Access to HBase running in a virtual machine with a client in the host system

Question

I am trying to write some data to hbase using a client program

HBase @Hadoop runs in a preconfigured virtual machine from Cloudera @ubuntu.

The client runs on the system that hosts the virtual machine, and the client runs directly on the VM.

So now I want to use a client outside vm to access the servers on vm

I am using NAT. To have access to servers such as HBase Master, HUE..running on vm, I configured port forwarding in the virtual field:

That way I can get to the HBase Master, HUE review sites ..

To start the client with servers on the vm hbase-site.xml I created with the content:

<configuration> <property> <name>hbase.zookeeper.quorum</name> <value>localhost</value> </property> <property> <name>hbase.zookeeper.property.clientPort</name> <value>9997</value> </property> <property> <name>hbase.master</name> <value>localhost:9999</value> </property> </configuration> 

So, I expected forwarding to work:

The error messages in the log when starting the client are as follows:

 11/09/07 17:48:00 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection 11/09/07 17:48:00 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181 11/09/07 17:48:01 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) 11/09/07 17:48:01 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181 11/09/07 17:48:01 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Address family not supported by protocol family: connect at sun.nio.ch.Net.connect(Native Method) at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:500) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1050) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1077) 11/09/07 17:48:03 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181 11/09/07 17:48:04 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.ConnectException: Connection refused: no further information at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method) at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:567) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1119) 11/09/07 17:48:04 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181 11/09/07 17:48:04 WARN zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect java.net.SocketException: Address family not supported by protocol family: connect at sun.nio.ch.Net.connect(Native Method) at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:500) at org.apache.zookeeper.ClientCnxn$SendThread.startConnect(ClientCnxn.java:1050) at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1077) 11/09/07 17:48:05 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181 

The correct connection log (when starting the client directly on vm) looks like this:

 11/09/07 09:05:29 INFO zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection 11/09/07 09:05:29 INFO zookeeper.ClientCnxn: Opening socket connection to server localhost/0:0:0:0:0:0:0:1:2181 11/09/07 09:05:29 INFO zookeeper.ClientCnxn: Socket connection established to localhost/0:0:0:0:0:0:0:1:2181, initiating session 11/09/07 09:05:29 INFO zookeeper.ClientCnxn: Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid = 0x132449d36df0006, negotiated timeout = 40000 

Therefore, I see only in the log line before the first problem that the connection URL is incorrect, because the port is correctly redirected, but the IP address is still local, and not 10.0.2.15, as it is configured in the port forwarding settings:

 Opening socket connection to server localhost/127.0.0.1:2181 

Only the hint that I found disables IPV6 -> disabled on host (win7) and vm (Ubuntu) and port check -> they are redirected correctly

Does anyone have an idea?

Answer 1

To give added value to users reading this, here is a solution:

 Could not resolve the DNS name of cloudera-vm 

To fix this, I simply added to C: \ Windows \ System32 \ drivers \ etc \ hosts:

 192.168.56.101 cloudera-vm 

Thus, whenever cloudera-vm is used, it resolves to the corresponding IP. The VM now runs in "Host Only" mode and is now assigned 192.168.56.101. Therefore port forwarding is not required.

Just for comparison:

HBase-site.xml

 <configuration> <property> <name>hbase.zookeeper.quorum</name> <value>cloudera-vm</value> </property> </configuration> 

persistence.xml

 <persistence xsi:schemaLocation="http://java.sun.com/xml/ns/persistence http://java.sun.com/xml/ns/persistence/persistence_1_0.xsd" version="1.0"> <persistence-unit name="hbase-addressbook" transaction-type="RESOURCE_LOCAL"> <properties> <property name="datanucleus.ConnectionURL" value="hbase:cloudera-vm" /> <property name="datanucleus.ConnectionUserName" value="" /> <property name="datanucleus.ConnectionPassword" value="" /> <property name="datanucleus.autoCreateSchema" value="true" /> <property name="datanucleus.validateTables" value="false" /> <property name="datanucleus.Optimistic" value="false" /> <property name="datanucleus.validateConstraints" value="false" /> </properties> </persistence-unit> </persistence> 

After that, I got another error (the connection was rejected, although all the connection strings were correct), so I investigated the days for a reason.

The solution was to disable IPv6 Ubuntu running on the virtual machine by adding the following to the file: /etc/sysctl.conf

 net.ipv6.conf.all.disable_ipv6 = 1 net.ipv6.conf.default.disable_ipv6 = 1 net.ipv6.conf.lo.disable_ipv6 = 1 

After the reboot, everything connects and works correctly :)

Answer 2

I think you can clearly define the code configuration, for example HbaseConfiguration conf = HbaseConfiguration.create (); conf.set ("hbase.zookeeper.quorum", "ubuntu1, ubuntu2"); ... can help you ..

Answer 3

Have you tried simply:

 <property name="connectionURL" value="hbase:master.standalone.hostname:60000"/>