Problem using hbase from java on Amazon EMR

So, I am trying to query my hbase cluster on Amazon ec2 using a custom jar i running as a MapReduce step. Im my jar (inside the map function) I call Hbase the following:

public void map( Text key, BytesWritable value, Context contex ) throws IOException, InterruptedException {
    Configuration conf = HBaseConfiguration.create();
    HTable table = new HTable(conf, "tablename");
      ...

The problem is that when it gets into this HTable line and tries to connect to hbase, the step fails, and I get the following errors:

2014-02-28 18:00:49,936 INFO [main] org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=localhost:2181 sessionTimeout=180000 watcher=hconnection
2014-02-28 18:00:49,974 INFO [main] org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of this process is 5119@ip-10-0-35-130.ec2.internal
2014-02-28 18:00:49,998 INFO [main-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn: Opening socket connection to server localhost/127.0.0.1:2181. Will not attempt to authenticate using SASL (unknown error)
2014-02-28 18:00:50,005 WARN [main-SendThread(localhost:2181)] org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused

      ...

2014-02-28 18:01:05,542 WARN [main] org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid
2014-02-28 18:01:05,542 ERROR [main] org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
2014-02-28 18:01:05,542 WARN [main] org.apache.hadoop.hbase.zookeeper.ZKUtil: hconnection Unable to set watcher on znode (/hbase/hbaseid)
org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase/hbaseid

      ... and on and on

I can use the hbase shell just fine, and you can request data and everything that is in the shell. I have no idea where to start, and I've been looking for many hours, no luck. Most issues like this on the Internet don't talk about Amazon special fixes. I thought that zookeeper and hbase should automatically connect appropriately using the amazon bootstrap.

Im, jbase 0.94.17 jar amazon, hbase 0.94.7 im , , , Java . - , .

+4
1

, 30 . , .

Im amazon emr hadoop2 (ami 3.0.4) Hbase 0.94.7 Im hbase java.

, , hbase - / IP-, EC2. , HConfiguration ( localhost) , , Amazon ( /home/hadoop/hbase/conf/hbase -site.xml) .

:

Configuration conf = new Configuration();
conf.addResource("/home/hadoop/hbase/conf/hbase-site.xml");
HBaseAdmin.checkHBaseAvailable(conf);

-, hbase PACKAGED . , hbase 94.x hadoop1, cloudera hbase hbase-0.94.6-cdh4.3.0.jar( ), hadoop2. , , , org.apache.hadoop.net.NetUtils.

+6

Source: https://habr.com/ru/post/1529474/


All Articles