Before asking my question, let me first explain the scenario. I installed the hbase cluster on ec2 with three instances:
i-xxxxxxx
Which worked perfectly. Now I tried to connect to the master of this cluster with a remote EMR instance with the hive installed. So, I followed this amazon aws documentation link:
http:
What is said before creating any table. Let the bush know the public DNS server of the remote hbase. set hbase.zookeeper.quorum = public-DNS-name; I did exactly what was mentioned. But I could not contact the master. I tried to create one table using the following script:
CREATE TABLE hauto(cookie string, timespent string, pageviews string, visit string, logdate string) STORED BY 'org.apache.hadoop.hive.hbase.HBaseStorageHandler' WITH SERDEPROPERTIES ("hbase.columns.mapping" = "m:timespent, m:pageviews, m:visit, m:logdate") TBLPROPERTIES ("hbase.table.name" = "hauto");
So he gave this error:
FAILED: Error in metadata: MetaException(message:org.apache.hadoop.hbase.MasterNotRunningException: Retried 10 times at org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:127) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.getHBaseAdmin(HBaseStorageHandler.java:74) at org.apache.hadoop.hive.hbase.HBaseStorageHandler.preCreateTable(HBaseStorageHandler.java:148) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:467) at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.createTable(HiveMetaStoreClient.java:460) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.invoke(RetryingMetaStoreClient.java:74) at com.sun.proxy.$Proxy14.createTable(Unknown Source) at org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:600) at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:3791) at org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:258) at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:144) at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:57) at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1355) at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1139) at org.apache.hadoop.hive.ql.Driver.run(Driver.java:945) at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:310) at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:231) at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:466) at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:819) at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:674) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.apache.hadoop.util.RunJar.main(RunJar.java:187) ) FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask
FYI, EMR installed installed instance, and EC2 Hbase-cluster are in the same security group. I am capable of telnet to ports 2181, 60000, 60010, 60020 from EMR to EC2, as well as from EC2 to EMR. And in this group I allowed tcp, udp, and ICMP connections on all ports within the group.
My wizard / etc / hosts file looks like this:
XXXXXXXXXXX master localhost XXXXXXXXXXX zkserver1 rgserver1 XXXXXXXXXXX zkserver2 rgserver2 XXXXXXXXXXX zkserver3 rgserver3
I am using hadoop 1.0.1 , hbase 0.94.11 , hive 0.11.0 and zookeeper-3.4.3 . And zookeeper is an outside manager.
Also, after that I tried the other way around. This time I tried to connect from a CLI cluster installed on an EC2 Hbase cluster to EMR Hbase. Now I was able to create the same table based on EMR hbase.
Updated question:
This seems to be a problem with amazon ec2. I provide you with a snapshot of these issues and a link:
http://hbase.apache.org/book.html
