I'm new to Cassandra, and I was tasked with launching it on EC2 in several regions, so if the whole EC2 region goes up, our app will continue to have fun. I read as much documentation as I could find regarding Ec2MultiRegionSnitch, and came to a standstill. I am running cassandra 1.0.10.
My problems are as follows:
1) when I run bin / cassandra I get an error: failed to start register mbean in JMX. Although I can run bin / nodetool -h ring on any of the nodes, and I get the display that you expect from a healthy system. I added the mx4j library to my cassandra deployment. I could try to remove this, I suppose.
2) when I run bin / cassandra-cli -h, I can create a keyspace like this:
CREATE KEYSPACE mykeyspace WITH placement_strategy = 'org.apache.cassandra.locator.NetworkTopologyStrategy' and strategy_options = {us-east-1:2,us-west-1:2};
3) After running "use mykeyspace", I can create a column family as follows:
CREATE COLUMN FAMILY people WITH comparator=UTF8Type AND key_validation_class=UTF8Type AND default_validation_class=UTF8Type AND column_metadata=[{column_name:FIRST_NAME,validation_class:UTF8Type}, {column_name:LAST_NAME,validation_class:UTF8Type}, {column_name:EMAIL,validation_class:UTF8Type}, {column_name:LOGIN,validation_class:UTF8Type, index_type: KEYS}];
4) After that I can run bin / cassandra-cli -h on any of the 4 nodes, start using mykeyspace; to describe; and each node correctly describes mykeyspace, including the column family and seed list.
5) But when I try to execute a simple one:
set people['1']['FIRST_NAME'] = 'John';
I get a stack trace as follows:
null UnavailableException() at org.apache.cassandra.thrift.Cassandra$insert_result.read(Cassandra.java:15206) at org.apache.cassandra.thrift.Cassandra$Client.recv_insert(Cassandra.java:858) at org.apache.cassandra.thrift.Cassandra$Client.insert(Cassandra.java:830) at org.apache.cassandra.cli.CliClient.executeSet(CliClient.java:901)
My configuration:
I performed ec2 authorization for ports 22, 7000, 7199 and 9160
I have 4 nodes in my cluster: one node in each of the following areas: Availability.
us-east-1:us-east-1a (initial_token: 0) us-east-1:us-east-1c (initial_token: 85070591730234615865843651857942052864) us-west-1:us-west-1a (initial_token: 1) us-west-1:us-west-1c (initial_token: 85070591730234615865843651857942052865)
Each instance of EC2 is associated with a public IP address.
In each node, I configured cassandra.yaml as follows:
seeds: <set to the public ip address for the us-east-1a and us-west-1a nodes> storage_port: 7000 listen_address: <private ip address of this node> broadcast_address: <public ip address of this node> rpc_address: 0.0.0.0 rpc_port: 9160 endpoint_snitch: Ec2MultiRegionSnitch
In addition, in each node cassandra-env.sh I included:
JVM_OPTS="$JVM_OPTS -Djava.rmi.server.hostname=<Node local IP Address>"
My request. I hope I have provided someone with enough information to help me get this job as I would like.
Further information Stack tracing from the first mx4j problem:
WARN 22:07:17,651 Could not start register mbean in JMX java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:616) at org.apache.cassandra.utils.Mx4jTool.maybeLoad(Mx4jTool.java:66) at org.apache.cassandra.service.AbstractCassandraDaemon.setup(AbstractCassandraDaemon.java:243) at org.apache.cassandra.service.AbstractCassandraDaemon.activate(AbstractCassandraDaemon.java:356) at org.apache.cassandra.thrift.CassandraDaemon.main(CassandraDaemon.java:107) Caused by: java.net.BindException: Cannot assign requested address at java.net.PlainSocketImpl.socketBind(Native Method) at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:353)
My cassandra-topology.properties
aaa.aaa.aaa.aaa=us-east-1:us-east-1a bbb.bbb.bbb.bbb=us-east-1:us-east-1c ccc.ccc.ccc.ccc=us-west-1:us-west-1a ddd.ddd.ddd.ddd=us-west-1:us-west-1c default=us-east-1:us-east-1a
My output of the nodetool __ ring
Address DC Rack Status State Load Owns Token 85070591730234615865843651857942052865 aaa.aaa.aaa.aaa us-east 1a Up Normal 11.09 KB 50.00% 0 bbb.bbb.bbb.bbb us-west 1a Up Normal 6.68 KB 0.00% 1 ccc.ccc.ccc.ccc us-east 1c Up Normal 11.09 KB 50.00% 85070591730234615865843651857942052864 ddd.ddd.ddd.ddd us-west 1c Up Normal 15.5 KB 0.00% 85070591730234615865843651857942052865
I am sure that I correctly added the areas / accessibility zone. At least I think I picked up what appears in the documentation. (Look at Ec2MultiRegionSnitch at this link) http://www.datastax.com/docs/1.0/cluster_architecture/replication
I don’t think I can just list the regions as us-west and us-east, because there are two regions in the west (us-west-1 is the California region, and us-west-2 is the Oregon region), so I don’t I think that simply surrendering us-west will successfully differentiate regions.