How to store complex objects in hadoop hbase?

Question

I have complex objects with collection fields that need to be saved in Hadoop. I do not want to go through the entire tree of objects and explicitly store each field. So I just think about serializing complex fields and storing it as one big part. And than desirialize it when reading an object. So what is the best way to do this? I’m at least talking about using some serialization for this, but I hope that Hadoop has the means to solve this situation.

An example of an object class for storage:

class ComplexClass {

<simple fields>

List<AnotherComplexClassWithCollectionFields> collection;


}

Answer 1

HBase only works with byte arrays, so you can serialize your object in any way that suits you.

Hadoop org.apache.hadoop.io.Writable. , org.apache.hadoop.io.WritableUtils.toByteArray(Writable ... writable).

, , Hadoop, Avro, Protocol Buffers Thrift. , . - , Hadoop Writable .