Serialization / Hash Functionality Improvement

Question

I am working on a specialized hash table on disk (previous experiments with Berkeley, ManagedESENT, etc. did not come out). It has a fairly simple chain structure, with each key value pair (KVP) following in the file having a long (Int64) value pointing to the next KVP in the chain (and uses a value of zero if it is not). I am using MD5 to generate a hash code.

When profiling code to evaluate the speed of adding records, the hash function is responsible for about 55% of the time, which is not entirely unexpected. But about 25% of that time comes from calling binForm.Serialize(ms, obj) in the serialization function ObjectToByteArray . Both features are shown below. I suppose I cannot achieve much success in the hashing algorithm, but I wonder if I can extract some of the serialization functions?

  // Compute hash code long hash(object s) { byte[] y = md5.ComputeHash(ObjectToByteArray(s)); // Produces byte[16] long z = BitConverter.ToInt64(y, 0); long res = z & bitMask; return res; } // Convert an object to a byte array private byte[] ObjectToByteArray(Object obj) { if (obj == null) return null; MemoryStream ms = new MemoryStream(); binForm.Serialize(ms, obj); return ms.ToArray(); } 

Answer 1

Use protobuf.net, find here , it is much faster!

Update

From a look at your code, I assume there is no re-evaluation so that the computed hashes are consistent between AppDomains? If you are not calculating your HashCode, this might be simple:

 private static long GenerateHash(object key) { long typeHash = key.GetType().GetHashCode(); long keyHash = key.GetHashCode(); return (typeHash << 32) + keyHash; } 

For future use, your MemoryStream must really be in the block you are using, otherwise you risk a memory leak:

 private byte[] ObjectToByteArray(Object obj) { if (obj == null) return null; using (MemoryStream ms = new MemoryStream()) { binForm.Serialize(ms, obj); return ms.ToArray(); } } 

Answer 2

The binary formatter is known for its poor performance. try other serialization methods.