All geek questions in one place

Has anyone used LevelDB storage to store ActiveMQ messages?

We are currently using the KahaDB repository to store messages in ActiveMQ, and so far, good.
According to the release notes, ActiveMQ5.6 LevelDB provides enhanced performance.
Has anyone tried to level out LevelDB, and if you could provide the pros and cons?

+6
source share
5 answers

See this link: https://github.com/fusesource/fuse-extra/tree/master/fusemq-leveldb#how-to-use-with-activemq-56 There is a small comparison for leveldb vs kahadb.

I am currently testing a system with high message throughput, and I see better results already. I still need to make sure it is stable, but good so far.

+3
source

FYI: here is a link to white papers for ActiveMQ LevelDB Store

Minuses:

  • This is a new store, so there may still be some bugs.
  • LevelDB indexes sometimes have to be compact, which MAY DISABLE new entries.
  • You cannot just delete the index and rebuild it from the data files as you can with KahaDB
  • KahaDB handles disk corruption much more elegantly, restoring what it can, and discards corrupted records.

Pros:

  • Adding mostly disk access patterns improves performance on a spinning disk.
  • Fewer drives sync than KahaDB
  • Fewer index entries need to be inserted into each saved post.
  • Fewer index queries needed to load a message from disk to memory
  • Uses Snappy compression to reduce disk size on index records
  • Additional instant compression of data logs.
  • Sending a composite destination only saves the message to disk once.
  • Faster and more frequent GC data file.
  • It has a 'Replicated' variant , where it can independently replicate to "subordinate" brokers in order to provide an HA level at the message level.
+6
source

We use the levelDB repository for a month out of two, which are now being produced on NFS (with the standard failover rollback installed). Over the past few weeks, we had a damaged storage, with no errors in the logs ... only piling up queues and very low bandwidth. The only thing we could do to solve this is throw the store away and start over.

So, we are back to the old and reliable KahaDB store.

+5
source

Most of the performance requirements for LevelDB appear to be empty claims. It should support high concurrency reads, but multi-threaded testing shows no concurrency wins. https://github.com/ayende/raven.voron/pull/9#issuecomment-29764803

(In contrast, LMDB shows excellent linear performance gains when reading multiple processors. Https://github.com/ayende/raven.voron/pull/9#issuecomment-29780359 )

+2
source

I did extensive AMQ performance testing and could not get the statistically significant difference between LevelDB and KahaDB in my tests: http://whywebsphere.com/2015/03/12/ibm-mq-vs-apache-activemq-performance-comparison-update /

+1
source

Source: https://habr.com/ru/post/921905/

More articles:


All Articles