Strength Queuing

Question

I read an article about batch processing in java in JDJ http://java.sys-con.com/node/415321 . The article is referenced using a save queue as a batch update instead of immediately sending a single insert or update to the database. The author does not give a concrete example of this concept, so I switched to the Persistent Queue, but it did not cause much. Does anyone know a good example of this?

Answer 1

Take a look at this previous question about Stackoverflow:

Player / Queue Consumer Streams

The first answer talks about implementing a Producer-Consumer (which is exactly what you want) using the ExecutorService . This is one way to do what you want, but it uses a in-memory queue - you could easily change that to a JMS queue, though.

Answer 2

This scheme is known by several names (for example, Backing Store, Write-Behind, etc.). This is often observed in Data Grid / Cache technologies, as well as in others. Typically, domain objects are stored in a FIFO queue or priority, and then the dispatcher decompresses them on an alternative thread or process and passes them to your real DAL. Queues can be in memory or actually placed in a network queue, such as ActiveMQ or MSMQ. In addition, I saw a scenario in which there are separate queues for each type of database operation.

In many cases, this is implemented as a facade in front of your DAL, representing the same interface. Thus, other applications believe that they communicate with the “real” DAL and abstract from other related problems. This denouement allows you to change it if you need. When the object data is transferred to the facade, the data is queued and then control is returned to the calling application.

After the queue, you can simply perform batch updates / inserts or continue to process items one at a time. Keep in mind that the concept of a transaction has a completely different meaning, so you need to think about it.

Although there are technologies that do this ... This is more of an example than one technology. I do not have an example convenient, but you can look at the documentation on a data grid from products such as Oracle Coherence. Find backup stores.