Work / Job theft ThreadPoolExecutor

Question

In my project, I create a Java runtime that receives working requests from a client. Work (different size) is divided into a set of tasks and then placed in a queue for processing. There are separate queues for processing each type of task, and each queue is associated with ThreadPool. ThreadPools are tuned so that the overall engine performance is optimal.

This design helps us balance requests efficiently, and large requests do not cause system resources to freeze. However, from time to time, the solution becomes ineffective when some of the queues are empty and their respective thread pools are idle.

To do this better, I was thinking of introducing a job / task theft method so that a heavily loaded queue could get help from other ThreadPools. However, this may require the implementation of my own Executor, because Java does not allow multiple queues to be associated with ThreadPool and does not support the concept of theft.

Read about Fork / Join, but that doesn't sound like my needs. Any suggestions or alternative way to create this solution can be very helpful.

Thanks Andy

Answer 1

Have you considered the ForkJoinPool ? The fork-join framework was implemented using modular modulation, so you can just use the thread pool to work.

Answer 2

Java 8 has factory and utility methods for the Executors class. There is an implementation of a thread pool for processing work ( here ), which, I believe, is exactly what you want.

Answer 3

you can implement a custom implementation of BlockingQueue (I think you basically need to implement the offer() and take() methods), which are supported by the "primary" queue and 0 or more secondary queues. take always takes from the main support queue, if it is not empty, otherwise it can pull it out of the secondary queues.

in fact, it may be better to have 1 pool, where all employees have access to all the queues, but "prefer" a particular queue. You can come up with an optimal work rate by assigning different priorities to different employees. in a fully loaded system, your employees should work with the optimal ratio. in an underloaded system, your employees should be able to help other queues.