What's cheaper: crawling using n iterators of a single ConcurrentHashMap or n instances of HashMap

Imagine a producer-consumer scenario, stream A produces records, consumes them from one to many other streams.

To do this, I transfer a bunch of records to each consumer stream.

Do this, I ask myself if it is cheaper (primary in terms of processor utilization, secondary in memory):

  • so that each consumer stream represents a separate instance of HashMap . After the Map transferred to one consumer, a new Map instance will be created and used to transfer the next produced records to the next stream

or

  • use one ConcurrentHashMap and create an Iterator for each consumer stream and after passing the Iterator to the stream clearing the Map - so that each Iterator contains its own kind of base Map .

What do you think? Is a more or less general answer possible?
Or it depends heavily on some variables, such as the number of records, threads, etc.

EDIT: Or should I use some other data structure that can better solve these problems?

+4
source share
2 answers

The java concurrent parameter provides the data structure for the exact script.

@see java.util.concurrent.BlockingDeque

But please do some performance tests: because the results are highly dependent on your use case. And , if it's just micro-optimization, than: a clean, easy-to-understand approach with a threading support would be much better than optimizing performance without impact.

+5
source

The most expensive cpu-wise thing is thread conflict. It seems that your first approach will not lead to disagreement - each stream will have a local version of the card - due to increased memory consumption.
I would, for example, check out two scenarios for several installations (nr streams, map sizes, etc.) that make sense. It is hard to say the exact numbers without a guideline.

0
source

Source: https://habr.com/ru/post/1335969/


All Articles