How to maximize processor cores in an Elasticsearch cluster

How many instances and replicas do I need to install to use each processor core (I want 100% load, the fastest query results) in my cluster?

I want to use Elasticsearch for aggregation. I read that Elasticsearch uses several processor cores, but I did not find accurate information about the processor cores regarding fragments and replicas.

My observations are that one shard does not use more than 1 core / thread during a request (only one request at a time is taken into account). With replicas, querying the 1-blink index is not faster because Elasticsearch doesn't seem to use other nodes to distribute the load on the shard.

My questions (one request at a time):

  • Doesn't the shard use more than one processor core?
  • Shards should always be scanned completely, replicas cannot be used to share the load between nodes between nodes?
  • The formula for better performance is SUM (CPU_CORES per node) * PRIMARY_SHARDS?
+5
source share
1 answer

When performing an operation (indexing, searching, volume indexing, etc.), a chip in a node uses a single thread of execution, which means one CPU core.

If you have one request that is currently running, it will use one CPU core per fragment. For example, three node clusters with one index, which has 6 primary fragments and one replica, will have a total of 12 fragments, 4 fragments on each node.

If there is only one query in the cluster, for this index ES will request all 6 fragments of the index (regardless of whether they are primary or replicas), and each node will use from 0 to 4 core CPUs to work, because the round- algorithm The robin used by ES to select which copy of a fragment performs a search cannot select any fragments on a single node or a maximum of 4 fragments on a single node.

+6
source

Source: https://habr.com/ru/post/1235548/


All Articles