Using Elasticsearch High Memory

Question

Iam using elasticsearch on our development machine now. Will we move forward productively in a few weeks. Today I scored “top” and I was shocked by what I saw.

 PID   USER      PR   NI  VIRT  RES   SHR S  %CPU %MEM    TIME+   COMMAND
 28972 elastics  20   0   27.4g 1.4g  39m S  186  4.3     2:11.19 java

This is normal when elasticsearch uses so much memory. I never tuned like that. What is the ideal configuration if we have up to 5 indexes on the same machine with 32 GB of RAM. How many Replicas / Shards should I configure? How can I control memory usage?

I do not want to get the same problems that we have with Solr => Unexpected shutdowns.

Thank you for your help!

Answer 1

Es 1.0. Mmapfs. mamapfs HD.But . HD, RAM.It , .

, mmapfs . .

.

.configure ..

1) ulimit, , .

2) .

    # Search pool
threadpool.search.type: fixed
threadpool.search.size: 5
threadpool.search.queue_size: 200

# Bulk pool
threadpool.bulk.type: fixed
threadpool.bulk.size: 5
threadpool.bulk.queue_size: 300

# Index pool
threadpool.index.type: fixed
threadpool.index.size: 5
threadpool.index.queue_size: 200

# Indices settings
indices.memory.index_buffer_size: 30%
indices.memory.min_shard_index_buffer_size: 12mb
indices.memory.min_index_buffer_size: 96mb

# Cache Sizes
indices.fielddata.cache.size: 15%
indices.fielddata.cache.expire: 6h
indices.cache.filter.size: 15%
indices.cache.filter.expire: 6h

# Indexing Settings for Writes
index.refresh_interval: 30s
index.translog.flush_threshold_ops: 50000