We were amazed at Solr's behavior in production, which we cannot debug. To begin, consider the configurations for solr:
Solr Version: 6.5 , master with 1 slave with the same configuration as below.
JVM configuration:
-Xms2048m
-Xmx4096m
-XX:+ParallelRefProcEnabled
-XX:+UseCMSInitiatingOccupancyOnly
-XX:CMSInitiatingOccupancyFraction=50
All other defaults.
Solr Configuration:
<autoCommit>
<maxTime>{solr.autoCommit.maxTime:300000}</maxTime>
<openSearcher>false</openSearcher>
</autoCommit>
<autoSoftCommit>
<maxTime>{solr.autoSoftCommit.maxTime:900000}</maxTime>
</autoSoftCommit>
</updateHandler>
<query>
<maxBooleanClauses>1024</maxBooleanClauses>
<filterCache class="solr.FastLRUCache" size="8192" initialSize="8192" autowarmCount="0" />
<queryResultCache class="solr.LRUCache" size="8192" initialSize="4096" autowarmCount="0" />
<documentCache class="solr.LRUCache" size="12288" initialSize="12288" autowarmCount="0" />
<cache name="perSegFilter" class="solr.search.LRUCache" size="10" initialSize="0" autowarmCount="10" regenerator="solr.NoOpRegenerator" />
<enableLazyFieldLoading>true</enableLazyFieldLoading>
<queryResultWindowSize>20</queryResultWindowSize>
<queryResultMaxDocsCached>${solr.query.max.docs:40}
</queryResultMaxDocsCached>
<useColdSearcher>false</useColdSearcher>
<maxWarmingSearchers>2</maxWarmingSearchers>
</query>
Host Configurations (AWS):
RAM: 7.65GB
Cores: 4
solr , , , GC .
, . , . , , , Solr.
NewRelic Sematext (, ):
JVM Heap Memory
1 1
: JMap, SOLR , JVM xmx 12GB:
num #instances #bytes class name
----------------------------------------------
1: 11210921 1076248416 org.apache.lucene.codecs.lucene50.Lucene50PostingsFormat$IntBlockTermState
2: 10623486 934866768 [Lorg.apache.lucene.index.TermState;
3: 15567646 475873992 [B
4: 10623485 424939400 org.apache.lucene.search.spans.SpanTermQuery$SpanTermWeight
5: 15508972 372215328 org.apache.lucene.util.BytesRef
6: 15485834 371660016 org.apache.lucene.index.Term
7: 15477679 371464296 org.apache.lucene.search.spans.SpanTermQuery
8: 10623486 339951552 org.apache.lucene.index.TermContext
9: 1516724 150564320 [Ljava.lang.Object;
10: 724486 50948800 [C
11: 1528110 36674640 java.util.ArrayList
12: 849884 27196288 org.apache.lucene.search.spans.SpanNearQuery
13: 582008 23280320 org.apache.lucene.search.spans.SpanNearQuery$SpanNearWeight
14: 481601 23116848 org.apache.lucene.document.FieldType
15: 623073 19938336 org.apache.lucene.document.StoredField
16: 721649 17319576 java.lang.String
17: 32729 7329640 [J
18: 14643 5788376 [F
19: 137126 4388032 java.util.HashMap$Node
20: 52990 3391360 java.nio.DirectByteBufferR
21: 131072 3145728 org.apache.solr.update.VersionBucket
22: 20535 2891536 [I
23: 99073 2377752 shaded.javassist.bytecode.Utf8Info
24: 47788 1911520 java.util.TreeMap$Entry
25: 34118 1910608 org.apache.lucene.index.FieldInfo
26: 26511 1696704 org.apache.lucene.store.ByteBufferIndexInput$SingleBufferImpl
27: 17470 1677120 org.apache.lucene.codecs.lucene54.Lucene54DocValuesProducer$NumericEntry
28: 13762 1526984 java.lang.Class
29: 7323 1507408 [Ljava.util.HashMap$Node;
30: 2331 1230768 [Lshaded.javassist.bytecode.ConstInfo;
31: 18929 1211456 com.newrelic.agent.deps.org.objectweb.asm.Label
32: 25360 1014400 java.util.LinkedHashMap$Entry
33: 41388 993312 java.lang.Long
Solr - max - 2000 . , . , , solr , .
, , , , .