We are experiencing a problem with ES 5.4.2 for production. About 25% of our indexed documents appear only in the search results after 3 seconds or more after responding to a query from indexing. For some documents, this is more than 7 seconds. Our cluster consists of three nodes with 59 GB of RAM, of which 31 GB are for the ES heap. Using heap never exceeds 75%, while GC lowers it to ~ 25-30%. Indexing speed is relatively low, less than 10 documents per second. The average update time is ~ 300 ms. The default update interval is 1 second.
With these numbers, I expect no more than 1.5 s delay between indexing and search availability. What am I missing here? Can the search availability delay be reduced? Maybe I'm missing some kind of metric that is worth tracking?
source
share