I need to index / update a document in Elasticsearch and wait for it to be searchable (update completed). Github has a related issue: https://github.com/elasticsearch/elasticsearch/issues/1063
I will not force updates because it affects indexing metrics, and I will need to perform this operation very often. I tried to wait 1 second as described in the Github release. It works very well until Elasticsearch is under pressure, but when there is little RAM left (which can happen occasionally), I saw that the update took 5 or 6 seconds. So I tried another way.
I wrote a helper function in my backend that expects a searchable document to reach a specific version. It is pretty simple:
- GET the document with realtime=false - if there is a result - if result.version >= wanted.version. Return - else wait a little more and retry - else if the doc is not found - HEAD the document with realtime=true (test if the doc exists in the transaction log) - if the doc is found (then it has just been created) wait a little more and retry - else Return. (the doc might have been created and deleted really fast)
The required version is the version returned by elasticsearch when the document was indexed.
This algorithm works, but you can see that it is far from ideal.
he will first make more elasticsearch calls when he is under pressure, which is not a good idea.
I saw an elastic search reset the version number when the doc was deleted for some time. If for some reason the function skips this, we can wait until the document reaches this version. (which is why I also added a timeout).
Does anyone have a better solution? Scaling is not automatically an acceptable answer right now.
source share