The same error (or debugging instruction) still occurs in version 1.6.0 and is not an error.
When you create a new scroll request:
SearchResponse scrollResponse = client.prepareSearch(index).setTypes(types).setSearchType(SearchType.SCAN) .setScroll(new TimeValue(60000)).setSize(maxItemsPerScrollRequest).setQuery(ElasticSearchQueryBuilder.createMatchAllQuery()).execute().actionGet(); String scrollId = scrollResponse.getScrollId();
a new scroll identifier is created (except for scrolling if the answer is empty). To get the results:
long resultCounter = 0l; // to keep track of the number of results retrieved Long nResultsTotal = null; // total number of items we will be expecting do { final SearchResponse response = client.prepareSearchScroll(scrollId).setScroll(new TimeValue(600000)).execute().actionGet(); // handle result if(nResultsTotal==null) // if not initialized nResultsTotal = response.getHits().getTotalHits(); //set total number of Documents resultCounter += response.getHits().getHits().length; //keep track of the items retrieved } while (resultCounter < nResultsTotal);
This approach works regardless of how many fragments you have. Another option is to add a break statement when:
boolean breakIf = response.getHits().getHits().length < (nShards * maxItemsPerScrollRequest);
Number of items returned: maxItemsPerScrollRequest (for each skull!), So we expect the number of items requested will be multiplied by the number of fragments. But when we have multiple fragments, and one of them comes out of the documents, while the others do not, then the first method will still provide us with all the available documents. The latter will stop prematurely - I expect (did not try!)
Another way to stop seeing this exception (since it is "only" DEBUG ) is to open the logging.yml file in the config directory of ElasticSearch, and then change:
action: DEBUG
to
action: INFO