SearchResponse scrolling does not repeat when there are fewer results than scrollSize

Question

I have such a cycle ...

while (true) { scrollResp = client.prepareSearchScroll(scrollResp.getScrollId()).setScroll(new TimeValue(600000)).execute().actionGet(); for (SearchHit hit : scrollResp.getHits()){ // does this when totalHits > scrollSize, skips it otherwise } //Break condition: No hits are returned if (scrollResp.hits().hits().length == 0) { break; } } 

The request scroll size is 500. When scrollResp.getHits (). totalHits () <500 a for loop is never entered. If I change scrollSize to be <totalHits, a for loop will be introduced.

In general, I expect> 500 results per query, but I need it to work anyway. Not sure if maybe I'm trying to iterate incorrectly or what. Feedback is welcome.

Answer 1

I believe you can do something like this:

  // tested also with pageSize = 1 final int pageSize = 100; SearchResponse firstScrollResponse = client .prepareSearch("db") .setTypes("collection") .setSearchType(SearchType.SCAN) .setScroll(TimeValue.timeValueMinutes(1)) .setSize(pageSize) .execute() .actionGet(); //get the first page of actual results firstScrollResponse = client.prepareSearchScroll(firstScrollResponse.getScrollId()) .setScroll(TimeValue.timeValueMinutes(1)) .execute() .actionGet(); while (firstScrollResponse.getHits().hits().length > 0) { for (final SearchHit hit : firstScrollResponse.getHits().hits()) { // do smth with it } // update the scroll response to get more entries from the old index firstScrollResponse = client.prepareSearchScroll(firstScrollResponse.getScrollId()) .setScroll(TimeValue.timeValueMinutes(1)) .execute() .actionGet(); } 

The trick first gets the first page of actual results from the scroll, since the first search response does not contain hits.