Pagination in CouchDB?

Question

How do I implement the queries required for pagination?

Basically, when page 1 is requested, get the first 5 entries. For page 2, get the following 5, etc.

I plan to use this through the couchdb-python module, but this should not make any difference to the implementation.

Answer 1

The CouchDB Guide has a good discussion on pagination, including many code examples, here: http://guide.couchdb.org/draft/recipes.html#pagination Here is their algorithm:

• Query rows_per_page + 1 rows from the view
• Show rows_per_page rows, save last row as next_startkey
• For startkey page startkey save startkey and next_startkey
• Use the next_* values ​​to create the next link and use the others to create the previous link.

NB: The right way to get pages in CouchDB is to specify the start key, not the start index, as you think. But how do you know which key to start the second page? Smart solution: "Instead of requesting 10 lines for a page, you request 11 lines, but only show 10 and use the values ​​in the 11th line as a start key for the next page."

If you expect multiple documents to emit the same keys, you will need to use startdocid in addition to startkey to startkey correctly. The reason is that a startkey alone will no longer be enough to uniquely identify a string. These options are useless unless you provide a startkey . In fact, CouchDB will first consider the startkey parameter, then it will use the startdocid parameter to redefine the start of the range again if several potential looking lines have the same key but different document identifiers. The same goes for enddocid .

Answer 2

The CouchDB HTTP API provides many features for efficient paging.

The simplest method will use startkey and count . Count is the maximum number of entries that CouchDB will return for this request, something that depends on your design, and startkey is where you want to launch CouchDB. When you request a submission, he will also tell you how many records there are, which allows you to calculate how many pages there will be if you want to show it to users.

Thus, the first request will not indicate a start key, but only a count of the number of records that you want to show. You can then mark the key of the last returned record and use it as a start key for the next page. In this simple form, you get an overlap where the last entry of one page is the first of the following. If this is undesirable, it is trivial to simply not display the last page entry.

An easier way to do this is to use the skip option to design the source document for the page, but use this method with caution. The skip parameter simply causes the internal engine not to return the records that it iterates. Although this gives the desired behavior, it is much slower than finding the first document for a page by key. The more documents skipped, the slower the request.

Answer 3

This is what I came up with - to get the identifiers of all messages, and then get the actual elements for the first number x of identifiers.

This is not very effective, but more important than getting all the messages and then throwing most away. However, to my surprise, this seemed to work quite quickly - I ran the posthelper.page() method 100 times, and it took about 0.5 seconds.

I didn’t want to post this on the actual question, so that would not affect the answers as well - here is the code:

 allPostsUuid = """ function(doc) { if(doc.type == 'post'){ emit(doc._id, null); } } """ class PostsHelper: def __init__(self): server = Server(config.dbhost) db = server[config.dbname] return db def _getPostByUuid(self, uuid): return self.db.get(uuid) def page(self, number = 1): number -= 1 # start at zero offset start = number * config.perPage end = start + config.perPage allUuids = [ x.key for x in self.db.query(allPostsUuid) ] ret = [ self._getPostByUuid(x) for x in allUuids[start : end] ] if len(ret) == 0: raise Error404("Invalid page (%s results)" % (len(allUuids))) else: return ret 

Answer 4

• Here is the recursive method that I found below:

  Take two variables

  var lastOffset = 0; var counter = 0; function someRecursive(lastOffset,counter) { queryView(db, whereClause).then(result => { var rows_per_page = 5; //formula below var page = Math.floor((lastOffset == 0 ? 0: (result.offset - lastOffset) + (rows_per_page * counter)) / rows_per_page) + 1; var skip = page * rows_per_page; if (somerecursionexitcondition) { counter = lastOffset == 0 ? lastOffset: counter + 1; lastOffset =result.offset; someRecursive(lastOffset, counter).then(result => { resolve(); }); }); }