Getting the total number of key-value pairs in RocksDB

Question

Is it possible to efficiently get the number of key-value pairs stored in the RocksDB value key store?

I looked at the wiki and still have not seen anything discussing this topic. Is such an operation possible?

Answer 1

It is not possible to get an exact bill. But in rockdb 3.4, which was released recently, it provides a way to get key counts, you can try it.

https://github.com/facebook/rocksdb/releases

Answer 2

Essentially, you can use db->GetProperty("rocksdb.estimate-num-keys", &num) to get an estimated number of keys stored in rockdb.

Another option is to use the sst_dump tool with the --show_properties argument to get the number of records, although the result will be based on each file. For example, the following command will show the properties of each SST file in the specified rockdb directory:

 sst_dump --file=/tmp/rocksdbtest-691931916/dbbench --show_properties --command=none 

And here is an example output:

 Process /tmp/rocksdbtest-691931916/dbbench/000005.sst Sst file format: block-based Table Properties: ------------------------------ # data blocks: 845 # entries: 27857 raw key size: 668568 raw average key size: 24.000000 raw value size: 2785700 raw average value size: 100.000000 data block size: 3381885 index block size: 28473 filter block size: 0 (estimated) table size: 3410358 filter policy name: N/A # deleted keys: 0 Process /tmp/rocksdbtest-691931916/dbbench/000008.sst Sst file format: block-based Table Properties: ------------------------------ # data blocks: 845 # entries: 27880 raw key size: 669120 ... 

Combine with some shell commands, you can get the total number of entries:

 sst_dump --file=/tmp/rocksdbtest-691931916/dbbench --show_properties --command=none | grep entries | cut -c 14- | awk '{x+=$0}END{print "total number of entries: " x}' 

And this will generate the following output:

 total number of entries: 111507