Understanding KeyValue vs FileSystem Embedded Data Warehouse

I have a basic question regarding the use of FileSystem. I want to use KeyValue's built-in storage, which is very write-oriented. (constant). Say my size value is a) 10K b) 1M and reading and updating are equal in number

I can’t just create files containing this value, and there the name acts like keys.

Do not use it as fast as using KeyValue storage like LevelDB or RocksDB.

Can anyone help me understand.

+5
source share
1 answer

Basically, yes, the file system can be used as a keystore. Differences arise only when you look at individual use cases and implementation restrictions.

Without going into details, some things can be very different:

  • The file system breaks the data into blocks of a fixed size. Two files usually cannot occupy parts of the same block. Total block sizes 4-16 KiB; you can calculate how much overhead your 10 kilobyte example can cause. Key / value stores tend to take into account small amounts of data.
  • Directory indexes on file systems often cannot iterate over names / keys efficiently in sort order. You can efficiently search for a specific key, but you cannot get ranges without reading almost all the entries in the directory. Some key / value stores, including LevelDB, support efficient ordered iteration.
  • Some key / value stores, including LevelDB, are transactional. This means that you can combine several updates together, and LevelDB will make sure that either all these updates pass, or none of them performs. This is very important to prevent inconsistency in your data. File systems greatly complicate implementation, especially when multiple files are involved.
  • Key / value stores usually try to store continuous data on disk (so data can be retrieved with less search), while modern file systems do not intentionally do this in files. This can greatly affect performance when reading many records. However, this is not a problem with solid state drives.
  • Although some file systems offer compression functions, they are usually either for each file or for each block. As far as I can tell, LevelDB compresses entire fragments of records, which potentially gives better compression (although they shift their compression strategy towards performance in terms of compression efficiency).
+15
source

Source: https://habr.com/ru/post/1202030/


All Articles