Anyone using the Google Cloud Storage fuse in production?

How do you have performance and stability?

I probably want to implement this on a cluster in order to avoid a network or cluster file system, and it should work well with the automatic scaling of the server cluster. But what kind of participation in this? For example, the price?

+5
source share
1 answer

Access to Google Cloud Storage Fuse is ultimately access to Google Cloud Storage. All data transfers and operations performed by the Google Cloud Storage fuse are transferred to Google Cloud Storage transfers and operations and are charged accordingly. See the section for more information.

There are a few caveats to keep in mind when using Google Cloud Storage Fuse for your application:

  • Individual I / O streams are about as fast as gsutil .
  • Small random reads are slow due to a delay up to the first byte (do not start the database through the Google Cloud Storage plugin!)
  • Random entries are made by reading the entire blob, editing it locally, and writing the entire modified blob back to Google Cloud Storage. Small writes to large files work as expected, but are slow and expensive.

Note. A less obvious place to consider this is the Google Cloud Storage Fuse benchmarking. Many comparison tools use a combination of random and sequential recordings as the default settings. Be sure to set up comparative tools for serial I / O when working with the bucket mounted by the Google Cloud Storage fuse.

  1. There is no concurrency control for multiple authors in the file. When several authors try to replace the file, the last entries win, and all previous entries are lost - there is no merging, versioning or notification of the user about the subsequent overwriting.
  2. Hard links do not work.
  3. Some semantics are not quite what they will be in a traditional file system. Exclusion list here . For example, metadata, such as last access time, is not supported, and some metadata operations, such as renaming a directory, are not atomic.
  4. File authorization is determined by Google Cloud Storage permissions, not rights to Linux files that are not applicable.
+2
source

Source: https://habr.com/ru/post/1241821/


All Articles