File storage + permissions: mongodb vs filesystem approach

Question

The explicit java web application allows users to upload files (images and documents) to their profiles and define access rules for these files (determine which other users can view / download the file). The access control / permission system is customizable, and the rules are stored in mongoDB along with the user profile and the actual file entry.

Knowing that I need the application and storage to be distributed and fault tolerant, I need to figure out which one is the best strategy for storing files.

Should I store files inside mongoDB in the file collection, where is the file containing the description and access rules?

Or should I store files inside the server file system and save the path in the mongoDB document? With the file system approach, I can still enforce user permissions and how? Finally, in the approach to the file system, how can I distribute files on different servers? Should I use dedicated servers for this or can I store files on webapp servers or mongodb servers?

Thanks so much for all your ideas! Any help or feedback is appreciated.

Alex

Answer 1

There are several alternatives:

• put files in a storage service (e.g. S3): simple and a lot of space, but poor performance
• place files on local file system: fast but not scalable
• Placing files in mongodb documents is simple, powerful and scalable, but limited to 16 MB.
• use the mongodb gridfs layer. Functionality is limited, but it is designed to be scalable (thanks to shading) and pretty fast. Please note that you can put file information (resolution, etc.) directly into the file metadata object.

In your case, it sounds like the last option might be the best, there are quite a few users who switched from FS to gridFS, and it worked very well for them. What you need to remember:

• gridfs sharding works, but not perfect: usually only data is processed, not metadata. It doesn’t matter, but the metadata splinter should be very secure.
• It may be useful to use gridfs in a separate mongodb cluster from your master data, as the requirements (storage, backup, etc.) are usually different.