Django: how to serve images and thumbnails submitted by the user from separate, multiple servers?

Question

For my Django site, I would like to:

• Accept images submitted by users
• Creating thumbnails from these images
• Place both source images and thumbnails on separate, multiple servers for serving images

I need several separate image / thumbnail servers to provide sufficient I / O performance.

What is the best way to create a distributed image service system? Any open source software to help?

Thanks.

Answer 1

This is similar to a job for distributed task queues. My personal favorite Beanstalkd b / c is so lightweight and easy to use.

Server: https://github.com/kr/beanstalkd
Client Library: https://github.com/PeterScott/beanstalkc
Django Helper: https://github.com/jonasvp/django-beanstalkd

How it works: 1. The file is downloaded and saved locally
2. The work is created using the load view and placed in the beanstalk tube
3. Beanstalk employees observe the work of the pipes, they capture a new job
4. A worker can create thumbnails and distribute them to as many servers as you need.

The beauty is that you can have as many workers who feed the tube as required. Therefore, if you ever fall behind, just start more workers. Workers can be on the same machine or on different machines, and everything will work well.