How to set up celery and django with Amazon SQS

I am trying to convert videos uploaded by users in a django application. The problem is that it takes resources, and the site becomes unavailable during this time, and a new instance should appear to scale it (I use Elastic Beanstalk). I did some research and decided to use SQS with a working environment

I installed celery and added the necessary configs to the settings.py file

BROKER_TRANSPORT = 'sqs'
BROKER_TRANSPORT_OPTIONS = {
    'region': 'us-east-1',
    'polling_interval': 3,
    'visibility_timeout': 3600,
}
BROKER_USER = AWS_ACCESS_KEY_ID
BROKER_PASSWORD = AWS_SECRET_ACCESS_KEY

CELERY_DEFAULT_QUEUE = 'celery-convert-video'
CELERY_QUEUES = {
    CELERY_DEFAULT_QUEUE: {
        'exchange': CELERY_DEFAULT_QUEUE,
        'binding_key': CELERY_DEFAULT_QUEUE,
    }
}

I set the POST URL to / celery -convert-video / I also do:

video = TestVideo.objects.create (uploaded_video = videoFile)

then

ConvertVideo.delay (video_id = video.id)

send the task to SQS. It captures the downloaded file using the URL and converts it. Locally this works, but the problem arises in the cloud.

, , , "", , , , 100% 4xx. , 403.

SQS ( , , , ?) " ", , , - . , .

, :

  • ?

  • /celery -convert-video/view, ConvertVideo ?

  • RDS, ?

  • 403 ?

- -, , !

P.S. . , .

+4

Source: https://habr.com/ru/post/1608975/


All Articles