Deploying Django + Celery + Amazon SQS to AWS Elastic Beanstalk with Amazon Linux 2

In this article, we will learn about the configuration for deploying Django+Celery+Amazon SQS to Amazon Elastic Beanstalk with Amazon Linux 2.

I will assume here that you already familiar with Django/DRF and have running Django/DRF application. I will also assume that you know how to create a Beanstalk application with Amazon Linux 2 image.

Let’s drive into the configuration.

The first step into getting your application to run in Elastic Beanstalk is to create the “.ebextensions” folder in the root of the source. Now create a file named “02_package.config” add the code into that file showed in snippet below.

The openssl-static.x86_64, libcurl-devel, gcc are required for installing pycurl which will be used to connect with Amazon SQS.

To run DB migration and install python required packages, add the snippet below to a file named “04_db_migrate.config”.

Then It is needed to configure the Elastic Beanstalk to find/work with the Django/DRF settings and run the WSGI server. So create a file name “05_django.config” and add the code snippet to the file.

Now It is needed to define a pre-deployment action through the hook which used by many Amazon services for controlled action execution. the file named “03_pycurl_reinstall.sh” is needed to be created under the folder path “.platform/hook/predeploy”. It will reinstall the pycurl after installing the required yum packages mentioned above as it is system package dependencies.

Then I create a file named “01_create_celery_service.sh” in “.platform/hook/postdeploy” folder and add the code snippet below. The Celery will be configured with this service file.

Finally, the AWS compulsory config along with Amazon SQS configuration is needed to be added to the  file under the config folder of the Django/DRF app.

Predefined queue do not work with pooling so It has been avoided. Now zip the source code and upload to Elastic Beanstalk. Alternatively you can use AWS console or command line interface to deploy the application. Now we have successfully configured the application with Celery and Amazon SQS in Elastic Beanstalk with Amazon Linux 2. You can check the log file to check the status of the Celery. Also a new queue will be created in AWS SQS console.

This is just a basic configuration, and there are many other things that can be done to improve the performance and scalability of the application. For more detailed information on advanced configurations and deployment methods, you can refer to the official documentation provided by AWS. It provides a comprehensive guide on creating and deploying Python Django applications on Elastic Beanstalk.

The specific configuration that you choose will depend on your specific needs and requirements. However, the basic configuration that we have shown in this blog post should give you a good starting point for deploying your Django application with Celery and Amazon SQS to AWS Elastic Beanstalk with Amazon Linux 2.

Feel free to leave any comments for clarification, changes, or improvements. Also, you can contact with iXora Solution expert teams for any consultation or coordination from here.