Upgrading Your Django Celery App from Amazon Linux 1 to Linux 2

Learn how to seamlessly upgrade your Django Celery application from Amazon Linux 1 to Amazon Linux 2, with detailed steps and troubleshooting tips.
---
This video is based on the question https://stackoverflow.com/q/66963932/ asked by the user 'HenryM' ( https://stackoverflow.com/u/2740177/ ) and on the answer https://stackoverflow.com/a/66981244/ provided by the user 'Voronin Roman' ( https://stackoverflow.com/u/2732766/ ) at 'Stack Overflow' website. Thanks to these great users and Stackexchange community for their contributions.

Visit these links for original content and any more details, such as alternate solutions, latest updates/developments on topic, comments, revision history etc. For example, the original title of the Question was: How to upgrade Django Celery App from Elastic Beanstalk Amazon Linux 1 to Amazon Linux 2

Also, Content (except music) licensed under CC BY-SA https://meta.stackexchange.com/help/l...
The original Question post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license, and the original Answer post is licensed under the 'CC BY-SA 4.0' ( https://creativecommons.org/licenses/... ) license.

If anything seems off to you, please feel free to write me at vlogize [AT] gmail [DOT] com.
---
Upgrading Your Django Celery App from Amazon Linux 1 to Linux 2

Upgrading a Django application running with Celery from Amazon Elastic Beanstalk on Linux 1 to Linux 2 can seem daunting, especially if you're faced with issues related to script compatibility and service management. This guide will walk you through the transition and ensure that your Celery workers are properly set up and running in the new environment.

Understanding the Problem

You are currently experiencing difficulties while upgrading your Django application that utilizes Celery on Amazon Linux 2. Your script that worked perfectly on Linux 1 is producing errors during deployment on Linux 2. The errors indicate missing files and commands that are essential for running your Celery workers.

To pinpoint the solution, let's first summarize the key issues you're facing:

The script executed during post-deployment is unable to locate required directories like /var/app/current/env and /opt/python/etc/supervisord.conf.

The supervisorctl command is not found on Amazon Linux 2 as it's not installed by default.

Steps to Correct the Setup

Transitioning from a supervisord-based approach to using systemd for managing your Celery workers is the best way forward. Here’s how to redesign your script appropriately.

1. Create the Celery Service File

You’ll need to rewrite your script to create a systemd service file that manages Celery. Below is the structure you can use:

[[See Video to Reveal this Text or Code Snippet]]

2. Explanation of the Service File

[Unit] Section: Here, you define the service's name, description, and dependencies.

[Service] Section: This is where you specify how the service should run. Make sure to set:

Type=simple: Indicates that the service will run continuously.

Restart=always: Ensures that the service will restart if it crashes.

User and WorkingDirectory: Ensure that the service runs under the appropriate user and directory context.

ExecStart: Specifies how to start your Celery worker within the specified application context.

EnvironmentFile: Pointing to the environment variables needed by your app.

[Install] Section: This indicates how the service should be installed and started at boot.

3. Starting and Enabling the Service

After creating the service file, the systemctl commands provided will help you start the Celery service immediately and configure it to automatically start when the server reboots.

Troubleshooting Common Errors

When dealing with the upgrade, you may encounter various issues, such as:

File Not Found Errors: Ensure that all paths specified in your scripts and configurations are valid and accessible.

Permissions Issues: Verify that the user configured to run the service has the necessary permissions to access files and execute commands.

Conclusion

Upgrading your Django Celery application to Amazon Linux 2 requires some adjustments, specifically in how services are managed. By adapting your setup to use systemd, you can maintain robust control over your Celery workers and ensure they operate smoothly in the new environment. If you encounter further complications, don’t hesitate to consult AWS documentation or seek help from the community.

Feel free to share your experiences or additional tips in the comments below. Happy coding!