In this tutorial, we are going to see how celery is set up in production environment, where both workers and other processes such as monitoring tool like flower are to be run continuously. During development stage, both worker and flower processes used to get stopped somehow forcing me to restart the every now and then. A solution for this, as suggested in official site of Celery, is to make use of tools like Supervisor.
In production you will want to run the worker in the background as a daemon and some times there may be a chance of stopping of celery worker automatically then it should be restarted automatically. To do these tasks you need to use the tools provided like supervisord.
First we need to set up python virtual environment. Then run the following command to create a virtual environment for our demo projects :
Now move to this folder env and activate the this virtual environment:
(Now we need to install both celery and rabitMQ in this virtual environment using pip,)
Now install supervisor using the following command :
pip install supervisor
This would create a configuration file named echo_supervisord_conf
Now run the following command to generate the config file :
echo_supervisord_conf > supervisord.conf
This would generate a config file, supervisord.conf where lies all the keys for our magic………….
Now move this file to the destination folder where we have written codes for celery. In my case I have a folder named project inside this env folder (which contains files such as tasks.py etc )
Now cd to projects folder.
Now open the file we have just copied, and add the following lines
command=celery worker -A tasks –loglevel=INFO
command=celery flower -A tasks
Since we need to run both worker and flower processes, they need to be added as two separate tasks as written above. Also we can set these as a group so that they would be started and stopped together. Most of the fields in these are selfexplanatory, however, if you would like to get a clear picture, you can this
Now Starting the demons :
Just go to the project directory (folder where we copied the config file) and open the terminal
and run the following command,
This would start both flower and celery worker as demons.
If we want stop supervisord, just runthe following command,