HTTP Gateway : Invoking Celery Tasks from Java (Non Python Application) – Part #3

HTTP Gateway is the ideal way for celery tasks invocation and status polling from any non python languages like java. For this, first we have to set up a django application which would handle all the incoming requests for tasks invocation and status polling. As a first step, we need to install python virtual enviroment named env (not mandatory, But I prefer this as this wont affect python packages installed in the machine).

Step1 :
activate the virtual environment and install all required libraries like django,djecelery etc.

Step 2:
copy and paste the django app for http-gateway in this activated virtual environment,env.(I have uploaded the code for this app in github, https://github.com/nishijayaraj/celery-HTTPgateway)

This app contains all tasks in tasks.py file. Now run the app from this activated virtual environment using the following command :
python manage.py runserver

Now with this, our djnago app would start at localhost:8080, to which we can send rest api calls. It is likely to be getting some module not found errors at this point, in case u get them, just try to install the missing packages and libraries in this activated virtual environment. and then run the app again.

The structure of the entire app is described in the image added below :
Screenshot from 2016-07-14 18:21:27

Now we need workers to execute the tasks. So open a terminal and run the following command from the root directory of our application (env/celery-HTTPgateway),

celery worker -A tasks –loglevel=INFO

It would then list all registered tasks there. We would be able to see to tasks,
1.tasks.hello_world
2.UploadTask

Now let’s see how we can invoke a celery task and poll its status using REST apis. These rest apis can be called from any programming languages using appropriate native apis.

For the sake of simplicity, here I am using linux curl command for simulating REST api calls.
( / at end of the url is mandatory for this command to work)

Open a terminal and run the following command,

curl -X GET http://localhost:8000/apply/tasks.hello_world/

Then you would a json data as response,

{“ok”: “true”, “task_id”: “0fc2150e-b321-4cc6-aaef-b1ce9b30e7fe”}

The respose contains the id of the invoked tasks which can be used to track its status.

Status Polling:

curl -X GET http://localhost:8000/0fc2150e-b321-4cc6-aaef-b1ce9b30e7fe/status/

Response:
{“task”: {“status”: “SUCCESS”, “result”: “Hello world………”, “id”: “0fc2150e-b321-4cc6-aaef-b1ce9b30e7fe”}}

These api would display custom states,something at which celery flower apis fails most of the time. A task named UploadTask has been written in tasks.py file with a view to showcase this feature. For this, first we need to invoke the tasks and then track the status.

Task invocation :

curl -X GET http://localhost:8000/apply/tasks.UploadTask/
Response –
{
“ok”: “true”,
“task_id”: “cc51e093-372f-42c1-8344-c1def70c544a”
}

The status checking of above task, can be done :

curl -X GET http://localhost:8000/cc51e093-372f-42c1-8344-c1def70c544a/status/
Response
{
“task”: {
“status”: “PROGRESS”,
“result”: {
“progress”: 0
},
“id”: “cc51e093-372f-42c1-8344-c1def70c544a”
}
}

References :
http://sofc.developer-works.com/article/25718503/How+to+use+Celery+in+Java+web+Application
https://github.com/ask/celery/tree/master/examples/celery_http_gateway

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s