1

My google-fu must not be up to snuff this morning because I am having trouble finding information why Celery is not finding my celeryconfig file. There's nothing in the celery log. The settings I have in celeryconfig just don't seem to be taking effect. I'm using Django 1.7.6, Celery 3.1.17 and my project & app are running in a virtualenv. The app is hosted by Apache and wsgi.

I'm using the celeryd script linked to on github from the celery documentation. Here is my /etc/default/celery with comments removed:

CELERYD_NODES="worker1"
CELERY_BIN="/home/celery/.virtualenvs/reporting/bin/celery"
CELERY_APP="main"

CELERYD_CHDIR="/home/celery/dev/reporting"

CELERYD_LOG_FILE="/var/log/celery/%N.log"
CELERYD_PID_FILE="/var/run/celery/%N.pid"

CELERYD_USER="celery"
CELERYD_GROUP="celery"
CELERY_CREATE_DIRS=1

CELERY_CONFIG_MODULE="celeryconfig"

As you can see, I have CELERYD_CHDIR set to my project directory (the one with manage.py in it). Here below I've pasted the output of pwd and ls showing the file in that directory:

(reporting)celery@wb-devel:~/dev/reporting$ pwd
/home/celery/dev/reporting
(reporting)celery@wb-devel:~/dev/reporting$ ls
celeryconfig.py  csv_output  manage.py  reports.xml
celeryd          main        reporting  requirements.txt

The file is readable by all.

This question here suggests explicitly setting the celeryconfig like so: --settings=celeryconfig but I'm not sure where to explicitly set that (in /etc/init.d/celeryd?), and still, I would like to do things the right way by using the setting in /etc/default/celeryd to point to celeryconfig

One last thing. In my celery.py file I have the following set:

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'reporting.settings')

app = Celery('main.celery',
          broker='amqp://',
          backend='amqp://',)

app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

So how do I get Celery to find my celeryconfig? Might it be because celeryconfig is not on my python path? I can't see how that is the case because I have CELERYD_CHDIR set, and also, in my apache config, I have python-path set: WSGIDaemonProcess web-reports python-path=/home/celery/dev/reporting:/home/celery/.virtualenvs/reporting/lib/python2.7/site-packages

Thanks for any help

Community
  • 1
  • 1
Patrick Zurek
  • 121
  • 1
  • 2
  • 6

1 Answers1

0

I think the problem might be that you are mixing up the configuration for standalone vs. Django-based celery.

In general when you use celery as a standalone Python project, then you create a celeryconfig.py module. This is shown in First Steps with Celery and the key line is: app.config_from_object('celeryconfig')

However, when integrating with Django you typically put the configuration values directly in the standard Django settings.py module. This variation is shown in First steps with Django and the key line changes to: app.config_from_object('django.conf:settings')

Since you are telling celery to load configuration from the Django settings module, perhaps you just need to move your values to that file.

Chris Ward
  • 774
  • 5
  • 6