mardi 12 janvier 2016

Django Test with Celery(with db operation in task)

My situation is a little bit complicated. My project is using django + mongoengine(mongo support for django) + celery. The project has run quite well for couple months. But I got some issue when testing with celery.

The thing is, I have db operations in my celery task, and task is called by a view. Normally when testing django will create a database with name 'test_XXX' and db operations in test case will connect to 'test_XXX'。 But I noticed that my task in celery still connect to develop database 'XXX' rather than 'test_XXX'

My goal is to test my task without changing setting, or restarting celery.

Code example is below.

proj/settings.py

MONGODB_HOST = '127.0.0.1'
MONGODB_PORT = 27017
MONGODB_USER = 'aaa'
MONGODB_PASSWD = 'bbb'
MONGODB_NAME = 'proj'

TEST_RUNNER = 'utils.mongotest.MongoTestSuiteRunner'

mongoengine.connect(MONGODB_NAME, host=MONGODB_HOST,
                    port=MONGODB_PORT, username=MONGODB_USER, password=MONGODB_PASSWD)

proj/utils/mongotest.py

from django.test.runner import DiscoverRunner
from mongoengine.connection import connect, disconnect, get_connection

from daodao import settings


class MongoTestSuiteRunner(DiscoverRunner):
    def setup_databases(self, **kwargs):
        self._test_dbname = settings.MONGODB_TEST_NAME
        disconnect()

        connect(
            self._test_dbname, host=settings.MONGODB_HOST,
            port=settings.MONGODB_PORT,
            username=settings.MONGODB_TEST_USER,
            password=settings.MONGODB_TEST_PASSWD
        )

        return super(MongoTestSuiteRunner, self).setup_databases(**kwargs)

    def teardown_databases(self, old_config, **kwargs):
        conn = get_connection()
        print 'drop mongo database %s...' % self._test_dbname
        conn.drop_database(self._test_dbname)

        super(MongoTestSuiteRunner, self).teardown_databases(old_config, **kwargs)

proj/init.py

from __future__ import absolute_import

from .celery import app as celery_app

__all__ = ['celery_app']

proj/celery.py

# proj/celery.py
from __future__ import absolute_import

import os

from celery import Celery
from django.conf import settings

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'daodao.settings')

app = Celery('daodao')

app.config_from_object('django.conf:settings')
app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

app/models.py

from mongoengine import Document, StringField


class User(Document):
    nickname = StringField(max_length=30)

app/tasks.py

from __future__ import absolute_import
from celery import shared_task
from app.models import User

@shared_task
def debug_shared_task():
    # this will print the count of users in database xxx, not test_xxx
    print User.objects.all().count()

I've tried method below but they doesnot seem to work

I created a test_settings.py and set mongodbengine to connect to test_xxx, and set a new broker for test_settings then created a new celery app(let us call it test_celery),and started its worker using test_settings. I expected test to use test_celery, but it did not.

Any suggestion would be appreciated.

Aucun commentaire:

Enregistrer un commentaire