Celery tasks do not execute with Django like they are empty functions | Django | Page 1

frank yacht Feb 14, 2024, 1:12 AM

#

I can't get Celery to work with Django, when I start the worker and beat, beat sends the task to the worker but the function does nothing like it's empty function

civic mulch Feb 14, 2024, 1:33 AM

#

Are you sure the task is running?

frank yacht Feb 14, 2024, 1:34 AM

#

This is my tasks code

# my_app > tasks.py
import requests
from celery import shared_task
from .models import Port

@shared_task
def task():
   requests.get('http://localhost:8000')
   Port.objects.create(port_number = 1)

And this is my celery code

# proj > celery.py
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'proj.settings')

app = Celery('proj')

app.config_from_object('django.conf:settings', namespace='CELERY')

app.autodiscover_tasks()

and this is my settings file

# settings.py
CELERY_BROKER_URL = 'redis://localhost:6379/0'  # Redis broker URL
CELERY_RESULT_BACKEND = 'redis://localhost:6379/0'  # Redis result backend URL
CELERY_BROKER_CONNECTION_RETRY_ON_STARTUP = True
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'

CELERY_RESULT_BACKEND = 'django-db'

CELERY_BEAT_SCHEDULE = {
    'random-task': {
        'task': 'my_app.tasks.task',
        'schedule': crontab(minute='*/1'),  # Run every minute
    },
}

frank yacht Feb 14, 2024, 1:34 AM

#

civic mulch Are you sure the task is running?

Yeah the worker is running with no issues

civic mulch Feb 14, 2024, 1:35 AM

#

frank yacht Yeah the worker is running with no issues

But how do you know the task is getting executed?

frank yacht Feb 14, 2024, 1:36 AM

#

civic mulch But how do you know the task is getting executed?

I guess by this

[2024-02-14 03:35:48,201: INFO/MainProcess] Task my_app.tasks.task[f2ae55f4-0d34-4cd1-bf2d-f854920a5b8d] received

civic mulch Feb 14, 2024, 1:38 AM

#

frank yacht I guess by this ``` [2024-02-14 03:35:48,201: INFO/MainProcess] Task my_app.task...

Hmm. How do you know the task isn't running as expected? What do you expect to happen that's not?

frank yacht Feb 14, 2024, 1:39 AM

#

There is no new record is getting created and even the get request doesn't log in my other server

civic mulch Feb 14, 2024, 1:40 AM

#

Can you schedule the task from another shell with .delay()

#

You may also try adding a new simpler task that prints something.

#

Everything you've shown looks right. Maybe it's your celery commands? What are those?

frank yacht Feb 14, 2024, 1:41 AM

#

even print not working the function doesn't print

civic mulch Feb 14, 2024, 1:41 AM

#

frank yacht even print not working the function doesn't print

Sounds like the task isn't making it to the worker then

#

The task is being sent to the queue, but isn't being popped off.

frank yacht Feb 14, 2024, 1:41 AM

#

celery -A proj worker --loglevel=info
celery -A proj beat --loglevel=info

frank yacht Feb 14, 2024, 1:42 AM

#

frank yacht I guess by this ``` [2024-02-14 03:35:48,201: INFO/MainProcess] Task my_app.task...

this is the worker log

#

the task is being received

civic mulch Feb 14, 2024, 1:42 AM

#

Try using debug logs. That may have more info.

frank yacht Feb 14, 2024, 1:45 AM

#

this is the debug

[2024-02-14 03:43:41,132: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:43:41,569: INFO/MainProcess] Task my_app.tasks.task[4ba4b2da-c857-4e5b-be7d-be6b2fb19e9b] received
[2024-02-14 03:43:41,570: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x0000016D8D276440> (args:('my_app.tasks.task', '4ba4b2da-c857-4e5b-be7d-be6b2fb19e9b', {'lang': 'py', 'task': 'my_app.tasks.task', 'id': '4ba4b2da-c857-4e5b-be7d-be6b2fb19e9b', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '4ba4b2da-c857-4e5b-be7d-be6b2fb19e9b', 'parent_id': None, 'argsrepr': '()', 'kwargsrepr': '{}', 'origin': 'gen4044@hx00r', 'ignore_result': False, 'replaced_task_nesting': 0, 'stamped_headers': None, 'stamps': {}, 'properties': {'correlation_id': '4ba4b2da-c857-4e5b-be7d-be6b2fb19e9b', 'reply_to': '3be32abe-7554-3dd2-be37-7f012c5d76af', 'delivery_mode': 2, 'delivery_info': {'exchange': '', 'routing_key': 'celery'}, 'priority': 0, 'body_encoding': 'base64', 'delivery_tag': '659abdde-fa17-406a-b127-238282876440'}, 'reply_to': '3be32abe-7554-3dd2-be37-7f012c5d76af', 'correlation_id': '4ba4b2da-c857-4e5b-be7d-be6b2fb19e9b', 'hostname': 'celery@hx00r', 'delivery_info': {'exchange': '', 'routing_key': 'celery', 'priority': 0,... kwargs:{})
[2024-02-14 03:43:42,134: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.

#

And also there is more stupid issue the worker just stucks after 5 received

#

[2024-02-14 03:48:54,493: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:48:55,494: DEBUG/MainProcess] Timer wake-up! Next ETA 0.7340000000003783 secs.
[2024-02-14 03:48:56,231: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:48:57,232: DEBUG/MainProcess] Timer wake-up! Next ETA 0.26599999999962165 secs.
[2024-02-14 03:48:57,498: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:48:58,500: DEBUG/MainProcess] Timer wake-up! Next ETA 0.9849999999996726 secs.
[2024-02-14 03:48:59,486: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:00,487: DEBUG/MainProcess] Timer wake-up! Next ETA 0.7340000000003783 secs.
[2024-02-14 03:49:01,223: DEBUG/MainProcess] Timer wake-up! Next ETA 0.26599999999962165 secs.
[2024-02-14 03:49:01,491: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:02,492: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:03,493: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:04,495: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:05,496: DEBUG/MainProcess] Timer wake-up! Next ETA 0.7340000000003783 secs.
[2024-02-14 03:49:06,232: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:07,233: DEBUG/MainProcess] Timer wake-up! Next ETA 0.25 secs.
[2024-02-14 03:49:07,485: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:08,486: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:09,489: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.
[2024-02-14 03:49:10,490: DEBUG/MainProcess] Timer wake-up! Next ETA 0.7340000000003783 secs.
[2024-02-14 03:49:11,226: DEBUG/MainProcess] Timer wake-up! Next ETA 0.26599999999962165 secs.
[2024-02-14 03:49:11,494: DEBUG/MainProcess] Timer wake-up! Next ETA 1.0 secs.

only doing this

civic mulch Feb 14, 2024, 1:49 AM

#

Have you tried purging the worker?

#

--purge

frank yacht Feb 14, 2024, 1:51 AM

#

celery worker shell also stucks not quitting with CTRL+C

#

i did it with --purge also the same issue

civic mulch Feb 14, 2024, 1:55 AM

#

civic mulch Can you schedule the task from another shell with .delay()

What about this?

frank yacht Feb 14, 2024, 1:59 AM

#

>>> task.delay()

def group(self, tasks, result, group_id, partial_args, add_to_parent=0):
    return 1


def xmap(task, it):
    return 1


def chain(*args, **kwargs):
    return 1


def xstarmap(task, it):
    return 1


def chord(self, header, body, partial_args=0, interval=1, countdown=2, max_retries=3, eager=4, **kwargs):
    return 1


def accumulate(self, *args, **kwargs):
    return 1


def chunks(task, it, n):
    return 1


def task():
    return 1


def unlock_chord(self, group_id, callback, interval=0, max_retries=1, result=2, Result=3, GroupResult=4, result_from_tuple=5, **kwargs):
    return 1


def backend_cleanup():
    return 1

<AsyncResult: c7c18121-b3bf-4478-be47-8e19ae8ffbaf>

frank yacht Feb 14, 2024, 2:06 AM

#

civic mulch What about this?

is this output even normal?

civic mulch Feb 14, 2024, 2:10 AM

#

It output all of that?

#

No that's not normal

frank yacht Feb 14, 2024, 2:11 AM

#

yeah that's the whole output

frank yacht Feb 14, 2024, 2:12 AM

#

civic mulch No that's not normal

so what is the issue 😢

civic mulch Feb 14, 2024, 2:12 AM

#

It might be worth uninstalling celery and reinstalling. It's also worth renaming your task to something other than task. It may be shadowing a name.

frank yacht Feb 14, 2024, 2:12 AM

#

i'm using windows btw

civic mulch Feb 14, 2024, 2:13 AM

#

I'm out of ideas after the above two any way.

frank yacht Feb 14, 2024, 2:13 AM

#

that is so weird and i felt that from the first day

civic mulch Feb 14, 2024, 2:14 AM

#

Celery is a pain to configure properly

frank yacht Feb 14, 2024, 2:14 AM

#

is there another good tool?

#

i just want to execute a django query on a model

#

and then execute a code

#

every minute

civic mulch Feb 14, 2024, 2:15 AM

#

A crontab and a django management command?

#

Django-q2

frank yacht Feb 14, 2024, 2:16 AM

#

I'll try this approach

#

Thanks for your help and your time

civic mulch Feb 14, 2024, 2:17 AM

#

You're welcome. If you solve the celery issue, try to follow up with me. I'm curious what it is

#Celery tasks do not execute with Django like they are empty functions