🔒 Setting up Celery with Flask and docker | Python | Page 1

crisp rain Jan 25, 2025, 1:57 AM

#

To get a better understanding of the project I am working on. I provide a link to pastebin containing the README.md file.
I am working on the planned enhancement number 1.

I need help debugging the following error I'm getting when running docker-compose up :
celery_beat-1 | AttributeError: 'function' object has no attribute 'user_options' the celery_beat as well as the celery_worker are logging this error on the console. The api service runs fine. This is the docker-compose.yml file inside the root of the project.

For the rest of files I will also provide links to pastebin.
Modified app/init.py
Created a file app/celery.py
Created a file app/make_celery.py
Modified app/tasks.py
Modified Dockerfile

And as last, a the full traceback of the celery_worker service error: https://pastebin.com/hmGyBt20

I have been struggling with this for more than 30 hours already. I have read multiple sources online from the documation (Flask and Celery) to articles, to videos (PrettyPrinted's yt channel). I even asked multiple LLMs (GPT-01mini, Claude, Copilot). I can't seem to wrap my mind around as to why I am struggling so much to set this up. I can't take it anymore so I'm asking for help here.

lean finchBOT Jan 25, 2025, 1:57 AM

#

crisp rain To get a better understanding of the project I am working on. I provide a link t...

#

@crisp rain

Python help channel opened

Remember to:

Ask your Python question, not if you can ask or if there's an expert who can help.
Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

magic pilot Jan 25, 2025, 1:59 AM

#

you left out a () somewhere

crisp rain Jan 25, 2025, 1:59 AM

#

In my question?

magic pilot Jan 25, 2025, 2:00 AM

#

you probably have app = XXX someplace, where it needs to be app = XXX()

magic pilot Jan 25, 2025, 2:00 AM

#

crisp rain In my question?

No, in your code

#

#

try changing the last line to return app()

#

well ... maybe no

crisp rain Jan 25, 2025, 2:01 AM

#

TypeError: Flask.__call__() missing 2 required positional arguments: 'environ' and 'start_response'

#

yeh

magic pilot Jan 25, 2025, 2:01 AM

#

do you have your code on github?

crisp rain Jan 25, 2025, 2:02 AM

#

It's a private repo

magic pilot Jan 25, 2025, 2:02 AM

#

too bad

#

if I could repro the problem, I might be able to fix it

crisp rain Jan 25, 2025, 2:02 AM

#

give me a moment

thick idol Jan 25, 2025, 2:02 AM

#

!traceback

Can you show the whole traceback? Not just the TypeError itself, all thecontext.

lean finchBOT Jan 25, 2025, 2:02 AM

#

Traceback

Please provide the full traceback for your exception in order to help us identify your issue.
While the last line of the error message tells us what kind of error you got,
the full traceback will tell us which line, and other critical information to solve your problem.
Please avoid screenshots so we can copy and paste parts of the message.

A full traceback could look like:

Traceback (most recent call last):
  File "my_file.py", line 5, in <module>
    add_three("6")
  File "my_file.py", line 2, in add_three
    a = num + 3
        ~~~~^~~
TypeError: can only concatenate str (not "int") to str

If the traceback is long, use our pastebin.

crisp rain Jan 25, 2025, 2:03 AM

#

https://github.com/Beefy-py/synergy-daily-basket-forked

GitHub

GitHub - Beefy-py/synergy-daily-basket-forked

Contribute to Beefy-py/synergy-daily-basket-forked development by creating an account on GitHub.

crisp rain Jan 25, 2025, 2:04 AM

#

thick idol !traceback Can you show the whole traceback? Not just the TypeError itself, all...

I've provided links to pastebins

crisp rain Jan 25, 2025, 2:05 AM

#

lean finch

oww we have our own pastebin. I'll use that one from now on.

crisp rain Jan 25, 2025, 2:06 AM

#

magic pilot if I could repro the problem, I might be able to fix it

Did you get to clone it?

#

These are the env vars I can provide you with: ```
FOODBASKET_API_URL=http://localhost:5000

CELERY_BROKER_URL="redis://redis:6379/0"
CELERY_RESULT_BACKEND="redis://redis:6379/0"
DATABASE_URI=sqlite:///db.sqlite3

DEBUG=1

SECRET_KEY=''
RESEND_API_KEY=''

FLASK_APP=app
FLASK_ENV=development
FLASK_DEBUG=1
FLASK_RUN_PORT=7000```

magic pilot Jan 25, 2025, 2:10 AM

#

I've got it running ... but I don't see a docker-compose file

#

fwiw, your Dockerfile doesn't do all the steps that you mention in the README; I wouldn't expect it to work

#

unless your build context includes whatever you've got lying around in the work tree, in which case ... eww

#

anyhoo, it's running and the web page looks like this

#

and I probably can't help further without the docker compose file

crisp rain Jan 25, 2025, 2:14 AM

#

oohhh

#

yeah right right

#

my current changes are not pushed

magic pilot Jan 25, 2025, 2:14 AM

#

make a branch, push 'em there

crisp rain Jan 25, 2025, 2:15 AM

#

bet

#

new branchfeat/celery-integration

magic pilot Jan 25, 2025, 2:16 AM

#

I don't see it. have you pushed the branch to github?

#

#

that should say "2 Branches"

crisp rain Jan 25, 2025, 2:17 AM

#

oohh righht right I didnt push to the forked one

#

smh

magic pilot Jan 25, 2025, 2:18 AM

#

ok now your branch is present, but it doesn't have any new commits on it.

crisp rain Jan 25, 2025, 2:19 AM

#

should be available now

#

made a pr

magic pilot Jan 25, 2025, 2:19 AM

#

got it

#

ok, it's running in docker. I still just see that plain home page. What do I do to reproduce the error?

crisp rain Jan 25, 2025, 2:21 AM

#

check in the logs

magic pilot Jan 25, 2025, 2:21 AM

#

crisp rain Jan 25, 2025, 2:21 AM

#

the celery worker and beat should be running aswell

magic pilot Jan 25, 2025, 2:21 AM

#

ya they're running

#

 * Serving Flask app 'run.py'
 * Debug mode: on
WARNING: This is a development server. Do not use it in a production deployment. Use a production WSGI server instead.
 * Running on all addresses (0.0.0.0)
 * Running on http://127.0.0.1:7000
 * Running on http://192.168.148.3:7000
Press CTRL+C to quit
 * Restarting with stat
 * Debugger is active!
 * Debugger PIN: 597-179-842
192.168.148.1 - - [25/Jan/2025 02:20:29] "GET / HTTP/1.1" 200 -
192.168.148.1 - - [25/Jan/2025 02:20:29] "GET /favicon.ico HTTP/1.1" 404 -

#

that's all I see in the logs

crisp rain Jan 25, 2025, 2:22 AM

#

did you run docker compose?

magic pilot Jan 25, 2025, 2:22 AM

#

which logs have the error? Celery?

#

yes, I did "docker compose up --build"

#

crisp rain Jan 25, 2025, 2:22 AM

#

what do the logs say

#

docker-compose logs -f celery_beat

magic pilot Jan 25, 2025, 2:22 AM

#

ah I see it in the celery_worker log.

#

which I cannot paste here, apparently

#

gimme a while

crisp rain Jan 25, 2025, 2:23 AM

#

I'm trying to test this with the hello world task first.
Im tryna get it to run each minute.

#

Before trying to schedule the send_email function inside of routes.py

magic pilot Jan 25, 2025, 2:24 AM

#

maybe unrelated but ```
[2025-01-25 02:21:00,026: ERROR/MainProcess] Received unregistered task of type 'app.tasks.hello_world'.
The message has been ignored and discarded.

Did you remember to import the module containing this task?
Or maybe you're using relative imports?

Please see
https://docs.celeryq.dev/en/latest/internals/protocol.html
for more information.

The full contents of the message body was:
b'[[], {}, {"callbacks": null, "errbacks": null, "chain": null, "chord": null}]' (77b)

The full contents of the message headers:
{'lang': 'py', 'task': 'app.tasks.hello_world', 'id': '20c60648-c582-43f5-bd57-bf3644d78d6f', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '20c60648-c582-43f5-bd57-bf3644d78d6f', 'parent_id': None, 'argsrepr': '()', 'kwargsrepr': '{}', 'origin': 'gen1@d0b4abd05e77', 'ignore_result': False, 'replaced_task_nesting': 0, 'stamped_headers': None, 'stamps': {}}

The delivery info for this task is:
{'exchange': '', 'routing_key': 'celery'}
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/celery/worker/consumer/consumer.py", line 659, in on_task_received
strategy = strategies[type_]
~~~~~~~~~~^^^^^^^
KeyError: 'app.tasks.hello_world'

crisp rain Jan 25, 2025, 2:24 AM

#

I am not getting this at all

magic pilot Jan 25, 2025, 2:24 AM

#

🤷

#

maybe we're not running the same code

crisp rain Jan 25, 2025, 2:24 AM

#

we are

magic pilot Jan 25, 2025, 2:25 AM

#

ah well that's the only error I see in the logs.

crisp rain Jan 25, 2025, 2:25 AM

#

im finna go insane

#

Did it create a sqlite instance?

magic pilot Jan 25, 2025, 2:25 AM

#

99% sure we're not running the same code. Maybe you haven't told docker compose to "--build" recently.

#

what is "a sqlite instance"?

#

and how would I know if it created it?

crisp rain Jan 25, 2025, 2:26 AM

#

crisp rain Jan 25, 2025, 2:27 AM

#

magic pilot 99% sure we're not running the same code. Maybe you haven't told docker compose...

I am going to run docker-compose up --build again.

#

No errors now.

magic pilot Jan 25, 2025, 2:28 AM

#

not surprised.

crisp rain Jan 25, 2025, 2:28 AM

#

sound:0:1

magic pilot Jan 25, 2025, 2:29 AM

#

I'm pretty sure you've set things up so that your docker image depends on more-or-less random stuff lying around in your working tree. This is Bad®

#

you should be able to do ~~git clean -dxff ; docker-compose up --build~~ and have it work.

#

note: do not run that command

crisp rain Jan 25, 2025, 2:29 AM

#

why nah

magic pilot Jan 25, 2025, 2:29 AM

#

because it'll trash files that you haven't checked into git

#

and I suspect you have some of those

#

like, it'll trash "celerybeat-schedule", which ... I don't know where that came from, but the docker-compose stack needs it.

crisp rain Jan 25, 2025, 2:31 AM

#

ah

magic pilot Jan 25, 2025, 2:31 AM

#

getting this stuff right is tricky, and you don't always need it. Sometimes I think I'm the only one who insists on doing it properly 😕

#

but if you do have it right, then you can be very casual, and don't need to execute 12 tricky steps to get it working

crisp rain Jan 25, 2025, 2:32 AM

#

I want to do it right.
And it needs to be robust

#

I want the emails to arrive every day.

#

This is hosted on a Digital ocean droplet.

magic pilot Jan 25, 2025, 2:33 AM

#

well, "the emails arrive every day" is one thing; "any fool can easily bring up the docker-compose stack" is another

#

I was talking about the latter, not the former.

crisp rain Jan 25, 2025, 2:35 AM

#

Right

#

Thank you for the help!

#

I'll be debugging this further.

#

!close

lean finchBOT Jan 25, 2025, 2:45 AM

#

Python help channel closed with !close

This help channel has been closed. Feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.

#🔒 Setting up Celery with Flask and docker