#is multiple `with_exec` allowed?

1 messages · Page 1 of 1 (latest)

rocky laurel
#

I have my sample postgres/django pipeline looked like this.

It seems curl do not wait django from ready before go ahead and result issues. I suppose I made some mistake somewhere?

import sys

import anyio
import dagger


async def main():
    async with dagger.Connection(dagger.Config(log_output=sys.stderr)) as client:
        postgres = (
            client.container().from_("postgres:14.7-alpine")
            .with_env_variable('POSTGRES_PASSWORD', 'postgres')
            .with_exposed_port(5432)
            .with_exec([])
        )

        src = client.host().directory(".", exclude=['.venv'])
        django = (
            client.container().from_("python:3.10.6-slim-buster")
            .with_service_binding('postgres', postgres)
            # mount cloned repository into image
            .with_mounted_directory("/app", src)
            # set current working directory for next commands
            .with_workdir("/app")
            # install test dependencies
            .with_exec('pip install -r requirements.txt'.split(' '))
            .with_exec('python manage.py migrate'.split(' '))
            .with_exec('python manage.py runserver 0.0.0.0:8000'.split(' '))
            .with_exposed_port(8000)
        )

        curl = (
            client.container()
            .from_("alpine")
            .with_service_binding("django", django)
            .with_exec(["wget", "-O-", "http://django:8000/api/blog"])
        )

        await curl.exit_code()

        # execute
        version = await curl.stdout()

    print(f"Hello from Dagger and {version}")


if __name__ == "__main__":
    anyio.run(main)
viscid pulsar
#

no, this is the expected behavior. Since the django "service" doesn't have a healthcheck, the curl pipeline starts running as soon as the django one starts. cc @shrewd veldt do we have a "suggested" pattern for waiting service containers to be "ready"?

#

I could think about multiple things like lockfiles, waiting for TCP ports to be binded, but maybe there's something else I'm not aware about

shrewd veldt
#

the with_exposed_port should be handling that, no? the client will wait for that port to be bound

#

@rocky laurel what error are you seeing?

viscid pulsar
shrewd veldt
#

it's possible that waiting for the port isn't enough, maybe it listens before it's fully ready to go?

rocky laurel
#

I think the issue is with with_exec for it seems I have a bunch of them running at once

shrewd veldt
#

can you paste log output? hard to guess what might be going wrong

rocky laurel
#

I.e. migration and runserver didn’t execute one after another

#

Ah, ok let me put my laptop back

#

I think it may be sometime with the dagger engine

shrewd veldt
#

hmm you could try running with export _DAGGER_DEBUG_HEALTHCHECKS=1

#

I doubt it's related to chaining with_execs though

rocky laurel
#

fresh run seems to be fine, but sub sequent run will cause issues

shrewd veldt
#

I'm wondering if the service container is somehow exiting successfully and getting cached

rocky laurel
#

do I run export _... with the dagger script?

shrewd veldt
#

which shouldn't happen, because we interrupt it, but it's theoretically possible

shrewd veldt
#

like in your shell

rocky laurel
#

right

#

yeah, this time it worked fine

#

I think it should be caching behavior or something

#

ah, I think I got it

#

django 4 version required ALLOWED_HOST to be set which don't bode well in this environment

#

so executing it will be fine. but

#

it will keep bad service container in even I updated the config

#

this is a successful run

#

yeah I believe it's caching behavior

#

I've done one in reverse, one with good config then a bad one

#

now the second run passes

shrewd veldt
#

a ha, so the wget was failing because of a bad response, not because it couldn't reach it?

rocky laurel
#

kind of footgun to me or I don't see the fineprint where

#

it could, it's just server return 400 due to allowed_host settings

#

as far as container concerned they are operating "properly"

shrewd veldt
#

oh, that was even inthe original logs, I just didn't notice:

Stderr:
Connecting to django:8000 (10.87.0.52:8000)
wget: server returned error: HTTP/1.1 400 Bad Request
rocky laurel
#

I think docker-compose mental model is very bad adapting dagger

#

I'll think change in source = fresh new container

#

I suppose this is not dagger rolls

#

tho I think it warrants a few warning here and there

shrewd veldt
#

yeah, it's not really suited for long-running stateful apps if that's what you're going for. better for ephemeral instances for testing.

rocky laurel
#

Tho I think it necessary to put warning there

#

If I don’t know this behavior I’ll assume somehow dagger skipped previous executions

#

Better, allow a dumb mode so service container always have the fresh build