#How do I Initialize a Service through another container?

1 messages Β· Page 1 of 1 (latest)

gray sandal
#

Trying to return a database service initialized, I'm seeing a very odd behaviour.

I can see 2 services have been spun in the engine and both have been exposed to the host, while only one of them has been initialized.
A reproducible example:

func (m *Tests) ServiceTest(ctx context.Context) (*dagger.Service, error) {
        // Define a database service
    db := dag.
        Container().
        From("postgres:14").
        WithExposedPort(5432).
        WithEnvVariable("POSTGRES_PASSWORD", "postgres").
        AsService(dagger.ContainerAsServiceOpts{
            UseEntrypoint: true,
        })

        // Initialize some schema in the database
    init, err := dag.Container().From("postgres:14").
        WithServiceBinding("db", db).
        WithEnvVariable("PGPASSWORD", "postgres").
        WithEnvVariable("PG_HOST", "db").
        WithExec([]string{"sh", "-c", "psql -h db -U postgres -c 'CREATE TABLE test (id SERIAL PRIMARY KEY, name TEXT)'"}).
        WithEnvVariable("CACHE_BUSTER", time.Now().String()).
        WithExec([]string{"echo", "Database is ready: %s", time.Now().String()}).
        Stdout(ctx)
    if err != nil {
        return nil, err
    }
    fmt.Println("Initialized: " + init)

        // Return the service
    return db, nil
}

Then, dagger call service-test up and trying to hit the db with the new table:

$> psql -h 127.0.0.1 -U postgres -d postgres -c "SELECT * FROM test;"
 id | name
----+------
(0 rows)

$> psql -h 127.0.0.1 -U postgres -d postgres -c "SELECT * FROM test;"
ERROR:  relation "test" does not exist
LINE 1: SELECT * FROM test;

Seems like 2 different instances of the service have been spun, and I get redirected to one of them each time.

#

I can see both services running in the engine container (docker exec -it dagger-engine-v0.18.17 sh):

ps axuww
PID   USER     TIME  COMMAND
    1 root     21:20 /usr/local/bin/dagger-engine --config /etc/dagger/engine.toml --debug
   35 root      0:00 /usr/sbin/dnsmasq --keep-in-foreground --log-facility=- --log-debug -u root --conf-file=/var/run/containers/cni/dnsname/dag
  933 root      0:00 [ssh]
  934 root      0:00 [ssh]
  951 root      0:00 [git]
  978 root      0:00 [git]
43238 root      0:00 [ssh]
43239 root      0:00 [ssh]
43240 root      0:00 [ssh]
43241 root      0:00 [ssh]
76472 root      0:00 [ssh]
76473 root      0:00 [ssh]
76474 root      0:00 [ssh]
76476 root      0:00 [ssh]
181992 root      0:00 sh
183911 root      0:00 buildctl dial-stdio
183921 root      0:00 buildctl dial-stdio
183931 root      0:00 buildctl dial-stdio
183941 root      0:00 buildctl dial-stdio
184961 root      0:00 /usr/local/bin/runc --log /var/lib/dagger/worker/executor/runc-log.json --log-format json run --bundle /var/lib/dagger/wor
184974 root      0:00 /.init docker-entrypoint.sh postgres
184987 999       0:00 postgres
185044 999       0:00 postgres: checkpointer
185045 999       0:00 postgres: background writer
185046 999       0:00 postgres: walwriter
185047 999       0:00 postgres: autovacuum launcher
185048 999       0:00 postgres: stats collector
185049 999       0:00 postgres: logical replication launcher
185413 root      0:00 /usr/local/bin/runc --log /var/lib/dagger/worker/executor/runc-log.json --log-format json run --bundle /var/lib/dagger/wor
185427 root      0:00 /.init docker-entrypoint.sh postgres
185440 999       0:00 postgres
185497 999       0:00 postgres: checkpointer
185498 999       0:00 postgres: background writer
185499 999       0:00 postgres: walwriter
185500 999       0:00 postgres: autovacuum launcher
185501 999       0:00 postgres: stats collector
185502 999       0:00 postgres: logical replication launcher
185504 root      0:00 ps axuww
still nova
# gray sandal Trying to return a database service initialized, I'm seeing a very odd behaviour...

Hey Nadir!!

Could you try moving the cache invalidation one step before? Right now the cache gets invalidated right after running the CREATE TABLE command, so that command will only run one time:

This is the current state:

    init, err := dag.Container().From("postgres:14").
        WithServiceBinding("db", db).
        WithEnvVariable("PGPASSWORD", "postgres").
        WithEnvVariable("PG_HOST", "db").
        WithExec([]string{"sh", "-c", "psql -h db -U postgres -c 'CREATE TABLE test (id SERIAL PRIMARY KEY, name TEXT)'"}).
        WithEnvVariable("CACHE_BUSTER", time.Now().String()).
        WithExec([]string{"echo", "Database is ready: %s", time.Now().String()}).
        Stdout(ctx)

This is my suggestion:

    init, err := dag.Container().From("postgres:14").
        WithServiceBinding("db", db).
        WithEnvVariable("PGPASSWORD", "postgres").
        WithEnvVariable("PG_HOST", "db").
        WithEnvVariable("CACHE_BUSTER", time.Now().String()). // <--- This is the change
        WithExec([]string{"sh", "-c", "psql -h db -U postgres -c 'CREATE TABLE test (id SERIAL PRIMARY KEY, name TEXT)'"}).
        WithExec([]string{"echo", "Database is ready: %s", time.Now().String()}).
        Stdout(ctx)
gray sandal
#

Hey. yeah, sorry. I was stripping a lot of code to make it a simple reproducible case. The cache buster should be further up as you suggest.
Either way, the result is still the same, 2 consecutive queries to localhost:5432 reach 2 different service instances running.

I'm not sure how that is possible? How is that the engine shows 2 processes running a postgresql service (no other CLI are connected to the engine)...

zenith granite
#

since functions can currenly return one service unless a proxy module is being used.

gray sandal
#
func (m *Tests) ServiceTest(ctx context.Context) (*dagger.Service, error) {
        // Define a database service
    db := dag.
        Container().
        From("postgres:14").
        WithExposedPort(5432).
        WithEnvVariable("POSTGRES_PASSWORD", "postgres").
        AsService(dagger.ContainerAsServiceOpts{
            UseEntrypoint: true,
        })

        // Initialize some schema in the database
    init, err := dag.Container().From("postgres:14").
        WithEnvVariable("CACHE_BUSTER", time.Now().String()).
        WithServiceBinding("db", db).
        WithEnvVariable("PGPASSWORD", "postgres").
        WithEnvVariable("PG_HOST", "db").
        WithExec([]string{"sh", "-c", "psql -h db -U postgres -c 'CREATE TABLE test (id SERIAL PRIMARY KEY, name TEXT)'"}).
        WithExec([]string{"echo", "Database is ready: %s", time.Now().String()}).
        Stdout(ctx)
    if err != nil {
        return nil, err
    }
    fmt.Println("Initialized: " + init)

        // Return the service
    return db, nil
}

I'm creating a single service (db).
I'm binding that service into another container to do some initialization (init container gets the db binded to create some schemas)
Then, I return the db service

From the CLI, I only do a single call:
dagger call service-test up

And I'm seeing the mentioned scenario...
I am very puzzled. Maybe this has to do with the postgresql container?

I can't pinpoint what I'm doing wrong πŸ€”

zenith granite
gray sandal
#

Sure πŸ˜„ I've been trying to debug this for a while and any pointer would be chefkiss

zenith granite
gray sandal
#

I'm not crazy. That's a good start... πŸ˜…

zenith granite
#

ok, seems like definitely a bug. Looks like the service is not being de-dupped correctly and started two times @gray sandal

#

checking if this happened with older versions of dagger

gray sandal
#

Let me pull the convo with the engineer that reported it, 1 sec. I think he started seeing something similar in an old version. 1 sec

zenith granite
#

I was just checking if that's something that we recently introduced and it clearly doesn't seem like it

gray sandal
#

Yeah, he's reporting it started braking on 0.18.9...

#

He was upgrading fom 0.18.2, which worked fine. So... between 18.2 and 18.9 πŸ˜…

zenith granite
gray sandal
#

🫢

#

Thanks, happy to report a bug in GH now that I know I was not doing anything stupid πŸ˜‰

zenith granite
gray sandal
zenith granite
#

ok, the error seems to have appeared in v0.18.3. I'll continue the bisect work tomorrow and pinpoint the commit which introduced this behavior πŸ™

gray sandal
#

Much appreciated!!

zenith granite
#

just replied in the issue πŸ™

zenith granite
#

@gray sandal we have the fix. wrapping up some tests and opening a PR now πŸ™

gray sandal
#

Amazing. Thanks for the prompt response

gray sandal
#

@zenith granite , you're a legend. Thanks for the fix πŸ™‚

zenith granite