#Multi service pipeline, DB-service not updating properly for some services

1 messages · Page 1 of 1 (latest)

thorny mango
#

Hey, I'm using Go SDK to build E2E test pipeline. Maybe I'm doing something wrong but it seems the backend is connecting some older version of our Database and changes made by other services aren't in effect. The Migrator creates our tables and python populater adds some test data to them. Those run without errors, but when the Next-app calls our backend we get error that database tables don't exist. Any ideas what I'm missing here.

#

Here's the pipeline:

func main() {

    // start the database
    pg := client.Container().
        From("postgres:15.4-alpine").
        WithExposedPort(5432).
        AsService()
    
    // create tables, etc...
    _, err = client.Container().
        From("golang:1.20").
        // ...
        WithExec([]string{"./migrator", "db/migrations"}).
        Sync(ctx)

    // add test data
    _, err = client.Container().From("python:3.12.0-slim-bullseye").
        WithDirectory(WORKDIR, golangDir.Directory("db")).
        WithWorkdir(WORKDIR).
        WithServiceBinding("pg", pg).
                // ...
        WithExec([]string{"python", "populate.py"}).
        Sync(ctx)
    
    // backend
    golangService := client.Container().
        From("golang:1.20").
        WithDirectory(WORKDIR, golangDir).
        WithWorkdir(WORKDIR).
        WithServiceBinding("pg", pg).
        WithEnvVariable("DB_HOST", "pg").
        WithExec([]string{"go", "run", "cmd/api-backend/main.go"}).
        WithExposedPort(8080).
        AsService()
    // ...

    node := client.Container().
        From("node:20").
        WithServiceBinding("backend", golangService).
        WithDirectory(WORKDIR, frontendDir).
        WithWorkdir(WORKDIR).
                // ...
        WithExec([]string{"npm", "ci"}).
        WithExec([]string{"npm", "run", "build"}).
        WithExec([]string{"npx", "playwright", "install", "--with-deps"})

    // frontend
    next := node.
        WithExec([]string{"npm", "run", "start"}).
        WithExposedPort(4000).
        AsService()
        // trigger the tests...
}
thorny mango
#

Ah, it was about the services restarting after 10 seconds if not in use. Adding .Start(ctx) when creating the pg-service seemed to help.

midnight grove
steep atlas
#

possibly, @thorny mango how did you end up figuring it out? docs?

thorny mango
#

yeah, I ended up reading this https://docs.dagger.io/757394/use-services/ but only when I read the line "Another way to avoid relying on the grace period" I started suspecting there's something there.

Use Services in Dagger | Dagger

Dagger v0.9.0 includes a breaking change for binding service containers. The Container.withServiceBinding API now takes a Service instead of a Container, so you must call Container.asService on its argument. See the section on binding service containers for examples.

steep atlas
candid charm
#

In the "reference" section of the doc, we do mention this in various places:
Dagger cancels each service run after a 10 second grace period to avoid frequent restarts.
Note that this example relies on the 10-second grace period, which you should try to avoid
Depending on the 10-second grace period is risky because there are many factors which could cause a 10-second delay between calls to Dagger
Can you help me understand the point we should highlight up front @steep atlas? I'm guessing it's the first one, but want to confirm

steep atlas
candid charm