How to create database container with migrations already applied? | Dagger | Page 1

placid meteor Jul 10, 2024, 11:31 AM

#

This is something I just can't wrap my head around. I need to create a database service and apply migrations with a migrator service (using with_service_binding), and persist the data of the database (/var/lib/postgresql/data for postgres). This should be persistent only for the final database container to contain the data using with_directory. I thought maybe I could use with_mounted_cache, but that would mean the final container would could potentially modify the data, which I don't want.

How should I go about this?

placid meteor Jul 11, 2024, 10:47 AM

#

Regarding this, I would also like to ask another thing. I was trying to use a mounted cache and tried to copy the entire directory using .directory(). It seems that this isn't possible either. What I had in mind was like this:

Create db and mount cache into /var/lib/postgresql/data.
Execute migrations against the db.
Create a dummy container and mount the cache to /volume.
Try to export the directory with .directory().

If I was able to do this, I could leverage the cache and have my db with migrations always available.

marsh olive Jul 11, 2024, 6:23 PM

#

placid meteor Regarding this, I would also like to ask another thing. I was trying to use a mo...

Regarding this, I would also like to ask another thing. I was trying to use a mounted cache and tried to copy the entire directory using .directory(). It seems that this isn't possible either. What I had in mind was like this:

correct, it's not currently possible via the directory/file API, you need to use WithExec and call cp at this moment 🙏

marsh olive Jul 11, 2024, 6:27 PM

#

placid meteor This is something I just can't wrap my head around. I need to create a database ...

I'd do something like this:

func (m *Lala) Lala(ctx context.Context) (*Container, error) {
    pgCache := dag.CacheVolume("postgres-data")

    postgres := dag.Container().From("postgres")

    _, err := postgres.WithMountedCache("/var/lib/postgresql/data", pgCache).
        WithExec([]string{"run", "migrations", "here"}).Sync(ctx)
    if err != nil {
        return nil, err
    }

    pgWithMigrations := postgres.WithMountedCache("/data", pgCache).
        WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/data"})

    return pgWithMigrations, nil
}

#

^ same pattern applies if you're using a postgres service. Take into account that cache volumes persist across dagger runs, so you need to be aware of the side effects of calling this pipeline multiple times if the cache volume name is always the same

placid meteor Jul 11, 2024, 7:26 PM

#

marsh olive I'd do something like this: ```go func (m *Lala) Lala(ctx context.Context) (*Co...

Hmm. Last time I tried, this really didn't work. But I will give it another shot. I am indeed using services, and the only difference here from my original attempt is that I am using a separate migrator.

marsh olive Jul 11, 2024, 7:28 PM

#

placid meteor Hmm. Last time I tried, this really didn't work. But I will give it another shot...

make sure that the container you end up returning ** is not** a reference of the WithMountedCache one. Otherwise, any files you put in the path that you set the cache mount to, won't get exported

placid meteor Jul 11, 2024, 7:32 PM

#

I will make my attempt with Go this time, using the CLI, not the SDK (if I am using the terms right).

marsh olive Jul 11, 2024, 7:32 PM

#

cc @ivory knot and @upper pumice not sure if we discussed the feasilibty of having a WithoutMountedX SDK method.

placid meteor Jul 11, 2024, 7:32 PM

#

Then I can convert the CLI-compatible Go version into the Rust SDK. Maybe something is off with the Rust SDK.

marsh olive Jul 11, 2024, 7:33 PM

#

placid meteor Then I can convert the CLI-compatible Go version into the Rust SDK. Maybe someth...

I can put together a quick example in Go if that helps Deniz

#

shouldn't take me more than 5'

placid meteor Jul 11, 2024, 7:37 PM

#

That'd be appreciated. I am going to do it myself now, too. Had to go between chats for a few minutes 😅

#

Btw, you can use .with_env_variable("POSTGRES_HOST_AUTH_METHOD", "trust") for easier auth. :)

#

I am still waiting for it to be finished, but currently what I have (dagger call database as-service up --ports=5432:5432):

func (m *M) Database(ctx context.Context) (*dagger.Container, error) {
    data := dag.CacheVolume("db")

    postgres := dag.Container().From("postgres").WithEnvVariable("POSTGRES_HOST_AUTH_METHOD", "trust")

    db := postgres.WithMountedCache("/var/lib/postgresql/data", data).AsService()
    migrations := dag.Directory().WithNewFile("/migrations/00_test.sql", "CREATE TABLE test (test text)").Directory("/migrations")
    m.Sqlx().
        WithServiceBinding("db", db).
        WithEnvVariable("DATABASE_URL", "postgres://postgres@db").
        WithDirectory("migrations", migrations).
        WithExec([]string{"migrate", "run"}).
        Sync(ctx)

    migratedDb := postgres.WithMountedCache("/data", data).
        WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/data"})

    return migratedDb, nil
}

func (m *M) Sqlx() *dagger.Container {
    return dag.Container().From("rust").WithExec([]string{"cargo", "install", "sqlx-cli"}).
        WithEntrypoint([]string{"cargo", "sqlx"})
}

marsh olive Jul 11, 2024, 8:08 PM

#

placid meteor I am still waiting for it to be finished, but currently what I have (`dagger cal...

lol, just finished my example

#

it's 99% there Deniz

#

func (m *Lala) Test(ctx context.Context) (*Container, error) {
    pgCache := dag.CacheVolume("postgres-data")

    postgres := dag.Container().From("postgres")

    _, err := postgres.WithServiceBinding("postgres",
        postgres.WithEnvVariable("POSTGRES_PASSWORD", "postgres").
            WithMountedCache("/var/lib/postgresql/data", pgCache).
            WithExposedPort(5432).AsService()).
        WithEnvVariable("PGPASSWORD", "postgres").
        WithExec([]string{"psql", "-h", "postgres", "-U", "postgres", "-c", "create table foo()"}).Sync(ctx)
    if err != nil {
        return nil, err
    }

    return postgres.WithMountedCache("/data", pgCache).
        WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/"}, ContainerWithExecOpts{SkipEntrypoint: true}).
        WithExec(nil), nil
}

#

note the SkipEntrypoint and the last WithExec(nil) before returning the container

#

that's probably the main difference

#

btw, that won't be required anymore in v0.12 per: https://github.com/dagger/dagger/pull/7136

placid meteor Jul 11, 2024, 8:11 PM

#

I am trying to decipher, hold on a minute :)

marsh olive Jul 11, 2024, 8:11 PM

#

WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/data"})

that will put the content in /var/lib/postgresql/data/data in the final container

placid meteor Jul 11, 2024, 8:11 PM

#

Hmm. Postgres within Postgres. The inner Postgres would be my migrator, I would assume.

#

No, wait

marsh olive Jul 11, 2024, 8:12 PM

#

placid meteor Hmm. Postgres within Postgres. The inner Postgres would be my migrator, I would ...

yes, I'm just running a psql create table

placid meteor Jul 11, 2024, 8:12 PM

#

Okay, the outer Postgres is the migrator, sorry.

#

Hmm. yeah, basically we have come up with nearly identical solutions. One thing regarding mine is it fails with this: error: error returned from database: the database system is starting up

#

The migrator is the outer Postgres in your case.

#

ExposedPort I miss, maybe it is required

marsh olive Jul 11, 2024, 8:18 PM

#

note the SkipEntrypoint and the last WithExec(nil) before returning the container

you also need that

placid meteor Jul 11, 2024, 8:18 PM

#

Okay, the final code:

type M struct{}

func (m *M) Database(ctx context.Context) (*dagger.Container, error) {
    data := dag.CacheVolume("db")

    postgres := dag.Container().From("postgres").
        WithEnvVariable("POSTGRES_HOST_AUTH_METHOD", "trust").
        WithExposedPort(5432)

    db := postgres.WithMountedCache("/var/lib/postgresql/data", data).AsService()
    migrations := dag.
        Directory().
        WithNewFile("/migrations/00_test.sql", "CREATE TABLE test IF NOT EXISTS (test text)").
        Directory("/migrations")
    _, err := m.Sqlx().
        WithEnvVariable("CACHEBUSTER", time.Now().String()).
        WithServiceBinding("db", db).
        WithEnvVariable("DATABASE_URL", "postgres://postgres@db").
        WithDirectory("migrations", migrations).
        WithExec([]string{"cargo", "sqlx", "migrate", "run"}).
        Sync(ctx)

    if err != nil {
        return nil, err
    }

    migratedDb := postgres.WithMountedCache("/data", data).
        WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/data"}, ContainerWithExecOpts{SkipEntrypoint: true}).
        WithExec(nil)

    return migratedDb, nil
}

func (m *M) Sqlx() *dagger.Container {
    return dag.Container().From("rust").WithExec([]string{"cargo", "install", "sqlx-cli"})
}

marsh olive Jul 11, 2024, 8:18 PM

#

marsh olive > WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/data"}) that w...

watch for this one

placid meteor Jul 11, 2024, 8:18 PM

#

✘ M.database: Container! 2.4s
! call function "Database": process "/runtime" did not complete successfully: exit code: 2
  ✘ Container.sync: ContainerID! 0.8s
  ! process "cargo sqlx migrate run" did not complete successfully: exit code: 1
    ✔ start docker-entrypoint.sh postgres 1.1s
    ┃ 2024-07-11 20:18:07.142 UTC [31] LOG:  database system was interrupted; last known up at 2024-07-11 20:17:32 UTC
    ┃ 2024-07-11 20:18:07.142 UTC [31] LOG:  database system was interrupted; last known up at 2024-07-11 20:17:32 UTC
    ┃ port is up: 10.87.0.31:5432
    ┃ 2024-07-11 20:18:07.488 UTC [33] FATAL:  the database system is starting up
    ┃ 2024-07-11 20:18:07.488 UTC [33] FATAL:  the database system is starting up
      ✔ 5432/tcp 0.2s
    ✘ exec cargo sqlx migrate run 0.3s
    ! process "cargo sqlx migrate run" did not complete successfully: exit code: 1
    ┃ error: error returned from database: the database system is starting up

Error: response from query: input: m.database resolve: call function "Database": process "/runtime" did not complete successfully: exit code: 2

Stdout:
invoke: input: container.from.withExec.withEnvVariable.withServiceBinding.withEnvVariable.withDirectory.withExec.sync resolve: process "cargo sqlx migrate run" did not complete successfully: exit code: 1

Stdout:
error: error returned from database: the database system is starting up

#

Let me see, sorry.

#

I based my modifications on the code you've provided. It also fails with the same error. Here is the code I used this time:

type M struct{}

func (m *M) Database(ctx context.Context) (*dagger.Container, error) {
    pgCache := dag.CacheVolume("postgres-data")
    postgres := dag.Container().From("postgres")

    migrations := dag.
        Directory().
        WithNewFile("/migrations/00_test.sql", "CREATE TABLE test IF NOT EXISTS (test text)").
        Directory("/migrations")

    _, err := m.Sqlx().WithServiceBinding("postgres",
        postgres.WithEnvVariable("POSTGRES_PASSWORD", "postgres").
            WithMountedCache("/var/lib/postgresql/data", pgCache).
            WithExposedPort(5432).AsService()).
        WithEnvVariable("DATABASE_URL", "postgres://postgres:postgres@postgres").
        WithDirectory("migrations", migrations).
        WithExec([]string{"cargo", "sqlx", "migrate", "run"}).
        Sync(ctx)
    if err != nil {
        return nil, err
    }

    return postgres.WithMountedCache("/data", pgCache).
        WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/"}, ContainerWithExecOpts{SkipEntrypoint: true}).
        WithExec(nil), nil
}

func (m *M) Sqlx() *dagger.Container {
    return dag.Container().From("rust").WithExec([]string{"cargo", "install", "sqlx-cli"})
}

marsh olive Jul 11, 2024, 8:24 PM

#

wondering where the 2024-07-11 20:18:07.488 UTC [33] FATAL: the database system is starting up is coming from.

placid meteor Jul 11, 2024, 8:25 PM

#

Oh, wait

#

I have a syntax error.

#

in... sql string

#

@marsh olive I think I found something. Would it be alright if I ask you to run this:

func (m *M) Database(ctx context.Context) (*dagger.Container, error) {
    pgCache := dag.CacheVolume("postgresdata")
    postgres := dag.Container().From("postgres").
        WithEnvVariable("POSTGRES_HOST_AUTH_METHOD", "trust").
        WithExposedPort(5432)

    migrations := dag.
        Directory().
        WithNewFile("/migrations/00_test.sql", "CREATE TABLE IF NOT EXISTS test (test text)").
        Directory("/migrations")

    _, err := m.
        Sqlx().
        WithServiceBinding("db",
            postgres.
                WithMountedCache("/var/lib/postgresql/data", pgCache).
                WithExposedPort(5432).AsService()).
        WithEnvVariable("DATABASE_URL", "postgres://postgres@db").
        WithDirectory("migrations", migrations).
        WithExec([]string{"cargo", "sqlx", "migrate", "run"}).
        Sync(ctx)
    if err != nil {
        return nil, err
    }

    return postgres.WithMountedCache("/data", pgCache).
        WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/"}, ContainerWithExecOpts{SkipEntrypoint: true}).
        WithExec(nil), nil
}

func (m *M) Sqlx() *dagger.Container {
    return dag.Container().From("rust").WithExec([]string{"cargo", "install", "sqlx-cli"})
}

#

Using dagger call database as-service up --ports=5432:5432

marsh olive Jul 11, 2024, 8:32 PM

#

checking

placid meteor Jul 11, 2024, 8:32 PM

#

Then have run psql postgres://postgre@localhost in your host machine

#

When you type \d test in the psql shell, you should see the table description.

marsh olive Jul 11, 2024, 8:33 PM

#

placid meteor When you type `\d test` in the psql shell, you should see the table description.

that's not working for you?

placid meteor Jul 11, 2024, 8:35 PM

#

Here is the thing: When I execute this code snippet for the first time, using a clean cache, it works. It all works. But if I modify the CREATE TABLE statement, sqlx fails with database isn't up error that I mentioned previously. Then I clear the cache again (I am renaming the cache), it works again.

#

I modified the CREATE TABLE statement to simulate file modifications.

#

(I assume that's what it would do, no?)

#

What I modified in the query was to just add NOT NULL on the column: CREATE TABLE IF NOT EXISTS test (test text NOT NULL).

marsh olive Jul 11, 2024, 8:37 PM

#

placid meteor Here is the thing: When I execute this code snippet for the first time, using a ...

I get this error if I modify the CREATE TABLE statement:

    ! process "cargo sqlx migrate run" did not complete successfully: exit code: 1
    ┃ error: migration 0 was previously applied but has been modified   ```

#

is that what you're getting?

placid meteor Jul 11, 2024, 8:37 PM

#

No, I get the database not up error.

#

To what did you change the query?

marsh olive Jul 11, 2024, 8:38 PM

#

can you run ti with dagger call -v and share the logs please?

marsh olive Jul 11, 2024, 8:38 PM

#

placid meteor To what did you change the query?

the table name

placid meteor Jul 11, 2024, 8:38 PM

#

Yes, let me see.

marsh olive Jul 11, 2024, 8:38 PM

#

from test to foo

placid meteor Jul 11, 2024, 8:40 PM

#

✔ connect 2.0s
  ✔ starting engine 1.5s
  ✔ starting session 0.4s
✔ initialize 0.5s
✘ M.database: Container! 2.3s
! call function "Database": process "/runtime" did not complete successfully: exit code: 2
  ✔ Container.from(address: "postgres"): Container! 0.7s
  ✔ Container.from(address: "rust"): Container! 0.2s
  ✘ Container.sync: ContainerID! 0.9s
  ! process "cargo sqlx migrate run" did not complete successfully: exit code: 1
    ✔ start docker-entrypoint.sh postgres 1.2s
    ┃ 2024-07-11 20:39:59.499 UTC [7] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
    ┃ 2024-07-11 20:39:59.518 UTC [30] LOG:  database system was interrupted; last known up at 2024-07-11 20:39:09 UTC
    ┃ port is up: 10.87.0.53:5432
    ┃ 2024-07-11 20:39:59.999 UTC [32] FATAL:  the database system is starting up
    ┃ 2024-07-11 20:39:59.999 UTC [32] FATAL:  the database system is starting up
      ✔ 5432/tcp 0.4s
    ✘ exec cargo sqlx migrate run 0.4s
    ! process "cargo sqlx migrate run" did not complete successfully: exit code: 1
    ┃ error: error returned from database: the database system is starting up

Error: response from query: input: m.database resolve: call function "Database": process "/runtime" did not complete successfully: exit code: 2

Stdout:
invoke: input: container.from.withExec.withServiceBinding.withEnvVariable.withDirectory.withExec.sync resolve: process "cargo sqlx migrate run" did not complete successfully: exit code: 1

Stdout:
error: error returned from database: the database system is starting up

#

I think this might be related to the migrations, not not dagger. I may not be thinking as clearly as I used to during the day, though. It is getting very much past the normal sleeping hours of mine, but you know the urge. :D

marsh olive Jul 11, 2024, 8:41 PM

#

have you added -v ? you should see more logs

placid meteor Jul 11, 2024, 8:42 PM

#

Oh, after call

#

I used dagger -v...

marsh olive Jul 11, 2024, 8:42 PM

#

👍

#

I think it might be related to a race maybe around the healthcheck and when the DB is effectively up

placid meteor Jul 11, 2024, 8:43 PM

#

✔ connect 1.1s
  ✔ starting engine 0.7s
  ✔ connecting to engine 0.1s
  ✔ starting session 0.3s
✔ initialize 0.7s
✘ M.database: Container! 2.2s
! call function "Database": process "/runtime" did not complete successfully: exit code: 2
  ✔ Container.from(address: "postgres"): Container! 0.7s
  ✔ Container.from(address: "rust"): Container! 0.2s
  ✘ Container.sync: ContainerID! 0.8s
  ! process "cargo sqlx migrate run" did not complete successfully: exit code: 1
    ✔ start docker-entrypoint.sh postgres 1.0s
    ┃ 2024-07-11 20:42:39.345 UTC [7] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
    ┃ 2024-07-11 20:42:39.363 UTC [30] LOG:  database system was interrupted; last known up at 2024-07-11 20:39:09 UTC
    ┃ port is up: 10.87.0.56:5432
    ┃ 2024-07-11 20:42:39.750 UTC [32] FATAL:  the database system is starting up
    ┃ 2024-07-11 20:42:39.750 UTC [32] FATAL:  the database system is starting up
      ✔ 5432/tcp 0.4s
    ✘ exec cargo sqlx migrate run 0.3s
    ! process "cargo sqlx migrate run" did not complete successfully: exit code: 1
    ┃ error: error returned from database: the database system is starting up

Error: response from query: input: m.database resolve: call function "Database": process "/runtime" did not complete successfully: exit code: 2

Stdout:
invoke: input: container.from.withExec.withServiceBinding.withEnvVariable.withDirectory.withExec.sync resolve: process "cargo sqlx migrate run" did not complete successfully: exit code: 1

Stdout:
error: error returned from database: the database system is starting up

#

Btw, I also ran the code against the modifications that you did. I didn't receive the same error you did. I still got the db not up error that I have been getting.

marsh olive Jul 11, 2024, 8:43 PM

#

weird.. I see way more data on my DB startup service:

📎 message.txt

placid meteor Jul 11, 2024, 8:44 PM

#

This is the exact command I should use, right: dagger call -v database as-service up --ports=5432:5432

marsh olive Jul 11, 2024, 8:45 PM

#

yeah, same. could you try with --progress plain please instead of -v one last time?

#

that shouldnt collapse anything

placid meteor Jul 11, 2024, 8:45 PM

#

Okay.

#

📎 message.txt

#

You think there's something fishy regarding 11 : [0.7s] | 2024-07-11 20:45:51.411 UTC [32] FATAL: the database system is starting up?

marsh olive Jul 11, 2024, 8:48 PM

#

placid meteor You think there's something fishy regarding `11 : [0.7s] | 2024-07-11 20:45...

yeah, I don't get that. But I think it's a race

#

and the cargo sqlx is not retrying

placid meteor Jul 11, 2024, 8:48 PM

#

I wonder if that could be fixed with a simple loop.

#

Also, let me see if it has any option for retrying.

marsh olive Jul 11, 2024, 8:49 PM

#

yeah.. you can surely do that I'd assume

placid meteor Jul 11, 2024, 8:49 PM

#

(I would normally assume it would do that by itself)

placid meteor Jul 11, 2024, 8:50 PM

#

marsh olive and the cargo `sqlx` is not retrying

Hmm. To be fair, maybe it isn't specifically caring about the "the database system is starting up" error. It must be a catch-all clause on their code.

#

That might be why it doesn't retry.

marsh olive Jul 11, 2024, 8:51 PM

#

placid meteor Hmm. To be fair, maybe it isn't specifically caring about the "the database syst...

yeah, probably...

placid meteor Jul 11, 2024, 8:51 PM

#

So, far, though, I am pretty happy with the result. I discovered that why it had been failing. Many many thanks.

#

I will now convert this into rust and share the snippet here for who might need it in the future.

marsh olive Jul 11, 2024, 8:52 PM

#

there's this: https://github.com/launchbadge/sqlx/blob/d1f180fbc5b09159f019bc5655b10ff3297a4a9c/sqlx-cli/src/lib.rs#L114 but seems like it's only connection errors

GitHub

sqlx/sqlx-cli/src/lib.rs at d1f180fbc5b09159f019bc5655b10ff3297a4a9...

🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, and SQLite. - launchbadge/sqlx

#

in this case it effectively connects, but receives a proper postgres error that the DB is still not up

#

I guess you can do a for loop, yes

placid meteor Jul 11, 2024, 8:52 PM

#

Exactly. That's what I was gonna say.

placid meteor Jul 11, 2024, 8:53 PM

#

marsh olive I guess you can do a for loop, yes

Yup. Also, this could be reported (maybe with a PR to upstream).

#

(reported to sqlx, i mean)

marsh olive Jul 11, 2024, 8:54 PM

#

placid meteor (reported to sqlx, i mean)

yep, that'd be nice. I guess that they'll probably ask if some other tools consider this specifically or not

#

or if they push that to the user to make sure the DB is "fully ready" before running the migrations

placid meteor Jul 11, 2024, 8:56 PM

#

I mean, it all really depends on them, but I would think "database is starting up" would be such an error that can really well be recovered from.

marsh olive Jul 11, 2024, 8:56 PM

#

placid meteor I mean, it all really depends on them, but I would think "database is starting u...

yep, agree

placid meteor Jul 11, 2024, 8:57 PM

#

Anyway, I am really glad this is out of the way. Many thanks!

#

Wish you a nice one. o/

marsh olive Jul 11, 2024, 8:59 PM

#

placid meteor Wish you a nice one. o/

yeah, same! LMK if that loop finally fixes it 🙏

placid meteor Jul 12, 2024, 7:47 AM

#

I now have guarded the sqlx command with a loop that ensures the db is up and okay serving connections using pg_isready. It works well, and I have confirmed (with simple print-like statements) that it works. But I will tell you something that doesn't work. You see, when we modify the cache, I think it isn't somehow reflected in WithMountedCache on the final image. I can confirm it isn't because no matter what I do, the migrations are only applied on the first db image, not the last. I can get around this by just using a cachebuster right before the copy. I will provide the examples here.

Usage: dagger call database --migrations=migrations as-service up --ports=5432:5432
Confirm it works: psql postgres://postgres@localhost -c '\d'

#

type M struct{}

func (m *M) Database(ctx context.Context, migrations dagger.Directory) (*dagger.Container, error) {
    cache := dag.CacheVolume("db-data")
    base := dag.Container().From("postgres").
        WithEnvVariable("POSTGRES_HOST_AUTH_METHOD", "trust").
        WithExposedPort(5432)

    db := base.WithMountedCache("/var/lib/postgresql/data", cache)
    _, err := m.
        Sqlx().
        WithServiceBinding("db", db.AsService()).
        WithEnvVariable("DATABASE_URL", "postgres://postgres@db").
        WithDirectory("migrations", &migrations).
        WithExec([]string{
            "sh", "-c",
            `
                pg_isready -d "$DATABASE_URL";
                while test "$?" -eq 1; do
                    pg_isready -d "$DATABASE_URL";
                done;
                cargo sqlx db reset -y;
                psql $DATABASE_URL -c '\d';
            `,
        }).
        Sync(ctx)
    if err != nil {
        return nil, err
    }

    return base.WithMountedCache("/data", cache).
        WithEnvVariable("CACHEBUSTER", time.Now().String()).
        WithExec([]string{"cp", "-r", "/data", "/var/lib/postgresql/"},
            ContainerWithExecOpts{SkipEntrypoint: true}).
        WithExec(nil), nil
}

func (m *M) Sqlx() *dagger.Container {
    return dag.Container().From("rust").
        WithExec([]string{"cargo", "install", "sqlx-cli"}).
        WithExec([]string{"sh", "-c", "apt-get update && apt-get install -y postgresql-client"})
}

#

Little explanation regarding the loop: according to the pg_isready manual, the exit status code of 1 means that the db is up, but not ready to accept connections. Quoting directly:

pg_isready returns 0 to the shell if the server is accepting connections normally, 1 if the server is rejecting connections (for example during startup), 2 if there was no response to the connection attempt, and 3 if no attempt was made (for example due to invalid parameters).

#

So, now the issue here is that the last db image (the one at the end of Database method) doesn't invalidates its cache when the actual cache is changing, thus doesn't execute the cp command. If my logic is correct, that is. It might be not how Dagger suppose to work.

#How to create database container with migrations already applied?