Seed postgres db with data before running tests

[hot cypress]

I want to seed some data into the db before I run the test that depends on it

but for some reason I am getting error that the psql command is not found becuse it atached to the worng contaiiner how can I resolve it
the code is in the file

the error:

19  :   ! process "docker-entrypoint.sh psql -U postgres -d bookstore_db -a -f /src/scripts/db/bookstore-schema.sql" did not complete successfully: exit code: 1
19  :   [0.1s] | node:internal/modules/cjs/loader:1145
19  :   [0.1s] |   throw err;
19  :   [0.1s] |   ^
19  :   [0.1s] | 
19  :   [0.1s] | Error: Cannot find module '/src/psql'

[broken shoal]

It looks like you dont have psql installed, I don't think it comes by default in the node:21-slim container, that is where you are executing the command from.

This should work if you add a new step to your build_env function to apt install -y postgresql-client

[fresh marsh]

This module might also be useful 🙂
https://daggerverse.dev/mod/github.com/quartz-technology/daggerverse/postgres@627fc4df7de8ce3bd8710fa08ea2db6cf16712b3#Postgres.client

[sour otter]

I love this use case by the way!

[hot cypress]

Thx for the help
I managed to overcome this issue using this docs
https://docs.dagger.io/manuals/developer/services/

but I still not able to persist the creation of the tables that happening in the seed stage

I have test that depends on those table and it not exist

if it will help this is the code of intgretion test
https://github.com/Yedidyar/remult-cli/blob/main/integration/integration.spec.ts

Am I in the right direction?
I am still new to this
I have a reference with working CI using earthly

https://github.com/Yedidyar/remult-cli/blob/main/Earthfile

[hot cypress]

by the way, I listened to the podcast on devtools.fm, it was great talk

[hot cypress]

I will try to simplify it using the module.

[broken shoal]

@hot cypress do you have a branch with the Dagger code in it, it would be great to try to reproduce on my end

[hot cypress]

I will push it soon and ping you

[hot cypress]

@broken shoal https://github.com/Yedidyar/remult-cli/tree/create-dagger-pipeline

here

[fresh marsh]

@hot cypress This is working 🙂
https://github.com/jpadams/remult-cli/blob/create-dagger-pipeline/dagger/src/main/__init__.py

dagger call test --source .

[hot cypress]

can you explain this function usage

def debug(self, source: dagger.Directory) -> dagger.Container:

[fresh marsh]

I just removed it in opting for sometihng even simpler 🙂
I just moved everything into build_env, but yeah, since you can return a container, you can then do things with it on the CLI.

For example, I can call my version of the build-env function from my fork/branch using your repo for the source. Since thebuild-env func returns a Container, I can do things like run a terminal to jump into it 🙂 or I can pass it to the test function to use for running tests 🙂

dagger call -m github.com/jpadams/remult-cli@create-dagger-pipeline \
build-env --source https://github.com/Yedidyar/remult-cli terminal --cmd bash

(anyone can run that ⬆️ )

And in a Container like that, you can look around, run commands, or install software to figure out what you need or debug an issue with CI and then maybe put the fix in code 🙂

So in an interactive session like the one above I can run:

psql postgresql://postgres:postgres@db:5432/bookstore_db -a -f scripts/db/bookstore-schecma.sql
psql postgresql://postgres:postgres@db:5432/bookstore_db

and I'm connected to the DB with all my test data loaded. \dn to see the schemas.

Then I can exit psql and run pnpm test:ci

Incidentally, on the main branch of your repo above, the tests fail 🙂

[hot cypress]

Wow

[fresh marsh]

but they pass if I specifiy the branch

dagger call -m github.com/jpadams/remult-cli@create-dagger-pipeline \
build-env --source https://github.com/Yedidyar/remult-cli\#create-dagger-pipeline terminal --cmd bash

[hot cypress]

This is amazing

[hot cypress]

how can I use terminal --cmd bash in any call command
for instance, dagger call test --source=. --progress=plain terminal --cmd bash

didn't work for me

[fresh marsh]

We're able to call the terminal function on the result of the build-env function since build-env returns a Dagger Container (which has a ton of methods/functions on it).

You can run and compare
dagger call build-env --source=. --help
vs
dagger call test --source=. --help
to see the available functions to chain

Since test just returns a string, it just gets shown on the stdout and we don't have any functions to act on it.

but....something like what you're describing is on a lot of our minds
https://github.com/dagger/dagger/issues/4463

[hot cypress]

Yeah its a great feature that exist in your alternative earthly

[fresh marsh]

Yep! We want to support that use case AND keep the sandbox really secure.
So what I do now is create my own "breakpoints" with functions 🙂

base + command A + command B works
base + command A + command B + command C breaks, so I return a Container there and open a terminal 🙂
(like my debug function before)

[hot cypress]

Looking forward for this it is killer feature in my opinion

[hot cypress]

@fresh marsh Follow up question
How can i cache the build kit docker layers beetwen runs?

I required to use the cloud solution?

[broken shoal]

If you're trying to do this in CI envionrment with ephemeral builders then yes Cloud is one option, but you could also run the engine yourself on a large server. This way your CI servers become "thin clients" that dispatch jobs to the engine.

There are also other options like running in Kubernetes. You can see how we run our own CI in this post: https://dagger.io/blog/argo-cd-kubernetes

[mild wind]

Have you guys looked into Kargo?