#how to use alternative init process for container

1 messages · Page 1 of 1 (latest)

abstract lake
#

Our product containers use the s6 init system instead of the common default dumb-init. We want to be able to test that our container service starts up correctly using s6.

However using the container.WithExec([]string{"echo", "hello"}), it seems to always use the dumb-init echo hello on the container. Really we'd like to run the /init s6 program as PID 1, and no other init system involved. This would be the default behavior on our container by just setting the entrypoint correct and doing "docker run <our-image>".

Is there a way to use a custom entrypoint for exec on dagger? Or is there a way to "run" a container in dagger?

unique shadow
#

cc @wise trellis just realized this is not documented in our docs 🙏

abstract lake
#

I don't think it is working. It seems like it's prepending both the "/.init" and the user supplied "entrypoint", so there's effectively two entrypoints.

Example:

func (m *Treasury) InitTest(
    ctx context.Context,
) (*dagger.Container, error) {
    container := dag.Container().
        From("alpine:latest").
        WithEntrypoint([]string{"/bin/ps", "axu"}).
        WithExec([]string{}, dagger.ContainerWithExecOpts{UseEntrypoint: true})

    return container, nil
}

https://dagger.cloud/conor/traces/fea359d73ad18d9f38c85703789f297e
https://dagger.cloud/conor/traces/fea359d73ad18d9f38c85703789f297e?span=28e5281641b0c959

I would expect the only binary to run is my example /bin/ps and it should have PID 1. Instead PID 1 is going to /.init still.

unique shadow
#

I think there's a stopgap way where you can replace Dagger's init process but that will mean that all containers spawned by dagger with use the same one if that's ok with you

#

You can basically build your own dagger engine image based on ours and replace the /usr/local/bin/dumb-init binary inside of it with whatever you want.

abstract lake
#

maybe i can overwrite the init binary before executing something daggerfire

unique shadow
abstract lake
#

Do you know how I can tell daggerCLI to use a custom image?

wise trellis
abstract lake
#

I made a working example that involves overwriting dagger engine's dump-init as suggested by marco.

https://dagger.cloud/conor/traces/b04a3a936d26e8880229be231c837824

The replaced init does something like this:

  # use dagger init
  exec /usr/local/bin/dumb-init.bak $@
else
    # use expected docker init
    exec $@
fi```

I'll post the full workaround on the github issue.
unique shadow
burnt jay
#

possibly off-topic: would this custom runner be a way to allow the process for a Service to be restarted while an Exec runs a test that consumes the service?

unique shadow
#

Could you describe with more detail what you're trying to achieve? Maybe there's another way of achieving that..

burnt jay
# unique shadow hi! no, not really. IIUC you need the Service to be programatically restarted at...

yeah, essentially I need to test what the client sees when the service is restarting -- I guess I'm thinking that if custom runner means I can have custom init then I can have that init restart supervised processes that exit with whatever code

the specific case here is a database migration: for one phase I need to fail all incoming writes while all recent writes are migrated to the new db, and I want to step through these phases, showing that the mechanism to advance phases and the online behavior work together as expected ... on the one hand, dagger seems very well-suited to this because it's naively 3 or 4 containers (1 client/tester, 1 app, 2 db's), but it seems like the way dagger wants me to do this is by setting up each phase and testing what would happen in that phase, which kind of removes some of the ambiguity that I want to simulate

unique shadow
# burnt jay yeah, essentially I need to test what the client sees when the service is restar...

I think you can still achieve this. Here's something that I think works:

func (m *Lala) Test(ctx context.Context) (string, error) {

    svc := dag.Container().From("nginx").WithExposedPort(80).AsService()

    svc.Start(ctx)

    go func() {
        time.Sleep(5 * time.Second)
        svc.Stop(ctx, dagger.ServiceStopOpts{Kill: true})
    }()

    return dag.Container().From("alpine:latest").
        WithExec([]string{"apk", "add", "curl"}).
        WithServiceBinding("nginx", svc).
        WithExec([]string{"sh", "-c", "while true; do curl --connect-timeout 1 -s http://nginx; sleep 1; done"}).
        Stdout(ctx)
}
#

^ as you can see, I'm manually stating and stopping the service so curl will eventually start failing after the service has been stopped

burnt jay
#

Sure, but what I need is to have the service operate with different config, not simply stop listening. IIUC, I have no way of achieving config reloading without some additional service providing mutable state and my application or some supervisor entrypoint watching. So there's no good way (IIUC) to say "I plan to accomplish this by rolling all the k8s pods to update the application". Is there a way to keep the Service Binding intact while stopping the Service, changing some state and/or providing a new container exec, and letting that updated/new container then be accessible via the same binding, ie without stopping and starting the consuming container?

unique shadow
#

IIUC you need to stop the service, change some config and start it again, right?

#

You can achieve this by making the service reading its configuration from a cache volume. Here's an updated example:

func (m *Lala) Test(ctx context.Context) (string, error) {

    ns := dag.CacheVolume("nginx-service")

    dag.Container().From("alpine").WithMountedCache("/cache", ns).
        WithExec([]string{"sh", "-c", "echo Hello world > /cache/index.html"}).Sync(ctx)

    svc := dag.Container().From("nginx").WithExposedPort(80).
        WithMountedCache("/usr/share/nginx/html/", dag.CacheVolume("nginx-service")).
        AsService()

    svc.Start(ctx)

    ep, _ := svc.Endpoint(ctx)

    go func() {
        time.Sleep(5 * time.Second)
        svc.Stop(ctx, dagger.ServiceStopOpts{Kill: true})
        dag.Container().From("alpine").WithMountedCache("/cache", ns).
            WithEnvVariable("BUST", time.Now().String()). //bust the cache to force execution
            WithExec([]string{"sh", "-c", "echo Hello Changed! > /cache/index.html"}).Sync(ctx)
        time.Sleep(5 * time.Second)
        svc.Start(ctx)
    }()

    return dag.Container().From("alpine:latest").
        WithExec([]string{"apk", "add", "curl"}).
        WithServiceBinding("nginx", svc).
        WithExec([]string{"sh", "-c", "while true; do curl --connect-timeout 1 -s http://" + ep + "; sleep 1; done"}).
        Stdout(ctx)
}
#

^ as you can see from the above, in this case I use a cache volume to store nginx's index.html file and then I stop the service, update that volume and start it again with the modified config

#

LMK if that works for your use case

#

BTW, seems like what you're building with dagger looks pretty cool. Would you like to present your use-case in our community call? cc @woven heath