#Trying to create separate containers in parallel with separate IP address

1 messages · Page 1 of 1 (latest)

ruby kayak
#

I'm trying to create multiple containers in parallel and network them so they can reach each other. It is a cyclic dependency so Services currently cannot be used.

I am chaining together some .WithExec commands to log the IP address of each machine to a shared file on a cache mount. This is the discovery workaround.

container = container.WithExec([]string{"bash", "-c", "ip route get 1 | ip route get 1 | awk '{print $7}' | xargs >> /shared/peers"})
container = container.WithExec([]string{"bash", "-c", "while [ $(wc -l < /shared/peers) -lt 2 ];  do sleep .5 ; echo waiting for peers to log IP; done"})
container = container.WithExec([]string{"bash", "-c", "echo peers have IP addresses $(cat /shared/peers | paste -sd ',' -)"})
// ..
// later start the services
for _, s := range export {
    go func() {
        s.Start(ctx)
    }()
}

However it seems that the IP addresses that get logged are not stable. It seems in subsequent .WithExec commands, the IP addresses logged beforehand are no longer valid, or that the IP address changed, or that the IP address is the same for both containers.

How can I start containers/services with stable IP addresses?

https://dagger.cloud/cordialsys/traces/7e4b6c1e8ccde4ec50285466ee18a709#970891ce5eb413d5

in this example they came out with the same IP:
https://dagger.cloud/cordialsys/traces/7e4b6c1e8ccde4ec50285466ee18a709?span=73addf0226a7d911

another run, same code, they ended up with different IPs:
https://dagger.cloud/cordialsys/traces/38308549de6d03230a5d7c45d83dcf2b?span=71163e2c220c83ae

ruby kayak
#

.

ruby kayak
#

I tried cramming them all into one Exec , which seems to resolve the IP addresses ending up the same or inconsistent.

#

.

vapid birch
ruby kayak
#

yes it's okay to be ephemeral IP, but i was expecting them to be the same inbetween different WithExec innovactions on the same container.

ruby kayak
#

I was planning to use multiple Exec's on the same container to prepare to run the tests.

  1. Figure out IP address, echo to shared cache
  2. Wait for everyone to echo their IP to shared cache
  3. Run tests with IP addresses input.

It seems to work if i cram into a single Exec, but not as multiple Exec.

vapid birch
ruby kayak
#

I have two containers that depend on each other because they use P2P networking. The services don't work with cycle dependency

#

I'm trying now to not export as services and just manually block on getting Stdout from each container.

vapid birch
#

i.e: this prints the same IP both times:

func (m *Lala) Test(ctx context.Context) (string, error) {
    nginx := dag.Container().From("nginx").WithExposedPort(80).AsService()

    return dag.Container().From("alpine").
        WithServiceBinding("nginx", nginx).
        WithExec([]string{"ping", "-w", "3", "nginx"}).
        WithExec([]string{"ping", "-w", "3", "nginx"}).Stdout(ctx)

}
#

but you can bind them to the container that performs the WithExec to run the tests?

ruby kayak
#

Okay I can try something like this. The issue is I needed two services to discover each other's IP addresses, and I was using manual Exec's on each container before they become services. Often they would have the same IP address at the time of those Exec's, and later not have the same IP address as a service.

Perhaps if I just bind them to a 3rd container then they can just use DNS and not bother with this IP discovery.

vapid birch
vapid birch
chilly sonnet
#

You can just call .start to have it run until the client goes away (or until you call .stop)

#

You'll get a different IP from each withExec because each withExec is actually a separate container

#

So to get around the cyclic dependency you'd need to record the IP and start the service in one shot

ruby kayak
#

is it possible to wait for a service to exit?

#

basically my test is I have two containers that use P2P networking to reach each other, then they exit 0

#

would be much easier if I can use services since DNS could be used

chilly sonnet
vapid birch
#

@ruby kayak you should still use services IMO, you just don't bind them together

#

and perform the service discovery via a cache volume as we spoke before