Running on standard gh runners (ubuntu-latest). we've noticed some pretty slow "connecting to engine" traces (~1m) and now also hit a time out (at 10m).
#connecting to engine timeout or slow in github runners
hushed crest
That ~1m time looks like its been the case for awhile (just hadn't noticed). Is that expected / normal span time ?
worldly comet
Its probably pull time for the engine container. I think the only way to get around this is (at time of writing, the dagger team is hard at work on a lot of stuff!) to self-host a runner, and an added benefit is that you get all the other caching along with this approach.
hushed crest
hrm, i was wondering about that - I could be interpreting the trace wrong, but the exec docker pull registry.dagger.io/engine:v0.14.0 happens in the starting engine - create phase (going by the trace), and thats usually pretty reasonable.
worldly comet
hrm, i was wondering about that - I could be interpreting the trace wrong, but t...
Oh I see what you're saying. I dunno maybe someone from the team will chime in. We went to self-hosted so the pull time is nonexistent (for most runs, save version updates) and the connect time is about 3-4s. I'm sure at least some of the "connect" time is the dagger engine startup time, but a minute does seem like a while. Looking at my older traces on GHA I still don't get that much time on connect, although it is close to 10s, but I get a pretty long prepare step installing the module, which we don't see self-hosting our runners...
spark flax
@hushed crest Is this an OSS repo by any chance?
hushed crest
@spark flax its not unfortunately. I could try to repro it on a public repo if that would help. fwiw - tried it on a larger runner size (the 4 core) but didn't really seem to make a material difference.
spark flax
<@336241811179962368> its not unfortunately. I could try to repro it on a public...
Let me try s quick repro on a public runner
I have a repo to try this