Can't start session with kubernetes

[simple mauve]

When I try to build a image from a kubernetes container with the mounted dagger socket, I get the following error message.
The dagger engine is deployed on the same kubernetes cluster using a daemon set. We are using this environment variable: _EXPERIMENTAL_DAGGER_RUNNER_HOST=unix:///var/run/dagger/buildkitd.sock
Setup was already discussed in https://discord.com/channels/707636530424053791/1169301950811099256 (by @gilded galleon )
Error message:

│ 1: [2.52s] Failed to connect; retrying... name:"error"  value:"make request: Post "http://dagger/query/": rpc error: code = Unknown desc = failed to get client metadata for session call: failed to JSON-unma │
│ 1: [8.29s] Failed to connect; retrying... name:"error"  value:"make request: Post "http://dagger/query/": rpc error: code = Unknown desc = failed to get client metadata for session call: failed to JSON-unma │
│ 1: [12.9s] Failed to connect; retrying... name:"error"  value:"make request: Post "http://dagger/query/": rpc error: code = Unknown desc = failed to get client metadata for session call: failed to JSON-unma │
│ 1: [35.2s] Failed to connect; retrying... name:"error"  value:"make request: Post "http://dagger/query/": rpc error: code = Unknown desc = failed to get client metadata for session call: failed to JSON-unma │

[languid ice]

Hey @simple mauve!! Can you tell me which Dagger Version you are running both in the client and in the engine?

[simple mauve]

Hi @languid ice We updated the dagger sdk version to the version of the engine (0.11.4) and the pipeline gets executed now. After the image is built an error occured:

  ✔ export layers 0.7s
  ✘ remotes.docker.resolver.HTTPRequest 0.1s
    ✘ HTTP HEAD 0.1s
  ✘ remotes.docker.resolver.HTTPRequest 0.1s
    ✘ HTTP HEAD 0.1s
  ✘ remotes.docker.resolver.HTTPRequest 0.1s
    ✘ HTTP HEAD 0.1s
  ✔ remotes.docker.resolver.HTTPRequest 3.3s
    ✔ HTTP PUT 3.3s
  ✔ remotes.docker.resolver.HTTPRequest 1.4s
    ✔ HTTP PUT 1.4s
  ✔ remotes.docker.resolver.HTTPRequest 1.3s
    ✔ HTTP PUT 1.3s
  ✔ remotes.docker.resolver.HTTPRequest 1.4s
    ✔ HTTP PUT 1.4s
  ✔ remotes.docker.resolver.HTTPRequest 1.4s
    ✔ HTTP PUT 1.4s
  ✘ remotes.docker.resolver.HTTPRequest 0.0s
    ✘ HTTP HEAD 0.0s
  ✘ remotes.docker.resolver.HTTPRequest 0.0s
    ✘ HTTP HEAD 0.0s
  ✔ remotes.docker.resolver.HTTPRequest 0.1s
    ✔ HTTP POST 0.1s
  ✘ remotes.docker.resolver.HTTPRequest 0.0s
    ✘ HTTP HEAD 0.0s
  ✔ remotes.docker.resolver.HTTPRequest 1.5s
    ✔ HTTP PUT 1.5s
  ✔ remotes.docker.resolver.HTTPRequest 1.5s
    ✔ HTTP PUT 1.5s
  ✔ remotes.docker.resolver.HTTPRequest 2.6s
    ✔ HTTP PUT 2.6s
  ✘ remotes.docker.resolver.HTTPRequest 0.0s
    ✘ HTTP HEAD 0.0s
  ✘ remotes.docker.resolver.HTTPRequest 0.0s
    ✘ HTTP HEAD 0.0s
  ✔ remotes.docker.resolver.HTTPRequest 0.2s
    ✔ HTTP POST 0.2s
  ✔ remotes.docker.resolver.HTTPRequest 9.6s
    ✔ HTTP PUT 9.6s
  ✔ remotes.docker.resolver.HTTPRequest 1.4s
    ✔ HTTP PUT 1.4s
  ✘ remotes.docker.resolver.HTTPRequest 0.0s
    ✘ HTTP HEAD 0.0s
  ✔ remotes.docker.resolver.HTTPRequest 0.7s
    ✔ HTTP PUT 0.7s

[languid ice]

Did the overall dagger command failed? Those "x" in the HTTPRequest are confusing and do not necessarily indicate a failure to the overall pipeline. We are working to remove some of this OTEL spans that get exposed to users since they are not very helpful and often generate noise (more details on the discussion here: #daggernauts message)