command works locally but fails in github actions -- what to check?

[chilly turret]

The command in question is running a suite of tests that consume a service and the last test in the suite fails with a tcp/io error, even when reordering the tests in the suite. It feels like some kind of teardown is early in CI and/or somehow more tolerant on my workstation. Plenty of other commands that run tests with the same service dependencies work fine. I'm not sure how to get started here. My laptop is M1 and the GHA machine is ubuntu and it's the same version of dagger: v0.14.0.

[torpid pelican]

👋 are you able to share a few elements from your pipeline, perhaps the commands you use in your CI configuration? And the signature of the function you call, in particular how the service is received

ideally a trace url also

[chilly turret]

the GHA pipeline does nothing but clone and auth to the docker registry then call dagger:

name: pycassa-tests
on:
  pull_request:
    paths-ignore:
      - '**.md'

permissions:
  contents: read
  packages: write
  # Allows fetching of an OIDC token
  id-token: "write"

jobs:
  
  test:
    uses: ./.github/workflows/dagger.yml
    with:
      args: pycassa-tests --directory-arg "$PWD"

and the reusable workflow:

name: dagger
on:
  workflow_call:
    inputs:
      args:
        required: true
        type: string

permissions:
  contents: read
  packages: write
  # Allows fetching of an OIDC token
  id-token: "write"

jobs:
  
  test:
    runs-on: ubuntu-latest
    steps:
      # this repo
      - name: Checkout repository
        uses: actions/checkout@v4

      - name: gcp auth
        uses: 'google-github-actions/auth@v2'
        with:
          REDACTED

      - name: "Set up Cloud SDK"
        uses: "google-github-actions/setup-gcloud@v1"

      - name: "GAR Docker auth"
        run: |-
          gcloud auth configure-docker us-docker.pkg.dev

      - name: dagger call
        uses: dagger/dagger-for-github@v7
        with:
          version: "latest"
          verb: call
          args: ${{ inputs.args }}

I haven't set up traces yet

one potentially questionable thing I'm doing is some db init shenanigans in the dagger function that sets up the db service ... so maybe that can cause dagger to lose track of the DAG in some cases?

I'm doing Exec with Sync using another container that consumes the service before returning the service

(discord failed to send when I tried to paste the code)

[torpid pelican]

yeah the db init use case is a known issue . Not sure why it would work in one environment vs another though 🤔

ah it might be caching

or lack thereof in CI ephemeral runner

[chilly turret]

so, the init is happening in both environments and none of the logs indicate caching, which in different from another function over here: https://discord.com/channels/707636530424053791/1311421776118550608

it's literally just at teardown the last test will fail with unexpected EOF on the last thrift call, which seems inexplicable because what signal is anyone getting that things are done before pytest exits?