Can't export an asset and retrieve the command exit code

[gaunt quarry]

I'm running into a design limitation with Dagger. I have an npm audit step that looks like this

   await client
          .container()
          .from('AUDITING_IMAGE')
          .withEnvVariable('SEVERITY_THRESHOLD', 'critical')
          .withEntrypoint(['/bin/sh', '-c'])
          .withExec('node /npm-audit-script.js')
          .file('/npm-report.html')
          .export('./vulnerability_reports/npm-report.html');

I want the the container to export the report showing a failed scan whilst returning the exit code of the command. However, this is not possible as Dagger will halt execution if a non-zero code is returned.

What I have tried to do is capture the exit code into a variable. However, this won't work either as you can't chain exitCode after an export call (or vice versa).

This looks related: https://github.com/dagger/dagger/issues/3192

What are my options? All I can think of is writing the exit code into a file and handling it outside of Dagger, but that feels wrong...

[half trellis]

All I can think of is writing the exit code into a file

Yes, this is your best workaround. May I ask why you need the exit code value? Just trying to understand the use case and best solution.

[gaunt quarry]

May I ask why you need the exit code value?

To determine whether or not the pipeline should fail.

[half trellis]

Is it enough to know if it succeeded or failed?

[gaunt quarry]

Yes, as if there are vulnerabilities reported we block the pipeline to ensure that developers fix it. They can then go look at the produced report to see why it failed

[half trellis]

So, I assume you're using Node.js. When .export() fails, it'll raise an exception. You just need to catch it to handle the failure case. If no exception is raised, then that was a success.

But for you to get the report file either way, you can either split into two steps or store the status somewhere and make sure you return 0 exit code so the pipeline continues.

Example: .withExec('node /npm-audit-script.js; echo $? > /exit_code')

I'm not sure if the report is enough or if you really need the exit code to switch behavior.

[gaunt quarry]

.withExec('node /npm-audit-script.js; echo $? > /exit_code')

Yup that's basically what I ended up doing.

When .export() fails, it'll raise an exception.

Export isn't failing in my example. The problem is primarily about how you can't retrieve an exitCode and export an asset in the same chain. Even if you could, Dagger wouldn't allow it in the event that the exitCode came before the export call. This is because the exitCode call errors in the event that a non-zero code is returned.

Personally I would prefer to just have exitCode return the exit code for me to handle myself. A bit more code is required, but it's more flexible.

[half trellis]

Yes, but the exit code is included in the error.

What do you need to do with the exit code value?

[gaunt quarry]

Maybe I'm not explaining this clearly. What happens is this:

• Script that dagger runs reports failures, pipeline fails
• That script will also produce a report that can be viewed. This cannot be exported onto the host without doing an file().export() call. But because Dagger halted execution, you can't see the report. You also can't call exitCode and file.export() in the same chain.

What would make this possible is if Dagger's mounts had an equivalent to Docker's bind mounts. I'll need to try this again as I don't think it worked, but at the moment if I mounted a directory and wrote a file inside that mounted directory it won't show up on the host without an export . Whereas in docker you can just do this and it works:

docker run -v /host/path:/container/path image audit_scan # non-zero code produced, but I still get to see the file produced

[half trellis]

Dagger is more like a dockerfile than docker run. Mounts from the host are equivalent to the context you send for docker build. That’s why you need to export.

My question was because I'm trying to understand why not just withExec('node /npm-audit-script.js; true'). Do you need to do something with the exit code value or check if it failed later in your script?

[gaunt quarry]

Sure, I'm open to other suggestions on how to solve this one.

withExec('node /npm-audit-script.js; true'). Do you need to do something with the exit code value or check if it failed later in your script?

Because you need to block the pipe if vulnerabilities are reported.

This is the full example:

        console.log('NOW RUNNING NPM AUDIT SCAN');
        await vulnerabilityScanner()
          .withEnvVariable('BUST_CACHE', Math.random().toString())
          .withEnvVariable('DISABLE_NPM_AUDIT', disableNPMAudit || 'false')
          .withEnvVariable(
            'SEVERITY_THRESHOLD',
            'critical'
          )
          .withExec(
            'node /tmp/spanners-vulnerability-scan.js; echo $? > /tmp/reports/npm-exit-code.txt'
          )
          .withExec('cp /usr/src/app/spanners-npm-report.html /tmp/reports')
          .directory('/tmp/reports')
          .export('./vulnerability_reports');
        console.log('FINISHED RUNNING NPM AUDIT SCAN'); 
        if (
          fs
            .readFileSync('./vulnerability_reports/npm-exit-code.txt')
            .toString()
            .trim() !== '0'
        ) {
          process.exit(1);
        }

[half trellis]

Thanks! I have a suggestion, but first of all, if you don't need the exit code in the host, you can just await dir.file("/npm-exit-code-txt").contents() and keep it in the pipeline.

[gaunt quarry]

But I still need to export other files in that directory, or is that still possible?

Specifically there's two files:

• spanners-npm-report.html
• npm-exit-code

In the code I only use the second one. But a human can go and look at the other one in their web browser

[half trellis]

Yes, no problem.

const dir = ....directory('/tmp/reports')
await dir.export('./vulnerability_reports')
if (await dir.file("npm-exit-code.txt").contents()) !== '0') {
  process.exit(1);
}

You can put that file somewhere else so it doesn't pollute your host (via export).

[gaunt quarry]

That's useful, thanks

[gaunt quarry]

Just a heads up that your variant produces an issue where the terminal output seems to be emitted after what looks like the process has exited. A Ctrl + C fixes it. I had a similar issue with SIGINT that was reported here:

https://github.com/dagger/dagger/issues/4823

[half trellis]

You’re forcing an exit with process.exit, but dagger still has to properly close the engine session and flush the rest of those logs. Better to use a return to let it finish normally and catch exception at higher level to control the exit.

In the python sdk there’s no issue because of how the exceptions (including exit) are handled, but I’m not familiar with how this is setup in node.js. Will need to take a look at it.