#Is there a way to feed output of a command into a LLM prompt?

1 messages · Page 1 of 1 (latest)

errant basin
#

I'm trying to think of ways to combine the CI and LLM use-cases.

For example, if I could do something like:

  • run unit tests
  • if failure:
    • append to prompt asking to summarize the error + suggest fixes
    • post that as a comment in the github PR
    • still exit with a fail exit code, so CI marks as failed properly

I could see that being pretty powerful, but I'm not seeing any straightforward way to wire that up in Dagger.

errant basin
#

Maybe something like this is the way to go?

    try {
      // run dagger ci job(s)
    } catch (error) {
      const command = error.cmd.join(" ");
      console.error(`Error running: ${command}`);
      const summary = await dag
        .llm()
        .withPrompt(
          `
You are a continuous integration diagnostic tool.
Extract the relevant error(s) from the below output and return it.

${command}
${error.stdout}
${error.stderr}
`,
        )
        .lastReply();
      console.error("AI Summary:");
      console.error(summary);
      throw error;
    }

Sort of works - does feel a bit wonky though. Would be neat if there was a way to chain it and make things more 'dagger-esque'?

errant basin
#

Does get logged 4 times, lol 😓

quaint coyote
random edge
#

Good news, there's a video 😄 This covers the exact flow you're describing: https://www.youtube.com/watch?v=VHUi9ABdASA
and here's the code that handles that flow https://github.com/kpenfound/greetings-api/blob/main/.dagger/main.go#L65-L82

YouTube
Dagger
AI Agents in CI

This demo shows how an AI Agent can operate in a CI environment to assist in resolving test failures.

Code: https://github.com/kpenfound/greetings-api

Have questions? Ask us in Discord: https://discord.com/invite/dagger-io

▶ Play video
GitHub
greetings-api/.dagger/main.go at main · kpenfound/greetings-api

Contribute to kpenfound/greetings-api development by creating an account on GitHub.

sage badger
# errant basin I'm trying to think of ways to combine the CI and LLM use-cases. For example, i...

Hi! I have a couple of examples of this in GitHub

  1. Self-healing CI - New PR. CI runs tests with Dagger. When tests fail, it triggers an agent which reads the logs and tries various fixes until the tests work. Once tests pass, the agent posts a comment in the PR with an explanation of the fix, a code diff and a sub-PR with the changes.
    -- Example broken PR: https://github.com/vikram-dagger/fastapi-sample-app/pull/89
    -- Agent-generated comment with fix and diff: https://github.com/vikram-dagger/fastapi-sample-app/pull/89#issuecomment-2863870915
    -- Workflow: https://github.com/vikram-dagger/fastapi-sample-app/blob/main/.github/workflows/dagger.yml
    -- Dagger module: https://github.com/vikram-dagger/fastapi-sample-app/blob/main/.dagger/src/book/main.py

  2. OpenAPI spec writer - New PR which updates REST API endpoints. PR triggers an agent which reads the changes and creates new PRs to update the changelog and OpenAPI spec. Agent posts a comment in the main PR explaining its actions.
    -- Example PR: https://github.com/vikram-dagger/gin-sample-app/pull/48
    -- Agent-generated changelog update: https://github.com/vikram-dagger/gin-sample-app/pull/50/
    -- Agent-generated spec update: https://github.com/vikram-dagger/gin-sample-app/pull/51/
    -- Workflow: https://github.com/vikram-dagger/gin-sample-app/blob/main/.github/workflows/dagger-spec.yml