#replay

1 messages · Page 1 of 1 (latest)

leaden carbon
#

I think that journal file feature was an early prototype that we removed later. how are you accessing it?

#

we never stabilized the format and doing so at the moment would be too constraining

#

“dagger within vitest” that is intriguing, what does that look like in practice?

old sun
#

I believe @safe vine mentioned using _EXPERIMENTAL_DAGGER_JOURNAL to get the journal and using it for real time events in our discussion about getting the engine output. Looking in the code it looks like that flag just adds another progrock writer to a list of writers to output to. It sounded like it's also used to get the interactive TUI to work?

safe vine
#

Yep - the journal file is still there, but it prints JSON stream of Progrock events now. It's still used by the Cloud team for troubleshooting afaik

old sun
#

Here is some snippets of code for how I'm using dagger within vitest

export const TEST_OUTPUT_DIR = "test-output";

export const testConnect = async (context: TestContext, cb: CallbackFct) => {
    const testSuite = context.task.suite.name.replace(/\s+/g, "-");
    const testName = context.task.name.replace(/\s+/g, "-");
    const logPath = `${TEST_OUTPUT_DIR}/${testSuite}-${testName}.log`;
    const log = createWriteStream(logPath);
    console.log(`Logging to ${logPath}`);
    await connect(cb, { LogOutput: log });
};

describe.concurrent("apk", () => {
    it("should install from main", async (context) => {
        await testConnect(context, async (client) => {
            await client
                .container()
                .from("alpine:3.18")
                .with(withApkPackages(["aws-cli"]))
                .withExec(["which", "aws"])
                .sync();
        });
    });

    it("should install from community", async (context) => {
        await testConnect(context, async (client) => {
            await client
                .container()
                .from("alpine:3.18")
                .with(withApkPackages(["kubectl"]))
                .withExec(["which", "kubectl"])
                .sync();
        });
    });
});
old sun
safe vine
#

Ah right, there are languages other than Go. 😅 Well, it'd be easy enough to write a little Go binary that just slurps up the journal file and writes it to the same TUI, or to the console writer, which might be more practical here

#

Jynx - the former

#

Should be mostly just gluing together APIs, let me know if you need any pointers

old sun
#

Will do, thanks!

old sun
old sun
#

@safe vine I'm getting some strangness with the journals that I'm getting. Some of the events are malformed JSON like this one

{"vertexes":[{"id":"sha256:2eaf60b1da55757d511ec1247ba9e556f67ca9416ec8e78c3de743aa324cc01d","name":"pull docker.io/library/alpine:3.18","started":{"seconds":1695919875,"nanos":438{"vertexes":[{"id":"sha256:2eaf60b1da55757d511ec1247ba9e556f67ca9416ec8e78c3de743aa324cc01d","name":"pull docker{"vertexes":[{"id":"sha256:63646db255a3d2462fa9626aa63991dc9e538765a315f61011a165e6834ee4a6","name":"exec apk add --repository http://dl-cdn.alpinelinux.org/alpine/edge/main --repository http://dl-cdn.alpinelinux.org/alpine/edge/community aws-cli","inputs":["sha256:2eaf60b1da55757d511ec1247ba9e556f67ca9416ec8e78c3de743aa324cc01d","sha256:d992cee8196052d190c4e3ab98c46199d4fbf1f5cc301c778732bde899deddfd"],"started":{"seconds":1695919875,"nanos":448749667},"completed":{"seconds":1695919875,"nanos":448749667},"cached":true},{"id":"sha256:2b8bd5ffbd50fac14bccd5700fcfd4f8221cbb06f3fec0db2219d52b0eb3a994","name":"exec which aws","inputs":["sha256:63646db255a3d2462fa9626aa63991dc9e538765a315f61011a165e6834ee4a6","sha256:d992cee8196052d190c4e3ab98c46199d4fbf1f5cc301c778732bde899deddfd"],"started":{"seconds":1695919875,"nanos":448749667},"completed":{"seconds":1695919875,"nanos":448770792},"cached":true}],"sent":{"seconds":1695919875,"nanos":449177084}}
#

formatted

{
  "vertexes": [
    {
      "id": "sha256:2eaf60b1da55757d511ec1247ba9e556f67ca9416ec8e78c3de743aa324cc01d",
      "name": "pull docker.io/library/alpine:3.18",
      "started": {
        "seconds": 1695919875,
        "nanos": 438{
          "vertexes": [
            {
              "id": "sha256:2eaf60b1da55757d511ec1247ba9e556f67ca9416ec8e78c3de743aa324cc01d",
              "name": "pull docker{"vertexes":[{"id":"sha256: 63646db255a3d2462fa9626aa63991dc9e538765a315f61011a165e6834ee4a6","name":"exec apk add --repository http: //dl-cdn.alpinelinux.org/alpine/edge/main --repository http://dl-cdn.alpinelinux.org/alpine/edge/community aws-cli","inputs":["sha256:2eaf60b1da55757d511ec1247ba9e556f67ca9416ec8e78c3de743aa324cc01d","sha256:d992cee8196052d190c4e3ab98c46199d4fbf1f5cc301c778732bde899deddfd"],"started":{"seconds":1695919875,"nanos":448749667},"completed":{"seconds":1695919875,"nanos":448749667},"cached":true},{"id":"sha256:2b8bd5ffbd50fac14bccd5700fcfd4f8221cbb06f3fec0db2219d52b0eb3a994","name":"exec which aws","inputs":["sha256:63646db255a3d2462fa9626aa63991dc9e538765a315f61011a165e6834ee4a6","sha256:d992cee8196052d190c4e3ab98c46199d4fbf1f5cc301c778732bde899deddfd"],"started":{"seconds":1695919875,"nanos":448749667},"completed":{"seconds":1695919875,"nanos":448770792},"cached":true}],"sent":{"seconds":1695919875,"nanos":449177084}}
#

Oop, I see what's going on actually. I'm getting a bunch of tests trying to write to the same journal 😛

safe vine
#

ha yep that'd do it

old sun
#

Getting this working in concurrent tests might be harder than I anticipated >_>

#

I did get something working though, and it is very nice so far. It makes it quite a bit easier to read how long something took.

image.png
#

I do see that logs get tailed. Is there an option to show more data? I might just be missing an option somewhere.

Will also have to see how I can get this to work with the interactive TUI next, which I think is going to need to fork dagger 😛

image.png
safe vine
#

dumb question, but have you considered just wrapping the test suite in dagger run [command to run tests]? You can even use the pipelines API so that each test is grouped in the TUI output

old sun
safe vine
#

ah, how does it break with vitest?

old sun
#

We get this far then it never finishes

image.png
#

This is trying to run a single test which is why I tried single thread, same problems happens with single thread disabled

old sun