Mounted Filesystem

[dire vapor]

Hey team! I have been trying to replace a bash script I have with with equivlant go/dagger solution. I am having issues with the mounted directory...please see output

This is a simpified version what what I am trying to do, but I would expect to see a file names /tmp/derp/derp when I view the filesystem on the host. Any thoughts? Thanks!

└─❯ mount  | grep derp                           
/home/derp/rootfs.ext4 on /tmp/derp type ext4 (rw,relatime)

┌─────────────── morpheus | 16-Jan-2024 13:43 [016] ────────────────
│ nats [localhost] 
│ 
│ ~/D/derp 🐹 v1.21.5
└─❯ file /home/derp/rootfs.ext4
/home/derp/rootfs.ext4: Linux rev 1.0 ext4 filesystem data, UUID=61ddeea0-e736-4fee-b43b-9fefc698dddc (needs journal recovery) (extents) (64bit) (large files) (huge files)

│ ~/D/derp 🐹 v1.21.5
└─❯ cat main.go                                  
───────┬─────────────────────────────────────────────────────────────────────────────────
       │ File: main.go
───────┼─────────────────────────────────────────────────────────────────────────────────
   1   │ package main
   2   │ 
   3   │ import (
   4   │     "context"
   5   │     "os"
   6   │ 
   7   │     "dagger.io/dagger"
   8   │ )
   9   │ 
  10   │ func main() {
  11   │     client, _ := dagger.Connect(context.Background(), dagger.WithLogOutput(os.St
       │ derr))
  12   │     defer client.Close()
  13   │ 
  14   │     // this is an ext4 fs mounted at /tmp/derp
  15   │     rootfs := client.Host().Directory("/tmp/derp")
  16   │ 
  17   │     client.Container().
  18   │         From("alpine:latest").
  19   │         WithMountedDirectory("/tmp/rootfs", rootfs).
  20   │         WithExec([]string{"touch", "/tmp/rootfs/derp"})
  21   │ }
───────┴─────────────────────────────────────────────────────────────────────────────────

[raw tapir]

Hey! Sorry I am having trouble seeing what the issue is - could you help me understand what you are expecting to see (or not see) in this output?

[dire vapor]

Hey! So I am trying to build a rootfs via dagger. I am currenly using this shell script (which works) and I am just trying to recreate it using the Go SDK

1 │ #!/bin/bash
2 │
3 │ set -xe
4 │
5 │ dd if=/dev/zero of=rootfs.ext4 bs=1M count=100
6 │ mkfs.ext4 rootfs.ext4
7 │ mkdir -p /tmp/my-rootfs
8 │ mount rootfs.ext4 /tmp/my-rootfs
9 │
10 │ docker run -i --rm
11 │ -v /tmp/my-rootfs:/my-rootfs
14 │ alpine sh <setup-alpine.sh #this just installs some things and copys the filesystem to /tmp/my-rootfs
15 │
16 │ umount /tmp/my-rootfs

For whatever reason, anything I do to that mounted directory isnt actually happening

[raw tapir]

Thanks, I see, give me a second and ill try to reproduce to make sure we are on the same page.

[dire vapor]

awesome, thank you

[raw tapir]

Do you have time to pair for a min on this in #911305510882513037

[dire vapor]

i will in about 5 minutes, does that work?

[raw tapir]

Yeah ill be there 🙂

[raw tapir]

Jordan and I did a pairing session, I think we got on the right track using Directory.Export() but we still are not able to see the file system locally, it shows up empty with just a lost+found, he is going to share a minimum reproduction for us to look at soon.

Does anyone else happen to know if there is anything special we need to do when we are working with a mounted filesystem instead of just a directory?

Also another question does WithMountedDirectory work the same way as docker -v or is there somethign different about it? We were confused about the difference between WithDirectory and WithMountedDirectory

[raw tapir]

Hey @dire vapor I was able to get this to work using a filesystem that I created using the same approach you shared, with the following code:

If I recall the only difference is is the directory that we are using for the export path. Can you try to change it to be the same?

package main

import (
    "context"
    "os"

    "dagger.io/dagger"
)

func main() {
    ctx := context.Background()
    client, _ := dagger.Connect(ctx, dagger.WithLogOutput(os.Stderr))
    defer client.Close()
    // this is an ext4 fs mounted at /tmp/my-rootfs
    rootfs := client.Host().Directory("/tmp/my-rootfs")

    client.Container().
        From("alpine:latest").
        WithDirectory("/tmp/rootfs", rootfs).
        WithExec([]string{"touch", "/tmp/rootfs/derp"}).
        WithExec([]string{"tree", "/tmp"}).
        Directory("/tmp/rootfs").
        // same as host where its mounted
        Export(ctx, "/tmp/my-rootfs")
}

Hm, I am still having an issue, I can see the file just file in the mounted directory, but if I try to umount and remount then it comes back empty :/

Oh hm, wait nevermind, it does seem to be working for me when I remount! 😄

ubuntu@nuc:/tmp/mnt$ ls
my-rootfs
ubuntu@nuc:/tmp/mnt$ tree my-rootfs/
my-rootfs/

0 directories, 0 files
ubuntu@nuc:/tmp/mnt$ sudo mount /home/ubuntu/dagger/rootfs.ext4 /tmp/mnt/my-rootfs
ubuntu@nuc:/tmp/mnt$ tree my-rootfs/
my-rootfs/
└── derp

0 directories, 1 file

[dire vapor]

That’s awesome! I will try as soon as I get home. Thanks a ton

[hushed tide]

👋 take into account that mounted files or dirs cannot be exported as they don't belong to the layered image being built. You need to use WithDirectory instead of WithMountedDirectory if you want that to be exportable

and/or you can copy files or directories from a mounted directory to a non mounted directory and export that instead.

we could clarify that here: https://docs.dagger.io/421437/work-with-host-filesystem/#export-a-directory-from-a-container-to-the-host
opening an issue for that: https://github.com/dagger/dagger/issues/6434 cc @patent stag

[raw tapir]

@hushed tide hm that’s confusing because in the example above it does seem to work, that file is exported from the mounted directory.

What is the recommended approach to get behavior similar to what you get when you run docker -v, that’s what we’re actually after.

[hushed tide]

this code you mean? #1196918439185494116 message

[raw tapir]

Yes that code, note we’re passing along a mounted file system, not sure if that has something to do with it

[hushed tide]

What is the recommended approach to get behavior similar to what you get when you run docker -v, that’s what we’re actually after.

you can't bind-mount things from your host to a pipeline. If you need to fetch something from the pipeline, you need to use the Export call

explained here: https://docs.dagger.io/421437/work-with-host-filesystem/#export-a-directory-from-a-container-to-the-host

[raw tapir]

Okay that makes sense and that’s the same conclusion we reached earlier. Thanks for clarifying!

it’s very strange that the code above works, can you take a look and correct any of my assumptions here?

[hushed tide]

checking..

[raw tapir]

I’m also around if you are to pair for a few minutes

[hushed tide]

can't talk now unfortunately, baby's sleeping 😶

@raw tapir I think your approach works because you're effectively exporting the results into the same path where the mounted thing is

so the export is re-creating the whole tree in the host os

[raw tapir]

Ah I see

[hushed tide]

on top of the mount

but it's not effectively modifying the contents of "source" filesystem

having said that the export shouldn't have exported anything

let me check

[raw tapir]

Its not obvious to me what the implications are, I definitely want to do the correct thing 😇

Sounds like all that code is fine except we should switch to WithDirectory?

[hushed tide]

yes, correct

[raw tapir]

@hushed tide is your comment about not being able to export Directories also true for things that are created as a part of the pipeline?

Because in my other dagger module that is building a hugo site that has also been working just fine exporting a directory even thought I am mounting it first.

        return m.Ctr.
            WithMountedDirectory(path, src).
            WithWorkdir(path).
            WithExec([]string{"hugo"}).
            Directory(outPath)

[hushed tide]

so, I was able to repro your case Lev. Seems like something changed recently that make this possible

I'm a bit confused now. probably Justin and/or Erik will have more context about it

[raw tapir]

@dire vapor I updated the snippet above to show WithDirectory instead - but otherwise should work on your end whenever you get a chance to try it out.

Thanks for looking @hushed tide! I think the biggest source of confusion that we had today was its not clear from the docs what the difference between WithMountedDirectory and WithDirectory is.

[hushed tide]

well.. I'm not sure if this is a regression currently

since the idea is that WithMountedX shouldn't impact the layered filesystem of the pipeline

let me do another test really quick

k @raw tapir so seems like changes do not effectively end up in the underlying image but seems like they can be exported. i.e: if you do:

    client.Container().
        From("alpine:latest").
        WithMountedDirectory("/tmp/rootfs", rootfs).
        WithExec([]string{"touch", "/tmp/rootfs/derp"}).
        // same as host where its mounted
        Publish(ctx, "ttl.sh/marcosnils:test1")
}

and then you run: docker run --rm ttl.sh/marcosnils:test1 ls -lad /tmp, you'll see that the container is empty, the rootfs folder doesn't exit.

I have the memory of Export not being possible for mounted directories. Seems like that changed along the way

[dire vapor]

This is all really interesting. Thank you both for the extremely comprehensive walk through

[hushed tide]

another weirder thing. Seems like you can Export the directory but if you call Export on the container, the underlying tar won't also have the files. I'll start a thread in the #maintainers channel

[dire vapor]

@raw tapir i wanted to follow up and let you know this is working now. I really appriciate all your help!

[raw tapir]

Hey @dire vapor that is so good to hear! 😄

Thanks for prompting the discussion, we found a couple of great points of clarification in our docs that will make things easier for everyone in the future.

[sinful tinsel]

hello, not sure if related to this, but I'd rather ask before opening a second separate forum thread. Is the export thing meant to work also in modules executed with dagger call ? In my attempt it looks like not, meaning that everything runs without errors, but the actual location on the host has no content.

[raw tapir]

@sinful tinsel yes it should work, could you share some more details, or if you can the code you’re using?

[sinful tinsel]

sure, it is here https://github.com/BondMachineHQ/BMBuildkit-dagger/blob/main/main.go#L44-L51

and should work out of the box with this example command from the root folder of the repository:

 dagger call --debug build --device-spec lattice/ice40/yosys --target dciangot/my_fpga_firmware:v10 --context-dir ./examples/blinky/ice40/

[proper compass]

When you Export inside the function, it'll export into the runtime container.

[raw tapir]

Thanks! This makes sense, I think I was wrong with my previous statement though.

I ran into this exact scenario when working with a simple hugo project. In my case I ended up not exporting the directory and instead passing it along to downstream jobs.

I think in this case you need to run dagger export instead of dagger call.

The reason why is that when you are running the new dagger modules you are running Docker in Docker so the idea of your host changes a bit.

In order to get this to work the Build function will need to return something exportable, like a Directory

[proper compass]

You need to return the directory/container and use dagger export instead.

[raw tapir]

Thanks @proper compass your answer is a lot more succint! 😄

[proper compass]

At the same time 😄

[sinful tinsel]

awesome! thank you! My suspect was exactly that, but I overlooked the dagger export command.. I'm giving it a try