I just had to share this, courtesy of @agile geyser ... https://macoscontainers.org
#macos
1 messages Β· Page 1 of 1 (latest)
Unfortunately it requires to disable SIP π¦
I have no idea what that is, which makes it OK! π
lol unfortunately Itβs not okay. You need to restart your MacBook in recovery mode and type in a command in the terminal to disable the SIP (system integrity protection) https://developer.apple.com/documentation/security/disabling_and_enabling_system_integrity_protection and itβs a very bad idea to encourage that.
Yeah, that... puts a damper on things...
I thought disabling SIP is what everyone does after getting a new mac. π
I can't decide if I should laugh or cry... π
You should rather laugh. It's better. π
Not on company MacBooks π
I just see that homebrew dagger is also available on official homebrew too https://github.com/Homebrew/homebrew-core/blob/master/Formula/d/dagger.rb. I'm quite sure that we should use from dagger official. And is it violate trademark because of its build on their own, not using binary from dagger?
I will check it out, thanks for the note
Since there's the homebrew tap maintained by dagger (https://github.com/dagger/homebrew-tap) maybe we just request they remove that recipe
Yes agreed. It's on my todolist, just didn't have time to get around to it yet
yo, I'm new to Dagger and I'm trying some stuff on MacOS
I'm running into "no space left on device" errors when running some functions
I exec-d into the dagger engine container and I noticed this
/ # df -h
Filesystem Size Used Available Use% Mounted on
overlay 109.7G 106.0G 0 100% /
tmpfs 64.0M 0 64.0M 0% /dev
shm 64.0M 0 64.0M 0% /dev/shm
/dev/vda1 109.7G 106.0G 0 100% /etc/resolv.conf
/dev/vda1 109.7G 106.0G 0 100% /etc/hostname
/dev/vda1 109.7G 106.0G 0 100% /etc/hosts
/dev/vda1 109.7G 106.0G 0 100% /var/lib/dagger
/dev/vda1 109.7G 106.0G 0 100% /etc/dnsmasq-resolv.conf
overlay 109.7G 106.0G 0 100% /etc/resolv.conf
does this make sense?
ah ok it seemed to be a local docker engine issue, I did a few prune commands and it's find now π
If you go to settings on docker desktop, you can increase the size of the virtual disk used by docker. This does not solve it, it only postpone the problem. Besides, if you need ever to reduce, it deletes all containers and images in it to create a new one with the decreased size.
If you donβt have space on your system to increase docker disk image, youβd have to prune. This link teaches how to do it. Just start all containers you donβt want deleted and run the command docker system prune
https://docs.docker.com/desktop/faqs/macfaqs/#how-do-i-reduce-the-maximum-size-of-the-file
Oh I just saw you fixed it!
Hi - just getting started with dagger and tried to run "dagger init --sdk go --source ./dagger" in my dev folder and notice that all of a sudden it tries to traverse and upload(?) my entire $HOME folder to what I think is the docker engine. Why on earth does it want to do that? I immediately hit ctrl+c to break it #1030538312508776540
Upload $HOME
I wonder if this could be used to add support for native mac execution? https://github.com/trycua/lume
--> We could provision a Mac VM on the fly and execute commands in there
i might be missing something, but this lume is basically lima, but also supports running macos VMs... the missing piece we'd need for native mac execution is something like buildkit-running-Virtualization.Framework-VMs-as-if-they-were-containers
and i don't think anybody's gonna build that bc there are too many limitations? like Virtualization.Framework, despite being fast and close to the metal, is still doing hardware virtualization, so every VM gets its own OS, FS, kernel, etc
I'm picturing something like a separate dag.Mac() core function, which would return a Host-like type, where you could execute command, read and write files, etc. You wouldn't get caching etc. but at least you could orchestrate xcode builds and get files in and out. The key is that it would be in a VM controlled by dagger, and isolated from the actual host so OK to give a module control of (maybe VM is namespaced by module, so each module gets exactly one VM to mess around with)
I'm eager to upgrade to v0.16.0 but am hitting a snag:
brew upgrade dagger/tap/dagger
==> Upgrading 1 outdated package:
dagger/tap/dagger 0.15.4 -> 0.16.0
==> Fetching dagger/tap/dagger
==> Downloading https://dl.dagger.io/dagger/releases/0.16.0/dagger_v0.16.0_darwin_arm64.tar.gz
########################################################################################################################################################################################################################################### 100.0%
Error: SHA256 mismatch
Expected: 5ae286047958a70e6ed541c81e5fbc4550b979ecb75d2fb95c1c132907c9945b
Actual: 4d19b0f10a2cbaeeff7a6a4e182fbcd40f12243bad0f68a4e6e0765a74105647
File: /Users/matt/Library/Caches/Homebrew/downloads/969b15c0ba09d53e123325d4f35c285bee4dda1a72217e31482886d5c03400ce--dagger_v0.16.0_darwin_arm64.tar.gz
To retry an incomplete download, remove the file above.
Error persists after removing the tar and trying again.
Thanks! We are currently investigating!
This should be fixed now
Yep, works great now. Thanks!
I wonder if we could co-opt this idea for solving the problem of native execution in dagger. https://news.ycombinator.com/item?id=43985624
GreenGames
Hey HN, we're excited to share Lumier (https://github.com/trycua/cua/tree/main/libs/lumier), an open-source tool for running macOS and Linux virtual machines in Docker containers on Apple Silicon Macs.When building virtualized environments for AI agents, we needed a reproducible way to package and distribute macOS VMs. Inspired by projects like ...
it shouldn't be difficult to make a dagger module that uses similar primitives, you can even re-implement part of the dagger core API so some of the chaining semantics still remain.
dag.MacVM().WithExec([]string{"xcode", "build). Under the hood the module can start the VM and setup some ssh or whatever mac has to perform RCE's against the VM
@polar ivy it can't be just a module though, something needs to natively execute the lume CLI on the mac host. My thinking is maybe dagger itself could do that as a hardcoded integration, similar to 1password integration
right... I was assuming the lume CLI part was going to be embedded in the Dagger CLI
It could work right?
yep, totally
we can do the same in Windows with HyperV also
yeah i never responded to this before but as you described it in february it sounds feasible to me
by following some of the Lume devs I've also came across with this https://github.com/cirruslabs/tart/
seems like as similar thing
"Push/Pull virtual machines from any OCI-compatible container registry."
"Easily integrates with any CI system."
How does that affect Dagger?
In a minor positive way. It makes it possible, in theory, to run the dagger engine on Mac without having to install Docker, Orb, Colima, etc.
When will Dagger support it (if it will at all)?
let's continue discussing here.
@desert cedar is currently doing some testing with this. Apparently it "seems" like we might be lucky and supporting that in Dagger could be straightforward. We're doing some tests now and will report back once we have a better understanding on the coverage of what works.
Yep! I'm running it today. Want to make it a great experience π
πππππ
Docs shipped!
https://docs.dagger.io/ci/integrations/apple-container/
Things to figure out.
https://github.com/dagger/dagger/issues/10666
GitHub
We have preliminary docs here https://docs.dagger.io/ci/integrations/apple-container/ but there are a lot of improvements to be made to make the DX of using https://github.com/apple/container great...
Just updated to Tahoe and wanted to try containers. I followed the instructions here, however I'm getting the following message:
βΌ connect 0.0s ERROR
! start engine: failed to run command [docker info]: exit status 64
β°ββΌ exec docker info 0.0s ERROR
β Error: failed to find plugin named container-info
β Usage: container [--debug] <subcommand>
β See 'container --help' for more information.
! failed to run command [docker info]: exit status 64
The engine is running:
$ container ls
ID IMAGE OS ARCH STATE ADDR
dagger-engine-v0.18.19 registry.dagger.io/engine:v0.18.19 linux arm64 running 192.168.64.2
$ echo $_EXPERIMENTAL_DAGGER_RUNNER_HOST
docker-container://dagger-engine-v0.18.19
Any clues?
π I was able to reproduce the issue on my mac. It looks like dagger executes docker info but there is no container info. My educated guess is that we recently introduced this change in the dagger CLI, and accidentally broke this docs page (we don't test this configuraiton in our CI). Let me look into the code.
this pr's description probably needs to make it in the docs but i got it to work by prefixing the host with image+apple / container+apple instead of docker-container
also, got it working without setting the runner host by removing docker from the PATH
@somber adder @slim ibex an update on this...
- Our docs are wrong π¬ aliasing
dockertocontainerdoes not work - As of the current stable release, Dagger will automatically pick
containerif it's available, and fall back todocker. - Make sure to remove your
dockeralias, and unset_EXPERIMENTAL_DAGGER_RUNNER_HOST. With just a vanilla system configuration, and a vanilla Dagger configuration, everything should work the way you want
Sorry for the confusion! We're going to remove the docs page.
cc @barren ether who clarified things for me π
Specifically in your case: yes what you did worked. And also, if you just unset the variable altogether, it should also work π
π€ I'm trying to have Dagger working with container, but it looks like not everything is working.
I have dagger core version working as expected and I can see the dagger container been created:
$ dagger core version
βΆ connect 3.1s
β loading type definitions 0.3s
βΆ version: String! 0.0s
β parsing command line arguments 0.0s
v0.18.19
$ container ls
ID IMAGE OS ARCH STATE ADDR CPUS MEMORY
dagger-engine-v0.18.19 registry.dagger.io/engine:v0.18.19 linux arm64 running 192.168.64.4 4 8192 MB
But when I'm trying to go further, I have several issues:
If I'm trying to just create an alpine container:
dagger -M
Dagger interactive shell. Type ".help" for more information. Press Ctrl+D to exit.
βΌ container | from alpine | with-new-file /hi Hello | with-exec -- cat /hi | combined-output 0.3s ERROR
βββ container: Container! 0.0s
βββΌ .from(address: "alpine"): Container! 0.3s ERROR
β ! failed to resolve image "docker.io/library/alpine:latest" (platform: "linux/arm64"): failed to resolve source metadata for
β docker.io/library/alpine:latest: failed to get credentials: error getting credentials - err: exec: "docker-credential-
β desktop": executable file not found in $PATH, out: ``
β β°ββΌ resolving docker.io/library/alpine:latest 0.3s ERROR
β ! failed to get credentials: error getting credentials - err: exec: "docker-credential-desktop": executable file not found
β in $PATH, out: ``
β β°ββΌ remotes.docker.resolver.HTTPRequest 0.3s ERROR
β β°ββ HTTP HEAD 0.3s ERROR
This might be because I still have some Docker Desktop things on my machine, but it means we can't really have Docker and container at the same time on the machine
One other issue is, sometimes, the dagger container just disappear...
$ dagger core version
βΌ connect 31.1s ERROR
! start engine: new client: interrupted
βββΆ exec container system status 0.0s
βββΆ starting engine 0.1s
β°ββΌ connecting to engine 31.0s ERROR
! new client: interrupted
β°ββΌ creating client 31.0s ERROR
! interrupted
βββ moby.buildkit.v1.Control/Info 0.0s ERROR
β ! connection error: desc = "error reading server preface: command [container exec -i dagger-engine-v0.18.19 buildctl
β dial-stdio] has exited with exit status 1, make sure the URL is valid, and Docker 18.09 or later is installed on the
β remote host: stderr=Error: notFound: \"get failed: container dagger-engine-v0.18.19 not found\"\n"
$ container ls
ID IMAGE OS ARCH STATE ADDR CPUS MEMORY
It looks like the client still think the engine is there, trying to access it, but it failed.
It makes me think about the container having a panic and be removed, but I'm seeing nothing in the logs, even if I container logs -f dagger-engine-v0.18.19
It looks like to be reproducible:
$ dagger core version && dagger core version
βΆ connect 3.0s
β loading type definitions 0.3s
β parsing command line arguments 0.0s
βΆ version: String! 0.0s
v0.18.19
Full trace at https://dagger.cloud/dagger/traces/591ea3db8150ee1eef284e6f3d68162c
βΌ connect 6.3s ERROR
! start engine: new client: interrupted
βββΆ exec container system status 0.0s
βββΆ starting engine 0.2s
β°ββΌ connecting to engine 6.1s ERROR
! new client: interrupted
β°ββΌ creating client 6.1s ERROR
! interrupted
βββ moby.buildkit.v1.Control/Info 0.0s ERROR
β ! connection error: desc = "error reading server preface: command [container exec -i dagger-engine-v0.18.19 buildctl
β dial-stdio] has exited with exit status 1, make sure the URL is valid, and Docker 18.09 or later is installed on the
β remote host: stderr=Error: notFound: \"get failed: container dagger-engine-v0.18.19 not found\"\n"
Second call is always failing
interesting, thats all new since I last tested (container 0.2.0), but I can try to have a look at this again
when apple container https://github.com/apple/container/releases/tag/0.5.0 version will be supported in dagger?
we can track this here https://github.com/dagger/dagger/issues/11205
What happens if you try to use it today? Are you sure it's not automatically supported?
Out team is currently exploring to migrate from docker to apple container and for that we just want to make sure all features of apple container we will be able to leverage with dagger or not? because it may break so many things in our whole end to end devops part
π for visibility: https://github.com/dagger/dagger/issues/11242
I had to try the native apple container tooling, I think it's supported... but something funny is going on and I'm not sure is me/apple containers/dagger π
The fix has been merged into main. You should now be able to use it with apple containers as expected π
you guys are π¦Έ
it takes me longer to report issues than you to fix them π
No problem, thank you to have report it. The issue was... interesting. Apple container doing crazy things sadly. But that should be good now, and it's nice for dagger to support apple containers, that's a great alternative to other existing container runtimes.
is there a way to enable NAT on the bridge interface created by the apple container? Essentially, I only want the "host <-> guest" communication only, making the container somewhat isolated.
IIUC you don't want the containers to be able to access to an outside network?
Correct. I want the container only to access the host
do you need to enforce this as a global engine config? Or adding it programatically as an option in a with_exec?
I would like it to be configurable per container.
cool, awesome. Since we have more control of the runtime now I think it's easier to do. cc @vital gust for input here
is there any way to combine two mac power to run one dagger engine? @polar ivy @ionic abyss
Dagger now calls runc directly (rather than via buildkit), so if there's some runc option for this it should be possible.
AFAIK It does. @random creek next step would be creating an issue
I am seeing an error with colima that I'm not seeing with docker desktop. When I run anything on dagger core engine local-cache entry-set entries I get this error
15:06:46 ERR cleanup failed msg="close dagger session" err="shutdown: do shutdown: Post \"http://dagger/shutdown\": command [docker exec -i dagger-engine-v0.20.1 buildctl dial-stdio] has exited with exit status 1, make sure the URL is valid, and Docker 18.09 or later is installed on the remote host: stderr=Error: dial unix /run/dagger/engine.sock: connect: connection refused\nUsage:\n dial-stdio [flags]\n\nFlags:\n --addr string The address serving the grpc api (default \"unix:///run/dagger/engine.sock\")\n -h, --help help for dial-stdio\n --timeout int The timeout in seconds for connecting to the grpc api (default 5)\n\n" duration=1.034878791s
Most everything else runs fine on colima. I can even prune the cache. But the above command fails.
what are the dagger engine and client versions?