Private modules with Dagger - windows binary & SSH | Dagger | Page 1

knotty harness Sep 3, 2024, 2:00 PM

#

How to use Dagger on Windows and private modules?

#

Context:

Windows doesn't have the same flexibility with the ~/.ssh/config file as macOS and Linux because Microsoft OpenSSH listens to a fixed pipe (\.\pipe\openssh-ssh-agent).

#

knotty harness Sep 3, 2024, 3:32 PM

#

SSH agent forwarding isn't supported on Windows, but you can set up the 1Password WSL integration to authenticate SSH requests in your WSL instance from your Windows host.

knotty harness Sep 3, 2024, 4:02 PM

#

So, i an git clone, from WSL using my Windows 1Password SSH Agent - by doing this:

[core]
    sshCommand = ssh.exe

This is because WSL integration has Windows exe support

#

When I run git clone from WSL, my 1Password app on Windows pops up, asking if i want to auth the request, and it works.

#

running either ssh-add -L or ssh-add -l also both work, but theyre actually using ssh-add.exe and the correct keys from 1Password are listed.

#

SSH_AUTH_SOCK=~/.1password/agent.sock ssh-add -L
ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIOVXHjvVPJrcqwq2f07LmmkKXHEwm9AZxhBX4gFW72xv Personal SSH Key

I was hoping somehow, I could get a workaround of using Linux Dagger from WSL (or maybe even still using dagger.exe, but from WSL)

spice wave Sep 3, 2024, 5:54 PM

#

hey @knotty harness! I don't see a way how the Windows Dagger binary could be currently using the private module support as the way it's currently implemented it solely depends un the SSH_AUTH_SOCK unix socket. IIUC what you're describing is the other way around, correct? It's a way for WSL programs to access the ssh-agent credentials that were set in windows?

knotty harness Sep 3, 2024, 5:57 PM

#

spice wave hey <@130061041580113920>! I don't see a way how the Windows Dagger binary could...

yeah, so my attempt was to try and use the linux dagger, but still use my Windows SSH Agent, i can use my windows ssh agent with 1Password and do git clones from WSL perfectly fine. Its obviously not the end result, but i want to try using private modules asap

#

still no luck right now

#

i think docs need updating about private git module supp - it should be specific, no point releasing a 1.26 windows binary if it doesnt really support it

spice wave Sep 3, 2024, 5:59 PM

#

knotty harness yeah, so my attempt was to try and use the linux dagger, but still use my Window...

ok, that's strange.. the linux Dagger should work in that case if the git clone command works in WSL by using the 1Password ssh-agent proxy

knotty harness Sep 3, 2024, 6:00 PM

#

it requires some shenanigans though, you can see my Linux git config is using "ssh.exe" but thats because WLS has 'interop=true' which enables calling windows processes from WSL

#

but this way, i am able to keep my SSH keys in 1Password vault, never put on my machine and i'd like to keep it like that

spice wave Sep 3, 2024, 6:21 PM

#

knotty harness it requires some shenanigans though, you can see my Linux git config is using "s...

oh, I see .. that's because for what I've just skimmed the 1password WSL agent pretty much relies in the interop mode. Googling some results, seems like the way to make this work natively in WSL is by using npiperelay and socat as described here: https://gist.github.com/WillianTomaz/a972f544cc201d3fbc8cd1f6aeccef51#file-wsl2-with-1password-info-md

Gist

Instructions for using 1Password SSH Agent with WSL2 (on Windows 11)

Instructions for using 1Password SSH Agent with WSL2 (on Windows 11) - wsl2-with-1password-info.md

knotty harness Sep 3, 2024, 6:23 PM

#

hmm

#

I am already able to run ssh-add -l

#

trying to understand why dagger cannot clone, but git can, even thouugh git in WSL does work with SSH

#

SSH_AUTH_SOCK=~/.1password/agent.sock ssh-add -L outputs the correct key for me

#

so i thought, because that works, it would have worked for linux dagger

#

#

1Password support WSL properly 8 months ago, and im following their guide https://developer.1password.com/docs/ssh/integrations/wsl/

Use the 1Password SSH agent with WSL | 1Password Developer

Learn how to set up the integration with Windows Subsystem for Linux so you can manage your SSH keys and authorize SSH requests from 1Password on your Windows PC.

#

We have this issue raised if another windows user comes across this issue https://github.com/dagger/dagger/issues/8313

GitHub

Private modules do not work with Dagger Windows · Issue #8313 · dag...

What is the issue? private-modules don't work with the Dagger Windows binary release, because Windows does not recognise export SSH_AUTH_SOCK. This is because on Windows, SSH is handled through...

spice wave Sep 3, 2024, 8:07 PM

#

knotty harness 1Password support WSL properly 8 months ago, and im following their guide https:...

yes, but their guide advertises using ssh.exe which is not the unix variant

#

does ~/.1passowrd/agent.sock actually work for you in WSL Patrick?

knotty harness Sep 3, 2024, 8:10 PM

#

This command SSH_AUTH_SOCK=~/.1password/agent.sock ssh-add -L gives me a result, so i assume it does? But being honest, i dont know what its actually doing.

#

maybe its not doing anything... ssh-add (which I aliased to ssh-add.exe) works

spice wave Sep 3, 2024, 8:12 PM

#

knotty harness maybe its not doing anything... `ssh-add` (which I aliased to ssh-add.exe) works

oh, that's why 😄

knotty harness Sep 3, 2024, 8:12 PM

#

"Instead of forwarding SSH agent requests, like with standard OpenSSH agent forwarding, you can effectively forward the entire SSH request from WSL to the ssh.exe process running on Windows"

spice wave Sep 3, 2024, 8:12 PM

#

lol

knotty harness Sep 3, 2024, 8:12 PM

#

that is taken from their page, so i assume it would work

spice wave Sep 3, 2024, 8:13 PM

#

@knotty harness can you check if you actually have the file ~/.1password/agent.sock ?

knotty harness Sep 3, 2024, 8:13 PM

#

yeah i checked it does exist

#

spice wave Sep 3, 2024, 8:14 PM

#

can you do file ~/.1password/agent.sockand ls -la please?

#

I'd assume that's not a "proper" unix socket

knotty harness Sep 3, 2024, 8:14 PM

#

#

#

Yeah, i have no idea what it's doing

#

the most i ever went with SSH is learning how to make a key and store it securely

#

😄

#

I think i'll give a try the thing you suggested above, see if there is some way to have a sock and somehow forward it

#

I thought i could install 1Password in WSL too.. but its awful/doesnt work properly due to no GUI

spice wave Sep 3, 2024, 8:23 PM

#

@knotty harness can you check this really quick please?

func (m *Lala) Test(agentSock *dagger.Socket) *dagger.Container {
    return dag.Container().From("alpine").
        WithExec([]string{"apk", "add", "openssh-client"}).
        WithUnixSocket("/agent.sock", agentSock).
        WithEnvVariable("SSH_AUTH_SOCK", "/agent.sock").
        WithExec([]string{"ssh-add", "-l"})

}

#

and then dagger call test --agent-sock $SSH_AUTH_SOCK

knotty harness Sep 3, 2024, 8:24 PM

#

yeah, one minute

spice wave Sep 3, 2024, 8:26 PM

#

knotty harness yeah, one minute

just editted the last command

#

to use ssh-add -l instead

knotty harness Sep 3, 2024, 8:27 PM

#

 patrick   ssh-test    dagger call test --agent-sock $SSH_AUTH_SOCK
Setup tracing at https://dagger.cloud/traces/setup. To hide: export GOAWAY=1

✔ connect 0.8s
✔ initialize 2.4s
✔ prepare 0.0s
✔ host: Host! 0.0s
✔ Host.unixSocket(path: "/home/patrick/.1password/agent.sock"): Socket! 0.0s
✔ sshTest: SshTest! 0.0s
✔ SshTest.test(
agentSock: ✔ Host.unixSocket(path: "/home/patrick/.1password/agent.sock"): Socket! 0.0s
): Container! 0.6s
✘ Container.sync: ContainerID! 0.1s
! process "ssh-add -l" did not complete successfully: exit code: 1

Error: response from query: input: container.from.withExec.withUnixSocket.withEnvVariable.withExec.sync resolve: process "ssh-add -l" did not complete successfully: exit code: 1

Stderr:
error fetching identities: communication with agent failed
Run 'dagger call test --help' for usage.

#

😦

#

brb! PIZZA DELIVERYY

#

Could I do ssh-add.exe (and see if it works??)

#

wait no, i guess that wouldnt work, because WSL is NOT where its running anyway

#

yeah, i guess not.

#

its cool that I am running WSL and using Windows Processes.... but this doesnt mean it translates well once its in a dagger function, then running in an actual linux container that isn't WSL

#

ill try the tool you mentioned soon

spice wave Sep 3, 2024, 8:33 PM

#

yeah.. I think that agent.sock belongs to something else

#

IIUC how the WSL / 1password ssh agent integration works

#

@knotty harness if you run sudo netstat -p -l -A unix | grep agent.sock does that show anything?

knotty harness Sep 3, 2024, 9:08 PM

#

okay so, it didnt work and i needed to install netstat from net-tools

#

1Password in WSL is buggy, something about wslg has some issues with GUI Stuff but i DID install 1password INTO WSL

#

i have to run it in background from wsl/unbutu using: 1password & - this pops up a Linux Gui on WINDOWS

#

#

Linux GUI app on WINDOWS.... like what is this magic

#

so, i enabled the 1Password SSH Agent (1password in linux)

#

I ran it again

#

#

#

couldnt get dagger -m working though 😦

#

also i think this 1Password wslg is not the best idea...

😄

spice wave Sep 3, 2024, 9:50 PM

#

Are you authenticating against GitHub?

spice wave Sep 3, 2024, 9:50 PM

#

knotty harness

Ok, this seemed to work

knotty harness Sep 3, 2024, 10:03 PM

#

i dont think its going to work properly this way, 1password seems to be pretty broken

#

yeah, ssh -T git@GitHub brings up a broken 1Password auth

knotty harness Sep 3, 2024, 10:44 PM

#

I have a mac laptop, i think a better alternative might be to setup sshd and ssh to my mac, from my windows pc and use dagger there becausse i have a feeling 1password on mac will probably work better

spice wave Sep 3, 2024, 11:17 PM

#

knotty harness I have a mac laptop, i think a better alternative might be to setup sshd and ssh...

I have a windows partition. I can give this a try tomorrow to see what a possible stopgap could be for WSL users

knotty harness Sep 4, 2024, 2:30 PM

#

@spice wave soon testing private modules on my mac from Windows terminal.

#

knotty harness Sep 4, 2024, 6:02 PM

#

Still having issues on a Mac, not sure if its how im calling the private module or if its to do with the SSH_AUTH_SOCK

#

dagger call -m ssh://git@github.com/pjmagee/dagger-private-hello-world@main
Setup tracing at https://dagger.cloud/traces/setup. To hide: export GOAWAY=1

✔ connect 0.7s
✘ initialize 0.4s
! failed to get configured module: failed to get module ref kind: input: moduleSource resolve: failed to resolve git src to commit: failed to load cache key: NotFound: NotFound: rpc error: code = NotFound desc = socket /agent.sock not found: stat /agent.sock: no such file or directory
✘ resolving module ref 0.4s
! failed to get configured module: failed to get module ref kind: input: moduleSource resolve: failed to resolve git src to commit: failed to load cache key: NotFound: NotFound: rpc error: code = NotFound desc = socket /agent.sock not found: stat /agent.sock: no such file or directory
✘ moduleSource(refString: "ssh://git@github.com/pjmagee/dagger-private-hello-world@main"): ModuleSource! 0.3s
! failed to resolve git src to commit: failed to load cache key: NotFound: NotFound: rpc error: code = NotFound desc = socket /agent.sock not found: stat /agent.sock: no such file or directory
✘ cache request: git://github.com/pjmagee/dagger-private-hello-world#main 0.0s
! NotFound: NotFound: rpc error: code = NotFound desc = socket /agent.sock not found: stat /agent.sock: no such file or directory

Error: failed to get configured module: failed to get module ref kind: input: moduleSource resolve: failed to resolve git src to commit: failed to load cache key: NotFound: NotFound: rpc error: code = NotFound desc = socket /agent.sock not found: stat /agent.sock: no such file or directory

#

/Users/mageep/Library/Group Containers/2BUA8C4S2C.com.1password/t/agent.sock```

#

Error: failed to get configured module: failed to get module ref kind: input: moduleSource resolve: failed to resolve git src to commit: failed to load cache key: NotFound: NotFound: rpc error: code = NotFound desc = socket /Group Containers/2BUA8C4S2C.com.1password/t/agent.sock not found: stat /Group Containers/2BUA8C4S2C.com.1password/t/agent.sock: no such file or directory```

#

It doesnt seem to be capturing the full path

#

 test-ssh-auth  dagger call test --agent-sock $SSH_AUTH_SOCK stdout
Setup tracing at https://dagger.cloud/traces/setup. To hide: export GOAWAY=1

✔ connect 0.7s
✔ initialize 0.3s
✔ prepare 0.0s
✔ host: Host! 0.0s
✔ Host.unixSocket(path: "/Users/mageep/Library/Group Containers/2BUA8C4S2C.com.1password/t/agent.sock"): Socket! 0.0s
✔ testSshAuth: TestSshAuth! 0.0s
✔ TestSshAuth.test(
    agentSock: ✔ Host.unixSocket(path: "/Users/mageep/Library/Group Containers/2BUA8C4S2C.com.1password/t/agent.sock"): Socket! 0.0s
  ): Container! 0.5s
✔ Container.stdout: String! 0.1s

256 SHA256:bl/oTC86rOHG9NGMWza9QXmp2eAWY4DRrRdE8S9Kcqg Personal SSH Key (ED25519)

#

The test code you sent me seems to work though

#

Got it working!! (On Mac)

#

The env variable has to be set to an absolute path /Users/mageep/Libary/ ...

#

the 1Password material/guide was using ~ which I copy pasted.

#

Somehow, your test go code worked, but when trying to run the module, it was failing

#

#

1Password auth pops up on Mac fine, i use my fingerprint auth and resumes 🙂

#

Not sure if this is a 'bug' - Your test go module code worked with ~ but the dagger command failed to recognise the env variable.

spice wave Sep 4, 2024, 8:39 PM

#

knotty harness

nice! it's awesome to see that working 💪

spice wave Sep 4, 2024, 8:40 PM

#

knotty harness Not sure if this is a 'bug' - Your test go module code worked with ~ but the dag...

yes, it migh be a bug. cc @green depot

#

seems like the SSH_AUTH_SOCK doesn't seem to be working with paths like ~/.ssh/agent.sock

#

it does seem to work though if you pass it as a function argument. Check example here: #1280527928861655112 message

green depot Sep 5, 2024, 5:06 AM

#

knotty harness Got it working!! (On Mac)

Hey thanks Patrick for testing things out 🙏 I got the windows computer set up I'll definitely take the time tomorrow to hack around

The ~ expansion definitely looks like a bug, and I think I know why: I convert the path to a relative one (lexically) and I might not expand first 🙏

#Private modules with Dagger - windows binary & SSH