Fetch private Git repo with Access Token

[jolly mountain]

Hello, I would like to fetch a private Git repo with a access token instead of the sshAuthSocket. How would this be possible in Node JS.

I have seen a example in go but i cannot really convert it to Node JS because i dont really understand the Go code.

Thx for your help.

[main kraken]

Can you point me to the Go example?

[jolly mountain]

Sure, https://discord.com/channels/707636530424053791/1107670041362706532

[main kraken]

Then main bit is executing in a shell to expand that contents of the token secret variable:

    token = client.setSecret("GH_ACCESS_TOKEN", process.env.GH_ACCESS_TOKEN!?.toString() || "")

    src
      .withSecretVariable("GH_ACCESS_TOKEN", token)
      .withExec(["sh", "-c", "git config --global url.\"https://$[e-mail]/.insteadOf\" https://github.com/"])
      .withExec(["git", "clone", "https://github.com/myuser/myrepo"])

This solution is based on a git config to force a different url (like a web url rewrite): https://gist.github.com/taoyuan/bfa3ff87e4b5611b5cbe. Then when you use git clone https://github.com/myuser/myrepo, it'll rewrite to add the token under the scene.

More on https://git-scm.com/docs/git-clone#_git_urls

[jolly mountain]

Ah I understand. For now I have been using const src = client.git(git).branch(branch).tree(); this gave me a Directory type. is there a way to replicate this functionality with the solution you provided?

[main kraken]

I'm afraid not, but we should add that option: https://github.com/moby/buildkit/blob/cdf28d6fff9583a0b173c62ac9a28d1626599d3b/client/llb/source.go#L327-L328. Right now we only support these: https://github.com/dagger/dagger/blob/00dc2cf3c4dd47dea568dc3e5c385b91a3e2f1f0/core/schema/git.go#L127-L135

[jolly mountain]

Yes this would be really nice to be able to just pass a token!

[main kraken]

I'm working on a prototype for this, I think it should be like this:

    const src = await client
      .git(git)
      .withAuthToken(client.setSecret("GH_ACCESS_TOKEN", process.env["GH_ACCESS_TOKEN"]))
      .branch(branch)
      .tree()

Wdyt?

[jolly mountain]

Oh thats really nice! or maybe like this
but im not sure if this fits with the code style of dagger

const src = await client
   .git(git, {
      authToken: client.setSecret("GH_ACCESS_TOKEN", process.env["GH_ACCESS_TOKEN"])
   })
   .branch(branch)
   .tree()

[main kraken]

Since there's multiple auth options, I think it's better to use withXXX. I'm going to propose the ssh options to move from tree() into those as well.

So it would add:

• withAuthToken(token: SecretID!)
• withAuthHeader(header: SecretID!)
• withSSHSocket(socket: SocketID!, knownHosts: String)
  You'd use only one of these really.

[pastel pond]

This is fantastic! I'll be looking forward to this too as we use internal PAT auth for our enterprise gh.

[jolly mountain]

if they are all in the same place it makes much more sense! Awesome improvement

[main kraken]

I'm having a hard time getting it to work with a github private repo:

❯ docker run --rm -it -e GIT_TERMINAL_PROMPT=0 bitnami/git git -c "http.extraheader=Authorization: basic $(echo x-access-token:xxxxx) clone https://github.com/helderco/myrepo
Cloning into 'myrepo'...
fatal: could not read Username for 'https://github.com': terminal prompts disabled

@jolly mountain is your private repo in GitHub?

In https://docs.github.com/en/get-started/getting-started-with-git/about-remote-repositories#cloning-with-https-urls it says:

Password-based authentication for Git has been removed in favor of more secure authentication methods. For more information, see "Managing your personal access tokens."

[jolly mountain]

Yes it is and i would be using the access token from a github app to access the repo

[main kraken]

In both github and gitlab, it works directly on the URL:

git clone https://x-access-token:<token>@github.com/helderco/<repo>
git clone https://x-access-token:<token>@gitlab.com/helderco/<repo>

But not with http.extraheader. At least it used to work, I'm not sure if I'm missing something, but from docs seems that github pushes more towards using a credential helper and doesn't support this anymore, however, it does accept via URL (isn't it worse security wise?), which perhaps is due to a lower level networking thing.

[main kraken]

I think I misread that. It's just saying to use a token instead of the account's password.

[pastel pond]

Yeah afaik GH fully supports token based https clones. They deprecated the user/pass

[coral hedge]

we use tokens via basic auth with the big 3 git hosts, BBC is the only one that requires a specific username, the others is arbitary iirc

A .netrc file is common as well, Go tells users to use this file for authd fetches, it just shells out to git anyhow

https://github.com/hofstadter-io/hof/blob/_dev/lib/repos/git/fetch.go#L185

[inner plover]

This feels related: https://github.com/dagger/dagger/issues/4243

Just a friendly ping to see if there was any progress on this. I just ran into this trying to pull a centralized vulnerability ignore file from our platform repo. We do this in github actions currently with a sparse checkout of just a .snyk file... drop it into the target repo then run our scan. I wanted to check back prior to trying to sort out the socket path. The withAuthToken seems great... we could just hook into our usual system of environment variables.