AsGit().Head() does not work always | Dagger | Page 1

gentle oxide Jun 5, 2025, 10:37 AM

#

    ref := client.Host().Directory(config.GitURL()).AsGit().Head()
    dir = ref.Tree()
    commit, err = ref.Commit(ctx)

Will work on a local machine on a fully cloned repo

But on Jenkins after a shallow clone will fail with

15  : GitRef.commit: String!
18  : │ git symbolic-ref HEAD
18  : │ [0.0s] | fatal: ref HEAD is not a symbolic ref
18  : │ git symbolic-ref HEAD ERROR [0.0s]
18  : │ ! git error: exit status 128
stderr:
fatal: ref HEAD is not a symbolic ref

The recommended git tool is: NONE
using credential jenkins-instance
Fetching changes from the remote Git repository
Using shallow fetch with depth 1
 > git rev-parse --resolve-git-dir /var/lib/jenkins/workspace/some-project-template_master@3/some-project-template/.git # timeout=10
 > git config remote.origin.url https://github.com/some-org/some-project-template.git # timeout=10
Fetching upstream changes from https://github.com/some-org/some-project-template.git
 > git --version # timeout=10
 > git --version # 'git version 2.43.0'
using GIT_ASKPASS to set credentials GitHub App
 > git fetch --no-tags --force --progress --depth=1 -- https://github.com/some-org/some-project-template.git +refs/heads/master:refs/remotes/origin/master # timeout=10
Checking out Revision 51523dc48df459c867c31e477ba17235d1ad6a0b (master)
Commit message: "Merge pull request #557 from some-org/renovate/master-patch"
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 51523dc48df459c867c31e477ba17235d1ad6a0b # timeout=10

#

The checkout is also using a reference repo for faster checkouts (which is why I use the jenkins clone instead of doing it inside dagger)

versed hill Jun 5, 2025, 1:33 PM

#

🤔

#

really hard to debug without knowing what the git state on jenkins looks like

#

the git symbolic-ref HEAD command is essentially just lokoing for .git/HEAD

#

if i shallow clone a repo, then it works

❯ git clone https://github.com/dagger/dagger.git --depth=1 .
Cloning into '.'...
remote: Enumerating objects: 4836, done.
remote: Counting objects: 100% (4836/4836), done.
remote: Compressing objects: 100% (3152/3152), done.
remote: Total 4836 (delta 1378), reused 3097 (delta 917), pack-reused 0 (from 0)
Receiving objects: 100% (4836/4836), 41.02 MiB | 20.09 MiB/s, done.
Resolving deltas: 100% (1378/1378), done.
❯ dagger core host directory --path=. as-git head commit
✔ connect 0.2s
✔ loading type definitions 0.5s
✔ parsing command line arguments 0.0s

$ host: Host! 0.0s CACHED
✔ .directory(path: "."): Directory! 1.4s
✔ .asGit: GitRepository! 0.0s
✔ .head: GitRef! 0.0s
✔ .commit: String! 0.1s

6639991582e791c94ea5b2e0bacaeb599e94183b

#

can you reproduce the issue outside of jenkins?

bitter grove Jun 5, 2025, 3:59 PM

#

I recall we had a similar issue about cloud telemetry being sent from Jenkins pipelines given how Jenkins currently handles git repo metadata. @vernal blaze do you recall what was the outcome here? This happened a while back and I have some fuzzy memories about it

vernal blaze Jun 5, 2025, 4:29 PM

#

Yeah, there's no such thing as a proper Git context in Jenkins, it does something weird to checkout code

#

I'm trying to find what it was exactly, because I remember having to read from the env to fetch the branch name for example

versed hill Jun 5, 2025, 4:33 PM

#

is there even a git directory?

vernal blaze Jun 5, 2025, 5:55 PM

#

IIRC there was not, there's just a Git plugin in Jenkins that does things for you

#

IIRC Jenkins creates a new empty repository and pulls the changes from the git repo that you want, it's very weird, but I cannot find the docs for it

versed hill Jun 5, 2025, 6:17 PM

#

vernal blaze IIRC Jenkins creates a new empty repository and pulls the changes from the git r...

okay wild 😛
yeah, if .git/HEAD does not exist, I am not really sure how you are supposed to get the "current commit"

#

that's what .git/HEAD is

vernal blaze Jun 5, 2025, 6:19 PM

#

@gentle oxide this might be unrelated, just surfacing in case you find it useful, we do have a section and a demo on how to use dagger in Jenkins
https://docs.dagger.io/ci/integrations/jenkins/

vernal blaze Jun 5, 2025, 6:20 PM

#

versed hill okay wild 😛 yeah, if `.git/HEAD` does not exist, I am not really sure how you a...

yes, exactly, it really doesn't work, that's why in the Jenkins env you literally have to read git-related env variables to get the branch name

#

https://www.jenkins.io/doc/book/pipeline/multibranch/#additional-environment-variables

bitter grove Jun 5, 2025, 6:25 PM

#

vernal blaze <@152554242169765889> this might be unrelated, just surfacing in case you find i...

For non-Jenkinsfile pipelines I recall we had message at some point to be careful about reference repo settings. I was almost sure we had it in writing somewhere but my memory might be tricking me at this point 🙏

gentle oxide Jun 6, 2025, 9:00 AM

#

This does work tho

func handleLocalGitSource(ctx context.Context) (*dagger.Directory, string, error) {
    var err error
    var dir *dagger.Directory
    var commit string

    client := config.Dag()

    dir = client.Host().Directory(config.GitURL())
    gitContainer, err := containers.NewGitContainer(ctx, client)
    if err != nil {
        return nil, "", fmt.Errorf("failed to create git container: %s", err)
    }
    gitContainer = gitContainer.WithDirectory(".", dir)
    gitContainer = gitContainer.WithExec([]string{"git", "rev-parse", "HEAD"})
    output, err := gitContainer.Stdout(ctx)
    if err != nil {
        return nil, "", fmt.Errorf("failed to get commit: %s", err)
    }
    commit = strings.TrimSpace(output)

    return dir, commit, nil
}

But this does not

func handleLocalGitSource(ctx context.Context) (*dagger.Directory, string, error) {
    var err error
    var dir *dagger.Directory
    var commit string

    client := config.Dag()

    ref := client.Host().Directory(config.GitURL()).AsGit().Head()
    dir = ref.Tree()
    commit, err = ref.Commit(ctx)
    if err != nil {
        return nil, "", fmt.Errorf("failed to get commit: %s", err)
    }

    return dir, commit, nil
}

I agree that Jenkins does something weird there, and to be honest, I'm looking for alternatives. Just not sure what yet.
Maybe I'll go back to GHA now that most of the logic happens on dagger. But I hate to get locked in GHA again and would prefer to not use yaml to orchestrate the pipelines.

versed hill Jun 6, 2025, 9:56 AM

#

🤔 okay, well there's potentially something wrong in dagger there

#

is there anyway you could share a tarball of the git directory? i don't how to reproduce this, so i don't know how to fix it currently

gentle oxide Jun 6, 2025, 1:37 PM

#

I'll try to export it this weekend and try to repro locally first

gentle oxide Jun 6, 2025, 4:20 PM

#

I can repro locally.

❯ pwd
/Users/skois/projects/my-escape/devops-dagger

❯ ls -la some/.git 
.rw-r--r--@ 6.1k skois  6 Jun 19:14 .DS_Store
drwxr-xr-x@    - skois 13 Feb 15:18 branches
.rw-r--r--@  263 skois  6 Jun 19:00 config
.rw-r--r--@   73 skois 13 Feb 15:18 description
.rw-r--r--@    0 skois  6 Jun 19:14 FETCH_HEAD
.rw-r--r--@   41 skois  6 Jun 13:31 HEAD
drwxr-xr-x@    - skois 13 Feb 15:18 hooks
.rw-r--r--@ 348k skois  6 Jun 19:00 index
drwxr-xr-x@    - skois 13 Feb 15:18 info
drwxr-xr-x@    - skois 13 Feb 15:18 logs
drwxr-xr-x@    - skois  6 Jun 19:00 objects
drwxr-xr-x@    - skois 13 Feb 15:18 refs
.rw-r--r--@ 8.6k skois  6 Jun 19:00 shallow

devops-dagger on  master [✘!?] via 🐹 v1.24.4 took 0s 
❯ cat some/.git/HEAD 
c3bf3112501a33ef53bd2f641eddeb3082101bfd

#

func handleLocalGitSource(ctx context.Context) (*dagger.Directory, string, error) {
    var err error
    var dir *dagger.Directory
    var commit string

    client := config.Dag()

    fmt.Println("Using local git source:", config.GitURL())

    ref := client.Host().Directory(config.GitURL()).AsGit().Head()
    dir = ref.Tree()
    entries, err := ref.Tree().Entries(ctx)
    if err != nil {
        return nil, "", fmt.Errorf("failed to get entries from local git source: %s", err)
    }
    for _, entry := range entries {
        fmt.Printf("Entry: %s\n", entry)
    }
    commit, err = ref.Commit(ctx)
    if err != nil {
        return nil, "", fmt.Errorf("failed to get commit: %s", err)
    }
    fmt.Printf("Commit: %s\n", commit)
    os.Exit(1)

    return dir, commit, nil
}

#

Using local git source: /Users/skois/projects/my-escape/devops-dagger/some
7   : │ │ connecting to engine DONE [0.0s]
8   : │ │ starting session
8   : │ │ starting session DONE [0.0s]
2   : │ connect DONE [0.2s]

9   : consuming /v1/traces
9   : consuming /v1/traces DONE [0.0s]

10  : consuming /v1/logs
10  : consuming /v1/logs DONE [0.0s]

11  : consuming /v1/metrics
11  : consuming /v1/metrics DONE [0.0s]

12  : Host.directory(path: "/Users/skois/projects/my-escape/devops-dagger/some"): Directory!
13  : │ upload /Users/skois/projects/my-escape/devops-dagger/some from fcw7dycpevo34eak9557zcczp (client id: ld45tzy1vw1gcjbvih226qa9y, session id: xvpeamqjtqum0l2ic1vef5jiv)

1   : /Users/skois/Library/Caches/dagger/dagger-0.18.9 session --label dagger.io/sdk.name:go --label dagger.io/sdk.version:0.18.9
14  : │ host: Host!
14  : │ host DONE [0.0s]
Error: failed to parse global flags: failed to get source code: failed to get source code for git protocol [local]: failed to get entries from local git source: input: host.directory.asGit.head.tree.entries load: failed to checkout HEAD: git error: exit status 128
stderr:
error: unable to normalize alternate object path: /var/lib/jenkins/git-refs/escape-website-b2c-template/objects
remote: error: unable to normalize alternate object path: /var/lib/jenkins/git-refs/escape-website-b2c-template/objects        
remote: error: unable to normalize alternate object path: /var/lib/jenkins/git-refs/escape-website-b2c-template/objects        
remote: fatal: bad tree object 8b9351bcebb35f70c1908106899b0ef146e4ad6d        
error: git upload-pack: git-pack-objects died with error.
fatal: git upload-pack: aborting due to possible repository corruption on the remote side.
remote: aborting due to possible repository corruption on the remote side.
fatal: protocol error: bad pack header

#

devops-dagger on  master [✘!?] via 🐹 v1.24.4 
❯ cd some                              
Using Node v22.16.0

some on  HEAD (c3bf311) via  v22.16.0 
❯ git rev-parse HEAD             
c3bf3112501a33ef53bd2f641eddeb3082101bfd

some on  HEAD (c3bf311) via  v22.16.0 took 0s 
❯ git rev-parse --abbrev-ref HEAD
HEAD

some on  HEAD (c3bf311) via  v22.16.0 took 0s 
❯ git status                     
error: unable to normalize alternate object path: /var/lib/jenkins/git-refs/escape-website-b2c-template/objects
HEAD detached at c3bf311
nothing to commit, working tree clean

#

some on  HEAD (c3bf311) via  v22.16.0 took 0s
❯ git symbolic-ref HEAD
fatal: ref HEAD is not a symbolic ref

versed hill Jun 6, 2025, 4:30 PM

#

uh. what type of object is .git/HEAD then lol

gentle oxide Jun 6, 2025, 4:30 PM

#

some on  HEAD (c3bf311) via  v22.16.0 took 0s
❯ cat .git/objects/info/alternates
/var/lib/jenkins/git-refs/escape-website-b2c-template/objects%

some on  HEAD (c3bf311) via  v22.16.0 took 0s
❯ ls -la .git/refs/heads

# empty

versed hill Jun 6, 2025, 4:30 PM

#

what's in .git/ ? and .git/refs/tafs

gentle oxide Jun 6, 2025, 4:30 PM

#

some on  HEAD (c3bf311) via  v22.16.0
❯ cat .git/HEAD
c3bf3112501a33ef53bd2f641eddeb3082101bfd

#

some on  HEAD (c3bf311) via  v22.16.0 took 0s
❯ ls -la .git
.rw-r--r--@ 6.1k skois  6 Jun 19:14 .DS_Store
drwxr-xr-x@    - skois 13 Feb 15:18 branches
.rw-r--r--@  263 skois  6 Jun 19:00 config
.rw-r--r--@   73 skois 13 Feb 15:18 description
.rw-r--r--@    0 skois  6 Jun 19:21 FETCH_HEAD
.rw-r--r--@   41 skois  6 Jun 13:31 HEAD
drwxr-xr-x@    - skois 13 Feb 15:18 hooks
.rw-r--r--@ 348k skois  6 Jun 19:21 index
drwxr-xr-x@    - skois 13 Feb 15:18 info
drwxr-xr-x@    - skois 13 Feb 15:18 logs
drwxr-xr-x@    - skois  6 Jun 19:00 objects
drwxr-xr-x@    - skois 13 Feb 15:18 refs
.rw-r--r--@ 8.6k skois  6 Jun 19:00 shallow

#

some on  HEAD (c3bf311) via  v22.16.0 took 0s
❯ ls -la .git/refs/tags

# emtpy

#

Insert more hate for Jenkins here

#

At this point I dont really care, cause I'll be replacing Jenkins hopefully soon.
But if you want me to get you more info, I can try

versed hill Jun 10, 2025, 9:59 AM

#

bleurgh yeah, sorry, i'm really struggling for knowing what happened here. apparently this used to occur also on super old versions of github actions, but doesn't seem very recent

versed hill Jun 10, 2025, 9:59 AM

#

gentle oxide ```sh devops-dagger on  master [✘!?] via 🐹 v1.24.4 ❯ cd some ...

the fact you get errors on git status is weird

bitter grove Jun 10, 2025, 2:02 PM

#

IIRC I think we should be able to address this by setting up a git alternate / reference repository. That's generally how Jenkins optimizes checkouts.

https://docs.cloudbees.com/docs/cloudbees-ci-kb/latest/client-and-managed-controllers/how-to-create-and-use-a-git-reference-repository

Using a Git reference repository

The Jenkins Git Plugin can use a reference repository as a cache to reduce remote data transfer and to reduce local disc use.
Reference repositories are defined per project via the Additional Behavior named Advanced clone options.
Reference repositories are created and maintained manually. The Git plugin does not maintain the reference repositor...

gentle oxide Jun 12, 2025, 8:37 AM

#

This is the ONLY reason I use jenkins checkout.
If that is supported natively in dagger, I'll skip jenkins checkout as well

#

Even shallow clones on a matrix with ~50 builds takes time.

gentle oxide Apr 13, 2026, 9:32 AM

#

Hey, @bitter grove are there any actions taken on implementing reference repository? I'm rewritting my dagger pipelines so I can move git checkout in the pipeline instead of having jenins do it, but really need the reference thing for speeding up the clones, sadly I've got some big repos with many images needed during build.

gentle oxide Apr 13, 2026, 1:51 PM

#

I currently did this (not battle tested yet tho)

📎 message.txt

#AsGit().Head() does not work always