How to run .gitlab-ci.yml file locally using dagger function

[sly tartan]

I have simple gitlab yml file that print "hello dagger", how should I run this yml file using dagger
how can we create "test" function, so when we call it - it will print hello_dagger

stages:

• test

test:
stage: test
script:
- echo "hello_dagger"
tags:
- vte-core-git
only:
- dev

[long totem]

👋

Dagger does't run your gitlab.yaml. What you can do is write the Hello Dagger part as a Dagger function and then run it the same way locally and in gitlab.

[sly tartan]

can you please guide me how can I do this in both ways?

[long totem]

You can write your first function by following our quick start guide here: and then run it in gitlab as mentioned here: https://docs.dagger.io/quickstart/428201/custom-function https://docs.dagger.io/integrations/734201/gitlab

[sly tartan]

I have gone through the docs, but didn't get idea how exactly it is working.

We can go with simple eg like
I have repo in gitlab which contains only one file hello.py and it prints "hello dagger"
now how can I move ahead step by step ?

in above docs, they have done it for golang - they have daggerize the repo then included golang dependencies n all
bu how it should be in my case?

[grand moat]

Hi @sly tartan I suggest that you first write and run a simple dagger function locally using dagger call. The first link Marcos gave you above has a good example. You could also read https://docs.dagger.io/manuals/developer/python/419481/first-module.

Once you have a function that runs locally, you can then try to integrate it with gitlab using the second link Marcos gave, which shows you how to use dagger call from within gitlab ci

Does that help? LMK if you need further guidance

[sly tartan]

Hi @grand moat @long totem
As i am able to call my first function

now I have created one version.txt file but I am not able to print data inside that file
Can you please help me in that?
Below is the code that I am using

prashant@ubuntu:~/potato/dagger/src/main$ ls
init.py version.txt
prashant@ubuntu:~/potato/dagger/src/main$ cat init.py
from dagger import function, object_type
import os

@object_type
class Potato:
@function
def hello_world(self) -> str:
return "Hello Daggernauts!"

@function
def print_version(self) -> str:

Place your code for accessing version.txt here

version_file_path = "/home/prashant/potato/dagger/src/main"

Check if the file exists

if os.path.isfile(version_file_path):

Read version number from file

with open(version_file_path, "r") as file:
version_number = file.read().strip()
return version_number
else:
return "Version file not found"
prashant@ubuntu:~/potato/dagger/src/main$ dagger call print-version
Version file not found

[grand moat]

Hi. Your function executes in a sandboxed container which cannot access host resources. So you won't be able to access the file as you have written it. Instead pass the file to the function as an argument. See https://docs.dagger.io/manuals/user/620164/host-fs for more information.

[sly tartan]

@grand moat So by considering my example how should I pass my version.txt file as argument?

[grand moat]

As an argument of type dagger.File

[grand moat]

Sure, I can help you in a call, but after a few hours as I'm in the middle of something else atm. Is that ok?

[sly tartan]

absolutely ok

[grand moat]

Around 7 pm IST?

[sly tartan]

sure np

[grand moat]

Ok talk then

[sly tartan]

ping me when you are available

[grand moat]

Hi

Try this

    # dagger call print-version --f=version.txt
    @function
    async def print_version(self, f: dagger.File) -> str:
        """Print file contents"""
        return await dag.container().from_("alpine:latest").with_file("/tmp/file.txt", f).with_exec(["cat", "/tmp/file.txt"]).stdout()

LMK if it clarifies, we can hop on a call if needed

^^ @sly tartan

[sly tartan]

I am tring , give me a min

Hi @grand moat still their is an error
can we have short call??

[grand moat]

yes

calling you

tried calling you, no answer

what error do you see?

[sly tartan]

call me

i am available

[grand moat]

Is your gitlab repo public? can I take a quick look?

also see https://github.com/dagger/dagger/issues/7218

[sly tartan]

yes it is public

[grand moat]

I hope the issue above clarifies. We are working on adding support for other VCS. Let us know if you need any further help

[sly tartan]

In Gitlab-CI integration (https://docs.dagger.io/integrations/734201/gitlab) code
If I run the dagger installation cmd inside build stage then it will also work fine

then what is the need to cmd above the build stage? like docker and all

[grand moat]

Dagger needs a container runtime like Docker

The previous lines are configuring a Docker service in the runner

  services:
    - docker:${DOCKER_VERSION}-dind

[sly tartan]

If I have docker installed already on my gitlab runner then their is no need to do it again, right?

[grand moat]

correct

[sly tartan]

to make it more clear -
On my gitlab runner, once I installed dagger and docker then no need to installed it from next time

So we can installed it once and from the next no need to provide this code in yml file
(note: by considering that I am not changing gitlab runner)

[grand moat]

considering that I am not changing gitlab runner
I'm not sure what you mean here, afaik gitlab runners are spawned on demand when a job is requested. so each runner starts "clean" and you would need to install all the requirements

are you using a self-hosted runner? https://docs.gitlab.com/runner/#use-self-managed-runners

[sly tartan]

yes

I think in self-hosted runner once we installed our requirement then we don't need to do it again, right?

[grand moat]

should work if the runner is not decommissioned at the end of each run and remains available to pick up future jobs

you'll also get the benefit of a warm cache

[sly tartan]

yes, same we are doing as of now with our runner
It is always available to take future jobs

thanks

[grand moat]

in self-hosted runner once we installed our requirement then we don't need to do it again, right?
not very sure about self-hosted runners, afair it depends on how it's configured/executor - https://docs.gitlab.com/runner/executors/index.html

we can installed it once and from the next no need to provide this code in yml file
keep in mind that in this case you will need to manually upgrade the Dagger CLI to get the benefit of fixes and new features https://docs.dagger.io/faq#how-do-i-update-dagger

[sly tartan]

ohhh understood
but docker we can installed once, right?

Unless docker does not require to upgrade

[grand moat]

same applies

[sly tartan]

Why it is printing Hello, World multiple times
Because in GitLab yml file I have called this cmd only once

[long totem]

it's mostly a visualization issue. The output effectively gets printed once in standard otuput, the other logs are printed to /dev/stderr to show more information about what Dagger is doing. We're currently working to improve that since it's somehow confusing. cc @shell oriole

[sly tartan]

@grand moat @long totem

initialize Dagger module

dagger init --sdk=python demo

by using the above cmd I have created one repo in my local machine and tried to run the below cmd
dagger call container-echo --string-arg='Hello Daggernauts!' stdout

so it working ... but when i push the same code on my GitHub and try to call via GitHub url then getting above error

[grand moat]

Try without https:// and .git

See the hello-world example in our quickstart and try to follow that format for GitHub URLs

[sly tartan]

thank you...it is working

[sly tartan]

Hi @grand moat @long totem

I above yml file - i am creating jar file and that jar I am trying to push on nexus repo using dagger function call
(here, my yml scripts runs on gitlab linux runner)

Now in other image - I have created dagger function that will push that jar on nexus using curl cmd

But i think their is issue while passing arguments and jar file to this function

how should I pass arguments in a right way and also how should I pass jar file that was build on linux runner??

Can you please help me in that?

[grand moat]

Can you please provide your function code here, as it's not readable in the screenshot?

Also and important: this is a public channel and your screenshot contains various credentials, please delete the image and rotate the credentials

Can you please also provide the YAML workflow (after removing all sensitive information from it)?

[sly tartan]

Deleted screeshots
ok, can you once confirm that their are still screeshots are available

[grand moat]

can you once confirm that their are still screeshots are available
I didn't understand this. But I don't see your screenshot any more in the thread.

^^ could you please use code formatting, it is still not readable because Discord is converting your YAML tabs to bullets

just to check: it looks like your intent is to use Dagger to only push the artifact to nexus, not for the actual build?

[sly tartan]

check now

[sly tartan]

yes

[grand moat]

you should pass the file to the function as a dagger.File argument. then you can move it to the container with Container().with_file() and use your curl command to push it from the container to your repository.

Container.with_file() API -> https://dagger-io.readthedocs.io/en/sdk-python-v0.11.4/client.html#dagger.Container.with_file

you should also use dagger.Secret to pass the credential

async def nexus(self, NEXUS_USERNAME: str, NEXUS_PASSWORD: dagger.Secret, MICROSERVICE_NAME: str, TIMESTAMP: str, RELEASE_VERSION: str, jar_path: dagger.File) -> str:

Here I have changed the credential from str to dagger.Secret so that it is scrubbed from logs. You will need to pass this to the function as an env var from your pipeline config or as a file. See https://docs.gitlab.com/ee/ci/variables/ with https://docs.dagger.io/manuals/developer/secrets for more on this topic.

[sly tartan]

can you please delete this msg

[sly tartan]

here dagger.secret file we need to pass?

[grand moat]

You can pass a secret using an env var, file, or command. See examples on https://docs.dagger.io/manuals/developer/secrets/

[sly tartan]

How do we call the dagger function for private GitHub repo?

[grand moat]

Private repos are not supported yet, work is in progress. See https://github.com/dagger/dagger/issues/6113
@long totem is there a more recent issue for this?

[sly tartan]

Hi @grand moat @long totem

In above code, is this the right way to pass "TIMESTAMP" value stored from the GitLab yml file to the dagger function?
And also check, is this the right way to access {timestamp} value in the function ".../dagger/v1.0.0/microservice/{timestamp}/"

Because I am getting error like
Error: required flag(s) "timestamp" not set

[grand moat]

it doesn't look like you're setting the variable correctly in the job, see the example in https://docs.gitlab.com/ee/ci/variables/#pass-an-environment-variable-from-the-script-section-to-another-section-in-the-same-job

[sly tartan]

Working now
Issue is with the double quotes in code
I think variables are right

thanks for the help

[sly tartan]

Hi @grand moat
Can dagger work on github enterprise account and on this enterprise account if we make our repo public then it will work or not?

Or dagger only works for github public repo (not enterprise account)?

[grand moat]

You can dagger call from a GitHub workflow on any type of account afaik

[sly tartan]

Like on private github account as well?

For eg. Company XYX has github enterprise account that means this account only accessible inside that company

Then if we create public repo which accessible within company only then can we use dagger in such case

[grand moat]

^^ @long totem do you know the answer to this? I'm not very familiar with github enterprise

[long totem]

hi, it currently can't. We're currently working to fix this and it'll be released very soon

[sly tartan]

Thanks for the reply
Please let me know once it will be available for enterprise

[grand moat]

I suggest tracking these issues to know what's happening on this topic
https://github.com/dagger/dagger/issues/7218
https://github.com/dagger/dagger/issues/6113

[sly tartan]

@grand moat @long totem

Now in our case we are not able to use Github public url.

So is there any other way we can execute the code or functionality without storing our code on public Github?

[grand moat]

I think this is being worked on in https://github.com/dagger/dagger/issues/6113
I'm not aware of a workaround, maybe Marcos has a suggestion

I suppose you can always git clone your private module and then install it from the local dir using dagger install. Then call it as usual.

[sly tartan]

Like I can call only in local dir, right?

[grand moat]

Yes or a public git url

[sly tartan]

Hi
How to pass username and password to dagger function using .env file

As currently I am inside docker container and when I hit

"dagger call demo --username=admin --password=admin"

Now how to pass these credentials using. env file