#GHA Marketplace Shim

1 messages · Page 1 of 1 (latest)

keen kettle
#

Here is the end user facing implementation

    inputs = GithubActionsInputSettings(settings, **{
        "paths": "/input/**/jacocoTestReport.xml",
    "token": settings.GITHUB_TOKEN,
    "debug-mode": "true"
    })

    if not settings.CI:
        payload_builder = MockPayloadBuilder(settings)
        mock_event = payload_builder.build_push_payload()
        # Write mock event to file
        with open(settings.GITHUB_EVENT_PATH, 'w') as f:
            json.dump(mock_event, f)
    await with_typescript_gha(client, result.directory("/jacoco"), "Madrapps/jacoco-report", "v1.6.1", inputs)

And the Typescript GHA container, which is just a simple wrapper around node with some changed permissions (I got EPERM errors without those): https://github.com/airbytehq/aircmd/blob/7364d6a9c2d793a2a1f4e3bd89d4021e6780cab9/aircmd/actions/environments.py#L24

The only tough part to mock here is the GHA event payload. This is the JSON that corresponds to the webhook event - it is stored on a file on the GHA runner that the environment variable GITHUB_EVENT_PATH points to. So on the local machine we end up writing the file out to /tmp and pointing the env variable there, and then passing that var into the runner container and mounting that path into the runner container. Some rudimentary mocks are here: https://github.com/airbytehq/aircmd/blob/main/aircmd/actions/githubactions.py

It's not as leaky of an abstraction that I had guessed after wondering this question at @modest terrace and @languid python , but there are still going to be some github actions that only ever work remotely in CI by very nature of what they do. Up to the end user to not make their pipelines overly complex by gating a bunch of behavior behind flags. I mostly recommend this for marketplace actions that don't really rely on being remote for that reason (upload X to pr, like in my above example, really only makes sense to run in a PR context).

#

I have only done some preliminary thoughts about how to make this off-the-shelf reusable, but my initial thought is that if Dagger supported their own marketplace of functions that wrapped containers supported by SDK's (I'm not sure if that's what an 'extension' is supposed to be) then that function would be available off the shelf for people.

something like .run_gha() where the method would determine what flavor of gha it was, pick the right container (e.g. use node:latest if it was typescript), and take in typed object with all of the dependencies to run the action

I think the problem this approa ch solves is somewhat orthogonal to the problem gale/ghx solves, btw, which to me is: "here is a way to instantaneously get value from dagger without making any changes to your CI." Rather, the problem this approach intends to solve is "I've already rewritten my pipelines in Dagger. We already have this modular library of reusable ci functions in the form of GHA marketplace, how can we use them in Dagger?" cc @acoustic stone

acoustic stone
#

Rather, the problem this approach intends to solve is "I've already rewritten my pipelines in Dagger. We already have this modular library of reusable ci functions in the form of a GHA marketplace, how can we use them in Dagger?

I understand where you're coming from and I appreciate your perspective. This is also a valid use case.

I believe we can easily handle this situation using #github along with assistance from #daggernauts . Ultimately, we will be executing sequential tasks. We can easyly do something like gale run step Madrapps/jacoco-report v1.6.1 --input path="/input/**/jacocoTestReport.xml" --input debug-mode=true
Do you think is it acceptable for your scenario @keen kettle ?

#

I'm not sure, what kind of DX we'll have at #daggernauts side but in worst case scenario, we could load gale lib to container and allow execution of the command similar to above

languid python
keen kettle
#

@aweris

I guess I would hope for the following things from such a solution:

  • I should be able to use it as a container seamlessly in my sdk workflow. Gale cannot be the entrypoint for the application, it has to be used as a container. This is because the entrypoints are quite opinionated right now for our setup
  • Ideally I shouldn't have to care what type of action it is (typescript, composite or container)
  • I deally I have a typed way to pass inputs into the gale container. This can be a simple hacky wrapper around gale container for now though - something like a python method that calls from("gale:latest") and dumps the requisite directory and variables into the container
acoustic stone
#

I should be able to use it as a container seamlessly in my sdk workflow. Gale cannot be the entrypoint for the application, it has to be used as a container. This is because the entrypoints are quite opinionated right now for our setup

I think we can use zenith here as the final version but I guess I can create a container image just to try our assumptions here. Just to clarify, you want to run Gale as a container. Can we use a basic container to hold the binary?

Ideally I shouldn't have to care what type of action it is (typescript, composite or container)

It should work right away, even though we haven't covered all possible actions yet. We will continue to add more with each version.

I deally I have a typed way to pass inputs into the gale container. This can be a simple hacky wrapper around gale container for now though - something like a python method that calls from("gale:latest") and dumps the requisite directory and variables into the container

I'm not sure about DX, but once it's functioning, we can enhance it.

keen kettle
#

For the first question i would assume so, do you mean like just getting an alpine, downloading the gale binary, and executing it inside of there? I would guess that should work fine

acoustic stone