Git only? | Dagger | Page 1

trail pilot May 24, 2024, 5:10 AM

#

I'm not familiar with athens sorry

#

We've discussed it in the past, but outsourcing storage and distribution to the existing git infrastructure has a lot of benefits. So far it feels worth it. Nothing is set in stone though, but it would take a pretty serious change in the perceived tradeoff to change our minds at the moment.

#

The thing is, there is another dimension to distribution of modules: the dagger cache.

#

Instead of forcing everyone to push all modules through an OCI registry, we are more likely to invest in a global distributed cache - which by the way could also leverage OCI registries, but as "dumb" swappable storage buffers.

#

Most registries are already used as caches for handrolled delivery pipelines anyway.

#

tldr it's not git only, it's git + dagger cache

cyan geyser May 24, 2024, 3:07 PM

#

https://github.com/gomods/athens is a go module proxy. Basically it ends up download the go dependencies in bundled format...making them portable.

GitHub

GitHub - gomods/athens: A Go module datastore and proxy

A Go module datastore and proxy. Contribute to gomods/athens development by creating an account on GitHub.

#

It also allows for rewriting module requests if you have to self host them.

#

Basically there exists infrastructure today for managing go modules for those who have internet restrictions and it seems like dagger hopefully will provide something equivalent...

cyan geyser May 24, 2024, 3:26 PM

#

Just for background in my world where my development box can access the internet but my CI/CD inf cannot. Right now dagger sdk is working great for us, but am thinking about how we can adopt modules in the future...

trail pilot May 24, 2024, 3:42 PM

#

I see. I think perhaps a robust vendoring feature is what would help in your case

#

Also, the module loading logic is available in the Dagger API, and not tied to git.

In other words, you can write your own Dagger pipeline to produce a module source code in any way you want, then dynamically load that module and call its functions.

#

unlike go, there is no complicated proxy system, and the full address of the module is not hardcoded in its source. So modules are in fact "portable" in the athens sense

cyan geyser May 24, 2024, 3:51 PM

#

Right but if I want to take advantage of the existing module ecosystem and if they have transitive dependencies doesn't that become challenging?

#

For the moment we are kinda writing our own module system just via the sdk...

trail pilot May 24, 2024, 3:53 PM

#

I think a well designed vendoring feature would make sure to also vendor transitive dependencies

cyan geyser May 24, 2024, 4:02 PM

#

We will explore the module loading via the API. Thanks for the tip.

#Git only?