#Namespaces
sweet dagger
Still up in the air, but the most recent idea @tender matrix mentioned during "Zenith week" was that we may be able to do some clever stuff with graphql interfaces.
The idea being that we could change type Environment to interface Environment and then when an environment is loaded, it stitches in an object that implements the Environment interface, but the concrete object also has all the fields specific to its own API.
This will obviously require we update our all our SDKs' codegen to handle interface, but that may be desirable in general anyways for other use cases.
The namespacing specifically would then just be a matter of namespacing the concrete object stitched into the schema, which should be pretty straightforward. We could name the object using the name of the environment (as specified in dagger.json currently)
stiff abyss
Ok, I just looked at the relevant code and it's clear to me now, thanks. 🙂 What I was really trying to understand was what the resolver for a nested command looks like in the schema. Even with the Environment interface, every subcheck and subcommand in the same environment is being flattened in the same namespace. So you can't have two functions named the same even from different types, and instead of dagger do sdk:go:lint you need to dagger do go-lint. Any ideas about that?
tender matrix
commands (now “tools”) should not be called programmatically and therefore won’t be nested
we’re trying (not easy) to split the human-facing and code-facing parts of the environment API
see “tools” and “graph” for an attempt at this in the zenith schema PR
stiff abyss
My questions are from the point of view of the SDK. So when I say schema, what I'm interested is in what the SDK has in resolver, in /inputs/dagger.json (in the SDK's runtime, when resolving from dagger do/check to the actual function in code), which used to be, e.g., Go.lint but is now (temporarily) Query.goLint, and I'm thinking ahead on how nesting will work with namespaces.
tender matrix
right but part of the answer is in the not-yet-fully-implemented graphql schema
my understanding (sorry if I’m missing something obvious in your message) is that namespacing only is needed at one level of depth, because any given client will only see the tools of the environment it’s querying (not any sub-environments / extensions) and same thing for the graph, artifacts, etc
stiff abyss
Yes, but in an environment, you have checks and each check can have subchecks. That's nesting.
tender matrix
nested checks then - not nested commands
stiff abyss
But according to your PR, a ToolCommand can have subcommands. Same thing.
tender matrix
maybe same thing 🤷🏻‍♂️ I don’t understand what “flattened namespace” you’re referring to, I thought it was the top-level graphql query type, but that shouldn’t happen with sub-checks or sub-commands hence my questions
I guess the graphql type name for each subcheck has to be namespaced like EnvCheckGoTestFoo
stiff abyss
Ok, I'll explain how things currently work in Erik's PR, for example with Go lint in our CI code. Currently with mage, you run it with sdk:go:lint. In main, the way you create that nesting as a top hat is by creating a struct SDK with field go that returns a struct Go with field lint that returns a string. This generates a GraphQL schema such that a type is created for each struct, and corresponding fields:
type SDK {
go: Go!
python: Python!
}
type Go {
lint: String!
}
type Python {
lint: String!
}
So when you do dagger do sdk:go:lint, the engine will run the SDK's runtime with "resolver": "Go.lint". The SDK then know's to run the lint method in the Go struct. The problem is there's a lot of types being created and it's easy to create naming conflicts with the rest of the schema.
In Zenith, any top hat function, wether it's a tool command, a check or a subcheck, they're all flattened right now in Query, however this will probably change to an Environment instance. Even though you have the static Check.subckecks to represent the nesting, the resolvers are dynamic additions to the schema, but they're being put under the same namespace so the Go functions need to be named accordingly, even if in different packages:
type Dagger implements Environment {
...
"Run all linters"
lint: CheckResult!
"Run the go linter"
goLint: CheckResult!
"Run the python linter"
pythonLint: checkResult
}
From this:
func main() {
ctx := dagger.DefaultContext()
ctx.Client().Environment().
WithCheck_(Lint).
Serve(ctx)
}
// Lint everything (engine, sdks, etc)
func Lint(ctx dagger.Context) (*dagger.EnvironmentCheck, error) {
return ctx.Client().EnvironmentCheck().
WithSubcheck_(EngineLint).
WithSubcheck_(PythonLint).
WithSubcheck_(NodejsLint), nil
}
I think we’re headed in a direction where the names of Go functions don’t matter as much. Which leaves the problem of graphql namespacing
stiff abyss
Yeah, that's what I'm trying to solve in my mind and seeing if you guys have already discussed it.
We already have the nested "path" by following Check.name to every subcheck and so forth.
We just need to connect that to the right resolver.
tender matrix
This is from my naive top hat guinea pig perspective (not deep knowledge of core): I expect the names of my go types & functions to be between me and my SDK, as I register them
naively that leads me to picturing something like a graphql query in inputs.json when the engine asks my sdk runtime for something
instead of a graphql query there is a “resolver path” with internally defined meaning that I don’t fully understand. So any namespacing question that involves that resolver-path protocol , is a mystery to me
stiff abyss
Yes, that's what I'm currently thinking: "resolver path".
tender matrix
does that resolver path leak go types outside of the sdk loading them?
stiff abyss
No. It would be unique to an environment.
tender matrix
is it like a graphql query but encoded differently?
stiff abyss
No, it's more like the ResolveInfo, I just need the path and right now it's just parent type, parent args and current field (resolver name).
With the path, I can trace it in code when you register functions with WithCheck and WithSubcheck for example. Because each of those fills the Check.name field, and they are done in relation to a parent check, so you can trace to the right function. Then you could have one generic resolver. The problem with that is it won't be automatically picked up by codegen as client.Environment(envid).GoLint(). It would be more like client.Environment(envid).Check("sdk:go:lint").
But you can make custom codegen from query (data) instead of just introspection query (schema).
tender matrix
I see. Maybe it’s ok if each individual check doesn’t get a codegen’ed call
since the return types are all the same anyway (a check result is a check result)
stiff abyss
I think a query based codegen would be pretty slick. Clean, not needing to change schema dynamically.
stiff abyss
I know, I'm just saying you don't need to change the schema for codegen to work with the dynamic stuff.
tender matrix
I think graph (or whatever we call it) is where codegen is most needed
stiff abyss
I still don't know what that is for 🙂 Can you brief me?
tender matrix
It’s a placeholder name for the env-specific schema extension. For scripting, chaining and composition
commands a-la dagger do have a dual identity as 1) lightweight CLI tools, for humans, and 2) scripting entrypoints, for code.
The idea is to split those in distinct entrypoints:
- “tools” (CLI tools) for the former
- “graph” (expose your environment as a data graph) for the latter
this is to avoid the eternal problem of shell scripts wrapping shell scripts