#I m exploring ways to get more mileage

1 messages · Page 1 of 1 (latest)

pallid mist
#

So is the general idea that save lets you store variables that can be referenced later via $var?

If so, I like the general idea here, the problem I'd anticipate with this exact approach is that it will be hard for us to get the types to work nicely.

For instance getenv couldn't output a String type, it would have to be our own object that has a save field on it. Totally possible, but reminds me of the route we went down before with laziness where you can't just use normal primitives anymore, you have to use wrapper types.

A variation that I think could work (but would need more verification for soundness) is something like:

{
    core {
        getenv(key: "GITHUB_TOKEN") @save("token")
    }
}

{ netlify { deploy(token: "" @use("token"), contents: "$build") } }

However the "" @use("token") part is quite ugly, so that doesn't quite work. Would avoid the type problem though I think

glad magnet
#

(The trick of course is that FS { save }, Secret { save } and FS { install } modify the state of the server for the next query)

#

Yeah I was thinking of extending only FS and Secret with a save

#

In the case of getenv it might require something like getenvsecret or equivalent

#

But you can substitute that with eg. vault { getToken(id: "netlify-dev") { save } }

pallid mist
#

Ah okay, yeah that could work then I think. We control the FSID and SecretID scalars so we can make the representation using $ work consistently.

I also like the use of universe(...) { yarn { install } } too, separately from the save idea

#

Actually just realized this somewhat less ugly variation on the directive approach might work:

{
    core {
        getenv(key: "GITHUB_TOKEN") @var("token")
    }
}

{ netlify { deploy(token: "token" @var, contents: "build" @var) } }

The only reason the directive approach could be preferable is that it might work with any type out of the box rather than just ones we've added support to.

glad magnet
#

We could have the directive version for power users. But the regular field version is easier to grok I think

pallid mist
#

Sure, that could work too

glad magnet
#

This is actually related to the "what to do with client stubbing" topic. Because both have to do with the role of gql queries, and how much to embrace them and expose them in the DX

#

I now understand better the distinction between queries (client-side) and "the graph" (types+fields+resolvers server-side)

#

And why you currently need to bundle both in an extension

#

But what if we embraced the distinction in our dx

#

And actually require clients to write their queries

#

maybe add a convention in project layout for where to store them

#

and then stubs could be generated from those queries

pallid mist
#

My thinking on this has been:

  1. Default case should be that queries (i.e. what's in operation.graphql) are autogenerated from the schema of the callee
  2. Optionally, the callee author can provide extra hand-written queries if they want
  3. Optionally, the caller client can also provide extra hand-written queries if they want
#

So base case is no extra work for either callee or caller, probably works 90%+ of the time. But if either party wants something extra+custom, they are able to do so in a fairly straightforward way

glad magnet
#

But what if we leaned on the query/graph distinction harder, and made it a core part of our DX? By making queries a client-side concern, and graph an extension-side concern?

#

We haven’t decided yet how to map dagger concepts (actions, services, artifacts) to gql. Now it appears there are 2 options we could follow: map them to the server-side graph, or map them to client-side queries

glad magnet
#

I think there’s something here that could work really well. I’ll try doing a poc tomorrow

pallid mist
# glad magnet I think there’s something here that could work really well. I’ll try doing a poc...

Sounds great, in the past I've thought about this whole schema vs. operation distinction and what it could enable (besides the boilerplate it currently causes), but have never been able to come up with something very concrete, so looking forward to it.

I'll go focus on some other dx fixups besides automating the creation of operations.graphql in the meantime, plenty else to do.

Just one addendum thought that @prime raptor and I have discussed a bit in the past: the fact that query operations currently have to be defined ahead of time also limits the usefulness of chaining in generated clients. You have to predefine your chained queries and then generate the client stubs for them, which takes away the power of being able to build chains on the fly in your code. Kind of similar to the schema-first vs code-first distinction I suppose, but in the context of operations?

The alternative of writing raw graphql queries is also bad though because A) it's devastatingly verbose in langs like Go and B) it prevents you from getting compile-time type checks (though you do get runtime checks via graphql at least).

Our vague thought has been to find a way to generate clients that can create query operations on the fly. So like from Go you would generate clients that enable something like:

core.Image("alpine").Exec(ExecInput{args:["..."}).ID()

Which, underneath the hood, creates the corresponding chained operation and submits it. There is almost certainly not any graphql codegen tool available for Go that can do this, so we will need to do our own thing, but we also have some (also vague) ideas on how to minimize pain there too (across all languages, besides just go).

Either way, feel free to ignore if this has nothing to do with what you're imagining, just wanted to throw it in the mix too.

glad magnet
#

Very helpful, thanks

prime raptor
#

Our vague thought has been to find a way to generate clients that can create query operations on the fly. So like from Go you would generate clients that enable something like:
core.Image("alpine").Exec(ExecInput{args:["..."}).ID()

Yep, indeed. Needs lots of experimentation but could be much nicer than any gql code generator out there

Could also handle "branching"/"parallelism", not sure how though yet. Something along these lines:

alpine := core.Image("alpine")
dagger := alpine.Parallel("dagger").Exec("curl", "https://dagger.io").Stdout()
github = alpine.Parallel("github").Exec("curl", "https://github.com").Stdout()

which somehow would translate to:

core {
  image(ref: "alpine") {
    dagger: exec(args: ["curl", "https://dagger.io"]) {
      stdout
    }
    github: exec(args: ["curl", "https://github.com") {
      stdout
    }
  }
}
#

Without something as sophisticated as this (chaining, branching, etc), codegen is inferior to raw gql queries (you lose a lot of LLB advantages by doing back&forth for every LLB op)

glad magnet
pallid mist