refactoring away from complex input types

[dreamy gyro]

Sure thing - so the plan is to stop using these input Foo { ... } types and instead use the ID + withFoo pattern to construct them, so we'll have an ID-able TypeDef object that can be used both as an input and an output. That way we won't have to deal with wanting both an input Foo { ... } and a type Foo { ... }.

I'm not sure yet if I'll extend this refactor to other existing input types that we don't care about having as an output type (like BuildArg). I'm inclined to for total consistency, since we've been mentioning things like making the core API itself just another module, and we don't have a way to represent input ... types for modules at the moment. It'd be nice if it was all just one way of doing things, but BuildArg is where it starts to feel a little strange.

[gusty stag]

Ok, is it just to reduce the duplication on input/non input representations and have more consistency or did this implementation end up giving other problems?

[dreamy gyro]

A bit of both - Erik framed it to me as an experiment that ended up leading to more problems than it solved. For example it required a breaking change to make input type args pointers in the SDK, because otherwise the TypeDef type infinitely recurses (it's self-referential), which meant BuildArg had to turn into a pointer.

The main benefit is just not having to deal with input/output representations of the same type, and to trend towards every object being ID-able, including potentially custom types in the future. I'm still not totally sure what to do about BuildArg though - i.e. whether that should also be an ID-able object or whether we still support inputs for plain simple structures that are only passed as inputs. (If so, it'd be worth trying to map out how SDKs/modules should support that too.)

If the current API is causing implementation problems for Python I can pivot and try to get it in the shape we want more quickly - I'm currently on a tangent exploring v2 IDs since I keep butting heads with problems that it solves. (For example the awkward cycle between Module IDs and Function IDs.)

(v2 IDs is this: https://github.com/dagger/dagger/issues/3923#issuecomment-1714090592)

[gusty stag]

Yeah, it's a good move to go back. I'm also not sure about BuildArg and PipelineLabel. We usually need a constructor and we repeat the fields of the target type as field arguments in the constructor. Like WithEntrypoint(name, value), instead of WithEntrypoint(Entrypoint{name, value}). Those two were made as inputs I suppose because they're both in an argument that is a list. In Python, instead of dagger.BuildArg("wat", wat) (input type), it would have to change to dagger.build_arg("wat", wat") (field in Query). And there's the slight performance hit for resolving the IDs before the main query. API wise there's more need for stitching with IDs as well. Even if we decide to make them IDable, let's revisit it later, with another PR.

[gusty stag]

It's not a problem per se, just anticipating that it's wasted work to implement for current API if it's going to change soon. I can just support basic types for now and expand later with the new core change.

[dreamy gyro]

Sounds good - I'll put a pin in my ID exploration and get this converted over today

[gusty stag]

I haven't eaten yet, but I'll take a look at v2 IDs after. Sounds interesting 🙂

[dreamy gyro]

Yeah, it's fun to think about! Poke any holes you can find 😄