thinking about enums

[mighty sleet]

okay, had a thought.
so the whole point is so that if module A has a declaration like ModValueFoo ModType = "bar" we create an enum with name foo and value bar. all makes sense - the value can be anything we like now, it's not just constrained to graphql names - the name foo (or actually, FOO, is what shows up in queries)

but one little effect of this is that this would get codegened by a different module as ModValueFoo ModType = "bar" (the same as before). But when I serialize this into a graphql query, by default, I'm getting bar. For go, I can kind of work around this by writing my own enum conversion logic, but wondering what this might look like for python or typescript 🤔 essentially, i want to serlialize the "name" of the enum entry instead of the "value" of it. i think it should be doable regardless, but just wanted to confirm that this is the desired behavior.

fyi @hearty compass @mental vale

[hearty compass]

Yes, that looks like what we want. Unless I'm misunderstanding. Shouldn't it be ModTypeFoo ModType = "bar" actually?

[mighty sleet]

yeah that 😄

cool, i was trying to think if there was a way to avoid the need to do custom marshalling/unmarhsalling/querybuilding here, but it does feel neccessary if we want this behavior

[hearty compass]

In Python it would be:

class ModType(Enum):
  FOO = "bar"

So the member is accessed as ModType.FOO, similarly to Go's ModTypeFoo.

[mighty sleet]

so when we do querybuilding, is there a way to do the equivalent of ModType.FOO.name() to get "FOO"?

(in python)

[mental vale]

Same for Typescript, it woud be

enum ModType {
   FOO: "bar"
}

[mighty sleet]

i mean, no worries if not, we can do a big dict lookup or something

[hearty compass]

Yes, it's ModType.FOO.name.

[mighty sleet]

😦 how neat

the go impl is just gonna be a chunky switch statement or something

that doesn't sound too bad then, i'll probably be able to the python bit myself then

@mental vale any idea of what it would feel like in ts?

[mental vale]

->

[mighty sleet]

right, but if i have a variable of type ModType, how do i get FOO from it (the name)

instead of just the "bar" (the value)

[mental vale]

Hmm

[mighty sleet]

for go i'll just add a custom method onto the ModType called .Name, and have a switch stmt in that

ugly, but we could do the same in ts

[mental vale]

Object.keys(ModType)[Object.values(MyType).indexOf("FOO")]

[mighty sleet]

😛 cool, must be nice to use one of these fancy languages with features

[mental vale]

Haha, I mean same as interface, you could handle that in a generic way, that should work if you do it outside of the user function

As long as user receive a enum type and output an enum, you can do some funny thing 😛

[mighty sleet]

@hearty compass i'm going a little bit insane. i can't seem to control the json serializazation of enums for python

i want an instance of Foo.X to json serialize to the name "X", not the underlying value of it

otherwise, the values are significant in some circumstances - specifically, they end up leaking into the user-api through things like defaultValue, which throws off codegen

[hearty compass]

How are you doing it (and where)?

[mighty sleet]

well this is the problem

i can't

i tried hacking this into cattrs

https://github.com/python-attrs/cattrs/issues/235

but you can't, because of this ^

you can't override a builtin type for json.dumps, even if you set the cls

potentially there's another json lib we could pull in that does allow this?

aside from that, i'm kind of really unsure of how to get it to work

i also though about just having the engine handle passing values around, and doing the conversion itself, but that means that all the engine enum logic just gets vastly more complicated, because it needs to handle both possibilities in way too many places

[hearty compass]

I don't see why this can't be done, I'll work it out.

[mighty sleet]

🙏

much appreciated lol 🙂 i'll move onto getting typescript sorted once i pick this up again

the latest state of things is in https://github.com/dagger/dagger/pull/9518 if you're interested, it should "kind" of work for python (the codegen bits are mostly good, it's just the marshalling/unmarshalling bits that need work)

[hearty compass]

>>> import enum
>>> class Choices(enum.Enum):
...     FOO = "foo"
...     BAR = "bar"
...
>>> import cattrs
>>> from cattrs.preconf.json import make_converter
>>> conv = make_converter()
>>> conv.structure("foo", Choices)
<Choices.FOO: 'foo'>
>>> conv.unstructure(Choices.FOO)
'foo'
>>> @conv.register_unstructure_hook
... def to_enum_name(val: enum.Enum) -> str:
...     return val.name
...
>>> conv.unstructure(Choices.FOO)
'FOO'
>>> @conv.register_structure_hook
... def from_enum_name(name: str, cls: enum.Enum) -> enum.Enum:
...     return cls[name]
...
>>> conv.structure("FOO", Choices)
<Choices.FOO: 'foo'>

[mighty sleet]

I think I got this far

But then, our enums extend str

And that stops working then I think, due to cattrs issue I linked above

And I think we need to extend str, or we need some other way to have the tuple syntax for enum value descriptions

[hearty compass]

I’ve been meaning to change that to use docstrings like codegen, I just need to get down to ast for that. Since we’re changing enums anyway now would be a good time to do this. However, people could still extend str on their own so it’s good to be able to support it.

Fyi, we're extending str not because of the descriptions, it's to limit to string values.

[mighty sleet]

Ahhh I see yeah

[hearty compass]

import enum

import rich
from cattrs.preconf.json import make_converter


class Enum(str, enum.Enum):
    """Custom enumeration."""

    __slots__ = ()

    def __str__(self) -> str:
        """The string representation of the enum member."""
        return str(self.name)


conv = make_converter()


@conv.register_unstructure_hook
def to_enum_name(val: Enum) -> str:
    return val.name


@conv.register_structure_hook
def from_enum_name(name: str, cls: type[Enum]) -> Enum:
    return cls[name]


class Test(Enum):
    FOO = "foo"
    BAR = "bar"


rich.inspect(conv.unstructure(Test.FOO), title="unstructure", docs=False)
rich.inspect(conv.structure("FOO", Test), title="structure", docs=False)

[mighty sleet]

Oooo I never got that working 👀

Not quite sure what's different between my attempts and this, I dont think I was using the decorators

But that looks perfect, I'll give it a spin tomorrow 🙂

[hearty compass]

That base enum comes from from dagger.client.base import Enum.

Also works without the decorators:

import enum

import rich
from cattrs.preconf.json import make_converter

from dagger.client.base import Enum

conv = make_converter()


def to_enum_name(val: enum.Enum) -> str:
    return val.name


def from_enum_name(name: str, cls: type[enum.Enum]) -> enum.Enum:
    return cls[name]


conv.register_unstructure_hook(Enum, to_enum_name)
conv.register_structure_hook(Enum, from_enum_name)

conv.register_unstructure_hook(enum.Enum, to_enum_name)
conv.register_structure_hook(enum.Enum, from_enum_name)


class TestA(enum.Enum):
    ONE = "1"
    TWO = "2"


class TestB(Enum):
    THREE = "3"
    FOUR = "4"


rich.inspect(conv.unstructure(TestA.ONE), title="unstructure A", docs=False)
rich.inspect(conv.structure("TWO", TestA), title="structure A", docs=False)

rich.inspect(conv.unstructure(TestB.THREE), title="unstructure B", docs=False)
rich.inspect(conv.structure("FOUR", TestB), title="structure B", docs=False)

What doesn't work is register only enum.Enum but then use dagger.client.base.Enum.

[mighty sleet]

right

Having to register both? I definitely was not doing that, how'd you work that out 😂

I was just registering on the base enum stdlib and thinking that would work

[hearty compass]

I knew that they're based on singledispatch, which does a issubclass check. So I knew registering dagger's Enum should work.

[mighty sleet]

😅 thank you for the help

[hearty compass]

Yeah, the str is taking precedence on that case, but we could deregister and replace it, but not worth it. Let's just have people subclass the dagger.client.base.Enum by convention, which makes it simple to detect.

[mighty sleet]

It honestly is kinda fun to try and work on features cross SDK though, even though I'm less familiar with python 🙂

Feels much nicer than just punting it over the fence to someone else

[mighty sleet]

another question for ya @hearty compass 😛 kinda unrelated.

it seems like in python doing something like:

@dagger.object_type
class Test:
    foo: str = "abc"

puts foo into the constructor, even though it's a private field. is there a way to not have it do that.

i'm just removing the top-level property, and now just setting it in __init__ but not sure if that's idiomatic

[hearty compass]

Yes, there is. The @dagger.object_type is a wrapper on dataclass. So you need init=False:

import dataclasses
import dagger


@dagger.object_type
class Test:
    foo: str = dataclasses.field(default="abc", init=False)

Docs: https://docs.python.org/3/library/dataclasses.html#dataclasses.field

This used to be explained clearly in https://docs.dagger.io/api/constructor but I suppose it was removed because the different permutations of dataclasses.field, dagger.field, and init=True, made the Python docs more complex than the others.

[mighty sleet]

Ahhhh awesome 🙏

Yeah was just hitting this when writing tests 😉

Would be nice to have some docs for it I guess

[hearty compass]

To expand the clarification:

• init: bool is for saying that the class attribute should be an argument in the auto-generated __init__ (defaults to True)
• The diff between dataclasses.field and dagger.field is that the latter exposes the property as a Dagger field. That's when you choose one over the other.
• You can also specify constructor-only args, i.e., a part of __init__ but not a property of the class, by wrapping the type in dataclasses.InitVar

[hearty compass]

Don't know when that was removed and what that discussion was like.

For a Python user, dataclasses should be well known. What's dagger specific is just replacing dataclasses.field with dagger.field when you want to expose a property as a Dagger field.

There's an info box on https://docs.dagger.io/api/constructor#simple-constructor saying that @dagger.object_type is a wrapper on @dataclasses.dataclass, but without the whole permutations that were documented more at length before, it could still be better by providing a bit more (succint) info in that box.