Type inference, undefined values and undefined behavior

[marble meadow]

I'm having the following function in my personal library:

pub fn FieldType(comptime T: type, comptime field: []const u8) type {
    return @TypeOf(@field(@as(T, undefined), field));
}

which can be used as

const S = struct { i: i32 };

pub fn main() void {
    const F = FieldType(S, "i");
    _ = F;
}

Now I want the same for pointers (so e.g. the const attribute is properly propagated):

pub fn FieldPtrType(comptime Ptr: type, comptime field: []const u8) type {
    return @TypeOf(&@field(@as(Ptr, undefined), field));
}

which would be used as

const S = struct { i: i32 };

pub fn main() void {
    const P1 = FieldPtrType(*S, "i"); // expect P1 == *i32
    const P2 = FieldPtrType(*const S, "i"); // expect P2 == *const i32
    _ = P1;
    _ = P2;
}

This however fails to compile

src/root.zig:6:21: error: use of undefined value here causes undefined behavior
    return @TypeOf(&@field(@as(Ptr, undefined), field));
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/root.zig:13:28: note: called from here
    const P1 = FieldPtrType(*S, "i"); // expect P1 == *i32
               ~~~~~~~~~~~~^~~~~~~~~

• Is this limitation necessary? (I see no point for undefined behavior here)
• Is there a simple workaround, or do I have to go the full route of parsing and copying the @typeInfo()?

[tender mantle]

could you do ```ts
pub fn FieldPtrType(comptime Ptr: type, comptime field: []const u8) type {
return *FieldType(@typeInfo(@TypeOf(Ptr)).child, field);
}

ah you want the const to remain too

[storm isle]

That wont work because the & operator needs actual data

If Ptr is a pointer type, you can use typeInfo to get both the pointed type and whether it's const or not

[marble meadow]

I see that the & operator requires actual data, but I don't see a reason why it would actually need it in the given context. That's my question.

Or is it that Zig (differently from C++) has no concept of "unevaluated context" and would literally evaluate the expression inside the @TypeOf before applying the @TypeOf?

[storm isle]

That's the case for that one, yes

builtin functions are still actual functions

[bold hare]

no they're not, the rules don't apply to them

(see: cimport, the various variadic builtins, etc)

however, it's not really possible to resolve the type of zig expressions without evaluating them, because of comptime

eg. consider the expression if (comptime something()) 42 else "hello, world!"

this expression has to be evaluated in order to determine its type, because its type depends on a comptime expression

it's far too complex to try and only evaluate the parts that are actually required to determine the type, so zig will just evaluate the whole thing

[storm isle]

(Corrected myself)

[marble meadow]

Actually, I'm wondering, is that really so difficult. Zig already has evaluation context determined by the expected result type (which is used by the casting intrinsics). One could add to this information a flag, specifying, whether the expression value is needed or only expression's type. Not sure if that would really work, but off the top of my head I don't know why it wouldn't.

[bold hare]

it's a huge amount of extra complexity for a feature that's not actually that useful

most cases where you want this behaviour, you can just use reflection instead

[marble meadow]

Reflection often gets a bit too involved. BTW where's the idea of "huge amount" coming from? I thought my proposal is rather simple (provided it actually works).

Simple case: say I wanted to write my own @field() function (actually smth like this is my actual use case). How would I infer the type of the result? I need to transfer all the pointer properties from the parent to the field. Actually not all of them, alignment may change. Maybe there are also some further fine details which I wouldn't be sure if I missed them. See what I mean?

[bold hare]

almost every construct in the language has to be special cased for this

[marble meadow]

I agree, but it doesn't seem like a major change: just one flag to suppress evaluation.

[bold hare]

i... what? my point is that "just one flag" has to be checked across the whole of sema

every branch construct has to check it and handle it specially

[marble meadow]

I'm not familiar with the sema arch, but my impression is that it'd need to be done in a single place per each syntactical construct. How many are there? Ca. 100 probably?

I mean I agree that it's not a minor or a local change. But I don't imagine that it's truly "huge"

[bold hare]

there's like 300ish zir instructions currently

the amount of additional complexity is so not worth it

like yeah it's "just" one extra check, but that extra check is subtle and complicates every single part of semantic analysis

[marble meadow]

I guess it's subjective. But ultimately it's up to the Zig team to decide, of course.

[bold hare]

for a feature that's completely not required at all

[marble meadow]

This is also subjective. IIRC I regularly come across use cases where it would have been so easy to use @TypeOf. Just look at my FieldType function at the top and compare to the respective function in the std.meta

[bold hare]

by "required" i mean "you can't do certain things without it"

[marble meadow]

that's also a subjective argument. ultimately no programming languages besides assembler are "required". actually even the assembler is not

[bold hare]

reflection works great. yes it's "more involved" but it often more accurately reflects what you're trying to do, and that userspace complexity is worth it to avoid adding a huge maintenance burden to the compiler

[marble meadow]

Reflection is a great feature, largely missed in C++. And it's indeed more powerful than @TypeOf. So if it's the choice of an either one, reflection all the way. However in many simpler cases reflection requires too involved code, which would have been one-liners with @TypeOf. FieldType is a great example of that (the std.meta counterpart takes one or two screens, depending on your montior size 😄 )

[bold hare]

how tiny is your monitor lol

[marble meadow]

Much worse, using the reflection sometimes requires essentially duplicating the language standard, which is a bit dangerous thing to do.

E.g. if I want to check whether one type coerces to another: I have to make sure that I understand the coercion rules perfectly (and that they do not change)

[bold hare]

tbh i don't think i've ever seen code that was harder to read due to using reflection over typeof

[marble meadow]

I know that reading is more important than writing by the Zig zen, but it's not that writing (or maintainability) are completely unimportant?

[bold hare]

this is a fair complaint, but not related to typeof

[marble meadow]

related to using reflection

[bold hare]

we're talking about reflection as an alternative to typeof tho

[marble meadow]

then just take the example at hand. how do I infer the return type for @field() correctly (without duplicating the language's own logic with edge cases etc.)

[bold hare]

i'd also like to note that a "type-only execuction" mode would be pretty confusing to users, because it would still execute code, just not all of the code

and understanding which parts of the code are evaluated and which aren't is nontrivial

[marble meadow]

now this is a fair argument, but I'm not sure things like this do not exist in Zig already 😄

maybe one could solve it with another keyword like comptime

[bold hare]

get the field type, create a pointer type to it and copy the pointer attributes across

[marble meadow]

that probably works, but actually is wrong

actually I already have code which does exactly that, and guess what - it's therefore buggy

[bold hare]

the only thing you have to be careful about is alignment, but that's a single @min

[marble meadow]

this is what I mean - there are subtleties in the language which one easily misses by writing such code

[marble meadow]

yeah, and it's a completely non-obvious one, easily missed (I did)

[bold hare]

i feel like if you're this deep into the type-level weeds, you should probably understand how alignment works?

[marble meadow]

there is a difference between understanding and full awareness of all edge cases

[bold hare]

i guess

idk that one seemed fairly obvious to me

[marble meadow]

wasn't to me. esp. coming from C++, where one tends not to bother with the alignment, unless absolutely necessary

[bold hare]

fair, i guess if you're not used to thinking about it

i think the fact that it's a thing you have to explicitly specify when constructing the type using @Type kind of solves that though

[marble meadow]

But coercion is actually another problem not exactly with typeof, but with the de-facto limitations of the reflection - it's very difficult for any language non-expert to write correct code testing the coercibility using the reflection

[bold hare]

if you're just blindly copypasting stuff you'll miss it, but as long as you actually spend a few seconds thinking about what you should be copying over, the alignment thing becomes obvious

[bold hare]

very true, and i'm not sure why the builtin that used to exist for this was removed

i guess because it's a niche usecase

[marble meadow]

What happened to "edge cases matter"? 😄

[bold hare]

to be clear i agree on the coercion thing, i think there should at the very least be something in std.meta or somewhere that's kept in sync with the compiler

[marble meadow]

std.meta generally doesn't look ripe or functionally complete to me. E.g. guess why I wrote my own FieldType function - because the one from std.meta wasn't usable for me (at least not without some extra metaprogramming helpers)

And there is nothing like FieldPtrType either AFAIK

[bold hare]

it needs some love for sure :)

[marble meadow]

Anyway. thanks for the discussion. It helps structuring the thoughts, much appreciated.

[marble meadow]

So, based on this discussion, I did some testing. It seems indeed, there is code generated for the argument of @TypeOf. Now I'm wondering how performance-safe are constructions like @TypeOf(@field(@as(T, undefined), field)) or @TypeOf(ptr.*) etc. Is it a good idea to rely on the compiler optimizing them out, or should I rewrite all this code?

[marble meadow]

NB. I think I now understand why std.meta.FieldType accepts an enum literal instead of a string - this only confirms my point how involved it often becomes to use reflection.

[marble meadow]

FWIW

pub fn FieldPtrType(comptime Ptr: type, comptime field: []const u8) type {
    const Data = @typeInfo(Ptr).Pointer.child;
    comptime var dummy: Data = undefined;
    const ptr: Ptr = &dummy;
    return @TypeOf(&@field(ptr, field));
}

I truly looked into using std.meta once again, but after studying the sources decided not to. There are unsolved performance problems (as a comment states), one has to deal with the mixture of using strings and enum literals to specify fields, etc. If even std has problems using reflection, what should be expected from the "mere mortals"?

[harsh loom]

theoretically there shouldnt be any runtime performance cost for comptime expressions/functions do you have a godbolt link for this?

[marble meadow]

I didn't say "runtime performance". This is the comment I've been referring to

[harsh loom]

oh, i guess i dont usually think of compiling time when i hear performance. i'd think you'll be alright as long as you're not dealing with large arrays of data (or apparently long for loops) during comptime, haven't heard anything about worrying how many TypeOf statements you use

[storm isle]

This is a comment about StaticStringMap, not reflection

[marble meadow]

std.meta.stringToEnum is a part of the reflection library, so it is about reflection. This function will be needed in combination with std.meta's field typing features. Since the built-in language features deal in strings and std.meta deals in enum literals, one needs to invoke std.meta.stringToEnum.

[storm isle]

stringToEnum converts a runtime string to an enum

[marble meadow]

How do I convert a comptime string to enum then?

[storm isle]

@field(TheEnum, the_string)

[marble meadow]

Doh. Never would have thought of that, thanks!

[marble meadow]

FWIW I realized that @field(@as(Ptr, undefined), field) doesn't really access the pointer and neither does &@field(@as(Ptr, undefined), field). Technically there is a pointer addition under the hood, but since it's implicit, there is no harm in allowing it and simply assuming that the address of the referenced entity is still undefined after applying the field access. This doesn't require the concept of "unevaluated context".

[marble meadow]

Got to try this for real. Still lots of boilerplate to get a field type by the field name, so std is clearly missing a function for that, IMHO

    const field = "a";
    const T = std.meta.FieldType(
        S,
        @field(std.meta.FieldEnum(S), field),
    );

[storm isle]

also works:

const T = @TypeOf(@field(@as(S, undefined), field));```

also: https://github.com/ziglang/zig/issues/21702

[marble meadow]

Yeah, but see the rest (esp. the opening message) of this thread 😄 😄 😄

[manic prism]

man ive spent so much time on this specific problem, and it sucks to do, there are grey areas involving comptime-ness, and due to anonymous struct literals being both impossible to reflect and significant for coercion, it is sometimes literally impossible to determine without extra info. To add insult to injury, the builtin @canCoerce used to exist in very old zig versions, but has since been removed

also, have you tried using a function like this instead of making coercing from undefined? Haven't tested this, but I think this should work.

fn declval(comptime T: type) T {
  const eval_msg = "The result of declval cannot be evaluated";
  if (@inComptime()) {
    @panic(eval_msg);
  }
  else {
    @compileError(eval_msg);
  }
}

I used @panic if inside comptime to try and prevent the compile error from being propagated up the @TypeOf statement