Zag

[languid canyon]

Well I do that

[fallen bobcat]

I might have BSP stuff thats set though

[heavy sandal]

lazy tpr is weird because its basically always faster on emulator and usually slower on modern actual ardware iirc

[fallen bobcat]

so might as well zero it and set it up properly

[heavy sandal]

i use default values for percpu stuff

[fallen bobcat]

no

[languid canyon]

Why doesn't it propagate to all cpus then?

[fallen bobcat]

because stuff like current_thread or whatever idk

i dont want stale state to be copied over then I get a race

might be simpler to just reinitialize it

[fallen bobcat]

idk what you mean by lazy, but I mean that moving between software levels doesnt cause a cr8 move

I dont do like windows where its just inlined to a mov cr8 and do self-ipi

my way might be better because I dont have the overhead of a self-ipi when setting up DPCs etc

[heavy sandal]

i dont think windows does a self-ipi for it?

[fallen bobcat]

(I actually stole it from will, thank you will)

[fallen bobcat]

idk how they would do it otherwise

if you solely use cr8

[heavy sandal]

the instruction is serializing iirc so you just dont get issues?

when you change cr8 the tpr is just changed on the spot

[fallen bobcat]

?

yes

so

that is not what I mean

[heavy sandal]

i dont see why youd ever need a self-ipi?

[fallen bobcat]

for a DPC

you want it to be queued when IPL is >= Dispatch

[languid canyon]

What do you mean by it here

[heavy sandal]

just check the ipl when lowering it and run dpcs then?

[fallen bobcat]

the fully hardware way of doing that is making a DPC interrupt be masked whenever cr8 is set to DISPATCH

[fallen bobcat]

but then it's not inlined to a simple mov cr8

[heavy sandal]

ahh i see now

[fallen bobcat]

I dont know if windows actually does a self-ipi though, they might do that

[languid canyon]

When do you self ipi exactly

[fallen bobcat]

whenever you enqueue a DPC

or not

only one

[languid canyon]

So u enqueue it but it doesn't get through until you lower it?

[fallen bobcat]

you only need one IPI

yeah

[languid canyon]

Oh ok

[heavy sandal]

they do, the vector is 0x20 (they also use self-ipi for triggering APCs which is vector 0x1F which is technically illegal but the cpu is basically required to support it because its windows and they do it)

[languid canyon]

Idk if thats good but sure

[fallen bobcat]

I do it all in software

I have a bitmap

[fallen bobcat]

that's a weird vector

[languid canyon]

Why is it illegal

Oh exceptions

[heavy sandal]

intel explicitly says that vectors 0x00 - 0x1F are reserved for exceptions yeah

[languid canyon]

I mean if its not used why not

[heavy sandal]

but also the IPL associated with a vector is that first hex digit

[fallen bobcat]

which is atomically set by yourself or other CPUs, and they send an ipi to nudge remote cpus

I think this is better than sending self-ipis

[heavy sandal]

yeah its unused and basically cant be allocated now because windows compat lol so its fine to do its just technically reserved for possible future exceptions

[fallen bobcat]

and also doesnt rely on any hardware interrupt priority

you could just do if > dispatch cli

[languid canyon]

Lol

Windows casually chipping off a reserved exception vector from Intel

[heavy sandal]

the downside to lazy TPR is the extra IRQs you have to take when the IPL is desynced, and also you kinda cant do lazy lowering anyway i dont think because otherwise youll lose interrupts, which i think those extra IRQs compared to the cost of mov cr8 on modern hardware is why windows doesnt do lazy IRQL

[fallen bobcat]

idk what is lazy irql

[heavy sandal]

yup lmao. i also use the same vector for the same purpose just because i know windows is forcing it

[heavy sandal]

irql is the windows term for ipl

[fallen bobcat]

yes

I know

idk what is lazy

[languid canyon]

You write to cr8 on first irq

[fallen bobcat]

ah

[heavy sandal]

so the thing that i was assuming you do that is usually called lazy irql/lazy ipl is you set a percpu in memory var with the irql, and if you hit an irq that is supposed to be masked by the current soft irql you update cr8 then

[fallen bobcat]

nah I don't do that, I just set cr8 whenever IPL is moved to a hardware level

[languid canyon]

Yeah that's gonna lose some uacpi points

[heavy sandal]

what do you mean by hardware level?

[languid canyon]

[fallen bobcat]

no

because there wont be any cr8 moves there

[languid canyon]

Why not

[fallen bobcat]

over dispatch

[fallen bobcat]

because the only ipl moves will be between passive and dispatch, which are fully software levels

unless you disable interrupts or something

[languid canyon]

Nah that's only used during shutdown

[heavy sandal]

ahhh ok. difference for windows then is that windows actually uses irqls APC and DISPATCH to mask interrupts

[fallen bobcat]

which is literally curcpu()->ipl = new_ipl

[fallen bobcat]

yes

[heavy sandal]

so all the IRQLs are hardware

[fallen bobcat]

ok now I should figure out why tf its not booting in passive

ah ok turns out I am an idiot

now it works

[heavy sandal]

what was the issue?

[fallen bobcat]

basically I have a self pointer for cpu-local accesses

I forgot to set it

for APs

so they were reading from the BSP

[heavy sandal]

oh lmao

[fallen bobcat]

ok now I have to write more assembly so that it handles the APIC ID reading 😭

 /* Get xAPIC base address from data */
    mov (DATA + 24), %rax
    test %rax, %rax
    jz 1f                   /* apic_base = 0 means x2apic */

    /* xAPIC mode */
    /* Set APIC base to the provided xapic address */
    mov %rax, %rdx
    shr $32, %rdx           /* high 32 bits of address */
    /* %eax already has the lower 32 bits */
    or $0x800, %eax        /* Global enable */
    mov $0x1B, %ecx         /* IA32_APIC_BASE MSR */
    wrmsr

    /* Read APIC ID from memory mapped IO */
    mov (DATA + 32), %rax
    mov 0x20(%rax), %edx
    shr $24, %edx
    jmp 2f

1:
    /* x2APIC mode */
    mov $0x802, %ecx        /* x2APIC ID MSR */
    rdmsr
    mov %eax, %edx          /* x2APIC ID is in EAX */

ok it wasnt that bad

[fallen bobcat]

ok nice I think I'm finally done

it handles booting in x2apic mode too (I tried the patch)

[round tangle]

Would you happen to know where I might read a complete, detailed account of this scheme (the original source)?

I’m afraid I didn’t quite get how it works from the fragments I’ve seen.

[cyan prism]

I have this document floating around, might be what you're after.

no idea if its the original source, honestly not even sure where I got it from.

[fallen bobcat]

I don't think that's it

[fallen bobcat]

It's just a bitmap of pending software interrupts

[cyan prism]

Yeah my bad, I only looked at a few messages and not the whole conversation 😬.

[round tangle]

I’m not sure I fully follow the connection with self-IPI?

[crimson sand]

iirc self ipi is what windows uses and zag uses that scheme invented iirc by @past dome

for handling all software irql purely in software

[fallen bobcat]

Both are ways to queue interrupts

[fallen bobcat]

https://dl.acm.org/doi/pdf/10.1145/2814270.2814312?download=true cool paper

[fallen bobcat]

mhm I can probably get the fireworks test running on amd64 tonight

id like to get a proper backtrace working first though

[fallen bobcat]

I wonder if I remove the base module and find a way to use root instead it would unfuck LSP

[fallen bobcat]

huh it did

nice

[heavy sandal]

nice

ive had a few issues with modules with zls tbh

just didnt really notice much because i dont have many modules lol

[fallen bobcat]

I ended up having to do this

which is weird

else it just wouldnt link the stuff

[heavy sandal]

yeah i had to do that too

[fallen bobcat]

now i can finally open my editor without zls crashing instantly

nice

[heavy sandal]

nice

[fallen bobcat]

probably more idiomatic to do that too

[heavy sandal]

yeah it is

[fallen bobcat]

huh, why does qemu report tsc-deadline but my CPU doesnt support it

[heavy sandal]

like your actual cpu doesnt support it? its an msr access to set the deadline so thats going to vmexit anyway, maybe the hypervisor is emulating it in software

[fallen bobcat]

it doesnt no

my command line is -cpu host,+invtsc

[heavy sandal]

mhm

then idk

[fallen bobcat]

and my actual CPU does not report tsc deadline in CPUID

but QEMU does

[heavy sandal]

maybe the +invtsc is implying it for some reason?

idk

[heavy sandal]

just googled it, apparently kvm does it in software for VMs even if the cpu has native support and qemu is evidently just always forwarding that info

[fallen bobcat]

huh

interesting

[heavy sandal]

idk if you have a way to check the kvm caps on the host side

but there it is (assuming youre running KVM anyway)

[drifting blade]

Zig gives no guarantees about the order of fields and the size of the struct but the fields are guaranteed to be ABI-aligned.
I noticed this while reading the Zig docs about structs. Has this affected you when kernel programming?

[pastel citrus]

yeah but you can just use an extern struct if you want a C struct

[fallen bobcat]

Not really, in fact it's probably better because the layout is optimal

[languid canyon]

Even in C this issue exists, the structs where you care about layout you're expected to use the packed attribute anyway

[fallen bobcat]

ok cool now we have apic timer

note about AI policy:
idgaf if you use AI for a contribution as long as it is high quality

[fallen bobcat]

oh my fucking god

I just spent two hours on the silliest bug ever

I restored hardware IPL but not software IPL after returning from interrupts so it got stuck in ipl device

anyway now fireworks run on amd64!

single threaded for now because I havent done IPIs yet

[fallen bobcat]

lol @languid canyon

remember when you told me to copy from the BSP

for the cpu data

well turns out I forgot to re-initialize the pmm state per-cpu so every CPU was getting a copy of the BSP's pages

[heavy sandal]

lmao

thatll be the problem yeah

[fallen bobcat]

so physical memory was getting allocated twice

or more

[languid canyon]

good thing i initialize per-cpu before the pmm so that cant physically happen

[fallen bobcat]

no that's not the problem

I have per-cpu caches in the PMM

I was copying over the per-cpu cache of the BSP to all the APs

[languid canyon]

yeah in my case it would've copied an uninitialized state to all APs also

[fallen bobcat]

why

[languid canyon]

because PMM is not initialized at the time the per-cpu data is allocated

[fallen bobcat]

so how do you allocate it

[languid canyon]

early memory map allocator

same thing I will use for my pfndb

[heavy sandal]

yeah

[fallen bobcat]

huh weird

[languid canyon]

so that alloc_pages can hand out actual struct pages

[fallen bobcat]

I caught this on UM with gdb

[languid canyon]

that are backed by memory

[fallen bobcat]

the poison feature of the allocator was actually useful!!

I saw a bunch of deadbeefs in allocator state

[languid canyon]

big

[fallen bobcat]

now hopefully it works on amd64

[heavy sandal]

i do the same as infy

and yeah zig allocators poisoning on free is so good

(and the concept in general is tbh)

[fallen bobcat]

it does!!!

nice

[fallen bobcat]

I'm the one doing it

it's a config feature

i do checks for use-after-free too

but for max performance it should be disabled

[heavy sandal]

i usually use undefined instead of like deadbeef or whatever because the undefined gets optimized out on unsafe modes and searching for aaaaaaaaaaaaaaaa is just as easy

[fallen bobcat]

yeah I realized it did 0xaaaaaa... because I was crashing on it lol

[heavy sandal]

lol

[fallen bobcat]

huh its slow as balls on tcg

[languid canyon]

how do u allocate the pfndb without an early allocator

[fallen bobcat]

sometimes I wonder if UM is worth it but then I use it to debug weird shit and it's awesome

[fallen bobcat]

I do have an early allocator

I dont use it to allocate per-cpu data

I use my regular allocator

[languid canyon]

why do u allocate per-cpu data so late?

[fallen bobcat]

I do it after the memory allocator is setup

the slowness might be a zig thing

tcg might not like it doing a bunch of extra checks

bruh it crashes on ReleaseFast in AP startup??

[fallen bobcat]

ah ok wtf it was doing a crazy relative load in an unmapped section (which is part of the executable but I had forgotten to map)

[fallen bobcat]

ok no maybe measuring with -d int isnt a good idea

ok yeah it is fast enough for a debug build running on tcg

before it was a slideshow

yeah ok great it runs really well even on 12 CPUs

[fallen bobcat]

we now have qspinlocks

for now I put them on zone locks and the global heap lock

I dont think they're worth it for per-cpu locks that are rarely contended

[fallen bobcat]

@west talon how do you get simics gui to work

[west talon]

I only use the GUI to setup the project

[fallen bobcat]

it instantly triple faults but doesnt show any window and when I do
board.console.con.show
Window already displayed.

[west talon]

Then use the CLI to actually run it

[fallen bobcat]

yes

but I want a window to pop up

no window pops up

[west talon]

Uhhhh 😭

It normally opens a window for me

[fallen bobcat]

are you on wayland?

[west talon]

Yes

KDE

[fallen bobcat]

I'm on X11 KDE

maybe that's why

lemme try it

nope

[fallen bobcat]

did you install something specific?

[west talon]

Just intel SIMICS

I installed all the packs into it

[fallen bobcat]

and then you install the package

yea

[west talon]

And I used my #resources guide and it worked

Idk what’s wrong

There is the vibe coded QEMU patches floating around if you want

[fallen bobcat]

I dont want fred

[fallen bobcat]

does it show the window for you as soon as you launch the CLI?

[west talon]

Yes iirc

[fallen bobcat]

[board.mb.cpu0.core[0][0] info] VMP disabled. Failed to open device.

WARNING: Simics failed to enable VMP. Enabling VMP substantially improves
simulation performance. The problem is most likely caused by the
vmxmon kernel module not being properly installed or updated.
See the "Simics User's Guide", the "Performance" section,
for instructions how to setup VMP.

do you also get this

[west talon]

Then why are you wanting SIMICS

[west talon]

Yes

[heavy sandal]

this means theres no accel

[west talon]

This just means it will run super fucking slow

[heavy sandal]

if youre on amd its guaranteed to have that

[west talon]

~3x slower than TCG

[fallen bobcat]

mhm ok wtf

it does say I have the gui package

it just says the window is already displayed but it's clearly not 😭

[heavy sandal]

theres a sub-window that it pops up that lets you toggle the fb and two serial terminal windows, so check that ig if thats there?

looks like this

[fallen bobcat]

well ok I give up

I'mma try vmware and vbox

[heavy sandal]

F

[fallen bobcat]

holy shit the broadcom website is cancer

[fallen bobcat]

well vbox instantly triple faults

lik simics

I think I did something wrong with how I handle cr0

[heavy sandal]

🤔

[fallen bobcat]

ah

lmfao

pub const Cr0 = packed struct(u64) {
    pe: bool,
    mp: bool,
    em: bool,
    ts: bool,
    et: bool,
    ne: bool,
    reserved0: u11,
    wp: bool,
    reserved1: u1,
    am: bool,
    reserved2: u10,
    nw: bool,
    cd: bool,
    pg: bool,
    reserved3: u31,
};

spot the bug

[heavy sandal]

theres an off by one here

pg is bit 31 not 32 so reserved3 should be u32

which means something else (im guessing reserved0) should be one bit smaller

[fallen bobcat]

it's wp

yeah reserved0 should be smaller

[heavy sandal]

yeah

and reserved3 bigger

[fallen bobcat]

also it crashes on vbox after a while for some reason

[heavy sandal]

F

[fallen bobcat]

and vbox is fucking annoying to debug

wtf it stack overflows

but giving it more memory fixes the problem

but I didnt increase the stack size???

I'm thinking I'm not handling OOM properly somewhere

[fallen bobcat]

ahhhhhh

lol

panic panicked

which caused the stack to overflow

[heavy sandal]

oh hey i was having this exact issue on simics earlier

where i forgot to make panic non-recursive so suddenly an early panic would overflow

[languid canyon]

Did you fix the simics crash?

[heavy sandal]

yep

[languid canyon]

What was it?

[heavy sandal]

well which crash?

the ioapic panic was solved by making irq requests not eager

the earlier crashes were just stupid bugs

and it does still panic but thats because it cant open the boot partition because simics isnt using nvme so theres not a storage driver it can use

[fallen bobcat]

what happened here was that I wasnt taking the debug lock with IPL high

so when I panicked in a function with IPL high it crashed because it tried to raise to IPL dispatch

[languid canyon]

Why wasn't it reproing on QEMU?

[heavy sandal]

because qemu didnt have the devices in the acpi tables that were causing it and/or uses different GSIs for them

[languid canyon]

Ah

[fallen bobcat]

I have had a few revelations while looking at the freebsd zone allocator:

• Bufctls are not needed, instead, a bitmap can be used in the slab header itself to mark which items are free (you can then get the address by doing bit * size_of_obj)
• To get the slab header from an address for large slabs, you can store a pointer to the slab header in the struct page itself (no hashmap!)
• Instead of fixing the threshold for large slabs, it is possible to instead scan for the possible slab sizes and trying both inline and offpage headers to find whichever format wastes less memory

@long pendant this might prove useful to you, I think keyronex has a fairly traditional solaris-style allocator (?) and these seem to be easy to implement while providing potentially lots of benefits

I think the bitmap + slab pointer in the struct page is a very good addition and avoids having to deal with hashmap resizing (and should be faster too?)

[long pendant]

i've got the slab pointer in the page struct but the bitmap i don't, that's quite a nice idea

[fallen bobcat]

so you still keep a hashmap?

for buffer-to-bufctl conversion?

[long pendant]

or i should say i actually store the slab struct in the page struct

[fallen bobcat]

ah that works too (and I think linux SLUB does this), but I think it's easier to just store a pointer (you can reuse list linkage too)

[long pendant]

i should do but i can't remember if i did, i might have left it at linear searching

[fallen bobcat]

one thing they also do that I stole is having distinct alloc/free buckets, I think that is useful for SMR

instead of having previous/current magazines where operations are mixed

[past dome]

The first part made me go "ohhhhhh why didn't I think of that"

[fallen bobcat]

yeah it's pretty smart

overall seems like a natural, modern evolution of the solaris allocator

[fallen bobcat]

@heavy sandal you were right

I tried vibecoding it and it was utterly useless

took less time for me to write it than to get the AI to fix it

because of zig

[heavy sandal]

mhm

[west talon]

even with C stuff its just easier to do it yourself tbh

somtimes at the very least

[heavy sandal]

yeah

[fallen bobcat]

the problem is zig moves so fast it just doesnt know how to write it lol

[fallen bobcat]

.{
    .title = "My Project Configuration",
    .entries = .{
        .{
            .kind = .bool,
            .key = "some_bool",
            .label = "Enable Feature X",
            .help = "Example boolean",
            .default = .{ .bool = false },
        },
        .{
            .kind = .int,
            .key = "some_ranged_int",
            .label = "Some ranged integer",
            .help = "A ranged integer",
            .default = .{ .int = 100 },
            .range = .{ .min = 1, .max = 1000 },
        },
        .{
            .kind = .menu,
            .label = "SubMenu",
            .entries = .{
                .{
                    .kind = .string,
                    .key = "some_string",
                    .label = "Some string",
                    .help = "Example string",
                    .default = .{ .string = "hello world" },
                },
            },
        },
    },
}

thoughts on something like this?

really easy to parse

[heavy sandal]

good format imo yeah

[fallen bobcat]

pub fn parse(allocator: std.mem.Allocator, source: [:0]const u8) !Schema {
    var diag: std.zon.parse.Diagnostics = .{};
    defer diag.deinit(allocator);
    const parsed = std.zon.parse.fromSliceAlloc(
        Schema,
        allocator,
        source,
        &diag,
        .{ .free_on_error = true },
    ) catch |err| {
        std.debug.print("Parse status: {any}\n", .{diag});
        return err;
    };

    return parsed;
}

very complex parser

[heavy sandal]

lmao true

[heavy sandal]

use {f} for the diag, its got a format method

[fallen bobcat]

it's cool but I dont get pretty colors

[heavy sandal]

id be shocked you were getting any before tbh

[fallen bobcat]

I wasnt either

this is much better

but ideally id get cool colors

but whatever

[heavy sandal]

yeah you need a Terminal object for colors

because windows doesnt do ansi they cant just hardcode that in

and zon diagnostics dont have a format function that takes a terminal, only a writer

[fallen bobcat]

this is what it looks like

it's fine I guess

[heavy sandal]

yeah sadly i dont think youre getting any better here

well

there is a way

[fallen bobcat]

I'd need to parse it by hand

I think

the diagnostic

I'd need to extract the info

[heavy sandal]

its got an iterator

so youd call std.debug.lockStderr to get a terminal

[fallen bobcat]

mhm do we wanna support depends_on multiple stuff

[heavy sandal]

and then use that to set colors

and youd basically copy the format method for the rest

[fallen bobcat]

👀

[fallen bobcat]

https://asciinema.org/a/bTQcIiC0FR1akd7b @heavy sandal

very cool

[languid canyon]

Did you make your own ncurses stuff?

[fallen bobcat]

Yeah

It should be called zonfig

[heavy sandal]

nice!

[heavy sandal]

I agree

[fallen bobcat]

ncurses is surprisingly nice to work with for that kinda stuff

I haven't done anything very complex yet tho but it's fine so far

[languid canyon]

did u vibecode it

[drifting blade]

That's sick

I do the second one in my kurnul already but the bitmap one is pretty smart

You said that it's a modern evolution, but I don't see why this couldn't have been implemented like this from the start?

[fallen bobcat]

No the AI was incapable of making something functional lol

[lavish meteor]

where's the list of complete slabs? Also how come you aren't using a single list like described in the paper?

[long pendant]

he has developed a technique which may represent a major improvement

in place of bufctls a bitmap of allocated v.s. free slots

[lavish meteor]

ohh, yeah I remember hearing about that

Was just kind of curious because I was wondering if having 3 separate lists would be quicker than one partially sorted one :p

[fallen bobcat]

That hasn't been implemented yet

[fallen bobcat]

full slabs = complete slabs

and I use multiple lists because

[lavish meteor]

ah :p

[fallen bobcat]

I think it's easier to work with but I don't think there's any performance consideration

[lavish meteor]

okie dokie :3

[heavy sandal]

btw is this up on git somewhere yet?

[fallen bobcat]

nah it only handles bools atm

[heavy sandal]

fair lmao

[fallen bobcat]

once i add more stuff I will put it on git

the only AI part is the ncurses bindings

btw

the rest I tried to vibecode and the AI would straight up produce incorrect code so I ended up having to write it myself ()

[languid canyon]

that might be too much for khitiara lol

[heavy sandal]

yeah

rip

[fallen bobcat]

bro what

how is this too much

this is boring tedious work no one wants to do

[heavy sandal]

theres literally a zig tool to generate those man

zig translate-c

[fallen bobcat]

it doesnt work

I cant import the headers directly

[fallen bobcat]

but like being mad this is AI generated is genuinely crazy

[fallen bobcat]

idk why, for some reason it wont compile ncurses, doesnt like its macros

[heavy sandal]

rip

[languid canyon]

will u make an ai free version?

[heavy sandal]

yeah i think theres some issues with it rn because they swapped off of clang to arocc for the translate-c

[fallen bobcat]

i mean if you want to rewrite exactly the same code by hand be my guest

[heavy sandal]

honestly the externs is fine tbh

[fallen bobcat]

there's no logic

[heavy sandal]

when you said bindings i assumed you meant something higher level with logic in it

[fallen bobcat]

no

[heavy sandal]

externs like this are meant to be auto-generated

[fallen bobcat]

yes

[languid canyon]

yep

[fallen bobcat]

imo if y ou are against that you are genuinely delusional

[heavy sandal]

(im still a dumbass and writing them all by hand for zuacpi but thats more sunk cost fallacy atp lmao)

(plus its a lot more about the structs than the functions there)

[fallen bobcat]

also it builds a DAG so it can detect cycles

which is neat

[heavy sandal]

nice

[fallen bobcat]

I havent figured out how to do stuff like depends on (A && B) yet

[heavy sandal]

i mean just put them both in and its still a dag

[fallen bobcat]

.depends_on = .{
  .and = .{"A", "B", .not = .{"C"}}
}
``` is kinda goofy

[heavy sandal]

wont be a tree

ah yeah true

[fallen bobcat]

yeah I know, I menat more like expressing them in the schema

which is why I built a DAG from the beginning even tho I only support single dependencies right now

[heavy sandal]

mhm

[fallen bobcat]

I want to have a good base

[heavy sandal]

yeah

[fallen bobcat]

I will try doing prompts now hopefully its not too painful

[heavy sandal]

gl

[languid canyon]

maybe .depends_on = .{ .expr = "A && B" }

[heavy sandal]

or just .depends_on = "A && B"

the downside is then you need a full parser for that

(so might be easier to make if function-like, so .depends_on = "and(A, B)")

[languid canyon]

imo in this day and age the format should be easier for human and not the parser

[heavy sandal]

mhm

[languid canyon]

this aint at&t asm lmao

[heavy sandal]

lmao true

[fallen bobcat]

no because then I have to parse

[fallen bobcat]

@heavy sandal https://asciinema.org/a/KUtJAEeFyONaAAyy

holy

[heavy sandal]

niiiice

good shit

oh btw a stretch goal suggestion: let the config schema specify an output format (for now only have zon) so that other options like the plain key=value or json or whatever could be added later

[fallen bobcat]

yeah for now it'll be zon

but I'm thinking something like

.{.kind = .builtin, .builtin = "optimize"}

Or something

[languid canyon]

nice

[heavy sandal]

im more suggesting add that field with only one enum option so its future proof on that

[languid canyon]

so how did u do dependants

[fallen bobcat]

and it would autogenerate

[fallen bobcat]

I build a DAG

[languid canyon]

a what

[heavy sandal]

directed acyclic graph

[fallen bobcat]

yeah that

[heavy sandal]

the standard solution for anything dependency-graph related

[languid canyon]

yeah but whats the string format

[fallen bobcat]

ahhh

[heavy sandal]

oh

[fallen bobcat]

for now it's just a single option

.{
    .title = "My Project Configuration",
    .entries = .{
        .{
            .kind = .bool,
            .key = "some_bool",
            .label = "Some boolean value",
            .help = "lol",
            .default = .{ .bool = true },
        },
        .{
            .kind = .bool,
            .key = "some_dep_bool",
            .label = "Some dependent bool",
            .depends_on = "some_bool",
        },
        .{
            .kind = .choice,
            .key = "some_choice",
            .label = "The Beatles",
            .help = "This is an int",
            .default = .{ .choice = "Paul" },
            .options = .{ "Paul", "John", "George", "Ringo" },
        },

        .{
            .kind = .string,
            .key = "favorite song",
            .label = "Favorite song",
            .help = "This is a string",
            .default = .{ .string = "A Day In The Life" },
        },

        .{
            .kind = .int,
            .key = "some_int",
            .label = "Some integer",
            .help = "This is an int",
            .default = .{ .int = 42 },
            .range = .{ .min = 0, .max = 100 },
        },

        .{
            .kind = .menu,
            .label = "A submenu",
            .entries = .{
                .{
                    .kind = .bool,
                    .key = "some_other_bool",
                    .label = "Some other bool",
                    .help = "This is a bool",
                    .default = .{ .bool = true },
                },

                .{
                    .kind = .bool,
                    .key = "some_otherother_bool",
                    .label = "Some other other bool",
                    .help = "This is a bool",
                    .default = .{ .bool = true },
                },
            },
        },
    },
}

this is the demo file

[heavy sandal]

id at least make depends on an array of strings

[fallen bobcat]

you can PR that

[heavy sandal]

and just like only support and for now

[fallen bobcat]

[heavy sandal]

lol

[fallen bobcat]

also an import directive

so like

.{
  .kind = .import
  .import = "whatever.zon"
}

and it would be a submenu from another file

so you dont end up in nesting hell

[fallen bobcat]

you can do that

now

surprisingly very easy

[heavy sandal]

cool

[fallen bobcat]

    var engine = zonfig.Engine.init(allocator, config, &out.writer, .{
        .format = zonfig.formats.KV,
    }) 

for KV

[heavy sandal]

nice

[fallen bobcat]

right now you have ZON (default), JSON and key=value

[heavy sandal]

not using decl literals for the .KV?

[fallen bobcat]

it's not an enum

it's a function

[heavy sandal]

yeah

[fallen bobcat]

so if a user wants to provide their own they can

[heavy sandal]

oh

yeah ok function wont work there for decl literal then rip

[fallen bobcat]

yea

unless tagged union

but I think this is better anyway

[heavy sandal]

(you can do it with consts of the right type no matter the type now not just enums, as well as calling functions on the result type that return the type itself, but not for arbitrary functions)

[fallen bobcat]

huh?

[heavy sandal]

decl literals

like if my type has an fn init() Foo function i can do const foo: Foo = .init();

[fallen bobcat]

mhm I dont think search is really needed

is it

idk

[heavy sandal]

idk

can add later

if needed

[fallen bobcat]

I think right now its pretty much done

pretty happy

I think the workflow for integration will be having a submodule and @importing it from build.zig

[heavy sandal]

if you make it a dep using build.zig.zon then if you @import the dep name itll import its build.zig

[fallen bobcat]

yeah but the point is its a buildtime tool

so it should be imported in build.zig

and nowhere else

[heavy sandal]

yeah

i meant that in build.zig you can import a dep's build.zig by name

the build.zigs of your depedencies get added as modules to the build.zig's compilation

even if the dep doesnt export any modules or anything

thats how i use dimmer in my build.zig

my gut rec is still to make this a compiled tool that you can make a run step for tbh? idk

@serene briar you have a better feel for build system integration than i do, whats your thoughts on this? its a ziggy kconfig-like using zon schema and outputs to zon (or json or kv pairs) with the intent of being called to change configs occasionally

[fallen bobcat]

what i thought to do was have some function like zonfig.defaultBuildStep()

and youd just bind that

[heavy sandal]

to create a step?

[fallen bobcat]

or something yea

idk

I have no clue honestly

[heavy sandal]

mhm

yeah im not 100 here either thats why i pinged xq lmao

when in doubt contact someone smarter than you lmao

part of the reason im iffy is i know theres a goal to eventually sandbox/separate config and make phases of zig build and that might muck with this setup

[fallen bobcat]

https://asciinema.org/a/lRNL6ZjOwU7m4VME

here's what it looks like now

also btw the border being weird is an artifact of asciinema

[heavy sandal]

mhm

[fallen bobcat]

this is what it looks like

oh shit ideally it should be able to load from an existing config too mhm

[heavy sandal]

yeah

[fallen bobcat]

do you think its sane to force a config path

like kconfig kinda forces .config

idk

[heavy sandal]

just take the path as an arg?

[fallen bobcat]

the build step can handle it

probably

idk

[heavy sandal]

id just straight take a path and not worry about it

[fallen bobcat]

that would not work

the config file is generated

the path would be in .zig-cache or something

[heavy sandal]

you can pass local paths too

not in the cache

using b.path to generate a lazypath

or by just yknow

using string paths

like the old timers did

if you want to make it take input and output paths that also works

and then you can write back to the origin one using https://ziglang.org/documentation/master/std/#std.Build.Step.UpdateSourceFiles

[fallen bobcat]

I don't think you understand how it all fits with the build system

I don't want it to be a cli tool

It's a thing you run as a build step

[heavy sandal]

a make-time step or something at configure time?

[fallen bobcat]

That loads a config schema and a previously generated config file (if applicable) and generates a new one which can be imported then by the build system

[heavy sandal]

because those operate rather different

[fallen bobcat]

Well you need to run config before anything else

so like zig build config then zig build

[heavy sandal]

that doesnt answer the question, it could be a step that everything else depends on that still runs at make time

[fallen bobcat]

Also I just realized I could abuse the zig ast API and parse custom syntax for the depends on

[fallen bobcat]

What is make time

what is config time

It doesn't depend on any build artifact, no

[heavy sandal]

config time is the execution of your build.zig's build function

after that returns the build runner uses that and the cli args to pick out what step(s) to run

and then builds a dag out of that and runs them

[fallen bobcat]

ah

[heavy sandal]

which is make time

[fallen bobcat]

Then yeah it's configure time then I guess

I think

hmm ok I see how it's annoying

[heavy sandal]

in the future the plan from the zig devs is to sandbox the config time stuff so it cant violate the intended separation there and instead have config generate a graph that gets serialized in a standard format and run by a separate entity (which apparently is also giving speedups in early testing)

the idea is that anything that actually manipulates files should be a make time

the nasty thing about this zonfig thing is its fully interactive and also has a circular dependency on its own output

[fallen bobcat]

Well could it be something like: in configure time I check if a config is already generated (somehow? even though it could be in zig cache) and if it is I load it, if not then I don't do anything and error if the user tries to build without configuring beforehand

[fallen bobcat]

It does not

[heavy sandal]

how does it not? presumably you want the config output to be used as the previous config the next time you run it to change options

[fallen bobcat]

I just need a way to somehow express that nicely, I think I could just expose a library that's imported in the build script

[fallen bobcat]

Ah yeah you're right nvm

Well I guess I have no choice but to make the output path known

so either like .config.zon or specified in the schema I guess

[heavy sandal]

again you can work around this by having it take separate input and output path args and then using the updatesourcefiles step to copy the generated output back into the main location later during hte make phase

[fallen bobcat]

And then the build script can just import that and so can the code

[heavy sandal]

youve got to let the user build.zig specify the config path imo

even if its just a string

[fallen bobcat]

Well right now it just takes a writer

I guess it could take a reader and a writer

But I'm not sure what you're on about with the update source files thing

[heavy sandal]

the real problem here is the fundamental way youre meant to do custom build steps is as a cli tool

[fallen bobcat]

Not really?

[heavy sandal]

yeah?

[fallen bobcat]

A cli has the same problem lol

[heavy sandal]

with a cli you can use lazypath to pass the input and output files

and the build system will figure out the relative paths and all that

[fallen bobcat]

Ah you mean like that

An external CLI tool

Well I guess that could work too but idk how I can integrate that

My idea at first was to put it all in a single zonfig.zig file that you'd just copy and import in your build script

[heavy sandal]

const config_file = b.path("my/config.zon");
const run_zonfig = ...;
run_zonfig.addFileArg(config_file);
const conf_output = run_zonfig.addOutputFileArg("config.zon");

const update = b.addUpdateSourceFiles();
update.addCopyFileToSource(conf_output, "my/config.zon");

boom done

the thing that sucks is doing this you cant access the content of the file in your build.zig directly but also thatll probably break when they sandbox the build.zig config time stuff later anyway

[fallen bobcat]

You can't?

ugh that is kinda annoying

[heavy sandal]

no because its a lazy path

the word lazy is there for a reason

[fallen bobcat]

Can you make it not lazy

lol

[heavy sandal]

that would require running it at configure time

[fallen bobcat]

Idk I'll wait on the other zig person to respond cuz idk how to integrate it nicely

[heavy sandal]

and idk if thatll keep being possible

you can still provide the contents of the file to other build steps

so it can be imported or embedfile'd or whatever

[fallen bobcat]

yeah but ideally I'd want to be able to configure e.g optimization

[heavy sandal]

ill ask in the zulip what the expected level of sandboxing is for configure phase stuff

[fallen bobcat]

Nice, thanks

This should be up on github soon btw

I just wanna get the build integration sorted out first

Why I'm doing this btw is I want my only dependency to be zig eventually

That would be neat

[heavy sandal]

if youre doing it at configure time fwiw it wont be a step in the sense that zig means it

because a step cant do anything at configure time

[fallen bobcat]

I can have a step to launch it manually but at configure time I can check if the file exists or not and if it doesn't then I launch zonfig

so it can't build without first being configured

Idk if that'd work

[heavy sandal]

mhm

idk

[fallen bobcat]

Which is why zap ™ is gonna be a thing eventually too

[heavy sandal]

zap?

[fallen bobcat]

Bootloader

[heavy sandal]

ah

[fallen bobcat]

™

So you just need a zig install and you can build it all

(and ncurses )

[heavy sandal]

mhm lol

the other workaround option for build integration is to have a separate build.zig in a subfolder for just building your particular configuration of zonfig and then run that as your configure script

and keep them separated

ok turns out the sandboxing isnt as much an issue as i thought @fallen bobcat so youre still probably fine to run this at configure time

[heavy sandal]

this is where im at for imaginarium and its great. only unmanaged deps are zig and qemu

(or simics instead of qemu for testing fred things when i do that)

[fallen bobcat]

i kinda hate the new zig io tbh

or maybe it's the lack of docs

but it's just super confusing

[heavy sandal]

what about it?

[fallen bobcat]

lots of functions do the same thing but differently

also do you know how I can import deps in build.zig

can i even do that

cuz I figured out something

[heavy sandal]

if you have it in the build.zig.zon you can import the dep's build.zig using the dep name

[heavy sandal]

thats kinda just a fact of io tbh, theres lots of ways to do things

[fallen bobcat]

ok so when I do @import() in build.zig it imports the dep's build.zig

[heavy sandal]

yeah

[fallen bobcat]

?

great

[heavy sandal]

so if my dep is named .dimmer in the build.zig.zon then @import("dimmer") imports that dep's build.zig to give a full example

[fallen bobcat]

ok great now it works really well

I think the MVP is complete

also yeah i should use depend_on error step instead of printing the error but who cares

[heavy sandal]

eh thats fine tbh

[fallen bobcat]

i think this is fine to publish for now but in the future:

• arbitrary depends_on expressions
• import directive
• more advanced choice directives (so choice options can also depend on stuff etc)

[fallen bobcat]

very cool

[heavy sandal]

nice

[fallen bobcat]

Wonder if choice should be exposed as a string or as a bool

like if arch is set to amd64 should it output arch = "amd64" or amd64 = true

[heavy sandal]

should be an enum

imo

[fallen bobcat]

Yes ideally

But thats not easy to do with zon

[heavy sandal]

you can just use enum literal syntax

and the load will fail if the enum menber is missing

so arch = .amd64

[fallen bobcat]

Yes ok but how do I define arbitrary enums like that

[heavy sandal]

oh in the schema

[fallen bobcat]

no

In the output

[heavy sandal]

yeah you literally just use enum literal syntax

no need to type it in zon

the type the user loads it as will cover that

[fallen bobcat]

The user does not load it as a type

It just imports the zon file

directly

So it can't define enums etc

[heavy sandal]

i didnt think you could directly import zon without a result type?

[fallen bobcat]

You can

You can't parse zon without a result type tho

Ah you can do const whatever: Config = @import

[heavy sandal]

yeah

thats what i meant by result type

for import

[fallen bobcat]

Mhm

Perhaps it should generate a zig file instead of a zon file

And handle defining all the enums and the types and whatnot

[heavy sandal]

thats what the addOptions build thing does

so you can look into the source of that to see how they do it

[fallen bobcat]

Well I know how to do it

It's just adding a few more strings

[heavy sandal]

true

[fallen bobcat]

Oh shit deserializing it will be a pain tho

[heavy sandal]

you can generate two files maybe

[fallen bobcat]

hmm I guess I could generate both a zig and a zon file and depend only on the zon file for the loading

[heavy sandal]

a zig file that has the type and the basic zon

yeah same idea at the same time lmao

[fallen bobcat]

Yea

zon is kinda painful to parse when you don't know the keys btw

JSON is much easier

idk why they don't provide the same API for deserializing into a hashmap

[heavy sandal]

yeah itd be nice but idk

[fallen bobcat]

And the AST literally has no docs

So I kinda had to figure something out by messing around with it until it worked

The way I do it is probably very hacky

[heavy sandal]

the other thing is if you make generating the config type from the schema separate you can make that run only on updating the schema

oh i know most of how the tokenizer and parser stuff works btw if you got Qs on things

or did know, its been a bit

i havent gone deep enough to start seeing ArrayHashMap(void, void) yet lmao

[fallen bobcat]

Yeah but that is tricky

And a bunch of complexity for no reason

[heavy sandal]

fair

[fallen bobcat]

Also idk how zig options work but perhaps i could use that?

Can they have enums and stuff

[heavy sandal]

you need the type in the build script to be able to use it iirc

[fallen bobcat]

As in, it goes through the zon file and adds options for everything

ah

so it does have to be a separate thing

[heavy sandal]

its basically generating a zig module that has type definitions and global constants for each thing you pass in

[fallen bobcat]

Ok like I wanna do

[heavy sandal]

where it uses comptime to figure out the type definition to then output a string of it

[fallen bobcat]

I'm thinking enums might be the way

[heavy sandal]

yeah

[fallen bobcat]

But choices will have to get a bit more complex to define

Cuz I'll also have to have dependencies on choices and for choices eventually

[heavy sandal]

yeah

[fallen bobcat]

like, show this option if X is set

and show this option if X is chosen

[heavy sandal]

the biggest benefit for enums for choices is being able to do an int comparison instead of iterating for mem.eql every time you want to check one (and also getting switch statements for it)

[fallen bobcat]

in the schema it should still be strings tho

[heavy sandal]

yeah ofc

[fallen bobcat]

Should I publish it already you think or wait

[heavy sandal]

for generating the config type btw,

https://ziglang.org/documentation/master/std/#std.zig.fmtId

[heavy sandal]

i would personally, once you pick a license anyway

[fallen bobcat]

Nah

I'll generate it by hand

I did that for the zon lol

[heavy sandal]

i forgot to specify the function lol

its a string escaper for identifiers

fmtId

converts any string (like the choices here) into a valid zig identifier you could use for an enum member for instance

[fallen bobcat]

ahh yeah that's cool

[heavy sandal]

i wouldnt use the namespace for generating the zig fully that sounds annoying

but for that one escaping thing id use it

[fallen bobcat]

I'm asking because if you'd like to work on it then I'd release it otherwise I'd just keep adding stuff until I'm satisfied

[heavy sandal]

i wont be able to do any dev work until saturday at the earliest

so no need yet if thats your reasoning

[fallen bobcat]

tomorrow night I'll put it on github

That gives me some time to add a bit more stuff

[heavy sandal]

yeah nw take your time

im nowhere near needing something like this for my kernel lol

so rather it be good than fast

[fallen bobcat]

yeah but I am planning on using it for zag so I kinda wanna be done with it asap

And add stuff as I need

[heavy sandal]

yeah fair

when i do eventually want it ill def need making bool options depend on having picked a certain thing for a choice option

for the various extra feature support on x86 (x2apic, fred)

[fallen bobcat]

Well any option can depend on a dependable option

And a dependable option would be either a choice or a boolean

[heavy sandal]

cool

[fallen bobcat]

but I'm kinda scared the choice struct will get huge

you need key, label, enum_label, depends_on

[heavy sandal]

mhm

[fallen bobcat]

enum_label vs key because you might want to have an enum member be .amd64 but then you need a unique identifier to depend on like arch_amd64

[heavy sandal]

mhm yeah

[fallen bobcat]

I guess I could enforce label be the same as the enum label tho

label = string that's shown to the user, key = unique identifier for the option

[heavy sandal]

yeah

[fallen bobcat]

I just realized I can't just import the zon

Because then I'd need to do something like config.options.whatever instead of config.whatever which is annoying

if only zig had some keyword to export all members of a namespace...

[heavy sandal]

why not?

like actually, can't you make the options top level in the zon?

[fallen bobcat]

You'd import the zig file not the zon

[heavy sandal]

ah right

either that or youd do something like

const Config = @import("ConfigType");
const config: Config = @import("config");

but that gets repetitive (source: am doing right now for my target info options lmao)

[fallen bobcat]

I'm thinking I could not rely on the zon and just write definitions

The zon would be used only by the tool and by the build system

[heavy sandal]

yeah thats also a good option

zon as storage not as final imported version

[fallen bobcat]

Either that or const config = @import(...).config

[fallen bobcat]

Yeah

[heavy sandal]

that also works lol

[heavy sandal]

i think this is what i like the most personally

[heavy sandal]

followed by this

[fallen bobcat]

Yes so do i

It's just a bit tricky to implement

[heavy sandal]

yeah

[heavy sandal]

@fallen bobcat just realised a reason i probably wont be able to use zonfig on my project 😭

unless ncruses is a thing on windows

which i dont think it is

[fallen bobcat]

it is

[heavy sandal]

ok cool then itll be fine

[fallen bobcat]

ncurses has been ported to everything

[heavy sandal]

would i need to build or install it myself then?

[fallen bobcat]

probably install it

unless you use mingw or something

in which case its probably installed already

[heavy sandal]

i use zig's normal toolchain which is llvm on windows

i hate bringing in external deps like this ugh

[fallen bobcat]

mhm

[fallen bobcat]

its either this or depend on some zig library that idk if its gonna stay

I chose ncurses because it was the most portable

also I figured out cooler expressions for depends_on

it works now

.depends_on = .{
                .not = .{
                    .any = .{
                        .{ .key = "some_bool" },
                        .{ .key = "some_other_bool" },
                    },
                },
            },

this for example

wow great formatting 😭

[heavy sandal]

lol

yeah that works

ill have to look into ncurses stuff

cause im not sure theres even a real package option for it for windows (at least not that ive found so far)

[heavy sandal]

i mean option 3 is to build ncurses from source using zig build as your dep

[fallen bobcat]

pdcurses maybe

[heavy sandal]

yeah i just saw that when looking

[fallen bobcat]

the real good option is to do it all from scratch

[heavy sandal]

idk if its got everything you need though but it is at least a thing for windows

[heavy sandal]

true

theres also stuff like https://github.com/dankamongmen/notcurses which has windows support

[fallen bobcat]

problem with that kinda library is what I wanted was that the user does not have to install anything

[heavy sandal]

yeah

which is true on nix-likes ig

for ncurses

[fallen bobcat]

yeah

now this works

this is cool

[heavy sandal]

nice

[languid canyon]

bro is truly a big brain

instead of doing .shit or .piss u did beatles and options that make sense

even for a demo

lol

[fallen bobcat]

im too much of a fanboy what can i say

the fruit demo is pretty fun too

[fallen bobcat]

that one

minor nitpick

but is it fine if the choice is shown as "paul" instead of "Paul"

.{ .value = "paul", .label = "Paul" },

i.e the label is ignored when showed in the menu view

but when you actually press on the choice menu it shows the labels

[heavy sandal]

imo its fine

[fallen bobcat]

because itd be like much trickier with how the code works now

ok now I need to output zig 😭

[heavy sandal]

gl

[fallen bobcat]

I think it'll just be another subcommand

[fallen bobcat]

pub const __some_choice_enum = enum {
    paul,
    John,
    George,
    Ringo,
};

pub const some_choice: __some_choice_enum = .paul;
pub const show_quiet_beatles = true;

is this fine you think

[heavy sandal]

yup

[fallen bobcat]

the __some_choice_enum is ugly but you wont use it anyway

[heavy sandal]

you can even inline it if you want (put the enum{} bit after the colon on the decl) but imo thats way messier

[fallen bobcat]

ah

mhm

maybe thats better actually

[heavy sandal]

as long as you dont expect to be sharing choice sets for multiple options and even then idk if for this case itd make sense to share em lol

[fallen bobcat]

no it wouldnt make sense

[heavy sandal]

yeah

[fallen bobcat]

ok great that makes it much simpler

pub const some_choice: enum {paul,John,George,Ringo} = .paul;
pub const show_quiet_beatles = true;

generates this now

[fallen bobcat]

ok i need your help

am i stupid or what

[heavy sandal]

ok whats the question

[fallen bobcat]

pub fn createConfigModule(b: *std.Build, dep: *std.Build.Dependency, schema_path: []const u8, config_path: []const u8) *std.Build.Module {
    const run = b.addRunArtifact(dep.artifact("zonfig"));
    run.addArgs(&.{ "--schema", schema_path, "--config", config_path, "--output-zig" });
    run.addFileInput(b.path(schema_path));
    run.addFileInput(b.path(config_path));
    const output = run.captureStdOut(.{});
    return b.createModule(.{ .root_source_file = output });
}

    const exe = b.addExecutable(.{
        .name = "example",
        .root_module = b.createModule(.{
            .root_source_file = b.path("main.zig"),
            .target = target,
            .optimize = optimize,
        }),
    });

    const config_mod = zonfig.createConfigModule(b, zonfig_dep, "config.zon", ".config.zon");
    exe.root_module.addImport("config", config_mod);

main.zig:2:24: error: no module named 'config' available within module 'root' const config = @import("config");

wtf???

it's present in the command line too

[heavy sandal]

uh ok thats weird

check the file content ig?

[fallen bobcat]

it's fine

[heavy sandal]

oh its probably because of the extension maybe?

[fallen bobcat]

maybe it doesnt like not being suffixed

yea

that's insane

bruh that's it

wtf

[heavy sandal]

that is strange

well you can set the basename in the options param to captureStdOut

[fallen bobcat]

yea I did that and it works now

great

[heavy sandal]

its probably so it can tell if youre importing zon or zig (or other possible future things ig) since the parser has different rules for the two

[fallen bobcat]

yeah but it should return a better error message lol

[heavy sandal]

that is def true lol

[fallen bobcat]

ok yay now it's way better

[heavy sandal]

pog

[fallen bobcat]

I think it's ready for release

[heavy sandal]

nice

[fallen bobcat]

https://github.com/rdmsr/zonfig

we are live

now to convert zag to it

[heavy sandal]

hell yeah

too bad i cant use it because windows

but it looks great so far at a brief glance

[fallen bobcat]

im sure you can find a way 😭

[heavy sandal]

maybe?

cant use wsl because zig is currently broken on wsl

[fallen bobcat]

yes but like maybe in the build script you could check if on windows and if so link pdcurses instead

can you try that

I dont have windows

[heavy sandal]

i may try that this weekend lol

pdcurses also doesnt come with windows so id need to figure out installing it too

and also isnt in any package manager i could find

also, idk what the system library "menu" is either lol

somehow i doubt i have that either

[fallen bobcat]

it's from ncurses too

[heavy sandal]

ah ok

[fallen bobcat]

idk typical windows L

not my problem

[heavy sandal]

lmao

[fallen bobcat]

like unironically use mingw

if you find a way to make it more portable be my guest

[heavy sandal]

ive tried mingw before

i find its usually more broken than the default options when i use it with my current setup

[fallen bobcat]

https://github.com/meszmate/zigzag

This looks good

no deps either

[heavy sandal]

oooo

[fallen bobcat]

Nice name too

Kinda don't want to rewrite tho 😭

[heavy sandal]

looks like it isnt updated for zig 0.16.0 yet is the only downside here

[fallen bobcat]

Ah right

Maybe I should start pinning a version honestly

[heavy sandal]

mhm

[fallen bobcat]

I think ncurses is fine for now until they update to 0.16

Cuz this looks really neat and no deps apart from zig

[heavy sandal]

yeah

[fallen bobcat]

the kernel now builds with zonfig 😎

and it works with LSP and stuff

pretty cool

[heavy sandal]

hell yeah

[languid canyon]

Does it have to be like one config, or can you have "source" etc

[fallen bobcat]

I haven't implemented that yet but that is planned yes

You'd have an import entry that'd import a submenu

https://github.com/rdmsr/zag/blob/master/config.zig.zon

This is the config now

Lots of nesting

[fallen bobcat]

oh man the new config stuff is so cool

also a zig update finally broke my code

they removed cImport

[heavy sandal]

yeah

thats part of why im pinned on 0.16.0 for now

[fallen bobcat]

I've implemented the bitmap thing in the allocator, seems to have improved performance a bit

These are my very accurate benchmarks, on the left is before on the right is after

this is for small objects

the performance difference seems to be the same for large objects too

also obviously less memory overhead (which is probably why it's faster, it needs less memory)

[heavy sandal]

nice

[fallen bobcat]

i am having a very weird bug

somehow when I'm allocating like 1 million large objects something is overwriting the hhdm mappings or something

inb4 its a bug in my fancy pmm

[fallen bobcat]

OMFG

i fixed the bug

it was so fucking stupid

omg

I lost so much time

[languid canyon]

What's the bug

[fallen bobcat]

also new perf comparison

not that accurate but its faster

[fallen bobcat]

pub inline fn pfn_to_page(pfn: Pfn) usize {
    return @intCast(pfn << 12);
}

this

pfn is a u32

cookie if you find the bug

[west talon]

You can have more than 4 billion pages in a system?

[fallen bobcat]

no that is known

i dont have 16 tb of physical memory

[west talon]

I don’t think you do 32bit stuff right?

[fallen bobcat]

no

[west talon]

So the return value isn’t an issue with PAE

[fallen bobcat]

nope

I will reveal the answer later if no one gets it :^)

[west talon]

Is it to do with the cast? Where you shift the pfn before casting properly?

[fallen bobcat]

perhaps

[west talon]

Otherwise I’m out of ideas

Oo hm

[fallen bobcat]

this is the fix

its so fucking stupid

which meant pages higher than 4gib were getting interpreted as pages much lower and they were corrupting the page tables

for fun I tried to give the problem to claude and it kept going back in circles and didnt end up finding anything

never saw it do that

it was tweaking

Anyway the bitmap stuff was a pretty good improvement

The allocator got leaner and faster

[languid canyon]

What do u use the bitmap for?

[fallen bobcat]

@languid canyon

[heavy sandal]

one of these days im gonna read through your slab and try to understand it

ive got a slab rn and im still barely understanding all of this lmao

[fallen bobcat]

It's gonna get much bigger soon

Because I plan on doing SMR through it

[heavy sandal]

smr?

[fallen bobcat]

Safe memory reclamation

[heavy sandal]

(i can always look at older commits if youre actually committing often enough if that makes it too complex to understand also)

[fallen bobcat]

Well as soon as this is pushed its good enough

I did find a few bugs here and there

[heavy sandal]

cool

[fallen bobcat]

But this overall simplifies the thing I think

And it's faster and has less overhead

[heavy sandal]

mhm