Zag

[languid canyon]

cool

[heavy sandal]

so including the file is still fine as long as you dont call any code that needs full kernel stuff

[fallen bobcat]

yeah ok the way I do it is a bit annoying since in_x2apic_mode is a cpu local

mhm

huh well I guess I don't really need that bool and can just read from the MSR everytime

considering this is just loader code idrc about performance

ah shit I guess I need to sleep as well for the AP startup

yeah no fuck that

[heavy sandal]

ap startup is kinda chronically fucked tbh

[fallen bobcat]

in the kernel I have a bunch of infrastructure I can reuse, like sleep and timers

[heavy sandal]

at least on x86

[fallen bobcat]

which i cant really do in the loader

[languid canyon]

yeah lol i def wouldnt do ap startup in the loader

[fallen bobcat]

it's a shame because it would be really convenient to do so

[heavy sandal]

eh

i still dont see much benefit if youre just going to park it immediately to resume later

esp if you want hotplug

[fallen bobcat]

well, it's nice to start the kernel in a convenient state with all the APs already set up etc

[fallen bobcat]

ok cool now there's dynamic worker pools

how it works is that if it notices there's a backlog, it signals a pool manager thread which spawns up to 4 (configurable) additional worker threads per pool

those worker threads then exit after 5 seconds of having no work

[fallen bobcat]

and so it begins...

[fallen bobcat]

yeah ok it actually wasnt that bad

granted I have like the most basic version possible atm

[fallen bobcat]

https://github.com/rdmsr/zag/commit/fd8676bec559203456c2fb87de8cbf2ce9ca2cdf

ok this is insanely cool

[inner python]

Is this an implementation of the thing you showed me in FreeBSD where they’d batch objects together in the same write sequence counter?

[fallen bobcat]

yeah

but it works differently because it relies on depot size and not magazine size

it uses the policies from XNU

https://github.com/rdmsr/zag/blob/master/src/mm/zone.zig#L79-L101

[fallen bobcat]

I got the clanker to write stress tests and I actually found bugs

there was a lock inversion between the scheduler and turnstiles where I relied on the thread lock to wait for a thread to switch off its stack in the scheduler while holding the current thread lock

so turnstile tried acquiring the thread locks in another order and it deadlocked

[fallen bobcat]

lol I found a very funny race in the zone allocator

[heavy sandal]

oh?

[fallen bobcat]

https://github.com/rdmsr/zag/blob/master/src/mm/zone.zig#L1125-L1129

I release the lock before calling this function as to avoid holding locks across the potentially blocking slab creation

yet self.color is racily updated

the stress test showed this and it lead to weird use-after-frees I couldnt figure out until I realized somehow an object was 8 bytes after the end of the slab

also the slab base page was too far

like, base = page+72 implies color=32, but it also had capacity=63 which is only valid for color <= 24

[heavy sandal]

mhm

[fallen bobcat]

I figured out something extremely cool that hasn't ever been done before I'm pretty sure

I'll update you all if it ends up working

[fallen bobcat]

https://klipy.com/gifs/gohan-its-so-peak

i have implemented preemptible SMR with priority inheritance to all stalled readers

[fallen bobcat]

https://github.com/rdmsr/zag/blob/master/src/kern/smr.zig this is really cool and i dont think any OS does this

maybe windows does it if it has preempt rcu

the preempt stuff is stolen from XNU but the PI is novel afaik, though tbf it's not really useful outside of that particular case since I only do single hop boosting (since blocking is not allowed in SMR sections)

[inner python]

is this so that low priority tasks in an rcu read section dont prevent some resources being reclaimed?

[heavy sandal]

ill be honest that im not sure (and @modern sluice might know more), but i dont think windows has smr/rcu in general as a thing. maybe bespoke for certain things but afaik not in general

[fallen bobcat]

windows does have RCU

[heavy sandal]

TIL then

[fallen bobcat]

I'm talking about modern windows

idk if reactOS people know about it

I only know of this because of a NT guy

[heavy sandal]

my source is the windows inside book which is for win10 but that also probably doesnt go too deep into details like that

[fallen bobcat]

yeah, if a high priority thread is waiting on a sequence to pass itll give its priority to the readers

[fallen bobcat]

this same person also said he had wished that a freebsd-style thing wouldve been implemented instead

im saying windows might have it because autoboost handles multiple readers, while turnstiles traditionally do not

I figured out how to do it for this specific case with turnstiles but it's not a generic thing u can apply

[heavy sandal]

mhm

[inner python]

why not use IBR or hazard eras if you’re trying to put an upper bound on the amount of memory that could be prevented from being reclaimed

[fallen bobcat]

I'm not trying to put an upper bound

im trying to keep preemption enabled in smr sections but still have decent latency

[inner python]

I’d imagine that using PI for something like this would end up with the side effect of a task getting its priority boosted needlessly when high priority tasks decide to retire an object

Which would probably break down if some of these tasks retiring objects are really high priority

[fallen bobcat]

wdym

it's not needless when a thread is blocked on it

[inner python]

I see

That makes more sense if it’s to handle tasks that are blocked until some objects can be retired