(fwiw if youre using the Type{ ... } syntax that is extremely non-idiomatic also, again because it results in shit like this)
#Zag
heavy sandal
fallen bobcat
makes sense
heavy sandal
(im not sure whats left that they havent been able to kill it yet but they sure have been trying lol)
(i think mainly its still needed for anytype parameters)
fallen bobcat
ok plan for tomorrow is to finish rewriting the slab allocator
then finally maybe I'll be able to do SMP
heavy sandal
unrelated side note i just realised i think @as could actually be implemented as a normal function:
pub fn as(comptime T: type, val: T) T { return val; }
imo as should be a keyword
@as(usize, @intCast(whatever)) is ugly af
@intCast(whatever) as usize
heavy sandal
i mean
the readable option is instead of @as(usize, @intCast(whatever)) you do
const whatever2: usize = @intCast(whatever);
// do something with whatever2 here but not inlined
less keywords is generally better anyway just because it puts less constraints on identifiers
heavy sandal
how would that work?
fallen bobcat
idk
heavy sandal
for the common uses anyway
heavy sandal
lmao
heavy sandal
as a name
secret technique: @"align" is a valid identifier if you need to use that exact name
fallen bobcat
yeah I know but then it's ugly
heavy sandal
yeah
mostly that syntax is useful for when youre parsing things based on the field name and want dashes instead of underscores tbh
since the thing after the @ uses the same parsing rules as a string literal and can have anything you can do with those
heavy sandal
btw for freelist_count, consider: @bitSizeOf
fallen bobcat
@heavy sandal is there a way to get compile errors even if I dont call a function
is that with like std.testing.refAllDecls or something
cuz its kinda annoying
heavy sandal
<@121733282676211714> is there a way to get compile errors even if I dont call a...
refalldecls should work, or just do a comptime { _ = somefunc; } which is basically what refalldecls does (except ref all decls is for all decls of the passed container)
the ref all decls should generally be in a comptime block like that afaik fwiw
fallen bobcat
refalldecls should work, or just do a `comptime { _ = somefunc; }` which is basi...
I get the lazy thing but it's annoying imo
heavy sandal
I get the lazy thing but it's annoying imo
it is sometimes but its also how zig does per-platform code so its kinda necessary
fallen bobcat
it is sometimes but its also how zig does per-platform code so its kinda necessa...
wdym
heavy sandal
wdym
even with functions swapped out using comptime, if e.g. fileReadPositionalWindows gets checked on linux its going to run into issues with windows shit not existing
or if an arm specific function with inline asm gets checked on x86 itll throw a billion compiler errors
fallen bobcat
var seg_zone: zone.TypedZone(Segment) = undefined;
// init
seg_zone.init("seg zone", .{});
// alloc
var segment = seg_zone.alloc();
// free
seg_zone.free(ptr);
this is neat
heavy sandal
👀
heavy sandal
so like std.heap.MemoryPool?
fallen bobcat
I could probably also grab methods from the type but that's not really idiomatic
fallen bobcat
so like std.heap.MemoryPool?
idk its a slab allocator
heavy sandal
std.heap.MemoryPool is more like a typed SLOB than a proper slab
fallen bobcat
idk what that is
heavy sandal
its literally a singly linked list of nodes that get allocated from a parent generic allocator where each node is min(sizeof(T), sizeof(usize)) bytes
slob = simple list of blocks
the most stupid and basic slab
fallen bobcat
idk this is a generic slab allocator but the typed wrapper over it just does the casts and stuff
heavy sandal
(as opposed to actually grabbing pages and breaking them up into multiple objects)
fallen bobcat
pub fn TypedZone(comptime T: type) type {
return struct {
zone: Zone,
const Self = @This();
pub const InitOptions = struct {
ctor: ?*const fn (*T) void = null,
dtor: ?*const fn (*T) void = null,
};
pub fn init(self: *@This(), name: []const u8, options: InitOptions) void {
self.zone.init(name, @sizeOf(T), .{
.alignment = @alignOf(T),
.ctor = @ptrCast(options.ctor),
.dtor = @ptrCast(options.dtor),
});
}
pub fn alloc(self: *Self) ?*T {
return @ptrCast(@alignCast(self.zone.alloc()));
}
pub fn free(self: *Self, obj: *T) void {
self.zone.free(obj);
}
};
}
it's literally just that
fallen bobcat
pffft returning errors
heavy sandal
(also zig usually calls the functions create and destroy for single objects and alloc/free for slices)
fallen bobcat
well it's not create
it doesnt create from a zone
it allocates from a zone
idk
create is good too
heavy sandal
yeah
im indifferent, though i use create personally for consistency with library code
heavy sandal
ive really got to refactor my slab out to be usable outside of the main list of slabs-per-size sometime tbh
fallen bobcat
why `?*T` instead of `error{OutOfMemory}!*T`?
also the idea behind this is that the only way it could ever fail is if it OOMs so
heavy sandal
I think the generic zone code can use alloc
yeah this is what i would do, generic zone is allocating a block of memory and typed is creating an object to return
fallen bobcat
but maybe it makes more sense to return an error
heavy sandal
the error is nicer because you can use try and propgate it
which gives you an error return trace when it hits the top and means i dont need to do unwinding myself
heavy sandal
(the trace is accessible anywhere with @errorReturnTrace which returns an optional trace - it returns null in modes with safety turned off like releasefast and releasesmall to save time appending and space storing it respectively)
fallen bobcat
i will look into proper backtraces soon
it's kinda annoying to have "KERNEL PANIC: division by zero" (from zig) and idk where it happens on amd64
heavy sandal
my start is literally main() catch |err| (print @errorReturnTrace here)
heavy sandal
it's kinda annoying to have "KERNEL PANIC: division by zero" (from zig) and idk ...
well if you use the limine file thing to get the kernel you can use std to do all the dwarf stuff for you and something like https://codeberg.org/Khitiara/imaginarium/src/branch/rewrite/src/krnl/debug/SelfInfo.zig (im sure youll do your own version but this shows off how it works) to override the std stuff and get it with std apis
fallen bobcat
should I use allocator in my kernel or not
pastel citrus
doesn't the dwarf stuff need a bunch of stuff you had on your own fork of zig?
heavy sandal
all you need to define is @import("root").debug as a container having SelfInfo, printLineFromFile (which noops if it returns an error), and getDebugInfoAllocator to return an allocator for the debug info stuff to use
heavy sandal
doesn't the dwarf stuff need a bunch of stuff you had on your own fork of zig?
not anymore
mlugg goated
pastel citrus
based
fallen bobcat
also I realized since LSP is useless anyway so I switched back to emacs
pastel citrus
so you can literally just give it a fixed reader from the limine kernel file thing and it works?
heavy sandal
so you can literally just give it a fixed reader from the limine kernel file thi...
file i linked plus ElfFile in the same dir and https://codeberg.org/Khitiara/imaginarium/src/branch/rewrite/src/krnl/debug.zig and thats all you need
heavy sandal
its literally just the SelfInfo type in std.debug is overridable now and a bunch of the internals it was using were made pub and refactored to not suck ass
they even threw out relying on posix for the thread/register context type
so std now has functions to capture register context
fallen bobcat
should I use allocator in my kernel or not
can a zigger (wow that sounds bad, idk what the name of a zig programmer is) answer this?
pastel citrus
ziguana
heavy sandal
can a zigger (wow that sounds bad, idk what the name of a zig programmer is) ans...
ziguana and the answer is depends. object cache stuff or stuff with fixed size shouldnt be an allocator
heavy sandal
if you have a thing that allows alloc(size) then make an allocator for it imo
fallen bobcat
I'm not an iguana
heavy sandal
if only so that you can pass the allocator to std or other library code
fallen bobcat
zigger is pretty funny tho
heavy sandal
zigger is pretty funny tho
one letter off, two keys on a qwerty board, from a very bad word is the issue there
fallen bobcat
if you have a thing that allows alloc(size) then make an allocator for it imo
yeah I will but I don't think I will pass around an allocator
heavy sandal
yeah I will but I don't think I will pass around an allocator
yeah i make one and put it in a global var
the thing with having an allocator is then you can use the std arena with it and stuff like that
fallen bobcat
passing around an allocator is annoying when I can just do malloc()
fallen bobcat
the thing with having an allocator is then you can use the std arena with it and...
I'm thinking those pieces of code that require a generic allocator can take one
but like 95% of the code wont
heavy sandal
I'm thinking those pieces of code that require a generic allocator can take one
yeah most programs will have one global allocator instance, passing is more for libraries or in cases where you want arenas or whatever instead of the global
fallen bobcat
one letter off, two keys on a qwerty board, from a very bad word is the issue th...
it's not two keys it's like 4
heavy sandal
it's not two keys it's like 4
right im stupid
fallen bobcat
I did know the zig mascot was an iguana but I didnt know programmers themselves were nicknamed ziguanas
like rustaceans
heavy sandal
yeah most programs will have one global allocator instance, passing is more for ...
on this note as of recently the zig ArenaAllocator is now thread safe and lock free in itself (obv if the backing allocator isnt thats its problem)
fallen bobcat
huh cool
fallen bobcat
I did know the zig mascot was an iguana but I didnt know programmers themselves ...
I will add a cute iguana ascii art at some point
or logo or something
heavy sandal
someone in the community got tired of arena not being thread safe and the locking threadsafeallocator existing and was like fuck this im going to make arena thread safe lock free and pr it
and then it took a month to get all the memory order edge cases worked out for all the arches zig supports lmao
fallen bobcat
honestly i could see myself getting too pissed off with zls and fix it myself
pastel citrus
i wish the smp allocator was a bit more flexible
it's so simple it'd be so nice to be able to just use it in a freestanding environment
heavy sandal
yeah
if my goal wasnt learning (and therefore wanting to do stuff like allocators myself) id feel the same more strongly
fallen bobcat
My opinion is that if you rely on the std for something like that you're most likely doing something wrong
heavy sandal
smpallocator is the default basically-malloc for usermode non-debug programs
heavy sandal
it's so simple it'd be so nice to be able to just use it in a freestanding envir...
it would be a bit hard because it uses threadlocal tbf
pastel citrus
i know, but imagine if it didn't :^)
but yeah sucks to suck
i just had clanker reimplement it
Lol
heavy sandal
im in this to learn i ignored it and wrote my own slub instead
pastel citrus
i've written them so many times it's just boring so i wanted to use smp_allocator instead
heavy sandal
mhm
(for those unaware, slub is the third and current-for-non-numa version/implementation of a slab allocator in linux)
fallen bobcat
My opinion is that if you rely on the std for something like that you're most li...
I don't want to blame someone but be if I run into issues
heavy sandal
i think it depends on context too
fallen bobcat
(for those unaware, slub is the third and current-for-non-numa version/implement...
The Linux allocators have such bad names lol
heavy sandal
we here are doing weird freestanding bullshit, the default allocators in std arent going to be ideal no matter how much you cludge them to cooperate
fallen bobcat
Do you store the metadata in the struct page like Linux?
heavy sandal
but for a usermode program id use the std stuff in a heartbeat
fallen bobcat
we here are doing weird freestanding bullshit, the default allocators in std are...
Yes obviously in userspace I wouldn't mind
heavy sandal
Do you store the metadata in the struct page like Linux?
yup
pastel citrus
ugh the stack trace stuff needs master
fallen bobcat
why aren't you on master
heavy sandal
ugh the stack trace stuff needs master
yeah thats the one thing, the pr was merged a bit after 0.15.2
pastel citrus
zvm doesn't have a way to pick a specific version if you want "master"
so that sucks
heavy sandal
you cant pick a hash for master
pastel citrus
^
fallen bobcat
Why would you need that
pastel citrus
reproducibility?
fallen bobcat
I just update it weekly and hope it compiles
heavy sandal
(its because the json on the ziglang.org download page is just "master" and a bunch of tagged versions)
pastel citrus
i want to know my code compiles
and i want it to compile in a weeks time
or months time
fallen bobcat
idk imma keep updating to master as long as this project is active
heavy sandal
reproducibility is the reason yeah
pastel citrus
kind of sucks when someone tries to build your project and they run into some issue that popped up because zig changed something
heavy sandal
there was a patch a while back in std.os.uefi that made my bootloader not compile for two weeks until someone caught it
and that was annoying
because i couldnt easily roll back without going all the way to a tag
and didnt want to have to patch std manually
fallen bobcat
Maybe when zig 1.0 releases I'll pin the version idk
fallen bobcat
there was a patch a while back in std.os.uefi that made my bootloader not compil...
what did it change
heavy sandal
what did it change
it was technically a change to switch case resolution in the compiler to fix an oversight, but std.os.uefi was unknowingly only compiling because of the oversight and noone ever tested it on uefi
heavy sandal
bootloader stuff is some of the most fun ive had in osdev actually
fallen bobcat
It shouldn't take too long, probably like a week
heavy sandal
mostly because it forces me out of hyperfixating on always getting the abstractions just right and the api just right and instead just make a fuckin thing already
fallen bobcat
It shouldn't take too long, probably like a week
And I write the kernel ensuring I rely on the bootloader as little as possible so it should be fine
heavy sandal
id offer to make tabula (my bootloader) a shared thing but a) its 99% done and b) its super tied in with my kernel's memory layout and wouldnt generalize that well
(you are as always welcome to reference how i do things if theres something you need help with though, ive gone pretty deep into uefi bullshit like setvirtualaddressmap with it)
fallen bobcat
Well the point of writing my own is to not reuse a bootloader lol
heavy sandal
also that yeah
fallen bobcat
But like I could probably rewrite like 90% of my brutal code
I probably won't put it in another repo
heavy sandal
but yeah i highly recommend making a bootloader its fun problem solving without the need to make it perfect its great
fallen bobcat
What does setvirtualaddressmap do
heavy sandal
What does setvirtualaddressmap do
relocates runtime services so they dont try to access the default uefi identity map for memory
fallen bobcat
Ah ok that's fine I don't need runtime services
heavy sandal
you might
theres some platforms where its the only way to get RTC
depends on your arches tho
heavy sandal
(specifically some efi firmware on arm and i think maybe also riscv will remove the RTC from all detectable firmware stuff like acpi or devicetree so you have to hardcode or use runtime services for it)
fallen bobcat
wtf
heavy sandal
ive heard of it on riscv and know it happens on arm because thats why i needed setvirtualaddressmap
fallen bobcat
who the fuck thought that was a good idea
fallen bobcat
the edk2 firmware does it unconditionally on arm
It does what
heavy sandal
It does what
removes the RTC from acpi tables and the devicetree
so you either have to hardcode for your board, not use uefi which theres not really many great alternatives there, or use efi runtime services for RTC access
fallen bobcat
Couldn't you use runtime services to do the shutdown too instead of acpi
heavy sandal
yep
heavy sandal
since my bootloader only supports uefi anyway i use runtime services for shutdown and rtc unconditionally
fallen bobcat
(ok I know you still want AML)
heavy sandal
you cant do sleep with uefi runtime services though
only reboot and shutdown
(and a "platform specific" option that has no generic meaning and would need to be determined for the board again)
fallen bobcat
Well I think the RTC stuff could be per board
Like on some boards it relies on UEFI on others it doesn't
heavy sandal
oh and uefi runtime services are also needed to set up uefi boot options if you want to make your thing be its own installer
if youre using an hhdm setvirtualaddressmap is super simple fwiw
i think the only reason limine doesnt do it is because it can only be called once so limine doing it would break anyone else wanting to do it themself
fallen bobcat
One thing I wanna do soon too is add a little graphical console
heavy sandal
(tbh as long as you can guarantee that the place you pick to map each thing to is a place youll be happy with it being at forever its easy, its just that a generic bootloader cant really do that)
fallen bobcat
Cuz I wanna test on real hardware
heavy sandal
you gonna flanterm or you gonna do it yourself?
fallen bobcat
My own
heavy sandal
hell yeah and same
fallen bobcat
Wait I could use ghostty 
They have a freestanding library
That would be pretty funny but nah
heavy sandal
mhm
graphical terminal to me is very much a "this doesnt look painful enough to override my distaste for having external dependencies"
(acpi on the other hand def looks painful enough to override that and hence im using uacpi lmao)
fallen bobcat
graphical terminal to me is very much a "this doesnt look painful enough to over...
well I also think that it's a thing I can easily push to userspace
So I don't mind if the kernel one is shitty
heavy sandal
mhm fair
fallen bobcat
shitty
heavy sandal
kernel one only needs to last until usermode takes over the framebuffer
fallen bobcat
Yeah pretty much
heavy sandal
shi`tty`
ok this is actually hilarious 😂
legit if i could use it easily with uefi i would be using vga text mode tbh
fallen bobcat
nah I want pretty colors
sec I need to find a screenshot of my old one it was awesome
heavy sandal
if i had the room the money and the time to restore one what id actually do is run my os on a teletypewriter
actually put the TeleTYpe in TTY
fallen bobcat
heavy sandal
nice
fallen bobcat
Tempted to reuse that code but maybe it was too much eye candy lol
shadows and stuff
heavy sandal
oh god i could get a used teletypewriter for $75 someone stop me
fallen bobcat
I will reuse the font because I love that font
heavy sandal
oh nvm this is just the case of the machine
heavy sandal
I will reuse the font because I love that font
what font is it?
heavy sandal
ok an actual teletypewriter is $1000 that makes more sense
heavy sandal
Sun gallant it's called
ill have to look into it
ive still not decided on a font for mine
heavy sandal
not a fan of spleen personally
gallant is nice tho
i think ill probably use peep as my font tbh
its MIT licensed though the original source link for it is dead now
fallen bobcat
added a small hardening measure
this isnt that useful since it's only checked on slab layer allocations and not on magazines tho
fallen bobcat
this isnt that useful since it's only checked on slab layer allocations and not ...
nvm now it is
languid canyon
the zag kernel shell 
languid canyon
eh
fallen bobcat
languid canyon
yes
languid canyon
how are u going to render text
languid canyon
like own flanterm?
fallen bobcat
yes
languid canyon
ig
fallen bobcat
but it'll be really simple for now
languid canyon
sure
languid canyon
i just slapped a quick downstream flanterm for that
fallen bobcat
I'm too lazy to use flanterm
languid canyon
oh ok
pastel citrus
you can just do zig fetch --save=flanterm git+https://codeberg.org/Mintsuki/Flanterm.git and thenzig const flanterm = b.dependency("flanterm", .{}); kernel_module.addIncludePath(flanterm.path("src")); kernel_module.addIncludePath(flanterm.path("src/flanterm_backends")); kernel_module.addCSourceFiles(.{ .root = flanterm.path("src"), .files = &.{ "flanterm.c", "flanterm_backends/fb.c" }, .flags = &.{"-DFLANTERM_FB_DISABLE_BUMP_ALLOC"}, });
so simple
languid canyon
DFLANTERM_FB_DISABLE_BUMP_ALLOC does that thing make it depend on malloc or smth
pastel citrus
nah, flanterm has an internal bump allocator for some initial allocations
but if you have a memory allocator you can just use that instead
pastel citrus
yeah, the two function pointers you pass in to init now can't be null
the alloc/free
pastel citrus
I dont wanna use flanterm
write your own language atp (half joke)
languid canyon
bro has the nih disease
languid canyon
hybrid thor/serenityos abi compat + grub + toaru libc + exokernel
languid canyon
Where did u get that font
fallen bobcat
i drew it
no I got it from /usr/share/kbd/consolefonts/sun12x22.psfu.gz
hey thats pretty cool
mhm which
one font can fit more text but another has way more aura
heavy sandal
left for sure
fallen bobcat
holy
pastel citrus
i prefer the plain white background with no decorations or borders tbh
looks cleaner
fallen bobcat
i prefer the plain white background with no decorations or borders tbh
I think you're right, + it does make it look more like the sun firmware
fallen bobcat
i prefer the plain white background with no decorations or borders tbh
do you prefer white on black or black on white?
good enough
heavy sandal
nice
fallen bobcat
this wont be pushed for a whiiile though
it's mostly a testing thing atm but in the future I'll have a proper console infrastructure and stuff
heavy sandal
better than nothing for now
languid canyon
Not a fan of the Skyrim font but for troll purposes I guess lol
heavy sandal
Not a fan of the Skyrim font but for troll purposes I guess lol
its literally from sun microsystems lmao
languid canyon
Lol
fallen bobcat
Not a fan of the Skyrim font but for troll purposes I guess lol
tf you mean skyrim font 😭
you have no taste...
also skyrim is a great game so skyrim font == great font
languid canyon
heavy sandal
skyrim is a great game but thats despite its font choices lets be honest
but this font predates skyrim by uh 20 years
sorry 22 years
it predates me being alive by 9 years 
languid canyon
good enough
How does your cpu not have invtsc
fallen bobcat
I think qemu might not report it
heavy sandal
I think qemu might not report it
are you using kvm?
fallen bobcat
yes
heavy sandal
and what is your -cpu?
heavy sandal
i meant the arg to qemu
because unless youre using host for the cpu arg youll need to pass ,+invtsc on it
and even if youre using host you might
otherwise qemu wont report it
fallen bobcat
because unless youre using host for the cpu arg youll need to pass `,+invtsc` on...
yeah ok thats why
heavy sandal
note that if you do that and use tcg itll throw a warning at you that it doenst support that cpu feature but thats for obvious reasons and i just ignore it
but also idk why youd use tcg if you can help it tbh
languid canyon
but also idk why youd use tcg if you can help it tbh
Debugging triple faults is the only reason I can think of
fallen bobcat
it predates me being alive by 9 years <:KEKW:913708372329644042>
You're 28?
unless I did my calculations wrong
heavy sandal
You're 28?
27, birthday at the end of the year so not 28 yet
heavy sandal
damn
wet kernel
hot
past dome
good enough
erm
u copied me
wait that ones not sun gallant demi
looks like it thoug
fallen bobcat
I didn't know you had ever used it for mintia tbh
fallen bobcat
This is way more copying tho
drifting blade
Why did you decide on Zig
fallen bobcat
Have you come across many bugs?
Not at all
Except one bug I had with atomics in a test program where it wasn't compiling at all on the default backend but I use llvm now
drifting blade
Not at all
Ah cool
I was considering learning Zig a few years ago but the language was changing too much and unstable
fallen bobcat
I was considering learning Zig a few years ago but the language was changing too...
Zig is slowly moving toward 1.0
It should release this year
weary jetty
I'm still super stoked about asynchronous page faults in Telix.
languid canyon
How does that work?
drifting blade
Zig is slowly moving toward 1.0
Cool
weary jetty
How does that work?
Asynchronous page faults? Blocking on the fault would only switch to some other scheduler activation of the process so the kernel thread itself doesn't block.
languid canyon
im just confused about whats the alternative?
just poll disk completion in the kernel?
weary jetty
im just confused about whats the alternative?
In Linux, the user context is hardwired to the kernel thread, so it blocking forces the kernel thread behind it to block too.
languid canyon
kernel thread behind it?
there's only a stack
i'm not sure what you're saying exactly
do u share kernel stacks between threads?
weary jetty
do u share kernel stacks between threads?
User threads are backed by a construct called a scheduler activation instead of a kernel thread/stack, so one of them being unready merely provokes the kernel thread to do its userspace return to a different scheduler activation's user thread state.
languid canyon
so u have per-cpu kernel stacks?
weary jetty
In short yes, kernel stacks are shared between user threads.
languid canyon
so its basically stackless coroutines
weary jetty
so u have per-cpu kernel stacks?
No, it's not interrupt model programming, but most of the reasons a kernel thread would block don't happen. IO completion just does the accounting on the memory and the scheduler activation blocked on the IO just gets moved to the ready queue within the task that has all of the kernel threads it could run on.
languid canyon
No, it's not interrupt model programming, but most of the reasons a kernel threa...
does this mean you cant preempt syscalls
weary jetty
does this mean you cant preempt syscalls
Almost all of them look like network IO operations internally, so when the time comes to preempt them, they're not usually in places where it's an issue.
languid canyon
but in general u can only preempt when something blocks
so u have a non preemptible kernel
weary jetty
The cases where things block are a lot more limited. Waiting on memory to be made reclaimable by IO & then freed is one of the few.
languid canyon
ill be interested to see how it turns out for you
weary jetty
ill be interested to see how it turns out for you
Full preemptability is in the mix of goals for the scheduler, so wherever non-preemptability is an issue, the plan is to clean it up.
past dome
i had an idea kind of like this before
ill try to find it
this was in the context of avoiding mmap()d files because they block a whole kernel thread and dont allow another goroutine to run, and potential solutions to that issue
languid canyon
yeah that makes sense
weary jetty
It's probably pretty close. Scheduler activations and a message-passing VFS/IO layer and the one-request-one-completion IPC model are supposedly all needed to combine together so it just naturally falls out of the design.
You also need to have other threads to switch to, so your idea about the start-up thread blocking would still apply if it's the only thread at the time.
rocky yoke
how does userspace register scheduler activations in telix?
languid canyon
does it even have a userspace yet?
weary jetty
does it even have a userspace yet?
There's a very limited shell & an init that runs regression tests for native userspace. Linux binary compatibility is in progress, with current work centring round doing an Xwayland + GNOME + Firefox demo. The syscall ABI/API has been written for 100% coverage. Debugging is ongoing to patch up where test failures are happening as test runs are done.
Darwin & Windows 64-bit binary compatibility are planned after Linux. I'll likely postpone them until after some other milestones like filesystem drivers and a full-fledged scheduler (right now a Linux -like O(1) scheduler is sitting there as a placeholder, and I've done the research for the rest of what I want from it).
weary jetty
I'll likely do Windows before Darwin. I'll likely need to grab a system image from somewhere. I've got a roadmap of milestones to cover & will just keep going through them. I included ticklessness and full preemptability into the scheduler rewrite. If a cpumask_t -like affair isn't there already it will be. Architecture ports for everything else 64-bit covered by qemu are in the roadmap too. IPv6, SCTP, it's going to be the all-singing all-dancing kernel people only ever dreamt Linux could have been.
(I say this as of yet never having booted it on real hardware.)
weary jetty
how does userspace register scheduler activations in telix?
There are some native syscalls for handling them.
rocky yoke
There are some native syscalls for handling them.
How do they work?
lavish meteor
I'm still super stoked about asynchronous page faults in Telix.
wut dat?
languid canyon
Darwin & Windows 64-bit binary compatibility are planned after Linux. I'll likel...
Isn't that a bit too much
weary jetty
How do they work?
create_user_thread() and such.
rocky yoke
How does switching work?
Does switching always involve the kernel?
If yes, doesn't the overhead of doing that negate the throughput gain from async pf handling?
weary jetty
Isn't that a bit too much
If it's going to be impressive, it needs to be.
lavish meteor
You do you ig 
weary jetty
Does switching always involve the kernel?
No, the creation API is for having the idea that the new user context exists in the background to be switched to in the event of blocking.
weary jetty
Or well, for employment?
Getting noticed to get the job talks going.
rocky yoke
No, the creation API is for having the idea that the new user context exists in ...
But what does the context actually do when it's swapped in? How is work actually distributed to contexts? Also, the original context needs to be switched to at some point in the future so how does that work?
weary jetty
If yes, doesn't the overhead of doing that negate the throughput gain from async...
The fault has been taken and so the user → kernel → user switches are committed regardless. The point is that the threaded process can just return from the kernel into one of the other user contexts when a given user context would block for the sake of the fault.
weary jetty
But what does the context actually do when it's swapped in? How is work actually...
The scheduler activations (user contexts) can be the recipients of wake-ups, at which point they're returned to the task's queue of available scheduler activations.
lavish meteor
I'm still confuzzled, what's asynchronous page-fault handling? Why's it important?
languid canyon
scroll up
languid canyon
If it's going to be impressive, it needs to be.
it took reactos 30 years to get windows xp compat, and its still flakey. What makes you think u can pull off 3x of that?
it had literal paid developers doing that stuff
weary jetty
I'm still confuzzled, what's asynchronous page-fault handling? Why's it importan...
Most of the relevant faults aren't really taken directly from userspace, but from syscalls that would access user memory, so almost nothing is truly asynchronous unless faults are. For instance, blocking on writeback IO to free memory for ZFOD receive buffers.
fallen bobcat
it took reactos 30 years to get windows xp compat, and its still flakey. What ma...
They didn't have opus 4.6
weary jetty
it took reactos 30 years to get windows xp compat, and its still flakey. What ma...
It probably won't get as good as theirs, but I might be able to get up to where a demo works.
weary jetty
They didn't have opus 4.6
Plus I'd be building on their & WINE's work.
All 3 together & maybe I don't get all the way as far as they did, but maybe still enough to be worth showing off.
languid canyon
if u want like a windows xp notepad.exe demo then sure
weary jetty
if u want like a windows xp notepad.exe demo then sure
I'll eventually hit a time limit and have to go on to other things. If Claude Opus can't cannibalise enough ReactOS (yes, I knew about them) and WINE code to get a meaningfully flashy demo going, I have the time management skills to cut my losses & turn the fire hose in a different direction.
Wait, ReactOS only did XP? No 64-bit? Its sources may be limited to WINE, then.
rocky yoke
The scheduler activations (user contexts) can be the recipients of wake-ups, at ...
Does the kernel perform preemption of user contexts within a thread or are these switches always cooperative?
weary jetty
Does the kernel perform preemption of user contexts within a thread or are these...
They're mixed. It won't do timeslice expiry on user threads.
If they all just burn full timeslices it'll rotate between them.
wet kernel
it took reactos 30 years to get windows xp compat, and its still flakey. What ma...
Now u can give claude code on a loop the nt source code and give it tens of thousands of dollars in tokens
weary jetty
Now u can give claude code on a loop the nt source code and give it tens of thou...
Licencing won't work.
wet kernel
They wont be able to prove if you make claude write the code different enough :P
lavish meteor
true 
weary jetty
They wont be able to prove if you make claude write the code different enough :P
The point isn't to lie. Hence the joint attributions to Claude itself beyond the papers cited and codebases cited and licenced with respect to derivation.
fallen bobcat
ok I think I've come up with a nice way of doing vmem reclamation with SMR
well it's not really novel or anything
actually I'm not sure, would it make sense to add the invariant "all pending tlb flushes are complete" to a quiescent state?
this makes sense for something like e.g context switching to a user thread
fallen bobcat
https://www.cs.yale.edu/homes/abhishek/kumar-asplos18.pdf @long pendant you will like this
I reinvented something like this and they reinvented mach
https://www.cs.yale.edu/homes/abhishek/kumar-taco20.pdf updated paper
I'm thinking this can be done with classic RCU
fallen bobcat
I'm thinking this can be done with classic RCU
for SMR this can be done in a background thread that calls smr_wait, in rcu you can do a call_rcu
weary jetty
https://www.cs.yale.edu/homes/abhishek/kumar-asplos18.pdf <@391791686881247234> ...
Ouch, no THP.
heavy sandal
thp?
weary jetty
Transparent Huge Pages, Linux' extremely limited partial support for superpages.
The pain for me comes because I architected Telix round superpaging & a particular way to enhance it, so leaving them out keeps me from using it.
heavy sandal
ah ok
granite kiln
done
languid canyon
damn
languid canyon
cpuid what
fallen bobcat
the tsc frequency isnt always available in cpuid
languid canyon
its only available on new intel cpus
drifting blade
Ah
fallen bobcat
https://www.cs.yale.edu/homes/abhishek/kumar-asplos18.pdf <@391791686881247234> ...
I will implement this
I think it will be done on DPC dispatch
oooh yeah I have a great plan
weary jetty
I will implement this
RCU TLB flushing?
fallen bobcat
RCU TLB flushing?
it's not RCU but similar I guess
it's a quiescent state approach
fallen bobcat
I'm either stupid or a genius but I have figured out how to improve the design the paper proposes considerably
faster and less memory overhead
rocky yoke
these guys just reinvented what Managarm has already been doing forever
actually, let me check if we already did it in 2018
fallen bobcat
I'm either stupid or a genius but I have figured out how to improve the design t...
ok so basically each CPU has a lock-less circular queue of 64 "reclamation states":
struct state {
uintptr_t va_start; // virtual address of start of flushing range
uint32_t npages; // Number of pages (This is smaller than just having an end address!)
struct addr_space *addr_space;
_Atomic uint16_t counter;
_Atomic bool active;
};
These fit in 24 bytes
Each cpu also has ncpus notification bitmasks, these indicate, for each cpu, which states are relevant to them:
struct per_cpu {
uint64_t valid_states[NCPUS];
};
When a VA needs to be freed:
unmap(va, npages);
flush_local_tlb(va, npages);
slot = allocate_slot_for_state() or do_synchronous_ipi();
// figure out how many CPUs we need to flush
// cpu_mask = mask of cpus we flush, ncpus = number of cpus
slot.va_start = va;
slot.npages = npages;
slot.counter = ncpus;
for (cpu in cpus we flush):
atomic_set_bit(my_cpu.valid_state[cpu], slot);
Then, on DPC dispatch (or context switch, or another quiescent state):
for (cpu in all cpus):
if (atomic_load(&cpu.valid_state[my_cpu])):
for each set bit (use intrinsics):
clear the bit
slot = cpu.slots[bit]
invalidate slot, decrement counter atomically.
background thread on each cpu (or perhaps a global one?):
for (state in my_cpu.states):
if (state.active and counter == 0)
reclaim the page
fallen bobcat
korona can you take a quick look at my thing with your phd brain?
you could have another per-cpu CPU mask for "which CPU has sent me a shootdown" too, which avoids iterating over all CPUs
and it's still less overhead and faster
rocky yoke
yeah what you describe should work
fallen bobcat
smh yale should give me a phd already
idk why they didnt think of those optimizations honestly, it's pretty trivial and much better than iterating ncpus x 64 times
rocky yoke
Managarm's algorithm works like this:
- Each page space has a queue of shootdowns
- Each shootdown has a seqnum
- Each CPU knows which page spaces it has bound to ASIDs and what seqnums it has already dealt with
- On shootdown IPI, the CPU checks for new shootdowns in the queue (skipping over ones that it has already dealt with)
- When it sees one, it invalidates and decrements a counter out outstanding CPUs
- When the counter drops to zero, the CPU removes the item from the shootdown queue and signals completion
fallen bobcat
with this you can pretty much avoid IPIs entirely though, which is neat
I also thought about an epoch-based thing, that could also work but I think in this particular case the quiescent approach seems better
rocky yoke
Managarm's algorithm works like this:
- Each page space has a queue of shootdown...
well, you can also do that with this ^ at the cost of checking on task switch
fallen bobcat
yeah
I'm thinking checking on every task switch is not that expensive, it can be just a single atomic load
rocky yoke
whether you use an IPI or some periodic check doesn't matter for correctness
fallen bobcat
I'm thinking checking on every task switch is not that expensive, it can be just...
I'm thinking I can do it on every DPC dispatch though, but idk if that is sane
I'm not sure how I can choose a "good" time to check
weary jetty
I'm still mindlessly IPI'ing when I should only be falling back to that under the direst of memory pressure. I'll clean it up eventually.
fallen bobcat
finally pushed allocator stuff and they're not even finished 🥀
at least for now I have enough to get smp started
@lavish meteor @languid canyon https://github.com/rdmsr/zag/blob/a7d9027c952fc6ecfb4a3182c62fb358e4f79d0f/src/mm/vmem.zig here is vmem if you wanna check it out, I've tried describing it a bit in the comments and also noted the simplifications
but like I think this is good enough for my use cases
rocky yoke
//! -
BestFit: scan freelist buckets for the smallest segment that satisfies the
//! request. Minimizes fragmentation at the cost of a fuller scan.
BestFit has poor worst case fragmentation
first fit is optimal under an adversarial model
weary jetty
I might be too tired to absorb what it's doing well enough. I usually like to have address-ordered allocation policies where, say, pinned kernel allocations are skewed one direction and movable pageable user memory allocations are skewed the other direction. This can, in principle, be extended to skew after ranking candidate free ranges by other criteria e.g. best fit.
rocky yoke
weary jetty
Something like an R tree might be worthwhile for such searches.
rocky yoke
BestFit has poor worst case fragmentation
this is a classical result, see the paper above
I don't know why anybody is still teaching best fit, best fit is quite bad
Maybe it makes sense if you know that your allocations are not adversarial and you only have very limited memory
but aside from that you should never use it
weary jetty
Address ordering being in the mix changes everything.
rocky yoke
note that first fit is not the only good algorithm though
heavy sandal
somewhat hilarious that the algorithm with best in the name is a bad one lmao
rocky yoke
you can construct other good algorithms that are in a sense "stable", i.e., that return exactly the same memory in the same order no matter what happens to other parts of the address space
rocky yoke
somewhat hilarious that the algorithm with best in the name is a bad one lmao
yeah the abstract also kinda makes fun of this
rocky yoke
you can construct other good algorithms that are in a sense "stable", i.e., that...
For example, what frg::slab (but not sharded_slab yet) does is "oldest" fit, i.e., it doesn't return the lowest possible address but an address of the oldest non-fully-allocated arena
that is equally good as first fit
fallen bobcat
BestFit has poor worst case fragmentation
yes I think the vmem paper also mentions that, they say it should only be used when you need it
instant fit is better and a good enough (very good) approximation
weary jetty
To be clear, I'm not saying address ordering wins with certainty. Rather, I'm saying that the empirical results without address ordering don't carry over to the address ordered case. Empirical results would need to be directly got from runs with address ordering.
rocky yoke
wdym by "address ordering"?
weary jetty
wdym by "address ordering"?
After the space of free ranges has been narrowed by sufficiency & possibly additional policies, the ranges are ranked by how low or high of addresses they cover, and then one can choose the highest or lowest.
fallen bobcat
ok so basically each CPU has a lock-less circular queue of 64 "reclamation state...
i have a genius idea
each state has a list head of physical pages to be freed as well
the linkage is stored in the pfndb itself when unmap notices a present PTE
fallen bobcat
the linkage is stored in the pfndb itself when unmap notices a present PTE
instead of flushing the entire VA (in a case where like you would munmap like 1 TiB but one page is used), when unmapping, try to create an invalidation slot for every contiguous range of backed VA, if it exceeds a certain threshold, instead of continuing and filling all the slots, just flush the whole tlb
the downside is that if you have some huge mapping and randomly access pages then you dont benefit from individual tlb invalidations
I'm not sure this is the way to go, perhaps it is common for memory managers to keep track of which virtual page is mapped or not, I don't know
fallen bobcat
instead of flushing the entire VA (in a case where like you would munmap like 1 ...
that may not be a good idea
fallen bobcat
omg I hate linux code 😭
long pendant
you could have another per-cpu CPU mask for "which CPU has sent me a shootdown" ...
nice addition
i might copy this
fallen bobcat
right now I'm trying to figure out how to handle sparse mappings
do you have any insight on this?
fallen bobcat
right now I'm trying to figure out how to handle sparse mappings
you could do it with a rmap-like mechanism, where you go through each physical page needed to be freed and invalidate its assoicated virtual mapping
rocky yoke
wdym by sparse mappings?
fallen bobcat
wdym by sparse mappings?
Well if you have like a huge virtual mapping of like 256 GiB but only like three pages are actually mapped and backed by physical memory, it would make sense to only flush those pages
rocky yoke
If you'd need to flush more than X pages, just flush the entire ASID/pt
fallen bobcat
yes, but in this case would X be number of backed pages in the range or total number of pages
rocky yoke
I don't think it makes sense to make invalidations very fine grained
fallen bobcat
it would be much simpler with the latter but it'd be less efficient
rocky yoke
What you could do is take a base va and bitmask
fallen bobcat
Yes but then the bitmask could be arbitrarily large
rocky yoke
Well, if it exceeds a fixed size (like 64 or 128 bits) or more than X bits are set, just flush the entire asid
fallen bobcat
then this comes back to just checking npages :^)
well i guess it would reduce the number of flushes for small sparse mappings
rocky yoke
The point at which it is faster to flush the entire asid is pretty low
Like 32 or 64 flushes
rocky yoke
So anything that optimizes batches of more than that many pages is not worth doing
fallen bobcat
You're right
ok well that design is handled so now I need to figure out AP startup
afaiu it's just going through the ACPI tables and doing this
heavy sandal
yup
the only annoying thing is they start in real mode and you need a trampoline but that isnt too bad tbh
obviously you use x2apic for the ICR if you have it
but yeah
and you shouldnt broadcast the sipi you should be doing it single-target iirc
fallen bobcat
obviously you use x2apic for the ICR if you have it
Yeah that's what I was wondering about
How do I know if the target cpu is x2apic mode or not
heavy sandal
ok lemme find the bit there was something about that in that section
fallen bobcat
and you shouldnt broadcast the sipi you should be doing it single-target iirc
Why not?
heavy sandal
Why not?
the code there using broadcast is iirc meant more for firmware - you should be using single target to only bring up APs you know from tables are ok
fallen bobcat
That makes sense
fallen bobcat
Yeah I get that, but like on the new Intel CPUs I can't assume that
heavy sandal
you can know what mode it was passed to you in tho
and that i believe is consistent across all processors that itll tell you about
fallen bobcat
ok so if I'm started in x2apic mode then I should start others in x2apic too
heavy sandal
yeah
the one whacky caveat is because the ICR is also iirc related to the format of the bus message, you have to put the 8-bit id in the upper bits even if your BSP is running x2apic mode already if the APs were put in xapic mode by the firmware
which i learned the hard way
you can use x2apic msrs to access the ICR tho
took me forever to figure that out
fallen bobcat
What
heavy sandal
so on x2apic mode the upper 32 bits of the ICR are the 32-bit x2apic id of the processor
and on xapic mode the upper 8 bits of the ICR are the 8-bit xapic id of the target
that 8 bit id is mapped to the lower 8 bits of the x2apic id
but is 24 bits offset in the ICR
the thing is that whether you put it in 56..63 or 32..63 depends on whether the TARGET processor is in x2apic mode
which is normally the same for all processors and can be determined by just going am i in x2apic mode
EXCEPT during ap startup
when you might have switched the BSP to x2apic mode but the APs are in xapic mode because of the firmware
if that sounds insane its because it is
one side thing though thats nice is if youre clever with your asm you can avoid having to do any relocations for the ap trampoline asm when loading it into whatever page number
fallen bobcat
when you might have switched the BSP to x2apic mode but the APs are in xapic mod...
Yeah so just keep a flag did_start_in_x2apic or something
heavy sandal
Yeah so just keep a flag did_start_in_x2apic or something
yep
(side note theres an acpi madt subentry that can let you boot direct into long mode if its present. i have yet to find a single example of hardware or emulator that has it though, so i wouldnt bother with it)
fallen bobcat
ooh pvclock is great
languid canyon
It is
fallen bobcat
oh well now it crashes on real hardware during table enumeration
idk why
I will debug it when I have a proper console in place
languid canyon
oh well now it crashes on real hardware during table enumeration
non uacpi user skill issue
fallen bobcat
const entry_count = (x.header.length - @sizeOf(SdtHeader)) / @sizeOf(u64);
const entries_ptr: [*]u64 = @ptrCast(&x.entries);
for (0..entry_count) |i| {
const phys = entries_ptr[i];
const hdr: *SdtHeader = @ptrFromInt(mm.p2v(phys));
format_table(hdr, phys);
}
this should be fine right?
I checked and the ACPI tables are below 4gib so they're mapped
anyway that laptop is fucked
its got this weird super high res touchscreen
languid canyon
I checked and the ACPI tables are below 4gib so they're mapped
i dont think theres a single x86 box where they're above 4g
languid canyon
```
const entry_count = (x.header.length - @sizeOf(SdtHeader)) / @sizeOf(u64);...
well if you're using xsdt then yes
fallen bobcat
ok great now I just need to figure out AP startup and lapic stuff and I think the amd64 port will be mostly complete
languid canyon
did u fix the crash
fallen bobcat
did u fix the crash
No, I'll fix it when I have a proper console thing in place
Because right now it probably faults but I can't see it because all the console does is print back what's already printed
languid canyon
wdym by that
fallen bobcat
wdym by that
the way the console works atm is it just drains the ringbuffer once at boot
its a super hacked thing
fallen bobcat
TIL zig has scoped logs
heavy sandal
yeah
its good
thats the single biggest place i use the whole @"some string" identifier syntax
since it uses comptime enum literals for the scopes
also you can specify a minimum log level on a per-scope basis
fallen bobcat
yes
heavy sandal
which is nice
fallen bobcat
I dont use log levels yet
heavy sandal
even if you ignore it the log levels are handled by std
with the minimums
what you arent doing is printing them lol
but if you just set the minimum levels then itll automatically elide stuff thats below the min
and its comptime so its fully elided
fallen bobcat
what you arent doing is printing them lol
yeah
or rather including them in the log data
heavy sandal
yeah
languid canyon
what are scoped logs
fallen bobcat
what are scoped logs
It's like pr_fmt
heavy sandal
sorta
fallen bobcat
Assigns a scope to the log
heavy sandal
its a log with a tag saying where its from
and the tag can be used to fine-grained filter out by log level
all the things in parens here are scopes
fallen bobcat
why always print imaginarium
heavy sandal
because the bootloader prints tabula. for everything
it makes it clear whats kernel and whats bootloader
fallen bobcat
strange
heavy sandal
its not really needed but i felt like it and havent bothered to really think about if i want to change it lol
heavy sandal
@fallen bobcat is your framebuffer stuff pushed yet btw?
heavy sandal
fair enough, just figured id ask for comparing against since im working on mine now lol
fallen bobcat
I will speedrun ap startup
heavy sandal
@fallen bobcat fyi that zig 0.16.0 got officially tagged btw
fallen bobcat
I am still on master 
heavy sandal
me too albeit a bit behind actually lol
stray kelp
<:letsgo:1282155513156468747>
nice 🙂
fallen bobcat
is the 10ms sleep even needed btw
fallen bobcat
between INIT and SIPI
fallen bobcat
if it works in qemu doesnt mean it'll work on real hardware..
stray kelp
nope
fallen bobcat
are you using kvm?
yes
stray kelp
else i could quickly test it on my laptop
is you pass me the ISO
stray kelp
tho i think the 10ms is just to give the cpu time to boot
languid canyon
just look at linux source
fallen bobcat
just look at linux source
do you do it
languid canyon
i dont have smp startup
languid canyon
do you do it
fallen bobcat
/* if modern processor, use no delay */
if ((boot_cpu_data.x86_vendor == X86_VENDOR_INTEL && boot_cpu_data.x86_vfm >= INTEL_PENTIUM_PRO) ||
(boot_cpu_data.x86_vendor == X86_VENDOR_HYGON && boot_cpu_data.x86 >= 0x18) ||
(boot_cpu_data.x86_vendor == X86_VENDOR_AMD && boot_cpu_data.x86 >= 0xF)) {
init_udelay = 0;
return;
}
/* else, use legacy delay */
init_udelay = UDELAY_10MS_LEGACY;
mhm
fallen bobcat
not doing ap startup in parallel might not be a good idea but I'm not sure how to do it in parallel
I can't broadcast the IPI because of firmware memes
I guess I can preallocate enough AP data then send all the IPIs at once
drifting blade
What firmware memes
rocky yoke
Just allocate a fixed number of pages for the trampoline and use these to launch a fixed number in parallel
languid canyon
I can't broadcast the IPI because of firmware memes
Nothing actually does that
Just starting each one individually is very fast
As long as you can do it without waiting for full ap boot
fallen bobcat
Just allocate a fixed number of pages for the trampoline and use these to launch...
Yeah, or I can allocate all of them up front
fallen bobcat
// Taken from Linux, on modern CPUs we can skip the long delay after INIT.
const skip_delay = switch (amd64.cpu_features.vendor) {
.Intel => amd64.cpu_features.family >= 0x06,
.Amd => amd64.cpu_features.family >= 0x0f,
.Hygon => amd64.cpu_features.family >= 0x18,
.Unknown => false,
};
I can do this
it's still sequential but it's fast
and anything older than that wont have more than like 8 cores
pastel citrus
you could just send out INIT IPIs, then wait, then send out SIPIs
i mean, maybe, but knowing how janky hardware can be it's probably not worth it and might even break
it made sense in my head when i first thought about it
stray kelp
```rs
// Taken from Linux, on modern CPUs we can skip the long delay after I...
why not do it in parallel?
like, send all the INIT at the same time
wait 10ms
and then send all the SIPIs
like local ban enjoyer said
it shouldn't really break
fallen bobcat
why not do it in parallel?
Because it will race on the AP data
stray kelp
hopefully
fallen bobcat
you could just send out INIT IPIs, then wait, then send out SIPIs
Yes I think that works
languid canyon
why would it race on ap data
fallen bobcat
Because I allocate the AP data at like 0x8200 or whatever (idk what the actual offset is), so if an AP gets started before another AP reads its data block they both could end up having the same stack
languid canyon
☠️
stray kelp
Because I allocate the AP data at like 0x8200 or whatever (idk what the actual o...
bruh
that is very very bad design
i just use the kernel heap allocator
fallen bobcat
that is very very bad design
lmfao
stray kelp
and pre-alloc the data from the main core
languid canyon
the way linux solves it is
my_id = apic_id_to_cpu_id[my_apic_id]
gs = my_cpu_data[my_id]
rsp = per_cpu_ptr(current_task)->thread.sp
Literally everyone here does that from what I've seen
languid canyon
Literally everyone here does that from what I've seen
most kernels here are shitos
dont do it like that
languid canyon
thats an L for mangarm then
stray kelp
I'm not talking about per-cpu data btw
then what other AP data do you even need?
languid canyon
^
fallen bobcat
I'm talking about the data block passed from the BSP to the AP, the BSP sets up the stack for the AP and passes it inside that block, I cannot start multiple APs at the same time because that block is the same for all APs
languid canyon
the way linux solves it is
```
my_id = apic_id_to_cpu_id[my_apic_id]
gs = my_cpu...
.
i literally showed u how its solved
fallen bobcat
Where does it do that
stray kelp
I'm talking about the data block passed from the BSP to the AP, the BSP sets up ...
pre-alloc that data using the heap alloc
languid canyon
in the startup trampoline
stray kelp
send it
fallen bobcat
pre-alloc that data using the heap alloc
I can't
stray kelp
why?
wet kernel
Literally everyone here does that from what I've seen
Just wait until mintia 2 has a x86 port and see what hyenasky does, anyoene elses opinion is invalid
fallen bobcat
why?
It needs to be in low memory and also the way the trampoline is written it wouldn't really work
stray kelp
then pre-alloc the data there
and use an atomic counter for the cpu
or a specific id for each AP
then the APs will read the array correctly
fallen bobcat
the way linux solves it is
```
my_id = apic_id_to_cpu_id[my_apic_id]
gs = my_cpu...
I think I could do this if I write more assembly in the entry
languid canyon
yes
fallen bobcat
then the APs will read the array correctly
No you can't do that, because writing the trampoline to handle a different offset is tricky
stray kelp
most of the time they are the same
languid canyon
BUG_ON(next_apic_id != current + 1)
drifting blade
On QEMU it's the same so good enough
fallen bobcat
most of the time they are the same
It can literally be anything
stray kelp
not really
fallen bobcat
Yes really
stray kelp
every time i tested my os on diff pcs, it was the same
fallen bobcat
TIL you were the leading authority on firmware correctness
stray kelp
thats not representative of every pc to ever exist
still, it can work as a temporary method, lol
drifting blade
Firmware devs smoke crack so it's not always so logical
stray kelp
XD
fallen bobcat
I do cpu id allocation in the cpu itself though atm so I'll have to change that
But I can also just not care and take like 10 microseconds per AP booted lol
languid canyon
iirc it was an order of magnitude diff when linux switched from your approach to current
fallen bobcat
I think what had a bigger effect was the delay thing
stray kelp
and you still have less delay by doing it in parallel
languid canyon
I think what had a bigger effect was the delay thing
nah
the parallel bringup is relatively recent in linux
its at max a few years old
before that they used smp_control_data
a global variable
fallen bobcat
So like me?
languid canyon
yeah
drifting blade
Why would you need more
fallen bobcat
It's a very very bad design though!!
fallen bobcat
I think it's manageable, I don't even have to do it via the thread even
Well I probably should
languid canyon
i stole linux design from the get go so i do have apic_id_to_cpu_id
fallen bobcat
Do you dynamically allocate that?
Are APIC ids all contiguous?
Cuz like what if you have id 69 and then id 420, you waste a bunch of space for 2 CPUs
languid canyon
Do you dynamically allocate that?
its actually a lot simpler
the real array is called cpu_id_to_apic_id, cpu_ids are contiguous, the AP has to just linearly scan the array to find its apic id
the array size is NR_MAX_CPUS
.Lsetup_AP:
/* EAX contains the APICID of the current CPU */
andl $0xFFFF, %eax
xorl %ecx, %ecx
leaq cpuid_to_apicid(%rip), %rbx
.Lfind_cpunr:
cmpl (%rbx), %eax
jz .Linit_cpu_data
addq $4, %rbx
addq $8, %rcx
jmp .Lfind_cpunr
(snippet from linux)
fallen bobcat
The AP cannot know its ID without the cpu block
fallen bobcat
Yes that's why you use the array
stray kelp
The AP cannot know its ID without the cpu block
you iterate trough the cpu id array
languid canyon
^^^^
stray kelp
and find the cpu id that has the correct apic id
fallen bobcat
ahh yeah ok
I can do something like GS = offsets[cpuid] then
and then do a cpulocal read
languid canyon
u32 cpu_id_to_apic_id[NR_CPUS];
u32 whoami(u32 apic_id)
{
for (u32 i = 0; i < NR_CPUS; i++)
{
if (cpu_id_to_apic_id[i] == apic_id)
return i;
}
}
yes
fallen bobcat
```c
u32 cpu_id_to_apic_id[NR_CPUS];
u32 whoami(u32 apic_id)
{
for (u32 i ...
Yea I get it I had a brainfart
languid canyon
and yeah they dont have to be contiguous
u can have huge gaps for various reasons
like disabled cpus, manufacturing defects, firmware etc
fallen bobcat
And iterating over the array even if it's big is fast since prefetching
languid canyon
i mean how many cpus are u expecting to have
fallen bobcat
My NR_CPUS is configured to 64 by default tho so it's not a problem
languid canyon
even for like 10k it would be near instant
maybe if u have millions u might have issues
languid canyon
yeah
fallen bobcat
In a bunch of places I make ncpus a u16
languid canyon
i think the max number of cpus linux supports is around 32k also
fallen bobcat
I think x86 architecturally doesn't support more than 32k?
fallen bobcat
ok I managed to do it
wow it's tricky
/* Get our APIC ID */
mov $0x1, %eax
cpuid
shr $24, %ebx
/* ebx = our APIC ID */
/* Scan cpu_id_to_apic_id[] to find our cpu_id */
xor %rcx, %rcx
mov $cpu_id_to_apic_id, %rdx
1:
mov (%rdx, %rcx, 4), %eax
cmp %eax, %ebx
je 2f
inc %rcx
jmp 1b
2:
/* rcx = our cpu_id, save it */
mov %rcx, %r15
/* Set GS base to offsets[cpu_id] */
mov $__cpu_offsets, %rdx
mov (%rdx), %rdx
mov (%rdx, %r15, 8), %rax /* offsets[cpu_id] */
mov %rax, %rdx
shr $32, %rdx
mov $0xC0000101, %ecx /* MSR_GS_BASE */
wrmsr
/* Load stack from start_stack */
mov %gs:ap_start_stack, %rsp
this doesnt work for x2apic, you need to use a different cpuid leaf for that
fallen bobcat
but may not be available
can I read from 0xb then assume if 0 then its not available
I either do that or just add a flag to the ap data block
heavy sandal
can I read from 0xb then assume if 0 then its not available
if the other registers are 0 yeah
in fact thats what intel recommends
fallen bobcat
wdym
heavy sandal
the x2apic id can legally be 0
fallen bobcat
yes
heavy sandal
i think its EAX or EBX i forget which thats guaranteed nonzero on that leaf
if supported
heavy sandal
yeah
fallen bobcat
/* Get our APIC ID */
mov $0xb, %eax
xor %ecx, %ecx
cpuid
test %ebx, %ebx
jnz 1f
/* leaf 0xb not supported, fall back to leaf 0x1 */
mov $0x1, %eax
cpuid
shr $24, %ebx
mov %ebx, %edx
1:
/* edx = our APIC ID */
/* Scan cpu_id_to_apic_id[] to find our cpu_id */
xor %rcx, %rcx
mov $cpu_id_to_apic_id, %rbx
and if that leaf is supported you use it instead of the 8-bit field in leaf 1
fallen bobcat
I can do this
fallen bobcat
well now I've done it
I had to add a ExportedCpuLocal primitive
that exports the cpu local under a specified name
heavy sandal
mhm
languid canyon
```x86asm
/* Get our APIC ID */
mov $0x1, %eax
cpuid
shr $24, %ebx...
This won't work
For old apic the id field is writable
fallen bobcat
why not
languid canyon
So it may be different
fallen bobcat
so
languid canyon
It won't match the cpuid value
languid canyon
So you won't find it in the array
fallen bobcat
I read the apic?
languid canyon
so how do I do it
Read the mmio
languid canyon
Yeah it makes it a bit more complex
fallen bobcat
cuz now I have to do the x2apic check
then get the lapic base
then write to it
😭
languid canyon
Yep
fallen bobcat
or I could just expose my zig apic_write function
languid canyon
Yeah
fallen bobcat
I mean read
languid canyon
The base may not match the bsp so be careful
heavy sandal
put the bsp apic base register in the ap data
since you want that to sync anyway
the shared data that is
languid canyon
Fair
rocky yoke
just support x2apic only, for x2apic the id is not editable 
heavy sandal
and then you can set it in the ap trampoline before reading your id
heavy sandal
thats probably what ill do when i do parallel bringup, instead of passing the gs base pass the lapic base register in shared data and then load that to the msr (plus pass the bootstrap cr3 and load that)
languid canyon
They gotta pay the salaries for something lol
heavy sandal
oh god i just realised
you cant use a function for the apic id read if it uses the stack
languid canyon
Lol yes
heavy sandal
because you need the apic id to get the ap stack
languid canyon
thats probably what ill do when i do parallel bringup, instead of passing the gs...
So your bringup right now is serialized also?
heavy sandal
So your bringup right now is serialized also?
yeah
languid canyon
Plan ruined
fallen bobcat
fucking intel
heavy sandal
i already thought the arm bringup was so much easier before this. now that feeling is even stronger
languid canyon
Yeah writable apic id was probably the stupidest decision
languid canyon
Not sure what flexibility lol
languid canyon
But relocatable base is not the same as modifiable id
heavy sandal
on aarch64 all i have to do is make a secure monitor call and give it the physical address of the trampoline and an arbitrary usize context value. and then it calls my trampoline with the context value in R0 and a bunch of extremely reasonable defaults lmao
languid canyon
Relocatable base makes sense
heavy sandal
you have vmem, it being relocatable doesnt really hurt anything
languid canyon
on aarch64 all i have to do is make a secure monitor call and give it the physic...
Well it has tons of different ways to bringup cpus right
Thats just one you mentioned
The most common one is the park proto no?
heavy sandal
Well it has tons of different ways to bringup cpus right
the one i mentioned is afaik the most common - theres a billion ways but most fully-fledged systems will implement them in firmware at EL2 or EL3 and then have that smcall/hypercall to do it for you
languid canyon
Oh ok
fallen bobcat
instant panic for some reason
heavy sandal
its basically just saying hey firmware please do the painful part for me thanks
fallen bobcat
ah they just have random IPLs for some reason
fun
const cpu_data = mm.heap.alloc(percpu_size) catch @panic("Failed to allocate per-cpu data");
@memset(@as([*]u8, @ptrCast(cpu_data))[0..percpu_size], 0);
shouldnt that zero it?
heavy sandal
Oh ok
there are two different variants of the setup i mentioned fwiw, but the literal only difference is if you use smc or hvc (also both versions have an aarch32 version too so ig four variants)
heavy sandal
```
const cpu_data = mm.heap.alloc(percpu_size) catch @panic("Failed to ...
is it loading IPL from the register at all?
fallen bobcat
IPL is a cpu-local variable
heavy sandal
yeah i know but the TPR exists so is the bringup synchronizing it at all?
fallen bobcat
I dont write to tpr
heavy sandal
instant panic for some reason <:nooo:750466870993223750>
also this implies the IPL is currently passive which id assume is 0
heavy sandal
well if youre memsetting to 0 then id expect it to be whatever IPL is mapped to 0
which id assume is .passive
languid canyon
Why dont you use the tpr if you use ipls?
languid canyon
Ah
fallen bobcat
Why dont you use the tpr if you use ipls?
also should I copy the per-cpu data from the BSP to the AP, no right