#Zag
1 messages ยท Page 2 of 1
the endgame plan for mine is to only attempt a trylock-and-flush if moving to dispatch is a raise
if its already above dispatch it just skips the flush
huh
and then ill just make the idle thread call flush 
well having threads do it is what linux does iirc
so raise to dispatch in the trylock but if already above dispatch dont bother flushing
I guess I can have a periodic flush or something
i might make it schedule a dpc if its above dispatch when it goes to flush
that's not a bad idea
tbh i think i might make a helper soon for stuff like this
takes a function pointer and if its below dispatch it raises to dispatch and calls the function and if its at or above dispatch it queues a dpc that calls the function
what's neat is that if its already locked it means your thing will get output(ted?) anyway
you can just do that in dpc_enqueue
yep exactly, thats why i use trylock because if its taken then its the other threads job anyway
yes but there are actually cases where i want to queue a dpc from low irql (mostly when i do a quick dpc self-test during init)
its not a true self-test so much as me just printing a test message from a dpc because im not confident enough in my code yet to trust em lmao
i do a lot of things like that, quickly using a thing early in init so if i break it it breaks early
I use DPCs a lot so I think they're not broken
the fireworks test ironed out a few bugs
mhm actually idk if I wanna do this
because enqueuing a dpc that will do essentially nothing when theres little contention is kinda icky
yeah
linux does a thing where a context can steal the flushing process
idk how it works
it locks in a way that makes the other thread break out of the loop when it starts waiting idk how
no this isnt bad considering a DPC can only be enqueued once
meaning?
send a dpc to all processors, or all except self
no why would i do that
instead of a single target processor
idk why you would lol, i have em but they make things so messy and i dont really have any uses for em so i might remove em
I have cool lockfree cross core calls tho
damn nice, ill have to take a look at how you do that
rip
i could make my DPCs lockfree if i was ok with making the queue a stack instead (aka prepend the DPCs)
you can make them lockfree if you dont need to dequeue them
but dequeuing is needed for e.g unloading drivers
oh i dont really plan to support that at this point
otherwise you dont need locks as they are cpu-local
oh lol true
also its still fairly easy to cancel a dpc on another cpu with a single atomic bool
just set a canceled flag in it that makes it get skipped
well yeah but thats clumsy when you have tons you need to skip over
yeah
way easier than trying to make a lockfree removal from the middle of a linked list tho
just do it with a lock
yeah ig
that is what i do now lol
my main problem rn is my wait code is a fuckin train wreck of edge cases and race conditions
and deadlocks actually i ran into a deadlock when trying to release a parked thread from an interrupt handler on the same cpu in the middle of queuing it to wait
at which point do I have too many files in the same folder
cuz i kinda wanna have log.zig and console.zig separate 
that is entirely up to you tbh
i made em separate lol
i tend to prefer more files over big files personally
yeah nah fuck it im keeping it in the same file
does the fireworks thing require a framebuffer btw?
cause i might be interested in trying that once i get my scheduler going more but i dont have a framebuffer yet
yeah
probably need to describe the implementation of the ring in more detail tho
oh also need to include kconfig with it
oh if you type up good comments for the log ring thatll be great
that shit is so hard to understand without comments if you arent an atomics knower
i tried explaining the orderings at least
but the wrapping arithmetic is pretty weird too
ill read thru it when its not 1am lol
i dont think i could explain it or remember it offhand but the wrapping logic i found pretty understandable, but ig thats just compensation for how i look at any atomic ordering other than whatever x86 atomics are equivalent to and my brain just short circuits about it
I will try writing a module comment like the scheduler one
well not now cuz it is, in fact, 1 am
non-blocking queues exist, it's basically just translating the paper into code.
yeah ik but its a pain to actually write that code
Have an AI agent translate the paper into code for you.
lockfree queue is not worth it in this case
i have never and will never use ai for anything, period. no exceptions
but yeah also not worth the effort in this case anyway
The thought is more that non-blocking queues are sort of a solved problem.
mhm wondering if it's sane to put the console stuff in Ke
Doesn't really make sense imo
did u end up understanding why we need acquire for the head?
or the seq num thing
Yes
wdym
.
Well I made this with plain ops and it works so
Only the state transition needs to be ordered which make sense
wait so the issue u had was with relaxing something else?
where
In desc alloc
You already do it I think
I thought I could be smart and make the first load relaxed
ah
I'll explain why it's wrong in more detail later am on phone rn
sure
Also what do you think about flushing
Right now what I do is try lock the console lock and if it succeeds it flushes it
bad idea
But that might not be good for stuff like nmi or whatever
you can just starve a random core that happened to be unlucky
if other cores keep feeding logs
no
not lose
just have a core that hangs because it keeps flushing messages that keep appearing
thats why a takeover mechanism is important
ah
so if u have multiple cores that keep spamming messages they all steal each others flushing work
the last core to log does the flushing
How do you do this
The stealing
It wasn't pointed at you but like in general
as for how linux does it just check the code
ah ok
lemme send a link
also do you think the logging stuff belongs in the core kernel or outside of it? Like the console registration and stuff doesn't really make sense to be there idk
I suppose I could figure out a better way to layer them nicely
i dont do nt larp so everything is the core kernel for me
but its hard to say
Gotta love their printk file being bigger than my entire kernel
since even the core kernel logs it probably belongs to the core kernel
i think its because it contains the /dev/kmsg interface impl etc
what can happen here if the first load is relaxed is that the head load is moved after the tail load, which means the prev_id and stuff are stale and it tries to advance the tail on something that is not actually the tail
I think what I can do to sensibly flush logs is:
on early boot, when no consoles are set up and threading and whatever is not initialized: flush synchronously messages in the ringbuffer up to an upper bound (to avoid starvation)
when fully booted, simply set an event when there is a new message and a worker thread waiting on that event will flush the logs
actually no, I dont need to flush synchronously at all, on early boot I can do:
publish_msg();
print_msg_to_platform_debug_write();
and all the console stuff can be put somewhere else
the temptation to add new features instead of porting to amd64
what new features are you thinking of
i would try to port it to amd64 but sadly i can't be bothered to install a separate zig version for this project
actually those can wait
I think
I wanted to reimplement the allocator and stuff
but the amd64 early init doesnt need it
I think I can get up to early init at least for now
I want to stay up-to-date with latest zig and dont wanna end up having to rewrite lots of code
so I do it incrementally
ok cool it builds with master, thanks
zvm install already does use as part of it
idk just to be sure
oh yeah i didn't have to do the use part :^)
run ./zig-out/bin/kernel-x86_64 -g -t 10 to see fireworks on ten CPUs
Damn you working on zag now??
fireworks
yes
it shouldnt hang
if it hangs tell me and I will cry for 3 hours then try to fix it
it does not hang
idk i'll try to do amd64
Cool
and then i'll see maybe i'll work on my own kernel
I can do the early init work honestly if u dont want to
idk up to you, i can try but idk what i'm getting myself into
it's annoying boilerplate
one thing I wanna try in the future tho is boot using hyper and maybe an efistub
that would be cool
Who creates this window btw
no I abstract over it
Ah
Sounds like what you would expect
also @pastel citrus you might wanna pull and rerun the config thing because I changed kconfig
now the log buffer size is set using it
cool, thanks
but it's really up to you, if you know other architectures id like that more than amd64 I think
because I already have amd64 code that I can rewrite so you dont have to think about it
aarch64 would be very cool if you know it
Naaah dawg
what ๐
Real
ring buffer assumes 64 bit
opinion rejected
I say aarch64 because I have a mac so running it virtualized would be cool
no committed state in the ring buffer
infy just hates arm in general iirc

no one needs commit
[iretq@raptor zag]$ zig build iso
iso
โโ install generated to myos.iso
โโ run xorriso (myos.iso) w
xorriso 1.5.6 : RockRidge filesystem manipulator, libburnia project.
Drive current: -outdev 'stdio:/home/iretq/sources/zag/.zig-cache/o/a9bd2e42e4b51123d6fa024dbf022427/myos.iso'
Media current: stdio file, overwriteable
Media status : is blank
Media summary: 0 sessions, 0 data blocks, 0 data, 948g free
Added to ISO image: directory '/'='/home/iretq/sources/zag/.zig-cache/tmp/0340e873cb132937'
xorriso : UPDATE : 11 files added in 1 seconds
xorriso : UPDATE : 11 files added in 1 seconds
ISO image produced: 2101 sectors
Written to medium : 2101 sectors at LBA 0
Writing to 'stdio:/home/iretq/sources/zag/.zig-cache/o/a9bd2e42e4b51123d6fa024dbf022427/myos.iso' completed successfully.
failed command: xorriso -as mkisofs -R -r -J -b boot/limine/limine-bios-cd.bin -no-emul-boot -boot-load-size 4 -boot-info-table -hfsplus -apm-block-size 2048 --efi-boot boot/limine/limine-uefi-cd.bin -efi-boot-part --efi-boot-image --protective-msdos-label ./.zig-cache/tmp/0340e873cb132937 -o /home/iretq/sources/zag/.zig-cache/o/a9bd2e42e4b51123d6fa024dbf022427/myos.iso```
๐ค
As a platform yes
Not as an arch
it says fail because xorriso is retarded and returns 1 on success
Hmm pretty sure it doesn't, at least my python thing never tripped on that
main.zig is a stub dont use that, the entry code should be in platform/amd64/entry.zig or smth
makes sense, gotta adjust build.zig then
i saw that it picks between platforms/um/entry.zig and src/main.zig 
actually ideally theres something like platform/boot/limine.zig with a limine_entry() or limine_main() that sets up the boot info and passes it to the platform entry
thats probably better
thats what i did with my old code
one thing i always like is having the arch declare a const with the calling convention
and then the limine main or whatever can pull that to set the callconv on it
well you can just comptime { _ = limine.kmain; }
in amd64/entry.zig
what does that do
just references the symbol
limine being the shared module that does the setup
i use it to force include the arch init stuff on mine but leave my krnl.zig as always the module root
I think it should be something like:
if config bootloader is limine:
entry = platform/boot/limine.zig
then in limine.zig y ou do something like:
fn limine_main():
convert limine to boot info
pl.impl.entry(boot_info)
that's what I did in my old code
acutally I can send you a tarball if youd like to take a look
most things are structured similarly
it's in C
i'm too lazy ngl
alright 
looks like i need to add some more code to config.zig
to extract shit like bootloader from the config
hm, i added it as a field to platform
optional field ofc
i can change it later
i'm gonna eat now, dinner came up
if you want it done relatively quick then you can do it tbh, i usually work pretty slow 
i'm mainly doing it for fun so don't feel bad
I have other stuff to do anyway
that's good, i'll see how far i can get with this after i'm done
but like if u can get up to first_init() working that would be great
I can do the rest
sure sounds good
oh fuck you might need to remove the stuff in kern/init.zig for it to build
I just assume std.heap.page_allocator exists
so remove all the thread stuff (or put it behind a if (false))
i saw some errors earlier related to that lol
moslty just comment out the function and the related code
should be easy, thanks
you need to define it for freestanding
yes you do
i just copied them from main.zig to root.zig and it works
well
memcpy the symbol you do
but @memcpy exists and __builtin_memcpy exists and is defined in compiler_rt
even on freestanding
thats fine for now but eventually I'll put it like in rtl or something
dunno
why arent you using @memcpy
also, xorriso doesn't return 1 on success
im not even using memcpy!
it just prints to stderr
and its siblings @memmove and @memset
zig does
then what are you linking that needs it??????
zig
zig defines it!
in compiler_rt!
clearly not ๐ญ
llvm might rely on it
something is wrong then
zig defines it for llvm in compiler_rt
its probably an llvm thing, but like I stole that from the zig template and it also defined its own stuff
im literally linking c code and didnt need to define memcpy/memmove/memset
whats your target triple?
and your build script for that matter
fn targetQueryForPlatform(plat: config.Platform) std.Target.Query {
var q: std.Target.Query = .{
.cpu_arch = plat.arch,
.os_tag = plat.os,
.abi = if (plat.os == .freestanding) .none else .gnu,
.cpu_model = .baseline,
};```
.cpu_arch = .x86_64 and .os_tag = .freestanding
wait does baseline work for x86_64? ive had problems there
yeah it should
otherwise it'll do native
because you aren't cross compiling
that's the logic in the build machinery
if (plat.os == .freestanding) {
kernel.linkage = .static;
kernel.pie = false;
kernel.bundle_compiler_rt = false;```
bruh
idk what kind of a shit template you found that does that
well then i blame whoever made that template being stupid
i took that and removed everything apart from the target stuff
yours uses make
lmao
xorriso issue
it's because it prints to stderr
yup
but it's really annoying ngl
thatll do it
doesnt matter it still works
๐ญ
๐ญ
very cool
nice
/// Panics!
pub fn panic(comptime fmt: []const u8, args: anytype) noreturn {
// FIXME: this is probably broken.
std.log.err("KERNEL PANIC: " ++ fmt, args);
while (true) {}
}```

beautiful
nice lol
For CXL support?
no I wanted to reimplement my allocator i had written
its a fairly standard bonwick slab allocator
with magazines and stuff
I'm leaning towards making something more freebsd-esque though, their allocator is quite nice
one of these days im gonna have to get my head around magazines tbh
I use that on my end. What's nice about FreeBSD's allocator?
one thing that is cool is they integrate it with their safe memory reclamation mechanism
they also claim it is faster than linux's SLUB
Go to the 10:40 mark to skip us fumbling around.
https://youtu.be/ZXUIFj4nRjk?list=PLxYop2KBDBV3pAsLgfo6b2ZiEFse_KZSH&t=637
slub is what ive got on my end
Holy cow, I knew him maybe 15 years ago.
Oh wow, I had no idea he had got into kernel hacking.
He was always a better programmer than I by far.
Samy Bahra, I don't know Jeff.
I haven't heard from him in 15 years either.
dude casually wrote like 1/4 of modern freebsd as a hobby while working as a farmer ๐
Ah, so Samy is just randomly putting up the video of Jeff's talk on his channel? Well, he always did have good taste in algorithms, too.
it's a systems conference or something, and jeff is doing the first talk for like 15 mins
about freebsd's memory allocator and safe memory reclamation
Hmm. Samy was always big on epoch reclamation. Maybe it's connected to that.
I'm making it cargo fmt and warning clean, will just make sure I can log in on all 3 architectures & then do the initial push to GitHub even without the documentation written yet.
curious to see what you came up with
is it all in rust? I know you mentioned sel4 but I'm unsure how it relates
I appear to have accidentally backed out my own pagetable sharing bits, but the critical VM idea is page clustering for a software-configured allocation unit larger than the MMU/TLB mapping size to do 2 things with superpages:
- guarantee superpage allocations will not fail due to external fragmentation for superpage sizes less than or equal to the allocation unit
- bridge wide gaps between the MMU/TLB base page size and the first superpage size by choosing allocation units to make it easier, thereby trading internal fragmentation to reduce external fragmentation
kewl
time for some smp ig
actually, i'm not even sure if i can do that considering there isn't an allocator
fireworks also need an allocator :(
It's all in Rust except, of course, for the assembly. The relationship to seL4 was just broadly basing the general design of seL4's.
yeah which is why i told you to only do early init
the allocator thing
yeah that is fair
for smp I think it will be bootloader dependent idk
like in limine.zig I can probably boot up other cpus
and for other bootloaders do it manually
yeah, that's kind of where i put ke.ncpus = 1; and ke.cpus = ...;
not ideal but whatever
in the future I'll do away with cpus[] entirely
I think I'll do the linux thing and only work with ids
What is this?
linux uses ids to access cpu-local variables
what I did before having cpu-local sections is I had a global array of cpu structs and used that instead
but now I'm transitioning towards ids
I've not done too much thinking about per-CPU affairs. I don't even remember how my magazines find their CPU-local data.
what I do for magazines is store an array of ncpus magazines per kmem cache
which is also what solaris does
I wonder why not to have a virtual range dedicated to per-CPU data.
cool but you should've put the thread stuff in ke/amd64

nvm it is
im stupid
yeah no i need to adjust a bunch of stuff (like the per-cpu stuff is just plain wrong) but this should be a nice start
thanks
it works!!! (in the one-cpu case
)
There's no way legacy features like segment registers can't have huge legacy emulation overheads.
yeah
memory addressing in general
even on long mode its all segments still
although segments dont affect virtual->linear address translation in long mode
fs/gsbase still does
iirc fs/gsbase is actuaslly the segment base register just extended and only directly accessible through the msrs
I think there's a default segment register. I don't know that other segment registers suffer large legacy emulation overheads.
That also sounds relatively reasonable.
also minor nitpick: thread context can be kept fully on the stack
I thought TLS was mostly done like an ELF section.
Oh, x86 context handling is insane enough that I have to do it straight out of the book every time.
The tech debt exists, but squashing it doesn't have to be done by hand, so in some respects the rate-limiting factor is my tech debt assessment & spotting speed.
ah I thought you were building on top of sel4
so is it a microkernel still?
It's supposed to be. I should probably go over it with a fine-toothed comb & make sure all of the design goals are being honoured.
@pastel citrus do you plan on continuing the port or not? if so idt and gdt setup would be cool otherwise I think this is enough for now for me to go clean up and implement the allocator and stuff before we move on to a full port
mhm well i guess setting up the boot info for real would be good too
i assume yall already know this since i think ive mentioned before but if you use export for the limine protocol stuff dont forget to call std.mem.doNotOptimizeAway so the compiler doesnt constant-fold the null default response pointer
uhh yeah i could still work on it, but if you want to do stuff now then i'd just get that merged in and continue yourself because i need to leave for a bit
yeah that would work
do you want me to make a pr?
yeah
alright
Ill merge it directly and clean it up
or I can do like you did with my limine PR 
Just a quick interactive login touch test on all 3 architectures now supported needs to get passed & the initial git push is imminent.
aaa you broke um
wait really?
its ok I fixed it
is it the platform entry file thing
like it shouldnt return error.noBootloader if on linux
and shouldnt use platform/x86_64
I'm not yet but it's coming fast.
what
idk if having it be boot/limine.zig is a good idea (that's not what you did though)
because like if i end up on an architecture where theres only one bootloader isnt it weird to have boot/weirdbootloader.zig
idk
ah well in that case I can just set it up in its own entry.zig
ok this is fine
The pool_cache(9) manpage doesn't describe the implementation in too much detail.
oh yeah, i forgot what exactly you wanted lol, itโs a trivial change tho
but in that case you can just have a function that decides the entry file path based on arch+bootloader
ok time for more amd64 stuff
also its crazy how discord hides my own thread after like 1 day of inactivity
I have to go search for MY OWN thread ๐ญ
it does that for all threads
but cant I set it so that it doesnt disappear for me
i have a discord server that only im in and i just made a link to my thread and put it there lmao
yeah itd be nice to keep your own threads there longer
you might be able to easily search for it if you press ctrl+t in the desktop app
i dont think there is a shortcut for that on the web version tho
there's hide after inactivity thing
but I think there was something in #announcements not to do that
I've been porting to LoongArch64 and MIPS64, generalising things so it's not merely assumed that there's only one superpage size, I dropped in some easy cases for pagetable sharing & will probably let things spin doing fixes for LTP (Linux Test Project) overnight. I wonder if there are things like that for other kernels so I can get some other personalities going. *BSD? Solaris? Windows NT? OpenVMS? NFI.
I cooked up a data structure BTW: https://github.com/NadiaYvette/hybrid-radix-btree-adaptive
I also pushed telix upstream to my GitHub just in case you didn't catch it.
oh didnt see you pushed it, nice
I never finished the docs but I got a first cut of the abstract & am going over revisions I need to make.
I'm gonna be interested for that more than the code since I'm not much of a rust guy
I can write FORTRAN in any language. Maybe I have to squint sometimes to get through Rust's syntax being different. I can still read & write it, albeit maybe not quite as swiftly as C or Ada or FORTRAN.
The Adaptive Radix Tree apparently has some relatively simple techniques to address its performance breakdown in sparse datasets, provided you're not shy about plugging B+ tree code for the sparse case.
I might bring it over to telix b/c it basically wins against B+ trees and ARTs by being the best of both by switching between them.
@heavy sandal do you use the limine header directly or did you write your own bindings
BTW the data structure code is in C.
i took https://github.com/48cf/limine-zig and modified slightly when i was using limine
i ended up writing my own thing
yeah afaik only decls cant be named type, because fields can only ever be accessed with dot-member syntax
For what architecture(s) does it do bootloading?
limine? a bunch, idk the list offhand
All the relevant ones
i'm probably gonna update that soon
i found myself needing limine bindings for my new wip project 
From looking at EFI docs, I can't piece together quite enough details to figure out how to get from end to end, but it looks like the environment it drops things e.g. bootloaders into is easily rich enough to not be difficult to operate in.
yeah efi aint bad at all
Wait are you using zig again
The problem is manufacturers are often writing broken firmware
trying to, yeah, but i am not gonna stick to that project for very long
Bruh don't be so hard on yourself
nah i'm not, i just have no motivation to work on it
wanting to do something cool isn't enough
There's never an end to that, true enough.
i tried to read your and infy's log rings and i just realised i have no clue what the hell i'm doing 
it's not like either of us could've implemented it without the Linux code lol
I did just write more comments tho if you wanna check that out
Yep
Why not
What is โitโ?
Log ring
I'm not sure I know what that is offhand.
a lockless ring buffer for kernel logs
[...] i just have no motivation to work on it
Damn
and i just don't have the ability to put in these hours researching stuff and learning everything like you do
Never lol
:(
U should make one
lol no
Your ring is more general purpose since its 32 bit
It's pointless to make that an embeddable library
Why?
I could see some utility in that
idk the thought of not NIHing stuff scares me
idk, adhd maybe (diagnosis pending)
A lot of people want one
To me it seems lazy
uACPI is where I draw the line
cuz I ain't writing that shit
Maybe
but like flanterm and stuff ehh
Fair enough
My goal is to learn
If I reuse a bunch of libraries I don't learn anything
Ig
What I do if I'm reading an OS book for example is read like 10-15 mins a day
MIPS is misbehaving here.
Usually MIPS has been the best-behaved of all architectures for me. I can't spot what's off about things that's making the first context switch to userspace fail.
It helps if it's not the same architecture every time.
I used to have a lot of cool stuff. Nowadays all I've got is qemu.
Lucky for you qemu emulates a ton of arches
To a very limited degree. A number of the systems' values were that testing on them was desperately needed.
@fallen bobcat in your UM target you do context switching right? Since it's just one target how do u handle different host arches?
I use ucontext
ucontext does have overhead though, so I'm considering just emitting different assembly stubs for every arch
its posix
tbh who cares about overhead for debugging
yeah true
ideally posix would introduce a new generic way though since ucontext is deprecated
qemu doesn't have the access to the systems to verify the accuracy of their emulation or simulation either, and so often doesn't even attempt to emulate the systems. It just emulates the most common ones that people don't have trouble getting to begin with.
i wanna make a um target as well, ill probably get inspired by your code
lol
I do have an idea but not sure yet, I think I'll just ignore the vma limit lol
In what respect?
well atm there is no memory at all, but in the old code I just mmaped it
i didnt handle switching address spaces tho which is what's actually tricky
how do u do fireworks and stuff without memory management?
its basically kernel mode threads now right
this worked though
fireworks is always kernel mode
oh ok
its just a stress test
this is what I mean by "solving memory" btw
do u have a slub?
no I have a bonwick-style slab
whats there to switch if its just kernel threads?
I also have cool ideas for that
eventually you might want to run user apps
the naive solution is mapping and unmapping everything
yeah
korona pointed out there was a VMA limit issue too
but apparently linux doesnt care
what linux does iirc is mmap only as a TLB of some sort
on every ctx switch
does it rely on userfaultfd?
um sounds like pain tbh
it is
it's not
just dont do user apps if you dont feel the need to
I dont plan to have them rn but eventually it'd be very cool
basically the only thing you gain is asan
this is literally the only tricky part
which kasan would be cheaper to do
ok true
valgrind probably would struggle but maybe
idk its like 200 lines of code so far and its really useful
maybe if u omit the usermode part
to me at least
its good
yeah you don't have to support user apps
you can also run unit tests/integration tests
someone here literally implemented ioapic etc emulation
to verify correctness
thats a bit too far imo
apache?
probably
personally I think it's a bit stupid to do that
like for stuff like that u can just boot qemu and get stdout
because you dont know if your emulation is correct, and if it is then you probably already know that your other code is correct
yeah
regression testing is still important but yes
well it's up to you if you implement it, but personally i have found it very useful
and I will keep shilling it
like if you consider most people here already move their code to userspace to test it first, I just go further and run the whole thing
probably
I've been running on qemu, so it's all debuggable anyway.
id say its not phenomenal value to run it in usermode but still useful
The ART-B+ tree hybrid seems like it should be deployable.
I have come up with a nice cpuid abstraction
const feature_info = CpuidRequest.execute(.feature_info);
const extended_features = CpuidRequest.execute(.{ .extended_features = .first });
const extended_processor_info = CpuidRequest.execute(.extended_info);
const power_management_info = CpuidRequest.execute(.power_management_info);
nice
Why is it needed since cpuid is arch specific features anyway?
it's just nicer to work with
Hm
@fallen bobcat btw do you trigger DPCs using an interrupt or do you just handle them as part of returning from interrupts/lowering ipl?
the latter
I do it all in software
You can do it all in hardware (using cr8) and use self-IPIs though
self-ipis is what i do but im running into race condition hell about it atm
On windows the irql functions are basically inlined to a mov cr8
(though i dont think its actually because of the route i took i think its just bad code i wrote)
What is it
yeah ik, was just curious how you did it
ive got a function that queues DPCs if youre at or above dispatch and just runs the passed function if you arent. my event sync primitive uses that to defer signalling to dispatch irql to prevent race conditions in the yield function, but im running into use-after free on that event signalling dpc if the event never gets waited on
which is weird because the place where it isnt being waited on is running at a level where it should use the immediate callback and be fine
Why are you dynamically allocating your DPCs in the first place
the use-after-free isnt the dpc, its the event object being passed
its just that i dont understand why the dpc is being delayed so much
bruh I got mails from the Linux mailing list but the mail client on my phone is ass
And they're all European so I'm fucked until tomorrow
what did they say
oh cool u got 2 reviewed-bys
Wanna tell them maybe we should merge this and do another patch for other fixups
I think there's enough instability ongoing in the USA that a lot of ppl likely made arrangements to go somewhere else like I did, which might explain why they're all here in Europe now (I myself am in Berlin).
What's a DPC?
deferred procedure call
basically just something you can queue up from inside an interrupt handler that then runs at a lower interrupt priority level
Ah, C can be painful to do continuation-passing style within.
i think DPCs are a windows-ism originally?
idk maybe someone else did it before NT
and it isnt really continuation passing
its more like starting a thread or queuing something in a thread pool
It's like linux softirqs
DPCs
yeah
except someone realised you can use the softirqs for other things that arent irqs too
Linux' softirqs got phased out 20+ years ago, no?
No
not afaik? and in any case the nt kernel's dpcs are still being used to this day
(well the executive's dpcs. nt things calling it separately
)
I seriously remember everything getting moved off softirqs and onto kernel work queues or similar.
I think it was even part of the Big Kernel Lock removal.
windows actually has something similar to that difference too
normal DPCs (similar to softirq) vs Threaded DPCs (similar to work queues)
The sweep was total IIRC. There should be no softirqs left in Linux.
Now there's tasklets
too
which are built on softirqs
If they're really still in there, I'm surprised I'm misremembering what went on.
A common problem in kernel development is controlling when a specific task should be done. Ker [...]
My (micro)kernel never had a Big Kernel Lock & whatever was wrong with softirqs to begin with. The message-passing -based IO/VFS layers also make asynchronous IO work & M:N threading via scheduler activations makes threading facilities for various programming languages actually work, too (tested with Haskell/ghc, though I should add more languages' runtimes).
ok so it's like WQ_BH then
i should probably implement threaded DPCs tbh (which are more like workqueues)
I will just have a generic workpool thing
thats also an option
ive been waffling back and forth between whether to implement a kernel thread pool or threaded DPCs
(threaded DPCs effectively act as a percpu thread pool with one worker iirc)
I'm not sure if it's exactly applicable to telix. Things run in microkernels' version of โuserspaceโ so a lot of things could be written in Haskell or Mercury or Idris2 or F# or F* or Scala3 or whatever.
yeah dpcs are only really applicable to stuff running in-kernel imo
so if youre doing microkernel stuff then it probably isnt as applicable
I'm kicking off a feature completeness kick with a bunch of filesystems and fancy network things and additional architectures.
I can probably push Xwayland + GNOME + Firefox up a fair amount. It would probably be good to have that to demo fairly early.
seems like the linux guys handled it
I literally just sent a 1 character patch and didnt say anything else 
zig language server
Oh yeah, LSP servers have trouble with projects that do funny things when they build.
impressive
i switched to zed like a week ago because it seemed to handle it better than emacs but now both dont like it
omfg I shouldve designed everything with cpu ids from the start ๐ญ
your own ids or the hardware ones?
well instead of passing around a *ke.Cpu is pass around an id now
so the big struct has been shattered
oh yeah no passing around an id is way better
well at least it builds now, so I can get back to amd64 stuff
mhm moving the directory structure around a bit:
arch/ -> ISA primitives (like cpuid, table formats etc)
platform/ -> platform-specific stuff (board etc)
then each subsystem has its own architecture subdirectory
bruh gdt init not ok
if youre using an "m" constraint to set the gdtr then thats the problem, if not then id need to look at it to see whats up
no I fixed it
oh pog
i forgot to pack the struct
I dont get the difference between packed and extern tho
packed is bitfields
extern is c abi
theres an open proposal to rename packed to bitpacked because that would actually make sense
align(1) after every field
bruh what
theres no keyword or simple option for it
I think packed works tho
packed requires it to be backed by an int
bruhburhbruhbruhbrub
so if its too big or the abi cant handle the int it dies
what you want here is extern + manually set field alignment
that's ass
nice
byte-packed structs are def on my wishlist for zig
esp once they rename the current packed to bitpacked (which was accepted at some point) so the term wont be as overloaded
mhm do I do zig comptime magic to generate the interrupt handlers
i used to do that
swapped to doing it with asm macros in an asm file just because it was a bit more readable
idk GAS macros
i do still generate my isr_common with comptime tho
like in nasm i can just do:
%assign i 32
%rep 224
INTERRUPT_NOERR i
%assign i i+1
%endrep
idk if gas lets me do that
it does but its a bit more fiddly
so before i learned how to do that i used comptimePrint etc to generate asm for the handlers at comptime
.rept 224
INTERRUPT_NOERR %32+\+
.endr
.global __interrupt_vectors
__interrupt_vectors:
.rept 256
.quad isr\()\+
.endr
``` lmao
amazing syntax as always
yeah
lesgo we have interrupts now
pog
tfw i have fucking 600 lines log ring but no interrupts 
lmao
oh btw idk if you plan to do aarch64 but if you are, what interrupt controller(s) are you planning on supporting?
I have no idea
I know nothing about arm
I know there's a thing called the GIC tho
yeah
but theres five different versions and for the most part they all behave very different
i do have a raspi, could be cool to target it even tho it's not well documented
or I could get one of those open source boards
apart from 4 which is back-compatible with 3
and i think 3 is sorta back-compatible with 2?
well I'm gonna do it per-board
but afaik 5 doesnt have any back compat and noone talks about 1 lmao
well if whatever board you end up with has GIC version 3 or 4 i got you covered for helping to understand that shit
i think mine is one of if not the only hobby os to have support for MSIs on the gic v3/4
so i got you covered there for the knowledge lmao
i dont have ascii art either
i think this is enough info for now but eventually it'll call into the generic panic thing
which shows current thread, ipl, etc
mhm
trick i do for that btw is to define a format function on the interrupt frame type and then i can pass a pointer to it through to the main arch-independent panic handler and because the type is still comptime known (depending on target ofc) itll call the format function
I just realized I cant do the addrspace(.gs) trick
very sad
since for zig *T != *addrspace(.gs) T so things like self functions dont work
ugh I think I need a self pointer or rdgsbase
I kinda stole your log ring, if that's okay
printk_ringbuffer documentation was an interesting read
did you finish implementing it
yes
where
bruh ok I had to do this:
pub inline fn percpu_ptr(variable: anytype) @TypeOf(variable) {
const offset = asm volatile ("mov %%gs:(%%rax), %[out]"
: [out] "=r" (-> u64),
: [self_off] "{rax}" (&cpu_self_offset),
);
return @ptrFromInt(offset +% @intFromPtr(variable));
}
no performance gain at all for using percpu stuff like that since I cant do gs loads/stores directly

I guess I could use rdgsbase if the machine supports it but it might just be faster to do that ^
Wouldn't it be easier to convert the c version
Or did u want his one specifically
tbh I forgot you had it as well 
and mine supports neither 
mine internally would support cont but i dont have the exposed api for it
The api is the easy part
yeah i just never wrote it lmao
what's that?
pr_cont?
yeah
lets you extend a message
ah
as long as its the latest written message
Print continued
Yeah
@languid canyon do you think there's a better way to do this
Not in zig
so there is a stupid workaround possible
which is to make the self-parameter to methods on percpu stuff anytype
and then itll use generics to instantiate for both gs and normal pointers
the problem with that is you lose literally all lsp type hints which makes the development side a pita
but it does work
F
๐
oh btw @fallen bobcat zig merged llvm backend incremental support
they dont have state saving yet so it only applies to if you use the --watch flag on the build process to have it watch for changes on the filesystem but it is there
Yeah
First real major project contribution
mention in your CV 
did they co-author it?
I've been wanting to work on a BSD for a while but not sure which or what to do
I asked on the netbsd IRC and they gave me stuff that was already done
Congrats!
Yeah I think so
I don't use that, also does that imply incremental linking or not
Like does it make linking faster
im not sure about linking, this was mostly the llvm emit side, but i think it will might do incremental linking too
Idk I think what's slow for me is linking
mhm
for me its def the emit so itll help there (albeit itll help more once they get state saving)
mhm
?
should I add a license header thingy?
idk you do you but I just think it's kinda crazy you 1:1 rewrite a piece of code and add a small "based on XXX" lol
oh sorry
you do what you want, I know you and youre not using for commercial purposes or something
I dont think this counts as licensed code since its rewritten tho
I don't really know licenses
I just find it funny 
I'll include a permalink
yeah just a link is fine
done
what kind are you planning to make?
for now I will do a simple freelist but I had a plan in mind to add per-cpu lists on top of that
it involved having one global "bucket" freelist which is a list of lists of 64 pages each
what about a buddy :P
and every cpu has like 4 local buckets
you allocate and free from buckets[0]
when you need to drain you shift all buckets down
it's basically magazines
I have no need for that
and a buddy allocator sucks anyway
I like it quite a bit
the buddy allocator in linux infamously sucks
if I need higher than one page i can do an llfree-like thing
don't you need to allocate more than one page frequently if the mmu doesn't support 4k pages
what kinda architecture doesnt support 4k pages
yeah really
it's an architectural thing
id have to double check
not a platform thing
arm supports a couple different granule sizes tbf
as the base level
but i think 4k is required to be supported
good to know
4kb is pretty much the standard for all archs nowadays
yeah
you could argue 16k would make sense nowadays
i would be unsurprised if windows used the 64k granule size if supported since like everything in windows's vmm uses 64k units if possible but idk
I think I've written my vmm in a way that it doesn't care if the page size is 4k or not
of course I haven't tested it in such a way :P
mhm well I need to implement my architecture abstraction first
I think I'm not gonna do anything fancy for now and just copy what I already ahve
mine is a mess
but eventually I will move to a cursor-like API over ptes
mines kinda meh abstraction wise but it works
requires the arch pte type to have an init function with a certain signature for setting up a valid pte and requires the page number to be accessible as the page field of a packed union in the addr field of the pte
oh fuck im gonna have to rewrite kmem
fun
I'm kinda tempted to move to a lock-free thing like mimalloc tho
lmao I found my old OS and it straight up gdt init... fail
limine v2.x was beautiful
[ FATAL] GDT init... FAIL
when limine was actually good ๐
ugh I feel like the "bump alloc from largest entry" thing is so hacky
what if it's a bunch of small memory entries
Lmao


