shkwve: an incomplete kernel/maybe bootloader/UEFI thing/maybe something else if i get bored again | Operating System Development | Page 5

untold basin Mar 5, 2025, 12:01 PM

#

okay i have bar alloc

#

time for irq routing

untold basin Mar 5, 2025, 3:23 PM

#

okay im setting up virtio-blk

#

time to set up virtqueues

untold basin Mar 5, 2025, 4:27 PM

#

hmm does virtio execute operations in order?

trim seal Mar 5, 2025, 4:39 PM

#

i dont think it guarantees ordering 🤔

untold basin Mar 5, 2025, 4:44 PM

#

ah

#

thats fine i think

untold basin Mar 5, 2025, 5:27 PM

#

okay i have a semi working virtio driver!

#

time to make it async (🚀🚀🚀)

#

okay lets figure out interrupts then ig

untold basin Mar 6, 2025, 10:02 AM

#

okay now i have a queue dpc

#

so hmm

#

should i write a shitty driver which submits at most one operation to the device

#

or should i write a good driver which can pipeline them

#

yeah no ill just put a lock around each virtqueue

#

i probably will have pipelined ops for xhci though

rose sonnet Mar 6, 2025, 10:59 AM

#

a good driver is not that much work if you use coroutines meme

zenith lion Mar 6, 2025, 11:00 AM

#

nooo

#

static PITUST: u128; is no more nooo

#

or const

untold basin Mar 6, 2025, 1:09 PM

#

zenith lion or const

constexpr is const

#

:^)

untold basin Mar 6, 2025, 1:09 PM

#

rose sonnet a good driver is not that much work if you use coroutines <:meme:575445094589661...

lol yes it is

#

but i wrote a bad driver that uses coroutines

#

and it works just fine

untold basin Mar 6, 2025, 1:09 PM

#

untold basin lol yes it is

~~i'd probably have to use at least one or two more semaphores, and that sounds hard~~

untold basin Mar 6, 2025, 1:53 PM

#

okay now i have handling for multiple virtqueues maybe

#

okay now i have a horrible virtio-console """driver"""

#

okay lets abstract it all out

untold basin Mar 6, 2025, 2:47 PM

#

virtio-console, with a semi not shitty driver!

#

async, too

#

so now i guess i could do blk

#

or i could do net

#

okay, now i have a network device i think

untold basin Mar 6, 2025, 3:17 PM

#

okay thats not quite right

#

(0xcc is what i fill new memory allocations with)

zenith lion Mar 6, 2025, 3:18 PM

#

untold basin constexpr is const

but your old username wasn't constexpr

untold basin Mar 6, 2025, 4:49 PM

#

okay now i have my shitty buffer wrapper actually working

untold basin Mar 8, 2025, 5:51 PM

#

hmm, im not sure how to design my network stack

trim seal Mar 8, 2025, 6:37 PM

#

In a bootloader?

untold basin Mar 8, 2025, 6:37 PM

#

yes

#

im writing a good bootloader!

#

and that includes pxe boot

vapid echo Mar 8, 2025, 6:38 PM

#

are you gonna do qr code booting

untold basin Mar 8, 2025, 6:38 PM

#

vapid echo are you gonna do qr code booting

now that you said it

vapid echo Mar 8, 2025, 6:38 PM

#

ive already figured out how to do it

trim seal Mar 8, 2025, 6:38 PM

#

untold basin and that includes pxe boot

Well unless you are the firmware uefi has helpers for that

vapid echo Mar 8, 2025, 6:38 PM

#

i had code for limine

untold basin Mar 8, 2025, 6:38 PM

#

"scan this qr code and upload binary to boot"

untold basin Mar 8, 2025, 6:38 PM

#

trim seal Well unless you are the firmware uefi has helpers for that

i am in fact the firmware

#

and the uefi network stack is shit

trim seal Mar 8, 2025, 6:38 PM

#

☠️

open osprey Mar 8, 2025, 9:08 PM

#

untold basin hmm, im not sure how to design my network stack

look at STREAMS

untold basin Mar 9, 2025, 9:47 AM

#

open osprey look at STREAMS

yeah im not sure if i like the streams-like design

#

the other thing is i only care about support for like 3 protocols total

#

udp, tcp, icmpv4/v6

untold basin Mar 9, 2025, 5:25 PM

#

okay i came up with a new way to do my network stack

open osprey Mar 9, 2025, 5:25 PM

#

untold basin okay i came up with a new way to do my network stack

Tell us all about it

untold basin Mar 9, 2025, 5:26 PM

#

open osprey Tell us all about it

i came up with this ultra cursed network packet building interface

#

        net::Packet p = net::build([&] <net::BuildPhase phase> (net::PacketBuilder<phase>& b) {
            // virtio
            b.u8(0); // flags
            b.u8(0); // gso type
            b.le16(0); // hedaer length
            b.le16(0); // gso size
            b.le16(0); // checksum start
            b.le16(0); // checksum offset

            // ethernet
            b.ether(net::EthAddr::broadcast);
            b.ether(ether);
            b.be16(cast<u16>(net::EtherType::IP));

            // ip
            IP ip{net::IPProto::UDP, net::IPv4Addr{255, 255, 255, 255}, net::IPv4Addr{0, 0, 0, 0}};
            ip.write(b);

            // udp
            usize len = b.payladSize();
            b.be16(68); // sport
            b.be16(67); // dport
            b.be16(len); // len
            b.be16(0); // checksum

            // test payload
            b.cstring("hello, world!"_s);
        });

#

something like this

#

and then i'll have higher-level interfaces for the higher-level protocols (udp, tcp, icmp), i think

#

then i do two passes, first to get the length

#

and then to write all the fields

trim seal Mar 9, 2025, 5:35 PM

#

truly revolutionary

open osprey Mar 9, 2025, 9:04 PM

#

untold basin i came up with this ultra cursed network packet building interface

but what if instead you allocated an mblk and set the b_wptr to base + maximum size of virtio, ethernet, ip, udp/tcp headers, and then sent it through a STREAMS stack such that each module subtracted its header's size from b_wptr?

untold basin Mar 9, 2025, 9:05 PM

#

open osprey but what if instead you allocated an mblk and set the b_wptr to base + maximum s...

hmm

#

the problem is calculating checksums

untold basin Mar 10, 2025, 12:42 PM

#

okay wtf somehow i have 160k of code

#

which is weird

#

hmm okay i think a lot of that is stack

#

its okay i think

#

thats my fdt::Node::is_compatible function

#

okay so some of it was -ffreestanding

#

now i have incompatible definitions of mem*

#

oh, oh no

#

my iterator wrappers have very bad codegen it seems

#

okay ill redo it then

#

to work less like rust and more like something approximating good codegen

untold basin Mar 10, 2025, 2:09 PM

#

i wonder how much code size i'd save if i compiled out all of the dchecks

#

like 4k

#

hmm i think i might be failing even before virtio-net

#

compiling out dchecks makes it crash earlier

#

hmm maybe i should do ubsan

#

okay i am hitting ubsan

#

trap: type mismatch

#

lmao what

#

also thats a wrong decode i think

#

so the first one is a function type check

#

wtf

#

specifically in the event claim function

#

oh fun!

#

llvm bug :^)

#

im just adding more and more checks and hardening at this point lol

#

i have SSP too now

#

doesn't trip

#

same global corruption bug

#

if this is another llvm bug istg

#

i think my allocator is fucked

sacred spire Mar 10, 2025, 3:13 PM

#

Duality of man

untold basin Mar 10, 2025, 3:15 PM

#

now i'm getting PointerOverflow ubsan traps

#

it's from a vector push?

#

oh its a copy to null

#

shit

#

ah

#

zero length memcpies are broken

#

okay more memory corruption

#

okay now i hit a FunctionTypeMismatch

#

sounds like more heap corruption tbh

#

okay i added a vtable check

#

(inserted a magic cookie inside of the vtable and started checking it before doing calls)

#

maybe my promise type is broken?

#

    vf->vmspace.ref()._authenticateVtable();
    Promise<> _detached = [tp = vf->transport, vma = vf->vmspace] () -> Promise<> {
        vma.ref()._authenticateVtable();
        DMAMemory mem = vma->allocDMAMemory(4096).unwrap();
        while (true) {
            co_await tp->submit(0, {virtio::Element{mem.physAddr, 4096, virtio::Dir::DeviceToHost}});

            parsePkt(Span{cast<const u8*>(mem.virtAddr), 4096});
        }
    }();
``` the second call to _authenticateVtable dchecks

#

yeah okay i can repro under addresssanitizer

#

stack uaf

#

okay i think i dont get c++ coroutines, part N+1

#

fucking hell

#

reads my own code again

#

what the actual fuck did i write

#

and why

#

oh, OH

#

what the actual fuck are these semantics lmao

#

wonderful i think i have to rewrite my promise type

#

and my blocking run

#

and add a detached run function too i think

untold basin Mar 10, 2025, 5:27 PM

#

okay i think im gonna try reading the old new thing and see if that helps me understand this crap

rose sonnet Mar 11, 2025, 1:44 AM

#

untold basin what the actual fuck are these semantics lmao

what's bad about deciding whether to delete self depending on if you suspend or not

untold basin Mar 11, 2025, 8:23 AM

#

rose sonnet what's bad about deciding whether to `delete` self depending on if you suspend o...

huh what?

#

is that how it's meant to work?

#

also i think now my allocator is broken lmao

untold basin Mar 11, 2025, 8:42 AM

#

ugh i think i might have more memory corruption still

#

oh this time its in-place OOB?

#

im somewhat tempted to port KASAN

untold basin Mar 11, 2025, 9:11 AM

#

ah the magic option i really want is -mllvm -asan-mapping-offset

#

okay now i'm FARing 0x1fffa00008011704

#

okay now i have a sane asan base

#

still need to map it

#

ehh it sounds annoying

#

for fucks sake

#

i think i have uaf maybe?

#

okay i think my changes fixed the allocator

#

hopefully

#

maybe

rose sonnet Mar 11, 2025, 2:17 PM

#

untold basin is that how it's meant to work?

It depends on initial_suspend and final_syspend

#

the latter decides who frees the coroutine afaik

untold basin Mar 11, 2025, 2:18 PM

#

wtf

rose sonnet Mar 11, 2025, 2:19 PM

#

#

untold basin Mar 14, 2025, 10:32 AM

#

i have a mildly cursed idea

#

what if i added go code to shkwve

#

okay maybe i wont go that cursed

glad bough Mar 14, 2025, 11:22 AM

#

untold basin what if i added go code to shkwve

💀

rose sonnet Mar 14, 2025, 11:53 AM

#

why not ultrameme

untold basin Mar 14, 2025, 1:13 PM

#

no i dont think i will that sounds a bit too cursed even for me

rose sonnet Mar 14, 2025, 3:36 PM

#

I mean I was planning to use it in pmOS for example, though the bad compiler kinda spoiled it

#

(though not in kernel)

untold basin Mar 14, 2025, 5:06 PM

#

ah okay

untold basin Sep 4, 2025, 4:15 PM

#

okay

#

im going back to shkwve

#

so far my progress is this

#

mapped into virtual memory in higher half

#

and full W^X

#

(there is a cpu bit to force enable w^x)

#

also i think my allocator is a bit fucked lol

vapid echo Sep 4, 2025, 11:08 PM

#

What do the log signs mean

#

Like ~

trim seal Sep 5, 2025, 12:29 AM

#

probably info/warn/error

vapid echo Sep 5, 2025, 12:35 AM

#

ah I guess ~ is debug now that i think about it

untold basin Sep 5, 2025, 7:38 AM

#

vapid echo ah I guess ~ is debug now that i think about it

yeah

untold basin Sep 5, 2025, 7:38 AM

#

trim seal probably info/warn/error

yep

untold basin Sep 5, 2025, 4:15 PM

#

my nvme is now nvmeing

#

and i need to think of a good way of implementing disk caches

jade dirge Sep 5, 2025, 4:24 PM

#

Bootloader page cache meme

untold basin Sep 5, 2025, 4:45 PM

#

of course!

#

i mean i already have a dma buffer allocator

#

and virtual memory

#

and a complement of synchronization primitives that would be acceptable for many full OSes

#

including something kinda like turnstiles ish

#

but more like parking_lot and go semaphores

#

including an entirely original and extremely unique API: ```rs
pub unsafe fn park(addr: usize) -> UnparkToken;
pub unsafe fn park_until(
addr: usize,
deadline: Instant,
) -> Result<UnparkToken, code!(Status::TIMEOUT)>;
pub unsafe fn unpark_one(
addr: usize,
callback: impl FnOnce(UnparkOneResult) -> UnparkToken,
) -> UnparkOneResult;
#[derive(Debug, Clone, Copy)]
pub struct UnparkOneResult {
pub unparked: bool,
pub be_fair: bool,
pub more_in_queue: bool,
}

#

and i have a one byte mutex based on that ~~hmm, where have i heard of that idea before~~

untold basin Sep 5, 2025, 4:50 PM

#

untold basin and a complement of synchronization primitives that would be acceptable for many...

semaphores, bounded queues, Event, sleeping mutexes with timeouts

#

and none of my sync primitives allocate, obviously

#

well ok thats not quite true

#

i use a third party queue library for my scheduling queue which does actually allocate

#

but i still have an allocation-free timer heap

trim seal Sep 5, 2025, 5:16 PM

#

untold basin including an entirely original and extremely unique API: ```rs pub unsafe fn par...

What does this do?

untold basin Sep 5, 2025, 5:17 PM

#

trim seal What does this do?

the same as the functions with the same names in parking_lot_core

parking_lot_core - Rust

This library exposes a low-level API for creating your own efficient synchronization primitives.

vapid echo Sep 5, 2025, 6:39 PM

#

untold basin i use a third party queue library for my scheduling queue which does actually al...

Scheduler in bootloader

#

Wtf

#

Honestly just turn this into a kernel

untold basin Sep 5, 2025, 6:39 PM

#

vapid echo Scheduler in bootloader

yes its a great idea

#

yea i mean its pretty much a bad kernel at this point

#

except it also has a funky self relocating bootloader combined into its image

#

and i want to make it work as a real bootloader at some point

vapid echo Sep 5, 2025, 6:40 PM

#

I should read about parking lot stuff

#

Is it another kind of lock?

untold basin Sep 5, 2025, 6:40 PM

#

no

vapid echo Sep 5, 2025, 6:40 PM

#

Or a lock backend

untold basin Sep 5, 2025, 6:40 PM

#

read https://webkit.org/blog/6161/locking-in-webkit/

WebKit

Locking in WebKit

We recently replaced all spinlocks, OS-provided mutexes, and condition variables in WebKit with the new WTF::Lock and WTF::Condition — these new primitives have some cool properties.

vapid echo Sep 5, 2025, 6:40 PM

#

Alright thanks

untold basin Sep 5, 2025, 6:41 PM

#

its mostly a userspace futex

trim seal Sep 5, 2025, 6:47 PM

#

vapid echo Honestly just turn this into a kernel

^

untold basin Sep 5, 2025, 6:51 PM

#

but thats boooring

#

plus it implies i have to care about things like actually having a userspace at some point

#

and it means i cant justify writing code with zero concern for OOM conditions under the guise of "yeah its fine its just a bootloader anyway"

trim seal Sep 5, 2025, 6:53 PM

#

Lol

untold basin Sep 5, 2025, 6:53 PM

#

also i have some cool ideas that dont make sense in a kernel

#

like for example, i had this idea to make an AML virtual machine which can be serialized back into functioning AML

#

unless i just say that my kernel has kexec support 🙃

trim seal Sep 5, 2025, 7:00 PM

#

untold basin like for example, i had this idea to make an AML virtual machine which can be se...

Damn

untold basin Sep 5, 2025, 7:01 PM

#

it would not be EASY

#

but it would be COOL

#

and i dont think it would actually be THAT hard

trim seal Sep 5, 2025, 7:10 PM

#

Like an AML interpreter in aml?

untold basin Sep 5, 2025, 7:10 PM

#

no like

#

i write an AML interpreter which can turn its entire state into aml

trim seal Sep 5, 2025, 7:13 PM

#

Hm

untold basin Apr 9, 2026, 8:47 PM

#

okay im resurrecting this

#

shkwve is backkkkkkk kinda

#

shkwve: an incomplete kernel/maybe bootloader/UEFI thing/maybe something else if i get bored again

tawdry surge Apr 9, 2026, 8:48 PM

#

is there a new repo?

untold basin Apr 9, 2026, 8:48 PM

#

no

#

i didnt do the whole version control thing yet

#

and i need to unhardcode some paths first

#

right now i have this hardcoded lmao

#

but i also have epic stacktraces (only for kasan errors, i didnt wire it up for panics lmao)

#

(this is a random crash i inserted on purpose to show how cool my stacktraces are)

oh yeah also i got usermode code executing

#

trim seal Apr 9, 2026, 8:51 PM

#

bro has usermode in a bootloader

untold basin Apr 9, 2026, 8:51 PM

#

no its not a bootloader anymore

#

im planning to maybe do kexec at some point but im going full userspace now

#

im gonna try microkerneling

trim seal Apr 9, 2026, 8:51 PM

#

lol

#

i like the direction change

untold basin Apr 9, 2026, 8:52 PM

#

the name "shkwve" is really just "whatever project im working on right now" ngl

untold basin Apr 9, 2026, 8:52 PM

#

trim seal bro has usermode in a bootloader

apple has that too

#

and microsoft has that on the xbox one

#

well actually idk if its a thing in the xbox one bootloader, i know its a thing in the xbox one bootrom

trim seal Apr 9, 2026, 8:53 PM

#

well the apple thing literally looks like macos

#

which it is probably?

untold basin Apr 9, 2026, 8:53 PM

#

trim seal well the apple thing literally looks like macos

wdym

#

no like

trim seal Apr 9, 2026, 8:53 PM

#

like when u pick a boot drive

untold basin Apr 9, 2026, 8:53 PM

#

apple uefi literally does usermode

trim seal Apr 9, 2026, 8:53 PM

#

it looks like mac os ui

untold basin Apr 9, 2026, 8:53 PM

#

trim seal like when u pick a boot drive

i think its just reconstructed

trim seal Apr 9, 2026, 8:54 PM

#

untold basin apple uefi literally does usermode

for what?

untold basin Apr 9, 2026, 8:54 PM

#

trim seal for what?

hardening

vapid echo Apr 9, 2026, 8:54 PM

#

untold basin but i also have epic stacktraces (only for kasan errors, i didnt wire it up for ...

oooh KASAN

untold basin Apr 9, 2026, 8:54 PM

#

option roms run in usermode iirc

vapid echo Apr 9, 2026, 8:54 PM

#

Nice

untold basin Apr 9, 2026, 8:54 PM

#

vapid echo oooh KASAN

to date, it has found 0 bugs

trim seal Apr 9, 2026, 8:54 PM

#

doesnt uefi require all memory identity mapped

vapid echo Apr 9, 2026, 8:54 PM

#

Is it proper compiler KASAN or just hardening features

untold basin Apr 9, 2026, 8:54 PM

#

vapid echo Is it proper compiler KASAN or just hardening features

yes its proper kasan

vapid echo Apr 9, 2026, 8:55 PM

#

untold basin to date, it has found 0 bugs

Yes but it's cool to have

#

I might try implementing it

#

Didn't know rust had it

untold basin Apr 9, 2026, 8:55 PM

#

untold basin to date, it has found 0 bugs

except bugs in the kasan impl

untold basin Apr 9, 2026, 8:55 PM

#

vapid echo Didn't know rust had it

it kinda does

#

its very janky

tawdry surge Apr 9, 2026, 8:55 PM

#

why is it janky?

vapid echo Apr 9, 2026, 8:55 PM

#

isn't rust supposed to be safe

trim seal Apr 9, 2026, 8:55 PM

#

no need for one since its memory safe trl

tawdry surge Apr 9, 2026, 8:55 PM

#

isn't it just done by generic llvm passes

tawdry surge Apr 9, 2026, 8:56 PM

#

vapid echo isn't rust supposed to be safe

unsafe code?

#

;D

vapid echo Apr 9, 2026, 8:56 PM

#

nooo

untold basin Apr 9, 2026, 8:56 PM

#

tawdry surge why is it janky?

it interacts very poorly with inlining

trim seal Apr 9, 2026, 8:56 PM

#

ur supposed to add a safety comment which makes it safe

untold basin Apr 9, 2026, 8:56 PM

#

#[cfg_attr(feature = "kasan", inline(never))]
#[cfg_attr(feature = "kasan", sanitize(address = "off"))]
pub unsafe fn write_noasan<T>(ptr: *mut T, value: T) {
    if cfg!(feature = "kasan") {
        let src = ManuallyDrop::new(value);
        unsafe { memcpy(ptr as *mut (), &raw const src as *const (), size_of::<T>()) };
    } else {
        unsafe { *ptr = value };
    }
}

untold basin Apr 9, 2026, 8:56 PM

#

trim seal ur supposed to add a safety comment which makes it safe

wait, shit

#

i forgot

#

now its good

untold basin Apr 9, 2026, 8:57 PM

#

untold basin ```rs #[cfg_attr(feature = "kasan", inline(never))] #[cfg_attr(feature = "kasan"...

idk if this kind of write also breaks

#

but like a bunch of the pointer operations end up calling helpers

#

which ends up breaking

vapid echo Apr 9, 2026, 8:57 PM

#

huh zig doesn't even have KASAN apparently

untold basin Apr 9, 2026, 8:57 PM

#

because the helpers dont have kasan disabled

untold basin Apr 9, 2026, 8:57 PM

#

vapid echo huh zig doesn't even have KASAN apparently

it surely has normal asan

tawdry surge Apr 9, 2026, 8:57 PM

#

it's weird that you need inline(never), on Managarm we compile with LTO and we never ran into an issue where nosanitize was not respected

vapid echo Apr 9, 2026, 8:58 PM

#

untold basin it surely has normal asan

I think it does, but they do a lot of it through their debug allocator too

untold basin Apr 9, 2026, 8:58 PM

#

tawdry surge it's weird that you need inline(never), on Managarm we compile with LTO and we n...

a c++ pointer load lowers to an llvm load

#

a rust pointer load lowers to a call to a helper function which does some checks

tawdry surge Apr 9, 2026, 8:58 PM

#

ah i see

untold basin Apr 9, 2026, 8:58 PM

#

that function is in libcore, and has sanitizers forced on

untold basin Apr 9, 2026, 8:59 PM

#

vapid echo I think it does, but they do a lot of it through their debug allocator too

my kasan isnt actually kasan

#

its normal addresssanitizer

vapid echo Apr 9, 2026, 8:59 PM

#

ah

untold basin Apr 9, 2026, 8:59 PM

#

built with this very simple set of flags

vapid echo Apr 9, 2026, 8:59 PM

#

I know there's bespoke KASAN too

tawdry surge Apr 9, 2026, 8:59 PM

#

but that's essentially KASAN

untold basin Apr 9, 2026, 8:59 PM

#

tawdry surge but that's essentially KASAN

i have -asan-mapping-scale=4 and a different offset

tawdry surge Apr 9, 2026, 8:59 PM

#

the difference to userspace ASAN is that the mapping offset is known a priori

trim seal Apr 9, 2026, 9:00 PM

#

untold basin i have -asan-mapping-scale=4 and a different offset

what does that do?

tawdry surge Apr 9, 2026, 9:00 PM

#

other than that, I don't think there is a difference

untold basin Apr 9, 2026, 9:00 PM

#

trim seal what does that do?

its more efficient

#

well

tawdry surge Apr 9, 2026, 9:00 PM

#

it halves the size of the shadow mapping compared to the default

untold basin Apr 9, 2026, 9:00 PM

#

yeah

#

it means you need half the amount of kasan shadow

trim seal Apr 9, 2026, 9:00 PM

#

so the checks are more coarse?

untold basin Apr 9, 2026, 9:00 PM

#

ish

#

in practice no

#

with kasan enabled, i put 16 bytes between each zone (~normal heap) allocation anyway

#

and the compiler does the same thing for stack stuff

untold basin Apr 9, 2026, 9:02 PM

#

untold basin built with this very simple set of flags

most of these dont really help much

vapid echo Apr 9, 2026, 9:02 PM

#

I only do poisoning when enabled and I think it's probably more than enough lol

untold basin Apr 9, 2026, 9:02 PM

#

i dont think any of the -asan-opt-* flags do anything useful

#

my kasan runtime is quite complicated in general too

because it can print those awesome errors with inline-aware stacktraces

#

also its c++, its like the one(ish) piece of c++ in my kernel binary

vapid echo Apr 9, 2026, 9:03 PM

#

Is that mostly dwarf

untold basin Apr 9, 2026, 9:03 PM

#

no

#

i have a custom format

#

thats easier to parse

#

which saves probably quite a lot over dwarf too

vapid echo Apr 9, 2026, 9:04 PM

#

How many languages do you have

#

Like 3?

#

I've never seen a pitust project not be a cursed mix of languages and I like it tbh

untold basin Apr 9, 2026, 9:05 PM

#

3 + assembly + a tiny amount of shell script

trim seal Apr 9, 2026, 9:06 PM

#

untold basin i have a custom format

how did u make clang output it

untold basin Apr 9, 2026, 9:06 PM

#

untold basin which saves probably quite a lot over dwarf too

my binary is like literally 93% dwarf lmao

untold basin Apr 9, 2026, 9:06 PM

#

trim seal how did u make clang output it

i didnt

#

i make rustc emit dwarf

#

and then i have an insane dwarf parser and converter in my build tooling

trim seal Apr 9, 2026, 9:06 PM

#

holy

tawdry surge Apr 9, 2026, 9:06 PM

#

actually a good solution

untold basin Apr 9, 2026, 9:06 PM

#

yes

tawdry surge Apr 9, 2026, 9:06 PM

#

that's how i'd also do it

untold basin Apr 9, 2026, 9:07 PM

#

if you dont want a binary thats 93% dwarf its basically the only viable approach i think

tawdry surge Apr 9, 2026, 9:07 PM

#

dwarf is pretty bad, especially if you want to use it on a hotpath (such as profiling)

trim seal Apr 9, 2026, 9:07 PM

#

have u checked how linux produces its orc stuff?

vapid echo Apr 9, 2026, 9:07 PM

#

And then you strip the binary?

#

what is orc?

untold basin Apr 9, 2026, 9:07 PM

#

no

trim seal Apr 9, 2026, 9:07 PM

#

vapid echo what is orc?

its unwind format

untold basin Apr 9, 2026, 9:07 PM

#

vapid echo And then you strip the binary?

the build process is a lot more complicated than that

vapid echo Apr 9, 2026, 9:08 PM

#

trim seal its unwind format

Even they don't use dwarf? Damn

trim seal Apr 9, 2026, 9:08 PM

#

they never did

#

they did frame pointer unwinding then added this thing

untold basin Apr 9, 2026, 9:08 PM

#

i mean for unwinding i just use frame pointers, i might do something like orc later

trim seal Apr 9, 2026, 9:08 PM

#

theres also a guesser unwinder on x86

untold basin Apr 9, 2026, 9:08 PM

#

im guessing its just an interval list of frame sizes?

vapid echo Apr 9, 2026, 9:09 PM

#

I'm probably gonna do dwarf because zig has a parser

trim seal Apr 9, 2026, 9:09 PM

#

untold basin i mean for unwinding i just use frame pointers, i might do something like orc la...

what do u use your format for then?

untold basin Apr 9, 2026, 9:09 PM

#

trim seal what do u use your format for then?

symbolizing

#

including inlines

trim seal Apr 9, 2026, 9:09 PM

#

ohhh

untold basin Apr 9, 2026, 9:09 PM

#

which is dwarf too

trim seal Apr 9, 2026, 9:09 PM

#

for symbolizing i just use kallsyms from linux

untold basin Apr 9, 2026, 9:09 PM

#

trim seal for symbolizing i just use kallsyms from linux

okay sure. but what if you want to demangle

#

rust demangling is not fun

vapid echo Apr 9, 2026, 9:09 PM

#

You don't mangle :)

trim seal Apr 9, 2026, 9:09 PM

#

isnt there a function which allows you to demangle

#

without allocations

#

at runtime

untold basin Apr 9, 2026, 9:10 PM

#

yeah in a crate

#

in a rust crate

trim seal Apr 9, 2026, 9:10 PM

#

is that bad?

untold basin Apr 9, 2026, 9:10 PM

#

so id have to ship a demangler

#

well i wrote this when i wanted to do kasan

#

and the kasan runtime is very intentionally not written in rust

trim seal Apr 9, 2026, 9:10 PM

#

why?

untold basin Apr 9, 2026, 9:10 PM

#

trim seal why?

you dont want to be using kasan-instrumented code anywhere within the kasan instrumentation impl

trim seal Apr 9, 2026, 9:11 PM

#

cant u just mark it as non instrumented

untold basin Apr 9, 2026, 9:11 PM

#

no

#

i mean yes i can mark my thing as not instrumented

#

but all of the things it calls will still get instrumented as normal

untold basin Apr 9, 2026, 9:11 PM

#

untold basin you dont want to be using kasan-instrumented code anywhere within the kasan inst...

so it needs a separate compile session and flags

#

and you cant really mix two different rust static libraries and have them work, plus thats really bad for code size

#

the crate also doesnt expose any of the info, i actually do a bunch of postprocessing to hide frames that dont matter

untold basin Apr 9, 2026, 9:12 PM

#

untold basin the crate also doesnt expose any of the info, i actually do a bunch of postproce...

and i also format some types specially

untold basin Apr 9, 2026, 9:13 PM

#

untold basin the crate also doesnt expose any of the info, i actually do a bunch of postproce...

so for example all of core::ub_checks doesnt show up

trim seal Apr 9, 2026, 9:13 PM

#

damn rust really is a pain to work wth

untold basin Apr 9, 2026, 9:13 PM

#

and a whole bunch of my kernel's helper APIs dont show up either

#

so for example sk_kern::mmio::{read, write}{8, 16, 32, 64} will never be visible on a stacktrace because i check for that and hide them

trim seal Apr 9, 2026, 9:14 PM

#

why?

untold basin Apr 9, 2026, 9:14 PM

#

trim seal why?

size + those functions are inline(always) and they take the address from the caller

#

and it looks ugly in a stacktrace tbh

untold basin Apr 9, 2026, 9:15 PM

#

untold basin size + those functions are inline(always) and they take the address from the cal...

so really the caller is at fault

#

if i want to know what exactly caused the issue i have a decompiler

trim seal Apr 9, 2026, 9:15 PM

#

isnt it useful to know it was mmio that caused a fault?

#

instead of having to read a disassembly

untold basin Apr 9, 2026, 9:15 PM

#

well it cant cause kasan errors so

untold basin Apr 9, 2026, 9:16 PM

#

trim seal instead of having to read a disassembly

ehh decompiler views are usually pretty good

#

and without a file name/line number its pretty hard to tell anyway

trim seal Apr 9, 2026, 9:16 PM

#

untold basin well it cant cause kasan errors so

if it cant cause kasan errors then why hide it?

untold basin Apr 9, 2026, 9:16 PM

#

trim seal if it cant cause kasan errors then why hide it?

size

#

each entry takes a few bytes

trim seal Apr 9, 2026, 9:17 PM

#

how many symbols do u have lol

untold basin Apr 9, 2026, 9:17 PM

#

not a lot lmao

vapid echo Apr 9, 2026, 9:17 PM

#

Infy one thing I wanna try is your fallible instructions

#

That's cool

#

pitust are you gonna do that too

trim seal Apr 9, 2026, 9:17 PM

#

yeah imo that shit is super convenient

untold basin Apr 9, 2026, 9:17 PM

#

90kb of unwind data total

#

less when its compressed (and my kernel is compressed, its self-decompressing)

untold basin Apr 9, 2026, 9:18 PM

#

vapid echo Infy one thing I wanna try is your fallible instructions

which ones?

vapid echo Apr 9, 2026, 9:18 PM

#

MSR ones I think?

trim seal Apr 9, 2026, 9:18 PM

#

i currently have abortable instructions for MSRs and try_memcpy

untold basin Apr 9, 2026, 9:18 PM

#

uhhhhh is that even reliable

untold basin Apr 9, 2026, 9:19 PM

#

trim seal i currently have abortable instructions for MSRs and try_memcpy

the fuck do you have try_memcpy for

#

lol

trim seal Apr 9, 2026, 9:19 PM

#

currently only used in unwind code

untold basin Apr 9, 2026, 9:19 PM

#

oh

vapid echo Apr 9, 2026, 9:19 PM

#

untold basin the fuck do you have try_memcpy for

User as well?

untold basin Apr 9, 2026, 9:19 PM

#

trim seal currently only used in unwind code

here is a fun function inside of my usercopy region

untold basin Apr 9, 2026, 9:19 PM

#

vapid echo User as well?

usercopy needs special shit though

trim seal Apr 9, 2026, 9:19 PM

#

yeah its just a building block so ill totally use it for copy_to_user also

vapid echo Apr 9, 2026, 9:20 PM

#

I'm not sure how it works

untold basin Apr 9, 2026, 9:20 PM

#

using that for copy to/from user is a bad idea

trim seal Apr 9, 2026, 9:20 PM

#

but it needs more work ofc, like enabling access to user memory etc

#

and validation

untold basin Apr 9, 2026, 9:20 PM

#

arm supremacy: i dont need to do validation

trim seal Apr 9, 2026, 9:20 PM

#

untold basin here is a fun function inside of my usercopy region

yeah thats basically that

trim seal Apr 9, 2026, 9:20 PM

#

untold basin arm supremacy: i dont need to do validation

why?

untold basin Apr 9, 2026, 9:21 PM

#

trim seal why?

arm has a "load as if you did it from usermode" instruction lmao

trim seal Apr 9, 2026, 9:21 PM

#

ohh

vapid echo Apr 9, 2026, 9:21 PM

#

You have to handle exceptions though? Do you have a list of fallible functions and on exceptions you check whether or not it was in that?

trim seal Apr 9, 2026, 9:21 PM

#

he probably has adhoc code to handle that specific one

vapid echo Apr 9, 2026, 9:21 PM

#

No I mean you

untold basin Apr 9, 2026, 9:21 PM

#

vapid echo You have to handle exceptions though? Do you have a list of fallible functions a...

if an exception is taken when PC is after arm64_usercopy_crit_start and before arm64_usercopy_crit_end, then ELR_EL1 (aka saved PC) is set to x8 and i return from the exception

trim seal Apr 9, 2026, 9:22 PM

#

vapid echo No I mean you

yea x86 code calls into generic handle_abortable_instruction

untold basin Apr 9, 2026, 9:22 PM

#

thats what the adr x8, .kasan_fail is for

trim seal Apr 9, 2026, 9:22 PM

#

just read the code lol

vapid echo Apr 9, 2026, 9:22 PM

#

Yeah ok fine meme

untold basin Apr 9, 2026, 9:22 PM

#

untold basin thats what the `adr x8, .kasan_fail` is for

    # Save x0 and x1
    sub sp, sp, 0x10
    stp x0, x1, [sp, 0x00]

    # Check if ELR_EL1 is in the usercopy region
    mrs x0, ELR_EL1
    adr x1, arm64_usercopy_crit_start
    cmp x0, x1
    b.lo 3f
    adr x1, arm64_usercopy_crit_end
    cmp x0, x1
    b.hs 3f

    # If so, fix stack and return to x8
    add sp, sp, 0x10
    msr ELR_EL1, x8
    eret
``` i have special asm for this even in the exception path

vapid echo Apr 9, 2026, 9:23 PM

#

Ah cool

untold basin Apr 9, 2026, 9:23 PM

#

does x86 guarantee that unknown MSRs fail with any specific exception

trim seal Apr 9, 2026, 9:23 PM

#

yeah, GPF

untold basin Apr 9, 2026, 9:24 PM

#

ah yes

#

okay neat

trim seal Apr 9, 2026, 9:24 PM

#

GPF is also used as -EINVAL

#

as like a bad input value for an MSR

untold basin Apr 9, 2026, 9:24 PM

#

wait wtf does your code do lmao

trim seal Apr 9, 2026, 9:24 PM

#

i mean x86

#

if u try to wrmsr something bad u get a gpf

#

even if the msr is known

#

so GPF is both bad input and/or unknown msr

untold basin Apr 9, 2026, 9:25 PM

#

ah i see

#

although i think that ```c
if (ai->flags & ABORTABLE_INSTRUCTION_ENOSYS_ON_ERROR)
registers_set_return_value(regs, ENOSYS);
else if (ai->flags & ABORTABLE_INSTRUCTION_EFAULT_ON_ERROR)
registers_set_return_value(regs, EFAULT);

trim seal Apr 9, 2026, 9:25 PM

#

why

untold basin Apr 9, 2026, 9:26 PM

#

idk maybe it makes more sense in your impl

trim seal Apr 9, 2026, 9:26 PM

#

all it does is reduce code duplication

untold basin Apr 9, 2026, 9:26 PM

#

i just handle this outside of the asm helper

#

and the entire mechanism is private to the arm64 port

#

oh yeah i have something that vaguely approximates a strong separation of generic vs per-isa code

trim seal Apr 9, 2026, 9:27 PM

#

yeah i mean all it does is it makes it so u dont need a wrapper

trim seal Apr 9, 2026, 9:27 PM

#

untold basin and the entire mechanism is private to the arm64 port

i think its useful to have it as a generic thing

#

at least looking at how it scales in linux

untold basin Apr 9, 2026, 9:28 PM

#

in my case i need a wrapper anyway

trim seal Apr 9, 2026, 9:28 PM

#

which i stole the thing from

trim seal Apr 9, 2026, 9:28 PM

#

untold basin in my case i need a wrapper anyway

because rust?

untold basin Apr 9, 2026, 9:28 PM

#

trim seal because rust?

no

#

because i want to handle dispatching faults to the right process manually

trim seal Apr 9, 2026, 9:29 PM

#

wdym by dispatching faults

tawdry surge Apr 9, 2026, 9:29 PM

#

imagine if there was a way for the arch to reject unsupported writes without an exception

untold basin Apr 9, 2026, 9:29 PM

#

trim seal wdym by dispatching faults

like if you had a page fault

tawdry surge Apr 9, 2026, 9:29 PM

#

intel apparently can't imagine this

untold basin Apr 9, 2026, 9:29 PM

#

riscv cant do that either i think

#

right?

trim seal Apr 9, 2026, 9:29 PM

#

tawdry surge intel apparently can't imagine this

yeah imo it was a mistake to make it like that

untold basin Apr 9, 2026, 9:29 PM

#

intel has nothing, arm has ldtr, riscv has ? (ofc depending on your extensions)

trim seal Apr 9, 2026, 9:30 PM

#

untold basin like if you had a page fault

wouldnt u still have high level code handle the e.g. copy_to_user failure?

untold basin Apr 9, 2026, 9:30 PM

#

trim seal wouldnt u still have high level code handle the e.g. copy_to_user failure?

ye but handling the two kinds of failure is extremely correlated

#

so its a bit of an architecture thing

trim seal Apr 9, 2026, 9:30 PM

#

like u handle copy to user errors and normal user faults in the same way?

untold basin Apr 9, 2026, 9:30 PM

#

no

tawdry surge Apr 9, 2026, 9:31 PM

#

it (riscv) can, it has write-any read-valid registers as well

untold basin Apr 9, 2026, 9:31 PM

#

i in general don't take [recoverable] kernel exceptions other than uaccess right now

trim seal Apr 9, 2026, 9:31 PM

#

ah ok

untold basin Apr 9, 2026, 9:31 PM

#

tawdry surge it (riscv) can, it has write-any read-valid registers as well

oh to system registers

untold basin Apr 9, 2026, 9:31 PM

#

untold basin i *in general* don't take [recoverable] kernel exceptions other than uaccess rig...

and user exceptions have really special code

tawdry surge Apr 9, 2026, 9:32 PM

#

recoverable kernel exceptions except for very few cases are a mistake meme

trim seal Apr 9, 2026, 9:33 PM

#

make a kernel where every kernel exception is recoverable

untold basin Apr 9, 2026, 9:33 PM

#

mine are only recoverable in uaccess

#

and in the smp trampoline

#

and i dont even have an error message otherwise

untold basin Apr 9, 2026, 9:33 PM

#

untold basin and in the smp trampoline

because arm moment

tawdry surge Apr 9, 2026, 9:34 PM

#

why does the smp trampoline need recoverable exceptions?

trim seal Apr 9, 2026, 9:34 PM

#

lol

untold basin Apr 9, 2026, 9:34 PM

#

tawdry surge why does the smp trampoline need recoverable exceptions?

well, memory isnt identity mapped

tawdry surge Apr 9, 2026, 9:35 PM

#

trim seal make a kernel where every kernel exception is recoverable

no, debug and single step from kernel mode should just cause an immediate boot drive erasure to make sure that developers don't do something stupid like that

untold basin Apr 9, 2026, 9:35 PM

#

and in general my translation is configured to writable implies execute never

#

and you cant execute from the direct map

#

you cant enable translation only for higher half

#

so if you don't take an exception, you are kinda stuck:

you need to enable translation
you need to be somewhere where after enabling translation, you will map to the same place in physmem
you need to be somewhere which is not covered by any page table
and the kernel is not identity mapped

tawdry surge Apr 9, 2026, 9:38 PM

#

i don't get it

untold basin Apr 9, 2026, 9:38 PM

#

well

#

the smp trampoline needs to be executable

tawdry surge Apr 9, 2026, 9:38 PM

#

you just identity map the page that the stub is located on

#

in the lower half

untold basin Apr 9, 2026, 9:38 PM

#

tawdry surge you just identity map the page that the stub is located on

thats annoying

tawdry surge Apr 9, 2026, 9:38 PM

#

in a specifically crafted address space

untold basin Apr 9, 2026, 9:38 PM

#

but yeah i guess that works too

trim seal Apr 9, 2026, 9:38 PM

#

so u manually fixup the pc of the cores when they start up or something?

untold basin Apr 9, 2026, 9:38 PM

#

you can also just intentionally take a prefetch abort

tawdry surge Apr 9, 2026, 9:38 PM

#

it's way less janky than catching a pf lol

untold basin Apr 9, 2026, 9:38 PM

#

trim seal so u manually fixup the pc of the cores when they start up or something?

no i set a special exception vector

trim seal Apr 9, 2026, 9:39 PM

#

crazy stuff

untold basin Apr 9, 2026, 9:39 PM

#

and then intentionally take either a prefetch abort or an undefined instruction

#

depending on how hard the core OoOs

tawdry surge Apr 9, 2026, 9:39 PM

#

we just treat it as a user page space except that we map it as a kernel page

untold basin Apr 9, 2026, 9:40 PM

#

the other annoying issue for me is that i dont really have a good allocator to use

#

allocator bringup occurs after i do smp

#

i have an early bootstrap bump allocator for smp, but that can't free

untold basin Apr 9, 2026, 9:41 PM

#

tawdry surge it's way less janky than catching a pf lol

actually is it

#

what i do is definitely simpler

tawdry surge Apr 9, 2026, 9:41 PM

#

well you need the infrastructure anyway if you want to run userspace :^)

untold basin Apr 9, 2026, 9:41 PM

#

tawdry surge well you need the infrastructure anyway if you want to run userspace :^)

well no

#

the userspace stuff only works after you have the memory allocator initialized

tawdry surge Apr 9, 2026, 9:42 PM

#

true. but there's also no need to do smp before allocation

untold basin Apr 9, 2026, 9:42 PM

#

uhhh kinda

#

i need to know at least how many CPUs there are

#

because my allocator needs to allocate per-cpu structures

#

i could split it up

trim seal Apr 9, 2026, 9:43 PM

#

u can kinda parse that info and not start cpus at the same time lmao

tawdry surge Apr 9, 2026, 9:43 PM

#

we enumerate CPUs before allocation but launch them afterwards

untold basin Apr 9, 2026, 9:43 PM

#

trim seal u can kinda parse that info and not start cpus at the same time lmao

yeah ofc. its more annoying tho

#

and there is zero benefit from doing so

untold basin Apr 10, 2026, 12:50 PM

#

okay and now im back to memory allocator hell

#

so i kinda want to allocate dma buffers

#

and like i could make them discontinous and say you cant allocate dma buffers larger than 1 page

#

hmm maybe i should just do that lol

untold basin Apr 13, 2026, 6:28 PM

#

ugh okay i decided to work on sync instead

#

and now im not sure how to implement sync lmao

#

i dont love the idea of taking a sleep lock

tawdry surge Apr 13, 2026, 6:39 PM

#

sync?

#

as in synchronization data structures?

#

or sync as in fsync()

untold basin Apr 13, 2026, 6:44 PM

#

tawdry surge as in synchronization data structures?

yeah this

#

not fsync lol im nowhere near disk drivers

tawdry surge Apr 13, 2026, 6:44 PM

#

what synchronization primitive do you want to implement?

#

you're writing in rust, right?

#

just do a block_on(future) :^)

untold basin Apr 13, 2026, 6:45 PM

#

lmao

tawdry surge Apr 13, 2026, 6:45 PM

#

that's what iretq did for thor-rs

untold basin Apr 13, 2026, 6:45 PM

#

futures are bad tho

#

they perform not great iirc

#

and they also dont match the rest of my model

untold basin Apr 13, 2026, 6:46 PM

#

tawdry surge what synchronization primitive do you want to implement?

well i was going for like a semaphore 🙃

#

or an MPSC queue or something

tawdry surge Apr 13, 2026, 6:46 PM

#

a semaphore sounds like a good start

#

and can basically be used as a primitive for everything else

untold basin Apr 13, 2026, 6:46 PM

#

i mean go does that

#

okay ill just go steal more code and ideas from go

#

// Puts the current goroutine into a waiting state and calls unlockf on the
// system stack.
//
// If unlockf returns false, the goroutine is resumed.
//
// unlockf must not access this G's stack, as it may be moved between
// the call to gopark and the call to unlockf.
//
// Note that because unlockf is called after putting the G into a waiting
// state, the G may have already been readied by the time unlockf is called
// unless there is external synchronization preventing the G from being
// readied. If unlockf returns false, it must guarantee that the G cannot be
// externally readied.
//
// Reason explains why the goroutine has been parked. It is displayed in stack
// traces and heap dumps. Reasons should be unique and descriptive. Do not
// re-use reasons, add new ones.

awesome

#

so yeah go unlocks the semaphore bucket lock on the scheduler stack

tawdry surge Apr 13, 2026, 6:59 PM

#

for what?

untold basin Apr 13, 2026, 6:59 PM

#

go "semaphores" are basically implemented as userspace futexes

#

and go does its own userspace scheduling

tawdry surge Apr 13, 2026, 7:00 PM

#

for low leveling blocking i'd just add Thread::park(), Thread::unpark()

untold basin Apr 13, 2026, 7:00 PM

#

tawdry surge for low leveling blocking i'd just add Thread::park(), Thread::unpark()

im not sure thats much better tbh

tawdry surge Apr 13, 2026, 7:50 PM

#

it's basically equivalent to a binary semaphore attached to each thread

open osprey Apr 13, 2026, 7:51 PM

#

untold basin and go does its own userspace scheduling

it was the fashion in those days

untold basin Apr 13, 2026, 7:51 PM

#

it still is i think

untold basin Apr 13, 2026, 7:51 PM

#

tawdry surge it's basically equivalent to a binary semaphore attached to each thread

yea

open osprey Apr 13, 2026, 7:51 PM

#

rust had green threads as well

#

so the first i saw of rust i took it to be basically a golike language and i still have the shadow of that prejudice in my mind

untold basin Apr 13, 2026, 7:52 PM

#

so now taking a lock means you take a semaphore, which acquires a lock, then you take from a binary semaphore, which acquires another lock

tawdry surge Apr 13, 2026, 7:53 PM

#

why does taking a semaphore take a lock?

untold basin Apr 13, 2026, 7:54 PM

#

well ig depends on the impl

#

oh wait no

#

yeah to enqueue yourself on the wait list you need to take a lock over the wait list

tawdry surge Apr 13, 2026, 7:54 PM

#

ah a spinlock

untold basin Apr 13, 2026, 7:55 PM

#

well yeah

untold basin Apr 13, 2026, 7:55 PM

#

untold basin yeah to enqueue yourself on the wait list you need to take a lock over the wait ...

or you do an "awesome" O(n^2) list traversal

untold basin Apr 14, 2026, 1:07 PM

#

ok i have something which may be a correct semaphore

#

use core::{
    hint::{likely, unlikely},
    sync::atomic::Ordering,
};

use crate::{
    atomic::Atomic,
    ex::sync::turnstile::{ParkToken, UnparkToken, park, unpark_one},
};

pub struct Semaphore {
    value: Atomic<u64>,
}
impl Semaphore {
    pub const fn new(value: u64) -> Semaphore {
        assert!(value < 0x8000_0000_0000_0000);
        Semaphore {
            value: Atomic::new(value << 1),
        }
    }
    unsafe fn put_slow(&self) {
        unsafe {
            let _ = unpark_one((&raw const self.value) as usize, |_, r| {
                debug_assert_eq!(r.unparked_threads, 1);
                if !r.have_more_threads {
                    self.value.fetch_sub(1, Ordering::Relaxed);
                }
                UnparkToken(0)
            });
        }
    }
    pub fn put(&self) {
        let value = self.value.fetch_add(2, Ordering::Release);
        if unlikely(value & 1 != 0) {
            unsafe { self.put_slow() };
        }
    }
    pub fn get(&self) {
        loop {
            let value = self.value.load(Ordering::Relaxed);

            if value < 2 {
                unsafe {
                    park((&raw const self.value) as usize, ParkToken(0), |_| {
                        if self.value.load(Ordering::Relaxed) < 2 {
                            self.value.fetch_or(1, Ordering::Relaxed);
                            true
                        } else {
                            false
                        }
                    })
                };
                continue;
            }
            if likely(
                self.value
                    .compare_exchange_weak(value, value - 2, Ordering::Acquire, Ordering::Relaxed)
                    .is_ok(),
            ) {
                return;
            }
        }
    }
}

#

now is it a correct semaphore? lol idk

#

it feels right

#

dies immediately in miri when testing basically this impl against parking_lot_core

#

error: Undefined Behavior: Data race detected between (1) non-atomic write on thread `unnamed-2` and (2) non-atomic read on thread `unnamed-1` at alloc7
  --> src/main.rs:80:44
   |
80 |                 unsafe { VALUE.get().write(VALUE.get().read() + 1) };
   |                                            ^^^^^^^^^^^^^^^^^^ (2) just happened here
   |
help: and (1) occurred earlier here
  --> src/main.rs:80:26
   |
80 |                 unsafe { VALUE.get().write(VALUE.get().read() + 1) };
   |                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   = help: this indicates a bug in the program: it performed an invalid operation, and caused Undefined Behavior
   = help: see https://doc.rust-lang.org/nightly/reference/behavior-considered-undefined.html for further information
   = note: this is on thread `unnamed-1`
note: the current function got called indirectly due to this code
  --> src/main.rs:77:9
   |
77 | /         thread::spawn(|| {
78 | |             loop {
79 | |                 SEMA.get();
80 | |                 unsafe { VALUE.get().write(VALUE.get().read() + 1) };
...  |
83 | |         });
   | |__________^
``` hmm yes

#

ok it turns out its just me dumb

#

lol

#

ok and now my turnstile/parkinglot knockoff is broken

tawdry surge Apr 14, 2026, 1:54 PM

#

Didn't you say that you don't want to do a parking lot knockoff? Or did you do it on top of semaphores as a basis?

untold basin Apr 14, 2026, 2:02 PM

#

tawdry surge Didn't you say that you don't want to do a parking lot knockoff? Or did you do i...

idk did i say that

#

anyway i did that so

tawdry surge Apr 14, 2026, 2:02 PM

#

You did say that you dislike the idea of using park() and unpark() as primitive

untold basin Apr 14, 2026, 2:09 PM

#

tawdry surge You did say that you dislike the idea of using park() and unpark() as primitive

i meant park() and unpark() as in rust std::thread::{park, Thread::unpark}

#

not as in parking_lot_core::{park, unpark_one, unpark_all}

tawdry surge Apr 14, 2026, 2:13 PM

#

Ah

#

Makes sense

untold basin Apr 14, 2026, 11:34 PM

#

okay ive got myself some one byte sleeping locks

untold basin Apr 15, 2026, 11:42 AM

#

untold basin i meant park() and unpark() as in rust `std::thread::{park, Thread::unpark}`

ok after further thinking i might implement those lmao

tawdry surge Apr 15, 2026, 11:49 AM

#

lol

untold basin Apr 15, 2026, 12:00 PM

#

untold basin not as in `parking_lot_core::{park, unpark_one, unpark_all}`

because they allow you to implement the other set of primitives without spraying spinlocks everywhere

#

and its a lot easier to test the primitives too

untold basin Apr 16, 2026, 9:05 AM

#

hmm does it even make sense to use a queued lock to protect my parkinglot meme's hash buckets

#

because disabling preemption is dirt cheap

trim seal Apr 16, 2026, 9:14 AM

#

depends how your per-cpu is implemented meme

untold basin Apr 16, 2026, 9:14 AM

#

trim seal depends how your per-cpu is implemented <:meme:575445094589661215>

wdym

trim seal Apr 16, 2026, 9:14 AM

#

you may have 50 indirections, debug checks, asserts, etc

#

because of rust

untold basin Apr 16, 2026, 9:15 AM

#

trim seal you may have 50 indirections, debug checks, asserts, etc

the fast path is this:```arm
ffffffff0000dd8c fe4fbfa9 stp lr, x19, [sp, #-0x10]! {__saved_lr} {__saved_x19}
ffffffff0000dd90 481640b9 ldr w8, [x18, #0x14]
ffffffff0000dd94 88030034 cbz w8, 0xffffffff0000de04

ffffffff0000dd98 08050051 sub w8, w8, #0x1
ffffffff0000dd9c 481600b9 str w8, [x18, #0x14]
ffffffff0000dda0 68000034 cbz w8, 0xffffffff0000ddac

ffffffff0000dda4 fe4fc1a8 ldp lr, x19, [sp], #0x10 {__saved_x19}
ffffffff0000dda8 c0035fd6 ret


pretend this got inlined

trim seal Apr 16, 2026, 9:15 AM

#

yeah im not qualified to judge arm per-cpu

untold basin Apr 16, 2026, 9:16 AM

#

you can count the number of instructions trl

untold basin Apr 16, 2026, 9:16 AM

#

untold basin the fast path is this:```arm ffffffff0000dd8c fe4fbfa9 stp lr, x19, [sp, ...

ill mark it inline(always) tbh

trim seal Apr 16, 2026, 9:16 AM

#

on x86 a good per-cpu would be inc gs:preempt_disable_count[rip]

untold basin Apr 16, 2026, 9:16 AM

#

trim seal on x86 a good per-cpu would be `inc gs:preempt_disable_count[rip]`

plus a branch

trim seal Apr 16, 2026, 9:16 AM

#

why a branch?

untold basin Apr 16, 2026, 9:17 AM

#

you need to check for pending preemption

trim seal Apr 16, 2026, 9:17 AM

#

for enable

untold basin Apr 16, 2026, 9:17 AM

#

oh sorry this is the enable path

trim seal Apr 16, 2026, 9:17 AM

#

ah ok

untold basin Apr 16, 2026, 9:17 AM

#

disable does get inlined

#

its 3 instructions: ```arm
ffffffff0000e30c 481640b9 ldr w8, [x18, #0x14]
ffffffff0000e310 08050011 add w8, w8, #0x1
ffffffff0000e314 481600b9 str w8, [x18, #0x14]
ffffffff0000e318 c0035fd6 ret

#

read, add, write

trim seal Apr 16, 2026, 9:18 AM

#

whats in x18?

untold basin Apr 16, 2026, 9:19 AM

#

percpu data

trim seal Apr 16, 2026, 9:19 AM

#

is that like

#

reserved in the abi?

untold basin Apr 16, 2026, 9:19 AM

#

untold basin Apr 16, 2026, 9:19 AM

#

trim seal reserved in the abi?

kinda

#

but -ffixed-[reg] makes whatever you want reserved 🙃

trim seal Apr 16, 2026, 9:19 AM

#

u pass that right?

untold basin Apr 16, 2026, 9:19 AM

#

yes ofc lmao

trim seal Apr 16, 2026, 9:19 AM

#

lol

untold basin Apr 16, 2026, 9:19 AM

#

its not reserved on linux

#

The role of register r18 is platform specific. If a platform ABI has need of a dedicated general-purpose register to carry inter-procedural state (for example, the thread context) then it should use this register for that purpose. If the platform ABI has no such requirements, then it should use r18 as an additional Caller-saved register. The platform ABI specification must document the usage for this register.

trim seal Apr 16, 2026, 9:20 AM

#

iirc linux stores per-cpu base in el0 sp

untold basin Apr 16, 2026, 9:20 AM

#

trim seal iirc linux stores per-cpu base in el0 sp

no they put the current task there

trim seal Apr 16, 2026, 9:20 AM

#

oh

#

what about per-cpu base

untold basin Apr 16, 2026, 9:20 AM

#

i have no idea

trim seal Apr 16, 2026, 9:21 AM

#

oh that might actually always do per_cpu_base[this_cpu_id]

untold basin Apr 16, 2026, 9:21 AM

#

its in tpidr_el1

trim seal Apr 16, 2026, 9:21 AM

#

base or just the id?

untold basin Apr 16, 2026, 9:21 AM

#

trim seal oh that might actually always do `per_cpu_base[this_cpu_id]`

it does tpidr_el1 + offset_of_percpu_var

trim seal Apr 16, 2026, 9:21 AM

#

ah ok

#

why dont u use tpidr_e1?

untold basin Apr 16, 2026, 9:22 AM

#

i do

#

it contains a mirror of x18

trim seal Apr 16, 2026, 9:22 AM

#

lol

#

whats the point?

untold basin Apr 16, 2026, 9:22 AM

#

it lets you manipulate percpu data without reading from a register

#

i need a copy because i need to restore it after running userspace

trim seal Apr 16, 2026, 9:23 AM

#

oh ok

untold basin Apr 16, 2026, 9:23 AM

#

i also store the current CPU number in TPIDRRO_EL0

#

(which is readable by userspace)

trim seal Apr 16, 2026, 9:23 AM

#

makes sense

untold basin Apr 16, 2026, 9:23 AM

#

i guess i could set x18=current task or something but idk

untold basin Apr 16, 2026, 9:24 AM

#

trim seal on x86 a good per-cpu would be `inc gs:preempt_disable_count[rip]`

this also requires writing per-arch asm

#

which is kinda annoying

trim seal Apr 16, 2026, 9:25 AM

#

hmm not really

#

__seg_gs

untold basin Apr 16, 2026, 9:25 AM

#

im writing rust

trim seal Apr 16, 2026, 9:25 AM

#

ah

#

right

untold basin Apr 16, 2026, 9:25 AM

#

also using seg_gs like that is definitely UB lol

trim seal Apr 16, 2026, 9:25 AM

#

how lol

untold basin Apr 16, 2026, 9:25 AM

#

actually maybe its not if you use an atomic

#

uhhh id have to think about the legality of that but it does not feel super robust lol

trim seal Apr 16, 2026, 9:26 AM

#

it's not allowed to cache gs: stuff

untold basin Apr 16, 2026, 9:26 AM

#

trim seal it's not allowed to cache gs: stuff

where does clang promise that lol

trim seal Apr 16, 2026, 9:26 AM

#

by being able to compile linux

#

lmao

untold basin Apr 16, 2026, 9:26 AM

#

thats not a promise

#

linux does a lot of UB lmao

trim seal Apr 16, 2026, 9:27 AM

#

but yeah i havent read the docs, but iirc it was defined to not be cached

#

or so i heard somewhere

untold basin Apr 16, 2026, 9:30 AM

#

idk i cant find anything about that

#

and it would be super dumb if this was a generic rule

#

this is used on amdgpu and nvptx for like normal memory

untold basin Apr 16, 2026, 9:32 AM

#

trim seal but yeah i havent read the docs, but iirc it was defined to not be cached

ok but i dont care that much about definitions if this code always returns 1 lol ```c++
using u32 = uint32_t;
using u64 = uint64_t;

u32 test2(__seg_fs u32* p, u64* q) {
*p = 1;
*q = 2;
return *p;
}```

#

and thats the result on clang

#

(this is subject to normal strict aliasing btw)

trim seal Apr 16, 2026, 9:32 AM

#

i mean its ub to do per cpu writes/reads without preempt_disable

#

so this doesnt break anything

untold basin Apr 16, 2026, 9:32 AM

#

thats not the point

tawdry surge Apr 16, 2026, 9:32 AM

#

trim seal on x86 a good per-cpu would be `inc gs:preempt_disable_count[rip]`

I think worrying about an extra indirection is most larp

#

It'd be surprised if this shows up in any meaningful benchmarks

untold basin Apr 16, 2026, 9:33 AM

#

tawdry surge I think worrying about an extra indirection is most larp

this 100%

#

if you want even cheaper, a load followed by a store of 1 is even cheaper

#

instead of a counter

#

but everyone parrots linux ~~including me~~ and has a counter because ???

#

i do want to get rid of the counter (because its really dumb) but i didnt get around to it

trim seal Apr 16, 2026, 9:35 AM

#

untold basin ok but i dont care that much about definitions if this code always returns `1` l...

also this is not even related to seg_gs lol

untold basin Apr 16, 2026, 9:35 AM

#

trim seal also this is not even related to seg_gs lol

how

trim seal Apr 16, 2026, 9:35 AM

#

its just a general race-is-ub rule

untold basin Apr 16, 2026, 9:35 AM

#

no this is strict aliasing

trim seal Apr 16, 2026, 9:36 AM

#

sure, but whats your point?

untold basin Apr 16, 2026, 9:36 AM

#

you cant get it to emit an inc without asm

#

afaict

#

you can get a lock inc, which i guarantee is worse than a load+add+store

tawdry surge Apr 16, 2026, 9:37 AM

#

I think linux is the only mainstream OS with a preempt disable counter (?)

untold basin Apr 16, 2026, 9:37 AM

#

which has only 20 cycles of latency on alderlake!

untold basin Apr 16, 2026, 9:37 AM

#

tawdry surge I think linux is the only mainstream OS with a preempt disable counter (?)

yeah i think so

#

its a lot more common in hobby OSes though

tawdry surge Apr 16, 2026, 9:37 AM

#

yeah

trim seal Apr 16, 2026, 9:37 AM

#

do u just not allow nested preempt disable calls?

untold basin Apr 16, 2026, 9:37 AM

#

trim seal do u just not allow nested preempt disable calls?

no you save the old value of the flag lol

tawdry surge Apr 16, 2026, 9:37 AM

#

you save the previous state

trim seal Apr 16, 2026, 9:37 AM

#

why would u do that if u can do less work with a counter?

untold basin Apr 16, 2026, 9:38 AM

#

trim seal why would u do that if u can do less work with a counter?

because using a counter does more work

trim seal Apr 16, 2026, 9:38 AM

#

untold basin you can get a `lock inc`, which i guarantee is worse than a load+add+store

untold basin Apr 16, 2026, 9:39 AM

#

trim seal

this is literally 10x slower than read+add+write btw

tawdry surge Apr 16, 2026, 9:39 AM

#

why would you use lock

untold basin Apr 16, 2026, 9:39 AM

#

tawdry surge why would you use `lock`

well you cant get llvm to generate a non-lock op without asm

trim seal Apr 16, 2026, 9:39 AM

#

yeah

untold basin Apr 16, 2026, 9:39 AM

#

so you need per-isa porting work

tawdry surge Apr 16, 2026, 9:39 AM

#

well, just use asm

#

yeah

trim seal Apr 16, 2026, 9:39 AM

#

if u want this optimized you will need asm

trim seal Apr 16, 2026, 9:40 AM

#

untold basin so you need per-isa porting work

not really

#

u can have a generic impl

untold basin Apr 16, 2026, 9:40 AM

#

or you can just do a flag which only needs generic loads and stores

trim seal Apr 16, 2026, 9:40 AM

#

and an isa can override it

tawdry surge Apr 16, 2026, 9:40 AM

#

you can just use relaxed loads/stores instead of inc

#

there's 100% no measurable overhead

untold basin Apr 16, 2026, 9:40 AM

#

if you lock inc, there is

trim seal Apr 16, 2026, 9:40 AM

#

relaxed still uses a lock

tawdry surge Apr 16, 2026, 9:40 AM

#

yeah if you're lock, that's bad

untold basin Apr 16, 2026, 9:40 AM

#

trim seal relaxed still uses a lock

nop

#

you do a relaxed load, add, relaxed store

trim seal Apr 16, 2026, 9:40 AM

#

well im looking at compiler exporer???

tawdry surge Apr 16, 2026, 9:41 AM

#

relaxed load/store, not relaxed fetch_add

trim seal Apr 16, 2026, 9:41 AM

#

untold basin you do a relaxed load, add, relaxed store

this needs the preemption to be already disabled before that

tawdry surge Apr 16, 2026, 9:41 AM

#

no, why?

trim seal Apr 16, 2026, 9:41 AM

#

otherwise u increment preempt disable on a random cpu

untold basin Apr 16, 2026, 9:41 AM

#

trim seal this needs the preemption to be already disabled before that

no

untold basin Apr 16, 2026, 9:41 AM

#

trim seal otherwise u increment preempt disable on a random cpu

well no

#

if you can get preempted then preempt_count == 0

trim seal Apr 16, 2026, 9:42 AM

#

what

untold basin Apr 16, 2026, 9:42 AM

#

the store is single-instruction atomic

tawdry surge Apr 16, 2026, 9:42 AM

#

yeah

#

pitust is correct, that's also how all non-linux OSes do it

trim seal Apr 16, 2026, 9:42 AM

#

cur_count = read(preempt_count)
cur_count += 1;
write(preempt_count, cur_count)

#

u can get preempted at any point before write

tawdry surge Apr 16, 2026, 9:42 AM

#

you don't do a counter

#

just 0/1

trim seal Apr 16, 2026, 9:42 AM

#

well ok in that case yes

#

but that still doesnt work

tawdry surge Apr 16, 2026, 9:43 AM

#

you cannot get preempted unless the disable flag is "committed"

untold basin Apr 16, 2026, 9:43 AM

#

trim seal u can get preempted at any point before write

doesnt matter, the preempt count is 0 on the target cpu

trim seal Apr 16, 2026, 9:43 AM

#

because u dont know if it was disabled before

tawdry surge Apr 16, 2026, 9:43 AM

#

yes you do, because context switch preserves the bit

trim seal Apr 16, 2026, 9:43 AM

#

or well i guess if u ended up on a different cpu u know it was enabled on that cpu

tawdry surge Apr 16, 2026, 9:43 AM

#

you cannot get switched in with preempt disabled

#

that'd break all kinds of stuff

trim seal Apr 16, 2026, 9:43 AM

#

yeah i suppose it works if u have a boolean

untold basin Apr 16, 2026, 9:44 AM

#

it works with a counter too

trim seal Apr 16, 2026, 9:44 AM

#

so if u read and it returns 0, you might end up on a different cpu, but u know its 0 there to because u got switched to it in the first place

untold basin Apr 16, 2026, 9:44 AM

#

yes

trim seal Apr 16, 2026, 9:44 AM

#

it doesnt work with a counter

untold basin Apr 16, 2026, 9:44 AM

#

it does

trim seal Apr 16, 2026, 9:45 AM

#

oh ig same logic applies lol

untold basin Apr 16, 2026, 9:45 AM

#

a flag is still better though

trim seal Apr 16, 2026, 9:45 AM

#

a flag?

untold basin Apr 16, 2026, 9:45 AM

#

sorry

#

bool flag > counter

trim seal Apr 16, 2026, 9:45 AM

#

why

tawdry surge Apr 16, 2026, 9:45 AM

#

it also works with a counter

#

because only the 0->1 transition is relevant

trim seal Apr 16, 2026, 9:46 AM

#

u have to allocate space for the flag on every call stack

#

it makes no sense

tawdry surge Apr 16, 2026, 9:46 AM

#

if it's non-zero already, you cannot change cpus

untold basin Apr 16, 2026, 9:46 AM

#

trim seal u have to allocate space for the flag on every call stack

what no

trim seal Apr 16, 2026, 9:46 AM

#

if u can optimize with a counter

#

yes, everything that disables preemption

tawdry surge Apr 16, 2026, 9:46 AM

#

and for the 0-1 transition, the same argument applies

untold basin Apr 16, 2026, 9:46 AM

#

oh i guess to save the old flag

trim seal Apr 16, 2026, 9:46 AM

#

yes

untold basin Apr 16, 2026, 9:46 AM

#

trim seal yes, everything that disables preemption

also lol wdym "everything"

trim seal Apr 16, 2026, 9:46 AM

#

every function you may call into

untold basin Apr 16, 2026, 9:46 AM

#

i mean im considering banning nested preempt_disable

trim seal Apr 16, 2026, 9:46 AM

#

sure

untold basin Apr 16, 2026, 9:47 AM

#

trim seal every function you may call into

you only have to save it when you disable preemption

tawdry surge Apr 16, 2026, 9:47 AM

#

optmizing a load+store to an inc is 100% larp

trim seal Apr 16, 2026, 9:47 AM

#

you will just end up with

do_thing()
do_thing_with_preemption_already_disalbed()

tawdry surge Apr 16, 2026, 9:47 AM

#

It doesn't offer any real benefit but it has real costs

untold basin Apr 16, 2026, 9:47 AM

#

tawdry surge optmizing a load+store to an inc is 100% larp

i doubt it makes any performance difference at all

tawdry surge Apr 16, 2026, 9:47 AM

#

namely that it becomes arch specific

untold basin Apr 16, 2026, 9:47 AM

#

inc has 3 uops

tawdry surge Apr 16, 2026, 9:47 AM

#

yeah, i doubt that you can show me a single benchmark where it makes a difference that is not within noise

untold basin Apr 16, 2026, 9:48 AM

#

so its just a rmw

trim seal Apr 16, 2026, 9:48 AM

#

yeah inc internally probably schedules the same sequence

tawdry surge Apr 16, 2026, 9:48 AM

#

on any competent CPU, the only difference is code size

trim seal Apr 16, 2026, 9:48 AM

#

its just less code bytes

untold basin Apr 16, 2026, 9:48 AM

#

oh wait no on zen5

untold basin Apr 16, 2026, 9:48 AM

#

tawdry surge on any competent CPU, the only difference is code size

i dont actually know if thats true

#

seems like on zen5 it might be a bit cheaper

#

but its not a measurable difference regardless its like 1 vs 3 cycles

untold basin Apr 16, 2026, 9:49 AM

#

untold basin you only have to save it when you disable preemption

and only a few things do that

trim seal Apr 16, 2026, 9:49 AM

#

technically u do run that on hot paths

#

in the scheduler etc

untold basin Apr 16, 2026, 9:50 AM

#

my scheduler runs with irqs masked

trim seal Apr 16, 2026, 9:50 AM

#

fair ig lol

untold basin Apr 16, 2026, 9:51 AM

#

i dont want to take irqs mid context switch

tawdry surge Apr 16, 2026, 9:51 AM

#

trim seal technically u do run that on hot paths

you have to consider the latency of the entire code path though

untold basin Apr 16, 2026, 9:51 AM

#

its also not a hot code path

tawdry surge Apr 16, 2026, 9:51 AM

#

iret takes what, 1000 cycles?

untold basin Apr 16, 2026, 9:52 AM

#

tawdry surge iret takes what, 1000 cycles?

less

trim seal Apr 16, 2026, 9:52 AM

#

eretu trl

tawdry surge Apr 16, 2026, 9:52 AM

#

so if you reduce the preempt disable by 1 cycle you gain at most 0.1%

untold basin Apr 16, 2026, 9:52 AM

#

idk on m1 a linux syscall is like 300

tawdry surge Apr 16, 2026, 9:52 AM

#

in reality the irq does mode than just iret

#

yeah, a syscall is cheap

#

an irq is really expensive on x86

untold basin Apr 16, 2026, 9:53 AM

#

why? iret?

tawdry surge Apr 16, 2026, 9:53 AM

#

yeah

trim seal Apr 16, 2026, 9:53 AM

#

is it a property of x86 tho?

untold basin Apr 16, 2026, 9:53 AM

#

just iret using sysretq /j

trim seal Apr 16, 2026, 9:53 AM

#

Dont u have to do something similar to iret on other arches

tawdry surge Apr 16, 2026, 9:53 AM

#

#x86 message

untold basin Apr 16, 2026, 9:54 AM

#

an arm64 eret is <300 cycles

tawdry surge Apr 16, 2026, 9:54 AM

#

when i was testing thread register restore code paths, replaced sysretq by iretq for testing

untold basin Apr 16, 2026, 9:54 AM

#

so its not as bad

trim seal Apr 16, 2026, 9:54 AM

#

is there data on how expensive fred returns are?

tawdry surge Apr 16, 2026, 9:55 AM

#

tawdry surge when i was testing thread register restore code paths, replaced sysretq by iret...

and it turned out that iretq was more than twice as slow as the entire syscall

trim seal Apr 16, 2026, 9:55 AM

#

is that some sort of a no-op syscall

tawdry surge Apr 16, 2026, 9:55 AM

#

yes

trim seal Apr 16, 2026, 9:55 AM

#

oh ok lol

tawdry surge Apr 16, 2026, 9:56 AM

#

not just 2x slower than sysretq, 2x slower than kernel entry, syscall handler dispatch, ipl checks, preemption and migration checks, kernel exit combined

untold basin Apr 16, 2026, 10:32 AM

#

ok anyway going back to my earlier problem

should i just use a spinlock for this lol

#

i think thats probably a better idea than a complicated pushlock-style mechanism

vapid echo Apr 16, 2026, 10:33 AM

#

untold basin ok anyway going back to my earlier problem should i just use a spinlock for thi...

What are you doing?

untold basin Apr 16, 2026, 10:34 AM

#

locking

vapid echo Apr 16, 2026, 10:34 AM

#

Yes ok but what

untold basin Apr 16, 2026, 10:34 AM

#

like im doing generic locking infrastructure

vapid echo Apr 16, 2026, 10:35 AM

#

But what is "this"

untold basin Apr 16, 2026, 10:36 AM

#

so basically im doing a ripoff of the wtf::parkinglot/turnstiles/futexes/go semaphores family

#

where you have a bunch of buckets with their own lock which you use to queue on (so that the inline sync primitive is smaller)

vapid echo Apr 16, 2026, 10:37 AM

#

ah yeah typically you do have a spinlock over the buckets

#

if you mean the table hashed by the lock address

untold basin Apr 16, 2026, 10:38 AM

#

yeah

#

WTF::ParkingLot and rust parking_lot take a sleeping lock

vapid echo Apr 16, 2026, 10:39 AM

#

But you might not be able to do that if you implement your sleeping lock using that

untold basin Apr 16, 2026, 10:39 AM

#

vapid echo But you might not be able to do that if you implement your sleeping lock using t...

you can make a second sleeping lock, that doesnt use the same infrastructure

#

which is larger

vapid echo Apr 16, 2026, 10:39 AM

#

Yes but is it worth it?

untold basin Apr 16, 2026, 10:39 AM

#

lol i have no idea

#

how would you even measure that

#

my vibe is no though

#

not for a kernel

vapid echo Apr 16, 2026, 10:42 AM

#

I'd say the lock likely won't experience that much contention

#

And it's a quick critical section

untold basin Apr 16, 2026, 10:42 AM

#

if you disable preemption at least

#

yeah

vapid echo Apr 16, 2026, 10:42 AM

#

Yea

#

but you kinda have to disable preemption

#

Huh reading about parking lot it's basically turnstiles

untold basin Apr 16, 2026, 10:43 AM

#

yes!

#

wait until you read about plan9 semaphores

vapid echo Apr 16, 2026, 11:02 AM

#

And they also call not doing handoff barging

tawdry surge Apr 16, 2026, 11:08 AM

#

does parking lot / futex style locking make sense inside the kernel?

#

or are we talking about futexes for userspace?

vapid echo Apr 16, 2026, 11:12 AM

#

tawdry surge does parking lot / futex style locking make sense _inside_ the kernel?

yes, you do want small locks

#

Also with turnstiles you can do priority inheritance

tawdry surge Apr 16, 2026, 11:14 AM

#

But parking lot is not entirely like a turnstile, isn't it?

#

because it doesn't actually use the key as a pointer

#

I think BSD-style turnstiles make more sense

#

well i guess parking lot is just the blocking side of a turnstile lock

#

if you think of it that way

#

and then it makes sense, yeah

untold basin Apr 16, 2026, 11:51 AM

#

awesome, i started to test my impl in miri

#

that was fast lol

#

ok apparantly my lock is broken???

#

yeah i used a fucking relaxed store when unlocking

#

me dumb

trim seal Apr 16, 2026, 11:58 AM

#

untold basin awesome, i started to test my impl in miri

How reliable is this software?

untold basin Apr 16, 2026, 11:59 AM

#

trim seal How reliable is this software?

what is?

#

miri?

#

~~its by definition reliable for all rust programs~~ nvm apparantly thats not the case

#

(well obviously it will have false negatives. but its guaranteed not to have false positives)

untold basin Apr 16, 2026, 12:00 PM

#

untold basin ~~its by definition reliable for all rust programs~~ nvm apparantly thats not th...

excluding compiler bugs ofc

trim seal Apr 16, 2026, 12:01 PM

#

does it work with barriers

untold basin Apr 16, 2026, 12:02 PM

#

uhhh i actually dont know

#

yes i think so

tawdry surge Apr 16, 2026, 12:04 PM

#

it is reliable as long as all the code is written in rust

#

it cannot reason across FFI boundaries of course

untold basin Apr 16, 2026, 12:05 PM

#

i think it can somewhat do FFI now

#

but yeah obv its only useful if you have "normal" userspace code

tawdry surge Apr 16, 2026, 12:06 PM

#

trim seal does it work with barriers

it works by interpreting the rust program and tracking extra state per memory location

#

so it's rather slow / not usable for full programs

#

but it does detect races and other memory unsafety issues

untold basin Apr 16, 2026, 12:07 PM

#

it detects basically all violations of rust's memory access rules

untold basin Apr 16, 2026, 12:07 PM

#

tawdry surge so it's rather slow / not usable for full programs

it can be usable, if your program is simple. but yeah its very much not fast

tawdry surge Apr 16, 2026, 12:08 PM

#

yeah but it makes much more sense to run unit tests etc through miri than full programs

untold basin Apr 16, 2026, 12:08 PM

#

yeah

trim seal Apr 16, 2026, 12:13 PM

#

tawdry surge it works by interpreting the rust program and tracking extra state per memory lo...

oh ok

untold basin Apr 18, 2026, 10:50 AM

#

okay im now trying to figure out a good ipc model

i still havent come up with a good one which isnt pain to implement

tawdry surge Apr 18, 2026, 10:54 AM

#

have you looked at existing ones?

untold basin Apr 18, 2026, 10:54 AM

#

a bit

#

so the big thing is that i want to make it work without dynamic allocation (i.e. allocation when opening a connection or setting up a server is fine, but anything else should be preallocated)

and ideally it would be asynchronous

tawdry surge Apr 18, 2026, 10:55 AM

#

asynchronous and no allocation kinda contracts each other, doesn't it?

untold basin Apr 18, 2026, 10:55 AM

#

well, lets call it "optimistically asynchronous"

tawdry surge Apr 18, 2026, 10:55 AM

#

unless you preallocate a bunch of inflight ipc data structures

untold basin Apr 18, 2026, 10:56 AM

#

tawdry surge unless you preallocate a bunch of inflight ipc data structures

my idea is to preallocate 1

#

and then you block if you cant allocate more

#

and if you can then you can obviously get more concurrency

#

im gonna look at managarm maybe

#

uhhhhh GCEs are not a good thing to rely on

#

they are more like a "this is getting removed possibly in a few months" kind of feature

#

not a "this is getting stabilized hopefully soonish" kind of feature

tawdry surge Apr 18, 2026, 11:01 AM

#

iirc our use of generic const expressions can be rewritten at the cost of making it more annoying / maybe using a proc macro

untold basin Apr 18, 2026, 11:01 AM

#

yea i see i see

#

assuming the only use is submit_async/Action, that looks easy enough to solve

tawdry surge Apr 18, 2026, 11:02 AM

#

yeah i think that's the only place

#

i think it's really only for the construction of the array

#

in let mut actions = [const { MaybeUninit::uninit() }; T::ACTION_COUNT];

untold basin Apr 18, 2026, 11:09 AM

#

also here is some functions in the managarm code i was just curious how you handle reading from shared memory

#

oop wrong paste

#

here is the correct 3rd picture

#

(you cannot just blind memcpy from random user-accessible pointers lmao)

#

this is probably not an issue but like

#

jfc

#

oh and also this is a controlled allocation primitive with whatever data you want on the only kernel zone, my favorite

#

at least my kernel attempts to do type isolation

#

even if it doesnt try super hard

tawdry surge Apr 18, 2026, 11:16 AM

#

wdym

#

there is no copy from user here

untold basin Apr 18, 2026, 11:17 AM

#

tawdry surge there is no copy from user here

the memcpy is from data shared with userspace

#

which you also cant just memcpy from safely

tawdry surge Apr 18, 2026, 11:17 AM

#

ah you mean because userspace could technically write and then its technically a data race?

#

yeah that's true

untold basin Apr 18, 2026, 11:17 AM

#

tawdry surge ah you mean because userspace could technically write and then its technically a...

yeah

tawdry surge Apr 18, 2026, 11:17 AM

#

but there's no way around that in current c++

untold basin Apr 18, 2026, 11:18 AM

#

in rust i'd tell you to do by-value freeze

tawdry surge Apr 18, 2026, 11:18 AM

#

unless you hand roll and asm memcpy i guess

untold basin Apr 18, 2026, 11:18 AM

#

which some people regard as being sound

tawdry surge Apr 18, 2026, 11:18 AM

#

there was a proposal to standardize bytewise atomic memcpy but it was not accepted

#

meme

untold basin Apr 18, 2026, 11:18 AM

#

just do like asm volatile("" :: "r"(&the_object)); lol

#

that is sound within llvm

#

because data races are not llvm ub

tawdry surge Apr 18, 2026, 11:20 AM

#

but there is no TOCTOU here right? userspace cannot change the count parameter post hoc, neither can it invalidate the buffer

#

it can just influence the buffer contents

untold basin Apr 18, 2026, 11:20 AM

#

tawdry surge but there is no TOCTOU here right? userspace cannot change the `count` parameter...

in the presence of inlining and llvm doing too much llvming you can actually get toctou here

#

imagine the compiler decided to remove the memcpy and read from the mapping directly (which is sound)

#

im not sure if you can prove that the stuff in the middle will have no side effects on memory, it could be enough of a barrier in practice

tawdry surge Apr 18, 2026, 11:21 AM

#

ah i see what you mean. that's true

untold basin Apr 18, 2026, 11:21 AM

#

untold basin imagine the compiler decided to remove the memcpy and read from the mapping dire...

and then it decided to remat the loads

tawdry surge Apr 18, 2026, 11:26 AM

#

i noted this down in https://github.com/managarm/managarm/issues/1331

GitHub

thor: Copies from buffers shared with userspace should use an atomi...

Otherwise, compilers may optimize out the memcpy and use the shared buffers directly (which could lead to UB). Since atomic_memcpy() is not in current C++, roll out own atomic_relaxed_memcpy() for ...

untold basin Apr 18, 2026, 11:26 AM

#

there is a very funny strategy here i think

#

i THINK llvm might have intrinsics that let you do this

#

that arent surfaced

trim seal Apr 18, 2026, 11:31 AM

#

tawdry surge i noted this down in https://github.com/managarm/managarm/issues/1331

yeah i rolled this exact relaxed memcpy for my log ring lol

tawdry surge Apr 18, 2026, 11:31 AM

#

how did you implement it?

trim seal Apr 18, 2026, 11:31 AM

#

it literally did relaxed loads on each byte

#

of source

tawdry surge Apr 18, 2026, 11:32 AM

#

that probably has horrible codegen though

untold basin Apr 18, 2026, 11:32 AM

#

yeah

trim seal Apr 18, 2026, 11:32 AM

#

i dont know how else to solve it but yes

untold basin Apr 18, 2026, 11:32 AM

#

definitely worse than doing the mildly UB thing of memcpy then by-value freeze

trim seal Apr 18, 2026, 11:32 AM

#

untold basin definitely worse than doing the mildly UB thing of memcpy then by-value freeze

what does that even mean

untold basin Apr 18, 2026, 11:32 AM

#

which one

#

by-value freeze?

trim seal Apr 18, 2026, 11:33 AM

#

yeah\

untold basin Apr 18, 2026, 11:33 AM

#

ok so rust has a concept of a "byte" which has 257* values: 0-255, inclusive, and a special value called "uninit"

#

the simplest case of by-value freeze is when you go from a value of type "u8 | uninit" to a value of type "u8" by doing something like ```rs
fn by_value_freeze_u8(value: u8 | uninit) -> u8:
if is_uninint(value):
return unpredictable_byte()
else:
return value


(not real code ofc)

#

its called "freeze" because LLVM has an instruction called freeze which does exactly this

untold basin Apr 18, 2026, 11:35 AM

#

untold basin the simplest case of by-value freeze is when you go from a value of type "u8 | u...

more generally, this works on any type where every bit pattern is valid

#

now the UB

#

in LLVM, data races are permitted: when two stores race, you store an LLVM undef value (which is similar to a rust uninit byte); when a load races with a store, you load an LLVM undef value

#

oh llvm has a real byte type now

#

instead of a weird and not used for good reason hack of <i1 x N>

trim seal Apr 18, 2026, 11:38 AM

#

tawdry surge that probably has horrible codegen though

ffffffff800041e4:       48 83 e2 fc             and    rdx,0xfffffffffffffffc
ffffffff800041e8:       31 c9                   xor    ecx,ecx
ffffffff800041ea:       66 0f 1f 44 00 00       nop    WORD PTR [rax+rax*1+0x0]
ffffffff800041f0:       44 0f b6 04 0e          movzx  r8d,BYTE PTR [rsi+rcx*1]
ffffffff800041f5:       44 88 04 0f             mov    BYTE PTR [rdi+rcx*1],r8b
ffffffff800041f9:       44 0f b6 44 0e 01       movzx  r8d,BYTE PTR [rsi+rcx*1+0x1]
ffffffff800041ff:       44 88 44 0f 01          mov    BYTE PTR [rdi+rcx*1+0x1],r8b
ffffffff80004204:       44 0f b6 44 0e 02       movzx  r8d,BYTE PTR [rsi+rcx*1+0x2]
ffffffff8000420a:       44 88 44 0f 02          mov    BYTE PTR [rdi+rcx*1+0x2],r8b
ffffffff8000420f:       44 0f b6 44 0e 03       movzx  r8d,BYTE PTR [rsi+rcx*1+0x3]
ffffffff80004215:       44 88 44 0f 03          mov    BYTE PTR [rdi+rcx*1+0x3],r8b
ffffffff8000421a:       48 83 c1 04             add    rcx,0x4
ffffffff8000421e:       48 39 ca                cmp    rdx,rcx
ffffffff80004221:       75 cd                   jne    ffffffff800041f0 <atomic_src_memcpy+0x20>
ffffffff80004223:       48 85 c0                test   rax,rax
ffffffff80004226:       74 17                   je     ffffffff8000423f <atomic_src_memcpy+0x6f>
ffffffff80004228:       48 01 cf                add    rdi,rcx
ffffffff8000422b:       48 01 ce                add    rsi,rcx
ffffffff8000422e:       31 c9                   xor    ecx,ecx
ffffffff80004230:       0f b6 14 0e             movzx  edx,BYTE PTR [rsi+rcx*1]
ffffffff80004234:       88 14 0f                mov    BYTE PTR [rdi+rcx*1],dl
ffffffff80004237:       48 ff c1                inc    rcx
ffffffff8000423a:       48 39 c8                cmp    rax,rcx
ffffffff8000423d:       75 f1                   jne    ffffffff80004230 <atomic_src_memcpy+0x60>
ffffffff8000423f:       c3                      ret

It did this at O3

#

if i force noinline

untold basin Apr 18, 2026, 11:38 AM

#

trim seal ```x86asm ffffffff800041e4: 48 83 e2 fc and rdx,0xfffffffff...

eugh

trim seal Apr 18, 2026, 11:38 AM

#

what is it even doing there

tawdry surge Apr 18, 2026, 11:39 AM

#

unrolling 4 iters

#

but not coalescing the loads

trim seal Apr 18, 2026, 11:39 AM

#

ah

trim seal Apr 18, 2026, 11:41 AM

#

untold basin the simplest case of by-value freeze is when you go from a value of type "u8 | u...

im too retarded to understand why this helps

untold basin Apr 18, 2026, 11:42 AM

#

trim seal im too retarded to understand why this helps

it helps in combination with the guarantee that loads which race load undef bytes

#

you know at the end if you had a race or not

#

so if you know you didnt, then you know you can safely read all of the bytes

#

actually you dont even need by value freeze lmao me dumb

#

well you need it in the managarm case you dont need it in the ringbuffer case

tawdry surge Apr 18, 2026, 11:44 AM

#

I think memcpy + asm volatile ("" : : : "memory") should be enough in the Managarm case, isn't it?

#

since that forces llvm to assume that the shared buffer has now changed in unpredictable ways

untold basin Apr 18, 2026, 11:44 AM

#

i mean it would make llvm not able to miscompile

#

but i'd also give it the pointer because i think that makes it also not llvm UB

#

(pointer to the copied buffer)

#

i guess that forces a "real" copy instead of reading into registers

tawdry surge Apr 18, 2026, 11:45 AM

#

reading into registers is fine though

trim seal Apr 18, 2026, 11:46 AM

#

untold basin (pointer to the copied buffer)

can I do that too?

#

instead of atomic memcpy

untold basin Apr 18, 2026, 11:46 AM

#

trim seal can I do that too?

i mean if you are fine with a little dose of technically kinda UB yes

trim seal Apr 18, 2026, 11:46 AM

#

well

untold basin Apr 18, 2026, 11:46 AM

#

its only a small dose, its not llvm ub

trim seal Apr 18, 2026, 11:46 AM

#

lmao

#

maybe its gcc backend ub

untold basin Apr 18, 2026, 11:47 AM

#

idk gcc is lame and bad anyway

#

just apply the linux strategy on gcc

trim seal Apr 18, 2026, 11:47 AM

#

whats the linux strategy

#

-fno-strict-aliasing?

untold basin Apr 18, 2026, 11:47 AM

#

complain if gcc produces correct code that breaks you because "standards shit just do the """obvious""" thing dumbass"

trim seal Apr 18, 2026, 11:48 AM

#

lol

#

i mean if your standard makes it impossible to write a custom allocator it is kinda shit

untold basin Apr 18, 2026, 11:49 AM

#

you can you just cant write a good one without writing insane code

#

also you can do it in c++

#

(not that the C++ features that let you work properly afaict)

trim seal Apr 18, 2026, 11:50 AM

#

what did u have to do to make it ub free again

untold basin Apr 18, 2026, 11:50 AM

#

launder or start_lifetime_as or placement new

#

or disable strict aliasing

#shkwve: an incomplete kernel/maybe bootloader/UEFI thing/maybe something else if i get bored again