shkwve: an incomplete kernel/maybe bootloader/UEFI thing/maybe something else if i get bored again

[trim seal]

launder on free?

[untold basin]

or asm which is just more launder

[untold basin]

uhh probably both alloc and free

[trim seal]

ig

[untold basin]

well you dont need it on alloc if you are doing c++

if you placement new the object

[tawdry surge]

you also don't need it on free

[untold basin]

which you almost always will

[trim seal]

yeah placement new works

[untold basin]

well you need to placement new any inline metadata

[tawdry surge]

but you need to be careful if you're placing meta data into formerly allocated objects etc

[untold basin]

which is on the list of things which make it ub free

[tawdry surge]

yeah true

[trim seal]

yeah depends on the impl details

[untold basin]

but yeah you need at least one of those fences

[trim seal]

if you reuse the object u have to launder

[untold basin]

or placement new or start_lifetime_as or asm

[trim seal]

is start_lifetime_as some new intrinsic?

[untold basin]

or i think there is some weirdness with automatic lifetime changes

[untold basin]

no its an STL function that is not supported by any compiler

[trim seal]

lol

[untold basin]

sorry thats not true its supported by msvc

[tawdry surge]

well, is it unsupported because the feature is missing or because compilers don't optimize "enough" based on lifetimes anyway?

[untold basin]

its unsupported because its missing; its missing because noone wants it; noone wants it because compilers dont actually optimize very much based on this

[tawdry surge]

the standard also only requires this for non implicit lifetime types

[trim seal]

adding new optimizations like this is probably fun because you will break a shit ton of working code

[tawdry surge]

since implicit lifetime types magically start lifetimes in memcpy() on other operations like that

[trim seal]

well if u have

struct thing x;
memcpy(&x, ...);

The thing lifetime has already been started before memcpy no?

[untold basin]

c++ lifetime rules are silly anyway

[untold basin]

this doesnt apply in c

its completely different afaik

[trim seal]

wdym

[untold basin]

the rules in C are very different and permit slightly different things afaik

[tawdry surge]

the cases where this applies is stuff like

my_struct x;
char b[sizeof(x)];
memcpy(b, &x, sizeof(x));
y = reinterpret_cast<my_struct *>(b);

[trim seal]

thats valid C++?

[untold basin]

if you did memcpy(b, &x, sizeof(x)); yes

[tawdry surge]

yeah

[trim seal]

u have to at least align b no?

[tawdry surge]

yeah

[untold basin]

oh well assume its aligned

[trim seal]

interesting

that only applies to pod types right

[tawdry surge]

this is valid for new enough c++ standards (unless they added implicit lifetimes as a defect report, idr) and if my_struct is implicit lifetime

[trim seal]

implicit lifetime -> no ctor/dtor?

[tawdry surge]

ah yes, it's a defect report

[untold basin]

at least one eligible trivial ctor and non-deleted trivial dtor

and all members are implicit lifetime

[trim seal]

so its ub now anyway? lol

[tawdry surge]

no, it's retroactively valid even for old c++ versions

[trim seal]

ohh

cool

[untold basin]

insane

[tawdry surge]

note that this is different from type punning via reinterpret_cast etc.

type punning is still illegal

[untold basin]

no

i want more async

and not boring async where you need three rounds of doing IPC to do anything like on managarm

[tawdry surge]

wdym

[untold basin]

afaict if you have an ipc with data you need to send a header and data

and then the other side needs to read the header, submit the ok i wanna read data action (and possibly another header idk if you can actually do that), and only then can it reply

[tawdry surge]

this depends a lot of the protocol

[untold basin]

okay so its only 2 ops

[tawdry surge]

and not on the kernel

[untold basin]

well yes and no

your kernel imposes constraints on what you can do efficiently

[tawdry surge]

if the protocol has fixed sizes, then it's request -> reply (2 ipcs) on the server side

and one on the client side because it merges send and receive into one call

[untold basin]

yeah its 2 submissions not 3 then

[tawdry surge]

so you want a "send response and retrieve next message" call?

[untold basin]

no

i want some way to make IPC happen without two submissions for each request 🙃

[tawdry surge]

yeah it's 2 submission but userspace does something in between

[untold basin]

i mean yeah

[tawdry surge]

so it's not that you could somehow fuse them

[untold basin]

ofc i dont disagree

and its definitely not an issue for bandwidth

definitely better than sync allocating enough space

but i want at least somewhat better latency

[lofty hearth]

one maybe silly idea i had is to give the kernel a buffer pool, and then a special action that retrieves the message head, grabs the tail size, grabs a buffer from the pool, and hands you the full message

[untold basin]

lol i was just about to say something like that

except with more cursed bytecode

and ofc you could make a protocol where you have one lane which has the lengths only

and then its still two submissions but at least you can perform two submissions from a single client in one go

[vapid echo]

Reading the convo, so data races are not LLVM UB?

[untold basin]

ish

[vapid echo]

I was told by zig guys they were

[untold basin]

ok but the zig guys didnt read the langref apparantly

[vapid echo]

So I had to implement an atomic memcpy too

[short kraken]

I just have a predefined set of messages and the kernel knows them all at compile time

[untold basin]

this definitely works

[short kraken]

of course it does

[untold basin]

and i like that a lot more than two submissions per ipc and if you have one client lol gotta open more channels if you want perf

[short kraken]

yes exactly

[untold basin]

but i want dynamic extensibility

[short kraken]

with some messages you just pass a reply mailbox handle

[short kraken]

why

[untold basin]

because its cooler

[short kraken]

imo it sounds lame

[trim seal]

so it only works for char buffers?

[short kraken]

because now you have to slow down/overcomplicate all of ipc... for what

only benefit being updates without reboots

[untold basin]

you can send them this screenshot 🙃

[short kraken]

but I just have reboots withourt session/data loss

[untold basin]

and adding new packages

i mean its cool in a hypothetical "oh but what if someone actually used my OS" way

[vapid echo]

idk what that means

[short kraken]

again, my latest message

[untold basin]

like i get it

[untold basin]

this is cool too

but i think its cool to not assume anything about userspace

well except that all processes use my vdso i do assume that one

[tawdry surge]

It works in a bunch of cases like memcpy or malloc but just a cast is not enough

You can use start_lifetime_as to do the object creation in place, that is more or less equivalent to a no op memcpy

[untold basin]

my plan is that:

• you cannot make a process without the vdso
• execution begins in the vdso, and without the executable being mapped
• i mean i guess you can use the currently nonexistend debug apis or something to bypass that but dont be lame

[trim seal]

yeah but i cant memcpy to a foo and reinterpret cast as bar anyway right

even if i copied a bar

[vapid echo]

I don't get the fixed IPC thing

You only have a fixed number of message types?

[untold basin]

you bake the rules for doing IPC into the kernel

which means that you can only do ipc with the statically known operations

afaiu

[tawdry surge]

good question. I'd have to check the standard

[untold basin]

which def solves the problem and its also faster than what managarm does

[vapid echo]

Solves which problem?

The need to send a header?

[untold basin]

solves the problem of how do you know where to copy

and in general sending a header etc becomes simpler

[vapid echo]

ahh ok I see

[untold basin]

but thats also lame and so im not doing that

i think im gonna do that except with more janky bytecode and a broken theorem prover

[vapid echo]

For IPC I'd do something like doors

Well that's synchronous though

[untold basin]

thats gotta be secure right

[untold basin]

what is? doors?

[vapid echo]

yeah

[untold basin]

yes and thats why i dont like it

[vapid echo]

You can support both sync and async tho

[untold basin]

the doors model is fundamentally sync on the receiving end

[vapid echo]

Most microkernels mostly do sync and they're fine

[untold basin]

"fine"

[vapid echo]

Yeah, I mean you can expose both

Doors and another async IPC model

[rose sonnet]

Microkernels mentioned?

[rose sonnet]

My kernel only does async and it's fine

[rose sonnet]

I take my words back, apparently I had synchronous recieve this whole time...

[untold basin]

lmao

[rose sonnet]

asynchronous send, synchronous receive

[untold basin]

okay thats like fine tho

mostly

[rose sonnet]

I wanted to add blocking send as well

[untold basin]

okay so i think i should go back to implementing things

im still not sure how to proceed with the implementation

heres the basic problem:

• i want to have "optimistically async" ipc (i.e. its async if you have free memory; on the callee side it always has to be ~async and you can't block on slow clients)
• no infallible allocation: my kernel does not have infallible allocation primitives except for like early startup shit
• cancellation: also you want to be able to cancel in-flight ipc
• lock ordering is hard: an async io operation is owned by the caller process, but you have to somehow send a notification from the target process

[tawdry surge]

wdym by the last point?

[untold basin]

well, where do you put the lock?

oh yeah io operations cannot allocate except when initially preparing to execute them and ideally you would allocate for "N generic operations" and not "N operations of type X"

[untold basin]

first of all, you cant really block the server on a copyin from the client

copyin can block if you have a pager etc

if notifying an io operation requires taking a lock on the target process, then you have a bit of a chicken and egg problem

while you are notifying you cant hold that per-process lock since if you did you could deadlock

but you obviously need to hold the lock which protects the list itself

for cancellation you also need to grab that lock

so do you take the queue lock, get the list, release the queue lock, take the list lock, and reacquire the queue lock?

idk that feels so sketchy

[tawdry surge]

what is the queue and the list here? maybe you could explain how your data structures work both on the server and client side

[untold basin]

i mean i dont have them but the sketch of the idea is you have a queue of tasks that userspace needs to do

or a tree or whatever else

and you have a list of iops a given client object owns as a client

[tawdry surge]

afaict the obvious solution is to refcount the inflight ops such that you can retain (shared) ownership of them w/o holding a lock

[untold basin]

yeah maybe

but i dont want them to be individually allocated...

idk maybe you cant avoid that

[tawdry surge]

the refcount in this case is probably at most 2 or 3 (part of list, completion, cancellation?) but that doesn't really help with optimizing it

[untold basin]

yeah ig

[untold basin]

okay i have an even better idea for ipc buffer handling

so i need to allocate a transfer buffer while im doing a copy from userspace

because i cant copy after i allocated space on the target queue

but what i can do is allocate a, lets say, 128k transfer buffer

and then allocate ~256 byte tranfer buffer entries inside of it with a simple local freelist

and if i fill up the transfer buffer then you can throttle clients

and i only need one transfer buffer per thread or process really

[untold basin]

and then for even bigger transfers i can allocate a bigger transfer buffer

[rose sonnet]

just use shared memory for those

[untold basin]

no i want it to be transparent

[untold basin]

if i use shared memory for those then i need to set up shared memory for each client which is really expensive

[rose sonnet]

I create a memory object, send its capability, and map it in the reciever...

yeah, it's expensive

[untold basin]

yeah thats like 3 expensive operations

well sending isnt that expensive ig but like

no

[rose sonnet]

you're sending anyway though

[untold basin]

ok but i dont want to allocate a big shmem object

i mean i also dont want to implement them rn because they are really annoying to implement

[rose sonnet]

why

[untold basin]

wdym why

[rose sonnet]

annoying

[untold basin]

you have to allocate a bunch of pages

in my design i basically cant allocate discontinuous pages

[rose sonnet]

what pmm are you using?

[untold basin]

basically just a freelist lol

[rose sonnet]

I don't get it then

[untold basin]

oh sorry i meant contignuous im dumb

i can only make discontinuous ones so id need to make like a radix tree or something

[rose sonnet]

why do you need them

[rose sonnet]

yeah

[untold basin]

but thats annoying to implement

[rose sonnet]

I mean I just store my pages in a linked list in memory objects

[untold basin]

WTF

so if you have a 1 gigabyte vmobject

and you fault at 1 gigabyte-4k

you traverse, uhh

[rose sonnet]

yeah

[untold basin]

256k nodes?

lol im not doing that

i have only a few linked lists and id like to not add any more

[rose sonnet]

I was planning to do a hash map, but my vector class was having a skill issue at that time, and I think I never changed that, but at least the lists are too simple to break randomly

[tawdry surge]

just use a radix tree

[rose sonnet]

it's an implementation detail

[untold basin]

a hashmap and a vector are both kinda bad

a hashmap is really bad

[rose sonnet]

I don't like radix trees, how are they better than hashmaps?

[untold basin]

but yeah im gonna use a radixtree but i dont want to implement it

[untold basin]

depends on your design

but they are generally slower

they have the same problem as a vector which is that they require large kernel allocations

[rose sonnet]

also, how would you even implement a radix tree there?

[tawdry surge]

not necessarily, you can make a radix tree that allocates e.g., 128 bytes at a time

ah you mean a hashmap

[rose sonnet]

why is doing large allocation a problem?

besides zeroing the memory

[untold basin]

because it requires writing an allocator which can handle it

[tawdry surge]

radix trees have better worst case performance

[untold basin]

yeah ofc you can do that for a radixtree

[tawdry surge]

and they have more predictable performance

[untold basin]

i think radixtrees also might have a better average case

[rose sonnet]

vmm?

[untold basin]

wdym

[rose sonnet]

how do you handle virtual memory?

[untold basin]

for what? for userspace?

[rose sonnet]

for kernel itself

[untold basin]

oh well i have some mappings ofc

[rose sonnet]

I use vmem, and it kinda just works with thatever memory it gets

[untold basin]

yeah i have something kinda like that but i also dont want to do that

[rose sonnet]

like what order do you choose?

[untold basin]

the smallest one that will fit the number of entries you need

lol

[rose sonnet]

It would basically be the same as with paging, but with pointers, right?

wait, no

[untold basin]

yes

[rose sonnet]

idk, it sounds horrible, but maybe I'm misunderstanding it

[tawdry surge]

you can also have radix trees with variable numbers of levels

[untold basin]

yeah

[rose sonnet]

but you would need a lot of levels still

[untold basin]

usually thats what you do

[untold basin]

not really

[rose sonnet]

why not just use a bst

[untold basin]

i mean 3 isnt a lot

[untold basin]

a bst would need many more levels

and would use more memory in the process

[tawdry surge]

also, radix trees are very easy to make RCU compatible (= lock free on the read side)

[rose sonnet]

but then you'd need huge allocations again

[untold basin]

4k is not huge

only more than 4k is like a big problem

if you have 512 child nodes (i.e. each node is 1 page), then a radixtree needs ~7 times fewer levels than an rbtree would

[rose sonnet]

you can just shove two pointers into page struct, so it won't?

considering that the pages aren't shared

between objects

[untold basin]

for what

a linked list of pages is insane lol

[rose sonnet]

for bst

[untold basin]

oh for a bst

yeah thats still insane imo

i mean it can work

but like no

[rose sonnet]

I mean linked list works right now

[untold basin]

in general i would prefer to implement things well

[rose sonnet]

I think mine are not

because otherwise it becomes more difficult to know which object the page is for

I also need to do something about anonymous mappings

[untold basin]

wait i also need a reply buffer

thats actually a bit more of an issue lol

[untold basin]

i think the only way to solve that would be to know statically how big the buffer is going to be

which is hard in the general case

like im not sure how youd even solve it

without dependent typing or in my ipc idl or some other bs that is really annoying to support well

idk this is hard

or just reserving a huge amount of buffer space for each reply just in case you get a big one

which is also not ideal

idk if this is a big issue in practice though, even

like maybe its not

i think one of my big issues is that the interaction can be possibly very complex

[rose sonnet]

You can just keep the buffer you've sent the message with, and let userspace choose the size...

[untold basin]

yes and then how do i calculate the right size

by hand?

thats annoying

[rose sonnet]

wdym

when you do a synchronous-ish request, you know what to expect back

or like preallocate something reasonable, yeah

how do you serialize your ipc?

[untold basin]

i dont have anything rn

[untold basin]

wow yeah and what is reasonable lol

thats kinda the issue

[rose sonnet]

Idk, what are you sending

Userspace would probably know

[untold basin]

so its like a structural issue on the userspace side

how does userspace know how many bytes to expect back

if i did know that yeah it would be no problem

i could also constrain ipc coming from pagers

and then youd be able to just copy between memory ranges and itd be no problem

[rose sonnet]

What do you have in userspace?

[untold basin]

one process

thats just a test stub lol

[untold basin]

pager-backed memory in my planned designed is already restricted to being ro

so its not crazy to limit using it for ipc

[rose sonnet]

Where/how do you do memory management?

[rose sonnet]

And like who enforces the IPC message format?

[untold basin]

i dont have ipc so

lol

but like my planned design is that ipc is enforced idk somewhere

if i could figure out how to validate/convert messages when streaming then obviously at streaming time

with some form of kernel based verifier or something

[untold basin]

wdym memory management

[rose sonnet]

Like some microkernels put pmm in userspace, I think

[untold basin]

ok no but thats dumb lol

i was more thinking of putting a vfs in kernel mode

but yeah i think the solution is to copy between the user address spaces

and then buffer sizing is a lot less of a concern

[rose sonnet]

Like my kernel dgaf and just sends messages as they are as arrays of bytes idk

And then it's on userspace to deal with what was sent to it

[untold basin]

ok yeah ill just copy between userspace memory thats so much easier

idk how ill deal with pagers

[rose sonnet]

what's the issue with them?

[untold basin]

they make it easy to deadlock

[rose sonnet]

also, do you mean managarm style thing?

[untold basin]

not really

[rose sonnet]

you can just trace dependencies

[untold basin]

wdym

like if i send a message from process 1 to process 2 and the message is located at memory paged by process 2

[rose sonnet]

like trace who's blocked by who, and chicken out if you see a cycle

yeah

[untold basin]

then to read the message process 2 needs to vmfault on behalf of process 1

[untold basin]

no im trying to avoid dynamic failure modes

[rose sonnet]

or you can just trust that userspace is not too dumb to do those things

the pagers would probably be disk drivers and such anyway, and if someone deadlocks with less privileged processes, it's on them

idk

[untold basin]

i still have to check that

[untold basin]

the problem is that i dont want to allocate a buffer for each message

and i cant let you deadlock the disk server by sending weird messages

[untold basin]

this is very very hard to do without dynamic failure

[rose sonnet]

another solution which I thought about was marking some processes as non-pageable

[untold basin]

i have a more cursed idea but its more complicated

where each process is assigned to a "group"

and i limit ipc such that a process can only talk to processes with a higher group using an ipc call

and then when you copyin you make sure that the pager has an even higher group

and then you are guaranteed to not have a cycle

[rose sonnet]

you can do that with capabilities

I don't think it needs a specific new abstraction?

[untold basin]

idk can you

i dont think thats true in general?

[rose sonnet]

idk

[rose sonnet]

Can you not make sure the message can always be read when receiving it or something?

Also, how do you quickly copy from one address space to the other in the first place?

[untold basin]

switch to one, copy into a buffer, switch to the other, copy out from the buffer

[untold basin]

i mean i guess you could prefault

[rose sonnet]

Like you pre-read it, and lock the memory or something

Though this has a different set of issues...

[rose sonnet]

On someone claiming a lot of memory

But I feel like those are solvable

[untold basin]

okay so ive been thinking about my ipc system some more

and now i think im going to do the awesome thing of making a global sharded lock

because it avoids some reference cycles

[open osprey]

when do all shards need to be acquired at once?

[untold basin]

never

i only ever acquire one shard at a time

although now that you mentioned that, its also a neat hack for dealing with that kind of problem

[rose sonnet]

"microkernels are fine with a global lock because you're not spending a lot of time in kernel anyway"

[untold basin]

lol

[rose sonnet]

(someone on osdev wiki/forums)

[tawdry surge]

well that's not entirely unreasonable

depending on what the lock is for

and how long it needs to be held

[glad bough]

void spin_lock(struct spinlock* lock) {
    global_lock.locked = true;
}

[untold basin]

lmao

[untold basin]

only slightly modified code from my kernel: ```rs
const NUM_LOCKS: usize = 32;
static LOCKS: [CachePad<RawSpinLock>; NUM_LOCKS] =
[const { CachePad(RawSpinLock::INIT) }; NUM_LOCKS];

fn lock(x: usize) -> &'static RawSpinLock {
let id = x.wrapping_mul(0x9E3779B97F4A7C15) >> (64 - NUM_LOCKS.ilog2());
&LOCKS[id].0
}

#[inline(always)]
#[track_caller]
fn with_lock<R>(_tok: PreemptDisableToken, addr: usize, f: impl FnOnce() -> R) -> R {
let sl = lock(addr);
if !unsafe { sl.lock_fast() } {
unsafe { sl.lock_slow() };
}
let res = f();
unsafe { sl.unlock() };
return res;
}

[glad bough]

what

what could the wrapping_mul possibly be doing

[untold basin]

wdym

[glad bough]

magic number

[untold basin]

https://docs.rs/parking_lot_core/latest/src/parking_lot_core/parking_lot.rs.html#352

a suspiciously similar magic number is there

its basically just a hash of the address

[glad bough]

so it's like a salt?

[untold basin]

kinda?

its more like a seed

in this case i just want to avoid having one huge global lock for it

but 32 big global locks is totally fine

[glad bough]

[untold basin]

anyway a lock hashtable solves a concurrency problem i had so its staying

ok awesome im getting more prefetch aborts

this time im trying to run .rodata

[rose sonnet]

are you still doing it in rust?

[untold basin]

yes

safety doesnt help much when you wrote 500loc of probably ub

i have KASAN too btw

[rose sonnet]

lol

average rust experience

[untold basin]

kasan experience too

[untold basin]

im doing rust because its nicer to use and not really because it helps with ub tbh

[untold basin]

i think that shit has never helped me find a real bug

but its all working so

[rose sonnet]

I don't have kasan, so I've just spammed asserts everywhere

[untold basin]

did they help?

ever

i think my issue rn is like a race condition

[rose sonnet]

yes

I mean maybe not with memory specifically

[untold basin]

well i have assert spam too tbf

[untold basin]

im guessing its a race with preemption

[rose sonnet]

just don't preempt the kernel

[untold basin]

ok there we go

[untold basin]

no bit i want to

i might actually remove it though. sooooo much is nopreempt anyway

the only value is finding bugs i would otherwise only hit with smp (and i dont have working smp)

[rose sonnet]

I do have smp though

[untold basin]

and smp isnt as good at finding bugs as turning up the preemption interval to 1ms in tcg is

[rose sonnet]

lol

how many instructions does it get to run before preempting?

[untold basin]

its a fair bit i think

probably like a few hundred thousand or something like that idk

i originally set it to like 100us

and i think it would sometimes get preempted with 0 instructions executed or something insane like that

okay i found the issue

i think

its in my preemption disable code...

my logger disables preemption

its like the one place where i have a recursive lock too

it turns out i added some prints to preemption code, to debug issues

and using logging enables IRQs due to a misfeature in my logic...

whatever, ill throw on a debug assert and ill fix it properly later

for now ill just mask IRQs entirely for uart logging

which is not ideal but who cares

[rose sonnet]

just copy the Ultra's log ring thing

[untold basin]

well no i cant

first of all i think i already said log rings are lame so it would be a bit hypocritical to then use one

but also i want to get tear-free uart logs

and you actually do have to mask irqs for that

[trim seal]

Why did you say that again

[untold basin]

idr

but anyway i dont want a log ring

i want to get tear-free uart

and you cant do that without a global lock anyway

also if you are producing enough logs for this to matter you have other issues

[rose sonnet]

yeah

[untold basin]

this only makes sense for linux because they actually have perf to worry about and i dont

[trim seal]

Wdym by tear free

[rose sonnet]

my kernel has very little logs once it boots

[untold basin]

two cpus can write to uart concurrently

you need a lock to prevent that

[trim seal]

But flushing logs is kind of a separate thing from producing logs

[untold basin]

ok so im gonna make my kernel nopreempt

preemption is dumb anyway

[rose sonnet]

seriously?

[untold basin]

i want the log flushed immediately

[untold basin]

yes

it makes things way more complicated

and i have some weird bug that makes it not work properly i think

[rose sonnet]

you can also make it stack per cpu then

[trim seal]

Thats what happens on Linux in a preemptible context

[untold basin]

no

i still maintain that is dumb

it makes everything so much simpler when you can sleep anywhere

[rose sonnet]

You can still sleep anywhere with that

I mean you also have rust with async stuff

[untold basin]

well no

you have to bail out all the way to the top

[untold basin]

im not doing rust async that shit is cancer

and makes everything slower

imo there are some big design issues with rust async that make it kinda unusable

[rose sonnet]

Yeah

[untold basin]

and i also dont really need it

[untold basin]

and slow

[rose sonnet]

I don't know what I'm gonna do once I implement blocking on memory allocation tbh

[untold basin]

dont do stack per core

[rose sonnet]

Since I just kinda malloc everywhere I want now

[untold basin]

thats also another argument against that

[rose sonnet]

I mean there is a very dumb solution

That's like maybe not dumb

[untold basin]

i mean i dont have blocking alloations rn anyway

[rose sonnet]

My syscalls are reentrant or whatever that's called

So instead of returning an allocation error, I can just block on memory, and repeat everything

[untold basin]

you just run it twice

yeah

which is stupid imo

[rose sonnet]

But like this would only happen when the kernel is very low on memory

At which point everything is slow

[untold basin]

yeah so might as well hard fail

[rose sonnet]

My kernel doesn't (shouldn't) hard fail though

Idk, I want to be nice to userspace and abstract the random ENOMEM bs away

Or like not abstract, but hide

[untold basin]

imo blocking on nomem is just worse in a microkenel

unless you only block in some programs or something like that

[vapid echo]

bruh

[rose sonnet]

I mean the plan is to have the important stuff not allocate

[untold basin]

yeah 100%

[rose sonnet]

But like you'd still need to block on pagers or stuff like that

[untold basin]

i think my kernels sync ipc never allocates

[rose sonnet]

So you're blocking anyway

Memory is just an extension of that

(anonymous)

[untold basin]

but for servers its a bad mix

because its really easy to livelock

posix hangs in allocation, and nothing can ever make progress

[rose sonnet]

Servers can preallocate their shit and run with higher priority

[untold basin]

i think for servers you should have hard enomem

[rose sonnet]

Also it's important for disk drivers and stuff more than anything

[rose sonnet]

This can be handled in libc

Also, I don't have 1 POSIX server

[untold basin]

oh you dont have monolithic posix?

[rose sonnet]

Like with handles you can make everything talk to each other without servers

[untold basin]

uhhh how

[rose sonnet]

It's more of a Managarm thing?

Or keyronex

Idr

[untold basin]

i think its kinda required for some things

unless you bake a lot of posixisms into the kernel

[rose sonnet]

I mean I don't really have process management and I'm rewriting my VFS thing...

[rose sonnet]

mmap flag

And just in general non-blocking flags for syscalls

[untold basin]

or just dont overcommit on anonymous mmap

[rose sonnet]

But that's so convenient

[untold basin]

oh and i guess MAP_DONTWAIT or smth

[rose sonnet]

Yeah

[untold basin]

rn i have no overcommit at all

[rose sonnet]

I don't track how much memory is used by everything at all

[untold basin]

lameeeee

[rose sonnet]

And then you need to account for page tables and stuff like that...

though I guess those can be made disposable

[untold basin]

thats harder

but yeah thats one solution

rn i eagerly reserve them

which isnt ideal

but it works

[rose sonnet]

I do everything lazily on pagefaults

I don't even think I have a way to eagerly map anything

[untold basin]

at least implement linux vm.overcommit_memory=0

[tawdry surge]

Linux also takes a global lock for uart logs

[untold basin]

yeah so lol

[tawdry surge]

I don't like linux's design for this

Imo if you want to log a ton of stuff (like the audit logs by linux LSMs etc) you should have a separate high performance channel

[untold basin]

yeah 100%

[untold basin]

lmao when i ported my locks to my kernel i added a deadlock

[untold basin]

okay i got preemption removed

[rose sonnet]

lol

[untold basin]

now i have to do more async op shit

hmm so i want to make sure you can reserve all resources ahead of time

but for some kinds of operations, it would actually be a bit better if you could instead block the caller at submission until we get enough free memory

or until some other condition is met

this is mostly the case for ipc

where its not crazy to force you to wait a bit for the other side to get enough resources to handle your request

although ig i could just make you queue on the remote side

[untold basin]

so that you can sometimes make more requests than you have resources for

but maybe thats not a good idea

and its better to just allocate ahead of time

[trim seal]

Cooperative kernel?

[untold basin]

nono its volountary preemption

but for usermode its still properly preemptive

so basically its fine

also it makes the kernel easier

[tawdry surge]

skill issue

[untold basin]

is managarm preemptible

[tawdry surge]

yes

[untold basin]

wait so the fuck are you doing coroutines for

[tawdry surge]

wdym

[untold basin]

whats the point of using coroutines at all if you are preemptible anyway...

[tawdry surge]

Concurrency?

Without spawning more threads

[rose sonnet]

but you get concurrency through coroutines already

[untold basin]

hmm

do you even use that

[rose sonnet]

where do coroutines run then?

[tawdry surge]

I agree that making a kernel non-preemptible is fine btw

or rather not-involuntarily-preemptible is fine

[rose sonnet]

I think I'll make mine preemptible by timers and ipis

[untold basin]

i think for a monolithic kernel its problematic at best

[rose sonnet]

only because of latency?

[tawdry surge]

At least you'd have to insert a lot of maybe_yield() calls in a monolithic kernel

[untold basin]

yeah it has latency issues

[tawdry surge]

yeah

[rose sonnet]

you can do preemption checkpoints

but maybe_yield() would be expensive¿

or not...

[tawdry surge]

One thing that Managarm does not allow right now is CPU migration at arbitrary points in the kernel

[rose sonnet]

you can do lazy irql

[tawdry surge]

maybe we'll fix that later but it's not that important

[rose sonnet]

why do you need cpu migration in the kernel at all?

I mean in a microkernel

you can just do costly work in low-priority kernel threads

[tawdry surge]

The main issue with this is that it makes per CPU handling more annoying and also it makes preemption disabling itself more annoying

It could be fixed by moving the preemption disable state into the thread state

but that doesn't sound very pleasant

[rose sonnet]

did Managarm get scheduler work rebalancing?

[tawdry surge]

Managarm has load balancing, yes

Not a very fancy algorithm but it works

[rose sonnet]

I'd need to steal it take a look then

[tawdry surge]

Essentially it just computes per-task load every X ms and then it moves threads in such a way that max_i(load of CPU c) is (locally) minimized

[untold basin]

i think linux does that

[tawdry surge]

https://github.com/managarm/managarm/blob/master/kernel/thor/generic/load-balancing.cpp

[untold basin]

also i think i might make my handle table a hashmap

@tawdry surge wtf why is a frg::hash_map doing chaining

also why please stop

dont use modulo reduction

i guess it makes something approximating sense if you have a hash which is not actually a hash and instead is fucking insanity but still

you dont even preserve insertion order or allow meaningful changes while iterating afaict

[untold basin]

meanwhile my c++ helpers library hashing a u64:

tbf my hashmap doesnt let you update while iterating either

but i have a smarter rehash policy than "just run it up to load factor 1 its fine right"

ah ok when you do separate chaining load factor 1 is actually low

[glad bough]

what does Value && even achieve

[untold basin]

its an rvalue reference

thats normal

[glad bough]

why can't you just Value &

[untold basin]

well i would add more PLZZ_CLANG_DONT_DUMB_ALWAYS_INLINE_KTHX but still

[untold basin]

because then you cant insert(whatever, 3)

and you cant insert(whatever, std::move(something))

thats more or less normal

well my hashmap takes it by value but i think its not crazy to take it by rvalue reference

[trim seal]

You mean universal reference

[untold basin]

no

i think

[tawdry surge]

frg::hash_map is pretty bad, yeah

[lofty hearth]

it is since it's directly on a template argument

[untold basin]

no because its the same type as the one on hash_map

[lofty hearth]

ah true, my bad

need to do more thinking and less talking

[untold basin]

in general i think you could benefit from a bit more always_inline (or maybe im just too aggressive with it)

llvm can often be dumb and miss things

[tawdry surge]

maybe. one thing that we could definitely do better is making sure that slow paths are factored out into [[gnu::cold]] or [[unlikely]] calls

[untold basin]

that too

[tawdry surge]

ty wiki bot

[untold basin]

idk how much that helps

[tawdry surge]

on the hash map side, open addressing is of course way faster

but it also has some disadvantages like the need to rehash on too many tombstones

i think i initially just did closed addressing to have a baseline similar to std::unordered_map

[untold basin]

baseline to a notoriously slow hashmap

but sure

[untold basin]

you can do a combination of having groups (which reduces how many tombstones you end up with) and refcounting the number of overflown keys to (afaik?) eliminate the need for that

[tawdry surge]

do you have a link to a paper that avoids this?

groups = subtables?

or wdym by groups

[untold basin]

folly (facebook nih stl) does this

which is where i got the idea

[untold basin]

groups like in swisstable etc

[tawdry surge]

folly uses SIMD though

[untold basin]

same idea works without simd

idk if its as fast because why would i benchmark

im more of a "yeah that feels fast, lets ship it" person anyway

[untold basin]

there is a SWAR trick to do the same kind of probing without simd

[tawdry surge]

okay, the ref counted tombstones idea sounds pretty neat

I wasn't aware of that

[untold basin]

well you refcount overflows not tombstones but yes

[tawdry surge]

I was aware of tables that use subtables (i.e., use part of the hash to select a subtable and then the rest to index into the subtable) to reduce worst case latency

[untold basin]

go does that i think

but its for iteration order or something

[tawdry surge]

what's also pretty cool for lookup heavy use cases is cuckoo hashing

which has constant time lookup even in the worst case as each key is only mapped to 2 slots (or in general, a constant number of slots)

[glad bough]

https://giphy.com/gifs/meme-battery-adarshinisharma-ObNvwW53YUUCtFZpAa

[untold basin]

at the cost of running at a load factor of lol, afaict

[tawdry surge]

no, the load factor can still pretty high

but the cost is that insertions are slower if the load factor is high

[untold basin]

ah

[tawdry surge]

because insertions have to displace a potentially large number of entries

on collision you find a way to move (potentially recursively) the element that you're colliding with to a free slot

and you rehash if no such move is possible (essentially if the graph of possible moves has no sink)

[tawdry surge]

okay, so if i understand the folly algorithm correctly it does this:

• on insertion, if there is a collision: increment the current slot's overflow counter, move to next slot
• on deletion, if the key is not in current slot: decrement the current slot's overflow counter, move to next slot
• on lookup, if the key is not in the current slot: if the overflow counter of the current slot is zero, return null

[untold basin]

yeah

i think

[tawdry surge]

that leaves the possibility of degradation if for a sequence a_i of identically hashed keys you do:

• insert a_0
• insert a_1, insert a_2, remove a_1
• insert a_1, insert a_3, remove a_1
• insert a_1, insert a_4, remove a_1
  ...
  or similar

afaict

[untold basin]

hm why?

[tawdry surge]

you're essentially creating a long chain of 1 overflow bits

[untold basin]

its a counter not a bit

[tawdry surge]

i know

but i did realize now that may example doesn't work

because you can re-use empty slots even if overflow != 0

[untold basin]

but you also have chunking so you have to insert 14 at once

at that point i worry about the quality of your hash function more ngl

[untold basin]

yeah that too

[tawdry surge]

not necessarily. if you have a long lived table with frequent inserts/deletes but moderate size, this is not an unrealistic scenario

[tawdry surge]

but yeah, this ^ is crucial

this approach seems quite nice actually, i might implement it in frigg

[untold basin]

you need to improve your hashes for that too though

there are other maybe good tweaks you can make: you can allocate the counts separately too try to get a full 8 or 16 key bucket

i think this helps more for non-simd impls

[tawdry surge]

for non-simd, linear probing is usually better than anything else IME

[untold basin]

youd have to benchmark tbh

but maybe

[tawdry surge]

of course

[untold basin]

the non simd parallel probing thing isnt much worse though

[tawdry surge]

of course you need to benchmark but i did some work previously on hash tables and a simple linear probing table is very hard to beat in non-degraded scenarios

[untold basin]

i see

[tawdry surge]

and it is also more cache efficient than the folly table (which is why they do chunking of couse)

[untold basin]

do you have an example impl of that kind of hashmap actually?

i want to see what the perf diff is like

[tawdry surge]

Not as a standalone library but it'd be easy to write

[untold basin]

okay time to do more async

i have implemented a fair bit around async ops already

pub fn sc_port_call_sync_hack(port: Handle) -> Result<()> {
    let port = port.into_object::<Port>(ObjectRight::WRITE)?;

    unsafe {
        block_on_async_qlock(&port.queue, |op| {
            let mut op = op.store(CallTxq {
                proc: Thread::current().proc().unwrap(),
                port: port.clone(),
                res_ptr: Ptr::from_raw(0),
                res_len: 0,
                req_data: user::Slice::from_raw_parts(Ptr::from_raw(0), 0),
                req_handles: user::Slice::from_raw_parts(Ptr::from_raw(0), 0),
            });
            let op_raw = op.get_raw();
            let ql = op.object_mut().state_mut();
            let was_empty = ql.workq.is_empty();
            ql.workq.push_back(op_raw);
            if was_empty {
                unpark_all(&op.object().ops_retire as *const _ as usize, |_| {
                    UnparkToken(0)
                });
            }
            op.retain();

            Ok(())
        })?;

        Ok(())
    }
}
``` rn i have a stub syscall like this

but the real api i want is for the syscall codegen tool to see e.g. ```rs
@kernel_async @__codegen("ksync_deadline") wait_for_event(mask: ObjectEvent, eq: ObjectEvent)

then it can already generate ```c
SK_EXPORT [[nodiscard]] SkStatus sk_object_wait_for_event_sync(SkHandle self, SkObjectEvent mask, SkObjectEvent eq, SkInstant deadline);
SK_EXPORT [[nodiscard]] SkStatus sk_object_wait_for_event(SkHandle self, SkHandle queue, u64 queue_key, SkObjectEvent mask, SkObjectEvent eq);

i just need to generate the kernel part of the API which bridge that to an API like rs fn sc_object_wait_for_event_async<'id>(op: OpTarget<'id>, object: Handle, event: ObjectEvent, eq: ObjectEvent) -> Result<OpSubmitted<'id>>; which submits if the op was set up or Err(err) if the operation fails immediately

and then the _sync variant does this with a local OpRef and the async variant does this with a queue-owned OpRef

[untold basin]

// setup
sk::sys::SkHandle queue;
SK_MUST(sk::sys::sk_queue_create(&queue));
SK_MUST(sk::sys::sk_port_create(queue, /* id */ 0x4141'4242'4343'4444, &test_port));

// thread 1
sk::sys::SkEventInfo evt;
u64 evt_count;
while (true) {
    SK_MUST(sk::sys::sk_queue_wait_for_events(queue, 1, sk::sys::SkInstant{0xFFFF'FFFF'FFFF'FFFF}, &evt, &evt_count));
    INFO("event: (key: {:}, kind: {}, data: [{:}, {:}])", evt.key, usize(evt.kind), evt.data[0], evt.data[1]);

    if (evt.kind == sk::sys::SkEventKind::PortInvoke) {
        SK_MUST(sk::sys::sk_queue_respond(queue, evt.data[0]));
    }
}

// thread 2
u64 datalen, handlelen;
u8 buf[16];
SK_MUST(sk::sys::sk_port_call_sync(test_port, sk::sys::SkCallArgs{}, /* buffer len */ 16, /* handle buffer len */ 0, /* deadline */ -1ULL, buf, nullptr, &datalen, &handlelen));

INFO("thread 2 reply, {} bytes", datalen);

got this working properly

you now have a full sk_port_call_sync api

[vapid echo]

i thought it was in rust

[untold basin]

the userspace is in c++

well "userspace"

its one file lmao

i need to write a vdso glue api for rust still

and thats gonna be a bunch of work too

[rose sonnet]

C++ kernel with rust userspace is much better though

[untold basin]

no

okay so i also need to implement allocating and moving the buffers too

and also i need to figure out what to do about my queue entry structure

my queue entry has two data args rn

and the problem is you cant easily encode a ptr+length in two arguments

although i guess i could just give you a pointer and then behind that pointer store the length inline

and also store the number of handles at the same time

the other idea is to do some shared memory shit where i map some info that is shared between the kernel and userspace including the pointer+length to both areas

i also need some way to give the kernel buffers to use

because the kernel cant really allocate them on its own

[rose sonnet]

Why?

[untold basin]

well because you also need a request id lol

although i guess i could write it into the message header

[rose sonnet]

You know it with handles/capabilities

[untold basin]

well no like my design has an explicit concept of a "call" vs "return"

so you cant just have a handle

[rose sonnet]

Wdym

[untold basin]

my ipc isnt symmetric, its not "send message to the other side"

its "send a call and wait for a reply"

[rose sonnet]

Yes, but how do you give out a handle to reply?

[untold basin]

you dont

there is no handle

its just a number that you look up on the queue

[rose sonnet]

My IPC is kinda the same

I'm not getting what the issue is

[untold basin]

well i have two numbers

and i need to pass more than two numbers of info in there

[rose sonnet]

If you give out an argument without validation, then you also get the issue that the other process might give you the wrong id in the reply, and you will hang

[untold basin]

the id is assigned by the kernel

[rose sonnet]

Or it can die or wherever

Where are those arguments stored?

[untold basin]

well the idea is that i copy them

somewhere idk where yet

thats also like the second half im not really sure about

[rose sonnet]

Like my IPC is just send() and recv() basically, and it's unidirectional

[untold basin]

yes so its completely different

my ipc is not unidirectional

there is a very intentional directionality here, because you cant really make unidirectional ipc guaranteed-failure-free without a lot of effort

[rose sonnet]

My IPC can fail on send, but not on reply

I mean kinda ideally if I implement it, but I haven't bothered

[untold basin]

you mean on recv?

[rose sonnet]

No

[untold basin]

wait how does that work

[rose sonnet]

You will send a message, and then the other side will send you a message back

[untold basin]

ok but what if they cant

what if you run out of memory in the middle

[rose sonnet]

So the right to the send back can hold the buffer to the reply message

[untold basin]

and you just cant allocate a message object

[untold basin]

ah sure i guess that can work

so you have a reply right

hmm

[rose sonnet]

Yes

[untold basin]

the other thing is i dont love the idea of like

using my handle table more than necessary

[rose sonnet]

I mean I have send one and send many rights, but basically the send once right is a reply right

[untold basin]

because its really not that good lol

but anyway i still have the issue that i need to somehow allocate the buffer to put the request into

[rose sonnet]

Both send back a message if you delete them (without replying in the send one case)

[untold basin]

and like conceptually i want to allocate a buffer thats close to the ideal size

[rose sonnet]

So the right itself is a request id

[untold basin]

but how do you allocate a buffer in kernelmode...

[rose sonnet]

I was thinking about letting userspace explicitly create a memory pool in the kernel (like a small slab allocator, idk how to explain), so then when you do that and don't send a lot of messages, send also can't fail on no memory

[untold basin]

well like there are some reasonably good answers here

[untold basin]

"can't fail unless you send a lot" is another way of saying "can fail"

but yeah kinda thats my idea

you just block requests until userspace gives you enough memory of the right shape

[rose sonnet]

No, you can just block if you are above some threshold

[rose sonnet]

Yeah

[untold basin]

the alternative is that when you are reading from a queue you have to supply a buffer

[rose sonnet]

But like explicitly in userspace

[rose sonnet]

This is what I do?

[untold basin]

and you have to size that buffer to be as big as the biggest message at least

and "recv from multiple" can put a bunch of messages in one buffer

but i dont love that

[rose sonnet]

I guess it can be an optimization

[untold basin]

wdym

i need a recv from multiple syscall like thats kinda important

[rose sonnet]

I don't have it

[untold basin]

lol

how do you wait for one of two things at once to happen

inb4 you dont

[rose sonnet]

Yeah I don't really

[untold basin]

like, reading one message every time you do a syscall is really bad

[rose sonnet]

If not, you can just spin up a thread which does it for you asynchronously

[untold basin]

but yeah the big downside of this is that if you have one really big message and one small message, the small message can keep everything live

which is also kinda not great

but i could actually just make it so that userspace copies from the buffer instead of retaining it

[rose sonnet]

(like that's how I'm planning to implement signals in userspace)

[untold basin]

which also has the advantage of being trivial

[rose sonnet]

But like in which sense?

[untold basin]

"wait for either process 1 or process 2 to exit"

[rose sonnet]

My receive handles are per channel

So like you'd ask 2 processes to IPC you on exit, and the messages will both end up in the same message queue

[untold basin]

okay but how do you know which one is which

(also i have some non-userspace events which i want to support)

[rose sonnet]

You get the receive right ID from each

[untold basin]

ahhh

okay sure

ig thats the same setup as i have

[rose sonnet]

Or if it's coming from the kernel, and the same thing can send you different messages, the message itself can hold it

[untold basin]

i have an id associated with this which also works ig

[rose sonnet]

Like I don't have processes explicitly, but I have task groups, and those groups can send you notifications when threads exit

(I need to rework it though, to use rights)

[untold basin]

ok yeah ill just make queue wait_for_events also take a receive buffer lol

[rose sonnet]

Then, since you can only read one message at a time, my plan is to keep a queue of changes in those groups, and a change id, so the same handle can just be once in the message queue, but read out different events to userspace without allocating any memory

[rose sonnet]

That's what I do

[untold basin]

ill let it get multiple messages at once though lol

[rose sonnet]

But like my IPC doesn't let you choose which event you're waiting on

You can only pop front

[untold basin]

mine also doesnt

you can only sk.Queue.wait_for_events, with this signature: ```rs
SK_EXPORT [[nodiscard]] SkStatus sk_queue_wait_for_events(SkHandle self, u64 max_events, SkInstant deadline, SkEventInfo* events, u64* events_actual_size);

[rose sonnet]

So that's what I said, it's the same as calling pop_front multiple times

But with 1 system call

So it's like an optimization in my eyes

[untold basin]

well i already have the ability to get multiple at once yeah

but i have to extend it to ports now

[rose sonnet]

What are your ports btw?

My receive queues are also single consumer and can't be transferred

[untold basin]

ports?

[rose sonnet]

Idk, my kernel is very bad with naming, so I'm not sure if what I call a port is the same thing as what your kernel would have

[untold basin]

my definition is this basically: ```ts
@kernel_handle interface Queue : Object {
/// Create a new queue.
@kernel_static create() -> (queue: Queue)

/// Allocate `count` asynchronous operation slots.
alloc_slots(count: u64)

/// Free `count` asynchronous operation slots.
free_slots(count: u64)

/// Wait for events to be delivered to the queue. Associated data will be
/// written into `data_max`.
wait_for_events(max_events: u64, data_max: u64, deadline: Instant) -> (
    @kernel_stub_size @sized_by("max_events") events: vector[EventInfo]:0,
    @kernel_stub_size @sized_by("data_max") data: vector[u8]:0,
)

/// Respond to a port message
respond(msg: u64) // this signature is wrong btw

}

type CallArgs struct {
@kernel_stub_size data_in: vector[u8]:0,
@kernel_stub_size handles_in: vector[Object]:0,
}
@kernel_handle interface Port : Object {
@kernel_static create(queue: Queue, queue_key: u64) -> (port: Port)

/// Send a message to the remote end and wait for a reply. The reply will be written into the given buffer.
@kernel_async @__codegen("ksync_deadline") call(
    args: CallArgs,
    data_out_max: u64,
    handles_out_max: u64,
) -> (
    @sized_by("data_out_max") @kernel_stub_size data_out: vector[u8]:0,
    @sized_by("handles_out_max") @kernel_stub_size handles_out: vector[Object]:0,
)

}

@__codegen("ksync_deadline") makes it so that you have a sync variant of the function

[rose sonnet]

Ok, I guess it's the same thing as my ports then

[untold basin]

yeah i think so

[rose sonnet]

But my message queue is an intrusive list of messages/events

[untold basin]

yeah thats how its implemented in my kernel as well

[rose sonnet]

Ok

[untold basin]

except with more funny state machines

[rose sonnet]

So then my rights can send themselves as messages when they expire

[untold basin]

also im going to make it so that the owner of a Port can say how many messages from one port can be sent at the current time

[rose sonnet]

And interrupt rights do the same

[untold basin]

as like a congestion control thing

[rose sonnet]

I want to do it per-right

[rose sonnet]

Yeah

[untold basin]

whats the difference between a port and port right in your design

an "object right" in my kernel is just a bitset of things you are allowed to do: ```ts
type ObjectRight bits {
Inspect :0
Admin :1
Read :2
Write :3
Debug :4
}

(no doc comments because i got bored of writing them)

all of the errors have doc comments though because that actually matters

[rose sonnet]

My port is a receive queue, and a receive right is an id of something that can send messages to you

[untold basin]

if you dont specify one you get crashes from the idl compiler

[rose sonnet]

The name is probably bad now that I think of it

[untold basin]

so your port is like my Queue ig?

[rose sonnet]

Initially I just had a right, but I've split it in two

[rose sonnet]

Probably

[untold basin]

and then a receive right (which sounds more like a send right id?) is what i call a Port

[rose sonnet]

It's the other side of a right that can send

[untold basin]

wait so what can you do with a receive right lol

is it just like the controller side of a send right

[rose sonnet]

Also, interrupts create receive rights, and I'm planning to use those for timers and pagers and stuff like that as well

Yeah

If you don't want to receive messages anymore, you can delete it

[untold basin]

for ports im gonna control that with just ObjectRight.Admin

and i can allow revoking with that kind of thing too ig

[rose sonnet]

So the other side can also be notified that it can't send you messages anymore

Or like in case of interrupts, it would mean that you're not servicing that interrupt anymore

[untold basin]

"revoking a right raises ObjectEvent.Closed on it" problem solved

[rose sonnet]

Yes

[untold basin]

i dont implement events properly. my thing used to have that

but i nuked it in the rewrite to rust

[rose sonnet]

So my send many right now has a shared refcounted structure, which is stored as a receive right in the port, and the send right is stored in the rights namespace

So then you can ask the other side to send you a notification when the sender can't accept your messages

Idk, I can send you the code

[untold basin]

do you have syscall definitions

like thats surely enough to understand right

[rose sonnet]

I have doc comments

But I think it's better to look at the kernel code lol

[untold basin]

yeah my idea for that is to have a "wait for(object.events & mask == eq)" syscall

and then you can wait for the event to be raised

[rose sonnet]

I just wait for all of the events with no filters

[untold basin]

so you get notified whenever anything happens?

[untold basin]

my experience with writing async userspace code is that really this kind of thing is what you want

[rose sonnet]

Yes

[untold basin]

oh yeah the events in my design is a bitmap its not an enum

so its like poll flags on linux

so that you can make race-free waits work properly

[rose sonnet]

So then with my idea of making it a restriction per right and not per port in general, you can allocate a large enough buffer in userspace for this, and no one can send any more things on that rights

Which solves my rust async problem

But like it won't block other things

[untold basin]

for rust async i explicitly added sync cancellation for my kernel api just to make that shit work properly

because lol

i dont have it implemented but like its the vibe yk

[rose sonnet]

All of my IPC definitions in kernel are these (then I also have interrupt rights which inherit from this, but the mechanism is the same)
https://gitlab.com/mishakov/pmos/-/blob/main/kernel/generic/messaging/rights.hh?ref_type=heads
https://gitlab.com/mishakov/pmos/-/blob/main/kernel/generic/messaging/ports.hh?ref_type=heads
https://gitlab.com/mishakov/pmos/-/blob/main/kernel/generic/messaging/messaging.hh?ref_type=heads

[untold basin]

holy shit so much c++

oh yeah and its the awesome super c++y c++

[rose sonnet]

Then the userspace part is more or less this https://gitlab.com/mishakov/pmos/-/blob/main/libs/sysroot/include/pmos/ports.h?ref_type=heads
https://gitlab.com/mishakov/pmos/-/blob/main/libs/sysroot/include/pmos/system.h?ref_type=heads#L172

Then I also have C callback wrapper for all of this, and async stuff/runtime for C++ and Rust

In userspace

[untold basin]

ports_request_t create_port(pid_t owner, uint32_t flags);

result_t pmos_delete_port(pmos_port_t port);
``` i love the naming consistency

[rose sonnet]

Yeah

[trim seal]

Lol

[rose sonnet]

It's a mess

[untold basin]

i have a really awful codegen tool which generates the syscall stubs + kernel syscall wrappers + syscall headers

[rose sonnet]

I'll fix all of this when I switch to mlibc

[untold basin]

so i at least have consistent naming

of sk_[object name]_[function name]

[rose sonnet]

I don't even have vdso

[untold basin]

(technically its [object name]_[function name] because the objects are in the sk package because its hacked on to what i want to become an ipc idl compiler)

[untold basin]

lameeee

[rose sonnet]

And I kinda really want that for i686

Because you can't just assume syscall and using int for them is lame

[untold basin]

and im targetting arm64 where i really dont need it

[rose sonnet]

But whatever

[untold basin]

but i also use it so that i can simplify the kernel ABI

so for example sk.PhysicalVMO.create_memory is implemented as ```c++
SK_EXPORT [[nodiscard]] SkStatus sk_physical_vmo_create_memory(u64 addr, u64 size, SkHandle* object) {
usize x0_in = (usize)addr;
usize x1_in = (usize)size;
usize x8_in = 16;
register usize x0 asm("x0") = x0_in;
register usize x1 asm("x1") = x1_in;
register usize x8 asm("x8") = x8_in;
asm volatile goto("svc #0; b.lo %[error_sk_physical_vmo_create_memory]" : "+r"(x0) : "r"(x1), "r"(x8) : "memory" : error_sk_physical_vmo_create_memory);
usize x0_out = x0;
object->value = x0_out;
asm("" :: "r"(x0_out) : "memory");
return SkStatusSuccess;
error_sk_physical_vmo_create_memory:
[[clang::unlikely]];
usize x0_err_out = x0;
return (SkStatus)x0_err_out;
}

[rose sonnet]

Also, my receive is 3 syscalls basically

[untold basin]

also btw

you might want to put most of pmoscxx/src/helpers inside of a header file

so that it can get inlined

[trim seal]

What the hell is an asm goto

[untold basin]

its asm where you can jump to a label as if with goto

[trim seal]

Is that a thing in x86 also?

[untold basin]

its a thing on all platforms afaik

[trim seal]

Never seen anyone use it

Why does it have to be in asm in this case

[untold basin]

because i branch on flags after the syscall instruction

the syscall sets PSTATE.C if the syscall succeded (or clears it idr)

[trim seal]

Ah

[untold basin]

its set on success

[rose sonnet]

These are my syscalls, and my solution to blocking https://gitlab.com/mishakov/pmos/-/blob/main/kernel/generic/processes/syscalls.cc?ref_type=heads#L802

[untold basin]

lol

[rose sonnet]

They were in headers, and I didn't like it

[untold basin]

yes but big fast.

[rose sonnet]

C has it afaik

[untold basin]

idk if it actually helps significantly

its a gcc thing yeah

its not like a per-arch thing

the name is unique because clang wants that

and the really dumb asm("" :: "r"(x0_out) : "memory"); thing is because otherwise it doesnt regalloc correctly

[tawdry surge]

Linux uses it in many places

for example for static branches

Maybe also for the alternatives mechanism

[untold basin]

yeah

[tawdry surge]

https://elixir.bootlin.com/linux/v6.6/source/arch/x86/include/asm/jump_label.h#L25

[tawdry surge]

Is a right in your system a capability or not?

[rose sonnet]

Yes

[tawdry surge]

I'd strongly advise against using ad hoc allocated IDs to identify senders and receivers

[rose sonnet]

why?

[rose sonnet]

I mean the send right is, I'm not sure if my recieve right would be a capability

[tawdry surge]

Because that raises a lot of issues like: when do you allocate, when do you deallocate, how do you verify the remote identity, etc

[rose sonnet]

I haven't had those issues yet?

It's 2 IDs actually, one for the sender, and one for the reciever

I'm not sure what you mean with allocation/deallocation

[tawdry surge]

You'd need to describe how your system works in detail of if you want somebody to critique it but I'm very skeptical that a good design is possible that identifies senders and receivers by numerical IDs

instead of proper capabilities

[untold basin]

i give senders numerical IDs but only for the benefit of the callee

i.e. the caller never sees them

for a port you are meant to put a pointer in and for a single request its just a number that is used as a handle for the kernel

for some rbtrees

[rose sonnet]

I do the same

[rose sonnet]

But I'm not sure what a proper capabilty would be

[tawdry surge]

To what do you attach these IDs? To the thread that sends something?

And why is the ID even useful

[untold basin]

for a port: to the port itself

[rose sonnet]

In the reciever, I have capabilities namespaces

[untold basin]

for an invocation: to each message (specifically, each AsyncOp with optype=CallTreeq has an id, assigned when it is assigned to the type)

[rose sonnet]

The ID is to find the right

[untold basin]

for ports the utility is obvious (you need to know what caused an event)
for invocations the utility is also obvious (the kernel needs to know what you are replying to)

[rose sonnet]

Yeah, when they are being sent, I transfer the ownership to the message, so the number in the reciever gets assigned when you recieve it, as it gets transfered to your active capability namespace

[tawdry surge]

So on ipc, a cap is inserted into the receivers cap space?

What prevents denial of service on the receiver side?

[rose sonnet]

When you recieve it

[tawdry surge]

What is a port in this context?

[rose sonnet]

Like you know what capabilities you've recieved when you get the message, you kinda accept them. Then you can either delete it immediately, or it will get deleted if you don't recieve the message

(like if the reciever dies)

[untold basin]

my ipc has two sides: a port which is the client part, and a queue which is the server part (also you can use a queue to do async io in general)

and each port is with a queue

[tawdry surge]

Re the kernel needs to know to whom you're replying: that's not necessarily true. For example, if your ipc always connects two endpoints, the receiver is implicit

[untold basin]

needs to know [within the context of my design]

obviously this isnt universally true

you could easily make everyone see every message

and then you dont need anything

it would be kinda dumb yes but it could work

[rose sonnet]

My ipc is unidirectional

[tawdry surge]

Is the port created from the queue and then moved to the client side or how does that work?

[rose sonnet]

wait

sorry, no, I've replied to the wrong thing

[untold basin]

yes