charmOS

[twin escarp]

Can you do both SMP and NUMA, or like, can you only do one of them?

[leaden magnet]

NUMA without SMP doesn't make sense

[eager yew]

the point of NUMA is because of SMP scalability issues

in particular memory bus lock contention became a problem

that's why we have NUMA

[twin escarp]

ahh, okie 👍

[leaden magnet]

btw i have a x86 numa system, if you want to test just @ me

[eager yew]

another issue related to the "local node exhaustion" problem is that you'd ideally also want the caller to specify some numa hint

for example, if you allocate an object that is infrequently accessed, and the local node is low on memory, it would be OK to allocate from a remote node that has a lot of memory

inversely, if you allocate an object that is very frequently accessed, and the local node is low on memory, it would be more optimal to use the local node's memory since it's a frequently accessed object

so that's also something that has to be taken into acccount

[leaden magnet]

but that stuff is hard to guess besides the basic quick allocation to get userspace data or similar

[eager yew]

wdym? stuff like a thread stack should definitely be node-local whereas other things like global DMA buffers that are used anywhere doesn't matter as much

[leaden magnet]

i was thinking more about slab/kmalloc

[eager yew]

you can always expand your alloc api to be something like
kmalloc(size_t size, uint32_t flags, enum numa_alloc_hint hint);

[leaden magnet]

do you support different archs? iirc x86 doesn't have that many 2+ numa systems?

[eager yew]

I might support different archs but for now it's not a high priority

I'm literally only doing this NUMA thing to have fun and learn things, the system I plan to run this on (Thinkpad t400) uses an intel core 2 duo and it's from 2008 or so lol

all of this is massively overkill for that hardware, I'm just doing it to learn and have fun

[eager yew]

pov: life without menix DAG-based INIT

[eager yew]

sanity 👍

(this exists because it would die on systems with too many cores with just a simple 64 bit bitmask)

(very lovely)

[eager yew]

nice

wait it broke with many threads per core nvm

💥

all is good

yippee

qemu would not boot with 1024 cores but the cpu mask thing should work fine

ok next up we will need to write traversal functions and also hold a pointer from the core to the given node so I can easily do that

I'll wanna traverse cores from SMT siblings -> NUMA node -> Full machine in an optimal scenario

gonna be great

all this work for a core 2 duo thinkpad 🤪 ✅

oooof I just realized some components will need rewrites to Not Suck on high core counts, like NVMe devices typically support 64ish queues so if you have more than that many cores things are suboptimal

ok all good, allocator rewrites are in order soon oh boy

hopefully rewriting allocators and then filesystem stuff is the right order since fs stuff loves allocators and if those arent fast stuff isnt the best

[eager yew]

https://bpa.st/SNVQ yap

[steel herald]

Wdym? Ofc it does, its even configurable via bios, some systems can be set from 2 all the way up to 8 numa nodes

[leaden magnet]

i mean like systems to test it on

or can you configure it to fake nodes?

[steel herald]

Not fake, no, it can pretend some stuff is within the same numa node to reduce the amount of them, but in reality for example latencies are ideally grouped into 8 numa nodes

Usually bios has a setting for that

[leaden magnet]

how is this setting called?

[steel herald]

Depends on the mobo and ofc only high end stuff would have that

Some examples

[steel herald]

As you can see even the cache can be a numa node

[gentle gazelle]

numa so good even the rpi4 firmware adds numa=fake=2 to the kernel commandline automatically :^)

[placid vessel]

[eager yew]

ok i think for the physical memory allocator I can create "Core Domains" which are effectively the "Middle Layer" between a core-local allocation and a machine-wide scan based allocation

the domains would just be groups of like 4-8 cores on UMA systems, but on NUMA systems would directly correspond to NUMA nodes

this way I can kinda make "invisible NUMA awareness" without hardcoding NUMA nodes everywhere by using generic domains, and UMA systems can probably also benefit from this

[eager yew]

"We have polymorphism at home" 💔

[eager yew]

processor topology classic

whoops package and numa node should be other way around

it usually doesnt matter but other way around is correcter

there we go

processor topology classic

[placid vessel]

what is CPUs=

[eager yew]

bitmask

possibly dynamically allocated too so it doesn't crashout with more than 64 total SMTs

[severe abyss]

what cpu is it

[eager yew]

qemu with some whimsical settings lol

    -smp sockets=2,cores=2,threads=2
    -object memory-backend-ram,size=2G,id=mem0
    -object memory-backend-ram,size=2G,id=mem1
    -numa node,cpus=0-3,nodeid=0,memdev=mem0
    -numa node,cpus=4-7,nodeid=1,memdev=mem1```

[placid vessel]

sockets=2

mfw

[eager yew]

yes 👍 you all should use these settings

these are peak settings for qemu

lets you aura farm

[placid vessel]

how to uacpi anti points

[eager yew]

struct cpu_mask {
    bool uses_large;
    union {
        uint64_t small;
        uint64_t *large;
    };
    size_t nbits;
};```

💥

[eager yew]

packages are just sockets 💥

I like how these days CPUs can mean like 3 different things

because you have

CPU as in "SMT"
CPU as in "Core"
CPU as in "Physical CPU/package"

🤪

[eager yew]

hmmmmm I just realized I may need to deal with heterogenous processor topology while I'm at this so the scheduler can differentiate between the P cores and the dysfunctional E cores

[vagrant cedar]

Don't forget CPU as in "All CPUs on this motherboard"

[eager yew]

oh right yes that one too

4 different things all referred to with the same word

very sane and normal 👍

and then when people refer to cores as CPUs and SMTs as threads

💥

"How many CPUs do you have?"

"One"

"No I mean CPU as in cores"

"Eight"

"Does that also include threads?"

"No, that would be sixteen"

rational dialogue

[severe abyss]

do you have ideas how you will handle heterogenous topology

[eager yew]

no

probably migrate higher priority threads to P cores

💥 llc data

[eager yew]

another processor topology classic

it looks identical here but i also implemented the logic for it to just copy packages if no llc data is present

I hope this all cooperates when I disable numa and make a fully flat topology lol

woah it actually works

that's shocking

i thought it would just die

nice

500 lines of code for a "Low Budget Tree" 💔

[steel herald]

How do u parse this info?

[eager yew]

ACPI SRAT and CPUID for now

I'll also be using the SLIT soon for memory distance yada yada

[steel herald]

Ah

[eager yew]

very simple

[eager yew]

i just rembered to do x2apic stuff coz that one has the 16 bit CPU limit that is chad and sigma and alpha so i went and did that 👍 took like 5 minute ezpz

anyway back to planning out the allocator

[stuck nexus]

see yo lator alloucater

[steel herald]

Rare playful hyena

[stuck nexus]

usually it means im suicidally depressed

[steel herald]

[steel herald]

Dont be sad

[stuck nexus]

okay i wont............. love you infy................. babygirl...................

[steel herald]

Good

[eager yew]

so the idea would be to use those "core domains" to group cores into "groups", on UMA systems these would just be groups of 4-8cores, whereas on NUMA they would directly correspond to nodes, since this way UMA systems can also benefit from this logic

then, the eventual kmalloc would look like

kmalloc(size_t size, uint16_t flags); where flags is a bitmap and has stuff like

ALLOC_FLAG_PREFER_LOCAL/ALLOC_FLAG_PREFER_BALANCED
ALLOC_FLAG_PREFER_CACHE_ALIGNED/OFF
ALLOC_FLAG_PAGEABLE/ALLOC_FLAG_NON_PAGEABLE

and then I can group these into things like

ALLOC_FLAG_FAST = ALLOC_FLAG_PREFER_CACHE_ALIGNED | ALLOC_FLAG_PREFER_LOCAL | ALLOC_FLAG_NON_PAGEABLE

or

ALLOC_FLAG_BALANCED = ALLOC_FLAG_PREFER_BALANCED ... and whatever, but the first big deal is the physical memory allocator

the physical memory buddy allocator would work with mini-buddy allocators in each "domain" with per-core arena caches, and the path with allocations can work like this

first check if we have any available pages in our arena cache, and just take it from there if anything is available

then check if our domain is balanced (memory usage is LEQ to average memory usage per domain) -> if it is, just pull from domain, ignore flags

if it is not balanced, check if flags want LOCAL or BALANCED, if they want balanced, figure out which domain is optimal for picking from, but if they want LOCAL try and see if we still have enough memory in this domain to pull from

each arena cache would have some given limit of how many pages it stores, and each core has a small ringbuffer of pages to free, so when a cross core free happens, the flow is

check if other core's "garbage collection" list is not full -> if it's not full just put our page here, if it is, acquire that other domain's lock and that other core's lock and do the free

something like that

idk

💥

[steel herald]

Does Linux have a numa node hint in kmalloc?

[eager yew]

me no think so, it's not explicitly "NUMA" related

same as how my hints arent explicitly NUMA

but they can be if the system is NUMA

[steel herald]

Ah

[eager yew]

I just don't want to tie a whole bunch of logic directly to NUMA and would prefer a generic abstraction that works well with NUMA and UMA

hence "core domains", like the linux scheduler domains

i think they call it that

[eager yew]

the big pickles here would be fragmentation (hugepages) and domain imbalance so that might take some fine tuning or dynamic runtime computation to avoid massive imbalances from lots of LOCAL allocs

[steel herald]

Thats why Linux has numa migration of movable pages

Using the same tech as it does for memory defrag

[eager yew]

yes very fancy stuff

[steel herald]

Its kinda complex but doable

You just put a fake pte entry and if it the process faults on it you spin until the migration of this page completes

[eager yew]

ya but i'll first try and get this buddy allocator revamp skeleton, that stuff comes later when I make the pager

[steel herald]

Yeah fair, I just think this is an important piece of numa support

And you would probably be the first hobby os to support live page migration

[eager yew]

numa would be a cute name for a pet or a person

[eager yew]

what about that zambesii thing

[steel herald]

What is that

[eager yew]

https://github.com/latentPrion/zambesii

do u rember this

they have some NUMA stuff

idk about page migration

[steel herald]

Iirc its mostly demo numa support to an extent where I wouldn't even call it support

[eager yew]

don't most mature hobby kernels have parallelized allocators (allocators that are "per-core" and don't have a Big Function Lock)

idk i havent looked too deep into hobby kernels i prefer the real ones

[steel herald]

Very few do, and if they do its definitely not numa aware in any way

[eager yew]

that's so sad

puter no go fast 😢

[steel herald]

Well it's kinda difficult to get access to numa hw as a normal person unless ur kinda rich

[eager yew]

understandable i am overly fortunate in that aspect 💥

i dont directly own any

but have frens that will lend me some perhaps

and also qemu numa flags coming in clutch fr

one reason behind me doing numa other than for the meme is because I'm doing this allocator rebuild and came up with the "core domains" idea and went "wow this is numa at home" so it just felt like a natural next step

[steel herald]

Yeah and qemu memory backend can allocate with numa binding so you can actually measure that your os is behaving correctly if running on a real numa host

[eager yew]

oh wait fr? that's so cool omg

qemu win once again

[steel herald]

Yeah there's like a policy=bind or whatever it was I dont remember

[eager yew]

I think my parsing of the Last Level Cache is entirely useless on Intel CPUs since I believe over here the LLC maps directly to a package (socket)

AMD has the funny business i believe

[steel herald]

And it has a numa node argument also

No clue

[eager yew]

    -m 4G
    -smp sockets=2,cores=2,threads=2
    -object memory-backend-ram,size=2G,id=mem0
    -object memory-backend-ram,size=2G,id=mem1
    -numa node,cpus=0-3,nodeid=0,memdev=mem0
    -numa node,cpus=4-7,nodeid=1,memdev=mem1``` my beloved

[steel herald]

I mean that the memory backend object has flags to bind to a real numa node and a numa node idx of host

[eager yew]

ohhh right yes

[eager yew]

I still occasionally get the giggles when I read past the part of linux init source code that just blindly maps the first 4gb of memory

[eager yew]

ok so due to the nature of how I bring my physical memory allocator online I think it'll have to be 3 stages

first we start with a static bitmap that tracks about 4GB of memory which is plenty for everything I'll ever need, and it's only 128KB of memory overhead, which is already not too big, but disappears if I throw it in __init because it just gets freed after early init

then we use this bitmap allocator to obviously do the early mapping and whatever, and we initialize the global buddy allocator afterwards

once the global buddy allocator is alive, we do mid-init activities, such as bring up the other allocators (slab, hugepage), wake the other APs, yada yada

then afterwards, once we know the processor topology, we go back and do a late stage init where we set up the fancy parallelized buddy allocator stuff such as the domains, per core arena caches, whatever

I can reuse the existing buddy alloc code but just pass in the domain buddy instead of the global one and free the global buddy once I construct the new ones

I can rely on the other cores being guaranteed to be in their idle threads to safely modify this here so I should be ok

💥

I wonder if I have an average 💥 rate I can derive

hmmm

it is 2.4%

472 booms, 19,750 total

highly insightful

[placid vessel]

bsd

[eager yew]

acpi 👍

[steel herald]

It might be retarded but it really simplifies firmware and IMO every table should've had this

[eager yew]

exploooooode

yey

I hope acpi tables dont lie about the existence of numa nodes like they do with cpu cores

ima check that rq

[eager yew]

hmmmmm my boot bitmap that tracks 128mb of memory (it's only 4KB in size) would only be able to work on systems with less than 4TB of memory since it takes approximately 128mb to do initial allocations that use that boot bitmap

rational concern 👍

[eager yew]

my OS has not gone crazy in a long time this is suspicious

[eager yew]

💥

neat

[eager yew]

I think when I do the ACPI C state thing I will just try on real hardware because I had a lot of type 3 fun today trying to get it to work under qemu

gargantuan issue of skill probably

[round geyser]

How’s your Numa allocator going

Super interested in that tbh

[eager yew]

it's going good, I've worked on setting up domains (a generic "group of cores" abstraction) and their free queues (cross core frees enqueue here, or they just directly acquire the lock and free) and per core arenas (fastpath to avoid domain lock contention)

currently trying to work out heuristics and just round the edges around the allocator API that I want to expose

👍

i got so tired of writing out all those lock/unlock boilerplate things

from now on, no more

macro 🤪

[round geyser]

Fuck spinlocks fr fr

[round geyser]

Dude same

All my damn init functions

Have so many spinlock shits

[eager yew]

i mean these

💥

oh wow using this macro is so clean omg

macros might be alright guys

"zero cost abstraction"

[round geyser]

Macros are great

Like

Really good

For many things

They’re over hated here

Because a lot of shit gnu software uses them

Everywhere

Imo macros are better than inline functions a lot of the time

[eager yew]

type safety is a myth

[round geyser]

Yes exactly

That’s the only “benefit” they have

Also

Macros are more optimized

Because they are simple text replacement

[eager yew]

rust macros better tbh

tt muncher

[round geyser]

This is extremely fucking true

Rust is just better lmao

It’s significantly better than C

[eager yew]

no

[round geyser]

Just not as well suited for kernel

[eager yew]

rust not as portable

[round geyser]

For applications programming and userspace programming rust is better

True

I’m talking more of the syntax and stuff

The portability isn’t there yet

[eager yew]

rust only has llvm backend (it can be dumb and i dont like how i cant tell it to use gcc)

[round geyser]

Unpopular opinion

Gcc is better

[eager yew]

lukewarm take

[round geyser]

Idk lots of people shit on gcc

[eager yew]

probably on their codebase

[round geyser]

c-parser.c

Yeah the codebase isn’t good

No level of modularization

Like

Why is the entire c parser in one file

That is SO braindead

[eager yew]

compile time optimization

write never read never codebase

[round geyser]

Lol

[round geyser]

But even stuff like sqlite compiles from one file but it is still modularized in the code base

[eager yew]

it's a write never read never codebase, their C parser isn't something that is modified often, and when it is modified, it's not like a huge rewrite, and for stuff like this it can be more optimal both for codebase organization and for compile time optimization to just have one blob

so big file

[round geyser]

Yeah true

[round geyser]

Yeah their c parser has barely changed in 20 years

Looking at the git blame

[eager yew]

gnu codebases are open source and open sores

six seven

[round geyser]

Yes sir

I only have 231

Mainly because I suck at version control

But I’m getting betyer

I at least update once a week now

[eager yew]

ok so I think the public API might need some tweaking

I think rather than having a simple LOCAL/BALANCED flag, I'll probably want to give this a "locality degree" value between 0-8

for example a value of 0 means "prefer not local at all" and a value of 7 means "absolutely positively make this local, and fail the allocation if it cannot be done locally"

this way, when I go and pick a place to allocate from, rather than looking at a binary flag, i have this scaling value I can use, and it also allows me to keep the same (size, flags) arguments (but have some bits of the flags used for locality preference rather than an on-off)

for example, if I get a locality value of 7 I know that there is zero reason to ever search other nodes, and I will only allocate from the local buddy and fail if that fails

meanwhile, if I get a locality of 4, I might add bias to the node balanced selection logic to steer it towards "closer nodes" even if there are unbalanced farther nodes

or if I get a value of 0 I could just simply pick the most optimal node regardless of distances and apply the tiniest bias away from farther nodes just for NUMA friendliness

of course all of this goes out the window, except for the value of 7, if the system is UMA

[round geyser]

Jesus

Lots of text

[eager yew]

bro is a yapper....

[round geyser]

I’m tryna understand it

[eager yew]

i think linux calls this Memory Policy or MPOL

I will probably need to start worrying about page migration soon coz once I fix up allocators I'll start doing the pager stuff and the first pager thing I'll probably get to is page migration

ok yeah this needs to be changed, I'll probably want allocation classes like the linux GFP doohickey, so I can pass in a (size, class, flags) where flags would contain locality distances and the class would control other business

I can probably also integrate my thread activity classes and stuff into this but that seems overkill (caller thread's activity class biases locality)

yea ok it seems like linux uses zonelists to determine which nodes to spill to so I can steal that idea

ah it seems like linux has a "round robin" method of interleaving allocations between nodes

so each core/thread could keep some rr_counter so its allocations would gradually get spread out across nodes

very cheeky

oh wait that is pretty smart yea

not my code this is qsort

[eager yew]

golang roleplay

[gentle gazelle]

fake, you have syntax highlighting enabled

[versed crystal]

macro_rules! my beloved

[eager yew]

accurate....

[eager yew]

anyways I got allocations up and running today with the domain (numa) allocator

paddr_t domain_alloc(size_t pages, enum alloc_class class,
                     enum alloc_flags flags) {
    /* We only care about INTERLEAVED at the buddy allocator level */
    if (class == ALLOC_CLASS_INTERLEAVED)
        return alloc_interleaved(pages);

    /* Fastpath: Get it from our local arena */
    paddr_t ret = try_alloc_from_local_arena(pages);
    if (ret)
        return ret;

    /* We don't care about any other flags in the domain buddy allocator */
    uint16_t locality_degree = ALLOC_LOCALITY_FROM_FLAGS(flags);

    /* No other options. Allocate from this buddy. */
    if (locality_degree == ALLOC_LOCALITY_MAX)
        return alloc_from_this_buddy(pages);

    return alloc_with_locality(pages, locality_degree);
}``` why does this end up reading like pseudocode

whatever

ok next up is just tweaking this stuff, using domain stats (I already keep track of) to bias decisionmaking, and then move on to frees

ah wait I was gonna implement an alloc_flag for ALLOC_FLAG_FLEXIBLE_LOCALITY that would make the locality not fixed to a number

right gotta do that 💥

[round geyser]

i think it reads like pseudocode because you have alot of subroutines doing sutff for you

like alloc_with_locality() and alloc_interleaved

but its not bad

[eager yew]

ok so domain freequeue flushing needs some work

firstly, I think I will need to allow allocations to perform a bit of local "flush our freequeue to our arena" logic, where I can flush a portion of the freequeue (say, 1/domain->core_count, since that allows other cores to do the same). these shouldn't touch the other arenas just for better lock contention

then, I can probably spin up one background thread per domain (threads that run once everything else finishes), and have this background thread get enqueued if it is not already enqueued (but not run, obviously, since we also enqueue this from a thread's context) whenever an allocation happens

then I can probably use my per domain stats to keep track of free memory usage and perform more aggressive flushing

[placid vessel]

isn't that a good thing?

[eager yew]

probably, i just find it curious

a lot of my code ends up reading like pseudocode, i guess it's just a byproduct of how i write things

when i design the thing I go top->down and try and figure out "what high level APIs I want to create" before figuring out the intricacies

then when i implement things i go bottom->up since I design the tiny helper functions before moving to bigger and bigger thigns

anyways

> git commit -a
[main aec4e5d] [buddy]: fully migrate to new domain buddy!
 8 files changed, 196 insertions(+), 81 deletions(-)```

numa aware allocator is done

I just need to go and double check that i dont have any devious race conditions in there and we are good

obviously no performance gains are present right now because all calls to this thing happen with a slab allocator with a Big Function Lock (or the hugepage allocator, but that rarely calls into this)

so next up we first

double check that my code isn't fried -> clean up some stuff -> add more API interfaces for the domain NUMA aware buddy allocator -> fix up the slab allocator and hugepage allocator to also make them handle these new features -> test and check those -> start working on the pager -> start implementing the basics of moveable pages -> rest of pager -> to be continued

why is the gh cli so astoundingly slow

whatever

"How many physical memory allocators does YOUR kernel have?"

wait a minute if i implement moveable pages this means i can potentially have hotswappable ram support

that's crazy

[eager yew]

done

just need to round out the edges and then add other support in the slab allocator

[round geyser]

Bet lemme look

[leaden magnet]

Or memory ballooning

[eager yew]

hyper v reference

I love that feature of that bum ass hypervisor

it's a pretty neat little trick it does

[eager yew]

note to self to remember to merge buddy page data with my struct page flags and information and make the page arrays for each node's memory region also reside in that node, and use the address offset to derive the pfn so the struct page can save 8 bytes

once I have that up and running*

I will probably try and keep struct page under 32-48 bytes so that it doesn't explode the memory usage and just hash the address of each struct page (or I can use the pfn, idk) into different hash tables to store more data about it

[eager yew]

feeling silly, might just

zabble zinkle zorp, forp zinky zoogle zeekybooble meep beeble deebo beebo, "Non Uniform Memory Access", boogle zinkbo doogle paggle zing zong zang, beeble meeble zart poogle. daaba dooba bing bong, poog doog, "Page Migration", zipple zoople zapple, boinky doinky zingle dingle pooble baggle binky bonky.

zooba dooba daba poodo moodo, zaaba daaba gaba dinko blinko, "Page Block", yibble yabble babble zabble

there is a message in here

[eager yew]

perl -pi -e 's/\bbool\s+\w+\s*=\s*(\w+_lock\s*\([^)]*\))/enum irql irql = $1/' $(grep -rlP "bool\s+\w+\s*=\s*\w+_lock\s*\(" .)``` sanity 👍

[round geyser]

this looks like a cursed kernel panic from hell

[eager yew]

just a find and replace to migrate out legacy spinlock things

[round geyser]

ah i see

[eager yew]

perl my beloved

https://tenor.com/view/arab-gif-15239312808874545560

[eager yew]

at the start of this kernel, I made a "One Size Fits All" spinlock that would just entirely disable interrupts but now that i have IRQLs and other fancies I figure out that that is quite suboptimal so I gotta go in and throw out all the legacy cruft

i will soon also be fixing my include/ directory and make headers not suck

[round geyser]

kinda like a bkl but slightly more sane

[eager yew]

no

[round geyser]

headers will always suck imo

[eager yew]

not a bkl at all

[round geyser]

i saw one size fits all

but i didnt read it all

but yeah i disable interrupts in my spinlocks

i dont have irql's

so maybe once i add those

[eager yew]

my plan to make it not suck is to have each subsystem keep **_internal.h files near the .c files, and use these for internal functions and structures that the outside world doesn't care about

then the include/xx directories would just contain public APIs

for example my allocators wouldn't have all their internal garbage collection, flush, etc. functions sitting around in include/ and would instead just expose

alloc, free, hint, and a few other high level functions

this way I can benefit from

a. shorter compile times
b. less header recursion issues
c. cleaner headers and separation of concerns
d. better expandability because external headers will be 50 lines instead of 500

🐸

[round geyser]

i see

yea i remember wux was doing something like that a while ago

we talked about it on vc

[eager yew]

average c experience

i feel like headers and things like forward declarations are just legacy things for C from the 70's

especially nonsense like forward struct declarations

[round geyser]

yeah its just an archaism

[eager yew]

#define SPINLOCK_GENERATE_LOCK_UNLOCK_FOR_STRUCT(type, member)                 \
    static inline enum irql type##_lock(struct type *obj) {                    \
        return spin_lock(&obj->member);                                        \
    }                                                                          \
                                                                               \
    static inline enum irql type##_lock_irq_disable(struct type *obj) {        \
        return spin_lock_irq_disable(&obj->member);                            \
    }                                                                          \
                                                                               \
    static inline void type##_unlock(struct type *obj, enum irql irql) {       \
        spin_unlock(&obj->member, irql);                                       \
    }                                                                          \
                                                                               \
    static inline bool type##_trylock(struct type *obj) {                      \
        return spin_trylock(&obj->member);                                     \
    }``` 🤪

oh boy

sily

[eager yew]

@round geyser you should probably whip up a macro like this soon, it makes a lot of code look significantly cleaner

[round geyser]

yeah because all my init functions have bullshit spinlock init fuckery

[eager yew]

rather than having raw spin_lock(&doohickey->lock); you can just say stuff like doohickey_lock(doohickey)

demure and mindful

[round geyser]

yeah

this is my bullshit

static spinlock_t buddy_lock_initializer = SPINLOCK_INIT;
buddy.lock = buddy_lock_initializer;
spinlock_init(&buddy.lock);

and i just called spinlock(doohuckey->lock)

in my pmm

[eager yew]

💥

[round geyser]

but youre right

i sohuld make a macro like that

currently

im actually rewriting lots of my smp stuff

[eager yew]

only good use of macros

jk

[eager yew]

im bored i'll give threads names

[eager yew]

wow linux has got a ____cacheline_internodealigned_in_smp

quadruple underscore

[round geyser]

Most readable Linux code

[eager yew]

nyaux is at risk of its spot being taken

anyways back to page compaction, this is what I understand so far

so linux has this abstraction known as a page block

there isn't an explicit page block struct, but each page block is a fixed size that typically corresponds to the size of the architecture's hugepages

basically, each Nth page, where N is the number of pages in a hugepage, acts as the pageblock for itself and the next N pages

so for example page 0 is the pageblock for 0-511, page 512 is the pageblock for 512-1023, etc.

then each pageblock is given a set of flags, such as the migration type, which is basically derived from the types of allocations that went into the pages in the pageblock

then there is also a skip flag to skip compaction after a pageblock is checked

but the idea is, compaction uses a pair of scanners, where there is a free scanner (goes forward looking for pageblocks that can receive pages), and a migration scanner (goes backwards looking for pageblocks with pages that can be safely migrated out)

then when both scanners line up on two compatible pageblocks, they attempt to migrate everything in the migration pageblock to the free pageblock

a pageblock would only be considered for migration if it's marked as migrate_movable, not skipped, and it has no pinned pages

something like that

☝️ bro thinks he can do paging

more hobby kernels should implement page migration and compaction for the Full Buddy Allocator Experience

I think for page compaction I can just have an "aggressive" compactor and a "lazy" compactor

the aggressive compactor runs on any failed allocation, where it will go through all the pages on a given domain and try and compact them and attempt the allocation again

then the lazy compactor would run as a background thread (creating this thread priority has unironically been incredibly helpful), that only runs once everything else has finished running

its job is just to passively wake up every now and then, check for anything to be compacted, compact until it reaches some maximum compaction limit, and then go nappy time again

linux basically does that but with their own doohickies

https://unix.stackexchange.com/questions/679706/kcompacd0-using-100-cpu-with-vmware-workstation-16 lol

ok I think I got that half of page migration figured out, the other half would be NUMA migration to optimize locality, but I think that is a bigger and harder thing to do so page compaction will come first when I build ye old pager and implement migration

[eager yew]

@chilly solar how did you scroll that far back in #schedulers and react to my message

kinda crazy

oops meant to ping in lounge

[chilly solar]

i just looked at images lmao

[eager yew]

understandable

discord mobile making notifications for reacts is so silly

[eager yew]

the next day or so will be the Great Fun and Pleasure of reorganizing things and making private internal.h headers for each subsystem so that the stuff in include/ stays clean

🤪

try not to turn a source tree for a codebase that uses C into spaghetti challenge (impossible)

[eager yew]

static bool enqueue_task(struct event_pool *pool, dpc_t func, void *arg,
                         void *arg2) {
    uint64_t pos;
    struct slot *s;

    while (1) {

        pos = atomic_load_explicit(&pool->head, memory_order_relaxed);
        s = &pool->tasks[pos % EVENT_POOL_CAPACITY];
        uint64_t seq = atomic_load_explicit(&s->seq, memory_order_acquire);
        int64_t diff = (int64_t) seq - (int64_t) pos;

        if (diff == 0) {
            if (atomic_compare_exchange_weak_explicit(
                    &pool->head, &pos, pos + 1, memory_order_acq_rel,
                    memory_order_relaxed)) {

                s->task = (struct worker_task) {
                    .func = func, .arg = arg, .arg2 = arg2};

                atomic_store_explicit(&s->seq, pos + 1, memory_order_release);
                condvar_signal(&pool->queue_cv);
                try_spawn_worker(pool);
                return true;
            }
            continue;
        } else if (diff < 0) {
            k_panic("Event pool overflow\n");
            return false;
        }
        cpu_relax();
    }
}```
least ugly c atomics

[round geyser]

You could probably make them cleaner yourself

Instead of using builtins

[eager yew]

whar

no I prefer this because stdatomic macros are the standard and I don't want to obfuscate by writing my own wrappers around builtin atomics

it's just the way it is

[round geyser]

Who cares about following the standard

[eager yew]

it's for readability

[round geyser]

I mean yeah it’s valid

[eager yew]

if I write my own funky things it won't be standard and other people and possibly myself in the future will have no clue what's happening

[round geyser]

I just think the c syntax is questionable

[round geyser]

Fair point

[round geyser]

I get this premise but personally if I don’t like the standard I’ll change it to something I like even if it’s unconventional

Like for example

I hate the terms acquire/release for locks

So my spinlocks are like spinlock() and spinlock_unlock

Which many would consider “not standard”

[eager yew]

huh

many people use those terms though

the trouble is when you change them to be unclear

atomic_compare_exchange_weak_explicit(&pool->head, &pos, pos + 1, memory_order_acq_rel, memory_order_relaxed) like if I were to change this to atomic_cmpxchg_weak_inc_var_acq_rel_relaxed(&pool->head, pos) or some other weirdness it would obfuscate it more

sure the former is quite ugly but it's clear and standard and you can't really do much to make it less ugly

it's just the way things are 💥

[round geyser]

Yeah true

[round geyser]

It’s like ANSI C… kinda cancer sometimes but worth following

[eager yew]

do you enjoy C atomics

I think they are okay

[round geyser]

I mean, it’s the best we’ve got

[eager yew]

it was a bit funky the first few days but I am understanding it a bit better

[round geyser]

Yeah I made some multithreaded programs for userspace and learned them easily through there

[eager yew]

static inline void spin_lock_raw(struct spinlock *lock) {
    bool expected;
    do {
        expected = 0;
        while (atomic_load_explicit(&lock->state, memory_order_relaxed) != 0)
            cpu_relax();
    } while (!atomic_compare_exchange_weak_explicit(&lock->state, &expected, 1,
                                                    memory_order_acquire,
                                                    memory_order_relaxed));
}``` also I don't understand why so many people use `atomic_flag_test_and_set` for their spinlocks, like sure it is easier to write, but that operation is always RMW and you get some cache thrashing on highly contended locks

it's like 2% more effort to write this

idk gng

[round geyser]

I don’t use flag test and set that’s cringe

[eager yew]

this one only CAS's when the lock is free because the busy wait loop just does a relaxed load

[eager yew]

have you implemented a lockless tree

[round geyser]

No, too retarded

[eager yew]

i might get bored one of these days and do that and I'll bet it'll be significantly slower

[round geyser]

I do this

static inline bool spinlock_trylock(spinlock_t* lock) {
    unsigned int expected = 0;
    return __atomic_compare_exchange_n(
        &lock->state, 
        &expected, 
        1,
        false,
        __ATOMIC_ACQUIRE, 
        __ATOMIC_RELAXED
    );
}

static inline bool spinlock(spinlock_t* lock) {
    bool interrupts_enabled = IA32_INT_ENABLED();
    __asm__ volatile ("cli");
    while (!spinlock_trylock(lock)) {
        IA32_CPU_RELAX();
    }
    return interrupts_enabled;
}

[eager yew]

ok so that's basically the same thing as the atomic flag test and set

you CAS on each iteration

even if the lock is held

so cache thrashing is a bit problematic sometimes

[round geyser]

Ohhhh

Is this like viable

Or should I probably add that to my todo list

[eager yew]

it works but it'll be a little suboptimal when many threads spin because of all the cache line bouncing and bus locking on CAS

relaxed loads don't have that issue since they just load in the shared state so there's no cache invalidation yap

[round geyser]

Should I do something like this instead:

void spinlock_raw(spinlock_t *lock) {
    while(true) {
        if(spinlock_try_acquire(lock)) return;
        while(__atomic_load_n(lock, __ATOMIC_RELAXED)) {
            IA32_CPU_RELAX();
        }
    }
}

And then probably turn off preemption

[eager yew]

that'll be fine

[eager yew]

once you acquire

[round geyser]

Yeah after

[eager yew]

the thing with a constant CAS loop is it requires the cores to broadcast cache invalidations because of the cache coherence protocol so you get bus traffic and cache thrashing

so that's why I went for this one

[eager yew]

static inline enum irql spin_lock(struct spinlock *lock) {
    if (global.current_bootstage >= BOOTSTAGE_MID_MP && in_interrupt())
        k_panic("Attempted to take non-ISR safe spinlock from an ISR!\n");

    spin_lock_raw(lock);
    return irql_raise(IRQL_DISPATCH_LEVEL);
}``` kablooey

panic thing is just for debugging since this one just masks preemption and doesn't do interrupt disabling

static inline enum irql spin_lock_irq_disable(struct spinlock *lock) {
    spin_lock_raw(lock);
    return irql_raise(IRQL_HIGH_LEVEL);
}``` other one

[stuck nexus]

i didnt read the whole convo so i apologize if this was intended as a bad example

but thats brokn

[eager yew]

how come

[stuck nexus]

there is a window of time between acquiring the spinlock and raising irql where a dpc can come in and try to spin on the spinlock

and then you have self-deadlock

you have to raise irql before acquiring the spinlock

[round geyser]

Good to know actually

[stuck nexus]

something is possible though where

you raise irql

try to acquire the spinlock once

if its already held, you can lower irql and spin on it (without acquiring it!) until it is freed

then you raise irql again before trying to acquire it

thats possible

then youre preemptible while spinning for a spinlock

idk how useful that is but you can do it

only works for the outermost spinlock as well

if youre already holding a spinlock and try to acquire another one you cant lower irql while spinning obviously

[round geyser]

How do you prevent this case

[eager yew]

my DPCs don't function like that

[stuck nexus]

wdym

[stuck nexus]

i amended that message

[round geyser]

Ohhh

[eager yew]

maybe I am calling my DPCs the wrong thing but they're just functions in a list that a few worker threads per core attend to

what do you mean by dpc here

[stuck nexus]

what is DISPATCH_LEVEL blocking off there

and what is HIGH_LEVEL blocking off

[eager yew]

it blocks off APCs and turns off preemption and high level disables interrupts

[twin escarp]

wut's an APC and DPC 🙃

[stuck nexus]

i have no idea how any of what youre saying is meant to invalidate my concern then

the same problem exists

[round geyser]

Deferred procedure call and asynchronous procedure call

[twin escarp]

ahh

[eager yew]

but how so? the way my workqueues work is that a thread currently executing work won't be in the threadqueue of available workers, so it will either spawn another thread or just wait for the current one to finish

so a dpc couldn't possibly come in at that moment there

I'm a little confused

[stuck nexus]

could you not get preempted by such a worker thread

even that case aside, this is still broken for ISRs

for the same reason

[eager yew]

ohhh if that's what you meant then yea ur right

I just thought that spinning first and then raising later might make some of my bad locks less bad because threads can get preempted as they spin

but avoiding this problem is a lot more important

[eager yew]

ugh why is my binary always above 400kb in size... oh well

lto inlining is actually mad

it's so ridiculous how much it can fatten the binary and make things disappear

I remember a little while ago I wrote some userspace stuff and lto inlined everything and I just had "one huge main function" in the binary

[round geyser]

Mine is 900kb

[placid vessel]

mine is 60k

but i have no code

[vagrant cedar]

Mine is nearly 800KiB (and that's stripped from debuginfo!)

[eager yew]

thanks guys I feel a little less bad about myself

binary size positivity

[round geyser]

Binary size matters

[eager yew]

ok I thinks my workqueues need some work, I might need some structured way to also represent kernel daemons since page compaction might benefit from that

[eager yew]

womp womp

[steel herald]

2.1.1 uACPI

[eager yew]

oh I think I am on very old version

[submodule "kernel/uACPI"]
        path = kernel/uACPI
        url = https://github.com/uACPI/uACPI
[submodule "kernel/flanterm"]
        path = kernel/flanterm
        url = https://codeberg.org/mintsuki/flanterm
[submodule "limine"]
        path = limine
        url = https://github.com/limine-bootloader/limine.git
        branch = v9.x-binary``` idk

I'll double check things

[steel herald]

The submodules file doesnt have a rev reference

[eager yew]

yes

kablooey

idk

it works fine on linux but on macos it does that

this isnt the allocator bug that happened last time

womp womp ig

boom
>> UACPI ERROR: aborting table load due to previous error: unimplemented```

looks like something hits that statement

hmmmm time to debog

[steel herald]

Enable uACPI debug logs

But this is probably memory corruption

Since it tries to assign to some bogus type

[eager yew]

>> UACPI LOG: starting uACPI, version 3.1.0

>> UACPI LOG: RSDP 0x00000000000F5250 00000014 v00 (BOCHS )

>> UACPI LOG: RSDT 0x000000007FFE2813 00000040 v01 (BOCHS  BXPC    )

>> UACPI LOG: DSDT 0x000000007FFE0040 000023B7 v01 (BOCHS  BXPC    )

>> UACPI LOG: FACP 0x000000007FFE23F7 000000F4 v03 (BOCHS  BXPC    )

>> UACPI LOG: APIC 0x000000007FFE24EB 000000B0 v03 (BOCHS  BXPC    )

>> UACPI LOG: HPET 0x000000007FFE259B 00000038 v01 (BOCHS  BXPC    )

>> UACPI LOG: SRAT 0x000000007FFE25D3 000001A0 v01 (BOCHS  BXPC    )

>> UACPI LOG: SLIT 0x000000007FFE2773 0000003C v01 (BOCHS  BXPC    )

>> UACPI LOG: MCFG 0x000000007FFE27AF 0000003C v01 (BOCHS  BXPC    )

>> UACPI LOG: WAET 0x000000007FFE27EB 00000028 v01 (BOCHS  BXPC    )

>> UACPI LOG: FACS 0x000000007FFE0000 00000040

problem
>> UACPI LOG: aborting table load due to previous error: unimplemented```

doesnt really say anything else

hmmmm

[steel herald]

U sure u enabled debug logs

[eager yew]

oh wait that's with the compiler flag right

[steel herald]

Nope

uacpi_context_set_log_level

[eager yew]

oh right yes

wow lots of prints

[steel herald]

Probably need the last few lines

[eager yew]

just piped it to a file

[steel herald]

Cant really check on the phone

[eager yew]

>> UACPI LOG: BEGIN OP 'ZeroOp' (0x0000)

>> UACPI LOG:     pOP: LOAD_INLINE_IMM_AS_OBJECT (0x1C)

>> UACPI LOG:     pOP: OBJECT_TRANSFER_TO_PREV (0x26)

>> UACPI LOG:     pOP: <END-OF-OP> (0x00)

>> UACPI LOG: END OP 'ZeroOp' (0x0000)

>> UACPI LOG: RESUME OP 'PackageOp' (0x0012)

>> UACPI LOG:     pOP: JMP (0x3C)

>> UACPI LOG:     pOP: IF_HAS_DATA (0x32)

>> UACPI LOG:     pOP: OBJECT_ALLOC_TYPED (0x1A)

>> UACPI LOG:     pOP: INVOKE_HANDLER (0x15)

>> UACPI LOG:     pOP: OBJECT_TRANSFER_TO_PREV (0x26)

>> UACPI LOG:     pOP: <END-OF-OP> (0x00)

>> UACPI LOG: END OP 'PackageOp' (0x0012)

>> UACPI LOG: RESUME OP 'PackageOp' (0x0012)

>> UACPI LOG:     pOP: JMP (0x3C)

>> UACPI LOG:     pOP: IF_HAS_DATA (0x32)

>> UACPI LOG:     pOP: OBJECT_ALLOC_TYPED (0x1A)

>> UACPI LOG:     pOP: INVOKE_HANDLER (0x15)

>> UACPI LOG: aborting table load due to previous error: unimplemented

[steel herald]

Yeah basically just the type field got overwritten with a bogus value

[eager yew]

ohh ok

[steel herald]

U can check it out in gdb

Just print out the object struct there

[eager yew]

type addr is 0xfffff0000001fbd4 ok found it i'll just watchpoint this then ig

[steel herald]

What's the value?

[eager yew]

type addr is 0xfffff0000001fbd4, val is 0x4 4

the watchpoint made it just get stuck and i had to send sigkill to qemu

rip

[steel herald]

4 sounds valid

Did u inspect the right one?

[eager yew]

this is the print right before it returns that unimplemented error

if (uacpi_unlikely_error(ret)) {
        k_printf("type addr is 0x%lx, val is 0x%x\n", &package->objects[i]->type, package->objects[i]->type);
        return ret;
}```

in static uacpi_status handle_package(struct execution_context *ctx)

kinda whimsical how this only occurs on macos now

same version (of everything) on arch lenox does not have this predicament

[steel herald]

Yeah dunno

I dont think there's anything special about Mac

[eager yew]

https://tenor.com/view/clash-royale-king-he-he-haw-sad-boi-sphere-gif-24045980

it's ok I can just work in the UTM ubuntu vm on my mac

things are sane over there

how peculiar

[late pecan]

wow

[late pecan]

hey gummi, really random question but did you make an isa before making an os

i want to make an isa too, but i decided to do osdev and i cant just stop halfway through

anyways bye

[eager yew]

a horrible one

don't look at that one

[late pecan]

ok

its pretty cool to me though

[gloomy slate]

@eager yew here

I guess we both forgot what it was called

[eager yew]

thanks gang 🙏

hotplug

[eager yew]

a lot of stuff happened in the many month hiatus lol i just got lazy (stopped yappin here) and school came back

[eager yew]

full time job

[eager yew]

working on stress testing hotplug to make it safe and make it so that wonky controllers that do spurious (dis)connects wont bring this all crashing down

import socket, json, time, random

def send(cmd):
    s.sendall((json.dumps(cmd) + "\n").encode())

s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.connect("/tmp/qmp.sock")

send({"execute": "qmp_capabilities"})

for i in range(10000):
    send({
        "execute": "device_add",
        "arguments": {
            "driver": "usb-kbd",
            "id": "kbd0",
            "bus": "xhci.0"
        }
    })
    time.sleep(random.uniform(0.001, 0.1))

    send({
        "execute": "device_del",
        "arguments": {
            "id": "kbd0"
        }
    })
    time.sleep(random.uniform(0.001, 0.1))
``` very stress

[eager yew]

[USBKB]: Keyboard disconnected 
usb last ref gone, freeing 
tearing down xhci slot 0xfffff000003ff2c0 
connect received 
port_connect work enqueued from isr 
port_connect work running 
Port 8 init 
Connecting 
disconnect received 
port_disconnect work 0 enqueued from isr 
connect received 
port_connect work enqueued from isr 
straggler found 
USB error in setup 
port_disconnect work running 
Disconnecting 
tearing down xhci slot 0xfffff000003ff2c0 
usb last ref gone, freeing 
disconnect received 
port_disconnect work 0 enqueued from isr 
connect received
[USBKB]: Keyboard connected 

insane log

average hotplug experience

i might go even more bonkers and start handling wild things like the controller randomly resetting

actually nah i might as well just work on the linux xhci driver if i wanna do that

i think this ok

[placid vessel]

i hate ts font

[last lichen]

ts =/= this

ts === this shit

[placid vessel]

i hate this shit font

[twin eagle]

ts fr pmo sm ngl

[last lichen]

ts 🥀

[eager yew]

i just ragebaited myself

i thought i somehow made my memory allocator slower by 3 orders of magnitude and i was spending so long hunting for where i made changes that could've slowed it down

turns out

i had optimizations off

The Fool spends hours micro optimizing hot paths, removing branches and improving cache locality. The Genius uses bogosort on a 10 item list and says to themselves "the compiler can optimize this anyways"

https://tenor.com/view/jgmm-cat-meme-ragebait-cat-think-cat-ponder-gif-7034802297606826390

[eager yew]

what font is it

im rocking berkeley mono nerd font (patched by meself) in the terminal

[placid vessel]

looks like Rubik

my terminal has Cascadia Code

[eager yew]

bro opens his suse

open the suse

let that suse in

[twin eagle]

https://tenor.com/view/riddler-paul-dano-new-rockstars-does-he-know-gif-26833694

Why does it have that second part

That was not supposed to happen

[lime wharf]

bro eats c for breakfast 😭 🙏

[eager yew]

okay i am content with the state of usb hotplug support now am gonna start making The Pager

those who swap

😂 ✌️

after this we do filesystem stuff and then networking 🥶

The Callisto Protocol

[lime wharf]

ram is just L4 cache
persistent storage is the only real thing

https://tenor.com/view/cat-meme-cat-silly-cat-kitty-cat-memes-gif-16920741247574028693

[eager yew]

i just realized i completely forgot about heterogeneous processor topology when i was making my in software representation of cpu topology

good thing i remembered that before i decide to go and rework thread migration

anyway time to go do that

are you a big or a LITTLE

[eager yew]

bruh qemu doesnt let me fake a heterogeneous processor i have to own one as the host machine and passthrough

this is widdawy heterophobic ‼️

scheduling has been a never ending plumbing job

every time i think i've done all i need i realize i forgot something or want to change the policy and then i go back

more recently scheduling has been evolving into two problems, the scheduling algorithm itself and the load balancing policy

i have the first one mostly figured out but the second one has a very long ways to go

oh brother

[eager yew]

the complexity increase of the latter is mostly due to my Arguably Bad Idea of supporting topology aware scheduling

i think i can give each logical processor a performance score and give each thread a "wanted performance" and migrate to a more performant processor if the one it is currently running on does not satisfy the wanted performance

i can derive the wanted performance from the thread's interactivity and other stats that i keep track of

basically i keep track of "What was this doing in the last 4 seconds?" for each thread and that contains run%, sleep%, etc. and i can say things like "if a thread is running for all of its time, then i will try to move it to a more performant CPU whereas sleepier goobers can go on efficiency cores"

something like that idk

just sketchin

oh boy energy saving will also make this Quite Fun

typically on homogeneous CPUs if you want to save power you can just say "on a machine with K processors, we can just force J of them to always be idle where J < K, J ∈ N, K > 1, K ∈ N" and be mostly ok but on heterogeneous processors you'd have to do a bit more work in figuring out which of your K processors you want to pick to be idle and tune it to your workload

[lime wharf]

thank u Obama 💔

[lime wharf]

I first thought this would be tuf but then the commit seems pretty straightforward :o

[eager yew]

hey guys i havent been posting here but i have been working on stuff, but i've also been doing some branch predictor experiments on my intell broadwell-U CPU to try and see if i can pick up on some patterns in its BPU since it's an older chip and may have more observable patterns

right now through testing i observe a VERY consistent pattern that goes as follows:
if i map a hugepage, fill it with blocks of "always taken jz to the next instruction", and space these blocks 16 bytes apart with a ret at the end of all the blocks, i seem to consistently get this result:

after ~200 blocks, there is a significant misprediction spike (e.g. if the most mispredictions on a previous run was ~200 – i do about 100,000 runs for each added block to the test code with 100,000 warm-up runs each time – the spike is around 10,000 mispredictions), and this spike stays there until i get to about the ~260th run, where it goes back down and the mispredictions stop

and this is reproducible every single time on this chip

very interesting stuff

anyways verabschiedung pals i am going back to the shadow realm

[eager yew]

and all this is being done from a pinned realtime thread and i'm using a hugepage since they are less likely to get evicted from the tlb

it seems like after that spike the misprediction count goes to zero

kinda wild

[molten night]

Could be some huristic, I don't know too much about branch predictions in modern designs, but I've heard there's a lot of huristics byond the base algorithm

Anyway cool stuff

[eager yew]

i think what's happening is multiple layers of predictors for different lengths

probably some kind of simpler TAGE

which is likely why they start all predicting correctly later on

maybe i will try adding some noise so they are sometimes taken sometimes not

wait it might be a loop stream detector

maybe the BP isnt doing anything at all

0 is a bit sus

yep it was probably the LSD

out of context messaegs

anyways

yeah i now flip eax and the BP doesnt breakdown and then have mispredicts drop to zero

[eager yew]

call me john graph the way that i be analyzing 🗣️

this is across 100 runs

the spikes are making this scale way off icl

cant see anything

those who graph

ok this is just 10 runs so trends may not be visible

there is a very clear starting point though

[eager yew]

ok first sane graph

anotha one

[eager yew]

i have observed some patterns

Dominant FFT Frequencies
Freq=0.2505 1/block, Period ≈ 4.0 blocks, Amplitude=3.8522
Freq=0.0020 1/block, Period ≈ 511.0 blocks, Amplitude=3.5465
Freq=0.2485 1/block, Period ≈ 4.0 blocks, Amplitude=2.3695
Freq=0.0039 1/block, Period ≈ 255.5 blocks, Amplitude=2.1668
Freq=0.1566 1/block, Period ≈ 6.4 blocks, Amplitude=1.7089

Autocorrelation Peaks (first 20 lags)
Lag 16: autocorr=0.2793
Lag 8: autocorr=0.2551
Lag 12: autocorr=0.2413
Lag 1: autocorr=0.1866
Lag 4: autocorr=0.1866```

very fascinating

a misprediction map

i wonder if i can make this a bit better and try and forecast where mispredicts will happen

that would be fun

Dominant FFT Frequencies (binary spikes)
Freq=0.0020 1/block, Period ≈ 511.0 blocks, Amplitude=16.0486
Freq=0.3112 1/block, Period ≈ 3.2 blocks, Amplitude=14.9531
Freq=0.3209 1/block, Period ≈ 3.1 blocks, Amplitude=13.4660
Freq=0.2505 1/block, Period ≈ 4.0 blocks, Amplitude=12.2449
Freq=0.4834 1/block, Period ≈ 2.1 blocks, Amplitude=11.4239

Dominant FFT Frequencies (weighted spikes)
Freq=0.0020 1/block, Period ≈ 511.0 blocks, Amplitude=2.1492
Freq=0.3112 1/block, Period ≈ 3.2 blocks, Amplitude=1.8623
Freq=0.3209 1/block, Period ≈ 3.1 blocks, Amplitude=1.6875
Freq=0.2505 1/block, Period ≈ 4.0 blocks, Amplitude=1.5414
Freq=0.2074 1/block, Period ≈ 4.8 blocks, Amplitude=1.5254

Autocorrelation Peaks (first 20 lags)
Lag 16: autocorr=0.2793
Lag 8: autocorr=0.2551
Lag 12: autocorr=0.2413
Lag 1: autocorr=0.1866
Lag 4: autocorr=0.1866``` biased things and changed stuff around a bit to focus on mispredict spikes

this doesnt seem to give a ton of info though

ok time to sleep this is not how i wanted to spend the first hours of 2026

gute nacht

actually that fft is entirely useless I forgot what I was measuring

it might be tricky to pinpoint exact misprediction locations

though actually, it might not be entirely useless

the graph basically shows misprediction frequencies when X conditional branches are inserted into a page but not misprediction locations

the plan after this is to target the spikes to figure out exactly where mispredicts are happening

mostly for time savings sake

[eager yew]

I just remembered that I might need some way for block devices to represent what NUMA node they are so that when I do things like the page cache I don't go and get pages from node 1 for a node 0 block device or something silly

just putting this here if I do a big dumb and forget

and similarly I might want to pin readahead and writeback workers to the bdev node

I wonder how cross node zero copy dma would work

seems like a funny predicament

[eager yew]

int node;

[eager yew]

I guess this wouldn't be hard at all to implement, I'll just need a few wrapper APIs to do allocations from nodes with the ability to fall back to other nodes if memory pressure is too bad, but that's kinda trivial and would realistically just mean that a few functions here and there would need extra parameters for the node rather that assuming it should use the caller thread's current node

[eager yew]

maybe they could do something like see if copying is genuinely faster than taking the locality hit and just do that instead

one copy 💔

ah wait but also nodes aren't block device specific meaning that DMA for everyone else should use a specialized DMA allocator that takes in a struct generic_device* and each generic_device has a node

I have previously been able to get away just fine without a dma allocator by literally malloc'ing dma memory because I'm on x86 where everything is coherent and I decided to not support non 64 bit devices at the start out of laziness lol

but I suppose I could do this too

anyways the plan now would become

• add numa locality to devices
• page allocator (I just have a slab allocator and buddy allocator and it would be nice to have that middle layer that automatically calls into the pager, mostly as a nice wrapper)
• dma allocator
• pager
• page cache
• head back into filesystem world

for now

and maybe I'll do a bit of scheduling work for better load balancing (heterogenous CPU) if I get truly bored

[eager yew]

will probably want iommu stuff too

[eager yew]

oh yeah and on the topic of this I found out that I could load all of my machine's acpi tables in qemu while avoiding the "no one table can be bigger than 16kb" by simply loading each table individually and that worked pretty well for trying to get c state stuff to work (doesn't work yet though but uacpi makes it show up)

while I'm yapping I wonder if I could abstract all of the "how local should NUMA stuff try to be" into a big centralized config option with optional per subsystem locality configuration

CONFIG_NUMA_LOCALITY 0-255 lol

and maybe CONFIG_PAGE_CACHE_NUMA_LOCALITY that can be defined and overwrite the global one

and other stuff

where 255 basically means "literally just fail and report OOM if you can't get it from ur node*

when your page cache ooms because its node ran dry 💔

enomem but you have mem 😨

[eager yew]

actually this is dumb I'll do runtime objects

as for when I eventually expose this stuff, likely via filesystem, I'll probably just do
/sys/mm/page_cache/numa_locality for the organization

and similar

all of this work for a little joke I wanted to do a long time ago that has lost its punchline

eh whatever reasoning about numa stuff has been quite fun actually 😍

for topology stuff I'll do

/sys/topology
  /cpu
    cpu0/
    cpu1/
  /core
    core0/
    core1/
  /domain
    domain0/
    domain1/
  /node
    node0/
    node1/
  /socket
    socket0/

light work no reaction

I think I will not support CPU hotplug though unless I start getting paid to do this because that's just masochism

@lime wharf do you have any ideas or suggestions

[lime wharf]

nono im just enjoying the yap 😭
im too dumb for this
for now

[eager yew]

I think one thing you could take time to think about is my first class abstraction of a "domain" which is a 1:1 mapping of NUMA nodes on NUMA systems or a group of a configurable number of logical processors on UMA

maybe try to find flaws in this design or something

I mostly have this for scheduling but also for reducing lock contention while maintaining balance in some subsystems

but you can have a thinky think

it might come in as a nice way to be lazy later on because I can tune topology stuff for groups of adjacent logical processors by just modifying their per domain variables (which get inherited to all LPs in the domain),

these domains would be independent from NUMA data in that they don't serve to hold the same data that NUMA nodes do (e.g. distance matrix) but don't harm NUMA locality

[lime wharf]

methinks (after tmrw's exam 🥀 )

[eager yew]

ok i realized that heterogenenity might not play super nicely with domains, but this is easily adjustable by adding a policy of "a domain's logical processors are all homogenous" at init time

also it might not be all that relevant anyways

i just thought it might be an oddity later on if i have domain with 3 P core LPs and 1 E core LP or something

[lime wharf]

I thought this would be okay?
cuz like
they are still differentiated by capabilities and performance rating stuff u have

[eager yew]

well from userland it might be wonky

because if you have a domain that's intraheterogenous then you can't exactly modify CPUs in groups in a way that isolates heterogenous CPUs into homogeneous groups

[lime wharf]

mmm
understandable

but wait wouldnt it be useful if domains could still be there as a 1:1 mapping to nodes and then in a heterogeneous node they could just refer to their cpus by their class
but then idk how would that be different from the node abstraction so nvm
i might be just hallucinating as well

[eager yew]

then i will prefer numa nodes

the splitting based on heterogeneity will only be for uma

[lime wharf]

oo
numa is scary 😨
also its hard to not be hetrophobic these days smh
with numa even harder

not shrimple at all

[eager yew]

yes i am still alive

i did realiz that rwlock priority inheritance might be a bit of a pickle

erm

like if i have a timesharing reader on the lock and a realtime writer wants to acquire the lock and there is another realtime thread, the reader never gets boosted to realtime and things just kinda get stuck

[lime wharf]

no way

[eager yew]

inundated with work and chasing butterflies with side projects 🥶

i'm mounting lights and screens to my backpack for synesthesia audiovisualization

[lime wharf]

🤑

those who fft

I wanna seee
plzplzplz

[stuck nexus]

shared locks are pickly for priority inheritance generally

usually there is only minimal effort into supporting PI across shared holders

maybe only one will be boosted for example

or none!

and its not airtight

so usually they just avoid shared locks where it matters

if it matters and a shared lock is too good to pass up, they will do priority ceiling protocol for that one lock rather than priority inheritance

that is, before trying to acquire it, they will have the thread raise its own priority to some maximum

[eager yew]

hello pals i am still alive and i'm making a dress right now so i wont have too much time for this side quest

the computer scientist to seamsperson pipeline is real

[lime wharf]

https://tenor.com/view/nub-nub-cat-silly-cat-cat-silly-gif-6876737291168577775

[eager yew]

sorry guys i got caught up reading immanuel kant's works, i have made major revisions to the APC model in my kernel recently, but more work will happen. also considering doing ARC instead of LRU

changes to apc model:
introduced "Boundary APCs" that execute at kernel -> userspace boundary (i think mintia has something similar)
"on-event APCs" -> you can create kernel event sources and enqueue these APCs that will execute one time every time that event is signaled for that thread (e.g. THREAD_EXIT is an event, and you can hook into it such that, say, a thread with a memory arena can create an APC that will run when the thread exits to clean up the arena)

[eager yew]

autoboost has inspired me

i've recently come to the realization that my timesharing thread PI is overly naive and can be more harmful than helpful

my big reason for PI is for cross priority class PI, such as a RT thread blocking on a TS thread and boosting it to RT so it can make progress (since priority classes literally do make things get stuck)

however, my inter-TS PI is just "copy the blocking thread's weight if it is higher than the owner"

this can end up in situations where threads get massive leaps in priority when they really don't need to, since my timesharing thread scheduler uses periods to guarantee that no one starves (i.e. it sets up a window of time in which everything runs at least once)

so i was thinking of introducing a "MicroBoost" framework for inter-TS priority "adjustments", as a means of facilitating progress more than rigorously policing fairness

this way, rather than a TS lock owner getting a big huge leap in priority it only gets boosts if it is seen that it isn't progressing very well

not sure though

I'll probably keep looking into autoboost

but i am fairly certain i'll want something beyond "Copy weights and call it a day" for inter-TS "priority adjustments" (PI isn't strictly necessary for inter-TS priority inversion since progress will still happen because of how the scheduler works)

[lime wharf]

charmOS is having regular commits again we are so back :O

[eager yew]

added io_wait_token structure to tie priority boosts to an object lifetime

i realized a lot of my IO code looks like this

block();
/* process things... */
unboost();``` because the waking side boosts the thread to the URGENT priority so that it can make progress upon the wake from IO

I thought this was quite silly so now I have a structure

struct io_wait_token {
    struct list_head list;
    struct thread *owner;
    void *wait_object;
    bool active;
    size_t magic; /* this is just taken from a global counter incremented at each instantiation of one of these structures */
};```

and i instead go
```c
io_wait_begin(tok, dev_ptr);
yield();
/* after waking, do whatever needs to be done */
io_wait_end(tok, IO_WAIT_END_YIELD);

and i track these on per-thread lists for debugging so that i can identify what is causing a thread to be boosted to URGENT

💥

idk tho

i think this is alright

[eager yew]

i keep forgetting to post things here

news:

• new priority boosting framework inspired by autoboost (CLIMB - Centralized Lock and IO Modular Boosting)
• logging rework with struct log_site and struct log_handle
• panic/assert/debug rework (soon)
• debug shell work (soon)
• plans to introduce swappable realtime thread schedulers and policies
• schedulers decide "Out of all the RT threads which should be run", policies decide "Should we run RT or TS if RT is available"
• RT schedulers (planned): EDF, FIFO, RR
• RT policies: True Realtime (as the name sounds), Soft Realtime (closer to linux, timesharing occasionally runs), Not Realtime (closer to windows, RT is effectively "unbudgeted TS" and is more of a very high prio TS thread)

https://github.com/users/BlueGummi/projects/1 this is where the to do list is

https://tenor.com/view/cat-meow-360-cat-meme-gif-11279902563780569961

[severe abyss]

did you manage to figure out how it works

[eager yew]

a bit

not fully

[severe abyss]

nice, itd be interesting to know your opinion

[eager yew]

👍 ya once i get a more coherent explanation for it formed i might write about it

[severe abyss]

so far i see you liked it since you plan something similar in your kernel

[eager yew]

XNU having ticket locks is quite the curiosity

maybe since XNU is primarily a desktop OS it is less bad

[eager yew]

implemented first iteration of priority boosting framework

it passes correctness tests (i.e. nothing synchronization related is blasphemously wrong) but i'll have to see how it holds up fairness wise

other people out there porting minecraft and doing all sorts of fun stuff and i'm here obsessing over autoboost and trying to do something a little similar

[eager yew]

man i've been wasting a lot of time on scheduling and still have a ways to go

the scheduling/synchronization to-do is looking something like:
Load balancing jokes:

• better NUMA load balancing (I have this for work-pushing to idle CPUs but not the other way around)
• better use of scheduler domains for load balancing (aggregation of load and shenanigans)
• better work pushing
• better use of per-period work items for load balancing (I can scan within the node/subsection of the node on every tick, maybe something like 4 CPUs)
• heterogeneous CPU detection and load balancing (I have a whole plan for this)
  Other humorous things:
• hot-swappable realtime schedulers and realtime policies (I also have a whole plan for this)
• maybe implement ticket locks
• much better lock instrumentation (tracking latencies)
• perhaps something like sched_ext in the long run
• idle thread state machine to allow it to do some work (idk, maybe)
• RCU needs big improvements (it is downright as bad as it gets, i need to first scale it up a little bit and then up to tree RCU)

this is far more planned out than any other subsystem's to-do list because of the immense amount of time i've spent thinking about this

i seem to be having a blast with scheduling and synchronization so i might genuinely end up wasting an unreasonable amount of time in this world

after these shenanigans though it'll be time to (ONCE AGAIN) re-enter the sad gloomy realm of filesystems

this time around I'll first implement a pager and rework parts of MM (multi-page slabs + the problems that brings), and likely also implement the page cache with a split cache model and build a DMA allocator wrapper over the page allocator (which will sit in between slab <-> buddy, i dont have that yet for some reason) before heading on into the filesystem world and rebuilding all of that (I have some fs code but large parts will need to get swapped out)

my plan afterwards is likely to then maybe flesh out the IOMMU interfaces (not too hard) and do some general driver QOL and touch up on some drivers, maybe then implement EHCI and an audio driver (for jokes) and hook them up to the filesystem layer (when it is there)

and as for swapping/page eviction i was thinking of devising a generic interface to allow myself to implement a variety of algorithms and select one at boot (NRU, LRU, ARC, etc.)

i'll probably go with LRU at first and then work my way up to ARC for fun

eventually it'll come full circle as i'll probably take an approach similar to sysfs in exposing knobs to userspace and with that i'll likely end up sprinkling all my code in wacky wonky sysfs nonsense

and then once all this is done i'll probably pursue the boring things that are completely irrelevant like security, networking, cryptography.....

oh yeah and kernel modules

💥

that is it for tonight though

[placid vessel]

bruh

mintia larp

[eager yew]

mintyb

[eager yew]

i was thinking

about linux sched_domains

and i realize that i can improve on this by modeling a heterogeneous NUMA CPU as a multi-dimensional space and use R* trees to create larger and larger "domains" going outwards

perhaps

and cache data as well

this way rather than hardcoding domain levels (socket, node, smt) i can just iterate up an R* tree that would be created a bit more intelligently than hardcoded levels

something like this

https://tenor.com/view/fire-writing-writing-fire-his-writing-is-fire-is-this-fire-hold-up-gif-15400153686296022809

wait this might actually be interesting to look into

I think it would scale a bit better sched_domain construction wise because I can keep increasing the number of dimensions

e.g. I can do

NUMA node
SMT sibling
Socket
LLC

as separate dimensions, and as I keep adding stuff (DVFS and stuff) I can keep adding dimensions

load balancing transcends the three dimensional universe

wobbly wiggly wobbly wiggly

i might change the way i implement my sched_domain and switch to a multidimensional R* tree

this is an interesting thought

[eager yew]

yes i am alive and yes i am still working on a goofy rabbit hole that no one in their right mind should care this much about

this time it's swappable realtime schedulers

and also reworking some init stuff

deep

when will we get a song called Ke

[eager yew]

wobbly wiggly wobbly wiggly

I was thinking about the split page cache + buffer cache model for my eventual filesystem layer (Soon™) and I think the way I will go about this will be by using a coarse grained bitmap for quick rejections and a radix tree for finer grained searches

for example I could keep a bitmap where each bit represents a 16MB range on disk and if the bit is set, something in that range is metadata

but if the bit is not set, it's all data

this is about 8 kilobytes for 1TB which I think is Fine™

^famous last words

but the idea is that I can use that data structure to quickly go "Nope, not metadata" on a write, or otherwise traverse the radix tree to figure out precisely what inside a write might be in metadata

bitmap can just be the top level of the radix tree probably

[eager yew]

I think I might start looking into supporting CXL tiered memory when I get back around to the pain and suffering of my memory subsystem

in particular I think DMA might benefit from it because things like NICs can write into CXL instead of precious node local memory

this may improve memory latency because the node local controllers won't be fumbling around with DMA and can service memory accesses from the cpu

[eager yew]

??? already

i thought this project was way smaller

[eager yew]

reconsidering the fault tolerant features of my swappable realtime scheduling

i think what i will adopt is a model where threads and realtime schedulers sign themselves up for fault tolerance

basically, the idea is that a realtime scheduler "module" can return an error code (FAIL_ASAP) from any function to tell the core scheduler "problem happened, switch to a known good scheduler" (I'll have a few known good realtime schedulers permanently resident – EDF, RR)

however, realtime schedulers must explicitly declare themselves as allowing faults, and any realtime thread must tolerate faults if it is to be enqueued on such a realtime scheduler

this is because once a fault trips, some threads might become unhoused (e.g. if a thread accepts only FIFO and your FIFO scheduler faults, the thread has nowhere to go), and threads must explicitly be aware of the fact that if their scheduler trips they are willing to say "starving me is fine"

the idea is that userspace can then use faults/failures as a way of debugging workload issues

for example, if an EDF impl. keeps missing deadlines over and over, it can trip FAIL_ASAP. this will then switch the scheduler to a known good scheduler (probably RR) and then move all the threads away

then userspace can inspect the threads themselves and their related data to dig deeper on what exactly happened

and if a scheduler says "I am not fault tolerant" (indicated at load time) and it FAIL_ASAP's I'll just crash the kernel :^)

this is actually kind of a funny hat trick

erlang lore

i feel like after a year or two my kernel will be inundated with weird hat tricks like this that are completely useless to 99% of people and just there because i thought it would be mildly humorous

[eager yew]

> git commit -a
git push[main 8806536] [alloc]: API revamp/improvements
 101 files changed, 564 insertions(+), 650 deletions(-)```

the humble macro spammer

https://media2.giphy.com/media/v1.Y2lkPTZjMDliOTUyZjdldGZyc3FmN2ZxYTc0cDMzanR2ajByc3NjOXdqcHZpNjBza2NxZyZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/OEn9NrFa7E35H7NmI1/giphy.gif

[gloomy slate]

More deletions than insertions 🔥

[eager yew]

that was such a nice thing to see, mostly because stuff like
kmalloc(big sizeof statement, ALLOC_PARAMS_DEFAULT); got shrunk to kmalloc(big sizeof statement); which reduced line wraps

i am so happy to have thought of this moumental abuse of macros

this is so pleasant

most cortisol draining API change to date

[eager yew]

look it's linux reddit

i wasted far too much time doing this shit

never touching frontend again

it's not even fully functional

https://media.tenor.com/5kCM0LbCZn0AAAAe/please-stop-matpat-please.png

lowk might vibe code the rest

[placid vessel]

bro what

https://tenor.com/view/fake-larp-larp-larp-sahur-sahur-larp-larping-gif-18307483096237429166

[eager yew]

I just wanna be lazy 💔

[placid vessel]

me

[eager yew]

gumbo larps as marvo 🥶

[eager yew]

i just had a thought

https://tenor.com/view/urethra-gif-27455842

I think I'll change my async model to create a generic wrapper around async requests that chooses between APC and worker dispatch

for a lot of kernel IO, worker dispatch (i.e. calling an async callback from a thread other than the one that submitted it) tends to be faster (marginally) since it can always run, whereas APCs can get blocked by a thread for some amount of time

but APCs are also a little better sometimes because if a thread is actively running and can run APCs, you can just send it there and make the thread yield and get it to run without loading up a worker thread

i was drafting maybe making something like a

enum completion_mode {
    COMPLETE_WORKER,
    COMPLETE_APC,
};

struct async_completion {
    enum completion_mode mode;

    union {
        struct {
            void (*fn)(void *);
            void *ctx;
        } worker;

        struct {
            struct thread *target;
            struct apc apc;
        } apc;
    };
};```

and then embedding one of these objects inside of things like async requests

idk tho

[eager yew]

working on drafts for better memory allocation (slab) usage once multi page slabs come about

it's like a "variant of LCM" to reduce unused space/internal fragmentation

[eager yew]

I may be getting into erlang soon

🥶

another idle period

distributed systems are too interesting

[twin escarp]

now's my chance to catch up with charmos

[eager yew]

nah i'll be idly working on patches and things, but i'll just be timesharing this work with other stuff

[twin escarp]

[eager yew]

i already have a big major revamp of mm planned (see the python slop above) and that is what is next

[twin escarp]

I'll get my chance eventually

[eager yew]

then i'll circle back around and make scheduler load balancing much better and maybe do some minor cleanup

once all this is out of the way i might then do IOMMU/DMA QOL improvements and then write page cache related stuff and revamp filesystems entirely

filesystems already exist as a small stub and the logic/walking/parsing/creation is there but it is all pretty bad and kind of non-scalable

oh and scale up RCU in the meantime, filesystem layer would love that

[twin escarp]

I barely understood any of that coolio tho

[eager yew]

https://cdn.discordapp.com/attachments/829403502349647952/1481147828649857052/speechmemified_Screenshot_2026-03-10_at_11.gif

[eager yew]

GRAPHSLOP!!!!

https://tenor.com/view/tiktok-meme-i-just-made-some-bullshit-gif-15690632352507439130

[eager yew]

my stress test for the slab allocator takes a few nanoseconds for random pairs of allocation/free under contention which i think is alright

not sure where the bottlenecks are

[inner rover]

How does this differ from a traditional kmem-style slab?

[eager yew]

wdym

the main goal that i want to do a bit better is space efficiency

basically my goal is to find page counts for slabs that minimize how many bytes go unused to reduce internal fragmentation

that's it

all this yap for something you can precompute and hardcode

[eager yew]

https://tenor.com/view/tiktok-meme-i-just-made-some-bullshit-gif-15690632352507439130

[eager yew]

that size_t in the loop should be int i do not know why i wrote size_t in this

[eager yew]

this project is still alive

i just dont really have anything to talk about here

but the repo has commits being pushed

[inner rover]

What did you add recently?

[eager yew]

quite a lot since the start of april

multi page slabs were one doohickey

that got some nice perf. improvements

and then i just did a lot of QOL stuff (that's why i never posted anything)

and IOMMU work begun recently

a few things planned for the slab allocator are dynamic slab caches (i.e. they are created/destroyed on demand and as memory pressure changes) and per-cpu *current_slabs for a better hotpath

though for the former i need to first make rcu not suck

rcu is a bit suckful right now

it "functions" (preemptible rcu too) but it is suckful

[inner rover]

Do you mean dynamic magazines

[eager yew]

no i mean creation of entirely new slab caches to reduce memory wastage if enough allocations of a given size are happening

i think solaris had a shenanigan like this

[inner rover]

I don't see how this would be helping memory wastage

Solaris did do dynamic magazines and depot size using a working set

[eager yew]

if one allocates a lot of, say, 80 byte objects but the next slab size has 96 byte objects you get 16 bytes of wasted memory for every allocation

so instead of that the slab allocator would just make a slab cache with slabs with 80 byte objects

[inner rover]

Oh I see

[eager yew]

👍

i can double check this though

[inner rover]

I'm thinking this can mostly be avoided

[eager yew]

there may be a better solution but this is what i hypothesize

[eager yew]

how so

[inner rover]

No I think I read something similar in freebsd

[inner rover]

Just statically check your code to not do that

[eager yew]

what about dynamic arrays

i think that the static approach also incurs the cost of potentially having lots of largely unused slab caches which also use up some memory

[inner rover]

Then it's likely to grow again anyway

So

[eager yew]

i also think this is a bit easier said than done

[inner rover]

They don't really use much memory

[eager yew]

if you have, say, 200 bytes per slab cache, and have one for say, every 4 CPUs (that's what I do, unless NUMA is enabled. then I have one for every numa node, but I might change this), that would be 800 bytes for every new size

so yea, not a ton of memory i suppose

but it also has the small cost of increasing the amount of traversals required to determine which slab size to allocate from

i suppose i could fix it a bit with a bsearch

that would probably work better

hmmmm actually

that might be somewhat viable

if i have a max object size of 2048 bytes and make one slab cache for every 8 byte size jump from 8 to 2048 that's 255 slab caches i think

even with 1,000 bytes of memory for every order that's only about 64 pages

i think i'll just make a lot of slab caches at boot

[inner rover]

That's not what I mean

I don't necessarily mean adding a size to your generic caches

But rather have a bunch of different caches in different subsystems

Actually the freebsd trick I remembered is that they use the same backing allocator for caches with the same object size

[eager yew]

i thought about this but i think i might introduce this as a side feature but avoid it at the start because in my opinion it has a few potential drawbacks

1. various subsystems can have the same cache sizes (suboptimal to have a bunch of these scattered about)

• the freebsd trick does indeed seek to solve this, but it's then still using some "global thing"

2. this can cause some places to get a bit messy on whether or not a slab cache should be created for it

it has the major benefits of

1. lower lock contention (probably)
2. better "separation of concerns" (less oddball sizes in the global allocator)

I have this thing:

SLAB_SIZE_REGISTER_FOR_STRUCT(thread, /*alignment*/ 32);```

which basically makes an object in a linker section that is read at boot to create custom slab caches in the global slab allocator specifically for that struct size + align

i will probably introduce different per-subsystem caches but just not as a central/core feature

maybe some places that really do want those latency numbers to go down might benefit

[inner rover]

Potential of object caching is also cool too

By separating the caches instead of making them generic

The section thing is a good idea though

[eager yew]

as in you also don't have to re-instantiate objects once they're freed, so that things like the additional object allocations (e.g. pointers to structs in an object) remain there in addition to not having to re-init?

[inner rover]

Yeah, or just initialization in general

[eager yew]

hmmm interesting idea

[inner rover]

Well that is literally why the slab allocator was invented lol

[eager yew]

💥

[eager yew]

code changes are happening i swear i am breathing i just dont really write stuff here

gumbi blog when

maybe some day

mostly i just feel like my thoughts are everything everywhere all at once and aren't nearly orderly enough to be published in text form

[eager yew]

just dropping by for another quick update on these jokes here, i fixed all of the xhci hotplug bugs so hotplug works under the most ridiculous stress tests, more qol stuff, random misc. things

next up is rmap, early page cache support, a venture off into filesystem land again, and then probably coming back around to scheduling and drivers for sysfs integration....

rmap will be a big one though

cow lore

larry

[twin eagle]

How are you stress testing it? I might steal that idea

[eager yew]

import socket, json, time, random, sys

def send(cmd):
    s.sendall((json.dumps(cmd) + "\n").encode())

times = int(sys.argv[1]) if len(sys.argv) > 1 else 1000
s = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
s.connect("/tmp/qmp.sock")

send({"execute": "qmp_capabilities"})

for i in range(times):
    send({
        "execute": "device_add",
        "arguments": {
            "driver": "usb-kbd",
            "id": "kbd0",
            "bus": "xhci.0"
        }
    })
    time.sleep(random.uniform(0.001, 0.01))

    send({
        "execute": "device_del",
        "arguments": {
            "id": "kbd0"
        }
    })
    time.sleep(random.uniform(0.001, 0.01))

[eager yew]

i like getting joybaited by a git diff that shows that i have more deletions than insertions in a feature/fixes commit

> git commit -a
Oops, these files are not formatted properly:
 - kernel/irq/irq.c
 - kernel/mem/slab/chunk.c
 - kernel/sch/rt/init.c
 - kernel/sch/rt/slot.c
 - kernel/thread/reaper.c
Format them? [Y/n] y
All formatted and re-staged.
[main 4ff130ce] [mem]: lots of goody goody QOL work before we go to folio land
 28 files changed, 409 insertions(+), 258 deletions(-)
 create mode 100644 include/.DS_Store
 create mode 100644 include/mem/alloc_or_die.h
 create mode 100644 include/mem/fixed_size_alloc.h
 create mode 100644 kernel/mem/fixed_size_alloc.c``` a joyous commit before we go back to the depths of the underworld

no hate to mm people

[placid vessel]

brochacho pushed ds store

[eager yew]

whoops

happens

[eager yew]

come to think of it i quite like apple's wording of "Wired Memory"

all this Nonpageable and stuff sounds funkier

[stuck nexus]

That's not apple

That's a common term for it

[eager yew]

vma and folio stuff is coming on shockingly not badly

soon we will have processes and hopefully cow fork

[eager yew]

the more I work on this the greater my hatred for unix fork becomes

and it's not even like unix fork is even used in a manner that warrants it being a standalone operation most of the time

since you get a fork and immediate exec

https://lwn.net/Articles/1076018/

valid crashout

for the non lwn subscribers that article is "Moving beyond fork() + exec()"