MINTIA (not vibecoded)

[warm mural]

mintia isn't even a real kernel... it's more like a toy

real kernels include serenity OS, redox, etc.

[twilit smelt]

This would have been the summer of mintia2 if I hadn't unexpectedly gotten this internship

[dense vigil]

anyway given you have a robust platform to build on, i can add uacpi support and even gfx stuff (ill hopefully have some of that figured out in 3 yrs)

[twilit smelt]

I squeezed in what I could during the first part where nothing was going on

Which was continuations

I'll keep trying to find time to work on it a bit but I also have a summer class

An online one

at uofu

which I'm terrifyingly behind in rn

[warm mural]

will i watched a 2 hour long documentary on mormons today

the utah lore goes deep

apparently it was supposed to be missouri instead

[vast valve]

And shrekOS

[warm mural]

what kinda class is it?

is it a challenging one

and is it synchronous or not (i.e do you have to attend classes?)

[twilit smelt]

It's asynchronous and not that challenging

I've had plenty of time to get and stay caught up but I've just been like gridlocked the second I get home from work and also all weekend

[twilit smelt]

I like the part where the federal government marched on Salt Lake City and was going to slaughter all the Mormons until they reached an agreement last second

[warm mural]

is it like a maths class or CS or something idk

[twilit smelt]

It's my last gen ed class I wasn't able to take earlier

[warm mural]

they also made them illegal by making polygamy illegal

but the thing you're referring to is in missouri I think? or maybe ohio or illinois idr nvm I can't read

[twilit smelt]

No

They marched on Salt Lake City

https://en.wikipedia.org/wiki/Utah_War

[warm mural]

ah but they also wanted to exterminate them before

Afaik the Utah war wasn't full blown conflict

[warm mural]

By considering them enemies in some previous state but I forgot which

Ah Missouri it was

"the Mormons must be treated as enemies, and must be exterminated or driven from the State if necessary for the public peace"

yay freedom of religion

[twilit smelt]

I mean they were an evil cult

They got better later bc they had to in order to integrate

Watch a documentary about the FLDS if you want to see how the early mormon cult was run and how awful it was

Some of them are still doing it

[warm mural]

I mean the polygamy stuff was corruption

but idk how evil they were

But the guy doing the documentary was an ex Mormon so maybe he didn't want to demonize them too much idk (Johnny Harris)

[twilit smelt]

It was basically a pedophile cult for sleazy old men to use their underage daughters as currency and trade them around as wives

[warm mural]

Now it's also like the richest church

[twilit smelt]

They were very evil

Very very evil

Sociopathic con artist pedophile nymphomaniacs

Child rapists etc

The men at least

But like when you read about federal army generals lamenting the fact they lost the chance to bayonet a bunch of Mormon guys that's not them just being genocidal it's because they were getting constant news out of that cult that was like the modern FLDS on steroids

In terms of being a child marriage and rape factory

[warm mural]

Damn, not cool

Did you know the stone on Washington monument says deseret

[warm pine]

an early logo for bhyve said that too

[twilit smelt]

Longest I've been outside Utah in my entire life

It's been weird having to consciously drop the assumption that everyone I speak to is a mormon until proven otherwise and just jump straight to the part where I can talk to them like they're a normal person

[ocean valley]

excuse me? I'm here 😭 that makes at least one person in utah

[twilit smelt]

Normal person*
*by Bay Area tech standards

[warm pine]

what's so difficult about speaking to mormons? they are few here and the only ones i've knowingly spoken to were missionaries from america here for uni

[twilit smelt]

going right from one wackjob culture to another... I never had a chance...

[twilit smelt]

Utah Mormons are extremely gossipy and judgmental

[warm pine]

and they were like the usual sort of american, always gay and smiling, but maybe a bit more so

[twilit smelt]

I found them to be profoundly untrustworthy

if you're not in their in group

[warm pine]

i thought they were supposed to be modelling a perfect lifestyle to present mormonism well to outsiders, but maybe it works differently when it's their own homeland

[twilit smelt]

Utah Mormons have a bad reputation among out of state Mormons

As being like really mean and cliquey

I've seen lots of ex Mormons who say they lost their faith when they finally moved from the east coast to "Zion" (the salt lake valley) and got bullied or witnessed bullying so relentless that it completely shattered the idea that this was where god's chosen people resided in an earthly paradise

They moved to Utah after hearing it was an idyllic land for their kind for like the entirety of their life as an out of state mormon and were like wow these are just a bunch of assholes

And their faith was destroyed

[vast valve]

Yes but you aren't real

You are a figment of my imagination

[ocean valley]

[vast valve]

Thus Utah is still empty

[ocean valley]

dang =/

[warm mural]

All Americans are gay

[warm pine]

does the evidence back this claim?

[warm mural]

yes, see e.g this server

No straight American

[earnest zenith]

I love George Galloway

[warm pine]

he's some guy

a fuller excerpt https://vxtwitter.com/earlbndip/status/2067692541554610358

)))LBD™️((( (@earlbndip)

🏳️‍🌈 ally

▶ Play video

vxTwitter / fixvx · Jun 18, 2026, 7:34 PM

[earnest zenith]

lmfao

[warm pine]

that quote that goes around of lenin saying "there are decades when nothing happens and there are weeks where decades happen", galloway literally just invented it himself

it was discovered recently

[earnest zenith]

oh, wow

I didn't know that

tbf, there's a lot of fake quotes going around

Louis XIV probably never said "I am the state"

[warm mural]

One that's annoying is people saying "An interviewer once asked <popular guitarist> what does it feel like to be the world's best guitarist? He responded: "I don't know ask <other guitar player the person who quotes this likes>"

I've seen many variations with all kinds of people

[twilit smelt]

https://soundcloud.com/loukeman/you-knowww

https://www.youtube.com/watch?v=c9KdfFCL7f8

[twilit smelt]

This is insane

[shadow ridge]

NT workbook manga

[twilit smelt]

It's the cover of the Japanese edition of Showstopper!

I forgot it looked like that

The Japanese title is "Fighting Programmer"

[twilit smelt]

bonus Japanese language box for some vax materials

VAX: 😑
VAX, Japan:

[true bloom]

lmaoooooo

[delicate plover]

This is good example of the sort of things that make most mainstream Christian denominations believe that Mormons are heretical

[dense vigil]

Mintia in shambles

[twilit smelt]

"NT shaped"

Is there code anywhere or

Guarantee you the author of that article isn't qualified to tell whether it's "a windows kernel" in any meaningful way

[short owl]

"NT Shaped" so they just wrote the code in the shape of "NT"

[delicate dome]

Right so it made a larp kernel that happened to use PascalCase identifiers

[queen torrent]

osdev has been solved lets all go home

[icy bridge]

osdev wiki bare bones but the symbol names are Pascal case

[mortal thunder]

The hell it did

[shadow ridge]

every time an AI OS is made

it looks legit at first

then you look at it closer and closer

and you start seeing that it's actually a bunch of kernel looking parts haphazardly thrown together and sometimes not even connected

like one time there was a vibe coded kernel that had a bunch of code for syscalls and shit but they werent used at all and everything ran in kernel space

[raven drift]

The reality is probably "Fable 5 wrote a demo kernel that looks like Windows in 38 minutes but don't look at the actual code because it's bloated and full of shortcuts"

AI is very good at making demos

What ppl don't get is that getting from demo to actual product is still a long way even with AI

If demo-to-product was easy, we would have seen an influx of new products that ppl actually use in the last 18 months or so

But that didn't happen

[warm pine]

Suiche is best known for his work in the memory forensics and computer security fields. His most notable research contributions include Windows hibernation file analysis and Mac OS X physical memory analysis
possibly he is

but until we see it we can only assume there are serious and probably fundamental flaws in what it's made

[vast valve]

so does this count

//       ___                 
//      /__/\          ___   
//      \  \:\        /  /\  
//       \  \:\      /  /:/  
//   _____\__\:\    /  /:/   
//  /__/::::::::\  /  /::\   
//  \  \:\~~\~~\/ /__/:/\:\  
//   \  \:\  ~~~  \__\/  \:\ 
//    \  \:\           \  \:\
//     \  \:\           \__\/
//      \__\/                

[blissful smelt]

nt shaped documentation

[vast valve]

I'm sure you could mutalate some code to be in that shape too

[true bloom]

Also making a kernel that boots isn't very impressive

You need like functionality too

[twilit smelt]

Being in the Bay Area for a few weeks has taught me that I picked up more Utah culture than i thought

Everyone here would be considered a mannerless slob there

I'm shocked by the stuff people casually do that would get you weird looks in utah

and would be considered rude

Like they'll just lick crumbs off their finger and then touch other stuff immediately

That is a HUGE no no in Utah you'd get called a toddler by all witnesses

[warm mural]

is it true that utah people are addicted to soda

[twilit smelt]

@warm mural lockless available page count management and per-list locking!!!!!!!

Coming to mintia2 in next commit

[warm mural]

very nice, what did you end up figuring out?

what I was thinking was probably having one lock over related lists like free list and standby list, and the rest are individual locks

[twilit smelt]

It's actually even finer grained than per list type

It's per page color

locking

[warm mural]

page color?

[twilit smelt]

Each list is already split into per color lists and im giving them each a lock

[warm mural]

color as in cache coloring?

[twilit smelt]

Yeah

[warm mural]

interesting

is that even useful nowadays

[twilit smelt]

Think so

dcaches are still not fully associative so yes it's still relevant to some degree

[twilit smelt]

The only problematic thing is the standby lists

But im figuring it out

I figured it out

Too bad I'll get killed before I can type it out

It was pretty clever

[twilit smelt]

the three places that are involved:

1. page lookup within object and ref'ing
2. page unreferencing
3. page reclamation from standby list

[twilit smelt]

what i thought up is smth like this (simplified)

LookupPage(object, vpn):
  lock(object)
retry:
  trackingpteptr = FindTrackingPte(object, vpn)
  pte = *trackingpteptr
  if pte.v == 0:
    pfe = LoadPage(object, pte)
    pte.pfn = PfeToPfn(pfe)
    pte.v = 1
    goto out
  pfe = PfnToPfe(pte.pfn)
  while true:
    refcount = pfe.refcount
    if refcount == 0:
      break
    if cas(&pfe.refcount, old=refcount, new=refcount+1):
      goto out
  // refcount is zero so its on standby list
  color = PfnColor(pte.pfn)
  node = FindNumaNode(pfe.nodeid)
  colorlist = &node.standbyLists[color]
  lock(colorlist)
  if pfe.object == object:
    RemoveFromList(pfe)
    pfe.refcount = 1
    unlock(colorlist)
    goto out
  unlock(colorlist)
  // reclaim path took the page away and hasnt cleared the tracking pte yet
  *trackingpteptr = pfe.location
  object.refcount += 1
  pfe.trackingpteptr = nullptr
  goto retry

out:
  unlock(object)

  return pfe


ReclaimStandbyPage(node, color):
  colorlist = &node.standbyLists[color]

  lock(colorlist)
  pfe = colorlist.first
  if not pfe:
    unlock(colorlist)
    return nullptr
  object = pfe.object
  pfe.object = nullptr
  RemoveFromList(pfe)
  unlock(colorlist)

  unrefobject = false

  lock(object)
  if pfe.trackingpteptr == nullptr:
    // lookup path already cleared it, and left an object refcount
    // we have to deal with
    unrefobject = true
  else:
    *pfe.trackingpteptr = pfe.location
  unlock(object)
  
  if unrefobject:
    UnreferenceMemoryObject(object)

  return pfe


UnreferencePage(pfe):
  while true:
    refcount = pfe.refcount
    if refcount == 1:
      break
    if cas(&pfe.refcount, old=refcount, new=refcount-1):
      return
  color = PfnColor(pte.pfn)
  node = FindNumaNode(pfe.nodeid)
  colorlist = &node.standbyLists[color]
  lock(colorlist)
  oldrefs = atomicDecrement(&pfe.refcount)
  if oldrefs == 1:
    InsertOnListAtTail(colorlist, pfe)
  unlock(colorlist)

pseudocode for my idea

[shadow ridge]

thats not utah culture

thats just what should happen

[twilit smelt]

i didnt say its hyper specific to utah

just that it is in the set of cultural traits of utah

and seemingly not in the set of cultural traits for the bay area

[dense vigil]

Rock is this true

[twilit smelt]

@warm pine u should review this and the following

[shadow ridge]

wait how is it possible for you to get a pte with 0 refc

or do ref counts not work the normal way for those

[twilit smelt]

pfe

page frame entry

an element of the page frame database

[shadow ridge]

ooh

ok my eyes are not working today

[twilit smelt]

https://github.com/xrarch/mintia2/commit/4b352d40a22eeb71a97240819dc0d416aa76075e

@warm mural

[mortal thunder]

what's the purpose of page frame coloring? to allow multiple allocations of physical pages to occur in parallel?

[keen stone]

the main purpose is to avoid cache aliasing

so that pages end up in different cache sets

but also yeah, if you have a list for every page color then you can lock individual lists

[mortal thunder]

i see. i'm curious if anybody knows how caching is implemented on x86? are there multiple sets or just one? how are the sets spread across memory? and more importantly, does it depend on the CPU? if so, can cpuid tell the OS info about the caching system?

[keen stone]

caches in x86 are n-way associative

you can query that from cpuid

[shadow ridge]

i think the intel docs actually explain how it works for a variety of intel cpus

[keen stone]

the scheme in x86 used for cache lines in l1 is vipt

i think that's not mentioned in any manual, but it's obvious if you calculate the index bits for any cpu

[warm mural]

I don't understand all of this yet, but one day I will 🫡

sounds good though, next step is removing the locks completely

[sharp igloo]

Patches are cheap, talk.

[shadow ridge]

cheap is patch, send talks

[keen stone]

patches

[lucid umbra]

cheep cheep

[tame phoenix]

cheap is cheap, cheap cheap 🐦

[digital pivot]

and are page channels related ?

[twilit smelt]

what are page channels

[digital pivot]

i dont know, nt mm in many places calls page channel realted functions

seems to be numa thing

https://en.wikipedia.org/wiki/Multi-channel_memory_architecture

i think its this

https://www.vergiliusproject.com/kernels/x64/windows-11/25h2/_MI_COLOR channel is a component of this

[twilit smelt]

What's not to get

[short owl]

Wtf is staff emeritus

[vast valve]

Former staff members

[twilit smelt]

The number of sets is (cache line count / cache associativity)

[twilit smelt]

Were did you see that

[short owl]

@warm mural roles

[twilit smelt]

I'm a former staff member and I don't have the rull

so it's probably former staff members that the admins like

[vast valve]

😭

[short owl]

That's because you're dog napper sky

[twilit smelt]

I've long been discriminated against due to the dog napping incident

[short owl]

Give the dawg back

[warm mural]

My memory management knowledge is lacking

[shadow ridge]

what is the perfect way to handle errors in a programming language

options?

exceptions?

return value?

return value as a tuplet of error and success?

errno?

[warm pine]

Remarkable breakup

The deferral of pte update/refcount change is smart and at a glance it looks entirely correct

I will look more close when im home

[warm pine]

In 80s it was considered ridiculous to not implement exceptions when designing a systems programming language

[keen stone]

will left...

[shadow ridge]

no??

@twilit smelt

wait whaty

he did

😭

[lucid umbra]

i prefer option/result types

[shadow ridge]

hmm

i might consider them as like

return value only stuff

[lucid umbra]

why special case it?

[shadow ridge]

so that it doesnt complicate the type system

[warm pine]

you can probably get the fluency of exceptions with return value based error handling

i think a new programming language called Zig does that

[shadow ridge]

with return value errors the problem is

you need multiple out parameters

or an optional type / return value as sandwichman says

[warm pine]

yes, a sum type of error and legitimate return might be preferrable, that makes misuse harder, doesn't leave in scope a variable which may not have been written to if there's an error-return

[warm mural]

Pretty much, yes, but one disadvantage of how zig does things is that it doesn't allow you to attach values to errors

like, you can only get an error code

I think the sum type/monadic return value is the best way to handle errors

[warm pine]

that's raatid, i don't see why it couldn't be modeled as a sum type of return or error where the error is whatever you want it to be

[warm mural]

Yeah, I think they plan to address that eventually

I don't think they want a full ML-like type system

[warm pine]

this is where the exceptions have the advantage

since they are orthogonal to normal return path the type system doesn't have to grow to deal with them

[warm pine]

incidentally it turns out a lot of Zig appears to be copied from Xerox Mesa/Cedar, which are sort of antecedents of DEC PILLAR

a lot of people from Xerox PARC left to go to DEC's SRC where they wrote the Topaz/TAOS operating system in Modula-2 and extended it into Modula-2+ with features they liked from Mesa/Cedar; PILLAR's design bears a close resemblance to Modula-2+ in many parts and a recent document hyena found proved it beyond all doubt

anyway as i have found out there was a paper implemented for Cedar called "Data Types are Values" which made it so that types were modelled as (const) values that existed at compile-time, and result in a distinctive of Mesa/Cedar: type declarations Point: TYPE = RECORD [ x, y: INTEGER ] v.s. constant declarations Origin: Point = (0, 0) are in fact the same thing, just a type definition is a constant definition of type TYPE. and this is copied by Zig

[warm mural]

Huh you're right

That's interesting

[warm pine]

this makes Zag some kind of cousin to MICA

[warm mural]

lol maybe

[warm pine]

https://xeroxparcarchive.computerhistory.org/cyan/cedar6.1/vm/.VMFaultsImpl.mesa!1.html your common ancestor

i need to comprehensively study the cedar OS codebase some day

it might be interesting

[warm mural]

I wonder if it's purely coincidental or andrew looked at old systems languages

[warm mural]

where does Mesa/Cedar descend from? That kinda syntax is still found in languages like ada, so I assume pascal?

ah well, that syntax mostly comes from algol ig

[warm pine]

the Mesa manual states pascal is the basis, with ideas from many other languages added, they name explicitly algol 68 and bcpl

[warm mural]

come to think of it, C is really weird in terms of syntax

well, B more like

[warm pine]

smalltalk also cited for Cedar

[warm mural]

no language looked like this before B I think

unless I'm missing something

[warm pine]

some of the C-like syntactic elements like {}-enclosed blocks in Mesa/Cedar seems to come from BCPL

in BCPL as you see it traditionally they use $( and $) instead for machines without curly braces

[warm mural]

ahhh

i was wondering why I didn't see any {}

I guess BCPL is the closest B-ish, but B really seems to be a syntax of its own

[warm pine]

i think with B and C conciseness was a real goal

[warm mural]

in traditional unix fashion

[warm pine]

C's type system on the other hand is roughly ripped from ALGOL 68

but while ALGOL 68 looks more like go or zig ([4] ref int) for an array of 4 pointers to ints, C did its own weird "declaration mirrors usage" with int *a[4]

[warm mural]

I feel like the C-ification of languages led to the reinvention of a lot of previous ideas from those years

[warm pine]

i think c should've done it the algol 68 way instead, that way it's like "declaration mirrors usage" (as in it's the mirror-image, inverted, of usage) with postfix dereference operator

[tiny swallow]

does auto here do infer the type as we know or is it something else

[warm mural]

B did not have types

famously

[tiny swallow]

ah

[warm mural]

only a word type

historically auto was a storage classifier however

until C23

[shadow ridge]

is it me or does the zig website have very deceptive examples

like they say this is the C code necessary for some sound stuff and it's much more complex than the zig stuff

this is the most badly written error handling code ive EVER seen written in C

like this has to be done on purpose right

[keen stone]

this person has taken "goto considered harmful" too much to heart

[shadow ridge]

im not saying zig is bad (in fact i like it so far)

but like

this is just deceptive marketing

[keen stone]

i ahree

[shadow ridge]

this would be literally like 5 lines of code with goto

[dense vigil]

Yeah thats not how you do C error handling

[shadow ridge]

oh my god

im totally stealing zig's error handling system

it feels like

you just have 2 return pointers

and error path returns to the second one

thats kinda genius ngl

it's like exceptions except 1000x better

[warm pine]

Its slower than exceptions but some like it

Exceptions are near zero cost if you don't actually raise one

[shadow ridge]

it only has spatial overhead, no?

it would still be 1 branch to call function and 1 branch to return from it, whether you're raising an error or not

[shadow ridge]

wait no thats basically the same as exceptions

[shadow ridge]

how is it slower exactly

[icy bridge]

isn't that aspect of it purely syntactical?

iirc it's lowered to (effectively) a result type

aka functions return struct { actual; error; } and the caller has to check whether it was an error

[shadow ridge]

idk how zig does it

but i would use 1 register for return value

and have 2 return pointers on stack

that the function chooses between

could also pass it in a register ig

[icy bridge]

I think that would be fairly difficult to get to play nice with cpu return target predictors

and in any case existing code generators like llvm (which zig uses iirc) don't support that kinda thing

[shadow ridge]

i think it would be managable since errors sit on the unlikely case

[vast valve]

I presume they could also do some packing of the error in certain cases too which could be more optimal in some cases?

[shadow ridge]

instead of a return you would just do a jump like a tail call optimization

i dont think that would confuse the cpu

[icy bridge]

the problem is that semantically it's a pop from the call stack and the cpu wouldn't recognize that

[shadow ridge]

hmm

[cursive charm]

i… must've missed a lot, apparently

[twilit smelt]

I won't doing smr stuff

Don't know how well it plays with pervasively paged and fully preemptible kernel and it also complicated af

Too risky

Already doing too much else

[twilit smelt]

@warm mural the general solution to the race you found i think is to check my own queue first before checking other queues when i do idle work stealing

[twilit smelt]

i lied

p sure i fixed it by just moving the thread stash update to indicate idle to under the ready queue lock

@warm mural see commit description to see the two other races i noticed while fixing the one you pointed out https://github.com/xrarch/mintia2/commit/b4e542948b951a3c44a0e1d0dbac0351a5258b82

[twilit smelt]

https://github.com/xrarch/mintia2/commit/d0368a6608c686023a9f9236a3457ecad6737266

lockless commit charging

by turning unsigned CommitUsage into signed CommitRemaining

that can become negative temporarily

@warm pine possibly of interest

we love our negative numebrs

[craggy spire]

The interactivity stash setter doesn't clear the interactivity?

[twilit smelt]

refresh

[raven drift]

That's also how we do it in Managarm

[warm mural]

Thanks

[warm mural]

how freebsd does it btw is that its idle thread priority is the lowest I believe, even "idle" background threads are higher priority

[keen stone]

isn't it like this in all oses

[warm mural]

what mintia does is that it has one idle priority i believe

[balmy zealot]

Could you possibly shed some light on what this is?

[warm pine]

tolerating them really relaxes things a lot

i once considered allowing them for reserving an allocation of physical memory but i found it unnecessary in that case

[warm mural]

Do you believe it sensible to split physical allocation in a reservation and allocation step?

[warm pine]

not anymore

i had cause for it before i split many vm locks into 'creation' vs 'stealing' locks

[warm mural]

I thought it might be useful to avoid partial allocations but I'm not sure

[warm pine]

my suspicion is these are the sort of things we ought to be relaxed about

[digital pivot]

like in windows before doing memory allocation the system ensures theres room for it

heres a similar windows function https://pastebin.com/iT4cg45h

its a bit more complicated but uses InterlockedCompareExchange as well

[twilit smelt]

my latest r/osdev interaction

redditors are so annoying lol

so damn smug

[queen torrent]

"Poop"

we had an ai bro actually doxx someone for complaining about ai

on r/osdev

like its insanity at this point

[twilit smelt]

wha

[queen torrent]

in the post someone did as a letter to the moderator some guy got so offended by it that they went on multiple comment threads there insulting people, calling their projects shit and even posted the linkedin of one guy ☠️

all with a holier than thou atitude

[queen torrent]

all of that got nuked by the mod already

[twilit smelt]

was it this

[queen torrent]

yeah

[twilit smelt]

wrong reply

oh my fucking god

the king of all redditors

i bow

[warm mural]

i wonder if this guy has a business in japan

[queen torrent]

his bio is just "cope and seethe"

💀

and when people asked him about his projects because if he was calling everyone's stuff shit then surely he had a good project he just ignored them

[vast valve]

How fucking fat is this one person

They are about to turn into a black hole

[ocean valley]

of course 😭

[vast valve]

What if we partnered with open osdev and nanobyte to make another open letter but this time with the infinity stones of racism and stupidity

[mortal thunder]

those subreddits dont even amount to anything

theyre just empty with him as the only member

[vast valve]

I mean

That's still chud behavior

[mortal thunder]

absolutely

it's even funnier though

like if this guy was a mod of some actual subreddits, that's one thing

but he's only a pretend moderator

[brittle locust]

oh that guy is fucking insufferable

[mortal thunder]

japan is one of the few countries that i actually DON'T blame for having strict immigration laws

[warm mural]

they're also just pretty xenophobic in general

[short owl]

Some of em are also racist too

[dense vigil]

This pic from twitter might be useful to you @twilit smelt

Learn something new for once

(Disregard the completely insane memory map on the left)

[warm mural]

as if will would even understand this schema...

bro probably does not even know what paging is

[dense vigil]

Gotta start somewhere

[twilit smelt]

This is too far advance for me

Don't intimidate me with these images it is a form of Bullying 😓

[dense vigil]

Man up

Pagig is when libraries at 4000000

[twilit smelt]

@warm pine upsetting observation

There are NO preemption-enabled safe memory reclaim schemes in any kernel except for RCU

XNU and FreeBSD's EBR impl disables preemption during read side and write side critical sections

[warm pine]

check again, epoch_enter_preempt

[twilit smelt]

why does it have two SMRs and one is called SMR and the other is called epoch

confusing

I bet there's an rcu floating around too and maybe hazard pointers

and the rcu is called smr2 and the hazptrs are called smrz

[twilit smelt]

It forbids sleeping

Not usable on paged memory

well

It is if you catch any page fault and then leave the section and then touch it outside there and then retry

[warm pine]

which paged structures would you anticipate being under SMR protection?

[warm mural]

I don't believe this is necessary

it's just sequence numbers

[twilit smelt]

It's not if you make the entities threads rather than CPUs

[warm mural]

also sleep rcu is a thing

[twilit smelt]

I'm aware

[warm mural]

Huh and that breaks the whole thing

Since polling is now extremely expensive

Unless you keep track of the lowest sequence per cpu or something from its threads idk

[twilit smelt]

It's honestly too expensive to poll all the CPUs already

on numa

polling node local CPU's isn't too bad but polling remote CPU's will cost you dearly

[warm mural]

the point of the freebsd thing is to not poll frequently

but yea it is pretty expensive

It's amortized by the magazine size

[twilit smelt]

Doing it at all is kind of obnoxious but possibly literally unavoidable because youd have to eliminate all cross node memory freeing otherwise and that is just not practical

[warm mural]

you should come up with numa aware preemptive SMR with fast reclamations I dare you

mhm this might suck but suck less than disabling preemption but if you pin the threads during critical sections it's fine

I think

[twilit smelt]

Handle table, per-file view space lookup trie, per-volume metadata buffer lookup trie, etc

But rn im leaning towards just foregoing SMR entirely for now

[twilit smelt]

I'm not smart enough

[warm pine]

aren't the latter two already secondarily paged? that is, they are torn down as primary paging dynamics demand

[warm mural]

is lock free really worth it vs fine grained locking and rwlocks?

Saying this as a lock free fanboy

[twilit smelt]

What's your reasoning

I don't see how paging kills those off like it does process page tables

[warm pine]

with both i assumed they reduced as memory pressure rises

[warm mural]

actually is preemption even a good idea in SMR critical sections

sure preemption latency and whatever but that could extend memory reclamation for a whiiile

ig you could do priority boosting

also, XNU does have smr_enter_sleepable

[warm mural]

ok i looked at freebsd epoch and XNU and I think i figured it out broadly

it's not that bad

NUMA polling is not really an issue if you do something à la tree RCU where there's one per-node summary of the current state

[twilit smelt]

whapever I'll just try to break up locks instead of doing this shniz

[warm mural]

i am currently describing it if u want me to send a wall of text

[twilit smelt]

or use hazard pointers where it'd be a substantial improvement

[warm mural]

but basically u can do it pretty easily and in the fast path (thread not preempted) it's as good as freebsd

[twilit smelt]

hazard pointers are the least bad to use across page faults and whatnot

[warm mural]

otherwise there's a sorting involved but could use a pairing heap

[twilit smelt]

they have a lot of overhead compared to other stuff but still could be more scalable than a lock, enough to be worth it sometimes

They scale to NUMA better also

[warm mural]

https://pastebin.com/HkQpTUZx

i think this is right

then poll is the same as normal but you do min(oldest_seq, stall_min)

then you can also do turnstiles and PI in smr_wait for cool points

and you get fully preemptible smr sections with ideally quick reclamation of memory

[digital pivot]

maybe look into srcu which is sleeping rcu

[twilit smelt]

This already came up

I knew of it

[twilit smelt]

this is just what tech twitter is now

this is hell

[ocean valley]

boy am I glad I only have twitter for art stuff

[twilit smelt]

dadpreneurs

[carmine grove]

dead internet theory

[twilit smelt]

"dad... did you have me just so that you could call yourself a dad on twitter and increase your trad cred among the other vaguely right wing bay area grinder p-zombies..."

"Of course not Asset #457! You're SO much more than just high signal to me"

[dense vigil]

How does he have time to reply to comments if he's building a startup

[teal trench]

waiting for claude

[night needle]

[raven drift]

building a startup around an AI written app that nobody will ever use

[sharp igloo]

"guys i bought claude max and a macbook how to make money from it" thats what twitter all is about nowadays

[twilit smelt]

@warm pine homework for you

How should a per-node commit tracking system that does expansion of node-local pagefiles etc to ensure there is enough room for all anonymous pages associated with that node to go somewhere, deal with remote node physical page allocations for backing a virtual page whose commit was charged to the local node

For example when a node is out of page frames and you want to look at a neighboring node rather than block

I came up with an answer but I want to see what u say in case it's smarter and I don't want to pollute ur brain with my idea

More explicit example of the issue this can cause if not handled in some way:

• A thread affinitized to node 0 makes an anonymous private memory allocation.
• Node 0 is charged commit for all these virtual pages and its paging space is expanded if needed.
• The thread faults on a page in this range, but no page frames are available on node 0, so it steals one from its closest neighbor, node 3.
• This node 3 page frame has been trimmed from the process and is on the modified page list for node 3.
• Because commit for this page was not charged to node 3, it does not have enough paging space for it! Strict accounting has failed

[warm pine]

this brings to mind cellular irix which has some kind of explicit page loaning concept, let me see if i can find out about that

[twilit smelt]

I now have 2 answers

[warm pine]

i can't find what i think might've been interesting in cellular irix so i will just give what comes to my mind instead

borrowing a page from another node becomes explicit and you track it distinctly

as i think i recall in cellular irix their equivalent of a vm_page_t is created on the node borrowing and the node loaning it has some record of that loan

this is problematic

when you loan out you are giving up control

you might decide as the loaning node you want it back so you can meet your own obligations but the request is fallible!

i think this must not be how cellular irix does it

[twilit smelt]

would u like to hear my two ideas now

Idk if either are any good

[warm pine]

can you just direct the borrowing node to write out the page? if he is the one charged commit, he has the capacity to outpage it

[twilit smelt]

That was one of my ideas:

1. When a pagefile runs out of space at write out time, check neighboring nodes' pagefiles for space and redirect the write there. Some pagefile will have space enough, because commit was charged to some node for this page.
2. Add a "destination node" field to the PFNDB entry which states which node's modified list a page frame should go onto. This is set to the node that its associated virtual page was charged to, and directs the page frame to be written directly to that node instead. When not modified, it still goes onto its home node's standby or free list when refcount drops to 0.

[balmy zealot]

I’m sorry to interrupt, but do you mean a NUMA node when you say node here?

[twilit smelt]

Yes

[warm pine]

option 2 i like

i want to find out about the page loaning infrastructure in cellular irix though

i'm sure there is an idea there that might be useful here

but i can't recall it

option 1 does have the nice property of being symmetrical with the means by which the page was borrowed in the first place

[twilit smelt]

#2 can mess up clustering

[warm pine]

it's a good point, you end up potentially writing out pages willy-nilly across page files

[twilit smelt]

There are numa systems where there are per-node disk controllers that can IO to any disk but cannot access remote node phys mem

So the storage stack would have to detect this page frame that is not on the same node as the others and go out of its way to fragment the request and send that fragment containing that weird remote page to its node's disk controller

This assumes they can all at least access the same set of disks which is at least usually true

In this scenario the remote node's disk controller would basically block waiting for the local one to finish its IO, then it steals ownership of the disk and writes that page to that pagefile

AViiON disk subsystem worked like this

This isn't very efficient .. .. and it assumes all disk controllers share a pool of physical disks that get dynamically reassigned which was true on AViiON but im unsure how true it is on others. If they don't share the physical disks then this just doesn't work because there's no physical way to write this page frame to the remote pagefile without a bounce buffer

Ideally there's some way to make it so this page frame is always directed to its local node

[digital pivot]

i think nt has a similar separation to #2 in a form that _MMPFN stores a partition independently of the pfn's node number

though saying write it to node 0 doesnt prove that node 0 pagefile has space when the page is trimmed?

[twilit smelt]

It sounds like all the disk controllers sharing and dynamically reallocating physical disks between them is a nearly universal thing on modern NUMA (that is, all disk controllers can access all disks equally, but they cannot necessarily DMA all physical memory equally)

possibly AViiON was the first to do it

I DONT WANT to assume that however

I am pretty sure SGI Origin's disk subsystem didn't work like that for example

just too big

Works well on physically smaller NUMA machines

Origin was like fills up a room type thing

@warm pine option 3: when you want to allocate from a remote node, attempt to charge commit on that node for that page. If commit is immediately available then steal the page. Otherwise move on checking others and ultimately just block on your local node for more memory

[balmy zealot]

It’s a naive question, but would separating the location of the page (node) from where the commit/swap file information is stored solve the problem?

[twilit smelt]

What do you mean by that

[balmy zealot]

Why is the latter stored alongside the node? Under normal allocation in remote NUMA node, only the first of the entities would be affected if they were separated.

Thus, even a set of NUMA nodes could be associated with a single entity containing information about the commit/swap.

[twilit smelt]

Commit tracks the space that is promisable in phys mem + paging space on disk. Paging space is often per NUMA node because disk controllers are often physically per NUMA node

So commit should also be

Because paging space is independent between nodes due to physical constraints

Which means the commit limit varies depending what node you're looking at

So a global commit limit would be inappropriate

[warm mural]

this is what i had in mind with my little mm knowledge

[twilit smelt]

The difficult thing about that is when do you uncharge it? I guess it would be at reclamation time or whenever it is removed from the memory object

And what do you do with the original charge for that page on the original node?

[warm mural]

ig if a node detects it needs to borrow, it uncharges the commit on its own node, and the commit is uncharged when the page is returned to its home node

idk if that would work I'm speculating

[balmy zealot]

Does connecting the controller to one of the NUMA nodes in itself make its disk space accessible exclusively to that node?

[twilit smelt]

As i mentioned on modern NUMAs it normally does not

They share a pool of physical disks which are reassigned ownership between controllers

depending on which one is accessing it the most

[twilit smelt]

Note to elf:

Wrap package for santa

Note to self:

Fix idle priority inversions with timeshared priority threads that I stupidly added in last commits

[dense vigil]

smh i use obsidian for notes like that

[twilit smelt]

I use Apple note app

[keen stone]

i try to remember everything that i have to do

[nocturne nymph]

I prefer to take notes by hand, but I was surprised at just how big the notes file on my pc is

[warm mural]

What do you guys take notes about

[nocturne nymph]

595 lines

[warm mural]

I feel like I can remember everything but I don't

[twilit smelt]

Every time i think of something and I can't immediately do it, I'll make a note of it

Which sometimes turns into a very informal design doc

[warm mural]

Makes sense

I like your nickname

[twilit smelt]

Random section of mintia1 notes from 4 yrs ago

That note is over 1000 lines long I think and has basically a dev log of the last 2 yrs of mintia1 development

[warm mural]

DMs with staticsaga 👀

[twilit smelt]

@queen torrent a lot of that is from BEFORE ChatGPT even EXISTED you little WEASEL

or whatever that animal is

in pfp

[warm mural]

It's a shame he's not very active anymore, he's scarily smart

[queen torrent]

bird raptor alien thing

[twilit smelt]

thought that was porn for a second

I covered my phone screen really quickly

[queen torrent]

☠️☠️☠️☠️

[digital pivot]

whats wob

[twilit smelt]

Windows of Business

not really

made that up

[twilit smelt]

He graduated

[nocturne nymph]

Work on behalf

[queen torrent]

bro needs to do a j*b now :(

[nocturne nymph]

The scheduler takes wob into account to resolve big.little inversions

[twilit smelt]

No I mean like from here

Idfk anything about him otherwise

I'm not in the osdev social spheres

I talk shop only

[craggy spire]

Isn't it the new Windows-on-Boron subsystem

[warm mural]

Only the europeans are

[twilit smelt]

They invited me several times and it would be good until someone said israel

Let me tell you European libs and American leftists are NOT in sync on that there issue

[warm mural]

i dont think anyone likes israel

[twilit smelt]

This was like 2024 last time

Things changed

Also me and mint are like water and oil for some reason

Presumably hyena water and mint oil

Idk what hyena water is

Probably tears

From being bullied by mint

We've finally reached a fragile equilibrium to end the cycle of cat fights and bans where we just pretend the other doesn't exist

[digital pivot]

whats a big.little inversion?

[twilit smelt]

An indirect consequence of this being that I don't know anything about staticsaga