#MINTIA (not vibecoded)

1 messages ยท Page 19 of 1

shadow ridge
#

do you listen to anything except liminal oddly nostalgic the backrooms LSD music

twilit smelt
#

yes this is just my "old ass kernels" music

shadow ridge
twilit smelt
#

u joke but

image.png
shadow ridge
#

holy shit

#

18.30 hours

#

most of my playlists dont go above 15

twilit smelt
#

its every song i associate with mintia in any way

#

ooh this is a good one

shadow ridge
#

if i listened to a playlist during winter while cozied up in my room

#

i will feel cozy again if i listen to it in the future

#

it's weird how that works

twilit smelt
shadow ridge
#

it really sounds like the sound is coming from mall speakers in the first seconds

twilit smelt
#

yeah whenever it would come on in the train id freak out for a second that my headphones came unplugged and it was playing out loud

shadow ridge
twilit smelt
shadow ridge
#

i dont think playing cool vaporwave is the worst thing you can do

twilit smelt
#

these are also good

#

"frez wave" is like 18 carat affair's cousin or something

#

dont ask how im finding all these like sub-10k streams songs

shadow ridge
twilit smelt
#

i dont remember

shadow ridge
#

damn

twilit smelt
#

oh god im disrespecting the OG

#

vanilla!

#

vanilla and 18 carat affair were like 90% of what i listened to while working on old mintia

#

part of my motivation slump on mintia2 might be that these songs dont feel exactly right for it and it doesnt have its own musical identity to me yet

#

i also like lots of very popular 2010s music that were typical of like high schooler taste when i was that age. like the weeknd and mac miller and whatever, mostly hiphop

#

i kind of lost track of new popular music this decade

#

thats the early stages of getting older i guess, although 23 might be an early age to become disconnected

shadow ridge
#

i see

shadow ridge
#

indie artists ftw

twilit smelt
#

given that my favorite genres tend to be like r&b, hiphop, and also like vaporwavey stuff, its fun when i find a hiphop song that was recorded over a vaporwave track

#

genre fusion......

#

ignore the artist name i dont avow that.

#

another adjacent example is solange (beyonce's sister)'s cover of boards of canada - left side drive

#

which is r&b vocals over an idm track

#

coo

shadow ridge
twilit smelt
#

i didnt even notice what that artist was called until just now lmfao

#

which is cool

#

imagine trying to rap over that

#

u can place the music that has in common (for whatever reason) that my brain likes to to associate it with mintia and stuff on a spectrum of how rappable it is

#

the one above is unrappable

#

entirely

#

this one is incredibly rappable (and is from the same album as "Sweeter" above, the one that sounded like mall speakers)

#

for example heres a hiphop song that samples it!
https://www.youtube.com/watch?v=nSOKXUqAZf8

YouTube
BEAST KOAST
Reon Vangeฬ€r - The sky is no limit

Join our Discord server: https://discord.gg/8VwEzJb
Follow our playlist on Spotify: https://spoti.fi/31Vyyb3

Follow Beast Koast:
http://beastkoast.com
https://facebook.com/BeastKoast
https://twitter.com/BeastKoast
http://instagram.com/BeastKoast
http://soundcloud.com/BeastKoast

Artist
https://soundcloud.com/reonvanger
https://twitter.com/reonv...

โ–ถ Play video
#

which i stumbled upon by chance in a hiphop mixtape and realized it sampled a letherette track

#

if ur checking this channel expecting to see mintia updates and are disappointed by music talk. rest assured the music talk contributes meaningfully to mintia's developmetn fr

#

cuz part of the psychology for me working on it probably is to have something to associate cool music with so the more im interested in cool music the more likely i am to do work on mintia

shadow ridge
#

listen to joe rogan on 2x speed

#

maybe you can finally move on after you start hating working on mintia 2

#

and go do kernel work at microsoft or something

junior pivot
#

mintiyya: an andjective when the os is considered to be minted/mintic/mintified

#

whatever that means

shadow ridge
#

will stole mintia from mint

#

mint stole mint from linux mint

#

it's a chain of theft all the way down

tame phoenix
#

that's mint, man

golden meteor
#

40? I think the lowest I've done is 72

#

and that's because I had a 104 hour shift that day

ocean valley
#

I would say stop flexing, but I don't know if that's a flex or not pepekek

shadow ridge
nocturne sentinel
#

coding less than 238 hours a day is just lazy ngl

#

(looking at my commit history however will show I'm extremely lazy ๐Ÿ˜”)

acoustic sparrow
shadow ridge
junior pivot
#

I am unemployed chad
only real men would relate

twilit smelt
#

Russian hyenasky be like: mintiya

shadow ridge
#

mintinovich

shadow ridge
#

so you're a woman now

twilit smelt
#

ancient roman hyenasky be like: mintius

#

ancient egyptian hyenasky be like: mnt๊œฃ

mortal thunder
blissful smelt
#

chinese hyenasky be like: ๆ˜Ž่ตท้›…

#

i think that directly translates to "bright rising elegance" or something

#

nothingburger statement

nocturne sentinel
#

I'm just seeing squares ๐Ÿ˜ญ I gotta install a better font

shadow ridge
twilit smelt
#

more mintia commits woooo

image.png
twilit smelt
#

I should make a blob

#

A blob

#

A blob

#

A blop

#

A blog

#

I have too much wacky historical autism bouncing around in my head it should not be confined to an uncrawlable discord server

#

Ik the guy who runs the virtuallyfun blog I might be able to make guest posts there

twilit smelt
#

LLMs won't get it

golden meteor
mortal thunder
#

also its unlikely llms havent already touched your source code by virtue of it being on github

golden meteor
#

(they have)

mortal thunder
#

there should be codeberg pages which is like github pages but codeberg

#

and builtin anubis

golden meteor
#

Isn't there?

#

I thought there was

#

Maybe I'm thinking of sourcehut's

nocturne sentinel
#

oh hyenasky left

golden meteor
#

yeah

mortal thunder
#

wonder if it's bc of that situation from 2d ago

shadow ridge
mortal thunder
#

we were talking about some AI bullshit in #lounge-0 then i casually made a remark which was perceived as a ToS violation and got banned

shadow ridge
#

what remark

mortal thunder
#

then they kept talking and will was also advised to shut up, and when he didnt, he got muted

mortal thunder
shadow ridge
#

did you say you were 1 years old or something

mortal thunder
#

no

shadow ridge
#

or was it political

#

like hate speech or something

#

did you say the n word

mortal thunder
#

it was hate speech towards the big elite

shadow ridge
#

oh yea you love the n word dont you

mortal thunder
#

no what

shadow ridge
#

you said yes

#

???

mortal thunder
#

no i didnt say the nword

shadow ridge
#

cant get out of this one by deleting your messages

#

history remembers

mortal thunder
#

i did NOT say the n word

shadow ridge
#

n word boy

#

whatever you say n word boy

mortal thunder
#

i wouldve been banned instantly by bots

mortal thunder
shadow ridge
#

@mortal thunder

image.png
#

genuinely wtf did you say

mortal thunder
#

I can DM you if you want

junior pivot
junior pivot
mortal thunder
#

fuck big endian

#

all my homies hate big endian

junior pivot
#

realest

dapper mulch
# mortal thunder then they kept talking and will was also advised to shut up, and when he didnt, ...

he wasn't banned, but left himself after being muted? how can one see this here? because I can't. there's no signs of him having left the channel/server whatever it's called. Also you were banned and then unbanned? And if you weren't unbanned, that would mean your Boron progress report would have become read only? And you wouldn't be able to post here at all? Just because of a single post in a shit channel without any warnings or something? that's quite fkdup I'd say. Nyauxmaster also got banned recently, right? The guy wrote about having depression and then next day got banned for rule 1 violation, which I didn't get what it is since it's a kilometer of discord bs nonsense I doubt anyone has ever read in its entirety. Nice "support" for a depressed person. Kind of mean and clueless. Wiping out those few interesting persons out there. sigh.
Will, get back. This server needs more NT admirers.

mortal thunder
#

the audit log doesnt say anything about will being banned or kicked

#

i think he left by himself

#

Yes, I also thought it was fucked up, and so did the mods. The mods unbanned me because I was a regular here

dapper mulch
#

I got, this server isn't shown in his profile. Well, it's sad.

junior pivot
# dapper mulch he wasn't banned, but left himself after being muted? how can one see this here?...

bro what
afaik they explained the situation to linuxmaster beforehand and afaik too he understands it
everyone here wishes him the best but also mods aren't gonna gamble nuking the whole server in the process
ppl here would love to offer support but this isnt the place for it
as for will idk
he mightve been having some growing frustration from the server lately but when he left there wasnt something that kicked him out specifically
he's an og member ig and i hope he gets back
but let's just not exaggerate what happened in either cases

dapper mulch
#

So many regulations, I don't even know if it's "safe" to ask this: what exactly he (nyauxmaster) did? if possible, can you tell me what those "multiple rule 1 violations" were? maybe I'm also doing them? ๐Ÿ˜

junior pivot
#

lol u aren't locked out of any channel ig
ppl had talked about this many times
it
was
not
a
unilateral
action

#

there's no big osdev gatekeeping a truth ffs LiPerformDelisquesceUponObjectW

raven drift
#

Nyaux did not get banned for having or speaking about depression

#

He got banned because of multiple incidents, most of which occurred in private chats

junior pivot
#

iirc portmaster said somethibf like that before when someone asked about it

#

wait ill look that one up

#

#lounge-1 message

warm mural
#

Damn I didn't even realize nyaux got banned

modern gate
#

Nyaux got banned?

delicate dome
#

Yeah. I hope he gets better. At least he still occasionally messages me.

earnest zenith
# dapper mulch he wasn't banned, but left himself after being muted? how can one see this here?...

Will didn't get banned, he left, and told Mint the conditions under which he would rejoin. We found these conditions to be unacceptable.
iProgramInCpp was only temporarily banned for making a call to murder people.
Regarding the Nyaux ban, please don't talk of things you have no clue about. We cannot go into detail as to why he was banned, but the situation was the polar opposite of your portrayal.

earnest zenith
delicate dome
#

Not that I know as much as staff do but still

modern gate
earnest zenith
#

yeah

#

I talked about the kind of stuff we have to deal with recently in #lounge-0

delicate dome
earnest zenith
#

Anyway, this thread is meant to discuss MINTIA, so I'll lock it until Will rejoins, because otherwise this will only invite people to discuss drama.

golden meteor
#

Unlocked on request! #modmail message

twilit smelt
#

minting

modern gate
#

pog?

tame phoenix
#

peppermint or spearmint

#

all I know it's it a bit mintier

heady bobcat
twilit smelt
#

i added slabs with magazine caching

#

and just now i did a huge codebase cleanup where i split up the kernel header files into internal vs exported and changed symbol names to reflect this (kernel internal symbols now have a -u suffix on the prefix (for example Keu- rather than Ke-) if undocumented)

#

plus i got rid of any lines > 80 columns which there were a few of

#

also ive been adding NUMA support

#

the memory management state is already capable of being completely split up per node (and into multiple partitions per node) including duplication of the heaps with each of the nodes getting their own region of kernel address space so that the page tables mapping them can be node-local

#

some other components need to be split up per-node still like the worker thread pool and thread reaping

# 
//  o Splitting the worker thread pool to be per-node.
//  o Splitting the object and thread reaper lists to be per-node.
//  o Splitting the object allocation caches (possibly by remodeling ObuType to
//    be itself a per-node thing).
//  o Splitting the IO packet allocation caches and other caches.
//  o NUMA awareness in the scheduler (just enough to isolate scheduling
//    decisions to processors on the local node).
#

thats the todo list for numa support

twilit smelt
#

theres no particular reason to think that way with a hobby kernel but its an easy way to improve the quality of the codebase and another thing to be aware of that most hobby kernels dont bother with so i did it anyway

#

numa support is also completely unnecessary rn but is looking forward to like 5 yrs from now when im trying to run amd64 mintia on some insane 4-socket machine with a jillion cores

#

easy to add now, rly hard to add later so im doing it now

#

the data structures involved for numa support need to change

#

for some reason i made it into a mm-private structure called MmpNode even though other components need node-specific structures like the worker thread pool

#

these are things that were mostly globals before being numized

image.png
#

this is MmpPartition which contains segregated physical memory state

image.png
warm mural
#

How are nodes structured? do you have things like neighbors, parents or whatever to find the closest node?

twilit smelt
#

each NUMA node has an integral partition but the idea is people will be able to create new ones and create little isolated memory universes for running jobs in and stuff

#

(NT idea i stol)

twilit smelt
#

im not going to be much fancier than that

warm mural
#

ah I see

twilit smelt
#

for example if a node runs out of pages im not gonna have someone go out to another node and try to find more pages over there

warm mural
#

I was planning to do something like solaris lgroups

#

but that may be trickier than your thing

twilit smelt
#

itll just block for pages on the local node until the per-node modified page writer has written out enough apges to the per-node pagefile or whatever

warm mural
#

you have to keep track of whatever pages you stole from another node so when you have enough free memory again you give them back

#

well ideally

twilit smelt
#

yeah no idc about going that far at least initially

#

thats something that could be added on top of this work

#

later

#

if i need

#

it

warm mural
#

true

twilit smelt
#

segregating stuff up front is easy enough but something like that would need lots of tuning from actually trying it out on a numa system

warm mural
#

maybe I'll just have rudimentary support like you do for now

twilit smelt
#

and i dont feel like spinning up an XR/frame emulator with NUMA latency simulation

warm mural
#

I dont expect to run on actual numa hardware but its fun

twilit smelt
#

the only thing cellular irix had on top of this was fault isolation

#

they managed to isolate kernel panics to be per-node and have other nodes be able to reboot the one that panicked

warm mural
#

oh that's cool

twilit smelt
#

unless it happened on the boot node (the "golden node") then the whole system would come down

#

because all IO was on the golden node

#

including the vfs and stuff

warm mural
#

ah that's weird

#

was there a specific reason why IO was only on that node?

twilit smelt
#

you could still initiate IO from another node on a ccnuma machine it would just be rly slow

#

and it was probably because drivers

#

used tons of globals and stuff

#

and also it probably wasnt obvious how to split the vfs to be per-node

#

seems like inherently global state

#

so they probably just went

warm mural
#

well the vfs I think you can just ignore numa

twilit smelt
#

"if you want to do IO fast then do it from the golden node, only do compute-heavy tasks from remote nodes"

#

i think VMS basically had the same philosophy with their NUMA support

#

which in VMS fashion they had their own terminology for, "RADs", Resource Affinity Domains

warm mural
#

i like the name locality group

#

better than node/partition meme

warm pine
twilit smelt
#

At least for mintia2

#

NUMA aware bootloader will do that transparently

warm mural
#

I guess that makes sense

twilit smelt
#

I'm aware of per node text and was intending on implementing it it's just not something I can test enough to bother with rn

warm pine
#

i say the best impression of cellular irix is to start by picturing every cell as a totally independent instance of irix

twilit smelt
warm pine
#

then the independence is relaxed by allowing very controlled borrowing between the cells

warm mural
#

huh I guess that is a pretty sensible design

warm pine
#

but so thorough is that independence that a cell can fail totally and not bring down the rest of the system

warm mural
#

is there a book on IRIX?

warm pine
#

i wish!

warm mural
#

how do you guys know technical details about it

warm pine
#

cellular irix 6.4 technical report and papers on its antecedent (stanford HIVE)

warm mural
#

ah well I guess this is kinda technical

warm pine
#

this was the thinking of the time, that even with cache coherent numa you had to start not with a scaling up of the traditional smp architecture but rather with what distributed systems did

#

it was a big topic of interest in those days

#

they were probably right about it too

digital pivot
#

as thread running on a cpu on node 0 should tell mm to allocate from that node

#

and then ke could migrate that thread to node 1

#

but your allocations were on node 0

warm mural
#

he's not gonna do migrations between nodes I think?

twilit smelt
#

I won't support migration (which is the real challenge) and a process will be bound to a node from birth

digital pivot
twilit smelt
#

all threads in the process will run on that same node

digital pivot
#

would you wait or run on another node

twilit smelt
warm mural
#

oops wrong reply

twilit smelt
#

The load discrepancy on one node compared to another has to be enough to justify like a million extra cycles

#

Or a billion

warm mural
#

does the cellular irix model support migrations

twilit smelt
#

That's the cost of node migration

fleet root
warm mural
#

should you then migrate all memory pages?

twilit smelt
warm mural
#

like you allocate on your new node and give the pages back

twilit smelt
#

Linux does at least

#

Also all the threads in the process are normally switched over to the new node

warm mural
#

yeah ok that's what I thought

twilit smelt
#

So it's very expensive

fleet root
# warm mural should you then migrate all memory pages?

I don't think so, at least I remember a good node and just try to get back as quickly as possible, instead of starting a full migration. This is a temporary measure until there are candidates where the thread wants to go.

warm mural
#

doing something very expensive for a temporary benefit doesnt sound good

#

or maybe its not that expensive actually

fleet root
#

If the node has exhausted its memory resources I also try the closest one from the graph.

warm mural
#

the thread would run but its performance would be affected obviously

warm pine
#

the administrator should probably be mindful to assign work appropriately to the nodes

fleet root
warm mural
#

Yeah true

#

Also I wonder, in mintia if you won't have migration or whatever, why do you even need per-node stuff? Can't you just spin up a kernel instance on every node

#

If the kernel practically runs independently then it can keep global stuff

twilit smelt
#

Also it's easier to manage this kind of system than to manage a distributed one because each node can directly start a process on another node and whatever

warm mural
#

can you have per-node text without distributing things like that

#

ah if you keep data global then it works I think?

twilit smelt
#

Yeah but ideally you have no globals and everything you'd previously have global becomes a field of the node structure

#

which is allocated in node local memory

dapper mulch
warm mural
#

yeah but you have to keep a global node array/tree no matter what I think?

#

unless you do that implicitly with neighbor pointers

twilit smelt
warm mural
#

you wanna keep knowledge of which node is closest to which

#

im talking about the case where you do migration

twilit smelt
#

i see what you were saying and no

#

that doesnt need to be global

#

you can duplicate that per node as well

twilit smelt
#

lookaside lists contain objects that are individually allocated in the heap and are a lockless singly linked list with protection against ABA using a big ass sequence number and cmpxchg16b

#

slabs pack together objects in big batched allocations to avoid allocator overhead (both in time & space) and whatever and maintain per-object free lists which are guarded by blocking mutex or spinlock

#

unless you use magazines then the slabs arent responsible for caching (only for packing allocations together) and instead the magazines are and they also do per-cpu caching

raven drift
#

one common example is that you have a large range of memory that many CPUs sequentially read

#

in this case, striping (alternating the pages between NUMA nodes) performs WAY better than separation

#

because the CPUs can fetch from different NUMA nodes truly in parallel while they are blocked if they're all waiting on the same nodes

#

and only allocating from the local node also leads to degradation for example if you have on thread that allocates buffers that are passed to another thread for processing

dense vigil
#

yup korona is right, its not that simple

#

its a bandwidth vs latency tradeoff

raven drift
icy bridge
#

and thus no two threads in different nodes will share memory

dense vigil
#

yeah but even for the same node the bandwidth will be slower if you only access numa local pages

raven drift
#

also, shared memory across processes is still a thing

#

via page caches or similar

icy bridge
#

from the description of the model i got the impression that each node would have its own page cache and coherency is maintained manually somehow

dense vigil
#

im curious how you could handle MAP_SHARED in this case

#

just on some file

icy bridge
#

no idea

raven drift
#

if you want separate page caches, you should probably just not promise coherency across NUMA nodes

mortal thunder
twilit smelt
#

Later it'll be easier to add bridges on top of full separation than it would be to dig trenches of separation through a concrete foundation though

#

So I'm just prioritizing maximizing separation for now until such time comes as I'm actually trying to do performance work on a real numa system

#

Which won't be for a long time

#

I wasn't trying to argue it's superior for performance or anything it's just what makes the most sense from an implementational standpoint

#

at this moment

#

looking forward

#

(way forward)

twilit smelt
tame phoenix
cunning mortar
tame phoenix
#

yw, soul warmed ๐Ÿ˜Š

heady bobcat
#

@twilit smelt I thought a bit more about the fastpath for page free from interrupt context that I mentioned the other day. I'm pretty sure that the following is possible:

Let there be some global atomic queue with two operations:

  • push(Page): insert a page into the queue.
  • clear(): clear the entire queue, giving you a pointer to all the pages within.
    This is implementable in terms of a singly-linked list (push is update Page->next, cmpxchg loop, and pop is xchg(nullptr)).

In interrupt context you trylock the allocator. On success, free the page, release the lock and return. Otherwise push the page onto this global atomic queue and then trylock again. On success, flush the queue.

The idea is that whenever you acquire the page allocation lock, you have to flush the queue of "pages to free", and whenever you release the lock, you have to test if the queue is empty. If the queue is not empty, trylock and on success, flush, then release again. All of this is to ensure that there is invariably someone who will acquire the lock and flush the page queue, without forcing anyone to block on the lock.

Pseudocode:

UnlockPageAlloc:
  While true:
    UnlockMutex(pgalloc)
    If AtomicLoad(pgqueue) == NULL:
      Return
    If !TrylockMutex(pgalloc):
      Return
    Flush(pgqueue)

The main problem with this is that of course this borrowing of work also applies to the happy-case of the interrupt-context fast free, and so now it is not so much of a happy-case anymore. Lots of pages being pushed to this queue can stall at the interrupt level for an unbounded time.

twilit smelt
#

ive begun an interesting project

#

i am doing a one-to-one rewrite of old mintia in jackal

#

because im upset that old mintia is the only semi usable OS on xrstation and its written in dragonfruit

#

that sucks

#

im reusing the mintia2 loader as the bootloader for the old mintia rewrite

queen torrent
#

damn, no more mintia2?

twilit smelt
#

that is not what i said

queen torrent
twilit smelt
#

this will help mintia2 development bc itll give me a platform i can do userspace development for it on in jackal

#

for when i get bored of kernel dev

#

and i can work from the top and bottom

#

until they meet

queen torrent
#

ohhhh

#

epic

twilit smelt
#

i think this will only be a few week detour

twilit smelt
#

Dude old mintia fucking sucked

#

This shit is ass

blissful smelt
twilit smelt
#

the improvement has been insane

twilit smelt
#

it sure was huge though

twilit smelt
#

the handful of new components ill have to write (like the XLO dynamic linker that the rewritten OSDLL will need) may be at least partly reusable in mintia2 later too

#

today i got done just starting it

#

adapted the mintia2 loader for loading old mintia's (soon to be 1:1 rewritten) kernel

#

and started rewriting some header files

#

ill rewrite all the header files first i think

warm pine
twilit smelt
#

deep in the 120k loc of old mintia

image.png
#

and i have dutifully translated it

image.png
blissful smelt
#

beOS ๐Ÿ™

twilit smelt
#

old mintia translation progress

image.png
#

the whole XRstation HAL is translated

#

to jackal

carmine grove
#

ah yes, mintia 1.5

blissful smelt
#

does jackal not have switch case or do you just never use it

twilit smelt
twilit smelt
#

at the rate im translating code at, this will take me at least half a year lol

#

prob should have done the math on that BEFORE i started

#

i could just get mintia2 to a workable userspace in that same amount of time

#

i could write a tool to do a lot of the heavy lifting/repetitive parts for me but idk im pretty lazy

twilit smelt
#

so i pressed the pause button on that and put a lot of work into more numa segregation work in mintia2

#

observe

#

whats left (broadly):

  1. make certain per-node things that pessimistically create as many per-cpu structures as there are cpus in the entire system (to avoid having to deal with weird races), only create as many as have been seen in the node with the highest number of cpus
  2. get rid of PsSystemProcess and replace with per-node system processes
  3. create per-node work queue threads inside the per-node system processes
  4. make all object type structures (and their contained pool caches) per-node
  5. make sure the process objects are being allocated from the requested partition and the thread objects are being allocated from their process's partition
  6. duplicate kernel text between each node automatically in the bootloader (kernel code paging will not occur when numa node count > 1)
  7. duplicate some constant (after boot time) data items into the per-node structure
  8. make string internment per-node
  9. make kernel stack pageout infrastructure per-node
  10. mock up a fake-numa xr17032 target to test that this stuff actually works in the node^.Id != 0 case
  11. disable BLD_NUMA on xr17032 and fox32 except on debug builds (only there for testing)
#

then i will have very skeletal but thorough NUMA mechanisms waiting for policies to be put on top of it to make it good later when im actually trying to run it on something NUMA

#

and will be able to continue working on the kernel without fear of accidentally making NUMA super hard cuz ill be designing for it the whole time

warm pine
#

Fast approaching a shared nothing style of kernel

#

Most of the existing kernels do a half and half kind of approach instead where they replicate a couple of things or try and do some kind of node local allocation preference but ultimately its a few tweaks they make to a shared everything kernel

twilit smelt
#

well "shared nothing" is not a goal but itll be highly segregated yeah

#

and then ill add bridges back later

#

is the policy i was referring to

warm pine
twilit smelt
#

another thing absent currently is any concept of like a distance map between each node mostly because i dont have any source of information for that but it wouldnt be difficult to add later when the scheduler needs it

warm pine
#

Do a shared nothing style approach and then relax that where you can actually share. I prefer it to the opposite style

twilit smelt
#

its usually just done as a 2d array afaik and you can duplicate that between each node for their own local consultation since its constant data

#

i looked up how ACPI represents it and its just a 2d array there

#

where the dimensions are the two node IDs

twilit smelt
#

for paging stuff in the kernel, instead of having a working set (or several) specific to the kernel, the kernel pages would just be faulted into each process's working set

#

and would not be eligible for pageout until trimmed from every single process that had touched them

#

the idea was to represent sharing of the kernel between them

#

i do not know why they changed this for NT, there is no hint as to what they discovered about it that was bad

#

they sounded excited about the idea in the mica workbook lol

#

and then its just gone in NT

#

so there must have been something bad

warm pine
#

I remember you mentioned that detail once

#

Its an interesting one

twilit smelt
#

the most likely thing i can think of is that it would take up like dozens of pages duplicated in each process's working set and wasting tons of cycles in the working set trimmer fruitlessly trimming these highly highly duplicated pages that almost never reach a share count of 0 anyway

warm pine
#

Given nt weren't originally to be getting pageable text maybe it just wasn't considered by rashid(?) When he implemented that

twilit smelt
#

I can minimize these by batching them (like I'm already doing), caching mappings, and also using per-node HHDMs for accesses where possible

#

For userspace shootdowns I can keep a bitmap in each process which has 1 bit for every core across all nodes and I set the bit when any thread from that process is scheduled on that core. When I do a shootdown I send IPIs to all cores with set bits and every few shootdowns (and also for very large shootdowns) I dump the TLB on all of those cores and then clear all the bits

#

And the scheduler will hopefully keep this mostly node local

#

actually

#

i only really need to send an IPI to cores that a thread from the process is actively running on and otherwise i can just increment a sequence number and when a core switches to my process, and the sequence number doesnt match, it flushes its TLB

#

that ones good because i already have a mechanism for the sequence number part because that is EXACTLY how i do asid management already

#

me ditching dragonfruit

#

its funny how neglected the IO system and "VFS"/"name cache"/"namespace" have been this entire past year while i just keep doing wacky stuff to the rest of the kernel

#

at some point i do need to give it a rest and make it start to actually do things

#

all the code i wrote for the name cache is probably going to be thrown away as soon as i get back to it

#

i jumped the gun on that big time

twilit smelt
#

each process stores a sequence number for each cpu, saved from the last time it was scheduled on that cpu

#

this is used for asid management stuff

#

on the cpu's end the sequence number increments and when theres a mismatch the tlb is flushed and the asid is reassigned when the process is next scheduled on it

#

what ill probably do is iterate over this whole array and decrement each sequence number, which is the opposite direction of the way the cpu is going so its unlikely to collide if they both wrap all the way back around and meet eachother, so that it sees a mismatch and flushes the whole tlb

#

currently each item in that per-process array of per-cpu sequence numbers is guarded only by the ready queue spinlock of the corresponding cpu (because thats whats held at scheduling time which is the only time it is modified)

#

but id want to add a per-item spinlock specifically for this

#

i should probably set a hard date for myself for when to abandon the numa work wherever its at and do the IO system

#

probably another 3 days

#

ill say on december 11th

#

the fact i wont actually rly get to test the numa work for performance and whatever for like probably 4 irl years doesnt discourage me from doing it now because

#

in 4 irl years it will be 1000x harder to add it if its not already there

#

this thing will be an actual monster by then itd be nearly impossible

#

also it makes it look cooler now

#

so thats another factor

#

it was sort of hard already and it doesnt even do anything yet and id partitioned up the memory manager already

twilit smelt
#

thing that comes up a surprising amount

#

the number of processors in the node with the most processors

#

its tempting to size stuff that is per-node and per-cpu with however many cpus are in that node

#

but that opens race conditions where the thread could take the pointer to a per-node structure for the current node

#

then get migrated to another node, specifically to a cpu with a node-local ID higher than the number of cpus on the old node

#

and then try to index that structure belonging to the old node with the new cpu ID and access out of bounds memory

#

this is not an issue if you give as many per-cpu slots for each per-cpu per-node thing as there are the max number of cpus on any node

#

the ID will always be valid no matter how it gets migrated

#

this can waste some memory but even if the nodes have an asymmetric number of cpus (cores) its probably not THAT asymmetric

#

and its a lighter weight solution than disabling preemption or something

blissful smelt
#

my solution to this pickle is to just do that

twilit smelt
blissful smelt
#

makes sense

#

I removed that from my code after you brought that up and some of my longer running tests saw big speedups (~10%), so now I just disable preemption when modifying CPU local data

twilit smelt
#

if the thread is pinning itself

blissful smelt
#

does mintia2 have a kernel memory model defined anywhere like how linux has the LKMM

twilit smelt
#

mintia2 i mean

#

so no lol

warm mural
#

relaxed is pretty much free afaik

#

wait can you use relaxed actually here? idk

twilit smelt
#

you can use a normal write

#

because other cores are only going to examine that when your thread has totally stopped running and is on a ready queue and by then there have been implied barriers

warm mural
blissful smelt
# warm mural did you use atomic relaxed?

no i was using acquire/release and another big overhead was probably also the frequency of this operation since i was doing it elsewhere and realized i didnt need to do that

#

i have also been considering eventually handrolling my atomics and abandoning stdatomic.h because i am starting to think that for architectures that i may want to port this thing to in the future (big one being ppc), stdatomic.h macros both

  1. won't be enough and
  2. cant trust the implementation to do it right to the degree i want it to be done right
#

smp_read_barrier_depends() ๐Ÿ—ฃ๏ธ

acoustic sparrow
#

if you cant trust your compiler to be correct

#

i have a lot of questions

delicate dome
#

I've never had problems with it

blissful smelt
blissful smelt
#

like on arm I believe the acquire and release ordering will do a full dmb ish when that's simply not needed

acoustic sparrow
#

consume isnt in C and C++ anymore anyway

#

and the second thing is not a bug either

#

because the reordering cannot be observed soundly

#

also those are from 9 years ago

glossy leaf
#

the bad thing about stdatomic is that it requires _Atomic which is why I just make my own using the compiler builtins

delicate dome
#

Of all things to blame, the C compiler? That thing that powers the world? GCC and clang are WAY too large for something like what you allege to slip through.

acoustic sparrow
#

thats like saying that stdint is bad for requiring int ๐Ÿ™ƒ

#

its a compiler intrinsic

glossy leaf
#

its just fundamentally stupid imo, like atomicity is a property of an access not type

acoustic sparrow
#

but it makes sense for it to not be exposed that way

#

you can also obviously wrap over the intrinsics with casts

glossy leaf
#

the compiler builtins take normal pointers

#

which is why they are fine

delicate dome
raven drift
#

fwiw there is std::atomic_ref in c++

acoustic sparrow
#

or well

#

not the c11 atomic builtins in clang

glossy leaf
#

you can always just use the gcc ones

acoustic sparrow
#

yeah or just use a cast lol

glossy leaf
#

I actually tried to use cast on msvc but it didn't like that KEKW so I needed to do the braindamage that msvc's stdatomic.h does because msvc only has builtins for seqcst atomics and everything else is a compiler barrier + volatile load/store

delicate dome
glossy leaf
#

?

acoustic sparrow
delicate dome
#

If you don't have an actual atomic instruction for anything but seqcst then only x86 gives you strong enough default memory ordering

glossy leaf
#

well yeah they also have intrinsics for the arm atomic insts that they use in the impl there

blissful smelt
delicate dome
#

If you want watcomc or some shit stop complaining

#

watcom is ass

blissful smelt
acoustic sparrow
#

on m2, Relaxed and Release appear to have no relative perf difference and the rest are a little bit slower

#

so i suspect they just decided to not expose them because on x86 there isnt a huge reason to

glossy leaf
# acoustic sparrow just dont use msvc ๐Ÿ™ƒ

that would be an option but I wanted to support something else than clang and gcc has a skill issue of not supporting seh __try/__except (though maybe I should just have stayed clang-only for sanity lol)

acoustic sparrow
#

clang-only is perfectly viable

#

also

blissful smelt
acoustic sparrow
#

msvc has builtins for things which arent seqcst atomics lmao

glossy leaf
#

that's not the intrinsic tho, and yeah on some architectures like aarch64 there are but not on x86

blissful smelt
#

should I handroll atomics though

#

I'm beginning to reconsider

acoustic sparrow
#

no

delicate dome
#

just use stdatomic

blissful smelt
#

since that would take a bit of time and the main benefits are supporting non-C11 compilers (who cares about them anyways) and slightly less strict barriers in a few places where I don't need a stricter one on some architectures

#

okay

glossy leaf
delicate dome
#

cursed

blissful smelt
#

fancy

acoustic sparrow
#

not without special options anyway

blissful smelt
#

try catch deadlock recovery ๐Ÿค”

glossy leaf
acoustic sparrow
#

which have debatable implementation correctess

#

(i.e. im pretty sure the impl is broken)

glossy leaf
#

at least from what I have tested it has worked fine

acoustic sparrow
#

its broken in a pretty subtle way afaiu

#

they also generate garbage codegen for that

#

because they promote every IR memory op to volatile

#

so sroa doesnt fire at all

glossy leaf
#

that I do know but its not a huge issue

warm pine
#

will you support filesystems with block size > page size?

twilit smelt
#

genuine question: does anybody support that? i dont think they do. i think thats why hard disk sector sizes stopped increasing at 4096 bytes

#

its extremely difficult

#

cluster sizes =/= block sizes, you can read partial clusters

#

i was assuming you meant the on-disk readable granular unit being larger than a page size

#

if you mean a virtual block that is composed of several sectors and is larger than a page size yeah that should be pretty easy, old mintia's FAT driver supported that

dense vigil
#

Logical is almost always hardcoded at 512

#

Physical can be very large

#

64k on some disks

twilit smelt
#

whatever is the smallest unit the driver can DMA into phys mem

#

which is probably logical

dense vigil
#

That one is usually 512, in rare cases 4k

twilit smelt
#

yeah its never larger than 4k for a specific reason which is that page caches in linux windows and macos dont support that

dense vigil
#

Yeah

#

Filesystem blocks can be of any multiple of those tho

twilit smelt
#

yes

#

it would probably be more doable to support on linux where they have readily available efficient phys contig allocation (the buddy allocator) but it would cause INCREDIBLE asspain for windows and macos

#

i dont think linux's page cache supports a sector size > page size at present

#

no such devices really exist

dense vigil
#

Who knows

#

You can make a virtual one like that easily tho

#

With just a qemu command line

twilit smelt
#

its a different story if the controller allows splitting the DMA for this one big ass sector across a virtual page boundary in the iommu or whatever

#

that would make it easier

dense vigil
#

Yeah abusing iommu is possible on modern hw

#

Most hw probably has one these days

#

Its funny some games even require iommu to run

#

Because there are cheats that work as a PCI device that snoops physical memory

twilit smelt
#

its another thing the PC was really late to

#

the 1989 MIPS workstations had an IOMMU chip which had a 1024-entry page table of 4kb IO pages which you could arbitrarily remap to any PFN

#

so there was a 4MB IO address space

#

that all DMA IO went through

dense vigil
#

Interesting, yeah idk why PC was that late to it

#

It solves a lot of problems

twilit smelt
#

VAX in 1978 used the normal MMU as the IOMMU

#

DMA would consult the CPU's MMU and would go through the normal address space

#

so youd set up a virtually contiguous mapping in kernel space

#

and direct it to the base of that

#

it used cpu virtual addresses

#

for DMA

#

this was more possible in 1978 when the MMU was its own board and not integrated tightly with the CPU on the same die lol

dense vigil
#

Thats convenient

twilit smelt
#

im unsure what they did with this in later VAXen that were actually microprocessors like NVAX in 1991

warm mural
dense vigil
warm mural
#

backwards compat is probably the reason tho

twilit smelt
#

this was the bug

#

no clue how i found that

#

apparently the IOMMU table here wasn't even on that chip it was just in some location in physical memory

#

some arbitrary location

#

which you could define programmatically

raven drift
#

Yeah I don't think linux supports disk controllers with minimum DMA access size > page size either

warm pine
#

Latterly work has been underway to support it at every level in linux

twilit smelt
#

in order to test numa support in mintia2 im going to add a numa configuration to xrstation lol

#

more specifically "xr/frame"

#

ill have a 2-node 8-processor configuration where each node has 4 processors

#

might even simulate the excess access latency for taking a cache miss on remote node memory

#

believe it or not this is still period realistic, big machines like this existed in like 1992

#

there was an m88k (not m68k!) based NUMA server from Data General called the AViiON 9000 or something

#

which had 8 m88k processors in 2 nodes

#

this will require extending the firmware's "device database" structure it passes to the bootloader to include info about the NUMA nodes but just growing this structure should be backwards compatible since its passed by pointer

#

so i wont break linux lol

twilit smelt
warm mural
#

hey will idk if you heard about this unix variant dg/ux it's pretty neat

twilit smelt
#

no i havent what is it!

warm mural
#

idk just some svr4 fork

twilit smelt
#

oh boring

twilit smelt
twilit smelt
#

which was pretty slow for 1992

#

youd have expected that more around 1989

#

the 1992 Alpha was 200MHz

#

(...which was so high that many people thought DEC was straight up lying about it until they got their hands on them and tested for themselves)

#

(more than 3x higher clock speed than the next highest clocked chip which iirc was a SPARC)

twilit smelt
#

the fundamental numa support stuff is p much done

raven drift
#

the DEC alpha was probably prohibitively expensive, wasn't it?

#

the 1993 Pentium had 66MHz

twilit smelt
twilit smelt
#

called AlphaPC

#

they sold this basically immediately

#

"high end PC range" for the time was like idk $7000 lol

#

ran NT

#

i was wrong that came like 3 years later

#

we do all of the funny pointer tricks in this house

#

sadly a branch is required for the case where a pointer could be in either node space or otherwise (for example the bss section of the kernel) so that it can default to the boot node rather than giving a garbage pointer

#

this could potentially be rectified by placing the data & bss sections of all kernel modules into node space for the boot node lol

#

the only time where i need to worry about this rn is the blocking mutex KeLock which can either be dynamically allocated (in which case theyre in node space local to where they were created) or statically allocated in the kernel's data section

#

and i need to calculate the pointer to the node structure for that because the turnstile hash table is per-node

#

i could simply forbid them from being statically allocated ig

#

that would pose a handful of chicken and egg problems though lol

#

during init

twilit smelt
# twilit smelt

to be clear this is a virtual mapping not some weird phys mem layout

#

i was going to reuse this strategy on future numa systems like after the amd64 port

#

on 64 bit the per-node spaces will be so large i can have a per-node HHDM in each one

#

and so on

#

should be cool af

warm mural
#

Static allocation is only useful for the idle thread

#

And my idle thread doesn't take blocking locks

twilit smelt
# warm mural This is what I do

you have numa support which led to the specific set of problems I described which you solved by disallowing blocking locks being statically allocated?

warm mural
#

no but I don't statically allocate turnstiles

twilit smelt
#

Neither do I

warm mural
#

ah you mean the locks themselves?

twilit smelt
#

Yes

warm mural
#

Also mintia does statically allocate boot turnstiles afaik so trl

twilit smelt
#

No those are per node now

#

At least in the numa branch

warm mural
#

I should look into numa

#

Can't wait to handle edge cases I'll never encounter outside of simulated NUMA VMs

twilit smelt
#

The initial turnstiles might be able to be removed since I'm not sure any contention actually ever occurs at the time they'd be needed

#

They're used to provide turnstiles for idle threads and a couple of initialization threads but the latter can give themselves turnstiles from the full blown normal allocator later probably and idle threads never do anything interesting enough to contend on a blocking lock

#

So that's something I'll look into ig

#

I think I added it out of paranoia when I first implemented turnstiles but it's become clear by now that they aren't needed

twilit smelt
#

@warm pine i try to minimize the amount of blocking mutex contention that can occur between nodes by making everything segregated per-node

#

but it can still happen

#

and because of the way turnstiles are donated

#

a thread that contended on a lock can end up taking away a turnstile that resides on a remote node

#

which will lead to long term inefficiencies

#

what do you think i should do about this

#

i was thinking that at certain safe moments (like before returning to userspace) i might have a thread check whether it has a node-local turnstile and if it doesnt, it allocates itself a new one and returns the old one to the remote node's turnstile cache

#

but then i need to find other 'safe points' for threads that never do that like system worker threads

#

i could just have all worker threads check for this case in their main work loop as a design rule

#

but thats difficult to remember and easy to mess up

#

for bespoke worker threads at least

#

im not thinking of any other good solutions though

warm pine
#

i had it in my head you once said you weren't going to have turnstile-based synchronisation between domains (despite the pain)

warm pine
twilit smelt
#

no i definitely have that

warm pine
#

it may be you can abstract workloop logic a bit so it's less on the onus of the workers to remember

twilit smelt
#

yeah i was thinking that

#

i could have a PsCreateWorkLoopThread function which creates a thread that runs an infinite outer loop that does this for you and calls your callback

twilit smelt
#

in a ccNUMA kernel

warm pine
#

because what immediately comes to mind is the case of objective-c per-iteration autorelease pools that are freed at the end of a workloop cycle; in that case the call to release the autorelease pools is made by an NSRunLoop class

twilit smelt
#

even if you make it really rare itll still happen once in a blue moon

warm pine
twilit smelt
#

inter-node turnstile contention was the main thing that motivate the "node space" design i have where you can compute the pointer to a node's control block just from a pointer inside of its node-local heap

warm pine
#

it may even have been a thought of my own

twilit smelt
#

i wanted to be able to go from a lock pointer to a per-node turnstile hash table

#

without making the lock bigger by adding a pointer to the node to it

#

or checking any other structures

#

and ideally really quickly

#

and indeed on my planned numa xrstation its just this lol

image.png
#

would be exactly the same idea on numa amd64 just with a 64-bit mask and gigantic node spaces rather than the puny 512MB planned for my absurd fantasy mainframe

#

i have no idea if this is a typical way to do this or if i invented something

#

i havent really been looking at anybody else's numa design for any of this

#

i read a few early NUMA papers from the 90s and i also read about VMS's NUMA support in the 2003 "OpenVMS Internals & Data Structures" book which was the last one they ever put out

#

but they were disappointingly poorly segregated in kernel mode

#

they didnt attempt anything like this to try to segregate the page tables themselves and stuff

#

theyd duplicate the kernel text on each node but beyond that they'd just put almost all of the kernel data on the boot node

warm pine
twilit smelt
#

yeah a node-local virtual access will only touch node-local memory even if you miss in the TLB on every level of the page tables

#

and the top level page table is always node local so you get that one for free even if you miss on a remote node space

#

one thing motivating this is that i think itll be necessary for trying to achieve tolerable compilation times for building mintia2 inside itself in xremu lol

#

by tolerable i mean like 6 hours instead of 3 days

#

and without doing completely unrealistic configurations

#

ill need mintia2 to scale to 16+ processors @ 25MHz in a simulated mainframe and fill each one with compiler jobs

#

ill even do disk striping and stuff lol

#

so that the IO scales

#

i want to simulate xrcorp's build machine lol

#

think that would be sick af

#

it would be extremely cool if i could one day develop it completely inside itself on my fake computer

#

the ability to do that is def a goal. whether i actually would do it is a different story because im way too ADHD to tolerate the speed of a 1992 build cycle even if im simulating it on what would have been a super beefy mainframe for the day lol

#

even if i changed just 1 source file and everything was in page cache itd still probably take 10 minutes just to link the kernel

#

would probably motivate changes to the build tool to automatically generate fragmentary linked objects for each subcomponent and stuff to accelerate the final link

#

id also want to look into precompiling header files

warm mural
#

is the hardware that slow?

twilit smelt
#

dude itd be 25MHz lol

#

in-order!

warm mural
#

ah yeah fair enough

twilit smelt
#

developing mintia2 on itself on amd64 id be more likely to do and take pleasure in lol

#

i could test it in amd64 qemu running on mintia2 on amd64

warm mural
#

Maybe you could skip certain slower passes in the compiler for faster build times

#

idk this is all hypothetical at one point so I have no idea where the bottleneck would be (even tho it's likely in the hardware)

twilit smelt
#

the compiler is fairly speedy

#

it runs on fox32os and can build a like 300 line source file basically instantly there

#

and fox32 is slower than xrstation by a bit

#

the jackal toolchain running on fox32os is already an extremely cool project on its own probably but we havent bragged about it publicly almost at all aside from a few low profile bluesky posts lol

#

i think me and ry both view it as a small feat in the course of a much larger project and not worth talking much about independently

warm mural
#

That's pretty cool

twilit smelt
#

you can write and test programs entirely inside fox32os which is more than the original macintosh could do

#

literally

#

lmfao

#

you had to use a Lisa for mac development for the first like year because they didnt have a native toolchain on the mac os

twilit smelt
#

im super annoying so i tend to feel smugly superior about not being that type of attention whore but at the same time i recognize that it is limiting my visibility a LOT which isnt pragmatic and is limiting my opportunities

warm mural
#

you should just start a blog

#

with these kind of updates

golden meteor
#

Xrnewsletter

warm mural
#

not necessarily related to xr stuff

#

any cool osdev stuff

heady bobcat
twilit smelt
#

but like turnstile allocation depends on a bunch of other stuff which is implemented using blocking mutexes which themselves rely on turnstiles

#

so it seemed complicated

#

however

#

every place like that is holding a mutex that blocks out kernel APCs by raising to KEP_IPL_APC

#

so it might actually work to replace the turnstile in the scheduler, if the previous IPL before the scheduler was invoked was < KEP_IPL_APC

#

i think this might be overkill though

#

like that might be too often

#

i think doing it once per outer work loop for kernel threads and on return to userspace for user threads is best

mortal thunder
#

@twilit smelt are you going to implement fork() support at some point? you mentioned at one point that you want to do linux compatibility

twilit smelt
twilit smelt
#

Going to attach an array (allocated in node local memory) of pointers to all the other nodes in increasing order of distance, to each node

#

I will walk this array to make scheduling decisions and also to obtain a free page when the local node is all out

#

Only for user memory though, kernel memory will remain strictly affinitized per node because of the longer term consequences of placing a remote page frame into another node's kernel heap

#

In order to quickly determine what partition a page frame is part of (and should be freed back to) I think I might just add another pointer to the PFN element on NUMA builds

#

Doing a binary search of physical ranges is still on the table though

#

O(log n) in number of contiguous physical ranges with respect to NUMA locality

#

Which would make freeing a range of virtual pages an O(m log n) operation where m is the number of pages and n is the number of ranges (which is at minimum the number of nodes). Probably an inconsequential extra cost

#

Also I might completely dissolve the MmpPartition and make it just per physical node and remove any plans for the ability to make user-created memory partitions

#

@warm pine thoughts on the above

warm pine
warm pine
twilit smelt
#

I don't think anybody supports that idea but NT which means nobody relies on it since everybody on big iron is using Linux lol

#

So it might be safely removable without any regrets

warm pine
#

There's sometimes ways of providing some level of memory partitioning but its usually a bit simpler and less thorough in how much is replicated

#

Linux cgroups for instance

#

While numa implemented in this much more fundamental and thorough going way that affects everything, as your partitions do

twilit smelt
#

I don't think cgroups involve reassigning page frames into strict groups right

#

It's more of a quota thing

#

where the actual page frames involved are immaterial and only the number of them counts

#

Lol why did they make it a filesystem that's ass

#

Linux abstractions are so goofy

twilit smelt
#

There's also a disclaimer at the top that the document is "hopelessly outdated" and it's from 2010 though lol

night needle
warm pine
#

You used to write to a file in the cgroup fs, a binary to be launched when the event happened

#

Cgroups v2 replaced that ridiculous approach

raven drift
#

it's amazing that linux has proper mechanisms in place for many things yet they use janky hacks in relatively new code all over the place

twilit smelt
#

It seems like partitions should be pared down to be more simplistic page frame containers but not removed entirely. The worker threads and pagefiles which were previously going to be per partition will be per node instead

warm pine
#

Review should really have been a bit more thorough there

twilit smelt
#

Partitions also need to allow a hierarchical construction

#

It seems like this is sufficient for the functionality the cgroups memory controller implements

raven drift
#

like, they could just have added a file that sets POLLIN on a notification with read() to dequeue notifications

#

or re-use eventfd

#

or use netlink sockets which they also added exactly for this purpose

twilit smelt
#

cgroups v1 was from 2007 tbf

#

v2 is from 2016

twilit smelt
warm pine
twilit smelt
#

Really a partition would just be reduced to a standby list and some memory resource limit numbers

warm pine
#

If there's a lack of pages then can ask the partitions to please reduce their standby lists

twilit smelt
#

The contents of their standby lists can be transferred to the per node ones until their min limit is reached

#

I want to pare memory partitions down to the minimum needed to implement the cgroups v2 memory controller

#

Ill put it on the todo list for after numa support is done

#

Also each node won't have like an integral partition or whatevr anymore and a partition's ultimate ancestor won't be some particular numa node

#

numa affinitization needs to occur on a per process basis within a partition

#

so the standby lists would be duplicated per numa node inside each partition object

#

I could also just recognize that the cgroup memory controller doesn't actually functionally require anything beyond simple accounting and that tracking individual page frames like this is only needed to enhance the behavior of advisory stuff

#

And just not do any of this and completely eliminate the partition object and roll the resource accounting part into job objects later

#

I'm leaning toward that lol

#

I'm in a simplification mindset rn

twilit smelt
warm pine
#

I know we both favour overengineering at times but the question does have to be for what would full partition objects be useful for you?

twilit smelt
#

Better benchmark scores running complicated docker or kubernetes workloads on big iron amd64 mintia in 2032

#

by 5%

twilit smelt
#

in the turnstile code, after i wake up from blocking on one, i can check if the new turnstile i grabbed is local to my node

#

if not then ill set a soft int pending at KEP_IPL_APC

#

the APC level soft int handler will see that the turnstile needs replacing and will do it then

#

so itll happen at the very next safe moment

#

itll go like this (pseudocode)

KeBlockOnLock():
  thread := KeCurrentThread()
  // ... do turnstile contention stuff ...
  thread^.Turnstile = donated turnstile
  if KeNodeFromPointer(thread) != KeNodeFromPointer(thread^.Turnstile):
    set software interrupt at KEP_IPL_APC
  return

KeApcInterruptHandler():
  thread := KeCurrentThread()
  if KeNodeFromPointer(thread) != KeNodeFromPointer(thread^.Turnstile):
    PsReplaceTurnstile(thread)
  // ... dispatch APCs ...
  return

PsReplaceTurnstile(thread):
  oldturnstile := thread^.Turnstile
  thread^.Turnstile = MmAllocateFromPoolCache(KeNodeFromPointer(thread)^.TurnstileCache)
  MmFreeToPoolCache(KeNodeFromPointer(oldturnstile)^.TurnstileCache, oldturnstile)
  return
#

its gonna be so funny if mintia2 turns out to be like a genuinely really really good numa kernel and it only runs on fake computers

digital pivot
#

_EJOB has VOID* PartitionObject according to vergelius

twilit smelt
icy bridge
#

i made a dragonfruit-to-c transpiler and compiled mintia with it, as a result i am fairly certain that the linux issues are caused by a miscompile

#

because mintia has the same issues

dense vigil
#

damn

twilit smelt
#

Also that's actually insane

warm mural
#

Monkuous casually dropping this and vanishing for another two weeks

twilit smelt
#

What did you do about the fact the C toolchain spits out ELF and old mintia expects XLOFF which I don't think is even documented

#

And also the ABI differences between C and dragonfruit you must have had to port a bunch of assembly to the new ABI

icy bridge
twilit smelt
#

Aw man you had to root around in the old lua SDK that i started when i was 15

#

I feel sorry for you

icy bridge
# twilit smelt And also the ABI differences between C and dragonfruit you must have had to port...

Less than you'd think actually - the only real differences in ABI are the return value registers (C goes a3, a2, a1, a0 and dragonfruit goes a0, a1, a2, a3) and the way varargs are handled (dragonfruit passes argc as the first argument and argv implicitly in the first free stack slot, C passes argc and argv as two extra arguments at the end). There's no handwritten assembly that does anything with varargs, so the only thing I had to do was edit the return value registers

#

Arguments are technically different but that's just a reversed allocation order so I take care of that by having the transpiler reverse the arguments in the output

warm mural
#

It's kinda weird the bug happens only in usermode

icy bridge
warm mural
#

you'd think code as big as the kernel would catch it

twilit smelt
#

Could it be stack frame corruption from interrupts

#

and exceptions

icy bridge
#

I highly doubt it, that's handwritten ASM that I didn't edit at all

#

The disassembly is exactly the same in dragonfruit-compiled mintia and transpiled mintia

twilit smelt
#

Is it deterministic like do you always get the same crash running ls there

icy bridge
#

Yeah

twilit smelt
#

That makes that easier to look into at least

icy bridge
#

It shouldn't be too hard to track down

#

But that's not what I'm doing right now (the main motivation for making the transpiler was to get mintia running on i386, and so that's what I'm doing)

twilit smelt
#

Just to make sure, you're disabling the red zone right

icy bridge
#

i386 doesn't have red zone

#

that's x86_64

twilit smelt
#

okie

twilit smelt
#

unless you're gonna do them in C

icy bridge
#

Yeah I'm just going to do them in C most likely

warm mural
twilit smelt
#

Also the fact old mintia lacks SMP entirely is a little sad

#

I last worked on it in earnest like a full year before SMP was even spec'd for XR/computer

#

Will at least make writing drivers easier lol

#

Synchronization is as easy as raising IPL or just disabling interrupts

#

(or blocking mutex if appropriate but probably isn't in a low level driver)

icy bridge
#

What might also make writing drivers easier is that I'm not targeting a modern PC

#

No ACPI, APIC, PCI, etc

twilit smelt
# icy bridge

How certain are you that this isn't due to mis-transpiles in your df to C tool

icy bridge
#

Quite certain but the i386 port would clarify that

twilit smelt
#

Well godspeed it would def be awesome to have old mintia on i386 pc

icy bridge
#

And on real hardware

twilit smelt
#

Yeah lol

warm mural
#

Will for the mintia rewrite you wanted to do you could use that tool and adapt it to jackal

icy bridge
#

Wasn't the point of the mintia rewrite to improve readability?

#

The tool just outputs something that compiles, it's not readable in the slightest

warm mural
#

Ah well make it readable

twilit smelt
#

Yeah i was rewriting mintia1 in jackal for readability and I gave up after rewriting the HALs by hand because i suddenly got daunted by how much code there is

icy bridge
twilit smelt
#

A tool would help but I felt it wasn't in my interest to spend time doing that

icy bridge
#

This is the result of transpiling LdrMain.df

#

Also one thing that really annoyed me about dragonfruit when making this tool is that it doesn't distinguish between non-volatile and volatile memory accesses

#

So I have to be incredibly pessimistic to ensure MMIO works correctly

#

Aka: volatile accesses are always used unless the operand is a symbol reference

twilit smelt
#

Yeah its very primitive i wrote the first df compiler in a single morning before school in like fall 2018 lol

#

Jackal actually also doesn't distinguish between volatile and nonvolatile accesses but it instead uses a keyword BARRIER which basically is equivalent to asm volatile ("" ::: "memory")

#

Which has caused complaints by other people

queen torrent
icy bridge
#

At least with that I could just insert that asm snippet

twilit smelt
#

When i say it caused complaints im referring to sandwichman who wrote a jackal frontend for his compiler (which is meant to target fox32, xr17032, aphelion (his own ISA), and AMD64) and BARRIER caused him a bunch of headaches

#

He tried to get me to get rid of it and replace it with a VOLATILE keyword which tags the next memory access as volatile but I didn't want to

#

Because there was already a ton of jackal

#

using BARRIER

#

He made it work

icy bridge
#

I mean going by that description barrier is something you have to implement anyway since calls to external functions are equivalent to asm volatile("" ::: "memory")

#

Unless it's stronger than that

twilit smelt
#

That's what I told him

#

I don't remember what the issue was

#

@lucid umbra what'd you have a problem with with BARRIER again

#

Also the correctness of my usages of that keyword in mintia2 is probably solid but untested because the jackal compiler isn't fancy enough to do a lot of the optimizations that it's trying to avoid anyway

#

It does some of them but anything involving complicated data flow is unlikely to be tested by the behavior of the jackal compler

#

The goal has been to either switch to sandwichman's compiler once it can compile mintia2 and is itself self-hosting, since he's a compiler guy and it's MUCH better than mine

#

Or if that doesn't pan out i will rewrite jackal compiler to be much nicer after mintia2 reaches UX parity with old mintia

#

Also an amd64 port might be performed that just uses the C backend the jackal compiler already has

#

Someone did a proof of concept of this by managing to get mintia2 to a bsod on amd64

icy bridge
#

Oh right jackal has a C backend, that could be used as another test of xr17032-gcc

twilit smelt
#

It needs changes to be appropriate for kernel dev since that wasn't its original intent

twilit smelt
#

The build tool also isn't currently set up for using something like a cross C toolchain for this

#

it's support for C is limited to what's needed to just build the tools with themselves

#

for boring mostly single threaded userland, on whatever host you're running it on

#

Would need some changes to work but it's doable

dense vigil
#

@icy bridge btw what's the reason you decided to do this for the old mintia instead of the new one?

twilit smelt
#

Probably just that it's usable

dense vigil
#

That could be ported to amd64 too

icy bridge
#

Because new mintia doesn't do anything yet

twilit smelt
#

I was guessing

dense vigil
#

Ah ok

twilit smelt
#

Are you gonna just use ELF on i386 btw

icy bridge
#

Nah I'd have to rewrite all the code that deals with DLLs

#

And have a completely separate build system for i386

#

What I'm going to do is add i386 support to the SDK linker and make it so .S files (not .s) are assembled using GCC instead of the SDK assembler

#

Then only use .S files on i386

twilit smelt
#

makes sense

mortal thunder
icy bridge
#

No?

#

Right now I'm using the same image format that regular mintia1 uses with a single very simple extension that adds addend fields to relocations

mortal thunder
#

I see

icy bridge
#

The build process is preprocessor-dfrttrans-gcc-elfconvert-link

twilit smelt
#

(fyi, XLO already has addends, which are encoded in-place at the spot where the relocation is pointing to in the text section or wherever)

#

(not relevant to this project but for future reference)

icy bridge
#

Ah the ELF REL relocations

mortal thunder
#

to allow relocations to be idempotent?

#

(so you can do them multiple times and not screw up the program)

#

because the space referred to by a rela relocation isn't being read from to determine the new address afaik

icy bridge
mortal thunder
#

I see

twilit smelt
#

i managed to do it by having special relocations that manipulate bitfields in discontiguous spots for certain idioms like the instruction sequence that the LA pseudoinstruction turns into

#

the addend is put together from whats already in those bitfields

#

smushed together

icy bridge
#

For example XR17032 has a HI16 relocation that places the upper 16 bits of the final value in the immediate field of an instruction. If you were to use REL here you'd lose the carry from adding the low 16 bits

lucid umbra
icy bridge
#

You could have multiple instructions between the LUI and the ORI/MOV for example

twilit smelt
#

yeah

#

a lot of the decisions i did around my object files and stuff was basically a priori without looking too much at what conventional toolchains already do

#

or i looked at something weird like PE

#

or the VMS object format

#

(which is unnamed aside from "VMS object format")

#

so there are weirdnesses

#

i believe XLO could be extended to add optional addend fields to relocations where the type indicates that an addend is used, in a backwards compatible way

#

because the relocation tables are never indexed or anything so the entries dont all need to be the same size

icy bridge
#

XLO or XLOFF? Because XLOFF relies on all the relocations having the same, consumer-compile-time-constant size

twilit smelt
#

i used the wrong term, it would be forwards-compatible; it wouldnt break old XLO objects from working with new tooling

icy bridge
#

Ah right

#

Yeah probably

#

But I didn't want to bother with that and it'd be pointless for this project considering the call convention is incompatible with old objects anyway

twilit smelt
#

im just spitballing for the future mintia2 port to amd64 which you might be interested in helping with or you might just do the entire effort yourself in 3.2 days immediately after ux parity with old mintia is met in like a year lol

#

since you seem to be INSANE in the membrane like this

#

porting mintia2 to a new isa is probably a little harder because it has a much higher concentration of highly arch-specific tricks (for performance and whatnot) than old mintia i think

#

things where each new ISA needs its own way to do it

#

f.e. old mintia has one slow kernel mutex codepath written in df so that wouldnt need any effort for a new port, whereas the kernel mutexes in mintia2 have fast noncontended codepaths written in asm for each architecture

#

which use ll/sc (xr17032) or restartable sections (fox32) or CAS (amd64 probably) or ...

#

the numa support also is now another point where theres some arch dependence because it relies on knowing the lay of the land of kernel space, which has varying numbers of valid address bits on diff 64 bit architectures

#

but thats fairly minor its just some constants in a header file

twilit smelt
# icy bridge Ah right

by the way while youre here, i wanted to mention that the "semantics" of physical memory for xrcomputer are going to be changing (in a backwards compatible way)

#

in the sense that theyll still be reported in the firmware's device db structure in the same way and will still be at the same locations in the phys addr space

#

but itll be split into fourths for each of four numa nodes on an "xr/frame" system

#

each node will be a board with up to 64MB of memory and up to 4 CPUs (so a fully loaded xr/frame would have 256MB and 16 CPUs, with the extra 8 CPUs being reported in an extended part of the device db structure im going to tack on to the end, which shouldnt outright break your linux port unless im mistaken, pls inform me if so)

#

and i was planning on simulating numa latency and whatever in the emulator on this basis so that i can test numa support in mintia2

#

its also a realism improvement because 256mb would have been a nutty amount of maximum memory to have on a pizzabox workstation in 1989-1991 whereas 64mb is more reasonable, with the xrstation and xrmp configurations becoming basically a single numa node from an xr/frame from a system software perspective

#

itll directly parallel how the real world AViiON line-up of computers was organized in 1992, thats my reference for realism for numa

#

this is how the device database structure is gonna change probably

image.png
#

the new NodeId field in FwProcessorInfoRecord fits in padding that was already there so that shouldnt break anything

#

and ExtendedProcessors goes at the end so it also shouldnt break anything

#

is this reasonable to you @icy bridge

icy bridge
#

are you sure NodeId is inside what used to be padding?

#

looking at the C definitions used by xrlinux FwProcessorInfoRecord has no tail padding

twilit smelt
#

no im not and actually its not because its used as an array. so instead i will have a separate NodeId array at the end lol

icy bridge
#

alright then yeah that looks good

twilit smelt
#

so itll be like this instead

#

its weird how many workstation computers around that period went out of their way to have circuitry that would detect the configuration of RAM and would remap it at reset time so that it would be physically contiguous starting at 0

#

not all of them but a lot of them did. AViiON for example

#

as near as i can tell its literally just because the ancient unix kernel had assumptions of physical contiguity baked in since pdp11 times

twilit smelt
#

in fairness i wasnt seriously anticipating there to be any OSes other than mine running on it and i figured i could just recompile and break every old image probably and that itd be overly larpy to add tons of reserved space as if anybody else is ever gonna rely on it

#

larping is the whole point of the project but it tended to feel overly presumptuous to do things that were attempts to account for anybody else writing anything for it, or even using it, aside from me

#

lesson learned