MINTIA (not vibecoded) | Operating System Development | Page 13

heady bobcat Jul 17, 2025, 8:37 PM

#

because it seems like a kind of null token (not a null statement)

twilit smelt Jul 17, 2025, 8:37 PM

#

so it might just be doomed to have a misnomer

dense vigil Jul 17, 2025, 8:37 PM

#

you should've just used ; as NOTHING, then it least it wouldnt be as ugly

#

and it would kinda make sense

twilit smelt Jul 17, 2025, 8:37 PM

#

possibly

heady bobcat Jul 17, 2025, 8:37 PM

#

a = MISNOMER b

twilit smelt Jul 17, 2025, 8:37 PM

#

yeah

#

if i called it MISNOMER thatd be a really fun mystery for ppl poking thru mintia2 codebase

#

i guess NOTHING is mysterious enough already

heady bobcat Jul 17, 2025, 8:38 PM

#

you should've just made a word up like a = MOXOF b

twilit smelt Jul 17, 2025, 8:38 PM

#

then people would go "okay so this gets the 'mox' of b. what is a mox" and then they google mox thinking its some arcane terminology theyre unaware of

#

and they find nothing

#

or a porn star or something

heady bobcat Jul 17, 2025, 8:39 PM

#

idfk

twilit smelt Jul 17, 2025, 8:39 PM

#

infy's suggestion is probably the most sensible lol

heady bobcat Jul 17, 2025, 8:39 PM

#

heady bobcat you should've just made a word up like `a = MOXOF b`

I stole this from a @ binder message

twilit smelt Jul 17, 2025, 8:40 PM

#

unfortunately its already called NOTHING and there are probably dozens of NOTHINGs everywhere in multiple repositories that i dont feel like changing

#

theres about 100k lines of jackal already

#

about 50k each in the sdk and mintia2

#

mintia2 will probably be in the 120-200k range when its "done"

#

old mintia was 120k lines (of dragonfruit 🤮 ) and mintia2 will do a lot more

blissful smelt Jul 17, 2025, 8:48 PM

#

mintia
> cloc . --read-lang-def=lang.txt
     637 text files.
     569 unique files.
     451 files ignored.

-----------------------------------------------
Language      files     blank   comment    code
-----------------------------------------------
Dragonfruit     316     28407         0   79260
Assembly         13      2734        84    6822
C/C++ Header    149      1747       289    4405
make             81       267         1     507
Markdown          3        81         0     154
Text              6        18         0      95
DOS Batch         1         2         0       3
-----------------------------------------------
SUM:            569     33256       374   91246
-----------------------------------------------
``` `cloc` might've counted wrong idk

#

maybe ur also counting whitespace lol

#

oh you just said "lines" not "lines of code" 👍

twilit smelt Jul 17, 2025, 8:49 PM

#

i usually use the same metric people use to measure templeos

#

which is a straight up count of text lines including whitespace

#

which is what ppl refer to when they say templeos is 120k lines

#

i count the same way so i get to say mintia is just as big and have it be fair

#

lol

#

last i checked the entire project including all 3 OSes, both sdks, emulator and so on is about 150k-200k sloc

#

yeah its 290kloc counting whitespace so probably 200ksloc is a conservative estimate

#

1/5 of a million lines in 5 years, i guess id get to a million sloc in another 20 years

blissful smelt Jul 17, 2025, 8:54 PM

#

only 700 more years before catching up with linux

#

you got this

twilit smelt Jul 17, 2025, 8:54 PM

#

its funny how its actually achievable that i could create an entire like 1990 tier computer ecosystem by myself in a fraction of my lifetime

#

but then you go like a decade later to y2k and nooooo way

#

its already orders of magnitude outside the realm of feasibility 10 years later

#

a "computer ecosystem" (in the sense of the total sources for everything youd have installed on an average personal machine) went from a million sloc to 80 million sloc in like 10 years

blissful smelt Jul 17, 2025, 8:58 PM

#

what even happened in those ten years

#

I was not alive

twilit smelt Jul 17, 2025, 8:58 PM

#

neither was i

#

i was born in 2002

#

but basically people rapidly shed a bunch of skill issues

#

and the number of software engineers exploded

#

in the 90s

blissful smelt Jul 17, 2025, 8:59 PM

#

what were some major OS advancements

twilit smelt Jul 17, 2025, 8:59 PM

#

do you mean from a home computer perspective or from a bleeding edge perspective

blissful smelt Jul 17, 2025, 8:59 PM

#

home computer stuff

twilit smelt Jul 17, 2025, 8:59 PM

#

they suddenly all got stuff the mainframes had since like 1979

#

so like paging, protected memory, preemptive multitasking, etc

#

modern kernels that look basically like the ones we study today trickled down to home computers for the first time in the 90s

blissful smelt Jul 17, 2025, 9:00 PM

#

twilit smelt so like paging, protected memory, preemptive multitasking, etc

they didn't have that before then?? was it because "home computers" before that were mostly stuff like DOS

#

oh wow I'm just now realizing that lol

twilit smelt Jul 17, 2025, 9:00 PM

#

macs and PCs were both cooperative multitasking, single address space systems

#

that started on processors without paging mmu

blissful smelt Jul 17, 2025, 9:00 PM

#

what happened to the bleeding edge of OSdev then

twilit smelt Jul 17, 2025, 9:01 PM

#

i think scalability was probably the most marked improvement in the 90s

#

people started building truly large machines in the 90s like the type you need NUMA for and this compelled scalability advancements in kernel design

warm pine Jul 17, 2025, 9:01 PM

#

blissful smelt they didn't have that before then?? was it because "home computers" before that ...

the closest to a modern OS on a home computer before then was amigaos

#

the creators knew about and modeled it on systems like unix

#

but an MMU was an unaffordable luxury in a home computer at that time

#

so they compromised and made a system which was an equal to the real OSes in all respects except for no virtual memory

twilit smelt Jul 17, 2025, 9:03 PM

#

twilit smelt i think scalability was probably the most marked improvement in the 90s

most of the modern fancy features that ppl here aspire to were part of any serious kernel by the end of the 80s, like page swapping and memory mapped files and COW fork and so on. the 90s was less new fundamental features and more improved implementation of the old ones

#

oh there is one other thing that compelled some big overhauls and new design tactics

warm pine Jul 17, 2025, 9:05 PM

#

twilit smelt most of the modern fancy features that ppl here aspire to were part of any serio...

i would go as far as to say, we spoke earlier of locking as a fundamental of the design space of a kernel, maybe even prior to in a lot of cases its data structures and algorithms, but almost the only qualitative change in kernel design post 80s was locking (because of need for performant SMP) assuming that position

twilit smelt Jul 17, 2025, 9:05 PM

#

plug and play

warm pine Jul 17, 2025, 9:05 PM

#

oh, that's a good one

twilit smelt Jul 17, 2025, 9:05 PM

#

plug and play caused the two major desktop OS vendors to redesign their IO systems

blissful smelt Jul 17, 2025, 9:05 PM

#

twilit smelt most of the modern fancy features that ppl here aspire to were part of any serio...

what has even happened since then? I mean I'm only really looking to match the kernels of the 70's to mid-80's, but what are kernels today (the 2020's) doing

twilit smelt Jul 17, 2025, 9:05 PM

#

the classic NT IO manager (which mostly assumed a static set of devices from boot much like old VMS) gave way to the Pnp manager

#

and NeXT driverkit gave way to xnu iokit

twilit smelt Jul 17, 2025, 9:06 PM

#

blissful smelt what has even happened *since* then? I mean I'm only really looking to match the...

scalability and security are probably the two biggest areas

#

security might be bigger now

blissful smelt Jul 17, 2025, 9:07 PM

#

memory encryption is pretty recent on consumer hardware I think

balmy zealot Jul 17, 2025, 9:08 PM

#

twilit smelt most of the modern fancy features that ppl here aspire to were part of any serio...

What features could you call characteristic not of the 80s or 90s, but even the 2000s? For hobby systems, I suppose what might be important is much higher automation and standardisation of devices and device drivers?

twilit smelt Jul 17, 2025, 9:08 PM

#

if i see lots of lock free stuff and RCU it makes me think 2000s

#

and strict security features like KPTI

balmy zealot Jul 17, 2025, 9:09 PM

#

twilit smelt security might be bigger now

Oh right, security. Lot of talk about it, but what actually changed in kernels in that regard?

twilit smelt Jul 17, 2025, 9:09 PM

#

thats a good question

#

i dont know as much about it as i should

#

other than the basic mitigations for meltdown and spectre and stuff

#

and ASLR and so on

#

i think Apple's XNU team and chip team are probably the security pioneers right now

#

since they put those both under the same roof they can work together to come up with really interesting security features

#

that require hardware support that doesnt exist with the same robustness on any other platform than apple silicon

blissful smelt Jul 17, 2025, 9:10 PM

#

twilit smelt i think Apple's XNU team and chip team are probably the security pioneers right ...

their disk controller handles encryption at the hardware level right

#

pretty neat

twilit smelt Jul 17, 2025, 9:11 PM

#

you could probably call linux scalability pioneers

#

idk if NT is pioneering anything other than being pretty

#

its keeping up at least

#

maybe memory management pioneers. NT's memory compression and stuff are really good

#

and they continue to abuse the page tables in unholy ways nobody else dreams of

#

continuing the vms tradition

balmy zealot Jul 17, 2025, 9:13 PM

#

twilit smelt and ASLR and so on

Meltdown still seems relevant for Intel systems, I suppose. Fortunately, Apple's moving away from them these days. Only recently learned about ASLR implementation in the kernel, surprised that something conceptually simple (?) requires such clever tricks to work effectively. It involves compiler tricks and probably more, right?

twilit smelt Jul 17, 2025, 9:14 PM

#

twilit smelt maybe memory management pioneers. NT's memory compression and stuff are really g...

they also employ other stuff like page deduplication that allegedly works well whereas if it exists at all in the other two its like an experimental thing

#

basically they have a giga memory management autist (affectionate) named landy wang who has been hacking away at their vmm for like 28 years straight

digital pivot Jul 17, 2025, 9:15 PM

#

twilit smelt i think Apple's XNU team and chip team are probably the security pioneers right ...

nt is very good as well, they first got securekernel thing and hyperv trust levels and stuff

twilit smelt Jul 17, 2025, 9:15 PM

#

the dedication of 1 guy for that long can really go a long way i think

digital pivot Jul 17, 2025, 9:16 PM

#

ig they try to partner with Intel and their security hw stuff like enclaves

dense vigil Jul 17, 2025, 9:16 PM

#

twilit smelt they also employ other stuff like page deduplication that allegedly works well w...

u mean this https://en.wikipedia.org/wiki/Kernel_same-page_merging?

Kernel same-page merging

In computing, kernel same-page merging (KSM), also known as kernel shared memory, memory merging, memory deduplication, and page deduplication is a kernel feature that makes it possible for a hypervisor system to share memory pages that have identical contents between multiple processes or virtualized guests. While not directly linked, Kernel-ba...

#

?

twilit smelt Jul 17, 2025, 9:17 PM

#

dense vigil u mean this https://en.wikipedia.org/wiki/Kernel_same-page_merging?

yeah but NT's implementation is apparently Better but dont ask me for details cuz i dont remember if i ever knew them

#

it uses some clever hashed avl tree thing

dense vigil Jul 17, 2025, 9:18 PM

#

oh ok

#

the one in linux is definitely not an experimental thing

#

its pretty stable iirc

twilit smelt Jul 17, 2025, 9:19 PM

#

dense vigil its pretty stable iirc

well im not

#

did you ever think of that

#

DID YOU EVER THINK OF THAT!!!!!!!!!!!!

dense vigil Jul 17, 2025, 9:19 PM

#

wdym you're the most stable person i know

digital pivot Jul 17, 2025, 9:20 PM

#

except for rcu like stuff nt has very rich set of synchronization primitives i believe

twilit smelt Jul 17, 2025, 9:22 PM

#

dense vigil wdym you're the most stable person i know

is this ironic

dense vigil Jul 17, 2025, 9:23 PM

#

irony is in the eye of the beholder meme

balmy zealot Jul 17, 2025, 9:29 PM

#

What about CPU scheduling in particular? Any notable innovations or pioneering efforts there?

twilit smelt Jul 17, 2025, 9:31 PM

#

dense vigil irony is in the eye of the beholder <:meme:575445094589661215>

itd be funny if you were serious because the quantities of bpd in my personal life are off the charts

#

successfully redirecting all my insanity away from the osdev community to trick people into thinking im employable

dense vigil Jul 17, 2025, 9:32 PM

#

well you definitely have the ability to consistently work on one thing which is commendable

twilit smelt Jul 17, 2025, 9:32 PM

#

thats what i want you to think

twilit smelt Jul 17, 2025, 9:33 PM

#

balmy zealot What about CPU scheduling in particular? Any notable innovations or pioneering ...

probably

#

i dont hear much about it so i imagine its nothing revolutionary but theres always room for more schedulers in the world

#

and refinements to old ones

#

i guess the windows processor group thing is interesting

#

but thats not really NEW

#

also probably isnt unique

#

they also made more scheduler changes lately

craggy spire Jul 17, 2025, 9:40 PM

#

Unique I don't know but they modify and tune it a lot

twilit smelt Jul 17, 2025, 9:40 PM

#

but those have had mixed reception

craggy spire Jul 17, 2025, 9:40 PM

#

As you know user threads can normally get a boost up to 16, but in recent versions I noticed that main thread of interactive apps has priority 21 and it's not something temporary

twilit smelt Jul 17, 2025, 9:41 PM

#

twilit smelt but those have had mixed reception

not to speak badly of our friend, just citing what ive seen

#

i saw some tech bro gamedev loser on twitter going like "WHATEVER IDIOT IS FUCKING WITH THE WINDOWS SCHEDULER NEEDS TO BE SHOT" or something because they were getting weird behavior and latency spikes on the most recent versions

#

i thought "no he doesnt..... i like that guy....."

#

rule 1 of the internet is that anything you say about someone will be read by the person you said it about

craggy spire Jul 17, 2025, 9:44 PM

#

The average user is unlikely to notice all these changes as a whole, but will be dissatisfied by default

twilit smelt Jul 17, 2025, 9:44 PM

#

rule 2 is that itll be read at least by someone who knows that person

twilit smelt Jul 17, 2025, 9:46 PM

#

twilit smelt i guess the windows processor group thing is interesting

basically there are two extremes:

big scheduler lock, with a single global ready queue shared between all processors.
pros: say you have N available processors. under this scheme, the top N highest priority threads are the ones currently running on those N processors, in perfect balance always
cons: high contention, scales poorly to many processors
per-processor locks, with per-processor ready queues
pros: contention basically disappears
cons: unavoidable and frustrating (but functionally harmless) races are introduced that can cause long term imbalances and starvation of high priority threads, which requires explicit rebalancing

#

NT people briefly switched from #1 to #2 in win 2k3 i believe (correct me if im wrong)

#

but then they asked

#

https://tenor.com/view/why-dont-we-have-both-both-celebrate-happy-gif-4997058

Tenor

#

and basically they group processors together such that groups of i think 4? will all share a single ready queue

#

so the members of these groups are always balanced among themselves

digital pivot Jul 17, 2025, 9:48 PM

#

this is so simple and clever

twilit smelt Jul 17, 2025, 9:48 PM

#

and the global imbalances are reduced

#

they still need explicit global rebalancing between the groups sometimes but its not as critical

digital pivot Jul 17, 2025, 9:49 PM

#

twilit smelt i saw some tech bro gamedev loser on twitter going like "WHATEVER IDIOT IS FUCKI...

the ke guy is super smart being able to solve problems in such elegant manner

twilit smelt Jul 17, 2025, 9:50 PM

#

twilit smelt and basically they group processors together such that groups of i think 4? will...

this probably came from the observation that option #1 i mentioned scales pretty much fine up until like 4-8 processors

#

so theres no real reason to chunk it up as fine grained as individual processors

#

you can chunk it up into groups of 4-8

#

instead

#

and get the scalability without nearly as big of a sacrifice of balance

#

so on a big 128 processor machine they might have like 32 ready queues with their own locks

#

i imagine this also makes the rebalancing algorithm more efficient because there are vastly fewer possible pairs of ready queues to have grave imbalances between

dense vigil Jul 17, 2025, 9:52 PM

#

is that a thing to this day?

twilit smelt Jul 17, 2025, 9:52 PM

#

yes

dense vigil Jul 17, 2025, 9:52 PM

#

i imagine its more complex with p/e cores etc

twilit smelt Jul 17, 2025, 9:52 PM

#

well yeah

#

but im 90% sure they still use this basic design

dense vigil Jul 17, 2025, 9:53 PM

#

thats interesting

blissful smelt Jul 17, 2025, 9:57 PM

#

twilit smelt this probably came from the observation that option #1 i mentioned scales pretty...

what about NUMA performance? if it's grouping multiple cores into one queue

twilit smelt Jul 17, 2025, 9:57 PM

#

they would just never have processors from different numa nodes in the same group

#

the groups align with the topology im p sure

#

so like hyperthreads on the same core will be in the same group

#

and so on

#

if they do work stealing between groups theyd do it in increasing topological distance

craggy spire Jul 17, 2025, 9:59 PM

#

twilit smelt i saw some tech bro gamedev loser on twitter going like "WHATEVER IDIOT IS FUCKI...

I don't know if games can use the benefits of CPU Partitions, but I think this new feature would be useful for them, as it allows them to avoid interference even from DPCs and improve latency in general

twilit smelt Jul 17, 2025, 10:00 PM

#

i just found it really comical that our friend was telling us about some of the new scheduler features on discord and right after that i see an explosion of outrage on twitter from gamedev influencers

#

about the exact thing i just heard about personally

digital pivot Jul 17, 2025, 10:01 PM

#

what was it particularly

twilit smelt Jul 17, 2025, 10:01 PM

#

they were just complaining about nebulous latency spikes that didnt happen before and suddenly were happening

#

i dont know what the root cause was or if it was even a big issue or just them being whiny and whipping up anti-microsoft hysteria among dumb cs kids for clicks

#

if it was a big issue im sure the smart people in the kernel team noticed quickly and were like ah oops that policy didnt work as well as we hoped / oops theres a bug in the implementation of this new policy causing latency spikes / whatever

#

and changed it in the next update

#

these things are way too fluid for people to be that outraged and whiny and disingenuous on twitter

#

especially now they can change it test it and push it out to everybody in the world in like a few weeks

#

even if it was the worst scheduler regression in history its still not THAT big of a deal lol

#

WAHHHH MY VIDO GAME RUN AT 80 FPS INSTEAD OF 100 FPS

#

IM A BIG OVERGROWN MAN BABY WHO LIKES JINGLING KEYS IN FRONT OF MY FACE ALL DAY

#

WAHHHH

#

etc

#

i hate gamers so much dude

#

especially dumbass gamedev idiots manufacturing crack for said manbabies and thinking thats respectable enough that they get to rudely throw stones at kernel people

#

who have real things to worry about like database server performance

#

mintia2's scheduler will dynamically detect if youre playing a video game and will purposefully starve the shit out of all of its threads JUST to piss you off

blissful smelt Jul 17, 2025, 10:14 PM

#

strstr the name of the process with a list of known mintia2 games

#

very good design decision

tame phoenix Jul 17, 2025, 10:15 PM

#

that sounds like the average anticheat lol... "we will just scan memory for strings that are in cheat programs whats the worst that can happen"

blissful smelt Jul 17, 2025, 10:18 PM

#

just write a rootkit 😃

twilit smelt Jul 17, 2025, 10:18 PM

#

mintia2 source leak:

IF RtlCompareString ( &thread^.Process^.Name[0], "valorant.exe" ) == 0 THEN
  thread^.Priority = IDLE_PRIORITY
ELSEIF RtlCompareString ( &thread^.Process^.Name[0], "genshinimpact.exe" ) == 0 THEN
  KeTerminateProcess ( thread^.Process )

blissful smelt Jul 17, 2025, 10:19 PM

#

twilit smelt mintia2 source leak: ``` IF RtlCompareString ( &thread^.Process^.Name[0], "valor...

use rust syntax highlighting with discord for jackal btw (if you didnt already know)

#

it looks better

twilit smelt Jul 17, 2025, 10:20 PM

#

i like it without the highlighting

#

makes it look ancient

#

carved in stone

#

all VPPERCASE like that

tame phoenix Jul 17, 2025, 10:36 PM

#

next youll be writing your comments in latin

dapper mulch Jul 18, 2025, 12:06 AM

#

When I see jackal code, I'm getting tempted to ... no, not kill myself, 😁 I want to write my own C compiler with black jack and ladies for riscv64. Would be a total ~~garbag~~ blast. 😁 But then I remember, that I don't like that architecture that much.

mortal thunder Jul 18, 2025, 5:52 AM

#

blissful smelt ``` mintia > cloc . --read-lang-def=lang.txt 637 text files. 569 uniqu...

cloc doesn't do a sum of this so its 90k + 30k = 120k, he counts blank lines too i guess

#

nvm it was answered sorry

mortal thunder Jul 18, 2025, 6:00 AM

#

twilit smelt mintia2 source leak: ``` IF RtlCompareString ( &thread^.Process^.Name[0], "valor...

IF RtlCompareString( &thread^.Process^.Name[0], "genshinimpact.exe") == 0 THEN
    Status = OsUnlinkFile( &thread^.Process^.ImagePath )
    IF Status != STATUS_SUCCESS THEN
        OsSecretCrashDontUse()
    END
END

dense vigil Jul 18, 2025, 8:47 AM

#

dapper mulch When I see jackal code, I'm getting tempted to ... no, not kill myself, 😁 I wan...

Lol

hybrid condor Jul 18, 2025, 9:25 AM

#

twilit smelt so theres no real reason to chunk it up as fine grained as individual processors

How does this fit in with CPU affinity if they support it?

hybrid condor Jul 18, 2025, 9:27 AM

#

twilit smelt so like hyperthreads on the same core will be in the same group

Do you know if they prefer to make smaller groups if the number of CPUs is not divisible by 4?

safe hollow Jul 18, 2025, 1:11 PM

#

hybrid condor Do you know if they prefer to make smaller groups if the number of CPUs is not d...

you can just use whatever hardware partitioning makes sense, like clusters with shared L2

harsh flame Jul 18, 2025, 5:58 PM

#

craggy spire The average user is unlikely to notice all these changes as a whole, but will be...

This sounds improbable, as they only release major updates after thorough testing and running the code millions of times. I know very few scheduler decisions, but they have a lot of expertise in dealing with them and it is unlikely that they would allow a major regression.

twilit smelt Jul 18, 2025, 6:15 PM

#

hybrid condor How does this fit in with CPU affinity if they support it?

I'm unsure

twilit smelt Jul 18, 2025, 6:15 PM

#

hybrid condor Do you know if they prefer to make smaller groups if the number of CPUs is not d...

Dunno but it's probably based on the topology like staticsaga said

twilit smelt Jul 18, 2025, 6:16 PM

#

twilit smelt I'm unsure

iirc they still have per processor ready queues which are specifically for affinitized threads or something but don't quote me

twilit smelt Jul 19, 2025, 5:08 AM

#

@blissful smelt it does seem that my SwapNext link in my thread struct is redundant and i can eliminate it and roll its two uses (thread stack outswapping & reaper list) into the ReadyEntry

#

thx for giving me the idea to check

#

i probably did it because it felt wrong using the ReadyEntry for that while the thread was still alive (during the brief period before it switches away from its context for the last time) but its not like it ever gets enqueued to a ready queue again

blissful smelt Jul 19, 2025, 5:09 AM

#

twilit smelt <@710548131992961074> it does seem that my SwapNext link in my thread struct is ...

WOOHOO!! micro-optimization!! saved a whole 8 bytes let's gooooo

twilit smelt Jul 19, 2025, 5:09 AM

#

4 bytes

#

32 bit

blissful smelt Jul 19, 2025, 5:09 AM

#

o ya i forgor

#

FOUR BYTES WOOOO

blissful smelt Jul 19, 2025, 5:10 AM

#

twilit smelt <@710548131992961074> it does seem that my SwapNext link in my thread struct is ...

did you also change the name of ReadyEntry or is that too much work

#

maybe leave a comment or something
(definitely not because I read your code)

twilit smelt Jul 19, 2025, 5:50 AM

#

blissful smelt did you also change the name of ReadyEntry or is that too much work

i didnt change the name

#

i did add a comment

#

also i changed the number of maximum global balance attempts to be the number of possible pairs between processors cuz thats the max number of imbalances

#

plus one for good measure

blissful smelt Jul 19, 2025, 5:53 AM

#

twilit smelt also i changed the number of maximum global balance attempts to be the number of...

i had the same approach and there was a bit of undesirable lock contention and I found that allowing 1/4th of the cores to do rebalances whilst maintaining a minimum of at least 1 core works great, but your system is probably wildly different

twilit smelt Jul 19, 2025, 5:54 AM

#

only processor zero does global rebalances and it only does it once per second

#

when a processor goes idle it also tries to steal some threads from the most loaded processor before it switches to the idle thread

blissful smelt Jul 19, 2025, 5:55 AM

#

ohh that second part is smart, I should probably do aggressive worksteals on idle

mortal thunder Jul 19, 2025, 6:25 AM

#

twilit smelt 4 bytes

you're going to port this to a 64-bit architecture one day

#

according to your plans

twilit smelt Jul 19, 2025, 6:28 AM

#

mortal thunder you're going to port this to a 64-bit architecture one day

ya but it doesnt run on one today

mortal thunder Jul 19, 2025, 6:29 AM

#

sure

#

not right now

#

but when it does this 4 byte save will manifest as an 8 byte save :)

twilit smelt Jul 19, 2025, 10:33 AM

#

is this cracked or wacked

#

also this

dense vigil Jul 19, 2025, 10:42 AM

#

me when arch without ffs

twilit smelt Jul 19, 2025, 10:42 AM

#

would ffs even work for the second one

#

circular search of a bit set starting from a particular index

dense vigil Jul 19, 2025, 10:42 AM

#

not directly

#

but with a combination of other bit helpers probably

#

or as a loop

twilit smelt Jul 19, 2025, 10:43 AM

#

i had to add a ROR operator to jackal for that

#

u can see it doesnt even have syntax highlighting

#

hot off the press

dense vigil Jul 19, 2025, 10:43 AM

#

lol

#

what does ROR do exactly

twilit smelt Jul 19, 2025, 10:43 AM

#

rotate right

#

i checked to make sure its masked off to the low 5 bits to avoid doing k = (k + 8) & 31 on all planned architectures, i can just do k += 8

dense vigil Jul 19, 2025, 10:44 AM

#

ah

twilit smelt Jul 19, 2025, 10:44 AM

#

so i can use a rotation amount >= 32 bits on those arches and its fine

#

i found

twilit smelt Jul 19, 2025, 10:45 AM

#

twilit smelt i checked to make sure its masked off to the low 5 bits to avoid doing `k = (k +...

its pretty typical to allow that because this is a common thing to want to avoid

#

the extra instruction from masking off a rotation amount

teal trench Jul 19, 2025, 10:57 AM

#

dense vigil me when arch without `ffs`

you'd need find last set here

#

afaict this is just (find last set(x) / 8)?

teal trench Jul 19, 2025, 10:58 AM

#

teal trench you'd need find last set here

so 31 - clz ig

dense vigil Jul 19, 2025, 10:58 AM

#

ye

twilit smelt Jul 19, 2025, 11:01 AM

#

teal trench you'd need find last set here

how would i do the circular search starting from given bit index

dense vigil Jul 19, 2025, 11:02 AM

#

do u have a bit ring buffer?

twilit smelt Jul 19, 2025, 11:02 AM

#

its a calendar queue

dense vigil Jul 19, 2025, 11:02 AM

#

lol

#

never heard of it

teal trench Jul 19, 2025, 11:02 AM

#

hmm

twilit smelt Jul 19, 2025, 11:02 AM

#

the timeshared thread queue is a calendar queue

teal trench Jul 19, 2025, 11:02 AM

#

i only looked at the binary search to find byte part

twilit smelt Jul 19, 2025, 11:03 AM

#

thats the first time ive ever had to use rotate right

#

it was actually broken in my emulator

#

worked perfectly on fox32 tho

mortal thunder Jul 19, 2025, 11:06 AM

#

twilit smelt i had to add a ROR operator to jackal for that

what would you have done in C since it doesnt have ROR

twilit smelt Jul 19, 2025, 11:06 AM

#

C compilers recognize ROR constructions

#

and emit ROR

teal trench Jul 19, 2025, 11:06 AM

#

ror(readybits, k) :^)

twilit smelt Jul 19, 2025, 11:06 AM

#

so id do that

twilit smelt Jul 19, 2025, 11:06 AM

#

teal trench `ror(readybits, k)` :^)

is this a real C thing

teal trench Jul 19, 2025, 11:06 AM

#

c23 adds it but under a different name iirc

twilit smelt Jul 19, 2025, 11:07 AM

#

twilit smelt C compilers recognize ROR constructions

also this

teal trench Jul 19, 2025, 11:07 AM

#

stdc_rotate_right

twilit smelt Jul 19, 2025, 11:07 AM

#

theres a common one-liner way to implement ROR with shifts and masks and modern C compilers can all recognize this and emit ROR

#

i googled this at some point

teal trench Jul 19, 2025, 11:10 AM

#

twilit smelt also this

what does this do btw? starting from k find the next set bit until k-1 with wraparound?

twilit smelt Jul 19, 2025, 11:10 AM

#

yeah

teal trench Jul 19, 2025, 11:12 AM

#

something like this i think works? ```
top = readybits >> k
bottom = readybits & ((1 << k) - 1)
k = top ? ffs(top) + k : ffs(bottom)

twilit smelt Jul 19, 2025, 11:12 AM

#

yeah probably

teal trench Jul 19, 2025, 11:12 AM

#

alternatively ig a second mask instead of a shift to get rid of the + k

#

but the branch has to be there since commonly ffs on 0 is undefined

#

actually even if it wasn't at best you could do ffs_into(top, &k) || ffs_into(bottom, &k) assuming it returns 0 for failure, nonzero for success

#

so still a branch

twilit smelt Jul 19, 2025, 12:02 PM

#

twilit smelt also this

im proud of this one this might be the fastest way to do this on an architecture w/o ffs clz whatever

#

esp cuz i replaced a naive one-bit-at-a-time loop with it

heady bobcat Jul 19, 2025, 12:04 PM

#

twilit smelt is this cracked or wacked

There is always room for wackier shit (C but can easily be translated to jackal):

uint32_t i = readybits;
i |= i >> 16;
i |= i >> 8;
i |= i >> 4;
i |= i >> 2;
i |= i >> 1;
i = popcount(i);

twilit smelt Jul 19, 2025, 12:04 PM

#

i dont have popcnt

heady bobcat Jul 19, 2025, 12:04 PM

#

ah

twilit smelt Jul 19, 2025, 12:05 PM

#

i have 60 instructions

#

popcnt is not one

devout geyser Jul 19, 2025, 12:07 PM

#

heady bobcat There is always room for wackier shit (C but can easily be translated to jackal)...

popcnt scares me after seeing intel's c fallback

COMPILER_RT_ABI si_int
__popcountdi2(di_int a)
{
    du_int x2 = (du_int)a;
    x2 = x2 - ((x2 >> 1) & 0x5555555555555555uLL);
    /* Every 2 bits holds the sum of every pair of bits (32) */
    x2 = ((x2 >> 2) & 0x3333333333333333uLL) + (x2 & 0x3333333333333333uLL);
    /* Every 4 bits holds the sum of every 4-set of bits (3 significant bits) (16) */
    x2 = (x2 + (x2 >> 4)) & 0x0F0F0F0F0F0F0F0FuLL;
    /* Every 8 bits holds the sum of every 8-set of bits (4 significant bits) (8) */
    su_int x = (su_int)(x2 + (x2 >> 32));
    /* The lower 32 bits hold four 16 bit sums (5 significant bits). */
    /*   Upper 32 bits are garbage */
    x = x + (x >> 16);
    /* The lower 16 bits hold two 32 bit sums (6 significant bits). */
    /*   Upper 16 bits are garbage */
    return (x + (x >> 8)) & 0x0000007F;  /* (7 significant bits) */
}

raven drift Jul 19, 2025, 12:17 PM

#

this is just divide and conquer

#

with the comments it's not hard to understand imho

#

if you see it w/o comments it looks way more arcane meme

acoustic sparrow Jul 19, 2025, 12:23 PM

#

twilit smelt C compilers recognize ROR constructions

i think llvm recognizes a funneled shift (so (x << k) | (y >> (32 - k)) == @llvm.fshl.i32(x, y, k)) and ror/rol is an isel-time special case of that

devout geyser Jul 19, 2025, 1:48 PM

#

raven drift if you see it w/o comments it looks way more arcane <:meme:575445094589661215>

true lol

twilit smelt Jul 20, 2025, 1:24 AM

#

FN KeAreApcsDeliverable () : UWORD

    // Return whether APCs are deliverable.

    RETURN KiCurrentIpl () == KI_IPL_LOW
END

KeAreApcsDeliverable:
.global KeAreApcsDeliverable
    subi t0, zero, 16134
    mov t0, byte [t0]
    subi a3, t0, 0
    slti a3, a3, 1
    ret

#

strange codegen here

#

i figured out its cuz my compiler is really stupid and isnt recognizing the subi a3, t0, 0 as being just a mov

#

not seeing the mov, its not applying hinting

#

so it doesnt assign these to the same register a3 and elide it

teal trench Jul 20, 2025, 1:28 AM

#

what does slti a,b,c do? a = b < c?

twilit smelt Jul 20, 2025, 1:28 AM

#

yeah

#

its doing a logical NOT there

teal trench Jul 20, 2025, 1:31 AM

#

ah

#

is KI_IPL_LOW 0?

twilit smelt Jul 20, 2025, 1:32 AM

#

yeah

teal trench Jul 20, 2025, 1:32 AM

#

ah i see

twilit smelt Jul 20, 2025, 1:32 AM

#

this is dumb codegen but ik how to fix it

teal trench Jul 20, 2025, 1:32 AM

#

well you could get rid of the subi and do slti a3, t0, 1 no?

twilit smelt Jul 20, 2025, 1:32 AM

#

i can add a table to the xr17032 backend indicating which instructions are nop if the second operand is zero

#

and convert those to movs

#

and then the regalloc will take care of this

#

assign these all to a3 and elide the resulting mov a3, a3

twilit smelt Jul 20, 2025, 1:34 AM

#

teal trench well you could get rid of the subi and do `slti a3, t0, 1` no?

yes

#

thats why this is silly

twilit smelt Jul 20, 2025, 2:17 AM

#

ok thats a lot better

#

i wonder what else this helped with

#

after:

$ ./newsdk/bin/xrlink.exe info mintia2/BuildRoot/XRstation-mpfre/mintia/mpx.sys
DateStamp     687C5186
Architecture  xr17032
Head Length   3936
Flags         STRIPPED

Sections:
text:
  Size     65076 bytes
  Virtual  65076 bytes
  Address  0x80010000
  Offset   0x00000F60
  Flags    CODE | MAP

data:
  Size     16936 bytes
  Virtual  31760 bytes
  Address  0x80020000
  Offset   0x00010D94
  Flags    MAP

INITtext:
  Size     9104 bytes
  Virtual  9104 bytes
  Address  0x80028000
  Offset   0x00014FBC
  Flags    CODE | MAP

PAGEtext:
  Size     35252 bytes
  Virtual  35252 bytes
  Address  0x8002B000
  Offset   0x0001734C
  Flags    CODE | MAP | PAGED

#

now i will revert the compiler changes and see how many bytes different and divide that by 4 and thatll be how many instructions it saved lol

#

$ ./newsdk/bin/xrlink.exe info mintia2/BuildRoot/XRstation-mpfre/mintia/mpx.sys
DateStamp     687C528F
Architecture  xr17032
Head Length   3936
Flags         STRIPPED

Sections:
text:
  Size     65080 bytes
  Virtual  65080 bytes
  Address  0x80010000
  Offset   0x00000F60
  Flags    CODE | MAP

data:
  Size     16936 bytes
  Virtual  31760 bytes
  Address  0x80020000
  Offset   0x00010D98
  Flags    MAP

INITtext:
  Size     9104 bytes
  Virtual  9104 bytes
  Address  0x80028000
  Offset   0x00014FC0
  Flags    CODE | MAP

PAGEtext:
  Size     35252 bytes
  Virtual  35252 bytes
  Address  0x8002B000
  Offset   0x00017350
  Flags    CODE | MAP | PAGED

#

text section changed by 4 bytes

#

it literally only helped with this one single place

#

thats beast mode

twilit smelt Jul 20, 2025, 3:01 AM

#

its really funny that this

FN KeAreApcsDeliverable () : UWORD
    // Return whether APCs are deliverable.

    RETURN KiCurrentIpl () == KI_IPL_LOW
END

compiles to basically

return *(uint8_t*)(0xFFFFC0FA) < 1;

#

fucked up

#

ive achieved incomprehensible compiler output

blissful smelt Jul 20, 2025, 3:03 AM

#

twilit smelt ive achieved incomprehensible compiler output

llvm, hire this man right now

blissful smelt Jul 20, 2025, 3:03 AM

#

twilit smelt ``` $ ./newsdk/bin/xrlink.exe info mintia2/BuildRoot/XRstation-mpfre/mintia/mpx....

when will jackal compiler get mintia specific optimizations to save a few bytes here and there

twilit smelt Jul 20, 2025, 3:04 AM

#

theres already mintiaisms baked into the toolchain

blissful smelt Jul 20, 2025, 3:04 AM

#

patching gcc so your binary is 2 bytes smaller moment

twilit smelt Jul 20, 2025, 3:04 AM

#

the executable format has a section flag defined for pageability

#

specifically for the mintia kernel

#

and modules

blissful smelt Jul 20, 2025, 3:04 AM

#

twilit smelt the executable format has a section flag defined for pageability

how would this actually work though

twilit smelt Jul 20, 2025, 3:04 AM

#

wdym

blissful smelt Jul 20, 2025, 3:04 AM

#

bootloader?

#

like why would this help with anything

twilit smelt Jul 20, 2025, 3:05 AM

#

by that i meant swappability

blissful smelt Jul 20, 2025, 3:05 AM

#

if it's just a section flag

twilit smelt Jul 20, 2025, 3:05 AM

#

whether it can be swapped to disk

blissful smelt Jul 20, 2025, 3:05 AM

#

ohhhh

twilit smelt Jul 20, 2025, 3:05 AM

#

itll be a nop for userspace modules (which are always swappable)

blissful smelt Jul 20, 2025, 3:05 AM

#

makes more sense 👍

twilit smelt Jul 20, 2025, 3:05 AM

#

but determines whether the section's pages can be swapped to disk for kernel modules

blissful smelt Jul 20, 2025, 3:05 AM

#

so you can swap out parts of kernel code at runtime?

#

very neat

twilit smelt Jul 20, 2025, 3:05 AM

#

old mintia could

#

new mintia cant bc it doesnt have swap yet

#

but it will also

blissful smelt Jul 20, 2025, 3:06 AM

#

how would I do this with standard ELF

#

or is that not possible

twilit smelt Jul 20, 2025, 3:06 AM

#

you could just define a name for sections that are pageable

blissful smelt Jul 20, 2025, 3:07 AM

#

does mintia use some way of immediately unmapping kernel code pages that are touched once at boot and never again? like I have custom sections for PCI device driver discovery and initialization and I could probably unmap those and free the memory

#

sounds like a cool idea

twilit smelt Jul 20, 2025, 3:07 AM

#

yes

twilit smelt Jul 20, 2025, 3:08 AM

#

twilit smelt ``` $ ./newsdk/bin/xrlink.exe info mintia2/BuildRoot/XRstation-mpfre/mintia/mpx....

thats what the INITtext section here is

#

mpx.sys is the mintia kernel

#

stands for "mintia parallel eXecutive" cuz i felt like larping

twilit smelt Jul 20, 2025, 3:11 AM

#

twilit smelt yes

for the record having a section of initialization code/data that gets freed after boot is relatively common

#

linux also has it

#

i added a preprocessor directive that lets me specify the section the following symbol will go in

#

you can do similar stuff but uglier with C pragmas and other stuff lol

blissful smelt Jul 20, 2025, 3:21 AM

#

twilit smelt i added a preprocessor directive that lets me specify the section the following ...

do you spawn all these things as work items? sounds like a good idea

#

I should probably parallelize my device initialization thanks for the reminder

twilit smelt Jul 20, 2025, 3:23 AM

#

blissful smelt do you spawn all these things as work items? sounds like a good idea

it would be a bad idea to get the scheduler involved for every object deletion

#

this is for object deletions that couldnt occur as soon as the refcount hit zero

#

because the thread was executing an APC or was holding a lock required for the object to be deleted (which would lead to self-deadlock)

blissful smelt Jul 20, 2025, 3:24 AM

#

twilit smelt it would be a bad idea to get the scheduler involved for every object deletion

so "work item" here is what then? is it an item that a worker thread picks up?

twilit smelt Jul 20, 2025, 3:24 AM

#

yeah

#

theres a pool of worker threads waiting for work items all the time

blissful smelt Jul 20, 2025, 3:25 AM

#

wait I'm working on this rn, what is your worker thread demand spawning policy?

twilit smelt Jul 20, 2025, 3:25 AM

#

none yet

blissful smelt Jul 20, 2025, 3:25 AM

#

I currently spawn one permanent worker thread per core and I'm tryna work out on-demand spawning

#

oh ok 👍

#

I was thinking that I could just reap worker threads after 5 minutes of no activity, which should be OK

blissful smelt Jul 20, 2025, 3:26 AM

#

twilit smelt none yet

how do your item pools where worker threads pick things to run from work? I just use ringbuffers that can hold up to 512 items each, and have one pool per core and a set of workers on each core

#

ringbuffers avoid allocations after the initialization

twilit smelt Jul 20, 2025, 3:27 AM

#

i have a "balanced queue" which internally just uses a linked list

#

but one other property is that it wont wake up a thread if theres a work item available if as many threads as there are cores are already running

#

because thats like maximum parallelism already

#

when one of them blocks (for any reason), another worker thread can then wake up

blissful smelt Jul 20, 2025, 3:28 AM

#

twilit smelt but one other property is that it wont wake up a thread if theres a work item av...

how do you submit things to the queue without an allocation

#

since you use a linked list

twilit smelt Jul 20, 2025, 3:28 AM

#

i just use a single statically allocated work item for thread and object reaping

#

well, one each

blissful smelt Jul 20, 2025, 3:29 AM

#

twilit smelt i have a "balanced queue" which internally just uses a linked list

but if you submit a new work item, doesn't it have to get allocated somewhere to get added to the list?

#

I'm a wee bit confused

twilit smelt Jul 20, 2025, 3:29 AM

#

the list links are internal

#

to the work item

blissful smelt Jul 20, 2025, 3:30 AM

#

twilit smelt to the work item

but does the work item still create a list node, or is it something like a static struct, where a function marked as a work item has a respective struct

twilit smelt Jul 20, 2025, 3:30 AM

#

STRUCT ExWorkItem
    QueueEntry : RtlListEntry,
    Routine : ExWorkItemF,
    Context1 : UWORD,
    Context2 : UWORD,
END

#

this is a work item

blissful smelt Jul 20, 2025, 3:31 AM

#

twilit smelt ``` STRUCT ExWorkItem QueueEntry : RtlListEntry, Routine : ExWorkItemF, ...

so I'm assuming that a function marked as a work item has a static struct like this associated with it? what happens when you add the same work item to the pool after it's already been added?

twilit smelt Jul 20, 2025, 3:32 AM

#

it breaks, so you dont do that

#

for thread reaping for example i atomically enqueue threads to a list of threads pending reaping

#

if the list was empty, i enqueue the work item

#

this guarantees the work item wont be double enqueued because only the work item pops the threads off the list

#

and by then it has been dequeued and is executing

#

same for object reaping

blissful smelt Jul 20, 2025, 3:33 AM

#

twilit smelt it breaks, so you dont do that

what if you have something like a call to free then? does that just mean that you have to add a free to the pool, you can only do it one at a time?

I went with a ringbuffer design because I had a lot of places where I wanted to defer frees to other cores and this allowed me to make multiple ringbuffer entries to the same function

twilit smelt Jul 20, 2025, 3:34 AM

#

blissful smelt what if you have something like a call to `free` then? does that just mean that ...

wdym

#

i enqueue the thing to be freed to a list atomically

#

like i mentioned

blissful smelt Jul 20, 2025, 3:34 AM

#

ohhh yeah that's how you handle frees ya

twilit smelt Jul 20, 2025, 3:34 AM

#

if the list was empty i enqueue the work item

#

i have this little abstraction i call a custody list for doing that

#

which is just an atomic singly linked list enqueued with cmpxchg

#

https://github.com/xrarch/mintia2/blob/main/OS/Executive/Ke/KeCustodyList.jkl

#

if it was empty it calls a callback

#

the "activation routine"

#

which for thread and object reaping just enqueues the work item

blissful smelt Jul 20, 2025, 3:37 AM

#

twilit smelt wdym

ah ok I didn't think of having a setup like that with a list of objects to free

basically, my rationale behind making it a ringbuffer is that I have a common pattern in ISRs

free(thing1);
free(thing2);``` and I need to defer both of those as events in the event pools because it's a bad idea to do those frees from within the ISR 

so in this case I would need to allow my event pools to permit duplicate events with different arguments or something 

so do you just avoid this problem entirely by using your object deletion list system and having a singular work item to handle that?

twilit smelt Jul 20, 2025, 3:38 AM

#

yeah

blissful smelt Jul 20, 2025, 3:39 AM

#

hmmm ok thanks then, I'll maybe consider borrowing your system if I find that my large ringbuffers are too much of a waste of space (they probably are)

twilit smelt Jul 20, 2025, 3:41 AM

#

they also have a failure mode

#

which is if they fill up

icy bridge Jul 20, 2025, 4:59 AM

#

twilit smelt but determines whether the section's pages can be swapped to disk for kernel mod...

how does this handle I/O errors on page in?

#

for userspace you can just send SIGBUS or similar but that doesn't work for kernel

twilit smelt Jul 20, 2025, 4:59 AM

#

icy bridge how does this handle I/O errors on page in?

if your paging disk is giving IO errors youre dead anyway

#

retry infinitely is the answer basically

icy bridge Jul 20, 2025, 5:00 AM

#

fair enough ig

twilit smelt Jul 20, 2025, 5:00 AM

#

userspace would also just fall to its knees anyway

#

there are other times where i need to be able to handle IO errors on page-in in the kernel though, for things other than paged code and heap

#

like when i access file view mappings

#

i need to propagate the error status from an IO error and handle it sanely

#

but thats a lot diff

icy bridge Jul 20, 2025, 5:01 AM

#

those would just be handled with something like user_memcpy right

twilit smelt Jul 20, 2025, 5:01 AM

#

yeah

blissful smelt Jul 21, 2025, 1:12 AM

#

huh

#

funny

#

is this ctz

#

looks like it

#

neat

twilit smelt Jul 21, 2025, 1:41 AM

#

blissful smelt is this `ctz`

i dont have such a thing

#

so thats what i do instead

blissful smelt Jul 21, 2025, 3:42 AM

#

twilit smelt i dont have such a thing

I am le guessing that this is also just for storing some bitmap about available threads or something

#

I did the same thing after I read that ULE does it

#

pretty great optimization

twilit smelt Jul 21, 2025, 3:46 AM

#

blissful smelt I am le guessing that this is also just for storing some bitmap about available ...

yeah i do this to find a thread in the real time queue and timeshared queue (left and right respectively)

blissful smelt Jul 21, 2025, 3:51 AM

#

twilit smelt yeah i do this to find a thread in the real time queue and timeshared queue (lef...

so I'm guessing you just have two queue levels? I was thinking about introducing realtime, timesharing, and background threads (I am sorry freebsd I keep stealing your ideas) but what is ur approach

twilit smelt Jul 21, 2025, 3:51 AM

#

blissful smelt so I'm guessing you just have two queue levels? I was thinking about introducing...

i also have idle threads

#

which go on a round robin idle queue and are only scheduled when no real time or timeshared threads are available

#

not to be confused with THE (per-cpu) idle thread which does not go on a queue

blissful smelt Jul 21, 2025, 3:52 AM

#

threads, plural? I give each scheduler (one per core) a singular idle thread to run rather than multiple

#

ohh ok nvm yeah

twilit smelt Jul 21, 2025, 3:52 AM

#

twilit smelt not to be confused with THE (per-cpu) idle thread which does not go on a queue

this one is only scheduled when neither realtime, timeshared, nor idle threads are available

blissful smelt Jul 21, 2025, 3:53 AM

#

twilit smelt not to be confused with THE (per-cpu) idle thread which does not go on a queue

so how do these idle threads (plural, not THE idle thread) work then? what kind of tasks run at that level

twilit smelt Jul 21, 2025, 3:53 AM

#

theyre just something really low priority that runs in the background soaking up excess cycles

#

the only thing i have running at that priority (which is priority 0) is the zero page worker

#

which just pre-zeroes free pages and puts them on a pre-zeroed page list

#

so that they dont need to be zeroed later

blissful smelt Jul 21, 2025, 3:54 AM

#

does that thread ever sleep once it finishes the work so that THE idle thread runs?

twilit smelt Jul 21, 2025, 3:54 AM

#

yes

#

it waits until there are more than N free pages where N is some number i forget

#

a few hundred

#

then it zeroes them and goes back to sleep

blissful smelt Jul 21, 2025, 3:56 AM

#

twilit smelt yes

I believe you had some threads that run once per second, right? what was your reasoning for doing this rather than having them only wake up on demand? like my reaper thread will wake up on demand, and if it successfully reaped a thread, it sleeps and wakes up after a second to check again, and if nothing gets reaped it sleeps without a wake up timer

so why do you choose to have threads that just run once per second (if I remembered correctly here)

twilit smelt Jul 21, 2025, 3:56 AM

#

i dont have anything like that

#

the only once per second thing i have is the global balancing DPC

#

which is enqueued on processor zero once per second by the timer ISR

#

DPCs are lightweight so this is nearly free

blissful smelt Jul 21, 2025, 3:57 AM

#

does your timer support interrupting on many cores like the hpet

twilit smelt Jul 21, 2025, 3:57 AM

#

wdym interrupting on many cores

blissful smelt Jul 21, 2025, 3:59 AM

#

twilit smelt wdym interrupting on many cores

so basically you can configure it so that rather than just interrupting on core 0 it can also interrupt on the other cores? idk how to put it

it's like how the HPET has multiple timers and you can set each of them to interrupt a different core, so each core can manage its own set of timer-based deferred events rather than a global queue of them

twilit smelt Jul 21, 2025, 3:59 AM

#

on any preemptive multitasking smp system youll need a timer that can interrupt each core

#

what youre asking is something different

#

youre asking if i have independent timers per core

blissful smelt Jul 21, 2025, 3:59 AM

#

ohhh ya

twilit smelt Jul 21, 2025, 3:59 AM

#

that can be set to different intervals

blissful smelt Jul 21, 2025, 3:59 AM

#

yes

twilit smelt Jul 21, 2025, 4:00 AM

#

basically i dont

#

if i wanted to make mintia2 tickless id need to add that

#

but rn it just uses a 100Hz tick

#

on each cpu

blissful smelt Jul 21, 2025, 4:00 AM

#

twilit smelt on any preemptive multitasking smp system youll need a timer that can interrupt ...

I remember seeing a scheduling approach where only one processor has the timer and it wakes up the other processors (if needed) upon the timer interrupt, so the other cores are essentially "tickless"

have you seen this approach before as well

twilit smelt Jul 21, 2025, 4:01 AM

#

ive heard of that but i dont relaly like it

#

i parallelize my timer queues per-cpu and that would require one processor to deal with all timers

blissful smelt Jul 21, 2025, 4:03 AM

#

do your DPCs have different priority levels in terms of which ones run first or do they just run in the order that they are enqueued (sorry if these are questions that are very explicitly and obviously answered by ur codebase btw, I have a bit of a harder time navigating ur codebase since it's just a format that I'm less familiar with)

twilit smelt Jul 21, 2025, 4:03 AM

#

i feel like this one is very easy for u to answer cuz KeDpc.jkl is not a difficult source file

#

its like 200 lines

blissful smelt Jul 21, 2025, 4:03 AM

#

oh ok ya I shall take a gander

#

yep ok I see thanks 👍

#

say, how did you come across the idea of turnstiles for mutexes? I never saw it in any beginner osdev material and only encountered it when I did a bit of a dive into synchronization primitives and locking across different OSes and saw solaris' turnstiles

twilit smelt Jul 21, 2025, 4:10 AM

#

studying existing kernels

#

theyre used in xnu and solaris

#

and maybe one of the BSDs?

blissful smelt Jul 21, 2025, 4:11 AM

#

I never saw linux mention turnstiles

#

maybe I wasn't looking hard enough

twilit smelt Jul 21, 2025, 4:11 AM

#

they were invented by solaris i believe

#

in like 1994 or soemthing

#

fadanoid found their first description in a book on solaris internals from around then

twilit smelt Jul 21, 2025, 4:12 AM

#

twilit smelt studying existing kernels

NT also accomplishes a pointer-sized lock but it uses a completely different mechanism

blissful smelt Jul 21, 2025, 4:13 AM

#

pushlocks orrrr what

twilit smelt Jul 21, 2025, 4:13 AM

#

yes

blissful smelt Jul 21, 2025, 4:14 AM

#

and iirc solaris does the "hash the address of the lock" to get their "pointer sized locks" as well?

#

very neat approaches

twilit smelt Jul 21, 2025, 4:14 AM

#

yeah

#

pushlocks dont work like that

blissful smelt Jul 21, 2025, 4:20 AM

#

typedef struct _EX_PUSH_LOCK { 
      union { 
                   ULONG Locked: 1; 
                   ULONG Waiting: 1; 
                   ULONG Waking: 1; 
                   ULONG MultipleShared: 1; 
                   ULONG Shared: 28; 
                   ULONG Value; PVOID Ptr; 
              }; 
} EX_PUSH_LOCK, *PEX_PUSH_LOCK;``` ahhh this is very interesting

twilit smelt Jul 21, 2025, 4:23 AM

#

i dislike pushlocq

#

because its wayyy harder to implement priority inheritance with them

digital pivot Jul 21, 2025, 10:12 AM

#

whats the biggest problem in your opinion

twilit smelt Jul 21, 2025, 3:12 PM

#

The turnstile gives a centralized source of information on the state of the lock contention

#

Protected by the hash chain spinlock

#

Pushlocks lack this

#

Because the information is on a list of blocks on the kernel stack anchored on the pushlock that it's difficult or impossible to traverse safely from a PI algorithm

#

So you need some secondary structures and now you need to design autoboost

#

Whic is complicate

mortal thunder Jul 21, 2025, 3:14 PM

#

twilit smelt The turnstile gives a centralized source of information on the state of the lock...

And if turnstiles are pointer size where is this data stored?

twilit smelt Jul 21, 2025, 3:14 PM

#

Turnstiles aren't pointer sized

#

The locks are pointer sized

#

A turnstile only exists for a lock during contention

#

It's in a hash table keyed by address of the lock

mortal thunder Jul 21, 2025, 3:15 PM

#

So the appropriate name is "turnstile based lock" or what?

twilit smelt Jul 21, 2025, 3:15 PM

#

yeah

mortal thunder Jul 21, 2025, 3:16 PM

#

And is the hash table dynamic? ie it expands if buckets are imbalanced

twilit smelt Jul 21, 2025, 3:17 PM

#

That would deadlock so no

#

Not to mention that'd be basically impossible to lock

#

You have to lock the hash chains individually that's the point

#

You can just statically allocate like 128 of them

digital pivot Jul 21, 2025, 3:39 PM

#

twilit smelt Pushlocks lack this

yeah, youll need some extra call for acquire and erlease paths to record/retrieve lock information from KeThread

#

but it will work with all kinds of locks

twilit smelt Jul 21, 2025, 4:53 PM

#

digital pivot but it will work with all kinds of locks

You can reuse turnstiles similarly

blissful smelt Jul 23, 2025, 6:40 PM

#

> ../newsdk/bin/xrbt.exe build.xrbt all
Building ObObject.jkl
OS/Executive/Ob/ObObject.jkl:527:16: Illegal usage of special macro
../newsdk/bin/jkl.exe OS/Executive/Ob/ObObject.jkl OS/Executive/Ob/obj/xr17032-upfre/ObObject.jkl.tmp header=OS/Executive/Ob/obj/xr17032-upfre/ObObject.jkl.dep incdir=OS/Executive/Include libdir=../newsdk/Rtl/Headers:OS/Executive/../Headers:OS/Executive/../../../a4x/Headers target=xr17032 BLD_BITS=32 ARCHITECTURE=xr17032 PLATFORM=XRstation BLD_MP=0 BLD_CHK=0 PLATFORM_FULL_NAME=XRstation-upfre BLD_BASE_LEVEL=2 BLD_TEST=1: 256
``` ruh roh

#

whatever, hopefully a working commit is pushed soon

twilit smelt Jul 23, 2025, 10:02 PM

#

blissful smelt ```c > ../newsdk/bin/xrbt.exe build.xrbt all Building ObObject.jkl OS/Executive/...

Do you have an old toolchain

#

Jackal changes all the time

blissful smelt Jul 23, 2025, 10:03 PM

#

twilit smelt Do you have an old toolchain

oh good point, I'll check again and git pull once I'm back at my puter

twilit smelt Jul 25, 2025, 3:28 AM

#

blissful smelt oh good point, I'll check again and git pull once I'm back at my puter

did it work

#

also i wonder how i could create an "old scanner" effect like this

Screenshot_2025-07-24_at_21-31-05_OneTouch_4.0_Sanned_Documents_-_bf-1-02.670421.io-modes.pdf.png

blissful smelt Jul 25, 2025, 3:31 AM

#

twilit smelt did it work

nope

#

most recent version of newsdk

twilit smelt Jul 25, 2025, 3:32 AM

#

did you rebuild it

blissful smelt Jul 25, 2025, 3:32 AM

#

yes

twilit smelt Jul 25, 2025, 3:32 AM

#

with ./buildall.sh [num cores]

blissful smelt Jul 25, 2025, 3:32 AM

#

this is an entirely different machine and i just cloned and built it 3 minutes ago

twilit smelt Jul 25, 2025, 3:32 AM

#

hm

blissful smelt Jul 25, 2025, 3:32 AM

#

idk

twilit smelt Jul 25, 2025, 3:34 AM

#

i found the issue

#

uninitialized struct field

#

pull latest and try again

blissful smelt Jul 25, 2025, 3:36 AM

#

works 👍

twilit smelt Jul 25, 2025, 3:36 AM

#

rate your satisfaction with how fast i fixed that on a scale of 1 to 10

blissful smelt Jul 25, 2025, 3:37 AM

#

10/10 please take notes linux package managers

twilit smelt Jul 25, 2025, 3:37 AM

#

that bug was only possible because of bad practices in the compiler code though

#

so it evens out

#

anyway does mintia2 run in the emulato

#

r

blissful smelt Jul 25, 2025, 3:37 AM

#

#

truly enthralling

#

I can say I am indubitably stimulated by this

twilit smelt Jul 25, 2025, 3:38 AM

#

if you open the emulator w/o any disks attached you should get to the firmware prompt

#

and then you can type setenv boot-args -dbg

#

and then close it and reopen it with the mintia2 image

blissful smelt Jul 25, 2025, 3:39 AM

#

looks like it

twilit smelt Jul 25, 2025, 3:39 AM

#

and then if you (left-alt) + tab you should be able to break into the debugger

#

right-alt switches between the framebuffer and the two serial terminals and the debugger will prompt over one of them

#

and then you have some incredibly primitive inspection stuff

blissful smelt Jul 25, 2025, 3:40 AM

#

#

something

#

does this use pthreads to do multicore emulation in the emulator

#

I'm only seeing the #include <pthread.h> and nothing else

twilit smelt Jul 25, 2025, 3:42 AM

#

it uses SDL threads which wrap pthreads

#

by default theres 1 thread for each 2 vcpus and any thread can execute code on behalf of any vcpu

#

using a trylock of a mutex around the vcpu to ensure they dont overlap

#

mintia2 doesnt support smp unless you build it with BLD_MP=1 btw

blissful smelt Jul 25, 2025, 3:44 AM

#

and if a vcpu is in a sleep state or something, I'm guessing that when it wakes it gets a host thread assigned to it? or does it not do that

blissful smelt Jul 25, 2025, 3:44 AM

#

twilit smelt mintia2 doesnt support smp unless you build it with `BLD_MP=1` btw

oh dear oops

twilit smelt Jul 25, 2025, 3:44 AM

#

that counts as a different target so the disk image it goes into is different as well

#

MintiaTest-XRstation-mpfre.img or something like that

#

note mpfre rather than upfre which is uniprocessor

blissful smelt Jul 25, 2025, 3:45 AM

#

the pinaccle of computation

#

this is mp

twilit smelt Jul 25, 2025, 3:45 AM

#

are you actually using multiple cpus though

#

-cpus [N] in the emulator command line

blissful smelt Jul 25, 2025, 3:46 AM

#

oh ok

#

it is now

#

this is incredible stuff

#

do you plan to use the fireworks for something functional in userspace like a screensaver or is this just a gimmick

twilit smelt Jul 25, 2025, 3:47 AM

#

its a multithreading demo

blissful smelt Jul 25, 2025, 3:47 AM

#

one thread per firework?

twilit smelt Jul 25, 2025, 3:47 AM

#

1 per firework particle yeah

#

so theres hundreds there

#

of threads

#

old mintia actually was semi usable

📎 mintia-XRstation-fre.img.zip

#

so if u want a peek at what mintia2 might look like later then u can try this out

blissful smelt Jul 25, 2025, 3:48 AM

#

yus i booted old mintia and ran a few commands it was cool

twilit smelt Jul 25, 2025, 3:49 AM

#

mintia2 will be like that but 100x better inside

#

smp support and so on

blissful smelt Jul 25, 2025, 3:50 AM

#

did old mintia not support smp?

#

that's interesting

twilit smelt Jul 25, 2025, 3:50 AM

#

there was no smp in the platform at all

blissful smelt Jul 25, 2025, 3:50 AM

#

oh wowzers

twilit smelt Jul 25, 2025, 3:50 AM

#

it didnt get smp until like december 2023 which was well after old mintia development stopped to start working on jackal and stuff

#

and the smp was untested and buggy until june 2024 when the mintia2 scheduler was written

#

which was the first thing that used it

blissful smelt Jul 25, 2025, 3:51 AM

#

twilit smelt it didnt get smp until like december 2023 which was well after old mintia develo...

how come?

twilit smelt Jul 25, 2025, 3:51 AM

#

because it was less ambitious at the time

#

i had no idea how smp would work at a system design level

#

i also didnt really know how smp support would work in a kernel

#

when i started writing old mintia

blissful smelt Jul 25, 2025, 3:52 AM

#

/home/gummi/mintia/../sdk/fstool.sh /home/gummi/mintia/build/mintia-XRstation-fre.img offset=4 wdf / /home/gummi/mintia/DELTA.xr17032.fre
couldn't mount image: bad magic 0
fstool: error: couldn't mount image``` old mintia is complaining a bit on this machine when i build it

twilit smelt Jul 25, 2025, 3:52 AM

#

by the time i started mintia2 i had basically learned everything i needed to know by osmosis though

#

yeah that happens sometimes

#

idk why

#

ill never fix it

blissful smelt Jul 25, 2025, 3:52 AM

#

oh well

twilit smelt Jul 25, 2025, 3:52 AM

#

just delete the image and retry i think is the solution lol

blissful smelt Jul 25, 2025, 3:52 AM

#

that is ok it builds elsewhere on my other machine

#

ok

twilit smelt Jul 25, 2025, 3:52 AM

#

its cuz of a race condition in the build process or something

blissful smelt Jul 25, 2025, 3:52 AM

#

ope worked fine

twilit smelt Jul 25, 2025, 3:53 AM

#

i used a tangle of makefiles in old mintia

#

unlike mintia2's chad fully custom self hosting build system

blissful smelt Jul 25, 2025, 3:53 AM

#

twilit smelt i used a tangle of makefiles in old mintia

are the makefiles racing?

#

how would this be racing

#

twilit smelt Jul 25, 2025, 3:54 AM

#

probably theres a dependency between targets that isnt properly notated

#

so two targets are running simultaneously that both access the disk image

#

which corrupts it

#

or smth

blissful smelt Jul 25, 2025, 3:54 AM

#

average makefile experience

#

never had that with cmake, idk maybe just me

#

uh what are some fancy things old mintia can do

twilit smelt Jul 25, 2025, 3:55 AM

#

if u type mousedemo youll be able to paint on the screen with a cursor

#

thats fun

#

u can also try reducing the emulator's ram size to see how far mintia's swapping gets it before it wont boot anymore

#

-ramsize [bytes]

#

default is 4mb

blissful smelt Jul 25, 2025, 3:56 AM

#

boy the sensitivity is high

blissful smelt Jul 25, 2025, 3:56 AM

#

twilit smelt default is 4mb

the tragedy with the limine bootloader for me is that it refuses to boot at 4mb (kinda sad)

twilit smelt Jul 25, 2025, 3:56 AM

#

you can use the mem -c command to get interesting memory statistics

#

and kstats

blissful smelt Jul 25, 2025, 3:57 AM

#

woah

#

hmmmm the ticks on kstats seems to constantly be increasing

#

are those scheduler ticks

twilit smelt Jul 25, 2025, 3:57 AM

#

clock tick IRQs yeah

blissful smelt Jul 25, 2025, 3:57 AM

#

did old mintia not have tickless mode

#

oh ok

#

i hope new mintia has tickless mode

#

that would be great

twilit smelt Jul 25, 2025, 3:58 AM

#

it doesnt

blissful smelt Jul 25, 2025, 3:58 AM

#

aw darn

#

so even if there is one thread to run you don't disable the scheduler ticks?

#

aw shucks

twilit smelt Jul 25, 2025, 4:00 AM

#

the argument for doing that is mostly power management

blissful smelt Jul 25, 2025, 4:00 AM

#

i just wanted to see my tests run faster

#

but fair enough

#

once I made my scheduler do tickless properly my tests ran a whopping [Unnoticeable, within the margin of error] nanoseconds faster

twilit smelt Jul 25, 2025, 4:01 AM

#

it wouldnt be particularly difficult to do that later

blissful smelt Jul 25, 2025, 4:02 AM

#

ya super easy stuff

twilit smelt Jul 25, 2025, 4:02 AM

#

my reasoning rn is mostly that xrstation doesnt have a per-core interval timer

#

just a global one

#

but i manage timeouts 100% per-core

blissful smelt Jul 25, 2025, 4:02 AM

#

oh ok so disabling on one would disable for everyone

#

that sounds a little bad

twilit smelt Jul 25, 2025, 4:03 AM

#

to do tickless id need to make one core responsible for all the timer stuff and IPI the others

#

and then my timers dont scale as nicely

#

theres also the property that a thread that goes to sleep on a core will have its wakeup timer enqueued to that core's timer queue

#

and that core's timer ISR will wake it up

#

and probably put the thread back on the same core

#

which helps with affinity being effective

#

i thought about changing the timer to be per-core

#

also theres another thing which is ULE's interactivity heuristic

#

it relies on there being a regular tick to tell if a thread has been running too long and downgrade it to timeshared

blissful smelt Jul 25, 2025, 4:06 AM

#

so ULE will downgrade interactive threads to timeshared? interesting

twilit smelt Jul 25, 2025, 4:07 AM

#

theyre only interactive for as long as theyve been blocked for at least 80% of the last 5 seconds

#

that can change if they run for a long time

blissful smelt Jul 25, 2025, 4:07 AM

#

and that means that interactive threads must always have the ticks enabled when they are running to do this, even if they are the only running thread?

twilit smelt Jul 25, 2025, 4:07 AM

#

you probably dont need a tick as frequent as i have but basically yes it does need to reassess at regular intervals

#

i do 10ms (100Hz) on XR/station and 16.67ms (60Hz) on fox32

blissful smelt Jul 25, 2025, 4:08 AM

#

I suppose once it gets downgraded to timeshared and it detects that it is still the only thread running then there is no need for the tick and disables it though

twilit smelt Jul 25, 2025, 4:08 AM

#

its 60Hz in fox32 because the only timer interrupt fox32 has is the vblank irq

#

btw did you notice the fireworks stuttering more with lots of cpus than with 1

blissful smelt Jul 25, 2025, 4:09 AM

#

not really

#

is that supposed to happen

twilit smelt Jul 25, 2025, 4:09 AM

#

it was stuttering a lot on SMP previously and i did a lot of work to improve it but @mortal thunder who first complained about the stuttering has yet to test it

#

i rewrote the emulator's outer execution loop, added pause instruction, and some other stuff specifically to fix the firework stuttering

#

and he wont even test it! shame him!

blissful smelt Jul 25, 2025, 4:11 AM

#

no problems here

twilit smelt Jul 25, 2025, 4:11 AM

#

blissful smelt I suppose once it gets downgraded to timeshared and it detects that it is still ...

how many cpus is that

#

wrong reply

blissful smelt Jul 25, 2025, 4:11 AM

#

4

#

this is a decade old laptop (intel mac) i5-5250U

#

so i only did 4

twilit smelt Jul 25, 2025, 4:12 AM

#

u cant rly test it well with a shitty host machine

blissful smelt Jul 25, 2025, 4:12 AM

#

ok i'll try on my M4 mac later

#

hopefully that is ok enough

twilit smelt Jul 25, 2025, 4:12 AM

#

because its unclear whether the stuttering is from the smp implementation or from the fact it just cant keep up with the vcpu interpretation

#

so thats why i need iprogramincpp to test it cuz he has a better pc than me plus hes the one who complained to begin with

blissful smelt Jul 25, 2025, 4:13 AM

#

don't you have an apple silicon mac

twilit smelt Jul 25, 2025, 4:13 AM

#

literally the very first one they made 5 yrs ago yes

blissful smelt Jul 25, 2025, 4:13 AM

#

alright understandable

#

is an 11th gen intel i7 with 16 cores ok

twilit smelt Jul 25, 2025, 4:13 AM

#

@mortal thunder TEST IT!!

twilit smelt Jul 25, 2025, 4:13 AM

#

blissful smelt is an 11th gen intel i7 with 16 cores ok

idk probably

blissful smelt Jul 25, 2025, 4:14 AM

#

just get that person who made a verilog implementation of the XR CPU to complete it and manufacture the processor to test on real hardware trl

twilit smelt Jul 25, 2025, 4:14 AM

#

thatll reveal how badly my compiler schedules the risc instructions and theyll pipeline terribly

#

@lucid umbra remember instruction scheduling exists

#

there are two big-ish projects i want to do that are on hold right now becuase of school business

#

which are

#

redo the kernel debugger to be a common front end which has two modes, local and remote. in local mode it is built as a kernel module (like the current debugger) and uses whatever symbol info is available and presents the prompt over the serial port and so on. the second is a remote mode where it runs on the host and can consume files with more comprehensive debug info, and talks to a debug server (that is also a kernel module) with a simple serial protocol. requires extending the toolchain (mainly the linker) to be able to emit these debug data files

#

redo the emulator core to be a pseudo-jit which keeps a "decode cache" of basic blocks that you pass to instruction implementations that directly tail-cail the next function pointer in the basic block. it should get like 10x faster

#

the benefit of #1 is that ill be able to provide tons more debug info (like line numbers, file names, and so on) for myself without needing to include them in the kernel binary which bloats it

#

additionally ill be able to debug builds of the kernel binary that dont include any debug symbols, just as well as ones that do, which will allow me to debug super-low-memory kernels a lot better

#

benefit of #2 is obvious which is just performance and power savings for the user of the emulator

lucid umbra Jul 25, 2025, 4:31 AM

#

twilit smelt <@454436336175415317> remember instruction scheduling exists

true

#

especially for the x86-64 backend cause it has cpu impls i can base it of off

twilit smelt Jul 25, 2025, 4:32 AM

#

doesnt it get less important with OoO

#

because itll just reorder it to pipeline better anyway

#

whereas instruction scheduling was like literally the single most important component of pipelined risc performance in like 1989

lucid umbra Jul 25, 2025, 4:33 AM

#

doesnt that go for most isas and a sufficiently advanced xr17032 impl could do that too

twilit smelt Jul 25, 2025, 4:33 AM

#

yes but xr17032 is meant to be from like 1989 so its unlikely to have OoO

lucid umbra Jul 25, 2025, 4:34 AM

#

fair

twilit smelt Jul 25, 2025, 4:34 AM

#

OoO existed by then but in like room-sized computers still

#

was still mostly theoretical in microprocessors

#

i have lore which is that xrcorp is like a spinoff company by a few DEC engineers founded in 1982 to make risc workstations and xrstation is what they were putting out in 1989-1991

#

old mintia is what ran on it in 1989 and mintia2 is like the last version of the OS to still support it in 1993-1994

#

they go bankrupt in the late 90s or early '00s

#

lol

#

i think this makes the most sense as to historical setting for all this

#

old mintia being written in dragonfruit is non canonical though.

lucid umbra Jul 25, 2025, 4:38 AM

#

twilit smelt Jul 25, 2025, 4:38 AM

#

it would have been written in jackal side by side with hardware development from 1982 until first release in 1985 and smoothly evolved into mintia2 i guess

#

yeah lol

#

xr/corp's biggest asset is mintia which it ends up porting to aphelion in the 90s anyway after deciding to discontinue its hardware

#

i forgot i said that

#

so yeah i guess xr/station is discontinued in the early 90s and xr/corp becomes primarily a software vendor in partnership with orbit systems

#

mintia2 is like the last version of mintia to run on the "legacy" hardware

#

orbit systems might even acquire xr/corp for its systems software division or something

lucid umbra Jul 25, 2025, 4:42 AM

#

funny enough i had some orbit systems logo variations that i tried to give a more 80s/90s feel (i.e. give the circle lines lmao)

#

so id consider all of this canon

twilit smelt Jul 25, 2025, 4:43 AM

#

also it might be called like XRSI (Xyzzy RISC Systems Inc)

#

rather than xrcorp

#

the "lore" is mostly superficial ive been too busy writing code to like put together an ARG or something with fake memos tracing its history up through the 80s or something cool like that

#

i def want the pdfs distributed of the books and documentation to be given fucked up old worn down scan feeling and things like that

twilit smelt Jul 25, 2025, 4:45 AM

#

twilit smelt also i wonder how i could create an "old scanner" effect like this

like this

#

i dunno how to convincingly imitate that though

#

the best way honestly might be to actually print and scan it with old equipment or something and then hand-correct the OCR so its still sanely copyable and stuff

#

i could use sandpaper lightly on the fresh prints to make them look aged

#

ink flaked off and stuff

lucid umbra Jul 25, 2025, 4:48 AM

#

avery(damia) in fox32 has a old printer/scanner that could probably get that kinda effect just by printing and scanning it back in directly

#

😭

twilit smelt Jul 25, 2025, 4:49 AM

#

even scanning it with a modern scanner gets you a good part of the way there

lucid umbra Jul 25, 2025, 4:51 AM

#

i love the xr corp alternate universe idea

#

with like fox32 and orbit as corporate entities

#

its rly fun to think about

twilit smelt Jul 25, 2025, 4:52 AM

#

i think xrcorp being a DEC spin off makes sense given how mintia is clearly derivative of like VMS

#

disgruntled engineers leave to make a RISC workstation company cuz management isnt taking it seriously enough

#

something liket hat

#

mintia would still exist in legacy systems in like 2025 and have boomer fan clubs i guess

lucid umbra Jul 25, 2025, 4:53 AM

#

arg where the prize is getting a realistically aged xr station system with mintia2 and aged manuals

twilit smelt Jul 25, 2025, 4:54 AM

#

lol thatd be cool

lucid umbra Jul 25, 2025, 4:55 AM

#

arg puzzles include writing programs in jackal or xr assembly to solve various issues / fix components in some virtual system

#

finding obscure details in internal communications

twilit smelt Jul 25, 2025, 4:57 AM

#

itd be a bit like the aperture science arg for portal 2's release

#

where they had you log in to a fake GLaDOS prompt made with flash on some website, aperturescience.com i think

lucid umbra Jul 25, 2025, 4:58 AM

#

ye ye

#

emails from frustrated orbit systems engineers having to port their compiler stack to support jackal and xr17032 cause their company bought xr corp out

twilit smelt Jul 25, 2025, 4:58 AM

#

probably one of the most famous args

#

this is funny because ive actually seen frustrated emails from mipsco engineers bitching among themselves about adjusting to SGI's culture and regearing around SGI's hardware after the buyout

#

for the life of me ive never been able to find that again, it was pretty fascinating

#

one of the emails was a memo called "hard truths" or something where someone pointed out all the inefficiencies in SGI IRIX and going like "im sorry but it is COMPLETELY RIDICULOUS that on my gigantic 16MB workstation, my paging disk STILL churns CONSTANTLY"

#

and he goes into deep technical reasons as to why and why ppl need to fix it asap

lucid umbra Jul 25, 2025, 5:00 AM

#

fuckin hilarious lmao

#

i have no idea where orbit systems would come from in this timeline

twilit smelt Jul 25, 2025, 5:02 AM

#

probably another risc startup in the mid 80s

lucid umbra Jul 25, 2025, 5:02 AM

#

i dont know enough about the corporate tech landscape back then

#

prolly a bunch of engineers trying to get ahead of the game straight into 64 bit

#

into scientific computing especially

twilit smelt Jul 25, 2025, 5:06 AM

#

yeah maybe ur like "we can deliver a 64 bit system in 1987" and everyone is like "thats crazy youre insane" and then you do it and it is actually dog slow and nobody cares and you almost go bankrupt but then you have a huge advantage in the early 90s from a head start on 64 bit izing your OS (which is a unix flavor)

#

and ur revenue explodes

#

and the 90s systems are really good

#

and youre kept relevant by the early 64 bit advantage while xrcorp starts stumbling into risc workstation oblivion

#

the issue with late 80s 64 bit isnt that it wasnt possible its that it would have been too big of a bottleneck on the clock speed

#

so youd be some crazy engineers who ignore that trade off and come out with this slow shitty system and everybody laughs and then youre the ones laughing like 3 yrs later

lucid umbra Jul 25, 2025, 5:09 AM

#

ill take that hell yeah

twilit smelt Jul 25, 2025, 5:09 AM

#

youre like barely kept afloat 1987-1990 by some labs who dont really care about performance they just have some application that really needs full 64 bit

#

and you have a slight performance edge doing native 64 bit versus other risc platforms synthesizing 64 bit ops

#

which is a tiny minority of applications but barely enough to keep the lights on

blissful smelt Jul 25, 2025, 5:10 AM

#

i got a little bored and tried to digitally add artifacting to the docs, idk maybe this could work if i made it less bad

#

prob not tho

#

a real scanner would be better

twilit smelt Jul 25, 2025, 5:10 AM

#

im restyling the documents rn

#

in the process of

#

to make them look more convincingly old

#

thats a fairly modern looking monospaced font so im switching to just Courier and stuff

blissful smelt Jul 25, 2025, 5:11 AM

#

understandable

twilit smelt Jul 25, 2025, 5:11 AM

#

and fluctuating font size less frequently

#

to make it look tapped out on a typewriter almost

blissful smelt Jul 25, 2025, 5:11 AM

#

I have a fairly old printer at home that's been around for longer than I have so it produces some grainy paper that I can scan

twilit smelt Jul 25, 2025, 5:11 AM

#

i bet there are like 20-30 yr old printers on ebay for like $50

blissful smelt Jul 25, 2025, 5:11 AM

#

wait I think using a phone-pdf scanner might be better at making the "old scan" appearance than an actual scanner, idk though. phone-pdf scans are real crap

twilit smelt Jul 25, 2025, 5:12 AM

#

much older than that and you start getting famous vintage like apple's 80s printers and the price goes back up though

#

but like a random printer from 2001 probably you could find for cheap

#

at a thrift store even

blissful smelt Jul 25, 2025, 5:13 AM

#

if you wanna go down the ARG route you'll also need other crappy hardware too

#

maybe for sound/audio logs

twilit smelt Jul 25, 2025, 5:14 AM

#

i feel like ive been developing this stuff too openly to make an interesting ARG

blissful smelt Jul 25, 2025, 5:14 AM

#

sounds like a neat idea tho

twilit smelt Jul 25, 2025, 5:14 AM

#

theres no like mystique

#

itd be cooler if this stuff just appeared with zero background and unknown who made it or why

#

and no source available just binaries and really old looking scans

blissful smelt Jul 25, 2025, 5:14 AM

#

mintia3 arg

twilit smelt Jul 25, 2025, 5:15 AM

#

i mean i guess its still basically unknown but if the ARG was a popular thing on /g/ or somewhere someone from here would inevitably spill the beans

lucid umbra Jul 25, 2025, 5:16 AM

#

i think it could still realistically go down a TIS-100 style route as a programming puzzle arg with lots of lore details

twilit smelt Jul 25, 2025, 5:16 AM

#

i looked away from my screen and there was a black rectangle that rapidly strobed up the screen repeatedly and as soon as i looked back it stopped

#

menacing

lucid umbra Jul 25, 2025, 5:17 AM

#

youve been visted by the rectangle

#

consider yourself blessed

#

or smth

blissful smelt Jul 25, 2025, 5:18 AM

#

lucid umbra i think it could still realistically go down a TIS-100 style route as a programm...

mintaf at xr's

twilit smelt Jul 25, 2025, 5:18 AM

#

lucid umbra i think it could still realistically go down a TIS-100 style route as a programm...

i think the mystique factor is the only thing that would make me interested in actually doing an ARG

lucid umbra Jul 25, 2025, 5:18 AM

#

fair

twilit smelt Jul 25, 2025, 5:18 AM

#

like imagine if i did this all secretly and then just randomly dropped a zip file on 4chan or something

lucid umbra Jul 25, 2025, 5:19 AM

#

the os and the arch are documented in a technical fashion but other than tidbits in the xr station book theres almost nothing externally documented about the story (apart from discord conversations like this ig)

twilit smelt Jul 25, 2025, 5:20 AM

#

the comments tend to also be written from an "in-universe" perspective

#

in the code

lucid umbra Jul 25, 2025, 5:20 AM

#

mm

twilit smelt Jul 25, 2025, 5:21 AM

#

but like the thing that ruins it a bit is the fact its open source and the developer is a known individual

#

i would def want to make it seem plausible it like crossed over from an alternate timeline or something

#

id consider it a success if i managed to convince some schizos that it actually did

#

if i distributed the sources id want to put it on like microfiche or something lol

#

i wasnt planning to do any of this though and i woul dhave needed to decide i was going to do this back in like june 2018 which wasnt ever gonna happen

#

oh well

lucid umbra Jul 25, 2025, 5:24 AM

#

welp time to rewrite everything with a completely different name and design in secret

#

:^)

twilit smelt Jul 25, 2025, 5:25 AM

#

twilit smelt id consider it a success if i managed to convince some schizos that it actually ...

"top ten internet mysteries. number 8: xr/station. another dimension crossing over with ours? or just an extremely elaborate hoax? on july 25th 2025 a mysterious zip file was posted on 4chan's technology board"

#

theres an alterate timeline where i did do that and ill have to be satisfied with that i guess

lucid umbra Jul 25, 2025, 5:27 AM

#

living vicariously through an alternate timeline version of yourself is a surprisingly effective strategy

blissful smelt Jul 25, 2025, 5:27 AM

#

lucid umbra welp time to rewrite everything with a completely different name and design in s...

language - Coyote
processor - HB48503
kernel - Ruelaris

trl

#

no one will suspect a thing

lucid umbra Jul 25, 2025, 5:27 AM

#

coyote compiler mentioned

twilit smelt Jul 25, 2025, 5:27 AM

#

twilit smelt "top ten internet mysteries. number 8: xr/station. another dimension crossing ov...

"containing extensive schematics, documentation, and leaked memos from a 20th century technology company. the weird thing is... this company... spooky sound effect never. existed"

twilit smelt Jul 25, 2025, 5:28 AM

#

blissful smelt language - Coyote processor - HB48503 kernel - Ruelaris <:trl:82931664119188688...

i ran into a coyote on a walk once

#

we scared eachother

lucid umbra Jul 25, 2025, 5:28 AM

#

cotoye

blissful smelt Jul 25, 2025, 5:28 AM

#

dobgy meets dogboy ? the missing letter on dobgy discovered ?

twilit smelt Jul 25, 2025, 5:28 AM

#

twilit smelt "containing extensive schematics, documentation, and leaked memos from a 20th ce...

id want it to be like a cult thing like boards of canada where some people just get like entranced by the mystique

#

the boards of canada people are known but theyre extremely secretive so not much more is known about them than their names

#

whereas i have nearly 100k messages in here documenting everything ive done

blissful smelt Jul 25, 2025, 5:29 AM

#

just ask the mods to ban you and delete the server and do the thing trl easy stuff really

twilit smelt Jul 25, 2025, 5:30 AM

#

they did delete the mintia thread once before

blissful smelt Jul 25, 2025, 5:30 AM

#

i wonder if there are any computer-related ARG things like what ur saying

twilit smelt Jul 25, 2025, 5:30 AM

#

when it had 6x as many messages as it has now

lucid umbra Jul 25, 2025, 5:30 AM

#

https://tenor.com/view/йокелемене-men-in-black-light-sunglasses-gif-15516530

twilit smelt Jul 25, 2025, 5:30 AM

#

by "they" i mean mintsuki

blissful smelt Jul 25, 2025, 5:30 AM

#

blissful smelt i wonder if there are any computer-related ARG things like what ur saying

aside from the cicada thing

blissful smelt Jul 25, 2025, 5:30 AM

#

twilit smelt by "they" i mean mintsuki

why tho

#

sad

twilit smelt Jul 25, 2025, 5:30 AM

#

they got mad at me

blissful smelt Jul 25, 2025, 5:30 AM

#

💀

lucid umbra Jul 25, 2025, 5:30 AM

#

lucid umbra https://tenor.com/view/%D0%B9%D0%BE%D0%BA%D0%B5%D0%BB%D0%B5%D0%BC%D0%B5%D0%BD%D0...

thats a much more annoying gif than i thought it would be wow

twilit smelt Jul 25, 2025, 5:30 AM

#

lost like the first 3 yrs of dev history which i almost exclusively logged here

#

it was very upsetting

#

still like once a week i try to find some convo with fadanoid to see our reasoning for some concept and find it was in the deleted thread and then i get upset again

blissful smelt Jul 25, 2025, 5:32 AM

#

do you keep a physical journal about these things

twilit smelt Jul 25, 2025, 5:32 AM

#

no

blissful smelt Jul 25, 2025, 5:32 AM

#

I have a weird habit of that

#

aw man

twilit smelt Jul 25, 2025, 5:33 AM

#

it was all in here :(

#

the like 3 yrs before that were in the first fantasy consoles discord that got deleted after someone hacked the owner's account

#

which prob should have taught me a lesson

#

anyway it like tangibly hurt me cuz it deprived me of a source for if i wanted to back-blog stuff

#

and there was far more in that thread than i could ever remember

#

may have contributed to some loss of motivation

blissful smelt Jul 25, 2025, 5:34 AM

#

time to subpoena discord for the logs

twilit smelt Jul 25, 2025, 5:35 AM

#

i have forgiven mint but still find it very disturbing that they impulsively did that within like 30 seconds of me doing something that personally pissed them off

#

(i forgave cuz they apologized to me)

#

so i tend to keep my own records now

blissful smelt Jul 25, 2025, 5:38 AM

#

do you record things elsewhere

twilit smelt Jul 25, 2025, 5:38 AM

#

cuz random project thread deletion is apparently a thing that can happen

#

wdym

blissful smelt Jul 25, 2025, 5:38 AM

#

like have a separate log of design decision reasons and timestamps for all of them

twilit smelt Jul 25, 2025, 5:39 AM

#

no

#

closest would be theory comments in the github repos

#

and how they change across commits

blissful smelt Jul 25, 2025, 5:39 AM

#

and git blame of course

blissful smelt Jul 25, 2025, 5:41 AM

#

twilit smelt no

were there any unique design decisions to mintia that you DO remember that may have been talked about in logs in the deleted thread

twilit smelt Jul 25, 2025, 5:41 AM

#

iterative IO enqueue was probably the big one

blissful smelt Jul 25, 2025, 5:41 AM

#

woah what does that mean

twilit smelt Jul 25, 2025, 5:42 AM

#

i could link you to the discussion but the thread was deleted

#

but basically its where each driver in a driver stack is called in sequence rather than recursively

#

we thought we were the first to implement this but minoca turned out to already do it because of course it did

blissful smelt Jul 25, 2025, 5:44 AM

#

twilit smelt but basically its where each driver in a driver stack is called in sequence rath...

every driver in a driver stack? wdym, like if you have one device with many drivers related to it?

twilit smelt Jul 25, 2025, 5:45 AM

#

for example if you mount a disk image then the driver stack will look something like

#

fs driver -> disk image layer -> fs driver -> hard disk driver

blissful smelt Jul 25, 2025, 5:46 AM

#

oh wow yea i never thought of that

#

so is there a real improvement with an iterative IO stack

twilit smelt Jul 25, 2025, 5:47 AM

#

stack usage

#

you evade the problem of indefinitely growing stack usage as a driver stack grows

#

which is a real issu

#

its "solved" in windows by like explicit stack extension upon entry to drivers where the driver stack is often deep

#

such as fs drivers

#

they have something htat looks like

#

if remainingstack() < 1024:
  extend kernel stack;

blissful smelt Jul 25, 2025, 5:50 AM

#

twilit smelt they have something htat looks like

so would iterative IO be done by having something like a list of IO "steps"?

#

this sounds neat i shall take a gander at it

twilit smelt Jul 25, 2025, 5:51 AM

#

blissful smelt so would iterative IO be done by having something like a list of IO "steps"?

each driver you call fills in a structure that tells you what to call next

#

with what parameters

blissful smelt Jul 25, 2025, 5:51 AM

#

so it's similar to defunctionalization in other contexts where you """simulate the call stack"""

#

neato

twilit smelt Jul 25, 2025, 5:52 AM

#

also it can ask you to call anything next

#

doesnt have to actually be on the same driver stack

#

you might not even model the driver stack in any high level way

mortal thunder Jul 25, 2025, 6:49 AM

#

twilit smelt <@700663172226482254> TEST IT!!

nice

#

its very smooth now

#

mortal thunder Jul 25, 2025, 6:50 AM

#

twilit smelt so thats why i need iprogramincpp to test it cuz he has a better pc than me plus...

do I?

twilit smelt Jul 25, 2025, 6:50 AM

#

mortal thunder nice

thanks

#

glad to see my tricks worked

mortal thunder Jul 25, 2025, 6:51 AM

#

im not sure how much powerful the r5 5600h is compared to the m1

#

honestly though maybe

maiden lance Jul 25, 2025, 7:35 AM

#

twilit smelt also i wonder how i could create an "old scanner" effect like this

My first guess would be to slap on some binary noise on multiply

#

There are probably some old scanner noise overlays online as well

twilit smelt Jul 25, 2025, 8:48 AM

#

i actually got the idea from neil christiansen i think his name is

#

theres a video interview of him on youtube with like 200 views where someone asks him if he wishes anything was different about NT's original design

#

(at the time he was (is?) a big name in the filesystems and filters group if you dont know him)

#

he says he wishes the IO stack was designed iteratively in the way i described

#

i agreed that was a good idea so i yoinked it lol

#

apparently the minifilter model was later designed iteratively in this way

#

by him

#

https://www.youtube.com/watch?v=TypWW6MFhW4

YouTube

NT

Neal Christiansen - Inside File System Filter, part I

File System Filters are kernel-mode non-device drivers that monitor inbound and outbound FileSystem IO.

A prime example of an FSM is anti-virus software (the primary function of an AV app is to monitor IO stream content looking for virus patterns, after all).

Anyway, we were introduced to Neal by Dana Epp (he's working with the filter driver t...

▶ Play video

#

it was either htis video or part 2

tame phoenix Jul 25, 2025, 9:30 AM

#

what does the filter stack do, is this stuff like encryption and compression?

digital pivot Jul 25, 2025, 9:42 AM

#

while you're here and i hope will doesnt mind, since you wrote that you planned to implement priority inheritance similar to windows , could you tell something about the windows approach ?

#

i mean how your current approach would transform

digital pivot Jul 25, 2025, 5:31 PM

#

is this called fast i/o ?

#

or fastio is a different thing

#

i think i got confused

digital pivot Jul 26, 2025, 8:37 AM

#

i see

#

btw thats what windows does, it has per thread structure that stores lock information

twilit smelt Jul 26, 2025, 2:35 PM

#

https://www.youtube.com/watch?v=mpQXqnOStAs

YouTube

Sesame Street

Sesame Street: Valentine - I Love You

If you're watching videos with your preschooler and would like to do so in a safe, child-friendly environment, please join us at http://www.sesamestreet.org

A girl writes "I love you" in a heart.

Sesame Street is a production of Sesame Workshop, a nonprofit educational organization which also produces Pinky Dinky Doo, The Electric Company,...

▶ Play video

digital pivot Jul 26, 2025, 3:03 PM

#

yeah i think its called KLOCK_ENTRY

#

6 of them in each thread or something like that

twilit smelt Jul 26, 2025, 4:33 PM

#

digital pivot 6 of them in each thread or something like that

they changed this

#

the 6 per thread thing came from descriptions of older versions of windows

#

i dislike the need to have per-thread lock entry structures though which is another reason i did turnstiles instead

twilit smelt Jul 26, 2025, 4:35 PM

#

twilit smelt the 6 per thread thing came from descriptions of older versions of windows

the array of 6 lock entries embedded in each thread structure got turned into a pointer to an out-of-line array of lock entries. take that as you will

#

the failure case implied by the expansion is scary

#

do you know if the NT team ever experimented with turnstiles

#

theyre used (by that name) in XNU, Solaris, and possibly the BSDs

#

i have the same requirement and they suit me

#

there is one little funny point though where i do a CAS on a lock word while holding a turnstile hash chain spinlock and i need to catch the potential page fault and make it abort the CAS specially in such a way that the calling code unlocks everything and then directly calls the page fault handler on the lock word (to bring in the containing page writable) and then retries the entire locking operation

#

but with that trick it (should) work

#

yeah

#

just pointing out i also had the requirement of pageable pointer sized locks and at first i thought pushlocks were the only way forward for that because all the turnstile impls i saw would, at some point, access the lock word while holding a spinlock (which is no bueno if you can take a page fault)

#

but then i realized i can just make it abort and retry the lock operation if such a page fault actually occurs

#

and turnstiles work after all

#

wdym

#

basically when you contend on a lock you put the turnstile block into a hash table keyed by the address of the lock

#

and theres per hash bucket locking

#

so thats how

#

spinlock

#

one per bucket

#

usually they just statically allocate like 128-1024 buckets and use a good hash function and they end up with nearly zero contention on these spinlocks in practice

#

the uncontended fast path for locking and unlocking is as brief as youd expect as well, no need to take any spinlocks or whatever its just a simple CAS (or equivalent)

#

i bet multics did it already

#

boosting outstanding packets is scary

digital pivot Jul 26, 2025, 4:55 PM

#

do you know how release path is optimized since it seems every release can call heavy tree related operations

#

i think will found out that every release of a pushlock calls some heavy autoboost stuff ?

#

like removal from an rb tree or something like that

#

no idea, is it like a flag to skip some operations ?

twilit smelt Jul 26, 2025, 4:59 PM

#

i got obsessed with NT like 5 yrs ago and it ended up being contagious

#

plus we had one or two NT kernel people pass through here at various points

#

and they would sometimes just talk openly about work they were doing

#

we've also had XNU ppl but i shant name them

twilit smelt Jul 26, 2025, 5:02 PM

#

twilit smelt we've also had XNU ppl but i shant name them

the urge in industry ppl to talk to random hobbyists about cool work theyre doing seems to be growing in recent years lol, might just be because the people working on these things are increasingly from more "internet native" generations

digital pivot Jul 26, 2025, 5:03 PM

#

inetersting, did you research autoboost?

#

itd be super interesting to see similar thing in your kernel

#

what a pity that its closed

twilit smelt Jul 26, 2025, 5:05 PM

#

twilit smelt we've also had XNU ppl but i shant name them

also had the google fuchsia architect in here before someone was rude to him and he left but that doesnt really count because he was a hobby osdev before he worked on fuchsia

digital pivot Jul 26, 2025, 5:06 PM

#

u worked on nt?

twilit smelt Jul 26, 2025, 5:06 PM

#

digital pivot u worked on nt?

not necessarily

#

there are many people who have access to its source code

#

including a bunch of governments and stuff who legally require it so they can audit it themselves to make sure its not full of american spyware before they use it on govt PCs

digital pivot Jul 26, 2025, 5:08 PM

#

no way

#

holy crap

twilit smelt Jul 26, 2025, 5:08 PM

#

digital pivot holy crap

youve literally met an NT guy before

#

now youve just met two of them

#

actually three

digital pivot Jul 26, 2025, 5:09 PM

#

twilit smelt youve literally met an NT guy before

now instead of each old one there will be a new one ?

twilit smelt Jul 26, 2025, 5:09 PM

#

?

digital pivot Jul 26, 2025, 5:09 PM

#

they left this server

#

now theres new one

#

which subsystem did you work on?

twilit smelt Jul 26, 2025, 5:11 PM

#

im unsure if youd know him but the person who is currently in charge of many of the scheduler functions was once in here and is still in an adjacent private server

#

dunno!

#

it sounded like he was important though

#

there was also another guy (who actually brought him here i think) who called him his boss

#

so he's someone's boss

#

what timeframe did you work on the kernel core

#

if it was anywhere in the last like 5 years you probably do know this guy, beyond that idk

#

i suspected he actually brought you here but it sounds like its just a coincidence

#

we are familiar with the cache manager

#

sometimes we call it the cache manafler cuz of bad OCR from a windows internals book scan

mortal thunder Jul 26, 2025, 5:19 PM

#

Does the cache manager do anything more than map views of files/sections in kernel space for fast regular read/write primitives?

twilit smelt Jul 26, 2025, 5:19 PM

#

yeah it has the lazy writer

#MINTIA (not vibecoded)