MINTIA (not vibecoded)

[cursive charm]

dave, i don't think you made a video on IE's download dialog

the file operation dialog isn't really the same as that

[twilit smelt]

a poster on the DEC notefile system in the early 90s called himself "A Soldier in Dave's Cutlerian Jihad"

didnt realize there were excellent shitposters back then

[twilit smelt]

cant believe i never thought of that before but of course i have to use it as my nickname now

[shadow ridge]

that coincidence is insane tbh

it's cool cuz it basically runs off of microcode

Because the top two bits of the 40-bit pointer were used for status flags, Objekt could only identify 238 objects in total. Since the objects were constantly being garbage collected, many of these values might point to non-existent objects, meaning the system could run out of identifiers in practical use. To address this, the entire system image was periodically written to disk, during which time all of the pointers were re-numbered to be consecutive.[9]

what the hell

[twilit smelt]

john vert saying something is "frying" him is strange

this one is going native

heres an engagement bait account for you all to block

for those of you unfortunate enough to use twitter

more of the counterfactual insistence that not only will AI replace everything but that it in fact already has

this person is suggesting that "kernels, protocols, router firmware, browsers, databases, etc" are no longer "coded by hand"

lol

reeks of insecurity that they have to lie like this

[shadow ridge]

thats not how i read it

[twilit smelt]

hes saying it firmly in the past tense

[shadow ridge]

the student thinks theyre no longer coded by hand

[twilit smelt]

he never uses the present tense to refer to it

even outside of the student's thoughts he uses only the past tense

the implication is clear

[shadow ridge]

oh right

he says "the idea that this was even possible"

like it's impossible to code without AI now

[twilit smelt]

he lives in a world of people addicted to agents

ive seen it described by themselves as an addiction and a problem

they have a hard time keeping the attention span to do anything by hand anymore which causes problems when they simply cant get the agent to do something right on its own

they then have to go reach into the codebase themselves at which point they get stuck in a loop of distractions by youtube and tiktok and whatever because its so boring to spend longer than 5 seconds doing it

[shadow ridge]

do they even know how to code

[twilit smelt]

for now

[cursive charm]

i'm not on twitter, but i guess i'm on mastodon, which is "dead" or something

[cursive charm]

miraculously, i just remembered a post on activitypub by david chisnall about NT's interrupt handling

david chisnall was on the freebsd core team, for those curious

apparently, they also worked at microsoft

[twilit smelt]

would you believe that this is not about dave cutler and lou perrazzoli of the nt kernel team

[warm pine]

he's very good

has an extensive exposure to many different systems

[twilit smelt]

im not seeing why a processor being hogged by a high priority interrupt thread is any more free or responsive, from the perspective of user-level threads (like UI threads), than a processor being hogged by an ISR

[warm pine]

he was a key figure in the Etoile project the principles of which i think were excellent

[twilit smelt]

those are both going to run to completion before the processor is freed up

also i think hes simplifying (consciously, for the benefit of readers so he doesnt have to write an expository paragraph about the difference) DPC to just 'interrupt handler'

[twilit smelt]

i mean i guess you could make the window server input thread that like redraws the cursor be higher priority than the interrupt threads

that might even make sense

that might make it FEEL somewhat more responsive

but idk

[warm mural]

cutlerian jihad 💀

[twilit smelt]

i think hes wrong on his point that windows doing too much work in interrupt context causes responsiveness issues

the same work but done in a high priority interrupt thread is no less of a responsiveness hazard

except for the fact the interrupted thread is pinned to the core and cant run until the interrupt is done, that is, it remains "running" on the core and cant be rescheduled elsewhere

but they solved this

DPCs that consistently take a longish time to execute (which is dynamically determined) now cause rescheduling to a "delegate" thread which frees up the interrupted thread to be rescheduled on another core

[warm pine]

the principle i think is that certain threads servicing work triggered by interrupts can be of a relatively lower priority

[twilit smelt]

how often does that happen in practice though

one lesson learned by the VMS team was that when you give driver developers the option between multiple priority levels for their drivers' work to run at

they will always pick the highest one bc everyone thinks theyre the most important lol

thats why theres only 1 DPC IRQL in NT

versus the like 8 in VMS

its because everybody would only use the highest one in VMS

[twilit smelt]

THIS is a problem of doing too much in ISRs versus a thread

[twilit smelt]

but now they just do this

so

[digital pivot]

"On FreeBSD and XNU, interrupt handlers do one thing: wake a thread (or some work-queue equivalent). The thread is then preemptive. A small number of things run with interrupts disabled but it’s very rare in drivers or subsystems outside the very core parts of the OS. In Windows, the driver model seems to encourage people to just do the real work in interrupt handlers."

he probably means dpcs right?

[twilit smelt]

yeah

[digital pivot]

but they dont disable all interrupts

if they mean that

[twilit smelt]

on NT you cant do much in an ISR other than touch your own driver's stuff and enqueue a DPC

[digital pivot]

idk about freebsd but windows has irqls

also afaik threading dpcs are enabled by default

but idk if the kernel automatically uses them

or a driver developers needs to specify some flag to a dpc

[hybrid condor]

does the delegate thread run at the user level?

[twilit smelt]

its kind of a fake thread they use as an implementation trick to free up the current thread to be rescheduled elsewhere

it only runs at DPC_LEVEL

irql

er sorry

DISPATCH_LEVEL

[glossy leaf]

there is a dedicated function for initializing threaded dpc's which makes sense as the dpc routine itself needs to be able to handle being called either at passive irql or at dpc irql

[digital pivot]

what counts as long enough?

to enable this

[twilit smelt]

i dont know

[shadow ridge]

are IRQLs really all that good

linux doesnt make use of them afaik

what benefits do they have that rivals complicating the kernel

[warm mural]

they have equivalents

preempt_{enable, disable}, softirqs, tasklets, etc

[shadow ridge]

arent those like callbacks

[warm mural]

and the unix family has traditionally had spl which is similar

[shadow ridge]

that are called at the end of an interrupt

[warm mural]

what do you think DPCs are

[shadow ridge]

i thought they ran in a separate worker thread

[warm pine]

those would just be work items

DPCs are logically interrupts at a priority (usually) lower than all interrupts generated by hardware

or rather they're a queue of callbacks drained by that logical interrupt

[hybrid condor]

yeah that's a pretty clever idea

[hybrid condor]

is that enough to justify the lack of work items

[twilit smelt]

it mitigates the fact that a lot of third party drivers overuse DPCs

they do have work items

but they cant force everyone to use them and unfortunately it also wouldnt be correct to just turn every DPC into a work item from their end or something

[hybrid condor]

so maybe I misunderstood, are they available to all developers or just microsoft?

[twilit smelt]

they are available to all developers

[hybrid condor]

also the thing is the delegaties help with starvation but not with very frequent dpc requests

I don't know the implementation details so it might be quite expensive

as far as I can tell there don't seem to be any limits on their requests or the number of operations in a single routine

[hybrid condor]

also, if a dpc is marked as long running, could that change in certain cases?

[hybrid condor]

for example, if I can change the dpc associated routine

[twilit smelt]

i think they always measure how long it takes to execute (in some way) and the flag can be flipped either way dynamically based on a pattern

so if it has stretches where it takes a long time and stretches where its very brief they can account for that

[cursive charm]

for the record, i asked in the server i first shared that post in about whether solaris does something similar with its interrupt handling

i think i only asked that because of rich felker of musl libc's Hot Take™ reply to david: https://hachyderm.io/@dalias/114415240903983708

Cassandrich (@[email protected])

-# ↩ David Chisnall (Now with 50% more sarcasm!)
@david_chisnall @amy A proper OS shouldn't even have a way to do things from interrupt handlers, or a concept of interrupt handlers (plural). There should be one interrupt handler that maps the interrupt to a list of hardware devices associated with it and fires a wake event for any drivers attached to each of them, then immediately returns, with no way to override that.

Hachyderm.io · Apr 28, 2025, 11:00 AM

"a proper OS" is doing the heavy lifting there, i think

[twilit smelt]

well i disagree with that

there IS a way to insist upon doing some ISR work in XNU and probably the others too

sometimes the needs of the device really do demand doing it asap

not even waiting for the scheduler to bring in that thread

[cursive charm]

youtube just recommended me the new standup episode

thought this chapter title was funny enough to share

[cursive charm]

how does it work in XNU, out of curiosity?

its interrupt servicing, that is

[twilit smelt]

iirc you hook up an IOInterruptEventSource to an IOWorkLoop (which is a worker thread) and whenever the IOInterruptEventSource fires itll cause the IOWorkLoop to execute your callback

these are C++ objects

in IOKit

you can also set it up to call a callback directly in ISR context but i dont remember how you do that exactly

its similarly easy though

the docs are like "DONT DO THAT! some people maybe HAVE to and they know who they are. if youre not sure whether you have to then DONT DO IT!"

[nocturne nymph]

It tracks DPC routines, not individual DPC objects

The long DPC flag is cleared as the queue drains

[shadow ridge]

https://www.youtube.com/watch?v=znRHdCDw9tQ

update to the casey thing

from primeslop

i just dont understand why theyre measuring this in FPS

[twilit smelt]

deeply enjoying these initial drafts of the alpha article for the digital technical journal

'remebered'

a shame that the CHM doesnt seem to have this 300 page VMS design document

[acoustic sparrow]

lol

[twilit smelt]

peak hand drawn diagram

didnt even fucking try

[dense vigil]

The x86 rings graphic from wikipedia

[twilit smelt]

good slogan

second attempt at the diagram

this is awesome

this is someones like personal files they donated so it has stuff like this in it

[twilit smelt]

"Alpha AXP will outlive all existing computing architectures"

if only man...

everything being Alpha would be an improvement

ooo

[coarse current]

meanwhile x86

[twilit smelt]

this is exactly what happened

[delicate plover]

Literally cockroach isa

[twilit smelt]

DEC went away and Alpha's microarchitectural technology went to AMD

[twilit smelt]

even intel didnt want it to survive

tried to kill it three times

[delicate plover]

Unfortunately IBM and everyone who likes upgrading computers said nay

[twilit smelt]

IBM didnt have a say

the clearest shot intel had at killing it was the i860

[delicate plover]

I think they did for the attempts in the 80's lol

[twilit smelt]

if the i860 had been a good ISA then it would have won and become the PC architecture of the future, everyone was on board

it failed bc VIVT cache

[delicate plover]

x86's operating environment is unusual though which might explain its longevity. It's the only isa where consumers with no budget can put a hard drive from one computer into another and expect it to boot without major issues. It's had a lot of pressure to keep all of its legacy cruft and to continue existing for the backwards compatibility even though its terrible

[twilit smelt]

it was still replaceable in 1989

primarily because the planned system software (NT OS/2) was going to feature 16 bit x86 emulation for DOS programs

[delicate plover]

Didn't that end up being butt-slow though?

[twilit smelt]

no

they did a study and emulating x86 on the MIPS R3000 was only like 20% slower than running it natively on the 286 which was well within the bounds of acceptability

and the i860 was much much faster than the MIPS R3000

and would have probably run 286 machine code faster than a real 286

(almost every PC user had a 286)

[delicate plover]

I remember reading that the i860 was like comedically fast iirc? It's just that it needed perfectly-optimized code which wasn't practical for the time?

[twilit smelt]

it was the fastest chip the year it came out

and the issue was not that it needed perfectly optimized code

it was a very conventional RISC ISA

the issue was the VIVT dcache

[delicate plover]

I see

[warm mural]

i860, Itanium and ?

[twilit smelt]

iapx 432

the one whose instructions werent byte aligned

and which had an "object oriented processor architecture"

[warm mural]

mhm

Wonder if Itanium could've ever picked up

[twilit smelt]

despite being a bitcode it still had extremely bad code density

[twilit smelt]

in the absence of amd64 it would have

amd64 outcompeted it in the "replace x86" space

[warm mural]

I think they had cool ideas

But idk if the entire thing was really good

Like the stack engine thing is neat

[twilit smelt]

it was considered a fairly good architecturá

there was a paper i read where they showed you could do faster L4 IPC on itanium than on any other extant architecture

they beat Alpha which was considered the generally fastest at the time

[warm mural]

I think the issue was with compilers?

[twilit smelt]

yeah which eventually got way better but only after it was too late

another major issue was that the first itanium chip was delayed by like a year

without a years worth of performance improvements to show for it, it was just because their design wasnt working yet and needed another year to be made to work

they needed more generations to catch up

which they did

but it was too late

[warm mural]

"In short, Intel tried to make a revolutionary leap with the IA64 architecture, and AMD made an evolutionary step with x86-64"

This sums it up pretty well

[twilit smelt]

intel had their own x86-64 project in the 90s which was cancelled

[warm mural]

I don't really buy in the "it failed because it was poorly designed" thing though

Like surely Intel engineers are not idiots

[twilit smelt]

idk if any of intel's original 64 bit x86's design documents are out there in the same way we r extremely fortunate to seemingly have every tiny piece of internal correspondence and documentation for DEC Alpha's development and history

would be interesting to read

see an alternate history for what 64 bit x86 could have been

[warm mural]

When will you start selling XR CPUs commercially

Can't be worse than risc v

[twilit smelt]

we have a higher percentage of documentation for dec alpha's creation than we do for xr17032's creation because of the Great Deletion of the OG Mintia Thread

[warm mural]

Maybe there was a great purge at Intel too

[twilit smelt]

recently i was like "wow its lucky my autistic hyperfixation has like 5,000,000 digitized documents, more than maybe any other chapter of that era of computer history"

before realizing i probably got it backwards

its my autistic hyperfixation BECAUSE there are 5,000,000 digitized documents

my earliest mention of mica in here

me being based as usual

[warm mural]

Is it fair to say NT is as close to VMS as BSDs are

Notably BSDs have ASTs and SPL

[twilit smelt]

no its closer

ive softened on my 2020 position that 'NT is not VMS-like'

it is VMS-like

not to the degree people want to say it is though

Mach having a greater overall influence on its design than VMS is probably accurate

[warm mural]

I was pretty surprised to learn BSD was directly influenced by it though which is cool

[twilit smelt]

well ofc

the idea that these things are completely separated belongs in the land of mythology, everyone would study everything they could about what everyone else was doing

i found a fuckload of dec documents on the chm website from the late 70s looking at unix and C and trying to figure out whether they should make use of it

they ended up deciding it was too shit

[warm mural]

yeah right

[twilit smelt]

https://www.computerhistory.org/collections/catalog/102737328

the second one is from the 70s

[twilit smelt]

lmfao

[twilit smelt]

@warm pine have we ever looked into datakit?

the datakit paper shows up in this archive

it was a STREAMS alternative

er

i guess, it was a piece of hardware whose original driver software was much like STREAMS

its sorta conflated together in this paper

[warm pine]

not at all, at least i haven't

[twilit smelt]

unix consistently got owned by VMS on benchmarks lol
context switching benchmark:

sequential page access benchmark:

random page access benchmark:

its funny how you never hear about this, that the clock algorithm was ass from the start

sequential file IO benchmark:

random file IO benchmark:

the algorithms causing bottlenecks here btw were essentially never changed in mainline unix thru to svr4 in the late 80s/early 90s

the commercial unix forks had to fix them

as a matter of fact the clock algorithm would get glazed in textbooks a lot

its still taught as like a baby's first page replacement algorithm due to the pedagogical history of using like 80s unix for such courses

even though it always STUNK

was well known to perform worse than VMS's much lower overhead algorithm

oh yeah i forgot he had a C compiler side quest

its funny he did that considering basically every C compiler at the time was doing really naive template-based code gen with just a couple peephole passes for optimization

hes basically like "i showed you FREAKS how to write a REAL kernel and now ill show you how to write a REAL compiler"

"even though i also dont know yet and i need to also learn how to do it and then write a book about the process which was often skill issued"

good book btw

[warm pine]

at this point i suspect working set based aging by directly aging the PTEs sequentially might be the single most optimal algorithm when you consider cache dynamics

[twilit smelt]

this book is like a narrative about the process of learning how to write a vaguely non-shitty compiler

i like the part where they force their intern to drive his truck like 100 miles every day with magtape in the back so that they could use a frontend running on a distant Multics system to generate IR to feed into their backend running on their local VAX, before they had their own frontend going

and what "internet" existed at the time was far too slow compared to just having their intern physically ferry the data back and forth

they actually did have an internet connection to the remote multics machine but it wasnt practical for this purpose

[shadow ridge]

it is still an internet connection

technically it's just IPv4 implemented over an intern

or v1 or whatever existed at that time

[twilit smelt]

i woke up at 3am after a few hours of sleep bc i was dehydrated

then i accidentaly knocked my glasses behind my bedside table while reaching around for them so i could get up and drink some water

and they got all dusty

so i had to get up and wash them

and then i sat down at my desk

and now its 5am

and im reading more ancient internal DEC stuff

i hope i run out soon im like fixated on it and havent been able to get anything else done for several days

wish i didnt notice how much the CHM digitized

i didnt know it could become painful to read these

theyve been busy

theyre basically doing all this for specifically me and like 3 boomers

[shadow ridge]

how did that take 2 hours

the fuck

[twilit smelt]

it didnt

it took a few min

but then i made the mistake of sitting down at my desk

[shadow ridge]

ah

[shadow ridge]

so it WAS you redigitizing tapes

i asked you this like 200000 times

and you never even acknowledged it

[twilit smelt]

this induced a like blinding white hot rage in my half-asleep state, the kind of irrational primal rage you can only feel in a hypnagogia

[shadow ridge]

do you also have that thing where you get extremely pissed when you wake up

just very irritable

[twilit smelt]

i half remember saying every swear word there is under my breath except for the slurs

[safe hollow]

@twilit smelt i just remembered about this

[twilit smelt]

no they are digitizing their entire collection (on their own, its their job) and have been in the process of doing so for several years now

i asked them to prioritize things in the past but i didnt ask them to prioritize any of this stuff, they just got to it over time

[twilit smelt]

do you remember what paper this is from

im 99% sure i read this before but i didnt really remember

[safe hollow]

Quality and speed in linear-scan register allocation

[twilit smelt]

also the bin-packing mentioned there came to DEC's awareness originally through Dave Cutler's C compiler project

which he got from the Multics compiler guy

whose name i forget

who invented it originally

he had a cool name iirc

Freiburghouse

Bob Freiburghouse

apparently cutler and crew would call him randomly to ask compiler questions to the point it was annoying and he would jokingly groan "what now" into the receiver after picking up

sounds like he basically taught DEC how to write compilers

which led to GEM

GEM has unfortunately been retired just recently

VSI wrote a GEM IL to LLVM IR translator (i think, they also might have just retooled the frontends to directly emit LLVM IR but that sounds like more work) and are now using LLVM for the backend for the DEC compilers on amd64, and GEM is still being maintained for Alpha and Itanium but will be discarded when those ports are finally discontinued (which will happen soon)

rather than trying to do a world class amd64 code generator and optimizer which they dont really have the resources for

i think VMS on x86 is now using ELF as the object & executable format too

which is just unnatural

but i understand the necessity

[twilit smelt]

their relationship with him started because i think they bought some compiler technology from him? i dont really remember

and he ended up going like wayyyy above and beyond the terms of the contract in offering personal phone support not just about the stuff they bought from him but also just about general compiler theory

for fun apparently

or perhaps because he was too polite to tell them to stop fucking calling him

lost to time...

[cursive charm]

for some reason, this reminded me to look over the old unix-vs-nt.org website again

in case you're wondering, that was an old website back in the '90s ran by a UNIX diehard who hated NT or something

said website no longer exists, but there was a link in the homepage to a piece authored by someone detailing "issues" with NT, iirc

[twilit smelt]

well NT did have issues

the object namespace was a huge missed opportunity imo

and i dont like how they did the interface between the kernel and the file system drivers

and also recursive IO enqueue

was bad

[cursive charm]

i just remember most of the complaints being directed towards system administration and the shell

think it was this piece: https://web.archive.org/web/20000815100517/http://www.unix-vs-nt.org/kirch/

[twilit smelt]

lmfao isnt this just corporate espionage

sending around an internal Bell Labs memo about the strengths and weaknesses of unix, their competitor, that says "the information contained herein is for the use of employees of Bell Labs and is not for publication"

from like 4 months earlier

how did they even get this

[cursive charm]

from the "reliability" section:

What causes "blue screens" in NT varies. In my own experience, the following can induce this state of failure:

• When both IPX/SPX and TCP/IP protocols are used and technicians put a machine with a static IP address on a different subnet;
• When some 16-bit Visual Basic applications are not being run in "separate memory space." NT does not run them in separate memory space by default. This is a manual configuration which should be set for each and every 16-bit application on the machine;
• Certain brands of memory modules or cache will induce this, even though the same hardware runs fine under other operating systems, such as Windows 95.
  why do they think the second item is caused directly by 16-bit programs running under the same VDM?

[twilit smelt]

its funny how this is nonsense

that list is the equivalent of like an ancient greek soothsayer trying to explain why the rains did not come last season

[tame phoenix]

sure sure it will outlive everything

there's so much cope in these docs

[twilit smelt]

i mean it was the best architecture with the best implementations

the hopium was that it wasnt too late after all (it was)

the PRISM cancellation caused them to have this 2 years later than they otherwise would have

which put the nail in DEC's coffin

[cursive charm]

that entire section, by the way, acts as if NT is like 9x in every way

and in regards to UNIX with their kernel panics:

Although I am sure that UNIX servers do crash on occasion, these are extremely rare events. If and when a UNIX server crashes, it is almost always due to a hardware failure of some sort. Any software induced problems in a UNIX environment generally make themselves known over a period of time, sometimes in the form of overall gradual performance degradation of the system, giving the administrator ample time to track down the source of the problem, correct it, and stop/restart the process (very rarely the entire machine!) causing the problem.

[tame phoenix]

doesnt really mean much though does it, best architecture doesn't mean it wins as we realise by the fact our desktops aren't all arm based... it's about who markets the best and is cheapest to integrate

[twilit smelt]

which would have been them if upper management didnt cancel PRISM

forcing them to do it from scratch all over again later when that ended up being a mistake

i disagree with laughing at them bc the engineers did by all accounts an EXCEPTIONAL job and these are mostly docs by representatives of the engineering groups

they got screwed over by management

and this is them trying to do their best to pick up the pieces

[tame phoenix]

but what was licensing going to be like if you wanted this arch in your "pc"? would they have IBM'd themselves to death and been hostile of compatibles?

[cursive charm]

very next thing is a list that somehow doesn't apply to NT as well:

In general, a UNIX server is halted only in the following situations:

• Due to a hardware failure, for instance, a hard drive fails;
• A hardware upgrade needs to be performed;
• A lengthy power outage has occurred and the backup power supply resources have been exhausted;
• The kernel is being upgraded.
• A beta kernel is being tested (not recommended for production environments).

[twilit smelt]

they tried to put it in PCs

there were high end Alpha-based PCs

if theyd done that with PRISM in 1990 which was during the time period where people were still hoping to ditch x86 soon then it might have been a wild success

[tame phoenix]

security vulnerabilities? not mentioned at all? was it assumed 100% secure and all vulnerabilitirs would be caught by kernel upgrades?

[twilit smelt]

bc they would have been the only RISC vendor with a high quality implementation and high volume, positioned to immediately put them in millions of PCs

MIPS tried to do this but didnt have the volume

DEC had the volume

[cursive charm]

vulnerabilities were only mentioned in regards to NT

[twilit smelt]

i860 had high volume and good positioning but was low quality, so it failed to replace x86 even though it was intel's own attempt to kill its child

[tame phoenix]

so they thought it was a non issue for them ok

"nt bad unix good"

[cursive charm]

they put up this quote from tim newsham with everything up to the last sentence making it seem like UNIX never gets DDOS'd???:

In the BSOD section you mention a few ways that a BSOD can be caused. I think this (small) list is misleading to the reader. There are so many ways that an NT system can crash, that by listing a small number you are likely to give the wrong impression. More dangerous yet is the fact that your cases mostly involve a person who is on the console doing something BAD to cause a crash. Many of the ways to crash an NT system happen inadvertently in the day-to-day operation of the system (indeed, leaving the system on too long while running a myriad of applications can cause bizarre crashes with little clue to their cause). Additionally malicious users can trigger crashes due to shoddy implementation in software modules such as the login program (LSA) or the tcp/ip stack.

[twilit smelt]

when i read these (and other similar docs from other companies) i dont go "HA what IDIOTS they didnt know x86 would live forever!" because that was an absolutely insane and unforeseeable outcome that literally nobody wanted or was expecting, including Intel and the PC manufacturers

we unironically could all be running PRISM on our PCs rn if it wasnt cancelled in june '88

or if the i860 didnt use a VIVT dcache, we'd be on that

x86 got extremely lucky lol

up until like 1992/1993 you will find roughly zero people who think x86 is going to live, the question being asked by everybody was which RISC architecture would win the day after x86's inevitable demise

[tame phoenix]

x86 was just what was in the thing everyone copied

[twilit smelt]

there were conscious attempts to replace it

[tame phoenix]

and IBM didnt like it, tried to kill compatibles and pc clones

but they failed

[twilit smelt]

the NT kernel is the child of Microsoft's end of Intel's project to do so

they were like "ahh x86 is going away we need a portable kernel"

"microsoft make us a portable kernel for a portable OS to run on the future RISC PCs plox"

[cursive charm]

i am curious about this part, though:

[twilit smelt]

IBM is like out of the picture by this point

irrelevant

x86 was never intended to last decades

by anyone

until like

1993

which is pretty darn late

and even then it was a niche belief that it would win out in the end

because the RISCs were still beating it technically to a degree that seemed like it would lead to an inevitable need to replace it

like ur talking like the PC manufacturers in the late 80s were fully bought in to x86 as like the forever-chip which would need some herculean miracle of God to unseat

which just isnt true at all

they were EAGER to replace it

there was just nothing to replace it with and nothing to run on the replacements yet

hence the projects to create chips to replace it with and create an OS to run on the replacements

which could simulate everybody's 286 programs with software simulation that ran faster than an actual 286, to have backwards compatibility

was the plan

(NT OS/2)

Intel tried to preempt competitors by offering the highest-volume, best positioned RISC chip to replace x86 with, the i860

which ended up having insurmountable technical flaws

it was the crummiest RISC architecture of them all, for general purpose computing

(great in embedded however)

if PRISM came out in 1990 that would have been the obvious alternative to the i860

in our timeline the i860 had no real alternative

MIPSco which was basically a startup didnt have the volume to put their chips in millions of PCs within a couple years

[cursive charm]

i guess the MIPS and PPC ports were already dropped?

[twilit smelt]

still, there were plans (by MS and PC manufacturers) to try to do the RISC PC with MIPS instead after the i860 didnt pan out. but it took so long that x86 ended up picking up steam (to EVERYBODY's surprise) and interest was lost in the RISC PC idea

if a giant like DEC had come out with a great and high volume chip in 1990 (and all evidence says they would have, it would have been essentially Alpha but 2 years early) then who knows

[cursive charm]

also, why did everyone think "variety of UNIX vendors" meant anything when most, if not all, people eventually ended up on linux, lol

[twilit smelt]

linux was like

mid osdev discord hobby kernel tier of bad

[cursive charm]

john kirch's article gives the complete opposite impression, believe it or not

there's a ton of linux praise throughout

in 1998, nonetheless

[twilit smelt]

it was fine for like personal usage but was not suitable yet for srs bsns

[cursive charm]

*1999

[twilit smelt]

yeah it didnt stop hideously sucking until like the '00s

and only because a couple companies put actually good kernel devs on the job full time

reassigned from teams like AIX and whatever

to go teach the linux troglodytes some actual technology

giving them fire

[cursive charm]

"unofficially?"

also, i hate to ask what the management vs. sysadmin wars of '99 would've looked like

[twilit smelt]

its just not true that linux performed better than NT lol

basically every actual benchmark from the time shows NT performing better on every task

mostly because it had an actual memory management algorithm

it scaled much better too

to more processors and more memory

and bigger processes

[cursive charm]

they do link to an article covering one such benchmark, but they're like, "oh, it's microsoft-sponsored, so we can't trust it!!!"

[tame phoenix]

.... did they have their heads in the sand then or something

pc compatibles were huge all through the late 80s and early 90s

[cursive charm]

some others reference coverages of benchmarks stating "NT trails solaris" and "UNIX trounces windows NT in testing," but not much for linux?

[twilit smelt]

IBM was not important for the RISC PC picture

[cursive charm]

"UNIX trounces winNT" is this one from stephen shankland of CNET, btw: https://web.archive.org/web/19990117020740/http://www.news.com/News/Item/0,4,29416,00.html

no mention of linux anywhere

[twilit smelt]

they only controlled a 20% share by like '89 when the RISC PC was picking up steam as a thing Intel, MS, and the other PC manufacturers were colluding on; and they didnt have a feasible architecture of their own for putting in a competing RISC PC until 1994 when that materialized as PowerMac

if the big RISC PC project (of which i860 and NT were both subprojects) panned out everybody would have moved to that leaving IBM with no supplier for PC chips aside from maybe themselves for ROMP or something

so theyd probably have moved their PCs to the same new RISC architecture, to avoid being kicked out of the market entirely

in fact this project was seen as such a sure thing that NT is literally named after the i860 lol

its codename (and popular name in the industry up until the marketing name 'i860' was given to it just before release) was N10

so NTOS just stands for "N10 OS"

ie the OS for the N10 (i860), these being two inseparable entities that will surely never be parted

[twilit smelt]

at one point on the OS/2 advocacy newsgroup someone went to a conference where microsoft was presenting on the NT project in late '91 and they took notes and posted them

and this was apparently the first time NT's feature set was shown to OS/2 groupies

and they all unanimously refused to believe it

they were like "Microsoft is blowing smoke. They seriously expect people to believe that 'NT', if its even real, has every good feature from every single UNIX system, and then some. Yeah right. This thing will never ship and if it does itll have maybe 10% of this at best"

[cursive charm]

LMAO

[twilit smelt]

they were calling it "Not There"

[cursive charm]

what is "(Pyramid)?"

[twilit smelt]

pyramid technologies

made unix multiprocessor machines originally with their own proprietary (unnamed) RISC architecture and then i think later they switched to MIPS

[cursive charm]

oh, okay

[twilit smelt]

source: some document in the bowels of bitsavers that i saw in like 2021

https://en.wikipedia.org/wiki/Pyramid_Technology

and also this wiki article which says the same thing apparently

dang i thought the mysterious unnamed risc architecture was deep lore in a bitsavers pdf from 1988

it was but its made less deep by the fact u can find it wikipedia skimming...

[cursive charm]

i don't know if i'd believe this anecdote they got from the unnamed sysadmin

there's so much bias throughout that it really makes it hard to trust everything being said

like here:

As for the claim that UNIX is behind the times, it is still the operating system of choice for science, engineering, research, and higher education. Most engineers would choose UNIX over NT without hesitation.

right after this is a quote about linux, though

[twilit smelt]

the commercial unixes were not behind the times

NT was extragalactic when it came out in 1993 but by the late 90s the others had largely caught up in general feature set and surpassed it in specialized areas like NUMA

[cursive charm]

and it's the sort of "linux is better than [insert X here]":

did the author really like linux or something?

[cursive charm]

that was under one of the "common misconceptions" the author debunks apparently labelled:

UNIX is this outdated, cryptic, command-line based operating system.

[twilit smelt]

oh im talking about kernel technology

it absolutely was outdated in UX and cryptic. CDE was dogshit

OSX was the first usable unix ever

[shadow ridge]

i wonder what would have happened if NT provided a Unix API

instead of winapi bs

[twilit smelt]

it did

[shadow ridge]

the posix thing was shit

and barely compatible enough

to qualify

[golden meteor]

nah it was actually pretty good

it was just Yet Another Unix Flavor you had to support

[cursive charm]

too bad the author disagrees:

Wrong! CDE (Common Desktop Environment) is a GUI desktop (Graphical User Interface: you use a mouse to point and click, or drag and drop on a colorful "desktop"; this is the basis for Microsoft's success.).
yes, the literal first sentence is just "well, CDE exists, so no u"

[twilit smelt]

lol jinx

edited simultaneous with that

[cursive charm]

LOL

[twilit smelt]

"(this is the basis for microsoft's success.)"

technically not wrong

but phrased in a way that tells me he doesnt know the history, probably

[cursive charm]

well, it was microsoft's (second?) success

so

??????

profit

technically, i guess DOS would be their second success

[twilit smelt]

also this would have been mica

cutler stayed on for 2 months after prism was cancelled, dutifully working on his new assignment to port mica to mips and retool it for sole use as the new ultrix/OSF kernel

until apparently he got in a heated argument with upper management where one of them said "You can either accept [thing he didnt like] or give us your resignation." expecting him to give in and apparently instead he resigned on the spot and stormed out of the building

never returned and left his personal belongings and everything

[cursive charm]

WHY DOES IT KEEP COMING BACK TO LINUX

[twilit smelt]

and raided his friends who just stopped showing up at decwest one after the other to go make a new workstation company

before getting preempted by ballmer

[cursive charm]

and then they had breakfast at a denny's or something according to davec

[warm pine]

i used to think this was wilful but apparently they had to deal with some bullshit from the posix specifiers

like the notion that "it is utterly inconcievable that any system should be posix compliant and it doesn't have a distinction between character and block devices"

[twilit smelt]

yeah its framed as willful by people with a chip on their shoulder towards microsoft

[warm pine]

which is my best recollection of a quote you found once

[twilit smelt]

yeah

that was said by a posix committee member to markl after he complained that one of their tests was unfair

[cursive charm]

that was from the source leaks

they had email convos between markl and the posix committee over various details of the spec

[twilit smelt]

it sounds like they had every intention of continuing the mica quest to show how to do a unix kernel right and come up with a world class unix implementation, before giving up like a year or two in

[cursive charm]

apparently, markl wasn't happy with parts of the spec

[warm pine]

so when that bs is accounted for it makes it quite understadandable that the authors of the posix subsystem would give up trying to appease them (the character v.s. block device distinction is an artefact of how the ancient unix buffer cache worked, and it's gone from several modern unixes, while others contort themselves to provide it)

[cursive charm]

he was the one who wrote the posix subsystem with some other person, iirc

[twilit smelt]

unfortunately this also implies that the project to make mica into a new kernel for OSF/1 might have failed or been very disappointing lol

they might just not have known how to do it

[warm pine]

it's a pity they didn't get very far there

[shadow ridge]

do you think it would have competed with linux

it wasnt open source until recently right

[twilit smelt]

it would have gone the same way as the other commercial unixes

what wasnt

[shadow ridge]

mica

[twilit smelt]

mica still is not open source

wdym

[shadow ridge]

i thought they released the source

am i thinking of something else

[twilit smelt]

no

mica source is lost

currently

and maybe forever

[warm pine]

but INTERIX on the other hand, which i think was adopted as the Windows Subsystem for Unix Applications much later, really is a competent unix, as far as i know, it could run X11 and all sorts. i would like to learn more about the challenges they faced with that

[twilit smelt]

the closest we have is ozix sources turning up in advfs

ozix was basically a continuation of using the same concepts as mica for an OSF/1 kernel contender

but written in C and with crazy ass stuff on top

they started it in 1989 and by 1991 it was cancelled

its only piece that shipped was advfs which ended up in OSF/1 (aka Tru64 UNIX) which ended up being based on Mach

[cursive charm]

okay, it was ellen aycock-wright who was the other author

apparently, a couple of others, steve wood and matthew bradburn, were also involved

[twilit smelt]

advfs was open sourced later

and seems to contain some pieces of ozix's IO subsystem

which it was apparently easier to graft onto mach as a kernel module than to retool the filesystem code to use mach's (BSD) IO subsystem

so that is the closest we've ever seen to mica sources and we may never see closer

we learned all this from documents the CHM digitized 4 me

it was unknown that advfs was originally an ozix component previously, until it came up in those documents

and after we looked at it we were like oh yeah this is grafted into here from another completely unique kernel

there were like #defines from ozix names for things to mach names for things

and whatever

[cursive charm]

stephen walli wrote a two-parter blog detailing their time working on interix: https://medium.com/@stephenrwalli/running-linux-apps-on-windows-and-other-stupid-human-tricks-part-i-acbf5a474532
https://medium.com/@stephenrwalli/running-linux-apps-on-windows-and-other-stupid-human-tricks-part-ii-c244b2ee535

both links taken from the interix wikipedia article, for the record

[twilit smelt]

its funny you can just take another kernel's IO subsystem and plop it in w/ ur filesystem driver and somehow it works

dont know what they meant by that

or how

theres like generic IO request enqueue code in there

and whatever

how are the storage drivers implemented... how are they calling into them with this

did they take ozix's storage drivers too?

we do not know

perhaps theres like a 'driver development for tru64' guide somewhere with clues

[cursive charm]

should also note that interix (originally opennt) extended from the original posix subsystem

that is, softway systems licensed source code to the subsystem from microsoft in order to make their own

[warm pine]

thanks, these ought to be a fun read

[cursive charm]

technically, they ended up doing a better attempt than markl's subsystem, but i guess MS stopped caring once they purchased softway

[warm pine]

the BS module is presumably adapted direct from ozix

[cursive charm]

last version of interix was in srv2012 and enterprise editions of win8, btw

[twilit smelt]

we need to annoyingly email the ppl who worked on ozix to see if any of that code is still around

[cursive charm]

notably, MS never bothered updating interix to the latest standards after they included it with the base system in vista

[twilit smelt]

imagine theyre like "oh yeah we finished the kernel and got it booting on DECstation to a full unix shell with utilities, heres a disk image"

[cursive charm]

there was no posix_spawn(), for example

never understood why

it would've made perfect sense for interix, but no

you gotta do a fork() and then exec()

i guess there was vfork() at least

if it wasn't already obvious, i'm kinda annoyed MS stopped maintaining interix, lol

cygwin and its brethren, msys2, are so shit, and WSL is kinda fine until you have to pass something over to the guest fs

it would've been fine had interix still been an option

[twilit smelt]

"poste haste"

bro had an argument to win

[cursive charm]

i wonder what OS/2 advocates thought of UNIX

[twilit smelt]

foreign strange and scary OS for bigger computers than they had

[cursive charm]

linux?

[twilit smelt]

they didnt think about linux

at least not until OS/2 was dead and the only advocates left were unhinged wackjobs

linux was not in the consciousness of a home computer user concerned about OS/2, even a tech-savvy one on usenet, in like 1995

its relevance is overblown in retrospect by its early advocates who became loud voices later

[cursive charm]

have you seen the OS/2 vs. NT shootout video before

what even was the story behind that

[twilit smelt]

there used to be such a thing as user groups

which were self-organized groups of users

that would have physical irl meetings

and had the clout to have direct liaisons with big companies

[cursive charm]

was it really out of usenet meetings

[twilit smelt]

that was a user group inviting a representative from MS and a representative from IBM to have a shoot-out at their monthly meeting

and they both came

bc neither of them wanted to look like a pussy

to the users

deciding between their OSes

[cursive charm]

i thought it was gonna be a little more than just user groups inviting reps

[twilit smelt]

no it was conceived and organized by the user group

HAL-PC

"Houston Area League of PC Users"

dissolved in 2014

there was a really powerful user group of DEC machine users called DECUS also

they come up constantly in these internal DEC documents ive been reading

[cursive charm]

idk why i said usenet here

[twilit smelt]

they had substantial sway over DEC and in return basically acted as a volunteer QA and field work force lol

user groups stopped being a thing bc of the internet

everyone became a member of one giga user group

unfortunately this giga user group is much more ignoreable than the previous ones

where ppl like paid dues and would go to irl meetings and had the power to summon reps from big tech companies to give an account of their work

if antennagate happened in like 1990 there would have been an Apple guy at every local Macintosh User Group meeting offering a personal apology

[cursive charm]

where did this claim even come from?

[twilit smelt]

its either a fabricated tale or has a pearl of truth that like MS Legal was being annoying about the copyright on that section

of the tape

which is easy to interpret as "theyre insecure" but could also just be overactive lawyers

seeing an opportunity for a license fee

"pay us for a license to the copyright for the slides for this section or you cant include this section in your tape."

[cursive charm]

i assumed it was because the video is titled "(uncut)"

but it's called that because this is the upload of the tape that wasn't spliced into parts: https://youtu.be/-DAojx2Hgec

maybe the multiple-parter uploads are different in some way

idk

27:34 features this slide from doug davis' presentation, btw

what

i mean, i guess it was initially NT OS/2

[earnest zenith]

it's just people drinking the "Microsoft bad, OS/2 good" sauce

[dapper mulch]

According to Gordon Letwin, OS/2 begun its existence at Microsoft. I think he knows, as he was its primary architect. And it predicted correct that IBM will fail with what they were trying to do with it.

[cursive charm]

i was about to ask if they meant microsoft was borrowing from themselves somehow

[dapper mulch]

For microsoft hating fanatics everything is possible.

[twilit smelt]

current plans for mintia2 improvements are 🔥

should be interesting

people go like "errrmmmm instead of doing improvements why dont you just add more features" and thats because during times where i dont have time to work on it i build up lots of good ideas for improvements for how things are. and then when i DO have time it only takes a couple days to run thru that list

[digital pivot]

in which areas?

[twilit smelt]

now that numa mechanism is basically complete i can implement numa policy ie topology awareness in page frame allocation and thread scheduling

currently a thread can only allocate from/run on its affinity node, making it able to do so with other nodes requires topology awareness

additionally i wanted to redo how system virtual address space allocation is done

and redo the heap

currently there are 3 regions of system VA space (per node):

1. pool space: composed of fixed size 1 page chunks, allocated via a free stack implemented as a linked list going thru the PTEs themselves, used to allocate mapped heap extensions
2. dynamic space: composed of arbitrarily aligned blocks, allocated via a flat bitmap, used to allocate >page size heap blocks and stuff
3. cache space: composed of fixed size N page chunks, allocated via a free stack implemented as a linked list going thru the PTEs themselves, used to allocate mapped file viewcache windows (NYI)

and the block allocator is layered thusly:

1. per type magazine caches on top of...
2. per type slab caches on top of...
3. size class segmented free list based heap on top of...
4. the page frame allocator

and im going to change it so that instead there are 3 regions of system VA space (per node):

1. dynamic space: composed of naturally aligned blocks, allocated via a O(log n) hierarchical bitmap or LLFree type algorithm (depending how much i want to tryhard scalability) that tries to pack allocations to the bottom of the space for reasons that will be clear momentarily
2. cache space: same as before
3. metadata space: composed of a virtual array of per-page MmpDynamicSpaceMetadata structures, sized large enough to contain as many such structures as the maximum number of pages of dynamic space

and the block allocator will be layered thusly:

1. per size class magazine caches on top of...
2. per size class slab caches on top of...
3. the page frame allocator

dynamic space block metadata, for example for a multi-page slab (or a single page slab), will go in the sparsely mapped metadata array which is indexed by page number

when a dynamic space allocation is made, the corresponding phys pages of metadata space (containing all the metadata blocks for each page of the returned dynamic space range) have a refcount in their pfndbe incremented. the ones that dont exist yet are mapped

hence the motivation for packing the virtual allocations to the start of the space

to minimize the number of metadata pages

[icy bridge]

how are you planning to handle the VA space for cached structures like inodes? it's not inconceivable that they might take up the full pool va space, especially if you're on a 32 bit system with a lot of physical memory, and if you then try to reclaim them to get pool va space back you might get significant fragmentation

not to mention that such structures sometimes wouldn't be evictable and would need to be counted for strict accounting, for example if you have an fd open you can't really evict the inode it's pointing to

[twilit smelt]

for this to fill up VA space you would need like hundreds of thousands of different files open simultaneously

and then youll run out of resources as youd expect you would if you have insufficient memory for that task

and opens will start failing

[icy bridge]

I'm mostly worried about the fragmentation

like if you open millions of different files and close every other one

[twilit smelt]

name me a kernel where that wouldnt fragment the fuck out of it lol

i think solaris does the most to try to fix fragmentation with compaction callbacks and stuff and i think even there you would get this problem

bc i dont think it can compact open vnodes

[icy bridge]

I'm not saying the fragmentation needs to be prevented, more that it needs to be made harmless

[blissful smelt]

page compaction ?

[icy bridge]

for example imagine a 32-bit x86 pae system with like 64G of ram, and your pool space is maybe 256M at most. opening enough inodes (and keeping them open) to fill va space wouldn't take a significant portion of phys mem, but it would deny all future kernel allocations

[twilit smelt]

there are quota limits that would stop you from doing this

[icy bridge]

fair enough ig

[twilit smelt]

assuming ur not a like sys op

[icy bridge]

idk it just feels like a dos vector

[twilit smelt]

as a normal user youd run out of disk quota trying to create that many files

or just exhaust on-disk inodes even

potentially

and trying to open them would result in running into a per-user open file limit

thx for reminding me to add an OpenFileLimit to the MmpQuotaBlock

almost forgot that

unsure why its not a PspQuotaBlock

prob should be

might move it

[hollow musk]

if a DPC crosses the threshold, the kernel moves it to the delegate thread, this will also set a flag that will likely be read when swapping to another thread, where it should be reset

[shadow ridge]

@twilit smelt what did you use to write the manuals

for xrstation

[twilit smelt]

WHY?

TELL ME...

[shadow ridge]

I realized I don't want to do x86 64 kernel dev

[golden meteor]

LaTeX :^)

[shadow ridge]

I'd much rather just continue my old project

[twilit smelt]

typst

[shadow ridge]

and I want to start by documenting my ISA first

so I can write an emulator and assembler

I'll also write a little kernel in assembly and then write a new compiler + kernel entirely within the emulator

[shadow ridge]

like I just dont see the fun, I'm just doing the things I did 5 years ago all over again

[shadow ridge]

thanks!!

[golden meteor]

I can't believe you would cheat on LaTeX

[twilit smelt]

i hate latex

it is evil

[golden meteor]

that's why I love it

[shadow ridge]

you're evil arent you

you're a nasty little sewage boy controlling the narrative from the shadows

[golden meteor]

nah I just like it when I can say I'm smarter than people

[twilit smelt]

typst is easier to Just Write A Document with

but they are about equal difficulty to do Fancy Document with

[shadow ridge]

what font do u use btw

[twilit smelt]

EB Garamond

the Apple corporate font from the 90s

or rather an open source version of it

[shadow ridge]

thx but i think ill just use Hack

ive been using this font everywhere for like a decade now

[dense vigil]

Noooo

Not the deadlock again

[shadow ridge]

dont worry this time I'm not making a compiler first

I'll write up a quick emulator

and just do everything in assembly

my goal this time is to just get up basic firmware and a kernel running asap

I'll alternate between this project and my game project every 2 days so I'll also release my game on steam in time

[twilit smelt]

if someone put me back to my 15 year old self and i had all the knowledge of vaguely what i had accomplished over the next 8 yrs on my project but none of the wisdom or technical experience or knowhow and was like "okay do it again." id take myself out back to be shot

[shadow ridge]

I had stopped working on my game for the past few days cuz I had given up on putting on steam but my family said they'll support me even if I ended up not making a profit and getting in debt

so I'll continue that as well

[shadow ridge]

and what if you kept all your knowledge and experience

would you still go through the trouble of

building your tech stack again

[twilit smelt]

yes i would take the opportunity to like materialize it in its final state out of thin air by like age 18

so that i could look like an incomprehensible prodigy

but if i had to learn again no i dont think i would it just would sound too obnoxious

i only got through it because at every step i didnt think i had that long left to go

if i knew how long it would take i would have done something else

[shadow ridge]

you could be an AI tech bro

grifting on twitter

holy shit i want to see that timeline

..or not

[twilit smelt]

if i got hyperfixated on ML instead of OS kernels when i was like 13 i might be making like $500k rn

[shadow ridge]

you would be funding the new LimnGPT data center in utah

by now

[twilit smelt]

i dont think i would even be that aware of the ongoing debates and whatever i think id probably just be keeping to myself

[warm pine]

you can't really expect something like that out any 18-year-old

in the end even the most intelligent prodigy has to come up against the limiting factors of time and experience

maybe the likes of von Neumann suceed anyway but the cost is an impoverished childhood

though i'm sure he didn't care an ounce about what he missed out on

[twilit smelt]

i was saying like if u set me back at age 15 with all the knowledge and wisdom and experience i have rn i would recreate my project from scratch just to look like an einstein level genius

to do that in 3 years apparently from thin air

its a lot less impressive when its like 8 years of research and study and trial and error

than doing that in a straight line in 3 years

[warm pine]

oh, that's another matter

[twilit smelt]

would b funny

[warm pine]

it's a classic fantasy really

everyone likes to imagine being put back into their body as a teenager but with all their knowledge

but when i think of that my first thought is usually "this time i won't sell all my bitcoins for $75 each"

[twilit smelt]

i would go back in time to buy 10,000 btc for a couple bucks in 2010 just so that i could then sell them for a $10 profit

so that i could tell people over and over later

"i had 10000 btc and i sold it for $10..."

which seems to give more fulfillment and joy than having the 10000 btc would today

there was a mc server i played where in 2012 some guys would sell diamonds for 1btc per stack of diamonds

they are now, apparently, millionaires

[shadow ridge]

lmao

[twilit smelt]

it was even better pay than that guy thought

[shadow ridge]

holy shit dude

[twilit smelt]

https://www.youtube.com/watch?v=74NluS3jzTo

[warm pine]

absolutely mental that they are from cullen

i don't know what comparable american settlement to name to make the import of that intelligible to an american

[delicate plover]

What kinda city is it

(as an American I shall make an attempt)

[twilit smelt]

iirc they grew up in canada hence the name

were from cullen

but grew up in canada

[warm pine]

well that's just the thing

it isn't a city

[warm pine]

apparently only a year or two in canada

[delicate plover]

This low-key gives me Amish country vibes

Which would be pretty mental for a gang of experimental musicians to be from Amish country 😂

[twilit smelt]

there was a tweet made by someone in spain that got huge recently where they said (in spanish, auto-translated) "I believed that Boards of Canada was from Canada because I believed in the honesty of the common man."

[warm pine]

there is an american connection in that, like many scottish burghs, it was (the newer part of it) designed by luminaries of the age of enlightenment (and most people agree that ours was one of the best enlightenments), who chose to build it to a grid shape, and americans liked the idea and borrowed it

[warm pine]

just wait until they find out where the Utah Saints were from

[shadow ridge]

@twilit smelt how would you change the stack pointer in the kernel in say an interrupt handler if the architecture doesnt define a "stack" or instructions for manipulating it

the only way I can think of is making R0 a zero register like you have

but allowing it to be used as a scratch register if a bit in a random CR is set

so you can use it to store the interrupting code's stack pointer while you swap it out for example

[warm mural]

if it doesnt define a stack then dont make interrupt handlers use the "stack"

[twilit smelt]

found out recently the only other language in the family of the language of the Aztec Empire is... the Ute language

they r cousins

[shadow ridge]

yes but the kernel code needs to set up a stack anyways

[twilit smelt]

a long time ago they split into 2 groups and one of them went south and became emperors and conqured shit and were glorious

[shadow ridge]

even if it doesnt exist in the arch

[twilit smelt]

and the other stayed in... utah

cuz they love utah so much or something

[twilit smelt]

i have scratch CRs that i use for this

[shadow ridge]

i see

i think ill do something like

what i explained

actually wait i cant

it would be a RMW operation

ok ill just use scratch registers as well

thx

[warm mural]

what advantages does not specifying a stack pointer have

[shadow ridge]

it's not necessary if your arch doesnt natively use the stack as like an active element in its execution

i dont have push or pop instructions

or anything that manipulates it

so i make a passing remark that R31 may be regarded by most software ABIs as a stack pointer

but it's not "official"

[warm mural]

why not have push/pop

idk i should learn ISA design cuz i think if I were to invent something new it'd be very x86y

[shadow ridge]

why do have it?

there isnt much to be gained from it imo

and i hate x86 for making the stack so important

like did someone have a stack fetish at intel

[twilit smelt]

simplifies

i mean you do have a stack pointer but its just conventionally specified in the ABI

the processor doesnt care

aside from like

sometimes there are microarchitectural caches and stuff

that are aware of which register is used by the software as the stack pointer

like the stack cache

which is used for predicting return branches

[warm mural]

also typst mentioned, I love typst

idk how people think latex is sane

[shadow ridge]

i thought of extending this philosophy to the link register as well

but like

idk if i want that

it might be the case that it is better to have it than not

[dense vigil]

I mean x86 doesn't have one

[shadow ridge]

yeah because it uses the stack

i dont have the stack either so

[dense vigil]

Yeah thats your only option

[shadow ridge]

yeah ig

[shadow ridge]

this is turning out really well @twilit smelt

thank you!!!

i was really bored of plaintext documentation

[dense vigil]

Me when I want to halt but accidentally disable the mmu

But also its kind of a bad design for virtualization, since such a complex control register will require vmexits whereas just halt on its own doesn't have to

But then again this arch won't have any virtualization anyway probably

[shadow ridge]

ok actually you're right this should just be an instruction

and not like a control register bit

[nocturne nymph]

I got the notification, but I didn't immediately understand what you meant. The way it's written there is definitely wrong.

The presence flag is set during enqueue when the target PRCB's run time history table says this deferred routine has previously run longer than the long DPC threshold. The DPC drain code just uses it as one reason to consider the delegate thread and it gets cleared when the queue is drained down to <=1 entries.

[hollow musk]

yeah there've been too many changes to the scheduler to remember everything

[twilit smelt]

https://raw.githubusercontent.com/xrarch/pdfs/main/xr17032handbook.pdf

restyling of the architecture manual

[warm mural]

this looks very nice

I prefer this to the blueish style

[twilit smelt]

the virtual memory chapter remains a mini textbook rather than a simple description of the architecture

which is on purpose bc i wanted ppl to be able to actually understand why the MMU is like that

[golden meteor]

idk I like the blueish one better. gives it more contrast

[twilit smelt]

this is less sickening to look at over extended periods

its also not super realistic for there to be color all over the place on an 80s architecture manual

[warm mural]

i think the blue does not really fit with the rest of the branding

[twilit smelt]

throwback to the limn2600 architecture handbook cover

the xr17032 book is a direct descendant of this one lol ive been editing it in place for over 5 years

it was google docs at the time

i think limn2700 is another name for xr17032

it was briefly called that b4 the great renaming

its the exact same instruction set and encoding as limn2600 but the kernel mode architecture was redesigned

so xr17032 is actually different from limn2600 but its not different from limn2700..

[golden meteor]

🍋

[twilit smelt]

limn2600 was the same encoding as limn2500 and i forget what the differences were

limn2500 was a total redesign from limn2000

in like early 2021

it was totally different it was like this

mintia1 originally targeted limn2000 and had to be ported

and we dont even TALK about limn1000...

which is from 2018

https://github.com/xrarch/mintia/blob/bf8e6fb8c164c8e1187a27486b1e2ec433e43624/OS/HAL/limn2k/HALLimn2kException.df

limn2000 exception handler in mintia1

yeah lets just throw in a single instruction that magically saves every single GPR starting from some base addr in a register

thats super realistic to have in a risc

[shadow ridge]

did i indirectly motivate you to do this

😭

[twilit smelt]

no ive been doing this for weeks

[shadow ridge]

ok so i just happened to write the docs for my thing

at a similar time

[twilit smelt]

ya

[shadow ridge]

your new bitfield stuff looks cool

i gotta do that too

my current bitfield tables suck

looks like some sort of extension to typst tbh

nvm it's a macro style thing

in config.typ

[twilit smelt]

not mine i stole it

@lucid umbra let me steal it

[blissful smelt]

30 is "a small set of control registers"?

[twilit smelt]

theres only like 13 with unique function

[mortal barn]

I'm not a huge RISC expert, but I do remember that PowerPC has stmw

[twilit smelt]

evil

genuinely wonder what conversation led to them adding that

i dont see the justification

maybe code size?

[mortal barn]

I think the primary usecase is exception handlers
Could be completely wrong on this, I'm tired and hungry, but iirc the vectors are like 16B apart? Not much wiggleroom for a bunch of code

At least, I know that's where I've seen it used most often

[twilit smelt]

if theyre that close together then theyre going to consist of like

a jump instruction

and thats it

xr17032 is superior bc its exception vectors are 256 bytes apart

so u can code golf to try to avoid that branch on certain paths

64 instructions

[mortal barn]

the form that ppcskel uses is

stmw 0, 0x2000(0)
li 3, <vector>
ba 0x2200

so, not quite just a jump instruction, it actually puts the stmw right there

[twilit smelt]

what is ppcskel

[mortal barn]

https://github.com/AndrewPiroli/ppcskel/

[twilit smelt]

draft virtual memory chapter i did for @sandwichman and friends' 64 bit aphelion architecture

this is (in function, not in form) what virtual memory would have looked like if i ever did a 64 bit version of xr17032 (which i probably wont because aphelion fills that niche)

[shadow ridge]

im also implementing my emulator

doing an extremely naive implementation to start things off tho

it's implemented as a library that provides the processor and nothing else

you register memory bus callbacks if you want anything of that nature

so the core itself wont actually handle tlb stuff

[twilit smelt]

that is epic but do u have any comments on the virtual memory chapter i wrote for aphelion architecture

then i will answer about ur cool emautlor

[shadow ridge]

that's also pretty epic (and much more featured than my arch lol)

i dont get why you show both binary rep and hex rep tho

here

[twilit smelt]

normally tlbs are part of the core though

[shadow ridge]

old cpu architectures go through the bus interface to access the tlb or the other caches

[twilit smelt]

how old

[shadow ridge]

intel pentium 4 / xeon

[twilit smelt]

i dont think thats true past like the vax 11/780

from 1978

mips r3000 in 1988 had on-die TLB and caches

as integral components, not connected over the bus

[shadow ridge]

i didnt look into it that deep tbh i just copied the intel design because it looked simpler, i dont want to think about cache coherency shit when im not there yet

[shadow ridge]

the bus here isnt the system bus

it's like a small unit in the die i think

[craggy spire]

"and i3 is the index into the third-level table"
i2?

[twilit smelt]

oop

[shadow ridge]

using 0 based index for page table levels is weird

[twilit smelt]

i like completely changed their original virtual memory chapter but kept some of the original verbiage and that was part of that

thats not my decision thats what they were doing already

[shadow ridge]

theyre weird

[twilit smelt]

but since they said i can do whatever i want i think i will renumber them

[shadow ridge]

anyways this all just means you need to write translate / read write / tlb invalidate callbacks

which is goated imo i can write several implementations on top of this

[twilit smelt]

the way i optimized tlb lookups in my cached interpreter would have forbidden this

i put a mini hash table inside the header of each cached block

(open addressed)

whose entries contained:

1. the matching part of a TLB entry (VPN and ASID)
2. a pointer to the TLB entry
3. a direct pointer to the mapped memory

and then upon executing a memory access instruction in that block, it'll look up the guest virtual address & current ASID in the hash table

if its there, then itll compare the cached TLB entry in the hash table entry, with the entry in the actual simulated TLB array which it was originally read from, through the pointer to it

if theyre still the same (meaning guest code didnt invalidate the TLB in the meanwhile), then the memory access can be performed directly through the host pointer

this can all be inlined and is quite fast

[shadow ridge]

thats actually pretty smart

i might do something similar when i write the jit for this

[twilit smelt]

leaving a pointer to my kernel allocation plans so that i can start working on thm latr

[balmy zealot]

To get a bit of inspiration for process resource accounting and ended up reading through the handle table code in ExHandleTable.jl. In the file ObHandle.jk, on line 105, you are not using the lock parameter. Why is that?

I can see that the HandleTableLock is associated with the process.

Without passing it, the parameter check in the EuCreateHandle function fails if there is insufficient memory. This means that MmAllocatePool may block whilst the table lock is held?

I'd be grateful if you could explain this protocol.

[twilit smelt]

the lock parameter is there for the PID table, which reuses the handle table structure

if a pointer to a lock is passed in, then ExuCreateHandle will unlock it before doing a memory allocation

this is needed for the PID table, to avoid blocking for free pages while holding the PID table lock

which could cause a deadlock with the thread reaper

because it needs to acquire the PID table lock in order to delete the thread's ID (which is also in the PID table)

the thread reaper is a memory-making thread, so it shouldn't block on a lock that someone else is holding while blocked for memory

because it can occur that the only way for free memory to appear is if the thread reaper can free up dead threads

(bc memory is saturated with dead threads)

note that the handle table structure as it currently exists is slated to go away

the PID/TID table will be replaced with a hash table, and allocation for each will be performed with a monotonically incrementing counter that wraps around 999999 or something

the hash table links will be intrusive so this problem will go away because no allocation will be required to insert

the handle table will also be redesigned as probably a multi level table

changes to handle table
    no longer use it for PIDs, use a monotonically incrementing counter and a hash table
        hash table has individually locked buckets.
        counter increments to a maximum like 99,999
        before using a PID allocated from the counter, check if it is already in the hash table
    use a multi level table for handles
        direct for first 12 handles, singly indirect for next 128, doubly indirect for next 16384

thats in my todo list

generally speaking holding any lock across a wait for pages, that can also be taken on a freeing path, is bad

a hazard

so, you may be right that as things are now, the handle table lock should be passed in to that parameter; because closing an object is a freeing path

& i will make that change since it seems simple

i wonder if in like 20 years there could actually be some value to this software ecosystem as being like the only bottom-up LLM-free one

and in 20 years it could acquire some usefulness beyond being a toy if it keeps being worked on for that long

who knos

[balmy zealot]

Thank you for the explanation

I look forward to hearing about future updates to this code

[twilit smelt]

i didnt come to my current full apprehension of this as a hard and fast rule until after i wrote that particular part, so

i dont think other kernels have that as a hard rule (never holding a lock across a wait for memory if it can also be taken on a freeing path), they just stop at making sure pageout deadlocks dont occur

which requires the less general rule of "never hold a lock across a wait for memory if it is also depended upon by the pageout machinery"

i think thats a mistake because it makes pageout the only way to get free memory in certain conditions when it may have otherwise been freed up naturally

like you COULD have gotten it just by this guy trying to free up some resources, but he blocked on this lock instead. and now he wont wake up until the guy who holds it is awoken by free memory made by the pageout thread

the fireworks test is actually what made me come up with this bc in the absence of a pageout thread, deadlocks caused by a lack of this rule will happen a lot while the fireworks test is running with low memory

[twilit smelt]

i like the part in zalloc.c in mach where they just give up on the nice naming scheme like zone_steal_memory and just go like

zinit

zcram

zfill

[hybrid condor]

what interesting things does windows do with the handle table?

[twilit smelt]

its also a scalability hazard in high memory pressure situations, which could cause very brief critical sections to become gigantic if you take a page wait

[twilit smelt]

last i heard its a multi level table

i dont think that it is lock-free but theyre possibly working on it seeing as theyve introduced RCU to NT

[cursive charm]

i hate how this reminded me of that OS design review thing where it argues that "everything is a file" is a better paradigm than NT's "everything is an object," and one of its points against NT in that regard was the handle lookup costs

[warm pine]

paradigmatically they are identical

the substantive difference between the nt handle namespace and the FD namespace is, as far as i know, just that NT has quite different rules

unix says a new FD has to be the lowest numbered which makes it hard to make FD table updating parallel

[cursive charm]

the quotes are really huge, so here's a snippet i thought was kind of amusing enough to save:

• I just don't like the vibe of kernel resources being "objects". It just feels wrong to me. Like I rather call ⁨pthread_join⁩ than ⁨WaitForSingleObject⁩ on a thread. To me, the latter feels overly abstract. Unix systems do not have a special system call for joining threads (not that there would necessarily be anything wrong with that), either. Instead, ⁨pthread_join⁩ is implemented in user space through the use of a futex. Cases like these show how Windows objects promote centralization of system components in the kernel.
  https://github.com/ElliotKillick/operating-system-design-review/blob/4459d385d6de333fce3f7b617708a3849f3e908f/data/windows/microsoft-complaints.md#dave-cutler-is-a-cringe-unix-hater

[warm pine]

this is just trite though

at a glance the whole article is

you could just as well have your threads on windows indicate exit to their peers with similar mechanisms

[cursive charm]

i honestly don't know how they refer to pthread_join() as an example of "everything is a file"

it isn't

it's really just WaitForSingleObject() but if it accepted only a single object type

[warm pine]

A file is the universal interface of Unix. The filesystem is like one big namespace everything in it is easily accessible through generic tools.
this is not what "everything is a file" means

the historic and accepted understanding is that "everything is a file" refers to file descriptors

having dirents and being able to readdir or ls is a whole other thing

[twilit smelt]

a thing im probably going to also do with mintia2 handles just so that i can have a 1:1 fd:handle correspondence for linux subsystem programs and dont need to like abstract the handle table with an fd table on top of it

[warm pine]

it's reasonable enough i think

so far it's hardly harmed linux to have serialised FD table updates

though some of the solutions now that some are upset about it are quite funny, like separate side FD tables for io uring

[cursive charm]

this really needs its own thread, but these last two bullet points from that section is really funny just for how trite it sounds:

• Dave Cutler could be taking "everything is a file" too literally, but I also want to know who these "grand poobahs of the academic world" are that Dave is referring to
• In the end, it is clear that Dave Cutler intentionally designed Windows NT with the goal of making it not Unix-like and in doing so he created an operating system with a completely backwards and hypocritical design

[warm pine]

it's just ahistorical nonsense