MINTIA (not vibecoded)

[twilit smelt]

i didnt catch that from seeing the scans and pictures of it

[flat dust]

microsoft did that with activation keys i think

[digital pivot]

what changed the most?

[flat dust]

or cdroms

the windows me disc had a bunch of pretty visual stuff

[warm mural]

new books are like 80$!

[flat dust]

they're pricing the books like video games now

:(

[hollow musk]

I don't even know what to say

scalability, topology awareness, power efficiency became first class scheduling input which is a big philosophical shift, timers evolved, priority inversion handling got much more sophisticated

[twilit smelt]

ignore this song playing in the background https://open.spotify.com/track/4TiMyX28DqJcIs427h9lQ3?si=c574936a372a4dca

[twilit smelt]

also fine grained locking

[hollow musk]

well, scalability

[twilit smelt]

until 2k3 (i think) the entire scheduler was under the big dispatcher lock

[mortal thunder]

vista actually https://web.archive.org/web/20101014182654/http://channel9.msdn.com/shows/Going+Deep/Arun-Kishan-Farewell-to-the-Windows-Kernel-Dispatcher-Lock/

the big dispatcher lock was split up in Windows 7

[twilit smelt]

that was when it completely disappeared

they had been splitting it up

earlier

[hollow musk]

It's not as deep as VMM, but it may be enough for an entire book, especially if you describe it chronologically

[mortal thunder]

oh okay

[warm mural]

yeah arun in the video says they started breaking it up earlier but the scheduler changes finally removed it or something

[hollow musk]

it may be funny, but this video was what interested me

[mortal thunder]

Prior to Windows 7 (and therefore Windows Server 2008 R2) the Windows kernel dispatcher employed a single lock, the dispatcher lock, which worked well for a relatively small numbers of processors (like 64).
apparently not?

or maybe you're thinking that the dispatcher lock originally had even more under its belt

[twilit smelt]

i cite that video in the mintia2 scheduler somewhere

[hollow musk]

this is not true

[twilit smelt]

https://github.com/xrarch/mintia2/blob/main/OS/Executive/Ke/KeWait.jkl#L538-L540

[warm mural]

I also do somewhere in my code

mine is simplified from NT's model however

they have a bunch more wait states than I do

[hollow musk]

the scheduler is in good condition, as far as I know, Dave Cutler, the original author, still works at microsoft

and I assume that he is still in charge of development

[twilit smelt]

he is not

[warm mural]

no he's not on the windows team

[twilit smelt]

he went to xbox and then hyper-v

[warm mural]

he's doing xbox or azure stuff iirc

[twilit smelt]

i think

hyper-v is another "cutler kernel" or so ive heard

would be interesting to RE

to see more of his work

[hollow musk]

he's not? to be honest, I'm upset because I was sure of it

[twilit smelt]

from more recently

yeah and i think that was by his own choice

he might have gotten tired of it

[warm mural]

tbf he's probably semi-retired

[twilit smelt]

he left the team in the late '00s i think

[warm mural]

works on whatever interests him

[twilit smelt]

he says hes not

[hollow musk]

oh, so many changes have already been made without his involvement

[twilit smelt]

he said in his interview with davepl from a year or two back that he still goes to work every morning and does normal work

[warm mural]

damn

9 to 5 at like 80 is determination

[twilit smelt]

he said he does it because he wouldnt know what else to do with himself

cuz hes been doing system software for 50 years

[shadow ridge]

thats sad

[twilit smelt]

kind of

lol

[warm mural]

he could join this server and make a hobby OS

[shadow ridge]

cutler os

feature list:

1. loads gdt
2. loads idt
3. a20 line enabled

maybe not.. lol

[hollow musk]

actually, it'd be interesting to see a professional's hobby system to learn their approach

[warm mural]

JW would count I guess?

dont think he's had an active project for a while tho

[digital pivot]

wasn't it you who listed some bugs in the windows scheduler

[hollow musk]

well, there were some odd bits

I don't know what some of them are related to, but I found it a bit strange that some kernel routines have a different naming style from all the others

[twilit smelt]

which ones

[hollow musk]

they usually name functions as the code name of the component, followed by a verb, but some new functions make an exception to this age-old rule

I know it sounds like a minor detail, but it caught my eye

here is an example of typical naming:
KiAdvanceReadyQueueEnumeratorToNextList
KiAdvanceReadyQueueEnumeratorToNextPhase
KiAdvanceReadyQueueEnumeratorToNextProcessor
KiAdvanceReadyQueueEnumeratorToNextRootScb
KiAdvanceReadyQueueEnumeratorToNextScb
(this is an example from the thread selection flow)

and these are mutants:
KeSwappablePageDereference
KeSwappablePageReference
KeSwappablePageGetLockedAddress

[twilit smelt]

lol we noticed that 2 years ago

[shadow ridge]

cutler normal form sucks anyways

verb should be at the end

dereference is a function of swappable page

not the other way around

[hollow musk]

curious

but yeah, it's not a big deal, just a minor thing

the other code problems I mentioned somewhere here on the server aren't major either

[warm mural]

verb first makes more sense english-wise

verb last makes more sense for namespace separation

[hollow musk]

have you found the answer to what it is?

[shadow ridge]

the latter is simply more useful

[twilit smelt]

like, what it does?

[hollow musk]

what is this page and why is it needed

[warm mural]

because of grep?

[twilit smelt]

iirc we found out that it is used to pin userspace pages in place that contain stuff needed for autoboost which has to be referenced while holding scheduler spinlocks

autoboost is used for userspace locks

so this is required

[warm mural]

you can grep all functions related to something using grep whatever_*

[shadow ridge]

yea thats a big factor

and when youre refactoring

[hollow musk]

the only use I've seen for it is in their priority inversion protection mechanism which is very optimized for user mode locks

and they not only swapout the kernel stack but also this per process page, hence the name

[shadow ridge]

it also just makes more sense because functions naturally exist in a hierarchy

[hollow musk]

for reference, they can periodically swapout kernel stacks for inactive threads

[shadow ridge]

you have Ke which has a ReadyQueueEnumerator which has a number of functions

[digital pivot]

what are these

[shadow ridge]

the opposite is like ordering a file system in reverse where all the leaf nodes are in root and they somehow extend upwards from there

your homework isnt in /homework/rustc/home

[warm mural]

yea but you gotta admit verb first looks better

even tho its less practical

[shadow ridge]

no i think it looks worse

[warm mural]

(I personally use verb last though)

[shadow ridge]

like i dont like it from an aesthetic viewpoint either

in fact that comes first

for me

everything else is just rational justification

[shadow ridge]

it's like asking if this looks better than /home/rustc/homework

like.. no.

[warm mural]

it's a matter of taste, but imo KeRaiseIrql reads better than KeIrqlRaise

[shadow ridge]

well i can see a parallel between how we write numbers (most significant digit first) vs how positive iq people order them in memory (least significant byte first)

and this

[warm pine]

i think noun_verb is an object oriented style

so it looks a bit off for non objects like irqls

[twilit smelt]

the NT naming style by the way is different from both mica and vms style which were the same

the explanation ive heard for this is that it is meant to be consistent with OS/2 instead

which tracks

look at the OS/2 api set

its stuff like

DosReadFile

DosWriteFile

etc

[shadow ridge]

irql is purely a software level concept right

i forgot

[twilit smelt]

(OS/2 refers to itself as "Dos" internally because it started as Multitasking DOS which was an experimental rewrite of DOS in C with preemptive multitasking)

[warm pine]

it's %cr8 on amd64

[twilit smelt]

the software levels are (in principle)

the hardware levels are hardware

[flat dust]

os/2 v1 was literally multitasking dos without a GUI i think

they were going to add it but they ran out of time or smth idk

[twilit smelt]

literally yeah

[flat dust]

deadlines :(

[twilit smelt]

they were forced to do a CLI-only release because things were taking way longer than they expected and they needed to get something minimal out for developers

that barebones beta release was like $600 too

[flat dust]

why so expensive

[twilit smelt]

because IBM

idk

[flat dust]

😭

ibs

[warm mural]

the start of IBM's downfall 😔

(idk probably)

[flat dust]

allegedly

what if the PS/3 was released and had a powerpc chip because screw backwards compatibility thats why

[warm mural]

kinda crazy IBM went from the equivalent of current day apple/google/microsoft to like oracle or something

[twilit smelt]

relative to the size of the computer industry at the time they were actually larger than all 3 of those

[warm mural]

apple+google+microsoft then

[flat dust]

IBM is still a huge company, they just left the personal computing industry like how they were before 1981 or smth

[warm mural]

I know it's still huge

also they did try to continue doing PC stuff

like thinkpads

[twilit smelt]

they did powerpc thinkpads that ran os/2

they sold like

10 of them

[flat dust]

weirdest cd drive position

[twilit smelt]

the only real world use of them ive heard is someone saying "we literally still have one of those its the only way to program this gigantic robotic arm we use at my job, the software runs on ppc os/2"

[shadow ridge]

they should springload the drive with a really strong spring so if you open it while it's in your lap it gives you a good smack in the stomach

[flat dust]

https://cdn.discordapp.com/emojis/816178972672131083.gif?size=64

[warm pine]

ppc os/2 would be an interesting read

[flat dust]

can you port it to a mac like windows nt

[warm pine]

they had ambitions for what they called the WorkPlace Microkernel as a universal OS

[twilit smelt]

its a somewhat weird mach

[twilit smelt]

the person who ported NT to the mac has considered doing the same with ppc os/2

[shadow ridge]

why does the most random shit always run on obscure and old tech

[twilit smelt]

it was the only part of taligent that ever shipped isnt it

and it only shipped as ppc os/2

[warm pine]

as far as i know, barring a few userland frameworks (opendoc might have been one?)

[flat dust]

there was literally only ONE version of powerpc os/2 i think 😭

[twilit smelt]

apparently ppc os/2 is a miracle that it shipped and the project was internally extended for far, far longer than it would have survived if it werent for a single IBM exec who was a diehard OS/2 fan and would not let it die

[shadow ridge]

he probably bought 9 of the 10 copies that were sold

2 for himself and the rest for his family

the last one runs a random robotic arm at some company

[warm pine]

intended architecture (A) of Workplace kernel vs actually delivered for ppc os/2

[flat dust]

https://klipy.com/gifs/lol-thrown

they tried i guess 😭

[warm pine]

ppc os/2 must be distinctive in being the only ever desktop OS shipped based on a microkernel

[twilit smelt]

is that a microkernel

look how much is in the box

on the right

[hollow musk]

#1322494605819908156 message cr8 on x86

[twilit smelt]

is that like NeXT calling it the "Mach microkernel" when it was actually larger than any of the other shipping unix kernels lol

[shadow ridge]

personality?

[hollow musk]

sorry, someone already answered

[twilit smelt]

API set

[warm pine]

i can't tell if the personality is in userland or not

if it is and the filesystem is in userland i'd count it

[shadow ridge]

thank you anyways

[twilit smelt]

so you could have an OS/2 personality, a Windows personality, a Classic Mac OS, a Taligent personality (these were all planned at one point)

basically IBM was trying to do the osdev beginner stereotype

"helo i want write os it wil run mac program linux program and window program" -ibm

[flat dust]

horrible case of brain damage

[warm pine]

tracing the footsteps of microsoft or indeed dec

[flat dust]

also add every single 68k os that exists

[hollow musk]

by the way, my explanation is that for some reason they sometimes do this naming when it's a simple helper-like operation with some object, an example of a similar naming from the memory manager: MI_PFN_IS_PROTO

[warm mural]

that wasnt really a bad idea

[twilit smelt]

it was seen as potentially the idea of the future

[warm mural]

NT also kept this design decision

[flat dust]

they need to emulate the stupid old mac os apps

[twilit smelt]

around the turn of the decade 1989-1991 it was ideated

[warm mural]

but they ended up dropping it over time kinda

[twilit smelt]

IBM was the last warrior for it

[flat dust]

centralization ends up bad honestly

[shadow ridge]

why didnt anyone think of writing a unified api set for userspace that could be shared by any kernel

[flat dust]

posix

[warm pine]

in industry at that time this was the main import that 'microkernel' had, other than just the newness of it, but the idea of schizophrenic kernels was what everyone wanted

[twilit smelt]

where did we read that schizophrenic kernel was the term for it again

[shadow ridge]

posix doesnt define anything at the abi level right

[flat dust]

theres a different thing for that

i forgot what

[warm pine]

for management the idea of merging the basis of AIX, OS/2, OS/400, and maybe even more must have been a really exciting prospect with a lot of opportunity to cull manpower

[flat dust]

i swear there was

[shadow ridge]

i dont mean like system v

like

[warm mural]

Pretty sure I've seen NT described as schizophrenic before

[shadow ridge]

just the syscalls etc.

[flat dust]

oh

[warm mural]

POSIX defines interfaces, not system calls

[shadow ridge]

system calls are an interface

if u think about it

[warm pine]

i wish i could remember, it might've been spoken or described casually in reference to this or that figure calling them that

[shadow ridge]

theyre just glorified function calls

[warm mural]

system calls are a way to call into an interface

[twilit smelt]

its funny how not only is calling a multi-personality kernel a "schizophrenic kernel" not something that would be considered politically correct to use as a technical term these days but its also literally wrong. they were confusing schizophrenia with DID

[twilit smelt]

im like 90% sure i read it in a formal technical document from the early 90s

like an academic paper maybe or a conference presentation

and it was used like it was the accepted term

[shadow ridge]

why is it not politically correct

[twilit smelt]

ableism

[shadow ridge]

how is it ableist

[twilit smelt]

its mocking ppl with schizophrenia

or something

[shadow ridge]

nobody says schizo kernels are worse than normal kernels though

ok well we were literally just saying that

but nobody was saying that at the time

they thought it was cool

[warm pine]

perhaps you can come up with some analogous names and see the format doesn't tend to sound complimentary

i have but i couldn't possibly post them here

[shadow ridge]

schizophrenia is a medical condition

i dont think you can say any "analogous" name

[hollow musk]

very briefly, so as not to interrupt the chat: they implement a multi dimensional walk over the scheduler's ready sources, so the new thread search code (slow path) can say give me the next eligible ready list

[warm mural]

It's like the "master/slave" terminology

[twilit smelt]

i consider myself really woke and that was just performative lol

[shadow ridge]

thats more understandable though cuz it does actually have something to it

like

if i shouted PANCREAS CANCER KERNEL! literally nobody would care

[twilit smelt]

the NAACP never came out saying like "master/slave terminology in technology is racist. it should be changed."

it was a thing white libs came up with

performatively

[warm mural]

Well yeah

No one cares

[twilit smelt]

but i miss the performative wokeness

these days its been replaced by like performative chudness lol

[warm pine]

now it just feels so quaint

[twilit smelt]

vice signalling

[warm pine]

it's gotten really awful

[twilit smelt]

respectable tech ceo: "I fucking hate humanity. Literally I want everybody to burn. I want you all dead"
passes $5T market cap the next day

[shadow ridge]

i think it left an extremely bad taste in everyone's mouth which is why people started turning far right more easily

and created stereotypes

that are still used to ridicule leftists

[craggy spire]

It's likely that it can be reused. AutoBoost doesn't really work with the swappable page directly. It works with an abstraction built on top of the swappable page, namely the scheduler shared slots.

[warm mural]

Corporations really doing anything to look good for the current administration smh

[warm mural]

That + isolation turned a lot of people far right

[shadow ridge]

literally the only thing that can unify 99.9% of people is like csa

it's why companies like roblox shit themselves

and why the epstein shit bothers even republicians

[warm mural]

What's CSA

ahhh nvm

I get it

[twilit smelt]

well thats a little too far out of scope for mintia thread.

[shadow ridge]

derailment averted?

[twilit smelt]

actually i want to chime in

the 99.9% figure here is far too high

its more like 80%

[craggy spire]

@hollow musk BTW, I looked into the problem with quantum target overflow that you mentioned. It's only possible if your CPU's nominal frequency reaches at least 34GHz and in some cases 6.5GHz.

[twilit smelt]

theres a campaign on the right to respond to the epstein thing by reframing him as some kind of like giga winner of life drowning in hot underage chicks that everybody is just jealous of and it is catching on

you will see this all over right wing spaces

[warm mural]

I think the only time humanity was last unified was pokemon go

Ever since it's been more and more division

[shadow ridge]

wasnt some guy robbed while playing pokemon go

[twilit smelt]

depending on the right winger the majority responses to the epstein thing are either "its democrat theatrics and a hoax" (80%) or "yeah and? men have wanted young girls since the dawn of time. its natural. youre jealous" (10%)

both of which are engineered by social media

[hollow musk]

iirc some of the newest CPUs get pretty close to 6.5 in turbo mode

[twilit smelt]

that they consume

[shadow ridge]

the math doesnt add up

[twilit smelt]

the remaining 10% are the ones who think its bad

thomas massie fans

[shadow ridge]

idk i thought most people agreed that children shouldnt be abused

[twilit smelt]

theyd 90% agree with that its just that the vast majority are being convinced its a nothingburger

[warm mural]

Nothingburger but literally all conspiracy theories are being proven right

[twilit smelt]

by the same news outlets that framed "the government no longer considers greenhouse gases as harmful to the environment and will stop enforcing all related regulations" as simply "you know that annoying feature on your car? thats going away now thanks to god emperor trump!"

[shadow ridge]

which car company was it that bypassed regulations by showing different stats on the benchmarks

so the car would detect if it was in like a test grounds or whatever

and stop greenhouse emissions if it was

cuz it wasnt fuel efficient

[warm mural]

Volkswagen

[warm pine]

the whole thing has been a bizarre spectacle

supporters of a known libertine who even publicly talked about grabbing 'em by the you-know-what spent years calling for the epstein files release

[twilit smelt]

it was insane how it got retconned from a cause trump was championing day and night while he was campaigning, to a nothingburger democrat hoax basically overnight

in the republican consciousness

[shadow ridge]

i love how trump shits (literally) himself every time he realizes people still care

[warm mural]

The CIA redacting names is also insane

[shadow ridge]

wasnt it the FBI

[warm mural]

(excluding victim names)

FBI yeah mb

[shadow ridge]

why would the CIA deal with the epstein files

[warm mural]

Sorry I mixed the three letter agencies 😔

[twilit smelt]

every time i see someone repost one of vance's old tweets about it i get whiplash

[shadow ridge]

anyways yeah it's insane that the KGB redacted everything except the thing they were required to

[twilit smelt]

because its a direct 180 from what they say now

vance's like 2020-2024 era tweets about the epstein files would fit right in with what a leftist would say about it today

[warm pine]

he is maybe one of the most despicable people in the world

that man has no scruples whatsoever

[shadow ridge]

still cant believe he killed the pope

who would do that

[warm pine]

he traduced his own people and his own family while claiming their identity for political benefit, he condemned trump then went quiet and obedient the moment it stood to benefit his career

[shadow ridge]

traduced?

ah nvm

[mortal thunder]

i wonder if windows still writes to cr8 even for DISPATCH_LEVEL and lower

[twilit smelt]

it does

[warm pine]

i read somewhere a relatively recent comment by someone who could be expected to be knowledgeable about it that said yes

[mortal thunder]

i thought os/2 was like tied to x86

im surprised they had a PPC version

[twilit smelt]

they implement software interrupt dispatch by setting interrupts pending that get triggered when cr8 is lowered below dispatch_level

so they are real hardware interrupts

DPC dispatch and so on

[shadow ridge]

apparently this is the only version of it that runs on ppc

[twilit smelt]

it was, thats why the NT OS/2 project started, to provide a portable base for porting OS/2 to RISC

[warm pine]

it's a whole other system, os/2 ppc was based on the 'workplace microkernel' instead

[twilit smelt]

IBM had to scamper to come up with their own portable kernel when they were viciously backstabbed by Microsoft a couple years into NT development

they came up with a Mach

which they called the workplace microkernel

[warm pine]

http://www.ibmfiles.com/ibmfiles/powerpc/os2ppc_first_look.pdf some description here

[shadow ridge]

man i forgot so much osdev stuff

gotta practice it to retain it ig

i dont even remember why microkernels are still used

[mortal thunder]

slightly later they created a MIPS laptop called the WorkPad Z50 https://en.wikipedia.org/wiki/IBM_WorkPad_Z50

unfortunately no OS/2 to be seen here

they just used windows CE

(not even NT smhsmh)

[warm pine]

in a charitable sense they are and probably running microkernel systems today are a few orders of magnitude greater in number than monolithic systems, but only because they are now almost solely used in some embedded uses

[twilit smelt]

MIPS and SuperH handhelds were quite common

[warm pine]

their last hurrah for use as general-purpose operating systems was BlackBerry 10

[shadow ridge]

999$ holy shit

[warm pine]

many of you are probably too young to remember the blackberry

but having one (and thus access to BBM) brought you into elite circles if you were at high school in the UK around 2008-2012

[shadow ridge]

ok thats true but the context is consumer pcs

[mortal thunder]

i don't "remember" the blackberry phones because i've never noticed one in person but i know they existed

[warm pine]

but i digress, blackberry 10 was a QNX-based OS for smartphones

[twilit smelt]

when i was little i had a PDA that my dad gave me after he stopped having a use for it and i learned how to program it a bit and i used it as a prop in games with my friends

at some point i lost it at a friend's house

:(

[shadow ridge]

maybe you would have died in a car accident if you kept it

look from the positive side

[warm pine]

i've heard somewhere actually that apple have their own messaging app for iphones and in america you get looked down on if you don't use it

[mortal thunder]

iMessage, yeah

[shadow ridge]

what a creative name

[mortal thunder]

if the USA were a rational country I'd laugh it off as a joke

[shadow ridge]

almost as creative as OS/2

[warm pine]

blackberry had a messenger too (BBM) but maybe 50% of us max had blackberries so it was no great misfortune if you didn't have one, but we did form something like a club or social circle with some immediate connection between us, a bit like how smokers have that connection with other smokers as soon as they get to the smoking room

[mortal thunder]

these are probably the same people who called it "iPhone"

[shadow ridge]

ridiculous branding tbh and it's insane it's normalized

if windows was called iWindows we would all point and laugh at it

[twilit smelt]

you could tell someone doesnt have it because instead of a blue text bubble itd be green

[shadow ridge]

isnt that just whatsapp

[twilit smelt]

and the green text bubble would tell you theyre poor

and lower status

[warm pine]

for the rest of us now of course whatsapp is what everyone outside of america uses, but i really miss that sort of club-like aspect that having a blackberry and BBM got you into

[warm pine]

is it really that much of a phenomenon? i thought it was exaggerated

[twilit smelt]

possibly the most impactful coloring design in history was whoever came up with the idea to make your texts to other iphone users blue and your texts to android users green

[twilit smelt]

ive heard from straight friends that some girls do get the ick and block you if they text you for the first time after getting your number on a dating app and its green

[shadow ridge]

holy shit

self optimizing dating pool?

[warm pine]

what a hard time it must be being straight in america these days if something like that can instantly ruin your prospects

[twilit smelt]

android = poor here for sure

thats the strong connotation

[shadow ridge]

american gen z population is so weird ngl

[twilit smelt]

at least in middle class communities

[twilit smelt]

it started with gen x and millennials

gen z are probably less crazy about it than they are

[shadow ridge]

are your straight friends not gen z

[twilit smelt]

they are

im saying it used to be a bigger deal

in previous generations

people still pay attention now

but its not what it was

[shadow ridge]

how is it a bigger deal than blocking you for a color

did they fucking kill you

because the text bubble was green

[digital pivot]

#1322494605819908156 message apparently yes

[twilit smelt]

dating apps are a special example because everybody there is extremely skittish and looking for a reason to get the ick

in the absence of physical signals the brain goes into overdrive overanalyzing everything else about them

[shadow ridge]

i dont use dating apps (or date) so i wouldnt know ig

[twilit smelt]

among millennials you could get bullied and excluded in high school for having green text bubbles

among gen z i havent heard of that

[warm pine]

they are probably not going to be remembered as something good for society

[mortal thunder]

little do they know you can get an iphone 6 for relatively cheaply and still use imessage in 2026

[warm pine]

i shouldn't talk because i met my gf on tinder but i still don't think they're much good for society

[mortal thunder]

i agree

[twilit smelt]

i probably have the most embarrassing possible story for how i met my partner

[shadow ridge]

it's better than random people practically being forced to marry cuz their family knows eachother (probably)

[mortal thunder]

like they're designed to extract as much as value from you (by keeping you on the app for as long as possible). that's what i believe

[shadow ridge]

if this was the case then they would just show you mismatches all the time

so you dont stick with one person and stop using the app

[shadow ridge]

if not gonna say it, why mention it

guys i have a secret vault in my room and theres something inside of it

(refuses to elaborate further)

[warm pine]

vagueposting is a tradition as old as time

[shadow ridge]

it started with people being like "hi.. can i ask a question" or "hi.. does anyone know C?"

and taking 20 mins before they ask the question

[twilit smelt]

do you guys think a chinese emperor ever walked in on some of his courtiers hanging out and just went like "It had to happen." and walked out just to make them freak out

[shadow ridge]

bro piped /dev/urandom to his stream of thought

is there an equivalent of /dev/urandom on NT

[warm pine]

i don't think that's reasonable to draw as if it's a dichotomy, i grew up in a highland village and generally (for those who stay there) their options are of course more limited, but that tends to work out well for everyone, the lack of options produces more diverse pairings

[shadow ridge]

actually how much of devfs has a direct equivalent in NT

[twilit smelt]

the devfs equivalent is the Devices directory in the object namespace

which contains device objects

[shadow ridge]

yes but how many of those objects are like

a match of unix ones

[warm pine]

they are still things which you open and you get a file object as a result, and you can do things like read, write, and ioctl them

[shadow ridge]

i mean as like, builtin devices that do a certain thing

[warm pine]

so the NT approach to devices is fairly unix-like, the most fundamental distinction is that on NT these operations are all profoundly asychronous

[twilit smelt]

i dont actualyl remember if you get a file object as a result of opening a device object

that might have been a mintia-ism

that we're projecting

on NT

[shadow ridge]

like, would there be a device like /dev/zero that ignores input and only reads 0

or /dev/null

like what is actually in the devices directory by default

[warm pine]

it sounds like you do:

The FILE_OBJECT structure is used by the system to represent a file object. To user-mode protected subsystems, a file object represents an open instance of a file, device, directory, or volume. To device and intermediate drivers, a file object usually represents a device object. To drivers in the file system stack, a file object usually represents a directory or file.

A file object is partially opaque. Certain types of drivers, such as file system drivers and network transport drivers, use some of the fields of file objects.

[twilit smelt]

what is said here is not the same as "NtOpenFile'ing a device object will work and yield a file object handle"

but that also might be true i dont remember

[shadow ridge]

what else would it yield

[twilit smelt]

idk

death by scaphism

[shadow ridge]

asteroid hits the earth

[warm pine]

well of that is said

Opens an existing file, device, directory, or volume, and returns a handle for the file object.

i think i read an absolute statement to that effect somewhere as well, possibly in windows internals

[shadow ridge]

do NOT google this

it's disgusting

idk why you know about this

[warm pine]

it's well known

[shadow ridge]

iron maiden is well known

that bull thing is well known

[warm pine]

everyone reads about this as one of the worst ancient punishments, i think it was practiced by the persians

[shadow ridge]

idk we didnt learn about persian execution methods in school

if we did i would have payed attention

[shadow ridge]

i dont get why it works though

[warm pine]

well presumably you at least do about what your own country did

[shadow ridge]

we didnt learn any execution methods at all

[shadow ridge]

insects dont just burrow into you

you would just die from bloatation cuz youre getting force feed huge amounts of milk and honey

[warm pine]

i certainly remember learning about what henry viii had done to his wives and about that ignoble episode in scottish history, the massacre of glencoe

[twilit smelt]

my favorite persian thing was where they had a 180 archery glitch where they could turn around on their horse with a saddle on a swivel and rain arrows down on you while retreating

and they defeated the romans like 3 times by doing this and nobody else could figure out how to do this

and they also had an infinite arrow glitch

where they just had insane arrow supply logistics

to maintain continuous arrowfall for like days straight

these 2 things are why crassus got crassused

[shadow ridge]

well

ig we did learn about beheadings and stuff

but nothing too gruesome described in extreme detail

do you guys remember these phones that had magnetic pens with them

i wonder if they still exist in some form

[twilit smelt]

this is what you can do with an infinite arrow glitch in antiquity

[shadow ridge]

did they patch the dupe

[twilit smelt]

who would win between 9000 horse archers on swivelly saddles with infinite arrows and 35,000 heavy infantry

i guess we know.

[twilit smelt]

yeah but the USA found an infinite bombs glitch

[shadow ridge]

ik 80 is highlighted but

it says like more than 3 times that

right under

[twilit smelt]

yeah

some poor laotian farmer gets a leg blown off like once a week

while trying to expand his field

we just ignore this

[shadow ridge]

imagine a very very post apocalyptic world where thousands of years have passed after humanity destroyed themselves and whoever's left doesnt even remember a war happened

so people are just walking around

thinking mines were left by gods

[twilit smelt]

somehow its the same farmer every single time

[shadow ridge]

who are punishing them for walking on dirt or smth

[twilit smelt]

hes on like negative 100,000 legs

just a fluke of statistics i guess

apparently at peak 1 laotian was killed per day from encountering unexploded us munitions

this has declined due to awareness efforts

the usa sent them one time reparations of like 5 million dollars once

which is a couple cents per bomb

usa! usa!

[digital pivot]

just reread this and uh... okay stack, but does it make sense to swap out a page?

[shadow ridge]

how many dollars does it take to remove one bomb

i would remove 1 bomb for like 100 bucks if safety gear + training is included

[twilit smelt]

at the most efficient its probably around there

[shadow ridge]

drop more bombs to force the unexploded to detonate

[twilit smelt]

the reason we did that by the way was because vietnamese troops would transport war materiel to the viet cong in the south via jungle corridors in laos and we couldnt really see them through the trees so we would just bomb the shit out of everything

1 planeload of bombs every 8 minutes

for 9 years

straight

we didnt even win

[shadow ridge]

yeah how tf did US lose like basically every big war since ww2

[twilit smelt]

we lost that one because we were fighting people who were fighting for their country

and had the juice and wanted to win more

nothing short of a genocide would have ended that one probably

luckily the prevailing winds were against genocide at the time so we lost

[acoustic sparrow]

i mean isnt that kinda by definition

[dapper mulch]

Is it Rajeev Nagar book? The section about disquette says:

Accompanying Diskette

A diskette accompanies this book and is often referred to in various chapters of
the book. This diskette contains source code for the following:

• A file system driver template
  Note carefully that this is simply a skeleton driver that does not provide for
  most of the functionality typically implemented by file system drivers. The
  code has been compiled for the Intel x86 platform. The code has not been
  tested, however, and should never be used as is without major enhancement
  and testing efforts on your part.
  This driver source is provided as a framework for you to use to design and implement a real file system driver for the Windows NT environment.

"Simply a skeleton". So wouldn't a driver samples like fastfat be more useful in this regard? Completeness and up to date.

[twilit smelt]

yeah fastfat probably is more useful

but you kinda need the sample code to follow along with the book at max effectiveness

so it would be useful to archive probably

since this book is still read

nobody has really written a better treatment of the NT IO system

[mortal thunder]

fastfat might be useful for a concrete implementation but at the same time hard to read and FAT-specific

[safe hollow]

the state diagrams on the book look like they were drawn with mspaint

[dapper mulch]

If you need Rajeev Nagar book in pdf I can send it.

[hollow musk]

@twilit smelt perhaps you know and can tell me what a "soft working set list" is?

[twilit smelt]

What's the context

[hollow musk]

the VA type called MiVaSoftWsles

I don't know about this one

[twilit smelt]

Me neither

[digital pivot]

https://www.vergiliusproject.com/kernels/x64/windows-11/25h2/_MI_WORKING_SET_TYPE has MiWorkingSetTypeSystemSoftWsles and its "...System..."

whats MMSUPPORT?

i thought it was a struct for working sets but its MMWSL_INSTANCE apparently

https://www.vergiliusproject.com/kernels/x64/windows-11/25h2/_MI_SYSTEM_VA_STATE has both _MMSUPPORT_FULL SoftWsleSystemVm and _MMWSL_INSTANCE SystemWorkingSetList[10]

[safe hollow]

what has the working set list + coloring info

[twilit smelt]

going to try to finish numa support today

[digital pivot]

coloring is numa related stuff?

[twilit smelt]

no

cache coloring is useful on UMA

[digital pivot]

is it like

a color per cache line

[safe hollow]

useful for single core and above

it's better to start from the opposite of cache coloring

[twilit smelt]

per cache set more like

[safe hollow]

you know that for example a small set-associative L2 cache stores at most 8 64-byte blocks per set

it uses the bits immediately after the offset to derive the index, and the remaining ones as tag (and ones with the same tag compete in the same set)

if you make so that a process only gets even indices, it can only use half of the cache

(which afaik it's actually useful in its own right for cache partitioning stuff for hypervisors)

[twilit smelt]

remaining numa work:

15. fix the messed up way the per-node system process page directory is not actually correctly set up
16. create an XR/frame target to test that this stuff actually works in the `node^.Id != 0` case
17. duplicate kernel text between each node automatically in the bootloader (kernel code paging will still occur when numa node count > 1, and will be local to the node, in the per-node system process's working set)
19. review the implementation of topology awareness in FreeBSD and how it's used in ULE
20. need a fast way to locate the node that a page frame is in and make it so that freeing a page frame doesn't require a node parameter (maybe a binary search over an array describing phys addr ranges and their association with nodes, replicated per node)
20. use topology information for scheduling and memory allocation decisions
21. refactor per-node initialization to be 1 or 2 outer loops around big subsystem initialization functions that now take a node parameter, rather than a billion scattered inner loops
22. fix the freakong turnstile propagation bug where it interacts incorrectly with explicit priority raises, which can prematurely terminate a wait chain boost

i merged the numa branch into main branch just now because its not aggressively broken

[digital pivot]

u dont do partitions in mm anymore?

[twilit smelt]

no i got rid of it to simplify stuff

the partition object is now just part of the node

[digital pivot]

ah i see

[safe hollow]

mr vmm man, are there some nasty differences between on reverse mapping between a list of VMAs and a list of PTEs (or wsles like windows but indirectly ptes still)

[twilit smelt]

i cannot parse this message

i dont understand the question

[safe hollow]

linux rmap stores a list of mapped regions that contain that page

so windows VADs

[twilit smelt]

sure but this question doesnt make sense tho

like syntactically

"differences between on reverse mapping between a list of VMAs and a list of PTEs" i dont understand

[safe hollow]

do you happen to know of nastiness than vma lists have that would be avoided with pte lists? the former save quite a bit of space and i think there has to be a catch

[twilit smelt]

by PTE list what do you mean specifically

[safe hollow]

mmpf stores the index to the first WSLE mapping the page, and that WSLE then links all the mappings in other processes right?

[twilit smelt]

no, thats not how working sets work

windows does not usually take a page frame and yank it from all of its current mappings

it goes the other way

a process is selected for trimming and PTEs in that process are aged

the ones that are invalidated cause the refcount on the page frame to drop

[digital pivot]

with a pte list you can go straight to the pte and with rmap you need to conpute the virtual address from vma start + page index and then walk page to locate the pte ?

[twilit smelt]

when it hits 0 it becomes a candidate for pageout

for pageout they go process->pte->page frame

not pageframe->ptes

so they have no need for something like rmap

or any kind of list of ptes that map a given page frame

the working set list was in former times a per-process array of virtual addresses that had valid mappings

so that they could iterate over this array and age PTEs and not walk the page tables

they have reverted to walking the page tables because its more parallelizable

[hollow musk]

that's interesting

[twilit smelt]

its optimized in interesting ways, its not just naively iterating over all of the page tables in the process

[blissful smelt]

do you happen to have any resources on reverse engineered parts of autoboost besides your messages?

i am quite interested in autoboost

[hollow musk]

as far as I know, there are no public resources

[blissful smelt]

do you have any pointers on where to go with reverse engineering it myself? I have a windows 10 binary

[mortal thunder]

how does that work? do they just walk every PTE possible, or what?

do they have any interesting optimizations?

[warm pine]

the page structs describing each page table page containing leaf PTEs is linked onto one of several lists headed in the working set description, depending on something to do with the age of PTEs in that table (average age? worst age? unknown)

[hollow musk]

I'm only familiar with the classic structure of the working set list

[mortal thunder]

same

[hollow musk]

what else do you know about the modern virtual memory subsystem? 🙂

[mortal thunder]

what if that same page frame is placed into multiple processes' working sets

[twilit smelt]

page tables are per-process so that cant happen

[warm pine]

being page table pages they are only in one process

snap

[mortal thunder]

oh

the PAGE TABLES are linked in a list

and specifically the ones that contain leaf PTEs

[warm pine]

it makes good sense really

[hollow musk]

do you have any specific questions? I have a lot of information in my head about the scheduler from 10+ different windows builds, but it hasn't been publicly described anywhere yet

autoboost is a rather complex subsystem, 95% of its implementation in modern Windows is neatly separated into a separate class called AutoBoost (and LegacyAutoBoost is the old version)

[warm pine]

the traditional working set list was a wonderful data structure, almost as well fitted to its use as any organ nature ever evolved, but when you implement a clock-like policy or indeed (as they have for some time) a policy for working set replacement based on maintaining several ages of entries, then it's no longer well suited; you have to traverse a data structure out-of-line with the page tables, and it may lead you about from table to table

[blissful smelt]

i think i will first dive into windows itself and poke around and see what i can find out, and come back with questions, thanks for your willingness to assist though!

[hollow musk]

let's see, maybe you'll share some new information

I didn't spend that much time on it, since I was mostly studying how the thread scheduler proper works

[hollow musk]

how long ago did it appear?

my knowledge remained at the level of windows 7

[warm pine]

some number of windows 10, either in the 1600s or 1700s, one of those two centuries

[hollow musk]

I remember reading about some windows 11 "lock-less" mechanisms, and specifically about scanning page tables

I wonder if that's related, cause I can see how that would cause massive performance issues if they didn't implement new locking mechanisms

I think I saw the working set changes but didn't have the context for them and didn't link them with the PTE changes

that's super interesting

[warm pine]

it's confirmed by an authoritative source that the scanning of page tables can proceed in parallel for different ages, and i would fairly speculate that it might be that at least aging can be done mostly locklessly with some atomic pte operations, to not block out page faults for instance

[hollow musk]

I noticed that they per each age?

I assume that 8 is the age

I have a question for you: are page importance and age related?

each PFN has a property called importance, and there are also 8 of them, apparently

[warm pine]

i don't know, all i know is that in a powerpoint about ? server 2003 or ? vista there is mention of installing the pages brought in as part of a clustered page fault with an elevated initial age, or maybe even as transitional ptes

[digital pivot]

i have a question, do you know whats KeReferenceCount?

[hollow musk]

this is easy one

KeReferenceCount counts how many scheduler deferred structures still contain a live reference to the thread and thread deletion asserts it goes back to zero

[twilit smelt]

does it cause thread deletion (sending the thread to the reaper) to be deferred until the count drops to 0

[hollow musk]

when thread teardown runs, it expects the KeReferenceCount pin to be gone and if it isn't, it tries to force progress, and if the pin still remains treats it as a kernel lifetime bug ie bugchecks

if KeReferenceCount:
KeFlushQueuedDpcs()
if KeReferenceCount:
bugcheck

I don't remember exactly, but something like that

one striking example is autoboost

initially, a counter was added for it, and I believe it was even called AbReferenceCount

enqueue deferred autoboost propagation worker, take ref, worker runs, drops ref

[digital pivot]

i see

[blissful smelt]

what "namespace" (function name prefix) are autoboost functions under, and which .dll or .exe would it be? I've found a few functions with AutoBoost in their names, and from my knowledge there should be functions like KiAbProcessContextSwitch but I can't find them

I'm looking around system 32 in windows 10 22H2

[digital pivot]

not an expert but it should be ntoskrnl.exe

[blissful smelt]

I'm looking in there and can't find KiAb

no mention of it

[digital pivot]

then idk

[blissful smelt]

i see a few Ab* fields in KTHREAD

[digital pivot]

what about autoboost not ab

[blissful smelt]

they seem to abbreviate autoboost into ab

[digital pivot]

huh have they really hidden autoboost so well

will says that they are deliberately hiding it for a reason

or something like that

[blissful smelt]

all this to bump thread priorities around 💔

[digital pivot]

or simpler

you searched in the wrong place

and just failed

[blissful smelt]

this bsod stack trace reveals a few symbols

[digital pivot]

note it was rewritten in win11 some version

i remember reading that it was rewritten but dont know the symbols

[hollow musk]

it's implemented in ntoskrnl.exe and ntdll.dll

in the kernel, it is implemented as part of the... kernel, "Ke"

[blissful smelt]

are they kind of all over the place? any notable/important autoboost functions?

[hollow musk]

in earlier versions, it's implemented in the autobst.c file, the interface for primitives has the prefix KeAb, internal functions are KiAb

[twilit smelt]

Ke is a component

[hollow musk]

in the new version, the current autoboost is located in the AutoBoost C++ class

data types are located in the same class and prefixed _AB_

internal functions to the kernel are KiAb, private functions of the class are KiAbp, public functions are KeAb

[blissful smelt]

by new version, do you mean closer to Win11?

I'll look at a win11 binary too and see

since this is 10 22H2

[hollow musk]

new files: ke/autoboost.cpp, ke/autoboost_api.c

[hollow musk]

it's the intermediate version, so to speak; the new version is available in preview builds and is likely planned for this year?

maybe start with something like mutexes and look at their implementation to see how they use the autoboost interface

legacy autoboost's representation was built around a very revealing structure: KLOCK_ENTRY

it answers 3questions:

1. which lock/resource identity is this about?
2. what role is this thread in -- waiter or owner?
3. what boost contribution is associated with this edge?

KLOCK_ENTRY answers these through:

1. it has both a tree node and two tree roots (OwnerTree and WaiterTree)
2. it has a HeadNodeBit
3. it has an embedded boost bitmap containing CPU boosts and also I/O and I/O QoS bits
4. it has explicit state bytes for "in tree", "waiting", "acquired", and "cross-thread"

in legacy autoboost the same struct can represent a thread's participation entry or a per-resource head node that owns the trees

(this has been changed in the modern autoboost.)

[blissful smelt]

would 26300.7674 potentially be "New"?

announced Jan27 of this year

[hollow musk]

if it's the Dev Canary channel, then I don't know

the latest windows 10 builds are perfectly fine, why not?

the structures of the new autoboost are closed and will be more difficult

[blissful smelt]

I couldn't find anything related to autoboost after decompiling windows 10 22H2's ntoskrnl.exe

perhaps it was too old?

[hollow musk]

it was already there in Windows 8.1

[blissful smelt]

I was using ghidra

hmmmm 🤔

[digital pivot]

its funny that it was rewritten into c++

considering that autoboost guy said he didnt like transition to c++ in the kernel or something along these lines

[blissful smelt]

who's the autoboost guy?

the person working on autoboost at microsoft?

[digital pivot]

"..., but when integrated into the kernel, I fear the alienation problem my team encountered when trying to start using C++ in the kernel, which at that point was firmly decided to be stopped ..."

[twilit smelt]

feel like i have way more of a case for a divinely inspired schizo kernel than terry

[blissful smelt]

devious numa awareness

absolutely schemeful

[craggy spire]

Lol, what are you doing

[blissful smelt]

trying to learn about autoboost

[twilit smelt]

use case

[craggy spire]

Numa-aware turnstiles?

[twilit smelt]

yeah

[craggy spire]

That's a new one

Per-numa node turnstile table?

[twilit smelt]

yeah

[craggy spire]

Right

[twilit smelt]

using whatever node the lock is allocated local to

hopefully allocated on your local node or a nearby node thanks to allocation affinity

[craggy spire]

Possibly the first NUMA-aware priority inheritance

[twilit smelt]

maybe? idk

i feel like pushlocks are more inherently numa aware since they have no central database like this at all

i dont think

[craggy spire]

They also have cache-aware pushlocks for bigger machines

They are replicated per NUMA node

And padded to cover a cache line

[twilit smelt]

isnt that where to acquire shared access you only need to grab the node-local one shared

[craggy spire]

Yeah

[twilit smelt]

but to acquire exclusive access you need to grab them all exclusive

i havent found much need for that because 99% of my locking needs are exclusive only

[craggy spire]

The memory manager and FS use them

FCB headers for sure

[craggy spire]

Ghidra

[blissful smelt]

I'm getting pranked

[blissful smelt]

is this 22h2

I downloaded 22h2 from archive.org and extracted the install.wim

[craggy spire]

This is the oldest thing I had in my Ghidra

22h2 has it 100%

This is a fundamental AutoBoost function that has been present in all versions, regardless of how you number them

[blissful smelt]

so I extracted ntoskrnl.exe from install.wim in the image I got on archive.org

is that wrong

how did you get your binary

[craggy spire]

From system32, it's not important

[twilit smelt]

wonder if one day theyll change the name and it wont be called ntoskrnl.exe anymore like how macos no longer has mach_kernel

[craggy spire]

And uh, did you download the symbols

[craggy spire]

ntrknlmp

[craggy spire]

autoboost is probably .c as well

They compile .c files as C++ anyway

[hollow musk]

there are suspiciously many people on this server who are knowledgeable about NT

when I joined the server, I assumed there would be intelligent people here, but I didn't expect this!

[twilit smelt]

nobody knew anything about it until around 2021

i got obsessed with it and started talking about it nonstop which annoyed ppl to the point where i almost got banned multiple times

[digital pivot]

why did you join by the way

[twilit smelt]

but over the span of a few years more NT enjoyers coalesced

[hollow musk]

I just wanted to ask one specific question, that's all

#osdev-misc-0 message

[hollow musk]

you're here, which means people don't find it that annoying

[twilit smelt]

things changed lol

there were a few people who would get weirdly hostile if NT was mentioned

some of them are still regulars they just gave up

this community was convinced to take NT seriously through like years of persistence by me and a couple others

[blissful smelt]

wdym

[hollow musk]

it's natural that people know less about the inner workings of windows, and the nature of their expectations and perceptions is also clear to me

[blissful smelt]

sob

[hollow musk]

in my circle, discussions about operating systems boil down to linux

I hope to change that

[twilit smelt]

pre-2021 the perception of NT (including mine) was definitely "lol its a micro$hit kernel theres definitely nothing interesting in there"

people just would not look at it

[blissful smelt]

i see a bunch of

[hollow musk]

I feel like microsoft has been trying to put a spoke in the wheel in recent years, and it's actually a bit of a shame that it's owned by microsoft

[twilit smelt]

i think i only had a case because i was like "actually its not a microsoft kernel its more of a DEC kernel" so i didnt need to like

defend microsoft in the process of defending NT

i could throw them under the bus and praise DEC culture instead lol

otherwise i might have just come off as a microsoft shill and unironically might have just been banned for being annoying on those grounds yrs ago

[craggy spire]

Well, Gummi, you haven't downloaded the debug symbols that contain names and types

You need to download them from the Microsoft server and load them into your Ghidra project

No, I don't remember the link off the top of my head

[blissful smelt]

ahhhhh thank you

[hollow musk]

it's a little funny that all this time I believed that Dave Cutler personally still manages at least the lower echelons of the kernel like Ke

and that this is partly why they maintain an engineering approach rather than a microsoft approach

[twilit smelt]

hes not completely out of the picture it sounds like he chimes in still

and keeps an eye on whats going on

NT culture is definitely running on inertia though

[hollow musk]

I don't remember, but someone famous said that he still works at microsoft, and I can't imagine that it's Xbox lol

[twilit smelt]

it was xbox for a time

a game console might have sounded like a welcome change after like 30 years of system software innards

[hollow musk]

he's probably working on some kind of cloud for xbox based on NT and Hyper-V

[twilit smelt]

his participation in azure was designing and implementing hyper-v

[hollow musk]

they are closely intertwined perhaps this is to his credit

[twilit smelt]

and he probably contributed to the xbox hypervisor

[hollow musk]

the modern NT scheduler can schedule virtual Hyper-V cores, which is quite interesting

I don't know exactly how it works, but it would be interesting to find out

this would be very useful in scenarios where host performance is important

such as Xbox which runs multiple virtual machines afaik

[twilit smelt]

isnt windows 11 always running as a hyperv guest now

[hollow musk]

this can be easily turned off, but by default it seems to be on

[twilit smelt]

alongside SK which is in another

(securekernel)

[hollow musk]

I don't know about SK

Is it connected with the scheduler? 🙂

[twilit smelt]

dont think so

[hollow musk]

perhaps one of the reasons for the transition to the concept of reschedule plans

apparently, the scheduler became too complicated, so they redesigned its heart

[blissful smelt]

thank you omwinyourhead

i appreciate and value your worthwhile insight 👍

[craggy spire]

Nice Gummi

I'm glad you figured it out

[warm mural]

This channel has a crazy amount of NT knowledge

Very cool

[digital pivot]

anything interesting so far

[blissful smelt]

i have just started my gander, i will report back in a few hours/days

unfortunately i am not dave cutler reincarnate so i wont be able to analyze this at lightning speed

NT code is also fond of long functions

[warm mural]

A fun thing is adding your own decompilations to the database and eventually you end up with a bunch of your own local decompiled NT code

[blissful smelt]

linux's code is a lot more like a bunch of haikus but NT often gives you a big epic which takes a bit to analyze

[warm mural]

Linux code is hard to get into if you don't know the codebase a bit already

which is why cross referencers help a ton

[blissful smelt]

do people actually use these and if so what for

seems like a funky feature

[warm mural]

Control flow?

[delicate dome]

Control flow analysis? This is basic reverse engineering.

[warm mural]

Actually I wonder if MS would sue you into oblivion if you published a decompiled part of original NT

[blissful smelt]

i find the decomp more useful than the control flow graph usually

perahps i should try this out

[warm mural]

Yeah the decomp is useful but it can be misleading/hard to read

[golden meteor]

omg yes

CFGs are so useful

[blissful smelt]

autoboost being hardcoded into the dpc execution engine is interesting

[digital pivot]

probably inlined

[blissful smelt]

yea but in either way it's a hardcoded call that executes the logic only if the caller is the Ab worker

which is interesting

[digital pivot]

what can you say about autoboost so far

[acoustic sparrow]

wow thats better decomp than i would expect from the sillyness that is ghidra

also wtf is LOCK() and UNLOCK() lmao

[blissful smelt]

oddball way to represent lock prefix instructions

[acoustic sparrow]

fucking what

this decomp is also just wrong lmao

[digital pivot]

is ghidra that bad?

[acoustic sparrow]

sometimes it can be yes

im a binary ninja enjoyer

which has its own stupids

[blissful smelt]

that's LOCK/UNLOCK

[acoustic sparrow]

yes but also lol

[blissful smelt]

which does beg the question

yes

idk

i cant figure out where that's coming from

[acoustic sparrow]

it probably got folded into the load

[dense vigil]

why is there a no-op lock

[blissful smelt]

ah

[acoustic sparrow]

maybe a seqcst load or a barrier or something

[blissful smelt]

that's the instruction

[acoustic sparrow]

lmao called it

its a seqcst barrier

[blissful smelt]

💥

[acoustic sparrow]

but yeah why not just lift it to like

FENCE(RW, RW); or something

[blissful smelt]

ghidra weird

this should really just decompile to CMPXCHG or soemthing

what an odd way to write this

[craggy spire]

What version is this?

[dense vigil]

i mean lock(); unlock(); kinda implies that

[acoustic sparrow]

well no not rly

[dense vigil]

what sort of spinlock doesnt act as a barrier

[acoustic sparrow]

but its all in a big lock held region?

hm

[acoustic sparrow]

well a non global one i guess

but a decompiler is all about readability

you are not producing code which accurately represents the machine code

that is impossible

you are producing pseudocode that is meant to make it easier to understand what the machine code is doing

[dense vigil]

but yeah this decomp kinda sucks

[acoustic sparrow]

whats the asm for this even

[blissful smelt]

win11 stable

[acoustic sparrow]

oh its just a fetchor lol

this is so stupid ngl

[dense vigil]

lock cmpxchg

[acoustic sparrow]

i wonder if binja does better

[blissful smelt]

lock cmpxchg dword ptr [r8], param_2

idk either

ghidra being odd

ah param_2 is EDX

[craggy spire]

IIRC in newer versions this is no longer the case

Possibly in older ones too

[acoustic sparrow]

i can't decide if this is like meaningfully better

__int64 __fastcall x(volatile signed __int32 *a1)
{
  return (unsigned int)_InterlockedAnd(a1, 0xFFFFC000);
}

``` oh and this is hexrays btw

which can actually recognize this CAS pattern

[digital pivot]

autoboost decompiles too poorly

[acoustic sparrow]

and also for some reason marked it volatile which is fucking insane

see https://dogbolt.org/?id=3ca2801e-b643-4a6a-899b-8efd29880f4b

most of them produce a cas loop and many do so incorrectly

this is really making me feel like i could totally make a top quality decompiler with this competition

[craggy spire]

Maybe, but it won't beat the memory manager

[acoustic sparrow]

is it really that hard to:

• not use signed __int32 as a type jfc you can just write i32 its not that hard
• not use volatile for atomics???
• also why is it marked __fastcall its a linux binary there is no fastcall
• not expand into a cas loop implemented without atomics
• ideally, just figure out what the atomic op actually is (which hexrays does but hexrays does other jank)
• not emit weird pseudofunction calls like ghidra

[teal trench]

everything is fastcall if you do it wrong enough

[warm pine]

the only decompilers anyone uses are either ida or nsa al-ghidra and both long predate even the low-quality __sync builtins of GCC, let alone the __atomic ones, which probably were added for the sake of the C++11 memory model

[acoustic sparrow]

i dont think thats even true

i mean ida yes

[warm pine]

the __sync builtins are only from gcc 4.1

[acoustic sparrow]

which is only a bit after the ghidra decompiler started development

[warm pine]

i don't think they were very popular

even today many big dogs like linux have their own memory model, barriers, etc, and i think they use volatile still

[acoustic sparrow]

perhaps. but LOCK() and UNLOCK() is still in no way readable

which is the most important purpose of a decompiler

i would be fine if they printed __cas or something like that

i dont hate how binary ninja on aarch64 prints ll/sc either (as calls to __ldxr and __stlr)

but LOCK() and UNLOCK() is really sucky. especially when you generate wonky decomp

[icy bridge]

my guess is they don't want the decompiler to lose information compared to the disassembly and making it something like __cas would result in cmpxchg/lock cmpxchg being decompiled identically

[acoustic sparrow]

cmpxchg shouldnt be __cas, that should just be turned into straight line code

only the lock variant is atomic and should have an intrinsic

[icy bridge]

the non-lock variant is atomic on UP machines

[warm pine]

wait, does it just do LOCK(); value = *ptr; if (value == expected) *ptr = desired; else desired = value; UNLOCK();?

[acoustic sparrow]

you dont need to surface every feature

[acoustic sparrow]

yes lmfao

[warm pine]

(or words to that effect)

[acoustic sparrow]

so does binja on x86

[warm pine]

well i can't say i care for that very much

i thought it would print a fake builtin or even some inline assembly (but then i suppose people don't want to see inline assembly in their decompiler output)

[acoustic sparrow]

isnt linux and msvc like the only ones which do that

[acoustic sparrow]

i would honestly prefer a fake builtin

[glossy leaf]

yeah the LOCK/UNLOCK/while loop stuff just makes it harder to read

[acoustic sparrow]

surfacing everything is impossible

strictly speaking any assembly sequence is atomic on UP machines if you do it in an rseq

[glossy leaf]

plus if you want to see whether its lock prefixed or not you could always look at the disassembly instead

[icy bridge]

fair enough

[warm mural]

IDA tho

[acoustic sparrow]

i dont have enough money

[warm mural]

🏴‍☠️

Is binary ninja worth it if I have IDA

[acoustic sparrow]

probably not

[warm mural]

Prettier UI though

[acoustic sparrow]

or well maybe

yeah the ui is way better

and the decompiler is usually worse

[warm mural]

if I wanted something legally I'd take binary ninja

Way cheaper

[sterile frost]

if you work with biiiig binaries then binja is much better

their analyzer is multithreaded lol

[warm mural]

Like how big

[sterile frost]

eh, like 30-40+MB probably will be noticeable

IDA is singlethreaded and it really shows sometimes lol

[warm mural]

Is that just starting time or overall?

like does it just affect how long it takes to start

[digital pivot]

KiAbReleaseTeeLockForEntry

i wonder what is a tee lock

is there such a concept in cs ?

[icy bridge]

is that not just a typo for tree

[blissful smelt]

ticket lock?

teecket lock

[icy bridge]

idk I'm not familiar with autoboost but it looks like a typo

[digital pivot]

lmao thats probably true

typo in autoboost spotted

[digital pivot]

so how do you like ghidra

[twilit smelt]

crazy that if prism weren't cancelled we'd all be sitting around talking about the typo in k$ab_release_tee_lock_for_entry

[blissful smelt]

🐉

[hollow musk]

nah, it'd be kern$...

[digital pivot]

btw is it patented

[warm mural]

I don't think its patented, because if it were then we'd have to know how it works

[twilit smelt]

It probably is patented and we just haven't found the patent yet

It's gonna be called something obtuse like "Method and system for efficient task interactions in a computer operating system"

[warm mural]

Time to look through every MS patent

[hollow musk]

not really, even if it is patented, I think you will see a very general description of priority inheritance there (based on my experience reading other software patents from microsoft and others)

Method and Apparatus for Assigning an Updated Priority Indicator to a Lock-Holding Actor in Response to Waiting Actors in a Computing Kernel

[craggy spire]

IIRC there are at least 2 Autoboost patents, but I don't have them on hand

[twilit smelt]

Patentese is the worst way to try to read about a technology

[craggy spire]

I saw both, but I don't remember anything really important

[twilit smelt]

It's a description of that technology with an audience of lawyers

[warm mural]

The RCU patent was surprisingly readable

[digital pivot]

anyone looked into xnu's rcu?

[hybrid condor]

so much chatter about AutoBoost lately

[warm mural]

Don't they do the freebsd thing?

GUS?

[hybrid condor]

what’s the deal with it?

[twilit smelt]

autoboost inspires interest because it's a way more complicated implementation of PI compared to turnstiles

[hybrid condor]

in a nutshell, it’s practically freebsd’s SMR

[twilit smelt]

it does weird stuff like the boosting of each hop in a waiter chain is apparently deferred

asynchronous

[hybrid condor]

I even found some bugs in their (reported)

so heavily spied on

and taken away

[warm mural]

wait what is GUS then

I mixed up my acronyms

[hybrid condor]

they call their SMR GUS

they = freebsd

[hybrid condor]

it’s windows, it can’t help but be weird

[twilit smelt]

me when I'm allergic to try/retry idioms so I make my boost hops into software interrupts

[citation needed]

[hybrid condor]

you made turnstiles for numa

🙄

[digital pivot]

its probably just one enqueue per all hops as a chain not per each hop ?

[twilit smelt]

I was mostly joking I'm unsure the details

[twilit smelt]

the best part is when you steal a turnstile that is local to another node and now for performance reasons at some point you have to free it to that node's turnstile cache and reallocate one for yourself and I do this as. an APC level software interrupt

[digital pivot]

anything wrong about numa ?

[twilit smelt]

the software interrupts cannot be escaped

sooner or later they come for us all

[hybrid condor]

will anyone be in cali in march?

there will be an opportunity to come and maybe even talk

there will be an event related to ios security

I think you're from california?

@liam

[twilit smelt]

I'm unfortunately from Utah

[hybrid condor]

that's not how I remember it.

[twilit smelt]

You might have spoken to the alternate universe Californian version of me

just tell me this... is he happy...

[hybrid condor]

then discord is back in hibernation after waking up from the announcement.

and what the hell is the verification nonsense?

if they don't stop I'll have to delete it

[delicate dome]

deep state show your govt ID if you want to access "18+" topics

[hybrid condor]

then it's good that I can survive without discord

[twilit smelt]

I mean it shouldn't come up just talking in places like this

This discord isn't 18+

[shadow ridge]

doesnt it also work for any server with like more than 1000 members

so you cant join unless u verify your age group

[twilit smelt]

Hope not

[shadow ridge]

it's so hard to get my friends to add me on matrix

at least i have one's irl phone number so it doesnt actually matter if we dont do it now or whatever

[hybrid condor]

you're no longer sure of the exact rules, and they may change

[twilit smelt]

I mean I could be in California for this event depending what we're talking about, I have friends I'm overdue visiting there so it might work out

[hybrid condor]

and what, they'll just remove people from the servers?

[shadow ridge]

no you just wont be able to use the server until you verify

[hybrid condor]

got it, can uninstall

[digital pivot]

so you uninstalled?

i wonder how many people really wont want to do verifcation

[shadow ridge]

50% of people are dumber than the average so