MINTIA (not vibecoded)

[icy bridge]

idk, and i don't really care about figuring it out because knowing that won't change how much of a pain it is to work with

[mortal thunder]

no support for recursive paging is one because the PTE layout is different between the leaf PTE entries and the non-leaf PTE entries

(i am not yehuda but here is my opinion anyway)

[mortal thunder]

namely, a non-leaf entry is described by having the R, W, and X bits in the PTE all equal to zero. any permission not equal to zero makes the PTE a leaf PTE

[signal zinc]

hmmm meow

[mortal thunder]

so even if the RISC V implementation hardcoded a maximum level of N, you still couldn't read the memory regions because they aren't readable

[twilit smelt]

plug n play will kill mintia2

if i dont find a way to get excited about it

[shut belfry]

https://www.youtube.com/watch?v=P0AJM6HMYjM

[twilit smelt]

ive been stuck on this for like a month

motivation wise

its just so boring

i just dont care

wheres the story man

wheres the sauce

wheres the juice

[shut belfry]

channel your inner steve ballmer https://www.youtube.com/watch?v=rRm0NDo1CiY

[twilit smelt]

my normal like 80s technology vibe tactics for getting myself hyped up dont work on plug n play because its too new

it feels wrong

unnatural

[twilit smelt]

at 22 years old i dont have 10% of the energy steve ballmer had at 40

[twilit smelt]

wait i figured out how to clean up an ugly thing thatwas bothering me

things might pick up steam

[marble socket]

microsoft loves developers

they even made a song about us!

[twilit smelt]

in my head mintia2 was still a really tiny stub

but coming back to it after like a month its actually pretty confusingly large already

sucks cuz it barely does anything

yeah damn there might be some major second system effect here that i just wasnt noticing previously

shit is gargantuan

for no good reason

seems sound at least though

at least its well commented

i think i did a lot more work than i was thinking

[mortal thunder]

my OS also suffers from second system effect, that's why progress is slow

[twilit smelt]

i was super insecure about getting like nothing done this year but now i think about it

this is like

40k+ lines in a year

thats probably a new personal record

?

idk what i was thinking

[mortal thunder]

but i did sacrifice some things to get things rolling

for example i wanted to get asynchronous IO right from the get go

but due to my non-understanding of IO request packets and how they're managed I didn't do it

[twilit smelt]

my major blockage right now is plug n play im still not like fully intuiting how the pieces all fit together even tho i have an overall plan

my current plan is to just try to forge ahead as directly as possible with doing like a disk driver or something and just do everything necessary to make that work lol

its possible i should scale back the io system goals

for now

everything else is so big already

mintia doesnt need to be osdev texas

i also fear its so like layered that its gonna be incredibly fucking slow

jackal compiler > dragonfruit compiler but also

mintia2 layering much more complex than mintia layering

[twilit smelt]

the trick with irps i think is just to understand how like custody of the packet structure passes around

who "owns" it and is allowed to manipulate it and stuff at any given time

[mortal thunder]

Yeah, I don't plan on moving to an async system immediately but at some point I might want to

[twilit smelt]

the synchronization reaaally confused me until i realized theres only like 1 context who ever touches it at any time and there are well defined points where its passed between contexts

where "context" could be a thread or an ISR or something else

or a spinlock critical section

or whatever

sometimes the context is owned by the system and other times its owned by like your driver

if you ever call like IoCompleteRequest (or whatever it is) on an irp, you owned the irp and you just passed custody back to the system (you no longer own it)

you cannot touch the irp ever again after calling that unless you get custody back from the system when it calls your completion routine or something

because it could be in ANY state now

bc u gave it away

stuff like that

[mortal thunder]

rn what i want to do is to implement the ext2 driver and get the userspace ball rolling

[mortal thunder]

what happens if an IO request issues another "child" IO request, and the parent IO request is waiting for the child to do more processing

for example reading in a directory entry

[twilit smelt]

each child that completes decrements a count inside the parent request

when the count reaches 0, the parent is automatically completed

[mortal thunder]

so the parent issues multiple IO requests that depend on each other serially I suppose?

[mortal thunder]

but I can't think of an example right away

for a file system driver for example metadata is cached and the actual file contents are not cached

[twilit smelt]

a better example is io that fragments on disk

so if you read like 5000 bytes from a file and theyre in like 6 different extents of disk blocks, that will generate 6 different child packets out to the disk

[mortal thunder]

and since its cached then a page fault might end up blocking to read that block of data into the page cache

[mortal thunder]

Yeah but those packets don't depend on each other serially

[twilit smelt]

the parent packet depends on the children

also the children can be truly parallel if you happen to have a striped disk or a volume set

[mortal thunder]

When you complete an IRP can you create subsequent IRPs before the current IRP is actually completed

in that IRP's completion handler?

[twilit smelt]

wym

ah

in that case youd return STATUS_MORE_PROCESSING_REQUIRED iirc and that basically tells the system's completion code to go "ope nvm" and bail out of completing the irp immediately

[mortal thunder]

ah

[twilit smelt]

and then its back in your custody

and you can do things like that with it

[mortal thunder]

that makes sense

[mortal thunder]

but what about this case? for example, when you're writing to a file and it needs expansion, you might need to allocate some blocks, increase the file's size etc

[twilit smelt]

NT gives up on avoiding a worker thread at that point

[mortal thunder]

to do that you need to update the file system's metadata

[twilit smelt]

if it notices something like that is required, it gets immediately tossed to a worker thread

[mortal thunder]

which is managed using the disk's page cache

[twilit smelt]

who does that work in a blocking fashion

[mortal thunder]

oh, fun

[twilit smelt]

this is probably the right choice because doing that asynchronously would be so obscenely complex its not even worth it

giga state machine

if metadata is not cached, that will also cause it to be tossed to a worker thread

[mortal thunder]

well what would you need to do? basically you'd need to check if that information is already paged in, if not, then issue an IRP that reads the metadata and would return "more processing required" on completion

this additional processing would put this data into the page cache (somehow, i don't know how you'd pull this off in a thread safe way) and then continue the file update

[twilit smelt]

btw in that situation old mintia doesnt even toss it to a worker thread

it just faults on the metadata pages in the viewcache

and blocks the requesting thread on synchronous page-in io on the metadata

[mortal thunder]

So it's only mostly asynchronous

[marble socket]

same

[mortal thunder]

wait nyaux isn't your first kernel?

[marble socket]

no

lol

[mortal thunder]

ngl boron suffers from like fifth system syndrome actually

the first two "nanoshells" (2019 and 2021) which were little more than bootable applications

the third "nanoshell" (2022) which is the publicly available one

the fourth "nanoshell" (2023) which was a 64 bit OS attempt

and then finally boron (2023+)

[twilit smelt]

ok if we're measuring by every single dinky project we did then im on my like 8th

[mortal thunder]

practically it's second system effect still though

also something that i wanna explore in the future is running 32 bit nanoshell applications on boron

even though it's useless since i could just port those to boron, i think it'd be a useful learning experience

[marble socket]

my first "kernel" was fucking written by chatgpt i "wrote" when i was 14

[mortal thunder]

i was 17 when chatgpt came out

oct 2022

[marble socket]

crazy

im 18

now

we getting old

[twilit smelt]

oh god it can NOT be that old now

it still feels like it came out a month ago

[mortal thunder]

oh no its november 2022

my bad

[mortal thunder]

that means you weren't 14 when you wrote your "chatgpt os"

15 is more plausible

[marble socket]

i was born march 2007

idk im not counting that shit

15 i was

crazy

[twilit smelt]

does that feel like a long time ago for you

[mortal thunder]

It does for me

[marble socket]

not for me

[twilit smelt]

late 2022 is like two milliseconds ago in my brain

[marble socket]

fr

i still think its 2016 sometimes

[mortal thunder]

clearly the sludge that was jackal development didn't hit you like a truck

[twilit smelt]

i have almost no memories of doing the jackal toolchain

[mortal thunder]

it took you like a year before you finally started to develop mintia2

[twilit smelt]

its just like compressed to nothing

[marble socket]

all i have is memories of watching fucking leafyishere when i was 9

leafyishere not beating the all his fans are 5 allegations

[twilit smelt]

yeah somewhere i have a message of me saying like "i bet i can do a new language and rewrite mintia in about 3-6 months" or something

from early 2023

mind you its been 2 years since i decided to do that

now

and im still grinding

old mintia's 5th birthday is coming up real soon

december this year

[mortal thunder]

i remember new year's eve 2023-2024

[marble socket]

-Mathewnd

[mortal thunder]

i was really trying to get the fireworks demo on boron stable and i didnt manage to do it in time

took me like a few days after to pull it off

it was a whole workout for that thing

in fact it helped me find bugs as late as september 2024

[marble socket]

never ran the fireworks demo on nyaux

might be worth it to run and see

when i feel like it

lol

[mortal thunder]

you should try it and see

[marble socket]

inb4 it triple faults

[mortal thunder]

when i get a relatively competent userspace i want to start porting boron to different platforms

like aarch64 (my raspberry pi 3b+)

and i386 i think

[marble socket]

crazy

port risc v

[mortal thunder]

possibly mips if i can manage to get my hands on a mips workstation without selling a rib

[marble socket]

this message has been approved by the marvin gang

@night needle

[twilit smelt]

this is relatable the #1 distraction from my project ive had this year is a particular dude ive been in an extremely melodramatic relationship

melodrama is annoying but its also fun its like im 15 again

[marble socket]

i should try and get into a relationship again ngl

maybe i should try and see what dating a boy is like

[twilit smelt]

we also have the most embarrassing possible answer to "so how did you guys meet?" we have to like prepare a lie to tell people

something normal sounding like coffee shop or school

[marble socket]

when anyone asks me about relationship questions i always say

so im dating your mother, might be ur new dad

that is how i avoid answering that

[twilit smelt]

ive been told by bi guy friends that relationships with guys are like a particularly intense friendship whereas with girls its a totally different thing entirely

i dont know what that means

[mortal thunder]

i wouldn't recognize anyone trying to marry my mom/my dad, as my new dad/mom

[twilit smelt]

but its data

[mortal thunder]

i was talking to a girl once

it was like trying to play minesweeper without numbers

trying not to fumble

and i did anyway

like casually it's fine. but when you have dating intentions? something else entirely

[twilit smelt]

i think ive only talked to girls from the angle of being one of the girls so i dont have any pertinent experience

lolz

in like a gay best friend way not a trans way

i fear the io system is going to be my great challenge with this kernel

i thought itd be the vmm

[mortal thunder]

you didnt even get started on the vmm

[twilit smelt]

but i might have no clue what im getting into with plugnplay and iokit-like driver matching

[marble socket]

vmm moment

[twilit smelt]

i already sweat blood to do the funnest hobby vmm once so im not that afraid of it, i just have to do the same thing but spruced up with some UVM ideas

[marble socket]

all i have thats "advanced" for the vmm in nyaux is demand paging

no swap vine boom no uvm vine boom

[twilit smelt]

but this is a whole different animal from anything ive done before

[marble socket]

https://tenor.com/view/jarvis-vine-boom-jarvis-vine-boom-gif-24471837

[twilit smelt]

its ok nobody else has swap either

except me and minoca

the cool kids

[mortal thunder]

what i have working is demand zero pages and page cache support (read only for now)

[marble socket]

crazy

[mortal thunder]

for now

[twilit smelt]

i got it like 3 years ago now too

and still nobody has caught up to where i was then meanwhile my power level has only increased

except like fadanoid i guess but i think that was short lived

[marble socket]

anyways i should be napping as it says in my status

i got very little sleep last night

[twilit smelt]

ppl been saying this for 3 years

[marble socket]

blame nyaux makefile bullshit

oh yeah and i starred mintia (didnt kno why i didnt have it starred)

[twilit smelt]

which one

[marble socket]

this one

[twilit smelt]

which this one

theres old mintia and mintia2

[marble socket]

mintia 2

[twilit smelt]

ok good

ty

[marble socket]

np

[twilit smelt]

https://open.spotify.com/track/3EWeWJHC9cxITJ0If6VX6J?si=1919b33883f74e5a

[marble socket]

https://open.spotify.com/track/6oodgVJsVi06ocDO9rwNig epic song

[twilit smelt]

boron viewcache when

[marble socket]

oh yeah question, how does ur language compile. does it go into llvm IR?

[twilit smelt]

no i did the whole compiler

[marble socket]

holy

damn

[twilit smelt]

it targets C, fox32 asm, and xr17032 asm

so only 2 fake computers :(

[marble socket]

still thats really impressive

ive never written my own language so

[mortal thunder]

soon™️

[marble socket]

must be really hard

[twilit smelt]

its more impressive than if it targeted x86 imo because i had to do all the other related infrastructure like the linker, emulator, firmware, etc

[mortal thunder]

probably it's one of the things required for my file system driver actually

not really required but itll be ass slow otherwise

[twilit smelt]

the OS however is less impressive as a result of only targeting the two fake computers lol

less opportunity to flex drivers

itll get an amd64 port tho

[marble socket]

oh yeah is the compiler also open source

[twilit smelt]

yes

[marble socket]

i wanna take a look at it

[twilit smelt]

its probably bad

but sure

[mortal thunder]

are you planning to tackle amd64 codegen or are you just going to use the CHost backend?

[marble socket]

repo link?

[twilit smelt]

https://github.com/xrarch/newsdk/tree/main/Jackal

[twilit smelt]

undecided

[marble socket]

epic

nice ur compiler is written in its own language

thats hella based

[twilit smelt]

yeah its self hosted

so that i can build mintia2 inside itself on my own computer architecture

one day

[marble socket]

holy shit IrGenerate.jkl is 2557 loc

[twilit smelt]

the toolchain has already been built for fox32os

and has been used to write programs from inside that

[marble socket]

based

[mortal thunder]

i wanted to write a shitty compiler too but it's tedious

[marble socket]

when Jackal LSP

[twilit smelt]

jackal compiler is bad but its big anyway because its a "production" compiler used to write a big ass kernel

[mortal thunder]

shitty because i wasn't planning to actually do many optimizations

[twilit smelt]

like it has to actually work and be useful for that task

[twilit smelt]

only in terms of drivers tho ill note

xrstation has SMP and paging and shit

so the kernel proper can be written taking all the most important trappings of a modern computer into account

and has been

and will be readily portable to amd64 when theres jackal toolchain support for that

also someone else is working on an alternate jackal compiler called coyote

@lucid umbra

[twilit smelt]

this sucks to have to explain though bc if you say "fantasy computer" people imagine a toy like chip8 and not like a literal 90s workstation computer except fictional

so im looking forward to the amd64 port when the os can shine more

it has to actually like

do stuff first though

[twilit smelt]

the parser is 5400 lines

for some reason

prob doesnt need to be that large

[warm pine]

the swap is still there, i got the current keyronex vm up to a mostly reasonable standard and robustness

[twilit smelt]

the bits of jackal that coyote has implemented have so far been more compact and more functional so im probably just shitty at compilers. id be willing to replace mine with theirs one day if it gets to that point

[warm pine]

but then i get tempted by things like that new working set aging approach

with the scanning of page tables queued onto a linked list

[twilit smelt]

speaking of robustness i really fear for mintia2 i feel like its gonna take a year straight of debugging to get it up to par

bc im not like slowly adding features and testing them thoroughly im like building the entire thing up at once in a huge monolith

i mean im testing stuff

but its not as thorough as the iterative development of old mintia was

where each little advancement got a lot of dedicated debugging time

cuz this time i planned the entire thing and im building it all up to the full feature set immediately, so i have no solid intermediate ground to stand on. when i get to userspace theres going to be like 1500 bugs suddenly revealed at once

[twilit smelt]

i forgot about that

yeah im definitely just gonna do the same fifo working set array thing

at least at first

[warm pine]

this is one area where i was really very tempted at first to dive into doing the fancy new thing but i've fixed bounds around it for now and i've managed to avoid pursuing it

same with some of the more extreme lock granularity reduction in the vmm

it has to wait until keyronex is at a similar level of development as it was in the last revision, with xorg etc running stably

[twilit smelt]

has development been steady

[warm pine]

for about 6 weeks now i have been intermittently looking at a function i started writing as part of the STREAMS stack, the function is to be the public interface to read messages from a stream

it's basically a trifling function to write. i already did the hard part when i invented a special locking mechanism specifically for STREAMS inspired by what OSF/1 was doing (and Linux's socklock is also similar)

but for whatever reason i open the editor, look at it, my eyes glaze over, i write nothing or very little

[twilit smelt]

was this what you mentioned in my discord

[twilit smelt]

im having the same issue with pnp

[warm pine]

if it's about the mechanism to serialise readers which can be either IOPs or synchronous readers with ever combination of MSG_PEEK, MSG_WAITALL, etc then it would be

[twilit smelt]

yeah that

ive probably asked this like 3 times before but where did you read about OSF/1's streams implementation

[warm pine]

design of the osf/1 operating system book

[twilit smelt]

im unsure im still going with streams because the stacking im doing now is flexible enough i might be able to accomplish the same basic thing without adding more orthogonal concepts

in a reasonable manner

[twilit smelt]

I'll make sure to read that

[warm pine]

i have the vague feeling it should be possible

[twilit smelt]

I'm reading magic garden way belatedly, prob should have read it like 4 yrs ago

[warm pine]

it's a good book, i don't think i was really captivated by its description of STREAMS but i barely remember that part of it

[twilit smelt]

I'll read the osf/1 book as soon as I'm done with it

And I'll have a better angle to ponder streams from

Do you have any pnp plans

[warm pine]

i was going to figure that out with a simple case (virtio-scsi hotplug) then pitust told me to write an xhci driver and i ended up doing so and not getting much further along with pnp plans, other than clarifying my understanding of what's needed for usb

[twilit smelt]

you should do a write up of whats needed for usb

that would be useful for me lol

i feel like i have a vague idea from skimming the docs for how iokit and NT handle usb but its definitely not as clear as yours

[warm pine]

i think it would be useful for myself as well, i've now forgotten a lot of it

[twilit smelt]

next thing i have to do is implement the io catalog i believe

ill have to change how the bootloader loads necessary drivers

itll need to load their little config files as well

to initially populate the io catalog for boot time driver matching

driver matching itself is a process that im unclear on how it should work other than i know i want it to be multithreaded

ive kind of figured that it will reveal itself to me if i just start implementing it

most things do

but im afraid

[twilit smelt]

first i have to do math homework which I still have at age 22 :(

[twilit smelt]

automatic push the ice

[twilit smelt]

@wicked lichen hello

[wicked lichen]

hi

[twilit smelt]

everyone this is my friend amelorate give them a warm welcome 👏 👏 👏

i met them outside a movie theater after leaving the minecraft movie last night

i said "im will" and amel was like "will? THE will? please let me into your legendary osdev discord thread" and i had no choice

[wicked lichen]

i dont know how i got here

[green pelican]

https://tenor.com/view/kitty-cat-hello-chat-gif-17407572284964392117

bonjour

[marble socket]

holy osdev met in real life????

damn didnt know that was possible

[devout geyser]

Dayum

[warm pine]

the way they did minimalism in those days was somehow better as wel

the font choice here is extremely modern

the observant will note that what people claim to like about modern minimalist aesthetic was already done better in 1969

(the above + barbara kruger's "your body is a battleground" are the direct inspirers of the keyronex logo)

[fossil crypt]

supreme ass image

😭

this shit I mean

[warm pine]

yes, they plagiarised her worse than i did

[fossil crypt]

ah

[twilit smelt]

on this day the second anniversary of the jackal language beginning to be implemented

let us take a moment to remember what came before

let us bless dragonfruit and thank it for what it did for us

and hold it in our memory

[twilit smelt]

changes to the boot process need to be made

currently the bootloader is capable of loading and dynamically linking modules

this will shortly be redundant because the kernel needs to be able to do this as well

nvm

its not redundant bc its nice to be able to dynamically link modules together before the kernel has really done much of anything (for example an optionally loadable debugger module)

its not too much code anyway

so anyway ill just allow it to be a little redundant

in any case the bootloader needs to load config files describing the drivers that it loads at boot time so that the io catalog can be populated

ill reuse the same "resource file" mechanism i already have for loading whole files uninterpreted for some later usage by the kernel

[twilit smelt]

mintia2's little known yaoi component

theyre STILL releasing posthumous mac miller albums??

[night needle]

SIMH

[dense vigil]

What does it stand for

[night needle]

single instruction multiple headaches

[twilit smelt]

feature i just kind of accidentally added bc it was suddenly easy

bootloader lets you select the system directory now

with an argument

by default its /mintia/ as the base for everything but you can now do SYSDIR=/some/other/path as a boot argument and itll use that instead. for this to actually be useful the rest of the OS will have to respect this too but that shouldnt be too hard down the line

will let multiple installations of the OS live on one partition

ill probably have a handle to the system directory be a standard handle that gets inherited in child processes for easy relative name lookup

[wicked lichen]

someone likes pic related as a bootloader
https://youtu.be/f6gEDUigoXw?si=Fc2sOn9oOQV6-zht

And yes I know I'm exposing myself as an x86 poser.

[twilit smelt]

unlikely criticism for u to receive

vast majority of ppl are doing x86

[twilit smelt]

looks like im going to be going for "drag and drop" system extension installation in mintia2

the bootloader will iterate the /mintia/Drivers directory looking for directories whose name is DriverName.kxt and itll load DriverName.cfg from inside as a BlResource

well no the bootloader wont do that actually

the kernel will later

the bootloader should load exclusively whatever drivers are required to access the boot device

it might automatically load all the filesystem drivers

[twilit smelt]

no it wont

only the appropriate filesystem driver for the selected boot device

[twilit smelt]

my fav thing is that my discord tag before they got rid of them was #1860 by chance and that looks like i860

and i get to keep that in my profile

[twilit smelt]

bootloader work ...

[twilit smelt]

note to SELF:

[twilit smelt]

bad

dont reuse the resource mechanism for this itll waste too much space on page alignment

need to do a special arena type thing instead for loading driver config files

too sleepy to do that rn

[night needle]

fox64 when

making an imaginary cpu and isa sounds fun

[twilit smelt]

ask ryfox that

that aint my imaginary computer

im just writing an OS that targets it

[night needle]

damn

[twilit smelt]

(along with my own imaginary computer that i did make)

(it runs on both)

(runs on 2 fake computer)

(portble)

there is only one major difference in functionality between the two platforms

which is that on mine it has full smp support

bc my platform can do smp

on fox32 it doesnt bc theres no such notion there

otherwise the goal is for them to work identically

we kind of have a larp going on where fox32 is like a fictional home computer platform

and xrstation is a fictional workstation/server platform

of like the early 90s maybe

we didnt collude on this we both started doing our own fake computers independently and ran into eachother later, its been fun

[night needle]

does fox32 have vma

[twilit smelt]

vma?

virtual memory addressing?

[night needle]

yes

[twilit smelt]

yeah it has a paging mmu

[night needle]

waow

[twilit smelt]

its roughly analogous to like a simplified 386

[night needle]

neat

[twilit smelt]

xrstation has paging and smp

and yes i have to do tlb shootdowns and stuff

on my fake computer

or it dont work

which is absurd

at that point u might wonder why im not just doing a PC operating system

good question!

.

[night needle]

yea

[sterile frost]

i did attempt an amd64 port of mintia

which went surprisingly well, considering i got pretty far where it did the idle thread memes

but then toolchain memes defeated me :(

can't wait for an official jackal x86 backend :^)

[twilit smelt]

sandwichman is doing a custom jackal compiler as the initial frontend for his own fancy compiler

and that may be most likely to get an amd64 backend first

[mortal thunder]

@twilit smelt what exactly should hold references to the view space objects? (file object? fcb?) I'm thinking of implementing a scheme similar to yours for view caching but I wanna be at least somewhat original

I've just implemented (but not tested) the code that maps views of files into system space

[twilit smelt]

well i got the idea from NT for the record

[mortal thunder]

yeah, it makes sense

[twilit smelt]

the implementation was mine though

[mortal thunder]

ah so your implementation is different from that of NT's

[twilit smelt]

well i didnt look at NT's implementations for old mintia

[mortal thunder]

makes sense

[twilit smelt]

i just read lots of books about NT and the workbook and stuff

[mortal thunder]

i suppose you picked the FCB to hold references to views mapped into system space?

[twilit smelt]

u could read mintia for this information

[mortal thunder]

i suppose i could do that

[twilit smelt]

but to sum up basically yes it was the fcb that would contain information about both the page cache and the view cache

it was split out into another structure called the cache block which would only be allocated when you actually did a cached io operation with the fcb though

to save memory

because most fcbs would never actually be used for that

[mortal thunder]

oh, i see

[twilit smelt]

but the cache block's lifetime was equal to that of the fcb after it was created, so its best thought of as just part of the fcb

that doesnt get allocated until its needed

[mortal thunder]

for the offset <-> view mapping did you use an AVL tree? radix tree?

and also i wonder what kind of sizes of views should i have for both 32- and 64-bit

for 64-bit they can be quite large because i set aside 512GB of VA space for them (and other things)

[twilit smelt]

NT uses 256k views i think

i used 32k bc i didnt want to allocate too many page tables for the view cache space

[twilit smelt]

avl tree

[mortal thunder]

i see

thanks for the answers

[twilit smelt]

youd want to use larger almost definitely though

one of the benefits by the way is that you make the recently used file cache pages part of a working set, which makes them subject to the full breadth of the page replacement policy

another one is that it removes a dependency on an identity mapping youd otherwise need in order to efficiently access file cache pages

[twilit smelt]

with working set based replacement like NT's, if you didn't do this, file cache pages would be consumed too quickly because theyd always be on the standby list for instant reclamation

which occurs preferentially over trimming any pages from working sets

[twilit smelt]

this was the specific thing that compelled it in old mintia

i got rid of an identity mapping i previously had, bc it was a bad thing to have on 32 bit

(or hhdm i guess in liminespeak)

which required a new efficient way to access file cache pages

for the record when i made this change i didnt really observe any difference in some (tbf fairly artificial) benchmarks i ran

while memory was plentiful

it didnt really get better or worse

[twilit smelt]

and it was difficult to artificially create a situation where this benefit would appear in the numbers

but its something i take as trivially true lol

[twilit smelt]

difference could be larger with much larger viewcache views like 256k

[mortal thunder]

wdym

like, an identity mapping that allows direct access into the pages?

[icy bridge]

hhdm

[mortal thunder]

yeah

[mortal thunder]

surely i'd use something larger on 64-bit

i'd think 1gb views or 512mb views would work

[icy bridge]

also I forget, are FCBs equivalent to vnodes or file descriptions

[mortal thunder]

in NT parlance FCBs are equivalent to vnodes and FILE_OBJECTs are equivalent to file descriptions

[icy bridge]

ah ok that makes sense

[fossil crypt]

the pdp-11 (iirc) emulator?

[twilit smelt]

thats somewhat absurd

thats welllllllllll past the point of diminishing returns

[mortal thunder]

i mean 64 bit address space

why not

free (but small) speed gain!

[twilit smelt]

just put the lock ordering inversions in the bag bro

[marble socket]

just put the code in the bag bro

put the mintia in the bag

[raven drift]

oof

maybe he should stick to his open source projects instead of posting his opinion on "sexual energy"

[twilit smelt]

nightmare fuel: manosphere adjacent andreas kling

[dense vigil]

Every man knows the number of locks you can have in a kernel is finite

thats why we have the big process lock in serenityos

[raven drift]

lol

[twilit smelt]

@warm pine is there source for for OSF/1 for Alpha

aka tru64

[warm pine]

no or at least not publicly distributed and well-known

wait never mind, it's there after all in the osf/1 1.0 source release

[twilit smelt]

can you link that again

[warm pine]

https://github.com/ryanwoodsmall/osf1/tree/master

[twilit smelt]

thanks

also https://vmssoftware.com/docs/dtj-v04-04-1992.pdf#[{"num"%3A1114%2C"gen"%3A0}%2C{"name"%3A"Fit"}]

bookmarking that

[mortal thunder]

the what bro

what does it guard

[dense vigil]

concurrent syscalls

[mortal thunder]

theres no way

what kind of syscalls

[warm pine]

presumably most of them

but as we can see klinge has abandoned serenityos in favour of giving advice on sexual energy

now who would be so kind as to take one giant step for all mankind, into the patented klinge orgone accumulator

[mortal thunder]

klinge has abandoned serenityos indeed

[mortal thunder]

apparently 21 syscalls out of their 156

but interestingly

read locks the process lock

for what reason, I cannot imagine

also i bet this isnt even a proper blocking mutex

[warm pine]

and write?

[mortal thunder]

i was about to check that. yes write too

[dense vigil]

any

a process cannot run two at the same time

but yeah they have tried getting rid of it

or have been for multiple years

[dense vigil]

well they have tons of issues with mutexes taken while under a spinlock etc

thats partially why smp is disabled by default

[mortal thunder]

why would you take a mutex when you're holding a spin lock

you should not be doing that

[dense vigil]

poorly designed code

[mortal thunder]

also that doesnt seem true

in the grand scheme of things 21/156 syscalls require the "big process lock"

[dense vigil]

yeah like i said

they've been trying

most misc syscalls dont have it nowadays i guess

[mortal thunder]

sadly no one competent has come and fixed it

they have this "fix it yourself if it bothers you" attitude

which is fine in some cases

[dense vigil]

yeah

now that andreas left it it's basically on life support for the most part

[mortal thunder]

i suggest we just replace the serenity kernel with the boron kernel.

[dense vigil]

yes

[mortal thunder]

although the boron kernel is still far from complete so

but ill get started on a more competent userspace soon

rn im like 3 calls deep

i wanted to start writing an ext2 impl which requires me to implement standard IO caching which requires me to implement file-views-in-system-space management

i have a lot of code already written but im not sure it's any good and i havent even gotten to test it

also i have my finals within the next like 3 weeks

[dense vigil]

that sucks

[mortal thunder]

sure it does

i dont need much effort to PASS but i should put in some effort to get some good grades

[warm pine]

they were supposed to be rewriting the serenityos kernel in jacket at one point

at some point that goal was dropped

[mortal thunder]

oh i seriously forgot the name of that language and i thought "ah yes, jackal"

for the record, it's jakt

[twilit smelt]

I asked them once "wouldn't this be a good time to redesign the kernel and add important features like a page cache since you're rewriting it anyway"

I got like 5 eye roll reacts and ppl going like "great another ideas guy. No. That's not practical. We can't do a new language and a new design at the same time. In the real world these things are done in steps."

They were comically skill issued basically

[wicked lichen]

I mean scope creep is a real problem.

I imagine nobody wants to accidentally rewrite NT.

But a solution to scope creep is just writing down the features you want and then removing everything you can live without.

I say this having never once battled scope creep because I do all my programming kitchen sink.

[icy bridge]

wait they don't have a page cache? how do they do mmap?

[twilit smelt]

silly style

theyd sometimes talk about page caches but as this like far off inaccessible, impossibly sophisticated thing

prob because thats almost how theyre presented in osdev forums adjacent material

[icy bridge]

lmao what

i'd say they're one of the easier forms of caching

[twilit smelt]

it does have its own issues but yeah

[mortal thunder]

by reading data within the page fault handler and just memcpying to the page

seemingly

i wonder how it deals with situations where multiple programs mmap the same file

[icy bridge]

yeah that sounds like it breaks in all but the most simple situations

[mortal thunder]

i would say it's simpler than one might think

but not necessarily very simple

theres a lot of cases especially involving write accesses where you can mess up

and you also have to write a PMM that knows how to deal with currently unmapped but still cached pages

[shut belfry]

lmfao

[twilit smelt]

from osf/1 for dec alpha's pmap module

just write code like a normal person. freak!

[twilit smelt]

i identified a huge embarrassing issue with the smp stuff in my emulator

@mortal thunder likely source of the stuttering in the fireworks demo

when a cpu hlts when it goes idle, the host thread for the cpu gives up the rest of its timeslice (a whole ass 16ms worth of cpu time) and goes back to sleep

if another thread ipis it during this time, it wont respond for another 16ms

if i remove the hlt in the idle loop and replace it with a pause the stuttering almost completely disappears

i think the only real solution here is to decouple cpus and threads

i think any thread needs to be able to run any cpu and can timeslice between them

and switch early in response to lots of pauses

i should probably just read hypervisor materials atp

this sounds like a problem set that was solved for hypervisors in like 2002

[twilit smelt]

hm

that seems to work nicely

@mortal thunder when u have the chance can you test the newest emulator commit to see if it helped resolve some of the smp stutter

theres a new -threads [number] option you can experiment with as well

if you do -cpus 4 -threads 2 for example itll do 4 simulated cpus and 2 host threads

[twilit smelt]

oh wow this is so much better lol

i changed the default number of threads to be half the number of cpus

if i do -cpus 8 it creates 4 cpu threads

and this now actually allows mintia's scheduler to scale - i can crank the number of firework particles way up and it keeps up

before these changes it would choke to death even with 8 simulated cpus, it wasnt actually scaling with them

it still sometimes stutters visually and im still unsure what causes this

but this was a huge W for performance scaling

fascinatingly it also improved cpu usage of the emulator

prob cuz it reduced the amount of spinning the cpu threads do waiting for spinlocks to be released and stuff

also theres still some cycle wastage as a result of hlt and stuff

but its now on a millisecond granularity rather than a real life frame time granularity

which seems to have been a massive improvement

[twilit smelt]

im pleased with today's labors this was a pretty small change that had a huge impact and resolved some longstanding issues with the smp simulation

i was supposed to be doing homework but this was cool too

[mortal thunder]

thanks for letting me know, i will try as soon as I can (which isnt very soon sorry)

[twilit smelt]

incredibly strange occurrence

this error message appeared on a build

<very long path>/immintrin.h:123:10: fatal error: 'avx512vlvnniintrin.h' file not found

i rebuilt again

no error

[mortal thunder]

avx512?

i guess you were building on intel, not on an m1? mac like usual

[twilit smelt]

i dont think the particular file it happened to is significant

yeah

its very strange

somewhat alarming

for the filesystem to randomly claim a file doesnt exist and change its mind a few seconds later

lol

[mortal thunder]

yeah, that's pretty weird

that seems to be a real filename so it shouldnt be like data corruption (single event upset etc) or anything

hopefully

[twilit smelt]

just debugged a crash in old mintia whose only symptom was this and it turned out to be buggy disk driver causing data to be beamed across page frame boundaries if a request's base wasn't page-aligned, causing mass chaos

took like 20 minutes from noticing it to fixing it

good to know i still have like sixth sense for debugging this codebase even though i havent touched it in a year

thats a good use of my limited synapses

the bug was introduced over a year ago when i changed how the disk controller works and hastily updated the old mintia disk driver

ive rarely used it since then so i didnt notice i introduced a family of evil crashes

i didnt do the "good fix" where i make it chunk up the transfers correctly, i just reverted to only doing 1 sector transfers

fixed it at the expense of slowing down disk IO a lot

dont really care enough to spend the time slogging through a bunch of tedious arithmetic in RPN dragonfruit

to make it work right

i was like just barely fitting a bunch of data squirreled into some fields in the io request packet structure and stuff anyway (code i wrote like 2-3 yrs ago) and it would have required rethinking that

too much effort for a dead project

making it work is good enough

the mintia2 disk driver can do it fancy

also i only debugged this at all to make sure it wasnt a bug in the emulator itself

cuz thatd be a party pooper later

i should start a blogge

maintenance stories from my fucked up 5 yr old miniature computer ecosystem

[twilit smelt]

I've been blocked on the IO system for mintia2 for soooooo loooooong

Motivation is low

I've never been defeated by smth i wanted to do though and plug n play can't be that much worse than like page swapping right.

[mortal thunder]

honestly i'm also blocked on view cache management but thats mostly because of my finals actually

[twilit smelt]

what must be done is

I must

make the bootloader load the initial config files for the drivers into a contiguous arena specifically for that purpose rather than allocating integral numbers of whole page frames for each one

this will save lots of memory

that's the very next thing for me to do

Then I need to crunch the config files into the IO Catalog

ugh I need to also ğħğħħğğ

I messed something up last time I worked on it

I need to load all the config files for all installed drivers in the bootloader. all of them.

more can be discovered at runtime but that's how it has to be at boot

that'll require iterating the driver directory and loading all config files

I added machinery for this to the bootloader but then I forgot why and removed it

Consequences of working on this once a month

[shut belfry]

You did commit those changes, right?

[twilit smelt]

i did

[twilit smelt]

im proud of this cpu work loop in the emulator https://github.com/xrarch/xremu/blob/91c2949ce138473ae0042480ff9cc9aa9a38e20a/src/main.c#L76-L106

its probably still shit but it actually scales

unlike how it used to work, which was 1-to-1 host threads to guest cpus

where each host thread would only execute simulated cycles from the same guest cpu each frame

and had no ability to "reschedule" to another guest cpu if the current one was doing nothing but spinwaiting

so it scaled TERRIBLY before but now it scales good

@mortal thunder did you ever test whether some of the fireworks stuttering was resolved by this

on your end

in fact i should probably turn this into a more general scheduler that can accept work items timed in virtual milliseconds into the future for things like disk and serial IO

and then i can get like nearly perfect millisecond resolution timing for those things

from the perspective of the cpus

i can probably do something clever like simulate disk io timing relative to the timing of the cpu that the disk interrupt is routed to

so like each virtual millisecond that ticks for that cpu, a virtual millisecond ticks for disk io

...or just tick it relative to the cpu that started the IO

thats probably better

[mortal thunder]

no I didn't get to do it yet

[twilit smelt]

i hadnt noticed some of the more interesting system services in mica before

VMS-y block oriented IO system

sparse files

kernel mode volume formatting

even vms didnt do that, i think

[warm mural]

@twilit smelt https://github.com/xrarch/mintia2/blob/main/OS/Executive/Ke/KeDispatch.jkl#L944 probably a typo, you should put the nullity check before dereferencing it

[twilit smelt]

o ya not good

good catch

im pretty sure it cant ever actually be null

so thats not actualyl a bug

[warm mural]

yea but still

[warm mural]

also any particular reason to use a next thread variable? Couldn't you just do something like:

sched_clock():
  td = pick_new_thread()
  switch(curthread, td)

instead of:

sched_clock():
  next = pick_new_thread()

timer_clock():
  sched_clock()
  switch(curthread, next)

idk if that shitty pseudocode makes sense

intuitively I would think it's because next can be manipulated by some operation in the meantime but the switch is literally just after picking it so idk

rn I'm going with the first logic, but idk if there's a problem I'm missing

[twilit smelt]

idk why youre associating this with the clock

look at the places where NextThread is set

[warm mural]

is it not set on quantum end?

which may happen on clock ticks

https://github.com/xrarch/mintia2/blob/main/OS/Executive/Ke/KeIpl.jkl#L76 here NextThread is set in KiQuantumEnd

it is later switched to on line 99

[twilit smelt]

it is

but thats not the only place

[twilit smelt]

what is sched_clock() even here

like where is that called

[warm mural]

it would be called on quantum end, but in my model a quantum = one clock tick

[twilit smelt]

whenever a thread is selected to preempt the current thread on a processor, the NextThread field is set

this happens for example when a thread is unblocked by cpu A and the best cpu X is selected to run it

and its higher priority than the current thread being run by cpu X

[warm mural]

so you just set X->NextThread to that thread?

[twilit smelt]

cpu A will set cpuX^.NextThread to this thread, set an interrupt pending at IPLDPC, and IPI it to get it to dispatch its soft ints

what this accomplishes is that cpuX doesnt need to search its queues to tell what to switch to

which would duplicate the work cpu A did

it knows that the value of NextThread is the highest priority thread assigned to it and that it should switch to it asap

the reason quantum end also does this is just for consistency, you could get excess context switches and stuff otherwise

[warm mural]

ah yeah that makes sense, I thought about that and went "you can just call pick_thread()" again but yea it does save the computation

[twilit smelt]

also cpu A checks if it should preempt X's current thread to avoid excess IPIs

[warm mural]

and it is chosen as the best CPU because that CPU does not have any threads with higher priority than that new thread, correct?

[twilit smelt]

no need to IPI it if this thread wont preempt the current one anyway

[twilit smelt]

iirc the order for "best cpu" is

1. the last cpu it ran on, if its current thread would be preempted; if not:
2. the cpu with the lowest load whose current thread would be preempted by this thread, if any; if not:
3. the cpu with the lowest load overall

load is just measured by the number of threads enqueued to the cpu

[warm mural]

ok that part I don't get, when and why do you do IPIs? Is it to signal the CPU to preempt whatever thread it's running because a newer higher priority one just came in?

[twilit smelt]

this tries to get the thread running asap for responsiveness and makes a best-effort attempt at having the N cpus of the system run the top N highest priority threads runnable at any moment (not perfect because theres no big scheduler spinlock, but its pretty close)

[twilit smelt]

how else would you get it to do that

[warm mural]

idk I was asking

but the thread is not actually preempted until its time slice expires, right?

[twilit smelt]

wrong

[warm mural]

if so, you can just add the new thread on the queue and that particular cpu will find it next time it calls pick_thread()

[twilit smelt]

thatd be really shit

thatd be ass

its preempted instantly

as soon as a higher priority thread is available

this is EXTREMELY important for responsiveness

its more important than timeslicing

[warm mural]

ah yeah I'm misremembering, ULE delays preemption but just until the thread is out of a syscall handler

[twilit smelt]

i dont think so

[warm mural]

to ensure locks are not held

[twilit smelt]

that makes no sense

its fine to preempt a thread that holds (blocking) locks

thats what theyre for

[warm mural]

https://youtu.be/jHROlCZV58I?t=1808

[twilit smelt]

what youre telling me is that the freebsd kernel is never preemptible under the ule scheduler

that sounds wrong

[warm mural]

idk man that's what marshall said

[twilit smelt]

i dont hear that at the timestamp you sent

[warm mural]

it's around that time, lemme find the proper timestamp

28:49

also

relevant i think?

this is from the ULE vs CFS paper

" In ULE, full preemption is disabled, meaning that only kernel threads can preempt others."

[twilit smelt]

if thats true thats not for any functional reason thats just their policy decision

which i disagree with on desktop bc it sacrifices responsiveness

[warm mural]

they probably tested it

[twilit smelt]

and found that it helped throughput at the expense of responsiveness most likely

and they decided they liked the former better

[warm mural]

also apparently ULE does not send an IPI unless the new thread is realtime

[twilit smelt]

thats also true of mine

timeshared threads never preempt other threads

[warm mural]

ok so basically when a new thread with a high priority is enqueued, and CPU X is chosen, set X->NextThread to that thread, and send an IPI if prio > REALTIME. So if the thread is timeshared even if it has a higher priority than the current thread, the current thread is not switched off until its timeslice expires, and if NextThread is realtime then it will get an IPI and the current thread will immediately get preempted

[twilit smelt]

thats not what real time priority means in ule parlance

its not a range of priorities

[warm mural]

realtime can also mean interactive

[twilit smelt]

yeah

[warm mural]

but there are realtime priorities

there's a range

[twilit smelt]

yeah that also exists

that range of priorities forces a thread to always go in the real time queues and behave like an interactive thread in preemption behavior and stuff

in mintia

i dont think thats a thing in freebsd ule idk

[warm mural]

not prio > REALTIME then, but rather if (threadqueue == realtimequeue) or something

[twilit smelt]

so this check is wrong

[warm mural]

yeah it is

if you have a realtime priority thread it will always be ran until it's voluntarily sleeping

if thats what u mean

same behavior for interactive threads, though I think their realtime priority is effectively lower

[twilit smelt]

ok so the freebsd kernel IS preemptible

you misunderstood what he was saying

[warm mural]

(a priority is calculated for interactive threads based on score)

[twilit smelt]

NEEDRESCHED is the freebsd equivalent of deferring reschedule using a soft int at IPLDPC

they only do that if the thread was currently holding a spinlock or something else

[warm mural]

ohhhh yea that makes a lot of sense now

I think what I was saying wasnt bullshit, but I misinterpreted it

also I think they slightly boost threads that are currently in syscalls, does mintia do that?

[twilit smelt]

no because that doesnt make a lot of sense

[warm mural]

maybe I'm misinterpreting again what he's saying, but I'm pretty sure he says that it's to "get out of the kernel as quick as possible"

[twilit smelt]

why would i let a thread get an execution time leg up over other threads just because its spamming syscalls

[twilit smelt]

whats his rationale for why this is inherently good

[warm mural]

sec lemme find the clip

[warm mural]

also I dont think this is a problem since priority is adjusted based on the time the thread ran in total / cpu time in total

so its cancelled out

[warm mural]

https://youtu.be/jHROlCZV58I?si=c04LWdZ4GfoBSmFh&t=117

[signal zinc]

i should hack some lints into coyote

i could farm so many "check for null after dereference" and such

[twilit smelt]

no u couldnt mintia2 is wonderful

[warm mural]

dunno if thats easy to do

[twilit smelt]

i think my kernel code is probably fairly good at least relative to most hobby kernels

but mintia2 rn has tooooooons of completely untested code

[mortal thunder]

i also wonder if on amd64 you plan to use the periodic or one-shot LAPIC modes (so, you can either program an interrupt in X LAPIC ticks every time, or have it automatically reload with the same value)

[warm mural]

tickless doesnt really matter if u have fixed timeslices I think?

freebsd actually does have variable timeslices kinda (the timeslice is the same for all threads but it is adjusted at runtime)

[mortal thunder]

no one said the timeslices have to be fixed

and also it could matter for things like timer latency (though, only on the order of microseconds)

[twilit smelt]

Yeah it does

[warm mural]

Why? I know there's a thing about power usage and stuff but idk if there's any other reason apart from energy saving

[twilit smelt]

You can more easily get higher resolution timing when needed

[warm mural]

I guess this would be useful for programs with real-time constraints or something?

Because if your tick is fast enough (something like 100hz) it's not that bad if you sleep one tick too much on a regular system

[twilit smelt]

I mean with tickless you can say your minimum resolution is something like 1ms and you coalesce all timers within the same 1ms span and otherwise you set a real timer interrupt for each one

And you then have 1ms resolution without needing a 1000hz tick

which in the vast majority of cases will be faster and so on because most timers will be further apart than that

including the thread timeslice timer which will be like idk 10-30ms

So it's a huge deal on multiple dimensions even if you have a fixed thread timeslice

[warm mural]

Yeah right

But presumably you're gonna have your timeslice timer always triggering at a fixed frequency so you can just handle timer expiration there even though that loses resolution

[twilit smelt]

(copypasted from elsewhere)
im gonna make a change to the platform
im going to make the interval timer per-cpu
right now its global and the timer irq is routed to all cpus in parallel
im going to make each cpu be able to set its own timer
and then im going to make mintia2 tickless

in fact

i may turn it into an architectural cycle counter

that counts down to 0 and then triggers an internal exception

also

massive document diving into the Copland IO system @warm pine https://www.pagetable.com/docs/copland_docs/D11E4/Modular I_O.pdf

its designed for actual microkernelness it seems

and has the only detailed description ive seen of a plug-n-play model for a microkernel system lol

[warm pine]

there's the family terminology that appears in iokit later

i'll have to read this more thoroughly after work, this is looking fantastically well-detailed so far

STREAMS present there too?

the "expert" terminology too and the driver/family matching stuff sounds iokitty too

[warm mural]

@twilit smelt I'm annoying but i have a last question https://github.com/xrarch/mintia2/blob/main/OS/Executive/Ke/KeDispatch.jkl#L527, how do you avoid the load of cpus changing between the iterations? I have to lock all queues before looking at the loads to ensure they dont change or I get a race condition

what's happening is that find_least_and_most_loaded_cpu is called and during that, on another cpu, a thread is picked and the load is decreased (and it is then increased again on switch) idk if that makes sense

[twilit smelt]

i dont. they can change

its inaccurate on purpose so i dont have to take locks

previously i did take a lock but the contention on that was extreme

[warm mural]

my problem is that they change so you end up not balancing anything

so like if u have 8 threads on 4 cpus, you dont end up always with 2 threads/cpus

which sucks

[twilit smelt]

no its fine

if theres a temporary imbalance itll get corrected

just a second later

it should be rare for the load to actually change during the time between scanning loads and moving threads

especially if theres a big imbalance itll be rare for it to change enough for the imbalance to switch directions or something

[warm mural]

but the problem is there's a fraction of time where a new thread is selected (load is decreased) and the previous thread is added back to the list (load incremented again)

[twilit smelt]

whys that a problem

[warm mural]

ah no nvm you edited

because the load does change

[twilit smelt]

i also dont know what you mean by this

[warm mural]

no nvm im stupid

it does seem to work pretty well without locking

[twilit smelt]

as for how bad the contention was when there was a lock around scanning loads

with that spinlock there, during the fireworks test every cpu was spending like 20% of its time spinning on it lol

when i removed it, no scheduler spinlocks showed up at all as being contended

[warm mural]

there's probably a way to do it properly lockless

[twilit smelt]

so it was a no brainer

[twilit smelt]

doubt it

[warm mural]

how do you measure that

[twilit smelt]

profiling

sampling program counter at intervals

[warm mural]

ah

[twilit smelt]

this is a manifestation of a common issue with fine grained locking schedulers where it becomes basically impossible to make the guarantee that like

the top N highest priority threads are always running on the N available cpus

this guarantee is automatic if you have a single set of ready queues shared between all cpus with a big lock (like 90s NT)

but when you break up the locks you lose that guarantee and can only make a best effort

and it will sometimes happen that theres a ready thread whose priority is higher than the current thread on some cpu

and it doesnt preempt that thread

when it "should"

[icy bridge]

wait NT did that?

[twilit smelt]

if you think thats bad wait til you hear what unix did!

[warm mural]

a big lock is fine for like 4 cores, which is more than what NT ran on

[warm mural]

freebsd's 4bsd scheduler ran fine apparently until like 16 cpus

[twilit smelt]

heres a comment on a wikipedia talk page that mentions the introduction of per-cpu ready queues in NT

dont ask how i found that and pulled it up so quickly

they didnt do it until 2003.

[twilit smelt]

i was referring to the "big kernel lock"

[warm mural]

oh

right

lol

[marble socket]

just put a lock on userspace

ez

:

[twilit smelt]

i like the guy making up fanfiction about dave cutler on a wikipedia talk page

[marble socket]

[twilit smelt]

for no discernable reason

[icy bridge]

multiple cpus can be in the kernel at the same time but only one in userspace

sounds like a great idea

[marble socket]

yes

[twilit smelt]

something extremely annoying and funny is that all of these taped presentations and stuff from the 90s that ive been looking at from the CHM mention dave cutler at some point as part of a joke that the entire room gets

dude was osdev chuck norris

[mortal thunder]

wait but how come there wasnt one queue per CPU

how did affinity work before

as in, the affinity mask

[twilit smelt]

it would scan the ready queues until it found a thread whose affinity mask allowed it to run on the current cpu

[mortal thunder]

wow that sucks

[twilit smelt]

this was typically the very first thread on the queue because only rare threads had different affinity masks

no it doesnt

because of what i just said

[warm mural]

what even is the use of pinning threads?

[mortal thunder]

just because 99% of the time its O(1) doesnt mean it isnt worse than a 100% O(1) solution

[mortal thunder]

better cache locality probably

but one program's caches tend to be mostly invalidated when you schedule other programs on the same processor anyway so

[twilit smelt]

for example i was watching a presentation from 1992 about the VAX->Alpha binary translator they wrote for VMS binary compatibility and the guy mentioned that it could identify untranslatable sequences that used stuff like nonstandard calling conventions and said the error message for this was "ERROR: DAVE CUTLER WROTE THIS CODE"

and the room laughed

dave cutler jumpscare in every presentation

[marble socket]

dave cutler is in my walls

[twilit smelt]

there was another presentation where someone mentioned his experience porting NT to the MIPS-based DECstation (as a DEC engineer, in partnership with MS) and he made a joke that he asked Cutler what kinds of debugging tools he developed while doing the MIPS port and Cutler said "debugging tools? it just worked the first time"

and the room laughed

legit osdev chuck norris

[warm pine]

It's called we do a little trolling

Zoomers don't know how to do it without being racist, bigoted, or threatening violence

[twilit smelt]

there was a period in the 90s where any joke involving dave cutler doing something badass told to a room full of industry OS people was an automatic knee-slapper

even if it was completely made up

the problem with the modern osdev scene is that it has no longer has a chuck norris

honestly i cant think of anybody who is worthy

terry davis is the pop osdev chuck norris but thats different because youd just seem weird and terminally online if you made a terry davis joke in a room of industry os people

if they even knew who you were talking about, that is

@warm pine can you think of any current osdev person where if you made a joke about them doing something badass itd be automatically funny

because nobody ever hears anything about them except badass things they did so it becomes automatic humor to make up badass things they did

like chuck norris

[digital pivot]

i wonder if he saves such level of influence to this day internally

or osdev industry lost all its heroes

[twilit smelt]

the internet killed heroes i think

[digital pivot]

sad

[marble socket]

we do a lil trolling

https://tenor.com/view/troll-get-trolled-we-do-a-little-trolling-gif-20266975

[twilit smelt]

fun fact that was originally a trump quote https://www.youtube.com/watch?v=PobQzVsj7GE

[digital pivot]

i find it cool and funny that we met a guy on the internet who can talk to cutler himself during the week

[marble socket]

OMG??? trump knows about internet humor!!?!?!? hes truly the best president ever everyone vote for him /s

brand x knows about x meme??? i will buy all their products

[twilit smelt]

my favorite trumpism is still "OBAMNA! amnesty, obama amnest."

https://youtu.be/YRlO2FdyT5s?t=13

why did he articulate it like that

if someone typed like trump spoke people would think they were putting on a cringe typing quirk to try to seem cute

[twilit smelt]

that was surprising to me for like 0.25 milliseconds before i remembered that every person in the west has access to the internet and itd be more surprising if i spent as much time as i did on osdev and didnt come within a degree or two of separation of people like that

[digital pivot]

well depends where on the internet

[twilit smelt]

its only surprising if you think of this community as a kiddie playground completely separate from the adult's table of industry/academia

when in reality its more like orbiting on the outskirts

and is a potential recruiting ground

[digital pivot]

well yeah but not that much of people from the group of industry devs you can meet here work with veterans like cutler

i think bindage met at google someone who invented something significant for today's networking

i forgot who and what

[twilit smelt]

likely place to meet such a person

[digital pivot]

i have too bad memory

[twilit smelt]

just like this is unsurprisingly a place you could probably meet a few people who are going to go into the industry or academia and make a name for themselves

[warm mural]

kelowerirql?

[twilit smelt]

this is only logical and has held true since like the late 80s when people with special interests were hanging out on like usenet

posting and replying all day

and then later became like Theo de Raadt and Linus Torvalds and so on

well they already were those people. they didnt shapeshift. but you know what i mean.

[digital pivot]

yeah

[twilit smelt]

ai people would say its different now though because "imagine thinking there will be an industry to have a name in in 10 years with the advent of AGI. if you arent 99th percentile at ML engineering rn youre ngmi."

i think they should jump into woodchippers

[twilit smelt]

as a hobby ive been surfing ancient usenet threads and theo de raadt somehow managed to be terminally online in 1989

he shows up EVERYHWERE

including arguments with avie tevanian over the design of nextstep and stuff

like "i was sponsored by the university to get a nextcube for my dorm room. this may be the stupidest computer ive ever used in my entire life."

and avie shows up in the thread

crazy stuff

[warm mural]

twitter but in the 80s and composed solely of gigantic nerds

[digital pivot]

now when people are hidden behind nicknames you don't see that a lot so transparently

[twilit smelt]

wow you werent kidding

STREAMS in copland, by name, using all the right terminology

generational level crossover

[warm pine]

i'd completely forgot about this thing you linked, i'll have to actually have a proper look at it asap

[twilit smelt]

they also have the interesting model of a whole streams stack residing in one driver server task

i mean thats not interesting thats just the trivially only way to make it performant

but its interesting in that they did that instead of some stupider thing

also fundamentally async up to the filesystem driver level

like NT and unlike xnu or linux

which i managed to guess some time ago just by looking at symbol names in the HFS driver for copland

also they use a worker thread pool per driver server

with a tunable number of threads (default is 1)

observe: the only semi competent attempt to do a desktop microkernel ever!

[twilit smelt]

its funny reading my suuuuper older messages in this discord from like 2020 on my old account

i would talk about basically the same stuff i do now (random old computer related pdfs i found and whatever) but for some reason it would summon an angry mob who would be extremely rude and bully me off the server repeatedly like this

mostly people who are now banned like racemus

so jokes on them!

[mortal thunder]

honestly i didnt have a great impression of this discord either but i was a 13-14 year old at the time so that may have played into it

[twilit smelt]

the regulars in 2020 seem to have had extremely fragile egos and would lose it if they saw someone know something they didnt

and/or were primed to dislike me and get instantly infuriated when they saw me post anything, by gossip that was going around about me at the time

in either case lucky for me theyre gone

[twilit smelt]

(started by people ive long since made up with and who are still active here, its ancient history so i wont go into details other than that)