MINTIA (not vibecoded)

[warm mural]

autism final boss (not in an offensive way)

[icy bridge]

it helps that i386 doesn't differ that much from xr17032 in the large platform-dependent areas like page table handling so i got to copy most of the code without needing to change more than a few constants

[twilit smelt]

ppc port would be nightmarish in comparison lol

[icy bridge]

like most of MiPTELinear.df is just xr17032's version with 5 replaced by 12 (the offset of the PFN bitfield in PTEs)

interrupt handling was the biggest difference between the arches but that's alleviated by not being particularly complicated on either of them

[mortal thunder]

nvm just ignore what i said

[twilit smelt]

i fixed this in the original repo too btw

[icy bridge]

nice

I wonder if it would've been caught by -Wall -Wextra

[night needle]

risc-v port wen

[sterile frost]

be the change you want to see

mintia2 rv32 port

[night needle]

don't threaten me with a good time

though i only have rv64 hardware

😢

[icy bridge]

with riscv you have to manually create the linear mapping instead of being able to use a recursive entry

which is annoying and might make a port impossible if mintia relies on the atomicity of page table insertion

[mortal thunder]

pretty sure mintia does rely on being able to create a recursive entry

[icy bridge]

it definitely relies on a linear page table mapping but that's not the same as relying on a recursive entry

[mortal thunder]

how can you achieve a linear pt mapping without a recursive entry

[icy bridge]

theoretically you could maintain the linear mapping manually without a recursive entry, but that's only possible if mintia doesn't rely on page table insertion being atomic

[icy bridge]

whenever you add a PDE you also add a PTE within the table at the entry that would normally be recursive

and whenever you remove a PDE you also remove the associated PTE

[mortal thunder]

i see

my OS doesnt really rely on that (because i use a lock on the address space)

maybe mintia doesn't either since it basically stops the entire world (by raising IPL) while modifying memory mappings (to my understanding)

but dont quote me on that

[dense vigil]

damn yeah crazy

[delicate dome]

Atomic access on page tables is cool because then I can do stuff like have mmap not stop the entire process (merely other mmap calls)

[mortal thunder]

page faults should be serialized against mmap calls as well imo

otherwise you'll be seeing half-mapped data and you'll encounter undefined/unpredictable behavior galore

[delicate dome]

Solution: Walk the page table on fault to check, and if it should have worked, flush TLB

[mortal thunder]

and besides the user process has no way to predict where mmap stuff will end up unless they use MAP_FIXED or spray the heap with a bunch of garbage to restrict the possible address spaces

[delicate dome]

What does this matter?

[twilit smelt]

dg/ux appeared to have locking on a per-pte granularity

"how! you cant fit a blocking lock in a pte!" you can if youre clever like them

[mortal thunder]

inserting a spinlock into a pte is one thing but a blocking lock!?

[delicate dome]

Yeah but POSIX mmap semantics are def easier to make if you have a single write lock

(In my model reads can occur concurrently with writes due to aforementioned atomic updates)

[mortal thunder]

i don't do eager PTE modification (yet). almost all PTE modification is performed within the page fault handler

and the only eager PTE modification I'll add is for clustered readahead faulting

[twilit smelt]

they cheated it slightly but yes

[mortal thunder]

does mintia 1 require atomic modification of PTEs?

[twilit smelt]

it doesnt rely on this because the kernel space page tables are established at boot time and arent changed

[mortal thunder]

but what about userspace page tables

[twilit smelt]

true but i still dont thin kit relies on that

[mortal thunder]

me neither

[acoustic sparrow]

i mean webkit-style locks fit in two bits, right? and they are blocking too

so you could do something like that/something with turnstiles to make a really tiny lock

[twilit smelt]

This was 1989 so pre turnstiles

And they only used a single bit in the PTE

[night needle]

almost every time i look in this thread someone is talking about turnstiles 😭

[twilit smelt]

i appreciate how DG/UX just calls them "demons" rather than "daemons"

[mortal thunder]

why do either of them call them demons

i never understood this aspect

its like they want to attract dumb american everything-believers into conspiracy theories

[twilit smelt]

its in an archaic sense of the word demon which just meant some unseen force in the background or something

in the sense of maxwell's demon rather than a christian demon

[mortal thunder]

i see

[twilit smelt]

@warm pine this sounds like the thing

(lwp = thread executing in kernel mode)

[warm pine]

just like osf/1

[twilit smelt]

@icy bridge i know youve moved past this but i found a weird bug lol

[icy bridge]

is this on i386 or xr17032

[twilit smelt]

i386

this is very consistent, across reboots

always says PQ

[icy bridge]

probably not passing args correctly or something

[twilit smelt]

its very strange because other commands are taking arguments correctly

[icy bridge]

https://github.com/monkuous/mintia386/commit/efc60c50f4c02bbf6c14ee4aa491d1076c1c08fd

found it: if strncpy ran out of space, it'd write the null terminator to src instead of dest. for relative paths with only one component, OSDirectoryName calls strncpy with max=0, so the output buffer wouldn't get null terminated

here's the updated image

[twilit smelt]

thank you much

i noticed this just after sending it to andrew_w for his os museum

ill send the new one

[dense vigil]

how did u find the bug so fast

[icy bridge]

first I added some printfs in rm's main to dump argc and argv. when they looked correct I added some more printfs all over the command to see where the weird string came from, which revealed that the weird string was the output buffer of osdirectoryname. I repeated the same process for that function, and found strncpy was misbehaving. that's a fairly short function in handwritten ASM, so I just looked at the code for a minute and caught the error

[dense vigil]

damn

[icy bridge]

would've probably only taken like 2 minutes if I could use gdb

[dense vigil]

btw are you planning to go back to linux and fix the random segfaults?

[icy bridge]

I want to do it at some point but am currently not very interested in doing so

[dense vigil]

what is your current project?

[icy bridge]

factorio

[dense vigil]

understandable

at least its not that huge of a project compared to other stuff

[heady bobcat]

I suspect that this is still a non-standards-compliant strncpy. IIRC:

1. strncpy(dst, src, N) writes up to N bytes, incl. null bytes.
2. strncpy copies up to N characters and fills the rest of the buffer with (possibly zero) null terminators.

For example, strncpy("abcde\0", 4) would give "abcd" with no null terminator and strncpy("abcde\0", 8) would give "abcde\0\0\0".

[twilit smelt]

It's not meant to be C standards compliant it's a dragonfruit strncpy

Which is confusingly named the same while behaving differently (more sanely imo)

Jackal's is not called strncpy thankfully

[heady bobcat]

oh I totally missed that, lmao.

yeah, the C standard library is very unsane.

I'm a simple man; I see strncpy, I assume C or C++.

[golden meteor]

Have you looked at beej's guide 🥺

[mortal thunder]

i completely agree and the way i usually get around this insaneness is by adding a null terminator at the end of the buffer directly

like strncpy(buffer, source, sizeof buffer); buffer[sizeof buffer - 1] = 0;

[blissful smelt]

strncpy feels slightly more sane if you have fixed length non-null terminated char arrays in structs or things tho

perhaps might've been more of what it was designed for

[mortal thunder]

Yeah, maybe

[delicate dome]

I literally just made my own functions that are actually sane and just used those.

Though the way I went about that left something to be desired sometimes, it worked a lot better than official strncpy for sure.

[twilit smelt]

DG/UX did the thing im planning on doing

where

page cache for file data and buffer cache for metadata

[twilit smelt]

thing to remember

devices can be localized to numa nodes as well

ALSO

@icy bridge do you want to put mintia386 on r/osdev or can i do it

and can i use one of your pics of it running on real hardware

[icy bridge]

you can do it and yes

[dense vigil]

Yeah pci express root ports have numa locality and it really matter for latency

[twilit smelt]

a menacing configuration i just read about is having per-node disk controllers which all control a common set of physical disks

wel its not that menacing i guess youd just represent them as all being logically 1 controller on the software end and then youd just direct each request to whatever the node-local physical controller is for the node the current thread is on

[dense vigil]

Never seen anything like that, but sounds bad

[twilit smelt]

This is how the CLARiiON disk subsystem worked

Or could be configured to work

[twilit smelt]

I will some time today

Do you have a reddit account that you want me to tag

[icy bridge]

i do not

[dense vigil]

Link the post once u do it

[pastel flume]

Sometimes I think I’m good at OSdev then I see what mintia can do then I realize I’m not

[blissful smelt]

no one is

[twilit smelt]

what can it do

isnt it just a black screen with some fireworks

[heady bobcat]

If you know what goes into making those fireworks, it's pretty damn impressive.

[heady bobcat]

Isn't this just impostor syndrome?

[warm mural]

I dont think thats what impostor syndrome really is

well it is but not the comparison to others part

[twilit smelt]

doubt it. the author of mintia is probably a total moron!

[warm mural]

it's more of a "comparison is the thief of joy" thing

[heady bobcat]

Yup. It doesn't make sense to compare one's own project with that of someone else, because you don't know how the time you put into it relates to that that they put into it.

[twilit smelt]

whats a cool title i could use for this reddit post for mintia386

[warm mural]

one day my kernel will do something other than show fireworks too 😔

[warm mural]

"My OS written in a custom language for my custom architecture running on real hardware" or something

maybe ported to real hardware would make more sense

[twilit smelt]

im thinking "I wrote an OS in a custom language for a custom RISC architecture, and my friend ported it to x86 on real hardware!"

[teal trench]

"real mode os running on real hardware" to get the upvotes on r/osdev

doesn't matter it's not real mode

[twilit smelt]

or "I wrote a multitasking OS in my custom language for my RISC architecture, and my friend ported it to x86 on real hardware!"

[queen torrent]

skibidi os running on 32 bit!!!

[warm mural]

I think this is good

not sure why multitasking is specified though

trust me tho im surprisingly good at making garbage projects go viral

[twilit smelt]

makes it sound more impressive

[warm mural]

maybe "with swap" sounds more impressive? idk

or

"with advanced features like ..."

idk

[queen torrent]

Oh god with swap is like foreign to r/osdev

[twilit smelt]

i think "multitasking" is best

[heady bobcat]

"paged virtual memory"

[twilit smelt]

yeah for r/osdev "multitasking" is best

[warm mural]

they wont understand "paged"

[twilit smelt]

ive tried this in the title of other posts about mintia and they consistently did poorly

like <70 upvotes

[heady bobcat]

"with downloadable free RAM"

[warm mural]

the sub has grown considerably since then tho

[delicate dome]

Gondamn they're REAL stupid in r/OSDev huh?

[twilit smelt]

aisix had the #2 top of all time post on r/osdev in like 2020

but i ended up deleting that post

it had like 250 upvotes

[warm mural]

for a while I had like top 3

er no maybe top 1 actually

idr

[twilit smelt]

literally yes

lol

[delicate dome]

I mean they're basically just normies atp

[queen torrent]

My minecraft in astral post got less than "C running in my 16 bit operating system" where the only thing written on the screen is "in skibido we trust"

[warm mural]

I hate the term "normies"

[teal trench]

officer down is still in the top 10 🥀

[warm mural]

it's so much redditspeak

[delicate dome]

If they don't even know what paging is

[queen torrent]

Its cr3 duh

[twilit smelt]

i think we should be legally allowed to attack 5 r/osdev users with hammers per year

as like a purge day

[warm mural]

these guys are normies, theyre not englightened like us redditors!!

[twilit smelt]

its funny you say that because its absolutely a 4chan term

[warm mural]

idk wherever it came from

its like a very chronically online term

[queen torrent]

If its on the internet assume it came from 4chan

[twilit smelt]

everybody who uses it knows this but its basically normalized

even normies say normie now

[delicate dome]

I don't use it basically at all but like how else do I describe it? A collection of noobs?

[warm pine]

skibidi rizzos running on 38667

[twilit smelt]

in common internet usage like lily just used it basically just means "neurotypical" and/or "mediocre"

[warm mural]

im not blaming you in particular I just think it's a stupid term that's condescending

[twilit smelt]

its just a funnier way to say that

[delicate dome]

806786

[warm mural]

neurotypical isnt really accurate

[twilit smelt]

it kinda is

[delicate dome]

Ehh

I disagree

[warm mural]

idk i think we have a lot of competent neurotypical people here

[twilit smelt]

we dont

if theyre competent here they have some tism

im convinced

[warm mural]

do you have tism

[delicate dome]

Neurotypical doesn't do anything to describe the level of knowledge they have

[twilit smelt]

competent osdevers who are neurotypical would not be in this discord theyre a postgrad at CMU and dont know what discord is

[delicate dome]

Now saying noob does describe the level of knowledge

[warm mural]

does neurotypical encompass adhd

[delicate dome]

No

[warm mural]

if not then most people are probably neuroatypical

[twilit smelt]

i feel like this should be obvious

lmfao

[delicate dome]

ADHD is very much considered neurodivergent

[warm mural]

I dont think people like korona have tism tho

idk i literally have no idea but from watching managarm talks and stuff it doesnt seem obvious

idk i wouldnt consider myself that competent but I dont have tism and ive been doing ok

[delicate dome]

Truth is you can hardly be 100% sure about someone's neurodivergence without just asking them

[warm mural]

yeah that is true

[queen torrent]

Korona was probably pulled into discord by nd ppl

[warm mural]

Im not claiming I know anyone's mental state either

[delicate dome]

Do neurotypical people really not use Discord that much?

[warm mural]

id consider myself fairly neurotypical and I use it

although maybe im not that neurotypical idk

[delicate dome]

I didn't think that would have much if any effect on whether someone uses it

[warm mural]

do people with a high degree of autism have trouble socializing online as well

i wonder

[delicate dome]

100%

At least in my own experience

Having a common interest that I'm sufficiently autistic about is nearly a requirement

[twilit smelt]

https://www.reddit.com/r/osdev/new/

posted it

[delicate dome]

Karma farming let's go

[twilit smelt]

averaging more than 1 upvote per minute so far

cant wait for that to immediately plateau to 1 upvote per 6 hours now that i said that

okay this guy will mog me

posted directly after me and has way more buzzwords

hes getting to 500 and im getting to 60

calling it now

[queen torrent]

Alpha karma farmer vs beta osdev

[warm mural]

for now yours has more upvotes

[golden meteor]

Worse: it's like ifunny/4chanspeak

[golden meteor]

They do

[pastel flume]

I mean sure, but the internals are insane. You’re one of the most read up individuals in this server fs, that’s what I mean

[pastel flume]

I’ve got it too 😔

[twilit smelt]

ok mine took off pretty well

the other guy got like 6

now i feel bad for wishing him poorly

wish i could share upvotes

[warm mural]

Stop being so selfless... this is a cruel world, he wouldn't have done it for you

[pastel flume]

Man I gotta make a post

[dense vigil]

Some person said you should make a yt video

They're not wrong

[sterile frost]

idk why i expected a higher level of comprehension from reddit users, but somehow i am baffled someone asked "what's your programming language called"

[mortal thunder]

hey im curious what you think of 32-bit arm's page table format, where the root page table is a 16kb "page" with 4096 PTEs which each map 1MB, and each sub-page-table is exactly 1KB with 256 entries

and there are two bits for "presence" status instead of one

[twilit smelt]

i have no opinion

[sterile frost]

sounds like brain damage

[mortal thunder]

i see

[twilit smelt]

im too tired from following this venezuela shit way past my bedtime to even parse what you just said

[mortal thunder]

i agree

when i was doing my port to 32-bit arm i had to modify my PMM to allocate 16kb regions, and also i just used 4 entries of the root page table at a time to get 4kb sized 2nd level page tables

@icy bridge hey do you know why this happens when i try to compile mintia386?

rm i386/LdrI386MapAsm.i386.fre.o.elf.o
gcc -m32 -march=i486 -mgeneral-regs-only -nostdinc -c -o PC/i386/LdrPCHeader.i386.fre.o.elf.o PC/i386/LdrPCHeader.S
PC/i386/LdrPCHeader.S: Assembler messages:
PC/i386/LdrPCHeader.S:91: Error: no such architecture: `default'
PC/i386/LdrPCHeader.S:98: Error: `movzx' is not supported on `i8086'
PC/i386/LdrPCHeader.S:99: Error: bad register name `%eax'
PC/i386/LdrPCHeader.S:100: Error: bad register name `%eax'
PC/i386/LdrPCHeader.S:104: Error: 32bit mode not supported on `i8086'.
PC/i386/LdrPCHeader.S:106: Warning: use .code16 to ensure correct addressing mode
.....

changing the line from .arch default to .arch i486 seems to fix it

then it doesnt recognize -std=gnu23

but thats probably just an outdated gcc moment

[sterile frost]

what archaic gcc version are u rocking damn

[mortal thunder]

gcc 11.4.0

[sterile frost]

lol why so old

[mortal thunder]

blame the package manager's maintainers lmao

[sterile frost]

ubuntu?

[mortal thunder]

yeah

[sterile frost]

i may just blame it on a skill issue

[mortal thunder]

funnily enough the cross compiler i built for my OS is newer

[sterile frost]

i'd hope so

[mortal thunder]

its literally gcc 13.1.0

but anyway i should be able to just install gcc13 and compile it now

[sterile frost]

fwiw you can probably drop the C standard to gnu11 or something

if you really want to keep using that old toolchain

github search through the repo shows it doesn't use embed, so unless monkuous used some funny new C feature it should just work

ah it uses auto

but you can just -Dauto=__auto_type

[mortal thunder]

anyway i wonder why the loader specifically checks for a 486

does mintia depend on alignment enforcement?

i would think it depends on write protecting kernel pages

but not necessarily alignment checking

gcc-13: error: unrecognized command-line option ‘-std=gnu23’; did you mean ‘-std=gnu2x’?
what the fuck

DriverAHCI.c: In function ‘InitializePort’:
DriverAHCI.c:453:22: warning: implicit declaration of function ‘__builtin_ctzg’; did you mean ‘__builtin_ctz’? [-Wimplicit-function-declaration]
  453 |     port->blockLog = __builtin_ctzg(blockSize);
      |                      ^~~~~~~~~~~~~~
      |                      __builtin_ctz
java -jar /mnt/c/Work/x/mintia386/dfrttrans/build/libs/dfrttrans-0.0-elfconvert.jar DriverAHCI.i386.fre.o.elf.o DriverAHCI.i386.fre.o
rm DriverAHCI.i386.fre.o.elf.o
/mnt/c/Work/x/mintia386/mintia/../sdk/link.sh link -nostubs /mnt/c/Work/x/mintia386/mintia/Root/mintia/BootDrivers/AHCI.sys.i386.fre    ./DriverAHCI.i386.fre.o   -d HAL.dll:/mnt/c/Work/x/mintia386/mintia/Root/mintia/HALPC.dll.i386.fre Dragonfruit.dll:/mnt/c/Work/x/mintia386/mintia/Root/mintia/Dragonfruit.dll.i386.fre OSKernel.exe:/mnt/c/Work/x/mintia386/mintia/Root/mintia/OSKernel.exe.i386.fre
xoftool: error: unresolved symbols:
  ./DriverAHCI.i386.fre.o: __builtin_ctzg

[sterile frost]

hm, might be a gcc 14 thing

[mortal thunder]

i guess i could compile gcc from source?!

[sterile frost]

i'd just make it work with C11

so far the only C17 feature i see being used is auto

and that g builtin thing

but you can replace that with a typed alternative

__builtin_ctzl or whaatever

[mortal thunder]

__builtin_ctzg seems to just be a type generic version of __builtin_ctz

[sterile frost]

yes

so replace with __builtin_ctz{,l,ll}

[mortal thunder]

ok

i gotta figure out how to keep the original ELF versions

(before converting to XLOFF)

nvm i believe it converts each object file to an xloff individually

[dense vigil]

Wait thats really stupid

Good thing 32 bit arm is dead

[mortal thunder]

my only guess is that it was designed with armv5 in mind which supported 1kb divisions of permissions

or something dumb like that

[mortal thunder]

"developing for dead platforms" is my middle name

sometimes, sadly

[dense vigil]

At least they realized it was stupid af in aarch64

[mortal thunder]

true

[dense vigil]

And iirc aarch64 has the most flexible format, like with bit granularity va width

[mortal thunder]

in my wip armv6 port i actually fill in 4 1st level PTEs at once just to keep the "4KB second level page tables" assumption going

[dense vigil]

I wonder how Linux handles 1k allocations there

Like a cache of allocated pages or something

[mortal thunder]

i havent looked too deep into linux but it seems like it uses some kind of software page tables and then converts them to hardware page tables

which is kinda dumb

[dense vigil]

Yeah thats every arch, but thats just an abstraction

[mortal thunder]

conversely, i don't blame the NT folks for avoiding inverse page tables on powerpc

because those are just.. bad

[dense vigil]

There aren't two sets of page tables in reality

[icy bridge]

maybe .arch default was introduced after whatever binutils version you use

[mortal thunder]

maybe

changing it to .arch i486 fixed it

GNU assembler (GNU Binutils for Ubuntu) 2.38
Copyright (C) 2022 Free Software Foundation, Inc.

[icy bridge]

CR0.WP

and cmpxchg/xadd

[mortal thunder]

bit 18 of eflags is AC or alignment check

[icy bridge]

yes

you check for i486 compatibility by trying to flip that bit

[mortal thunder]

does mintia depend on alignment checks?

i understand cr0.wp but not alignment checks

[icy bridge]

it doesn't need alignment checks, that's just how you check for i486 compatibility

[mortal thunder]

i see

that makes sense

[icy bridge]

cpuid is i586 (and some later model i486s but it's not architectural yet by that point)

[blissful smelt]

don't you mean the bits are "overloaded" because I think for the first level it was

00 - not present
01 - page table
10 - section (1mb)
11 - supersection (16mb)

and the second level is

00 - not present
01 - large page (64kb)
10 - small page (4kb)
11 - tiny page (1kb) <- deprecated I think

so it's less of "two present bits" and more like overloaded type bits

[mortal thunder]

the bits are actually for type, yes. 00 means not present.

on armv6 the meaning changed, 11 means small page execute never

tiny page is an armv5 relic

[mortal thunder]

lmfao

i agree

also when i was doing the armv6 port the implicit backwards compatibility bit me in the ass

cause i thought i was using the armv6 layout but in reality the armv5 layout was being used instead

[mortal thunder]

also according to the trm it's an "extended small page table" of 1kb pages

[pastel flume]

Brain rot page tables

[warm pine]

i've been doing this on m68k

[mortal thunder]

what do the page tables look like on there?

[oak spade]

depends on the cpu 😛

I've forgotten the specifics but the 68040 (and its successor iirc) have PTs with 128 PTEs at one level, and 64-32 at another, page sizes are 4k or 8k (globally configured).

there's an external mmu available for the earlier cpus that has an incredibly configurable page table layout too.

ah I should do another port soon, that was fun.

[twilit smelt]

@icy bridge did you see the post got like 300 upvotes

It did pretty well

[icy bridge]

that's nice

[twilit smelt]

he dgaf...

[icy bridge]

yeah idk I'm not particularly fussed about attention to my projects

[twilit smelt]

yeah r/osdev isnt the highest quality feedback

very arbitrary and lots of cliches like "terry... welcome back..." type comments

its pretty clear none of these people know what theyre looking at

not very fulfilling

i think it is considered a good idea to engage in brand building though or something

which ive completely avoided doing for like 8 years

[golden meteor]

What brand /j

[blissful smelt]

these sound ai generated

[twilit smelt]

not really

theyre just silly people

[queen torrent]

You are absolutely right — these are ai generated comments.

[blissful smelt]

u da real ai generated comment

[twilit smelt]

Today I will try finishing the baseline NUMA support and merging it

[twilit smelt]

I did not touch it

It is now 4am

The sun is down

it is getting cold.

[twilit smelt]

Perhaps today I will accomplish something

[blissful smelt]

what is the c version of mintia called

MINTCIA

mintia but produced by the cia

[twilit smelt]

Is there a C version of mintia

[blissful smelt]

isn't that what monokous is doing

I forgor

[twilit smelt]

He was but he gave up on it

He did a df to C compiler instead

[twilit smelt]

i feel like this

[queen torrent]

@twilit smelt

[twilit smelt]

Ab solutely terrifing

[twilit smelt]

Perhaps if I meditate I can rediscover the motivation I used to have to work on stuff

Ohmmmmmmmm

[night needle]

buy a dual intel x99 machine

50$ numa

[tame phoenix]

https://tenor.com/view/numa-ye-memecoin-numa-numa-onchain-summer-numaye-gif-15134192273665458710

oh wait not that numa? 😄

[twilit smelt]

https://fxtwitter.com/i/status/2013309382767833164

Cats with Aura 😺 (@catwithaura)

big yawnin

**💬 10 🔁 1.4K ❤️ 17.1K 👁️ 186.7K **

▶ Play video

FxTwitter · Jan 19, 2026, 5:55 PM

[mortal thunder]

@icy bridge Is the disk image generated (mintia-PC-fre.img) a fully allocated disk image? How much of it is actually used? I'm curious about something...

[icy bridge]

i believe it's just a fixed size that can be increased in the makefile

don't know how much of the default is used

[mortal thunder]

i see

i wonder if you can fit an image of mintia386 on a floppy disk

i have recently gotten my hands on an old pentium laptop, sadly it's missing a hard drive though

so i was thinking i could experiment with floppies instead

[icy bridge]

@mortal thunder i got it small enough to fit on a 1.44M floppy

that being said, there is no fdc driver, so it's a bit useless

this is where it gets to in qemu

if you're interested in implementing the fdc driver yourself, this is the diff to get it to output a 1.44M floppy image: https://hst.sh/otamibetuv.rb

compile with make -j16 SMALLDIST=1

[mortal thunder]

ah thats cool, thanks

i might start implementing drivers to get it on there

[icy bridge]

you should only need fdc

and isa dma since fdc depends on that

the challenge will be allocating isa dma buffers since afaik mintia has no mechanism for allocating memory with a physical address range restriction

[mortal thunder]

werent there other quirks that prevent mintia from running on a pentium (i586)

[icy bridge]

i don't think so

it should run on i486+

[mortal thunder]

although iirc you are using i486 toolchain

not i686

[icy bridge]

yeah it's compiled with -m32 -march=i486

and it doesn't use any i586+ features

[mortal thunder]

neat

[icy bridge]

it should just work

although it is vulnerable to f00f

[mortal thunder]

as soon as i manage to find a well priced fdc to usb unit and some floppies, and i finish my exams, i will start messing around with that

[icy bridge]

with fdc to usb unit you mean plugging a usb floppy drive into an internal floppy connector?

because otherwise you'll need to use a usb driver instead of an fdc driver

[mortal thunder]

no... i mean buying a floppy drive usb device so that i can write floppies

[icy bridge]

ohhh

[mortal thunder]

since this laptop doesnt have a hard drive i have no way of writing floppies

[icy bridge]

sorry brainfart

[mortal thunder]

although

technically i can solder wires behind the weird proprietary connector it uses

since its literally almost just IDE

slightly rearranged

so eventually i will have a hard drive/CF card/SD card connected to it

[icy bridge]

does the machine have PCI or is it ISA only?

[mortal thunder]

It has PC card slots if it counts

But no true PCI as far as I'm aware

[icy bridge]

hmm i'm not sure whether that's an indicator of whether its internal devices use pci

[mortal thunder]

The dock connector at the back has support for adding cards... but only ISA cards

for what it's worth the BIOS date is 1998

I guess it doesn't then

Here is the BIOS screen

[icy bridge]

yeah im not sure, something of that era should have internal PCI at the very least

it's important because IDE without PCI can't do DMA

[mortal thunder]

It probably doesn't do DMA

from what I can see

[devout geyser]

wow only 16 megs in 1998?

oh wait is this a laptop?

yeah i'm blind

[mortal thunder]

this is a laptop, yeah

[tame phoenix]

it cant?

what about 16 bit dma below 1mb

the oldschool stuff

[icy bridge]

nope ide doesn't use that

[tame phoenix]

ahhh nvm then

[icy bridge]

ISA DMA is so slow that even back then it was better to just use PIO

[tame phoenix]

even with a large sized request?

[icy bridge]

interrupt driven PIO of course but still

apparently yes otherwise it'd have had isa dma support

[tame phoenix]

pio is pain

[icy bridge]

not really

you get an interrupt and do a bunch of inw's

[tame phoenix]

most pcs i ever had, had 33 or 66mhz udma

but those were pci

[twilit smelt]

mintia: please work on me there are so many interesting problems to solve like NUMA things and vmm and IO and whatnot :)

the unknown ailment where i sleep 16 hours a day and spend all functioning moments either in class or doing homework or making money for tuition:

https://tenor.com/view/glass-breaking-monster-analog-gif-4955644972346747774

[fair jetty]

waiting for the day mintia gets a new progress update

[twilit smelt]

I look back on like November 2021 where i randomly wrote like 30,000 lines of code in a month the same way medieval Italians with no running water would look back at the Roman Empire

Abrupt and brutal fall off

[twilit smelt]

GUYS!!!! YOU DROPPED ONE!!!

[warm pine]

they have bigger problems than that

[twilit smelt]

one reason im having trouble with finding the motivation I used to have to work on this is that my outlook on the industry and this entire field is increasingly bleak

If uni were free I'd probably be trying to pivot away into a completely unrelated major right now

Sadly im almost done with my cs degree

I'm just not excited about it anymore i think the moral/ethical/aesthetic positions I feel are right will also lose materially and im not interested in getting my shit kicked in

[raven drift]

bleak in what way?

[twilit smelt]

im increasingly seeing that the only corners of it that i have any interest at all in being around any of the people involved in it

and dont think theyre bloodsucking soulless cretins

are places im unlikely to find myself because they recruit smarter people than me from like MIT and shit

i also think the aesthetics/morals of the field are going down a dark like fascist technocratic path and that its inevitable youll need to pretend to like the people peddling that shit at some point if you want to make good money doing anything in the field

i just cant pretend to like people that i dont like

never been able to do that

[raven drift]

I don't think this is true

[twilit smelt]

i also think the social/cultural integrity of those corners is doomed to collapse

anyways

so

i think they only still exist inertially

[raven drift]

which corners exactly?

[twilit smelt]

mostly systems stuff

and some parts of academia

[raven drift]

but "systems stuff" is very broad

[twilit smelt]

i dont know enough to define it more narrowly i just know that hardware people and like firmware and OS people are some of the least likely people to be insane

like the management and the outlook of these groups

i think part of it is a lot of them are populated/managed by oldheads who still have 80s/90s/00s era philosophies

but idk for sure

if you look at like Oxide Computer i think thats an example of a company that doesnt thoroughly repel me at every level

which is good

but i dont think it can last

we're entering the age of the grifter and theyre going to infiltrate and destroy everything before it gets any better

that is, if it ever does

civilization might just collapse instead

lol

[raven drift]

i think you're painting an overly pessimistic picture of the industry

[twilit smelt]

the industry captains are all moronic child rapists who are actively working to make life worse for the rest of the species for their own self enrichment

like

[raven drift]

there are plenty of companies that work on OS adjacent software

they are diverse

the industry does not only consist of sam altman and mark zuckerberg

[twilit smelt]

either child rapists or good friends with child rapists. the second part is almost universally true

[twilit smelt]

i think there are maybe 1 or 2 major tech CEOs who are not lizard people atp

tim cook is a good candidate for not being a lizard person

his eventual replacement is more likely than not to turn out to be a lizard person though

[pastel flume]

this is what i open up the thread to 💀

[twilit smelt]

i do believe there are structural forces that are making the lizard people overwhelmingly outcompete the sincere throughout the industry

historians will probably look back on the current cultural moment in the tech industry the same way we look at like precursor groups to the nazi party lol

its just evil shit

runaway individualism & contrarianism from incredibly powerful and wealthy idiots combined with a right wing moment

not sure why im meant to spend decades of my life working for entities that i truly believe are pure evil and will be responsible for the worst villainy of the 21st century just because i happened to get fixated on computers as an autistic preteen instead of cars or something

i often wish i was born like 10 years earlier i have a feeling things would be much more straightforward and id have been in the industry for 8-10 yrs by now and would have a good shield of cope to make it easier to pretend im not trying to get noticed and recruited by like, nazi troglodytes

[twilit smelt]

ofc things might abruptly swing back left and then id be in better shape but im never going to forget what ive seen with my eyes and heard with my ears from these people over the last few years

[digital pivot]

what left the biggest impression?

[twilit smelt]

the revelation that a few of the tech billionaires are active movers in plots to bring fascist authoritarianism (in some flavors, corporatocracy) to the united states in order to i guess stop having to pay taxes? or summon a machine god? or facilitate space colonization? or some other variation, and do not give a fuck about how many people suffer or die in the process or as a result

that these same people were deeply involved in an elite childraping ring for decades

and that the rest who arent directly implicated refuse to condemn them or talk about it and continue to go to fancy parties with them and shit

not to mention the fucking insane ties to US and Israeli intelligence

which would have sounded like an insane conspiracy theory if they arent spelled out so clearly in just the unredacted epstein emails

it all feels like a bad dream i cant wake up from lol

[raven drift]

again, the tech industry doesn't only consist of the mag7

[twilit smelt]

im aware its not a monolith but it operates 100% in their orbit

[digital pivot]

well according to this logic you live in this country and revolve around them no matter what

[twilit smelt]

i can only make the choices that are available to me

unfortunately there would be no complete escape from these fucking freaks other than suicide lol

or maybe china in 10 years

suicide or china take ur pick

[hollow musk]

and there is always the academy

[digital pivot]

depends what you mean by that, in my country it sucks

in cs specifically

and i think unless its a very good uni its not the best way to share your knowledge

[twilit smelt]

my opinions will probably relax when im not acutely freaking out about all this

[balmy zealot]

Am I right in thinking that you're not too keen to drop the priority boost from KeTurnstile if, say, the wait didn't finish as a normal lock hand-off?

I'm trying to get a feel for how common it is to leave a priority boost in place longer than strictly necessary.

[twilit smelt]

I don't understand the question

[balmy zealot]

I probably only understand parts of the picture, not the whole thing. Right now, I can't identify all such scenarios, but here are some that made me think:

• thread termination
• wait timeout
• your system has the concept of APC, which I am not very familiar with, but from what I understand, using this mechanism can interrupt the wait

I think this can be summarised as the waiter stops being a waiter, but not because the lock was released.

[twilit smelt]

If the wait is interrupted by any of these then you haven't received a priority boost from waiters on that lock

Only the owner is boosted

You would never have been the owner

[balmy zealot]

In these cases, the dependency the waiter -> resource > owner no longer applies, so the owner will be left boosted for a waiter who no longer exists.

[twilit smelt]

I see what you mean

Yeah for synchronization/efficiency reasons only the boosted thread removes its own boost

At lock release time

This isn't an issue

[balmy zealot]

Not quite the same thing, but from a similar category of things that complicate inheritance is this change in thread priority mid wait. I tried to take all the details into account on paper and failed.

On the one hand, my previous concern did not affect correctness, but this one might.

[twilit smelt]

I don't deal with that but one solution is to just break the thread out of its wait and let it go loop through the whole lock logic again and propagate its new priority

[balmy zealot]

And even if what I fear does not affect correctness, it is unpleasant to think that a thread with a significantly higher priority (perhaps gifted by some very important audio stream) may remain and thus disrupt the performance of others.

[twilit smelt]

Very unlikely that that audio thread will get interrupted

[hollow musk]

windows (autoboost) represents the boosts that must be removed later as a compact mask that travels with the dependency edge and/or the head's summary (a "head" representing the contended resource identity, reachable via hashed lookup)

unboost is separable (windows priority inheritance can apply and propagate 3 types of priority boosts): the token's lower 30 bits are CPU scheduling boost contributions

a boost contribution maps to one or more floor bits, floors are ref counted, so multiple dependencies can grant the same floor, and removing one dependency only decrements the floor

when the last contributor goes away, the bit is cleared and the thread's effective priority is recomputed from remaining floors

[hollow musk]

0 -- a priority change occurs and autoboost has established (a) a thread entry representing that thread's relationship to a resource (b) a head representing the resource identity (c) a cached key/token stored in the entry/head that summarizes the waiter pressure and any priority dimensions
1 -- mark affected autoboost state dirty and enqueue processing.
2 -- enumerate the thread's active autoboost entries and process entry-by-entry.
3 -- recompute the thread's priority key(s) used for ordering and for token generation.
4 -- if the computed key differs from the cached key, repair ordering in waiter/owner sets (reinsert in an RB tree according to the new key).
5 -- recompute head summaries; once waiter ordering changes, the head's cached max pressure and any derived target mask(s) may change. determine which owners are affected and compute the new composite token(s) that should apply to those owners.
6 -- apply token deltas. conceptually: (a) Remove = OwnerEntry->AppliedToken & ~RecomputedToken and Add = RecomputedToken & ~OwnerEntry->AppliedToken (b) UnboostThread( Owner, Resource, Remove ) for the removed bits and the symmetric routines for the added bits.
7 -- if an owner thread that just gained/changed boost is itself blocked behind another autoboost participating dependency, autoboost may need to propagate the new pressure: mark upstream state dirty and queue a worker for chain propagation, or push affected heads/threads into a processing list

[twilit smelt]

Strange location

[night needle]

liminal space

[blissful smelt]

i love how we have (at least from what i know) the first known instance of an explanation of autoboost's implementation in NT randomly dropped in the mintia channel

certified osdev discord activity

[twilit smelt]

Found in the network of underground tunnels under the UofUtah

Which I've been exploring between classes

[digital pivot]

btw.... " summary (a "head" representing the contended resource identity, reachable via hashed lookup) "

isnt this how it works in turnstiles

[hollow musk]

I may not be a good advisor here, as it turns out that my knowledge of turnstiles is rather poor

for example, I didn't know that they use an intrusive max heap and I was mistaken about how they work

[digital pivot]

with turnstiles it goes like lock address -> hash bucket -> turnstile object for that lock

what about autoboost?

[hollow musk]

then how do you know about it?

[blissful smelt]

windows internals

and an instructor

[hollow musk]

resource identity -> bucket -> autoboost head entry containing the ordered sets and cached metadata

I don't remember if turnstiles have a global hash table; maybe there's a clue there

in autoboost, it's somehow sharded and the key to its partitioning is not entirely clear to me, but apparently each shard has its own set of trees/lock

perhaps you have something similar with turnstiles?

[digital pivot]

i think its global for kernel and thats it

but idk for sure, @twilit smelt is an expert on turnstiles

[blissful smelt]

turnstiles do use a global hash table, but you could also split it up per-numa-node or whatnot if you so wished

turnstile_block():
    addr = mutex_addr
    turnstile = lookup_turnstile(addr) // looks in global hash table
    my_turnstile = this_thread.turnstile

    if turnstile:
       add my_turnstile to turnstile's freelist
    else
       donate my_turnstile as this lock's turnstile
       turnstile = my_turnstile

    add to turnstile's wait queue
    block on turnstile

a very simple slighly lobotomized pseudocode for turnstiles

[twilit smelt]

i might take an official break for a while and focus on my other hobby which is cropping up in my attention lately

from when i was a tween

conlanging

ive gotten interested in the etruscan language which is only known in pieces and fragments and it hink it would be an interesting learning exercise to try make a like historical fiction conlang that basically takes etruscan and fleshes it out with some reasonable guesses (or for most vocabulary just made up but phonotactically coherent roots) into a usable language

something like that

[hollow musk]

I didn't want to leave this point unfinished and thanks to the possibility of creating pseudocode, it became easier to look at it and understand from this fragment that the partitioning is by processes for user mode entries (the shard is chosen from a per-process bucket table, shards lead to RB trees of autoboost heads)

I apologize for intruding on your channel with my information, which is not entirely on topic.

[twilit smelt]

What you're saying is more on topic so that's fine

[digital pivot]

so how would a lookup for user mode look

[shadow ridge]

tbh it's more about discipline than motivation at a certain point

waiting for motivation to come back / focusing on side projects only works until a certain point and you will slowly abandon your project if you keep waiting for motivation to strike again

[hollow musk]

identity key -> select table partition -> select shard -> lock bucket -> lookup head entry in bucket RB tree -> if not found allocate a new head from a slist (pop from cache) -> ... -> on teardown remove from RB tree and free head (return to cache)

[digital pivot]

i see

do u study like only autoboost or more ?

[hollow musk]

everything related to the windows scheduler ("ke") and in general everything related to kernels

as a programmer, I like how the scheduler has been modernized, very nice architecture; as an enthusiast, I like the architecture as a whole

my compliments to both the current and original teams

and that's definitely off topic 🥲

[twilit smelt]

I don't mind at all

[shadow ridge]

@twilit smelt why are casts no-ops in the IR generation stage in jackal

do you expect the truncation etc. to happen in like the code gen

[twilit smelt]

they have implicit effects like generating the signed versions of certain IR instructions

truncation in local variables isn't a thing in jackal you can't rely on it to do that for you which is stated in NOTESFORSPEC

It may or may not do that

the current compiler doesn't

sandwichman's probably will since it's not terrible

[shadow ridge]

this isnt terrible either

[twilit smelt]

It is

[shadow ridge]

im looking at it cuz i want to go back into compiler dev

and finish my compiler

I never went past a frontend + transpiler

and i dont really have a full mental map for the process of AST->IR->Optimization->Assembly

and havent seen any books that actually provide a good mental model

[hybrid condor]

Please explain to me something I keep forgetting about Windows: thread states

I remember a table showing several ready states at once

[hollow musk]

in simple terms:
deferred ready -- the thread is runnable, but a CPU needs to be selected.
ready -- the thread is runnable and a specific CPU has been selected.
standby -- the thread is runnable and it has been selected to be the next one on a specific CPU.

[flat dust]

reasonable system

[digital pivot]

there's an interesting state called transition which means threads stack is swapped out which is i believe a unique feature of windows today

but what idk for sure does it mean thread in transition can't be put into ready state

or when the outswapped thread is readied it's put into transition while it's being swapped in

[twilit smelt]

older versions of mach used to do the same

does XNU still? lets find out!

https://github.com/apple-oss-distributions/xnu/blob/main/osfmk/kern/stack.c

nope!

the code to do so is all gone

they decided at some point to get rid of it

or the people working on mach 3.0 before them did

or whatever version of mach they used i dont remember

the copyright on that file only goes back to 2003

very interesting

lets look to see if the earliest versions of OSX from before that still retained the kernel stack swapping code

it does!

OSX had kernel stack swapping and removed it c. 2003 looks like

[digital pivot]

that's cool, wonder why they removed it

[hollow musk]

the thread is being treated as logically becoming runnable, but it cannot be placed on a ready queue yet

this is for the "transition" state (6), and the "wait for process in swap" state (9) is a process-level gating wait

[digital pivot]

didn't know about WaitingForProcessInSwap

it's new

https://www.geoffchappell.com/studies/windows/km/ntoskrnl/inc/ntos/ke/kthread_state.htm says it's nt 10+

[warm pine]

Common feature

Solaris, FreeBSD do it

[flat dust]

i dont think they want to give you any warranty

is there any reason xnu removed it

[twilit smelt]

lots of memory now and they might have judged its a reliability liability

hate that those rhyme

also it can make debugging really annoying

having written a kernel that does kernel stack swapping the most annoying thing in the world is when you break into the kernel debugger to go stack trace some thread and it says

"Kernel stack swapped out!"

like well fuck

now what

[flat dust]

:(

[flat dust]

fair

i was originally thinking it wasnt really practical on powerpc

gcc stores function arguments in r3 to like r15 i think

i feel like it uses the stack on ppc but not as often

idk how thats related, maybe the amount of stack usage couldve made it useless

[twilit smelt]

they were using kernel stack swapping for at least 5 years of xnu on mac ppc (assuming xnu starts somewhere around 1998)

[flat dust]

true

rhapsody existed

[twilit smelt]

idk if rhapsody had xnu

might have been the next kernel still

those are different apparently

[flat dust]

rhapsody was based on nextstep though, at worst its based on a mach version

[flat dust]

true

[flat dust]

or whatever the ns kernel is called

[flat dust]

oh ok nvm its based on mach

apple forked it into darwin

that was open sourced

basically xnu is based on this

i think

[warm pine]

what i have found is that XNU appears to be a sort of greenfield reconstruction of the nextstep mach 2.x kernel architecture, based on fresh and unencumbered sources in the form of 4.4bsd-lite2 and osf/mach

[flat dust]

insanity

[warm pine]

the nextstep kernel was derived from 4.3BSD and that was encumbered until 2002

it didn't stop apple publicly releasing its source in early versions of darwin however

[flat dust]

oh 😭

[shadow ridge]

isnt the kernel stack a few pages at most

why would you need to swap it if you have more than like a few megabytes of memory

also wouldnt it be possible to retrieve the part on disk where it was swapped out to

[warm pine]

a nextcube could have 8 mib of ram, suppose you have an 8 kib kernel stack (i don't know, it's possible it was that size), 40 threads eligible for stack swapping, you can free up 320k or so of ram, a reasonable proportion

and when stack swapping was invented the ratios would've been even more favourable

[shadow ridge]

okay thats a lot of ram

[shadow ridge]

but even then, is it not possible for the debugger to just dump this info

[delicate dome]

Yes but the debugger cannot read the disk

[shadow ridge]

not read the disk, just dump the disk device + lba of the pages

[delicate dome]

well sure but you can't use that since interpreting the stack manually is pain

[shadow ridge]

no pain no gain

[delicate dome]

that's just not how this works

[warm pine]

this introduces considerable complexity to the debugger

[shadow ridge]

i havent done osdev in years

i dont really remember the limits of debuggers

[delicate dome]

Debuggers really don't expect you to do anything but debug userspace and it shows. And it just gets worse if you use a custom language.

[shadow ridge]

ig u cant rely on any of the vmm data structures to be valid either huh

[shadow ridge]

i thought debugger meant like kernel's builtin debugging utilities in this context

not gdb or whatever

[delicate dome]

oh

[warm pine]

that's exactly the sort of thing i would expect a lot of debuggers to have at least a little resilience against

[delicate dome]

Most kernels are very limited in what self-debugging they do at all

[shadow ridge]

i mean, if the kernel panics, is there anything that you can expect to be in a valid state?

[delicate dome]

Technically no, practically a lot of stuff is usually intact.

My kernel does an attempt at showing a backtrace, the registers and virt2phys if applicable

[warm pine]

hopefully a debugger that's relatively independent of the dynamic state and doesn't trust it too much

[lucid umbra]

it does truncation and extension by default but if you provide the --xrsdk switch it doesn’t

that switch tells it to mimic the original compiler in terms of syntax and code behavior

[shadow ridge]

if your IR only assigns each variable once and most instructions simply create new variables to store the result

why not treat the instructions themselves as variables

[raven drift]

that's what many SSA IRs do

the only annoyance is dealing with multiple results

[shadow ridge]

hmm

do you know of any books / resources i can read about this stuff?

specifically about IR, code gen and optimizations in general

[warm mural]

There's the dragon book but it's famously very academic

[shadow ridge]

i read that one a few years ago but i recall it being more of an overview / explanations of the concepts rather than exploring it more in depth

ill skim it again

[twilit smelt]

the audience for the dragon book is not "random person who wants to write a working compiler as a personal project"

its like

a reference for a person who has been doing compilers professionally for 10 years and has a master's or phd in CS with a focus on compilers and adjacent topics like graph theory

type of thing

(im being a bit hyperbolic but essentially true)

[shadow ridge]

yeah im looking for something thats more textbook-ish ig

[blissful smelt]

how many human languages do you know

[twilit smelt]

zero

[blissful smelt]

wise words

[lucid umbra]

^ in my IR, inputs to instructions are just pointers to other instructions

multiple results is easy enough by using some sort of instruction that just selects output N from another instruction

often called a projection node/inst

[shadow ridge]

well

this makes things much simpler

by giving instructions a reference count

that could also help with like

idk it could help with something somewhere

[mortal thunder]

It could help with dead code removal and register allocation

[lucid umbra]

oh yeah use counts/lists are very useful

in my IR each instruction keeps a list of instructions which use itself as an input and the corresponding input index

i bitpack the index into the top bits of the pointer

[digital pivot]

what was the motivation for lazy apc thing?

it seems to be used only for suspend

[twilit smelt]

its a very permissive environment for APCs to run in

because lazy APCs only get executed before returning to usermode and also while the thread is blocked on behalf of usermode in the kernel

which are times where no locks are held (aside from possibly superficial locks like the one on a file object for atomic append)

so such an APC can do a lot of stuff including initiate IO and so on

safely

this is my alternative to the error-prone thing NT does where they increment an "APC disable count" when entering unsafe code like filesystem drivers where an APC shouldnt run

which i dislike because it requires spamming these increments and decrements all over driver entrypoints and whatever

god help you if you miss one

[digital pivot]

as i understand it windows would bsod in such case

i see, it removes the burden and works on the opposite principle with safe places when its good to do so

[digital pivot]

turns out theres its own apc disable count for both special and regular kernel mode apcs

why did they separate them originally?

[twilit smelt]

i have no idea

i think that is something recentish (post-'00s)

[digital pivot]

normally youd raise to apc irql and that way disable them

but there must be some reason for kernel apc disable count to exist

and be used

[hollow musk]

if I understand your question correctly, it's probably to avoid touching IRQL

the KeRaiseIrql is basically "mov cr8"

the scheduler uses APCs to terminate or suspend the thread, so when holding a lock you need to either frequently touch IRQL or use KernelApcDisable

otherwise this could result in a system deadlock, think of a scenario when a suspended thread is holding something other threads depend on and remember that thread suspension is performed by a kernel APC queued to the target thread

though not all locks require a critical region with KernelApcDisable, as some implicitly prevent suspension

[digital pivot]

i wonder is it that bad?

[hollow musk]

I think it could be expensive under virtualization because of vmexits

[hollow musk]

now I think that maybe they could've done the s/w IRQLs in s/w

[digital pivot]

i think thats what mintia2 is doing

or will be doing

[warm pine]

mintia1 also did it that way i think

[digital pivot]

btw do they queue an apc to suspend? i heard they now can convert into suspended state directly

ah wait itd work only for waiting threads

[hollow musk]

why not?

you could suspend the waiting thread immediately, but otherwise APC works well

I know that Windows has this optimization

[digital pivot]

yeah so that you dont need to wake a thread to suspend it

apparently its important

[hollow musk]

one pretty important reason I see is transition threads

NT can swap out kernel stacks of threads that wait 15 seconds or so and longer

if you try to suspend such a thread, when you try to wake it up the stack swapper will inswap the stack back to send the thread back to wait

[digital pivot]

i didnt think of that

[digital pivot]

and why th is there so much code there

[hollow musk]

it's basically

if CurrentIrql != NewIrql:
    writecr8(NewIrql)

if KiIrqlFlags != 0:
    if (KiIrqlFlags & 1) != 0 and CurrentIrql <= 15:
        if NewIrql >= 2 and NewIrql <= 15:
            if CurrentIrql == NewIrql:
                Mask := (1 << NewIrql)
            else:
                Mask :=
                    (((1 << (NewIrql + 1)) - 1)
                     AND (ALL_ONES << (CurrentIrql + 1))
                     AND 0xFFFFFFFC)

            CurrentPrcb.SchedulerAssist+5 := CurrentPrcb.SchedulerAssist+5 OR Mask

return CurrentIrql

so yeah, there is a SchedulerAssist update, but IRQL changes still happen via writecr8

[acoustic sparrow]

well you can always have a debug version of your kernel which verifies you dont have cycles in your locking, no?

[twilit smelt]

That's not a locking cycle

[acoustic sparrow]

incrementing the apc counter is like taking a lock though

[twilit smelt]

all you can efficiently check is whether the apc counter is 0 on exit from the kernel

youre right that in a debug version you could do something INefficient to more effectively ensure that these occur in pairs

but i still find it ugly to spam these increments and decrements all over code that frankly shouldnt need any knowledge of the detail of whether it can safely run an APC in the middle of it or not

like random driver code

[hollow musk]

what are you doing instead?

[twilit smelt]

a special type of APC that is only dispatched during a wait on behalf of usermode and also upon return to usermode

this is different from a user APC with only a kernel function because it doesn't interrupt the wait, the wait is resumed by an outer loop

[hollow musk]

what exactly qualifies as wait on behalf of usermode?

[twilit smelt]

a wait where the mode parameter is supplied as USERMODE

[hollow musk]

it doesn't necessarily mean that important locks aren't held or that non-reentrant regions aren't active, does it

[twilit smelt]

it is overloaded with several meanings

for example in NT it means "its safe to swap out the kernel thread stack" (because there are no pointers to anything on it that might be accessed by another context)

[hollow musk]

suppose you're mid-IO/FS path and you hit a user wait (or a boundary that qualifies as return to user), if such an APC can initiate IO, can it reenter the same stack while it's already active?

[twilit smelt]

that just doesnt happen along such a path

for example mutex waits are always designated kernel mode

and blocking on synchronous IO happens after the request packet has been fully enqueued and the driver stack has fully returned back to the syscall implementation level

so that can be designated usermode

"wait mode" might be a somewhat bad abstraction which ive copied from NT where it is also maybe bad

due to how many things it conflates

the original purpose on NT was just to designate whether a wait should be quit if an event arrives that would have interrupted usermode

then early in development it was overloaded with "is it safe to swap out the kernel stack?"

and other meanings that tend to align with a general theme of like

the final long lasting wait at the end of some in-kernel processing caused by a syscall

which occurs at a shallow level in the kernel where few or no locks are held and no pointers on your stack could be used from other contexts

especially if its indefinite (like blocking on user input)

ok it might be a good abstraction but the names are bad

you might call it "wait level" and have it indicate which logical level of the kernel the wait is occurring in

like you could rename usermode wait to like, superficial wait or something

and kernelmode wait to like

balls deep wait or something idk

not that

[hollow musk]

it also couples your user APC safety to a caller chosen flag?

what enforces that the waits where user APCs run are exactly those phase boundaries, not just a wait that happens to be tagged user-ish

[twilit smelt]

well these are not user APCs, my usermode APCs work basically like NT's do

these are "lazy kernel APCs"

or LAPCs

theyre lazy cuz they arent executed until the next wait with the mode designated as usermode or the next return from the kernel to usermode

as opposed to normal kernel APCs which are executed as soon as the thread is scheduled in and is below APC level

what enforces that you indicate a usermode wait at a time where it is appropriate is the same thing that enforces you dont access a null pointer and crash the kernel

[shadow ridge]

so balls deep wait is not called wait mode on NT and it's a user mode only thing?

[twilit smelt]

i may be being confusing

"wait mode" is a parameter on the internal kernel mode wait functions

on both mintia and NT

its not something you can specify from outside the kernel

[shadow ridge]

ooh i see

[twilit smelt]

the wait mode on NT is supplied as either usermode or kernelmode and indicates what its safe/expected for the kernel to do with the thread's wait at that point

same on mintia

this is how its described in NT's docs targeted at driver authors

which is extremely vague and doesnt at all capture what its doing lol

the workbook probably has a better description

[hollow musk]

so instead of drivers pairing APC disable, they must correctly classify which waits are superficial boundary waits?

[twilit smelt]

a driver on mintia will basically never do a usermode wait

[hollow musk]

then if someone mistags (intentionally?) a deeper wait as superficial you've reintroduced reentrancy hazards with a different annotation

[twilit smelt]

theyre all fundamentally asynchronous so they just execute in a straight line path without blocking hopefully (can still occur if contending on a mutex or etc, which is always a kernel mode wait) and then after kicking the hardware to start the IO and enqueuing the in-memory structure representing it, they return back up to the top

and the wait on behalf of usermode for the IO to complete is done in the generic code in the kernel proper

at the end of the read/write/whatever syscall impl

[twilit smelt]

yeah but like in the docs i can just say "if youre writing a driver you probably shouldnt ever tag a wait as usermode"

im unsure why NT's docs dont say that

i cant really stop a driver author from intentionally doing something malicious

there are easier and more useful ways to be evil as someone whose kernel driver has been loaded into the system than to exploit some misuse of a parameter to an internal kernel function

[mortal thunder]

Lowest-level and intermediate drivers should specify KernelMode.

pretty sure they do

[twilit smelt]

theres an implication there that theres some vague classification of a driver as "highest level" which may specify usermode

but they dont explain what this is

[digital pivot]

maybe one shouldnt directly export a function with such parameter to regular drivers ?

[twilit smelt]

perhaps not

but your options then are to either have a wrapper function that just calls the internal function with waitmode=kernelmode always

or to export the symbol, but only document a macro in a driver kit header file that does that

both kinda are a little ugly and convoluted

to avoid them potentially breaking a rule that is very simple and less complicated than 99% of the other rules for kernel programming

[digital pivot]

is it allowed for a lapc to block?

[twilit smelt]

yes

[hollow musk]

is there a difference between mintia1 and mintia2, and which one is this thread for?

[twilit smelt]

its for whichever one something is happening with but normally mintia2 now

the difference is mintia2 is rewritten in a new language

and has smp support

its basically a new kernel

that i didnt want to come up with a new name for since i like the name mintia

[hollow musk]

are they compatible with each other?

that is, a new kernel for the same OS?

[twilit smelt]

they are not compatible

also, they are both written for a toy risc computer architecture in a custom lang

mintia1 has an x86 port now

as well

[hollow musk]

what other supported platforms are there?

or are you referring to 32 bit x86

[twilit smelt]

mintia1 runs on 32-bit x86, fox32, and xr17032

the latter two are toy ISAs made by me and a friend

mintia2 runs on fox32 and xr17032

an amd64 port is planned

we have a little informal community of uber-NIHers who like writing compilers and designing architectures and stuff like that

we're collectively hoping to put together like a full semi-usable computer ecosystem made 100% just by us

between us we have like 3 architectures 3 OSes and 2 compiler toolchains at this point lol

which we like retarget to eachothers stuff for fun

i mostly use mintia2 as a kernel design vehicle, the fact it runs on a novelty toy isa is kind of way on the backseat atp hence the eventual amd64 port plans

cuz the fact it runs on a fake smp computer probably makes it sound puny

which its not

its pretty durn fancy for a hobby os

[hollow musk]

sounds more like fun

did you draw inspiration from something real for your own computer?

[digital pivot]

do u have an os yourself

[hollow musk]

maybe I ought to, but it will require a lot of patient work and time

[twilit smelt]

mostly risc workstation computers like sparcstation, decstation, aviion

[night needle]

speaking of risc

do you hate risc-v

[twilit smelt]

i think it contains one of the most unnecessarily silly goofy decisions ive ever seen in a modern architecture

[hollow musk]

how many redundant extensions to the base architecture does it cover? 😄

[twilit smelt]

im referring ofc to the fact that entries of non-leaf page tables have to have all permissions bits set to 0

which means leaf and non-leaf entries have a different format

and you cant do recursive paging

for like no good reason whatsoever

its one of those things that makes it evident its a student project made by people who had only looked at linux before

linux and mips

you can still create a virtually linear page table but you have to do it with explicit mappings

you cant do the top level self-entry trick

so it adds memory overhead to something that should be free

[balmy zealot]

I'd quite like to ask how useful this actually is, and forgive the slightly naive question, but is it something everyone tends to use?

[delicate dome]

It's a common debate of whether you should use them. Either way will work but of course it's extra effort to create page tables if you're relying on the trick and now need to port to an arch that doesn't have it.

[twilit smelt]

they allow some optimizations

[hollow musk]

probably hyenasky refers to mapping the page table hierarchy for the process in kernel virtual address space

this makes conversion between a virtual address and its page table entry very neat

[twilit smelt]

and also vice versa

pte addr -> vaddr

which is never needed but is useful implementationally to be able to do

[hollow musk]

there's a good book about the windows memory manager that explains many tricks that are useful not only in windows, the book isn't well known, so I must recommend that you buy it

What Makes It Page? is the name

[warm pine]

au contraire

my approach to working set aging & eviction by iterating the PTEs of page table pages with active PTEs in them needs a way to get from the PTE address to the vaddr it maps

[twilit smelt]

Good book

[warm pine]

so i have to stash the base of the range described by a page table into the page struct to do that

[hollow musk]

the guy basically broke down the entire virtual memory subsystem in windows into molecules and wrote a book in calibri font like a true boss

[warm pine]

if i had virtually linear page tables then i could use that to calculate instead

[twilit smelt]

it really is a disgustingly ugly book

but its very informative

[hollow musk]

it's written in a default font, clearly hasn't been properly proofread, and contains typos, but it is magnificent

the man wrote it and disappeared

[hollow musk]

what kind of virtual memory system do you have?

I found it interesting how you presented the aging of the working sets

[balmy zealot]

Thank you for the recommendation

[warm mural]

oh yeah ive read a bit of that book

[twilit smelt]

flexing some of my osdev journey books

books I couldn't find in pdf form or I just felt like owning a physical copy of

[warm mural]

how can you not find pdfs

[twilit smelt]

sometimes there just isn't a pdf

[hollow musk]

you have a wonderful collection, a good reason to show it off

[warm mural]

have you read them all?

[twilit smelt]

ive read those all in full except for pc interrupts and modern compiler design

[warm mural]

I havent read the linux one, how good is it

well tbf I've only read one and a half in that list

[hollow musk]

It's a good book about linux

[warm mural]

also I really like the VAX architecture handbook design

[digital pivot]

i see, you do nt re?

[warm mural]

also pro tip idk if you know about annas-archive

but like if you dont find it on there then i wonder if the book even exists

[twilit smelt]

i have looked there and failed to find anything before

[warm mural]

physical books are cooler

it's a shame I dont have infinite money or space

[twilit smelt]

i only knew about hitchhiker's guide to vms because someone mentioned it in a wikipedia talk page for rsx-11/m

couldnt find it online anywhere

found someone selling a used copy on ebay

[hollow musk]

yes and no, because normally I don't, I'm just interested in operating systems in general, and as part of some research on schedulers, it seemed to me that windows one wasn't described very well in its modern form, so I'm kinda digging into it

[warm mural]

like there's so many cool books that also have cool covers

[digital pivot]

i see

[digital pivot]

so has it changed a lot?

[twilit smelt]

what even is the cover of the linux one

a man polishing a gigantic glass orb

is the glass orb meant to represent the linux kernel

[warm mural]

o'reilly covers are all weird

but they look cool

you should buy the solaris internals book it's pretty good

[twilit smelt]

hitchhiker's guide to VMS is an underrated book about its internals

the author looks like this

[warm mural]

do you buy all of them used?

[twilit smelt]

a lot of these have been out of print for 20+ yrs so

[warm mural]

makes sense

[twilit smelt]

this copy of hitchhiker's guide to vms was printed in 1990

[hollow musk]

yes!

many fundamental things still apply, but yes

[twilit smelt]

My used copy of Windows NT File System Internals came with the sample code floppy

miraculous stuff

[warm mural]

crazy

[twilit smelt]

the seller didnt even mention that

[hollow musk]

succeeded in examining the contents?

[twilit smelt]

i should buy a usb floppy drive and try to digitize it

i havent tried yet

if i do ill put it on archive.org

[flat dust]

can you implement really old NTFS with that book

not just the floppy but like the entire book

[twilit smelt]

its not about NTFS

[flat dust]

oh

[twilit smelt]

its about NT filesystems more abstractly

what unixheads would call VFS maybe

[flat dust]

oh that filesystem

not the drive one :(

[twilit smelt]

its about implementing a filesystem driver for NT

[warm mural]

it's Windows NT + file system internals not Windows NT file system + internals

[twilit smelt]

not about NTFS

[flat dust]

ohh

https://klipy.com/gifs/tense-table-smash

[twilit smelt]

this book (not my picture) is about NTFS

[hollow musk]

essentially, a must-read if you are a developer of file system drivers or file system filters for windows

[warm mural]

wait books are actually pretty cheap on ebay

inside windows NT is like 20$

(cad)

[flat dust]

theyre basically called the same thing but are way different

[twilit smelt]

yes lol

[flat dust]

BUT WHY 😭

[twilit smelt]

a pleasant surprise with inside windows nt was the cool hologram on the cover