Keyronex

[hexed solstice]

A firmware binary is necessary for the boot process. It is a modified U-Boot under GPL but its source is lost so it cannot be included in QEMU.
lmao

[sullen fern]

wait I just realized amiga = friend

[hexed solstice]

gpl violation defence: "i lost the source code"

[sullen fern]

I knew the company existed but I never made the link

[valid sequoia]

also cheap macs

[sullen fern]

oh yeah some macs used ppc

[valid sequoia]

before they get way too expensive

ugh, they already have

[hexed solstice]

i had a powermac g4 before i got for £20 in 2009 or thereabouts with pocket money i saved up

i wish i asked my parents to buy me an sgi box at the same time, when they were also available for £20

prices for SGI octanes in 2009

the cost is the final number before the [Octane...

[sullen fern]

Port keyronex to M-chips

[hexed solstice]

and today...

[vale pelican]

keyronex the new router os

[hexed solstice]

incredible, i had no idea that m68k was king of 32-bit embedded in 1997

funny thing about that btw

a large chunk of the 68k share would be the coldfire

which is marketed as, quote, "a variable-length RISC architecture"

it is literally just the 68k without a few of its most hardcore addressing modes

there are a few other omissions but it still looks nothing like a RISC

[twin haven]

Awesome, let me know if you run into any trouble. I plan on adding a feature for passing the boot info (and ramfb) to the kernel at some point too, if that's of any use.

[hollow hull]

did you just call loong "mediocre riscv"?

i hope i misunderstood

[hexed solstice]

it will be, for the time being i will just hard-code it though

it's an interesting one because it doesn't really fit directly into limine protocol

presumably it should appear in the memory map as something special, and a new request added, to do it the 'right way'

[velvet horizon]

oh hi fadanroid

hows keyronex going

[hexed solstice]

first thing i am doing today is moving over to the virtually-contiguous pfndb

[hexed solstice]

it is more portable than when you were last here

[velvet horizon]

great to hear!

[neat steppe]

yo fadanoid who is this foo

[velvet horizon]

oh you got unbanned nice welcome back

[hexed solstice]

this is the nyauxmaster, the fourth

[neat steppe]

LOL

hello

[velvet horizon]

hi

my absence has been too long

[hexed solstice]

nyaux has actually done fairly well

[velvet horizon]

i lost motivation and stopped working on it for like 2 months

that is a problem

im afraid no human can escape the grasps of mental illness

[hexed solstice]

still, you managed to go from nothing to something fairly quickly when you last worked on it

[velvet horizon]

after getting help by like everyone in the server, some information was spoon feed to me while others i did manage to understand. i still dont think i did that great of an achievement

but its whatever now, good thing im back i suppose

[hexed solstice]

you shouldn't underestimate yourself

there are a lot of people who could be spoonfed and fail totally to make anything

[sinful agate]

agreed

nyaux is a pretty damn cool project

[velvet horizon]

really thank you all !!

means a lot to here this

[neat steppe]

i can concur that making something semi-functional is incredibly rare lol

[velvet horizon]

if nyaux still manages to compile after all this time i will be surprised lol

[hexed solstice]

quite, chatgpt or copy pasting alone can't get you that far

[hexed solstice]

something else i really ought to add: ASID management

[neat steppe]

its not that hard after you realize you can just deal with it per-cpu

[hexed solstice]

i haven't thought about how to do it yet so i still have to decide whether it's hard or not

virtually contiguous pfndb is now in on amd64

complete with smarter handling of memory that i want to track (kernel, modules, bootloader reserved) but cannot immediately (or ever for the first two, for now) free and reallocate

and the latest stats, kernel/ now standards at just over 32,000 lines, and kernel/vm specifically at exactly 8000

8 is an auspicious number to the chinese

[violet tinsel]

is it up on github yet? I‘m curious how you implemented it

[hexed solstice]

https://github.com/Keyronex/Keyronex/blob/virt-pfndb/kernel/vm/init.c no numa zoning yet

no non-amd64 yet

no freeing up of sub-parts of the largepages allocated to store the pfndb but which are not required yet, either

[violet tinsel]

do you ever replace limines page tables for the kernel?

[stone orbit]

can't you derive pfndb_page_describes from pfndb_pgsize?

[violet tinsel]

I‘m wondering if you reuse the pages used to create the mapping or if you just keep them forever

[stone orbit]

unless im misunderstanding what pfndb_pgsize actually means

[hexed solstice]

pfndb_pgsize is the size in bytes of a page that's used to map the pfndb

[hexed solstice]

i do now

[stone orbit]

oh and pfndb_page_describes is how big each page pfndb tracks is

[violet tinsel]

nice
i see you‘re using the same page_buddy function as I do : )

[hexed solstice]

yes, i quite liked your idea about keeping a max order

alternative option could have been continuing to check the phys segment to which a page belongs and check bounds

but i like this better

as then i would have to either iterate over all pregions, looking for which one contains the page, or keep an array of pregions and an index into these inside the page

[violet tinsel]

yeah my first idea was to keep a table of segments and an index into it per page

[hexed solstice]

i will probably bring the phys segments array back into existence but the use will be to validate paddr to vm_page_t translations

it's useful to have as a debug assertion that the paddr is within a valid phys seg

[violet tinsel]

my debug assertion is potentially page faulting

[violet tinsel]

my debug assertion is potentially page faulting

(or failing silently)

[earnest salmon]

we love it when assert page faults

[violet tinsel]

i didn’t mean it like that

when this part fails it pagefaults which is the alternative for asserting

[earnest salmon]

I hate page faults because I can't

stack trace through them with gdb

except on the m68k

[violet tinsel]

why not

[earnest salmon]

because no .cfi_* and stuff with nasm

so I can't use gdb to reliably stack trace

[violet tinsel]

oh yeah that’s annoying

but i‘m sure there‘s some workaround

[hexed solstice]

i just use gnu gas

[earnest salmon]

I'd rather do userspace dev for windows in assembly than use GAS when there are alternatives

[hexed solstice]

virtually contiguous pfndb is now in for aarch64 and riscv64 on that note

[violet tinsel]

It‘s not THAT bad

[earnest salmon]

there is only one human response to seeing AT&T syntax assembly: it is to curl up into a ball, start vomitting, crying and having a full mental breakdown
- DreamingWarlord

[hexed solstice]

the trouble for m68k is that i have to also adapt it to limine boot

however i think that's not such a bad thing

[sullen fern]

what do you use atm?

[hexed solstice]

i can declare from now on that on all arches a limine-like bootloader loads keyronex

[earnest salmon]

idk what that does, but good job

[hexed solstice]

a mess that i originally wrote for my amiga and later adapted to qemu virt m68k

no large pages on the 68040 unfortunately

[frank badge]

i agree that its pretty awful but i still use it

[violet tinsel]

basically linux struct page *pages;

[earnest salmon]

credit @ember geyser

[ember geyser]

ok bro that's it

im going fucking insane

why cant i go one day without hearing about AT&T syntax assembly

[wet apex]

movq %rax, %rbx boo

[earnest salmon]

rax=rbx

OH WAIT I FORGOT IT'S FUCKING ATT SYNTAX

rbx=rax

because for some reason that's logical

rax gets the value of rbx

no no

rbx gets the value of rax

like wtf

[violet tinsel]

well intuitively we read from left to right

[earnest salmon]

jokes on you, in arabic we read from right to left

[violet tinsel]

you win

i can’t argue with that

no but seriously, both are fine

it‘s not like it’s that big of a deal anyways

[earnest salmon]

in one sentence, explain what this does:

mov 8, %rax```

should be easy

move 8 into rax

Wrong.

Move *(uint64_t*)8 into rax

mov 8(%rsp), %rbx```

[violet tinsel]

are there assembler alternatives to gas for like aarch and riscv

surely there have to be

[hexed solstice]

ERROR: Argument is of an unacceptable type 'File'.
Must be either an external dependency (returned by find_library() or dependency()) or an internal dependency (returned by declare_dependency()).
i really really hate meson

just let me depend on a file outside the project for heaven's sake

[sullen fern]

I typically provide my own patches for non-meson projects

what are you trying to do specifically?

[hexed solstice]

make @twin haven's limine shim for m68k depend on the kernel, without having to build it as a subdirectory of the kernel

in other words it has to be built as a project in its own right

[sullen fern]

yeah you'll need to add a meson.build for that

[hexed solstice]

yes, that goes without saying, i have one

and i have no way that i know of to add a dependency in that meson.build on the kernel executable (which is built elsewhere, and if its timestamp is more recent than the loader, then the loader - or rather, one file of the loader, the one that incbins the kernel, should be rebuilt - but of course meson doesn't believe that individual source files should have dependencies, so...)

[zenith karma]

may i suggest not using meson

[hexed solstice]

of course

in the long term i am throwing meson in the bin

[zenith karma]

just make a build.sh script

its fine

[hexed solstice]

if i can be bothered it will be a makefile-based system

[zenith karma]

i dont think there's a point in doing that

ccache clang performs okay enough if you don't use C++

[hexed solstice]

and while i'm at it i will minimise xbstrap's role - i am not prepared to integrate gcc, binutils, and xorg into my own build system, as netbsd does, but everything else, i would like to

[vale pelican]

rax -> rbx

:^)

[sullen fern]

what if you declare_dependency in your kernel

then use dependency() in the shim

[hexed solstice]

i will look into it in the future

[earnest salmon]

just switch to cmake

[hexed solstice]

for now i just added rm -f path/to/Loader.cpp in my convenience makefile

[sullen fern]

nah cmake sucks

[earnest salmon]

I disagree

[sullen fern]

meson is better

[earnest salmon]

set_source_files_properties(
    "Loader.cpp"
    PROPERTIES
    OBJECT_DEPENDS "${OUTPUT_DIR}/oboskrnl"
)```

[sullen fern]

ok fair but the syntax sucks

[earnest salmon]

I guess

[hexed solstice]

it would all be much better if qemu-system-m68k -M virt let you specify a firmware

i think the chap who was enhancing virt with an xhci controller and such actually ported the das U-Boot along the way

now given that das U-Boot is reported to load limine

[vale pelican]

if the loader ends up being a flat binary then with some linker script magic you could make it possible to just do cat loader kernel > out-loader

[hexed solstice]

it would even be quite fun to port limine proper

[vale pelican]

and this would be trivial with meson, a custom target that invokes cat and is just given the loader output object and kernel executable object as arguments

[earnest salmon]

what stops you from doing -bios

[vale pelican]

and captures stdout

[hexed solstice]

mint might not care for anything 32-bit, but m68k limine will be delivered, at the end of a fork if necessary

the whole thing would be extraordinarily funny

[zenith karma]

the boot protocol isnt very 32 bit friendly though

[earnest salmon]

Replace All "uint64_t" -> "uint32_t"

[zenith karma]

lol

yes

perfect

[hexed solstice]

regardless of all that, adaptation to r4's lisp shim is getting along

[sullen fern]

it's in lisp?

[final hatch]

it should be movq *8, %rax to dereference 8

but why does it have to be like that...

[earnest salmon]

or at least make it consistent with dereferencing registers

movq (8), %rax```

[zenith karma]

why would you do that?

[earnest salmon]

to make it more clear

[zenith karma]

consistency is like

boring

[zenith karma]

yeah clarity is boring too

i propose we randomize the assembler syntax

to ensure maximum security

[hexed solstice]

now boots with r4's shim

[earnest salmon]

did you make mlibc support the m68k?

[hexed solstice]

yes, it's a bit rough and ready

i have a custom relocateSelf for m68k because i could not for the life of me get their existing one to work without it depending on relocations itself

[stone orbit]

no, it's just called LISP, Limine (Boot Protocol) In SIlly Places

[sullen fern]

ah ok lmao

[stone orbit]

yea lol

and for now it only includes m68k

[hexed solstice]

so that's that

i need to make sure i didn't leave any mess and then i can merge this

the virtually contiguous pfndb is nice in its own right but the nicer thing, i think, is that now i have a non ugly way to a) reclaim bootloader reclaimable memory b) back non-_USABLE pages with a vm_page_t

that would be useful if ever i decided that some part of the kernel should be paged

[neat steppe]

I wouldn't recommend it for kernel code

for your targets

it's just not enough memory to justify the extra risks

paged pool i still stand by though

[hexed solstice]

without a doubt, it would be useless

it might be a fun diversion but realistically there isn't that much to page

too many of the big things use spinlocks

[neat steppe]

It ended up about 50/50 when I did it

so it was like 150k paged 150k nonpaged

Which is of course utterly insignificant when you're targeting over like a megabyte or so

I anticipate more like 60/40 this time because of more extensive usage of blocking locks for everything

[hexed solstice]

at a very quick overlook of what code i could page: i can't page most device drivers (use spinlocks), could page parts of FS drivers (but that takes care), could not page most networking (because it runs in DPCs, and in any case it would be terribly stupid to page most of it, if it were latency-critical), could page much of the namecache code but that's not big anyway, could not page most executive services (there's not much there yet anyway, but e.g. the handle table is RCU'd)

implementing sleeping RCU would be nice and allow a bit more code to be paged, but my intuition is that it's dangerous to have to page in code involved in freeing up RCU'd data

[neat steppe]

Yeah it takes a lot of work to maximize what you can page

[hexed solstice]

it's a fun thing to do without a doubt

but i don't feel like i'm missing out much

[neat steppe]

I think it also ends up enforcing a certain system structure which might be beneficial

[hexed solstice]

i am happy that i am already going to greater lengths and can happily page the page tables. that deals with a cost that grows the more, the more virtual memory you are using. the kernel code is a constant cost by contrast, and one that is not a huge deal on my targets

i target the m68k but my real m68k box has 128mib of ram and my spare has 64mib, which is huge

[neat steppe]

Mach ended up winning me over on page table management

I now consider its scheme there to be superior to the VMS one if implemented right

It's also probably possible to meld them in a nice way to get the best properties of both

[hexed solstice]

i think for its time period and a while after, it was the completely right choice

how translation tables looked was so variable at the time (i am not thinking about software refilled tlbs, where mach will maintain conventional page tables as a cache for efficiency, but about inverted page tables and such)

[sullen fern]

what do you mean by that?

do you mean the pmap layer?

[neat steppe]

Yeah and the fact the page tables are disposable since they contain no information that can't be reconstructed from the map entries and the objects

[hexed solstice]

i was dead in favour of mach style originally, completely so, i am moderate on the issue now

[neat steppe]

No need to page them out you can just destroy them instead

[sullen fern]

oh that's basically what I do

[neat steppe]

And having a paged pool to store the objects' page tracking entries in makes the whole thing the indisputably best choice I think

[hexed solstice]

my intuition now is that you could have some fun tricks going on to allow quick servicing of page fautls without looking at the abstract data structures, but these being disposable

[neat steppe]

Yeah I'm planning on storing an encoded pointer to the map entry in the page table entries when a working set entry is trimmed

So if that's there you can skip a lookup

[prime coyote]

sounds cool, and memory domains seem similar to will's partitions

[neat steppe]

"Will's partitions" not mine lol

Did not invent that

[prime coyote]

i mean Mipartition structure in mintia2

[neat steppe]

Yeah

[hexed solstice]

i have had a bee in my bonnet lately about platforms with larger page sizes than page tables and dealing with those effectively will need a bit of reworking at least of how the abstract vm objects deal in their tables

[prime coyote]

how did you find out about them?

[neat steppe]

kelowerirql

[prime coyote]

i see

[zenith karma]

doesn't this also eliminates the requirement for page tables when you are using a software tlb?

[hexed solstice]

it would be completely unacceptable to allocate 64k tables for every file's page cache, for example

[zenith karma]

because you can just look up the map entry and object instead

[neat steppe]

It does but this sucks to have to do a full lookup on every tlb miss

[hexed solstice]

in principle yes but in practice, what a software refilled tlb is is a riscification of a hardware refilled tlb

[zenith karma]

a full lookup of all the data structures?

[hexed solstice]

the page tables in mach are a cache

[zenith karma]

that makes sense yeah

[hexed solstice]

if you sacrifice the cache, you pay the penalty

[neat steppe]

I need some form of page tables anyway to track the state of dynamic system space

[zenith karma]

but there's no reason to keep the cache in the traditional radix tree form i think?

[zenith karma]

but yeah fair enough

[neat steppe]

A lot of that exists with no backing in objects and map entries and so on

[hexed solstice]

loongarch at least incentivises that form

[neat steppe]

And would be inefficient to do so

[hexed solstice]

see the ldpte and lddir instructions

[neat steppe]

Recursive page table mapping is ludicrously convenient

[zenith karma]

fair enough

how do you actually set it up?

do you just do like new_page_tables[some entry] = new_page_tables | flags to make it rw uxn pxn

[neat steppe]

MIPS tlb also prefers a multi level page table mapped recursively somewhere

[zenith karma]

interesting

[hexed solstice]

this must be where they got it from

[zenith karma]

so i guess just because you dont HAVE to doesnt mean its still not a good idea

[hexed solstice]

as it is said that loongarch is mips with a lick of red paint and a chairman mao portrait

[neat steppe]

Prefers as in it sets up some control registers for easy access to the correct pte within such a mapping, rather than make you calculate it in your refill routine

But you're not forced to use this you can just do your own junk anyway

my arch does a similar thing but I take it further

[zenith karma]

that seems like it breaks KASLR

because you always have page tables in one place in memory

or in one of 256 places which is not a lot of randomness

[neat steppe]

the mips ppl gave up on the case that the bottom level page table isn't in the tlb and you take a nested miss; at that point they just dispatch to the general exception handler and you deal with it

but I let it cascade up the page table hierarchy til it gets to the wired page directory, which is cooler

[neat steppe]

Who caresss

[sullen fern]

software-refilled tlb sounds like you can lots of cool stuff

[zenith karma]

the security nerd within me does

[hexed solstice]

these partitions, i really want to figure out what's going on with the CrossPartitionReferences field in quite a few data structures

partitions surprised me from the getgo because at first they sounded like the common thing people do with NUMA systems where they replicate the allocator and paging queues and daemons

[prime coyote]

probably related to CrossPartitionCharges?

that's another symbol that contains "crosspartition"

[final hatch]

Then randomize its place lol

it's worse randomization than the aslr of the actual kernel (if it occupies the -2GB region)

but better than nothing

[twin haven]

Exciting! 🎉

[twin haven]

Yeah look not my greatest name lol, but it was late and it made me laugh.

[hexed solstice]

cheers for it, very useful

i want to do a quick review of locking around page faults on vnode memory objects

right now i speculatively permit a pagein which may be beyond the end of a file and let the FS driver veto it in its page io routine

i don't like that

but isn't it the most reasonable behaviour for networked filesystems like 9p?

excluding those, it would be perfectly reasonable to require that the vnode's vm object have a length field which is kept up to date by the FS driver, i.e. vnode creation initialises it, file extension/truncation alters it

it just doesn't feel quite right to me that someone could mmap with length 15 terrabytes a file which is really only one byte long, and then when they fault on high addresses, the tracking tables are allocated for those outrageously high addresses, and are immediately deallocated when the FS driver fails the io

i don't think i have a better solution currently so it stays

[sullen fern]

yeah it seems reasonable

you could also just do something like fs_get_size() on page faults

but that would require calling the FS driver twice (once for getting the size and another time for reading)

[hexed solstice]

it's tricky as well becuase that would have to operate while a spinlock is held, if i want to do it in the right place

[sullen fern]

true

[hexed solstice]

in the end with 9p at least there is an inherent tocttou involved

even if i request the size it can change before i request the bytes

[sullen fern]

I guess you could also keep a temporary buffer and map it if the read didnt fail

not sure about locks here though

[violet tinsel]

vmem_t vmem;
RB_HEAD(vm_map_entry_rbtree, vm_map_entry) vad_queue;

are vm_map_entry's for the procstate::map_entry_rb and vm_object::map_entry_list what describes an object/file/private, and vmem is just what you use to decide where to put the memory you're assigning in the VAS? that's how i'm understanding it, but I wanna make sure : )

[sullen fern]

vmem is the virtual memory allocator, yes

[violet tinsel]

but is it really only used for the underlying VA range allocation? not even address lookups?

ok seems like it is

thinking about it, seperating it from the acutal describing structures is reasonable

now i want vmem too

[sullen fern]

yeah

it can also be used for e.g allocating pids

[violet tinsel]

the algorithm itself?

[sullen fern]

yes

you may need to use a different allocation strategy than INSTANTFIT though

[earnest salmon]

imagine allocating pids when you can just have your pids as a uint64_t

then:
pid = next_pid++

[sullen fern]

pids are 32 bit sadly

[earnest salmon]

skill issue

[hexed solstice]

the vm_object::map_entry_list is for future truncation of files which have memory mappings beyond truncation point

and please welcome the vitrually contiguous pfndb to the master branch

[violet tinsel]

what do you do if one pagefaults in a privately cow’ed area which lies beyond the end of a file that‘s been resized?

does that make it bigger?

[hexed solstice]

if it's already cow'd, the page fault is handled without touching the vnode

if it should be cow'd, but wasn't yet, and it's past EOF, i think the right thing to do is to fail

and definitely wrong to extend the file

[violet tinsel]

that’s sensible yeah

thanks

[wet apex]

Posix says to raise a synchronous SIGBUS if you read past the last page

[violet tinsel]

also if it‘s contents are already faulted in?

then you’re just going to write to trash memory right

and upon unmapping it‘s going to be ignored

[wet apex]

If its a private mapping yeah, it wont be synced to the underlying resource

[violet tinsel]

with a shared one it also shouldn’t fail i think

[wet apex]

In astral if a file gets truncated the existing pages get thrown out of the page cache, so they act more like anonymous pages and dont get synced

[earnest salmon]

iirc that's similar to what I do as well

[hexed solstice]

i am considering to make a change

splitting the modified page queue into three: modified-compress, modified-pagefile, and modified-other

modified-compress would be the first place of queueing for dirty anonymous pages and pages from there would be subject to a compression attempt, if a good ratio is achieved, they'd be stored in compressed form. modified-pagefile would be where pages not compressed to a good ratio, as well as the pages backing the compressed page store, would be queued for putting to the pagefile. and modified-other would be for file/disk-backed pages to get ordinary writeback

[hexed solstice]

to act as a cache for pages that recently became inactive after having been dirtied, where they can sit for a little time and maybe be reactivated, and if they're not, to be written back to disk and then go on the standby page queue for either reactivation or replacement

the pageout process is split in two, there is writeback (to the backing store, which might be the pagefile) if needed and then there might eventually be replacement

the swapfile is on disk

[valid sequoia]

no, it's to better determine what to swap

[hexed solstice]

but cleaning and replacement are discrete processes

[valid sequoia]

none specifically

page compression is just like any pageout for the rest of memory manager

[hexed solstice]

only that i am thinking of splitting up the modified queue, one for each of the three possible destinations to which a modified page will be written back (either to swapfile, or to regular file/disk, or in the future maybe to an in memory page compressor)

which i am considering for two reasons, 1. it simplifies things for systems without a swapfile to not have to churn through dirty anonymous pages that can never be written to disk, and 2. it allows for some more parallelism

[valid sequoia]

no, compression is still expensive so you treat it like any swapping

it's to discern the pages that are accessed only somewhat infrequently from the pages that haven't been used in a long time

particularly there are some workloads like full scans where pure LRU approaches fail

[hexed solstice]

do you know the second chance algorithm? imagine if there were, instead of a single inactive queue where you have to writeback if you go to replace a page at the head of the inactive queue, instead two inactive queues - one for dirty pages, one for clean - and you gradually wrote back the pages on the dirty inactive queue and migrated them to the clean inactive queue

[valid sequoia]

and if you have actual second chance, it also acts as a third chance before pageout

[hexed solstice]

(then if you also modify it so that instead of a single active queue, you have an active queue per process, what you get is roughly what keyronex does)

[valid sequoia]

which is now trendy as linux does on more levels calling it multi generation LRU

[hexed solstice]

the preference to replace clean instead of dirty pages is worth noting typical of most of the practical page replacement algorithms

[woeful spear]

You can check how the windows memory manager does things. The kernel memory manager has a periodic event that can trigger page trimming, and during this the memory manager can move dirty pages from the process's set of pages resident in main memory to the standby or modified list. Without this, every time you access previously removed pages, you would have to hard fault them from disk

[hexed solstice]

yes, if it's dirty and inactive it's on the modified list, if it's clean and inactive, the standby list

that's the strict rule

there is a local and a global level of replacement

the local level are per-process working sets

those grow as there are page faults and are gently shrunk periodically

when a page is no longer in any working set, it is placed onto the modifeid or the standby page queue, depending on whether it's dirty or not

[woeful spear]

Did they reinvent aging?

[earnest salmon]

isn't aging described in the denning working set papre

*paper

[woeful spear]

I don't think so

[hexed solstice]

sort of

linux is very siloed and has their own distinctive language

in this case, what they've done is taken the classical two-queue second chance algorithm they've always used, but they've added more queues

they call page replacement in general "LRU" even though they were really doing two-queue second chance, which is equivalent to clock

[earnest salmon]

what's this then?

[hexed solstice]

some months ago i figured out how to make locking more granular, i forgot the details, now i have mostly figured it out again

there's one bit i need to figure out again though (or maybe i never figured it out initially?)

it has to do with page stealing

when dealing with private anonymous memory, the working set mutex of a process while held means no PTE will turn from zero to anything else, nor from swap to anything else, nor from valid to any sort of invalid (be that standby, swap, zero, etc). they can still transition from standby to swap, that's one of the things the big PFN lock guards

but that's not a problem, because you don't need to allocate a page when you encounter a standby PTE - just turn it into valid and update the reference count on the page

that's done under the PFN lock but in the future will be done under a per-process PTE state lock + if page refcount == 0, the page queues lock

for, say, a demand-zero fault, a new page has to be allocated. page allocation can look at the standby page queue and choose to steal a page from there; obviously it has to acquire the PTE state lock of the process the page belongs to in order to do that, so it can replace the standby pte with a swap pte

but since PTE zero-to-anything-else transitions don't happen without the working set mutex held - which all page faults acquire - that's no problem; just go and allocate the page without your own process PTE state lock held, it's fine, that PTE isn't going to change state because you found it was zero

what's trickier is when VM objects come into play

for them i just wanted to have a single spinlock for their PTE state. but suppose you fault on a shared mapping of a file, the page is not in the VM object, it's got to be brought in. and you found that out while holding the vm_object::pte_state_spinlock. so you want to allocate a page and map it in as a busy page while you post off the read-in request to the FS. but you can't allocate the page

because, case 1, there's not many free pages, but at the head of the standby page queue is a page belonging to the very same VM object. so the PMM tries to acquire that very lock so that it can delete the page from the VM object (or replace it with a swap PTE, if it's an anonymous VM object.) it never works, because the lock is already held, and you have to keep that lock held, because otherwise by the time you've allocated your page, someone else might've managed to fault in the page you wanted to!

and the general case when it's not a page of the same VM object is even worse. because all PTE state locks are sisters.

• if fault 1 is on VM object A, and the head of the standby list at that time is a page belonging to VM object B;
• and fault 2 is on VM object B, and the head of the standby list at that time is a page belonging to VM object A;
• then while fault 1 is holding vmobj A's lock, the PMM is trying to acquire vmobj B's lock, to evict that page;
• and while fault 2 is holding vmobj B's lock, the PMM is trying to acquire vmobj A's lock, to evict that page

[prime coyote]

is that the same as page trading ?

[hexed solstice]

what's meant by page trading?

[prime coyote]

https://github.com/j-wang2/memoryManagement/blob/master/coreFunctions/pageTrade.c do they call it page trade here?

it's something to do with page trading in windows, ithought it's the stealing?

[hexed solstice]

i need to take a closer look at this

something odd is going on

[prime coyote]

oh! they say "Note: Page trading is currently deprecated (not updated with rest of code base)"

[hexed solstice]

at first i thought by trading they meant what's commonly called stealing, which is repurposing a page on the standby list

[prime coyote]

do you know what are the quarantined pages?

saw that in windows but undocumented

[hexed solstice]

i can only really judge by the name

quarantine is where you put something diseased to protect others from it

but there's already a bad page queue on windows (i think?)

so i don't know

[prime coyote]

i thought maybe the ones that can't be traded

[hexed solstice]

if trading equalled stealing, then all pages on the standby list should be stealable

it looks like trading in this memory manager is just literally replacing one page with another, copying over the data

i don't know the purpose

it could be useful for relocating pages to make physically-contiguous pages available

longer spans of physically contiguous ones i mean

[prime coyote]

search for "trade" in windows:
MiTradeTransitionPage
MiTradePage
MiTradeBootImagePage
MiTradeActivePage
MiPrepareToTradeUsingAccessedBit
MiCheckContiguityTradeEligible
MiAddMdlPageToTradeBlock
MiActivePageTradeable

it probably is not page stealing

[hexed solstice]

i am confident that my guess was correct then based on this stack trace

page trading appears to be moving pages around in memory to create physically-contiguous space

[prime coyote]

this is interesting, i saw a trace where MiDefragmentSlabWorker -> MiClaimPhysicalRun -> MiTradePage

[hexed solstice]

here are two ideas i have on solutions to this: 1. give up the vmobj lock while allocating pages, then re-check after allocating pages whether they should still go into the object, i.e. that someone else didn't allocate and put pages into it; or 2. add a second vmobj lock, one that explicitly guards against creation of pages (or for anonymous vm objects, swap to in-memory transition)

[violet tinsel]

why does vm_kalloc() in keyronex return direct mapped allocations when npages=1 but map them into vmem_kern_nonpaged_va? Is this just to be able to satisfy allocations of where npages isn’t a power of 2 thus can’t be allocated directly from the buddy allocator without fragmentation, because if that‘s the case, you‘re not doing order > 1 buddy allocs anywhere, right?

[hexed solstice]

this one is explicitly for allocating kernel virtual memory with

i considered the possibility of returning direct mapped multipage allocations but i felt it would be better to save contiguous pages for other purposes

[violet tinsel]

ok, fair

i just found it inconvenient to have it sometimes return reliably direct mapped and sometimes randomly to KVM_WIRED mapped pages

but if it works it works ig

[hexed solstice]

it used to return only non-direct-mapped ones but as an optimisation i decided to skip mapping for the one-page case

it's also relied upon in the maintenance of the working set data structure that it be virtually mapped and not directly

as it expands by realloc'ing at the page level

[violet tinsel]

in terms of not having to copy to new pages? That‘s neat yeah

although i‘m not sure how that‘s related to the ws

[hexed solstice]

the structure that tracks what's in a working set is a bit like a ring buffer (not really, but sort of)

[violet tinsel]

struct wse *nodes; /*!< Entry array. */

This?

[hexed solstice]

that's the one

[violet tinsel]

ah, got it

i mean, you could still add a flag for allocation to, for example, prefer allocating contiguous

it‘s totally valid like you currently implemented it, just a thought i‘m having : )

[hexed solstice]

i think i am basically at the point now where i could legitimately start speccing out the posix server and implementing its prerequisites (IPC and such)

nothing that i am aware of is horribly broken, of a poor foundation, or incomplete enough to forbid it

the posix server will be responsible for at least unix ttys, signals (including posix interval timers), job control, and the process tree as opposed to the purely object oriented process model the kernel has (no tree, only handles)

[sinful jetty]

any advantage to doing that instead of making your kernel posix natively?

[hexed solstice]

elegance mostly

the kernel was going to be even more policy neutral originally

[sinful jetty]

arent u running into a potential risk of ipc bottlenecks

[hexed solstice]

that's why i adopted some posixisms for convenience

the handle table obeys posix semantics now

[sinful jetty]

i see

[hexed solstice]

what i wanted to avoid there was that every posix operation involving an FD round-trips to the posix server to resolve the FD

so now instead there's going to effectively be a 1:1 correspondence between FDs and handles

signals, ttys, and the process tree are not nearly as common as reads, writes, and ioctls so i don't see it being a problem

if it turns out to be a bottleneck i can multithread the posix server

[quick saddle]

Very interesting typo

[modern fable]

managarm 2.0

[hexed solstice]

i am continuing to slowly fill out some basic prerequisites for bash and other things so simple that they don't need the unix policy that the posix subsystem will provide

[hexed solstice]

epoll currently. it's a bit mind-bending to support it comprehensively with the numerous flags

[hexed solstice]

https://gist.github.com/netbsduser/007abfffcbd7b1812878ed3baf22ae04 decision made: try to be faithful to linux epoll semantics, don't bother with EPOLLET/EPOLLEXCLUSIVE/EPOLLONESHOT/EPOLLWAKEUP/etc yet

[zenith karma]

are you doing native epoll? or doing epoll->kqueue translation

something someting kqueue ftw

[hexed solstice]

kqueue is more advanced but it's hard (impossible?) to replicate some linux epoll semantics with kqueue

such as the getting epoll notifications for FDs you closed

[zenith karma]

new feature: EV_CLOSEFD?

[hexed solstice]

i think you interpreted that charitably

the notifications are normal notifications

[zenith karma]

what kind of crazy ass semantics does epoll expose?

[hexed solstice]

you can never get rid of them after you've closed an FD without calling epoll_ctl to delete it first

[zenith karma]

oh ohhhh

[hexed solstice]

basically epoll translates the FD into a file pointer and then uses the FD merely as part of the key

[zenith karma]

thats an interesting design choice

yeah

i see

[hexed solstice]

it doesn't actually have a notion of FDs

it might be that this is so unwanted a behaviour that it shouldn't even be emulated

[zenith karma]

why does kqueue track the fd?

i mean its a bit silly that epoll will just keep it around forever

[hexed solstice]

so it can do the right thing and unwatch it rather than keep a zombie open file alive

[zenith karma]

but automatic closing is funky

[sinful jetty]

I mean its still considered open right

[zenith karma]

ah. lol

[sinful jetty]

Just without an fd?

[zenith karma]

just make a flag thats like

[hexed solstice]

the open file exists but there may be no other way to get to it

[zenith karma]

EVF_EPOLL_STUPIDITY

[zenith karma]

well there's always a kernel exploit

[sinful jetty]

Is it still displayed in proc pid fd in that case?

[zenith karma]

nop

[sinful jetty]

I imagine not

[zenith karma]

because its closed

[hexed solstice]

this is what linux got for not looking at any of kqueue, /dev/poll, or i/o completion ports

[sinful jetty]

I wonder how that plays into the maximum number of open files quota

[zenith karma]

huh

good point

to be fair this mechanism IS easier to implement

so i see why linux does that

[hexed solstice]

it's a typically linux thing

[zenith karma]

or call it EVF_ZOMBIEFD

or something

[sinful jetty]

So u can technically work around the quota in that case and open a million files

[zenith karma]

but you cant do anything useful with them

of course!

linux!

[sinful jetty]

Just to dos the kernel

[zenith karma]

hmm i wonder if thats a dos

yeah lol

[hexed solstice]

epoll should not be slammed too hard, nowadays if you combine EPOLLET, EPOLLONESHOT, EPOLLEXCLUSIVE you can even avoid thundering herds in a similar way as kqueue with EV_CLEAR

[zenith karma]

so basically if you combine all their flags thats the same as one kqueue flag

that probably existed 5 years before they started

why are you doing epoll again?

[sinful jetty]

Wayland in keyronex

(I hope)

[zenith karma]

they surely have kqueue

for the BSDs

[hexed solstice]

yes, that's a goal of mine

however much we love X11 it's fading away

[zenith karma]

i mean at least wayland isn't that bad of a protocol i guess

[sinful jetty]

Hmm no idea

Fadanoid is doing it for a reason probably

[zenith karma]

oh i'm sure

[hexed solstice]

i am doing epoll mostly as practice

[zenith karma]

for implementing ugly linux interfaces?

[hexed solstice]

that and because some elements of it are similar to kqueue

[zenith karma]

oh hmm EPOLLONESHOT actually sounds incredibly useful

[hexed solstice]

there isn't really a world of difference between the two, kqueue is mostly epoll done right and more extensible

[zenith karma]

for my init system thing

i can arm an EPOLLONESHOT instead of adding and deleting events

which is also not racey

[hexed solstice]

in keyronex we can expect an EVFILT_IOP for asynchronous i/o packets and an EVFILT_PORT for I/O completion ports to be watched

[zenith karma]

are you planning to have IOCP?

[hexed solstice]

and EV_CLEAR (stops future delivery of an event until it's turned back on) for avoiding thundering herds when doing multithreaded kevent

[zenith karma]

you know there's a truly linux solution to this problem

called eventfd

which im pretty sure is just iocp

except with fd in the name

[hexed solstice]

yes, i think it would be fun to add

[zenith karma]

to be fair the linux naming scheme of <thing>fd isn't like that bad

although it would be nice if we got kmiscfd and not AF_NETLINK

[sinful jetty]

Indeed

[hexed solstice]

xnu for its part has some event filters which subsume some of the things that netlink is used for

[zenith karma]

indeed

[hexed solstice]

netlink is used for wildly diverse things

[zenith karma]

i think xnu kqueue lets you monitor directory changes

and linux instead created a notification object inotifyfd for it

[hexed solstice]

the last thing i used it for was the CN_PROC GNU/Linux® NetLink® Process Event Connector™️

[zenith karma]

WHA

what the hell is CN_PROC

i have so many questions right now

[hexed solstice]

a bit like EVFILT_PROC (very useful for init systems!) except it's a firehose that tells you every time someone forks, exits, etc

[zenith karma]

but also actually this is quite useful

except for netlink

[zenith karma]

that does sound kinda useful

although it should be noted that you can just use a process object pidfd

[hexed solstice]

beware it has fucked up interactions with PID namespaces

[zenith karma]

of course it does

do they give you the pid from the child namespace?

[hexed solstice]

that was meant for CN_PROC but true also of pidfd

[zenith karma]

what is the problem with pidfd???

[hexed solstice]

i think so, i forgot, it was a while ago

[hexed solstice]

it refuses to kill over PID namespaces or something like that

[zenith karma]

... why...

even if we are the parent namespace?

[hexed solstice]

arsen fixed it in a patch but i don't think it was merged

[zenith karma]

so the child can just escape into a new pid namespace and they can't be killed with a pidfd?

but anyway i can just forcibly kill it by manually signalling the corresponding pid

which cant be reused while a process object pidfd is held

[hexed solstice]

epoll (and kevent too) is a nice example of the way simple locking doesn't work in a kernel, because adding/removing/modifying a poll on an object has the obvious lock ordering epoll->lock then object->lock, but wakeups have the opposite

and another thing, epoll_pwait along with similar p-functions should atomically modify the signal mask, but the keyronex kernel is supposed to have no concept of signals and that's supposed to be a posix server thing, so that requires some thought

the posix server will not implement epoll (it's one of those things that's a hassle not to have coresident with the VFS) but what i could do is make epoll_wait asychronously wait and queue an AST on the thread that invoked it when it wakes

[vale pelican]

will the kernel have epoll proper in it or an abstraction that lets you implement both it and kqueue in the posix server

[hexed solstice]

this initial solution i just came up with does give the kernel something very like epoll, without the signal mask stuff, but an asynchronous epoll (in the case that there is nothing ready yet) that causes a message to be delivered to the posix server when the epoll_wait that would've had to sleep in the ordinary case has either timed out or has been satisfied

i like the idea of something more generic in the kernel but i struggle to think of how it would look

this solution is a bit overengineered but it would make for a very entertaining use of two features i want to implement (dpc-wait, which is where the usually synchronous wait on e.g. a mutex, event flag, semaphore, etc becomes an asynchronous one that queues a DPC on completion; and ASTs, the generic alternative to signals, just a way to cause some thread to execute a callback)

[hexed solstice]

epoll needs a locking revision to fix this case:
add epol watch on fd -> lock epoll spinlock -> lock file underlying object's poll list lock
file becomes ready for read -> lock poll list lock -> lock epoll lock of each waiting poller to move the poll list entry onto the ready list

this is a poor example as this can't(?) happen, change "add epoll watch" to "remove epoll watch" however and it turns true

[hexed solstice]

https://gist.github.com/netbsduser/007abfffcbd7b1812878ed3baf22ae04

alright

i think this protocol works

i haven't tested it but i think it does

i separate out locking of the ready list and the watches list

[hexed solstice]

it seems to. next on the agenda should really be implementing fork

[hexed solstice]

i took a detour. i think i have implemented epoll properly now, and i hopefully also made it so that vnode reference release works too

the kernel (+ drivers) currently amounts to just over 34,000 lines of code, by contrast the old one amounted to 25,000 and had features not yet in new keyronex (tmpfs, comprehensive unix/internet sockets, and lots of posixy stuff enough to run xorg and friends). granted this new one has features the old didn't have too. it's getting to become a complicated system

[hexed solstice]

now it really is time to implement fork()

[hexed solstice]

fork logic is underway

i planned out most of how it would work over a year ago

luckily i haven't fogotten the plan

[hexed solstice]

i am getting there, slowly

the bulk of the vm fork operation is done

i think the page fault handling logic for vm fork will be the heavier work though

[earnest salmon]

if you already have CoW what's the cost of fork?

[final hatch]

i guess only the thread that called fork() is going to be cloned?

[hexed solstice]

i have only asymmetric cow

[final hatch]

just to make sure

[hexed solstice]

fork is symmetric

[final hatch]

wdym

[hexed solstice]

if you mmap MAP_PRIVATE something then you read-fault on it, you map in a shared page; when you write-fault on it, you yourself copy that shared page at that point and you get your own private page

in symmetric cow, if anyone write-faults on the shared page, you copy it, unless there's no sharers left - then you just steal it

[final hatch]

So basically you just infinitely dish out that page of zeroes, I assume

[hexed solstice]

https://keyronex.readthedocs.io/en/latest/vmm.html#anonymous-mappings i discuss it here in the old keyronex manual

the basic logic is going to be the same

except vmp_anons are only allocated on fork() now as they aren't needed in the general case

[final hatch]

oh I didn't realise you were not talking about anon memory

well the same logic could work wouldn't it? like

on a read fault, you add 1 reference to that page and map it COW

on a write fault, you duplicate that page just as if you were duplicating a regular CoW page

except maybe itd still be referenced in the page cache

but not a problem cause the PFN goes into transition onto the modified/standby list

[hexed solstice]

for fork() the memory that's operative is private memory - both formally anonymous private memory, and the implicitly "superimposed" layer of anonymous memory that sits over a MAP_PRIVATE mapping of a shared memory object

[hexed solstice]

the reference cannot be done that late

[final hatch]

why not?

[hexed solstice]

if it were done at time of read fault then the parent can annihilate everything before the child has had the chance to fault on it yet

forked memory requires explicit tracking not of "how many resident sets is it in" but of "how many address spaces refer to this page, be it in their working set or no"

[final hatch]

what's the "parent" in this instance? the file that's mapped's page cache?

[final hatch]

ah

but huh

why would a forked page not be in the working set

[hexed solstice]

fork only affects private memory (or private memory + memory that's already been made fork-shared)

[hexed solstice]

the working set directs page replacement and the refcount in a vm_page_t is the number of reasons to keep it resident in memory, which is usually identical with how many working sets it's in; forked pages still have that, but there is a reference count (or you could call it a share count or something else) of how many address spaces could possibly refer to it

https://keyronex.readthedocs.io/en/latest/vmm.html#anonymous-mappings i think i describe it well here

[final hatch]

oh i think i get it

since the page may not immediately be present in the new process' address space

but could potentially be faulted in in the future

[hexed solstice]

and may indeed be evicted from every working set and swapped to disk

[final hatch]

this is a lot to think about and it's why i can't seem to make any progress on boron's memory manager

i wonder how i can learn all this stuff

[hexed solstice]

i read and read and read the uvm paper, the mach vm paper, and some others until it started to make sense

[final hatch]

i see

[hexed solstice]

if i look back at notes i made in june 2022

they are altogether wrong

i misunderstood uvm badly at that time

https://gist.github.com/netbsduser/4439a4d8b0518bb9a0490c6c4361a023/2a072034ee3126241e1dd5788b0432407f72b549 this has some things correct and some things wrong

(though having said that, i wonder what i was on to here: vm_object_t - a mappable object [...] also carries a size, and a queue to which all vm_map_entry_ts mapping the object belong with regard to the queue of all vm_map_entry_ts mapping the object - that's an interesting feature and one i have need of now to be able to do file truncations while files are mmap'd)

swapping (presumably an anon should be able to identify either a physical page or refer to where a page has been swapped out to. do we want pagers to operate at the anon level? logically then vnodes are implemented in terms of a vnode-pager for anon mappings.) --- answer: yes, this is what we will do.
this however is nonsense born of a misunderstanding

alright, i think the bulk of the fork operation is now done

next i'll implement the eviction logic, that should also be relatively easy

and then it's the slightly more challenging fault logic (i have the broad strokes of how it will work, but i have a feeling i will encounter some complication or other)

[hexed solstice]

the fault logic is basically done

only a few edge cases remain (namely cases where while a fault is being serviced by page-in, or while we are even doing a fork itself, another fault happens on the same address - the second faulter or the forker respectively need to abandon their locks and wait)

[hexed solstice]

gnu bash loads and can run builtins

i have mostly completed the VM logic for fork

i can't really do the rest of it because the native API has a handle-based process model and not a tree, and tree-based process model is to be a posix server responsibility

in any case it seems to work properly in the parent process, i haven't test the child process as that stuff is not yet done

[wet apex]

good work

[hexed solstice]

thanks, i really am burdening myself somewhat with the scope of some things here

the vmm has reached almost 9000 lines of code and my "immediately planned" improvements will push it up another 1500 or so

[twin haven]

wow the vmm is quite a significant chunk of the codebase then

[stone orbit]

how does keyronex achieve virtually contiguous pfndb?

it feels like a chicken and an egg problem, cant map memory before pmm is up but you need pfndb for the pmm

i am curious as to how you solved that

[twin haven]

you could bump allocate the pages for those mappings from the memory map

[stone orbit]

i guess i could have the pmm alloc/free be two function pointers

that point to a bump allocator before pmm proper is up

[twin haven]

would you even need free that early on?

[stone orbit]

well, i was thinking that maybe it would be nice if it all went through the same page table api

but now that i think about it

what i really am doing is setting up a "template" page table

so there is no need for it to go through all that code

i would set up a template page table, with the pfndb, higher half and kernel mapped, that would bump allocate pages out of memory map

and then with that up i can initialize the pmm and vmm nicely

[hexed solstice]

I bump allocate out of the largest block of available memory

First 2mib pages (that's what actually backs the pfndb) in dense succession, then the 4k pages follow

[stone orbit]

i am doing the same now!! it's a nice and simple idea, i like that, it will let me bootstrap pretty easily without worrying too much 😄

i am mapping the hhdm and kernel, as well as the pfndb

i am going to use 1GiB pages for the hhdm so hopefully i dont use too much memory

mapping the hhdm at 0xffff'8000'0000'0000 and the pfndb at 0xffff'9000'0000'0000, i also want to implement support for loadable kernel modules early on

im not yet sure where ill start mapping the kernel modules but i think +256MiB relative to the kernel base, so without ASLR it will be 0xffff'ffff'9000'0000, i don't think my kernel will use anywhere close to 256MiB of virtual memory so i should be safe and i will have plenty of space to map the modules

[twin haven]

any reason why you wouldnt put them above the pfndb but below the kernel?

[stone orbit]

no idea, i just thought putting it closer to the kernel would make sense, logically

there's still almost 2GiB for the kernel modules, which should be plenty given most of them will be a few virtual pages at most

but i could put it like 16GiB below the kernel or whatever

or at like 0xffff'fffa'8000'0000, i like that address

the layout of the kernel virtual address space will probably be subject to big changes as i change my mind though

[hexed solstice]

namely:

. 34700 0
cshim 10
dev 7450
executive 2883
fs 1263
include 3333
kdb 475
kern 3167
libkern 306
net 839
platform 3650
storport 1646
vfs 893
vm 8785

overall new keyronex is 9000 loc longer than old, and still so much to do

[sinful jetty]

does storport work?

[hexed solstice]

enough for storahci, vioscsi, vioblk, and stornvme to read and write blocks

[wet apex]

Neat

Do you have any on disk filesystems yet?

[hexed solstice]

read-only ext2, vfat

[sinful jetty]

thats really cool

did microsoft specifically design this to depend on so few things or something?

[hexed solstice]

drivers are supposed to just use the storport functions, viz.

but don't be fooled, StorPortNotification and StorPortExtendedFunction are monster variadic functions that do all kinds of things depending on a type parameter

[wet apex]

Cool

[zenith karma]

do they actually only use those?

or is it more like "well yes but actually no"

[hexed solstice]

in practice they are disobedient and use others, these cover the drivers above:

[zenith karma]

ah

[sinful jetty]

ohh, so like you are the storport, and drivers use that interface?

[hexed solstice]

yes, it's what i think they call a miniport-port driver system

the port driver is a storport and the miniport driver is storahci/stornvme/etc

the storport port driver takes care of a lot of the common issues and the miniport driver provides the concrete interface with hardware

[twin haven]

that was a confusing message preview to receive

interesting to see the breakdown

[sinful jetty]

and how do you call into the drivers?

[hexed solstice]

the driverentry routine registers a set of callbacks

[sinful jetty]

i see

why did they decide to confine the drivers to that set of functions a few of which are giant instead of letting them use normal kernel api?

[hexed solstice]

cleanliness, i assume

[sinful jetty]

does this mean u also support loading PEs in your kernel?

as dynamic kernel modules

[hexed solstice]

yes, quite a pleasant format

the two-level namespacing for imports is particularly nice

[sinful jetty]

your dynamic kernel modules are also PE or do u support both?

im guessing for storahci u just take the proprietary blob from a windows distribution?

[hexed solstice]

they'll be elves when i fix up support for them again (i don't think an m68k-pe toolchain exists)

i built my own storahci because i had some debugging to do (it's an open source driver), i think a prebuilt one would work, maybe not if it's for very recent windows, i don't know if i cover everything needed for that case

[sinful jetty]

what about nvme and others?

[valid sequoia]

stornvme works too

[sinful jetty]

is it also open source?

[valid sequoia]

nope

but it worked finr for me

[sinful jetty]

wheres the source for storahci btw

[valid sequoia]

https://github.com/microsoft/Windows-driver-samples

[hexed solstice]

the vio ones i also built for debuggings' sake, i don't think i had to build my own stornvme though, that one i didn't try til after i ironed out the problems with vioscsi, vioblk, and storahci

[sinful jetty]

its a sample but it literally ships with commerical builds?

[valid sequoia]

yeah

so is fastfat too

[sinful jetty]

so for stornvme u just grabbed the one shipped with your windows?

[sinful jetty]

interesting

[hexed solstice]

i think so, i got it somewhere anyway

[sinful jetty]

interesting

[hexed solstice]

ndis would make for another interesting target

but i know next to nothing about it yet

[valid sequoia]

the only annoyance is that usbstor is not storport based iirc

[sinful jetty]

i dont see any quirk workarounds there, interesting

how does it work everywhere

[hexed solstice]

what sort of quirk workarounds are usually necessary?

[sinful jetty]

https://github.com/torvalds/linux/blob/master/drivers/ata/ahci.c#L125

this is an array of per device quirk list

[hexed solstice]

very interesting

well, apparently windows' one just works

[sinful jetty]

Yeah that's what im surprised about

[hexed solstice]

the driver invokes a few acpi methods

perhaps something magic is going on you are in the unique position to understand

[sinful jetty]

can u link those

[hexed solstice]

https://github.com/microsoft/Windows-driver-samples/blob/b3af8c8f9bd508f54075da2f2516b31d05cd52c8/storage/miniports/storahci/src/pnppower.c i think all the StorPortInvokeAcpiMethod()'s are in here

[sinful jetty]

yeah, _GTF for get task file and _DSM (device-specific method) for setting some capabilities

task file is some ahci specific thing iirc?

[prime coyote]

doesn't it cause any problems that your implementation doesn't actually work according to msdn

[hexed solstice]

in what regards?

[prime coyote]

when drivers expects one behavior and your implementation return something else, or they are not intended to be used by drivers in that way

[hexed solstice]

well, i would simply try to return what's supposed to be returned

[prime coyote]

for example some functions are meant to be processor group aware and return based on the provided group number

so if you don't support proc groups, the return value might be different from what a driver expects? or maybe not

[hexed solstice]

i don't yet support > word size in bits processors, so no problem

when i do i'll see if there's any problem

[sinful jetty]

Keyronex cant run on my 128 cpu EPYC

[hexed solstice]

all in due time

i have classic RCU which will help provide a basis for scaling to a few hundred cores

there are a lot of areas that need revision ahead of that though (VM domains for NUMA-local allocation & page replacement; breaking up the PFN lock; probably more)

[sinful jetty]

Have you started on numa support yet?

[hexed solstice]

sort of, i have been prototyping the beginnings of it in a third repo

i know what will be done for it

the introduction of a vm_domain_t which will contain { pagequeue_t freelist[BUDDY_ORDERS]; pagequeue_t modified, standby; thread_t *writeback_thread; } etc

and of course its own page queues lock (which even makes it worthwhile to have multiple VM domains per NUMA domain as it decontends a lock that, even after vm locking reform, will remain relatively hot)

though per-cpu free page caches (magazines) will help with that too

[sinful jetty]

Nice

[velvet horizon]

boo did i scare you

hows keyronex going !!

[hexed solstice]

right now what i want to do is figure out how the kernel paged heap allocator(s) will look

[hexed solstice]

it goes well but there is much to be done

[velvet horizon]

I see

good luck !

[hexed solstice]

cheers

i am trying to move in a more careful and controlled way than i did before

keyronex last year was very much a sort of speedrun

it started in february and by april was doing this

but at the cost of a lot of shittiness in the implementation

[hexed solstice]

i don't think adapting the slab allocator to be able to managed paged slabs will be too much bother

it'll need a flag added to it which will cause it to synchronise with a mutex instead of a spinlock if set, and to use the paged heap allocator/freer instead

as to the paged heap itself, i think a fair thing to do is to allocate chunks of say 16mib or so, put a traditional (bitmap-based) buddy allocator's metadata at the end of that region to allocate spans out of that region, create new chunks as needed, destroy them when they're empty

[stone orbit]

@hexed solstice is there any reason you implemened curcpu the way it is implemented currently in keyronex? why not have gs point to the cpu info and get the thread from there instead?

i was curious how different kernels handle that and i wanted to know what the best way to do it would be lol, are there any drawbacks of one or the other?

[hexed solstice]

it was a stopgap measure to make getting curthread atomic but the long term solution is going to be a struct cpu_local { blah blah blah }; with %gs-relative loads on amd64, %tp-relative loads on riscv, %x18 relative loads on aarch64, and on m68k i don't need to worry because there is only one CPU

[stone orbit]

i wanted to have something likestruct pcb { thread *cur_thread; cpu *cur_cpu; };
and just have the address of that loaded into gsbase

so it seems like i was on the right track from the start

[hexed solstice]

and maybe in the future there could even be dynamic per-cpu variables (in that case %gs with register offset on amd64, %x18 with register offset on aarch64, on risc-v will have to be done in a no-preemption region because no addressing mode suitable, on m68k trivial again)

[hexed solstice]

that's the right way i think

[stone orbit]

i dont think im going to do any of that, so ill just stick with that for now, thanks for explaining 😄

[wet apex]

I do that too except gs points to the cpu structure and it has a pointer to itself

So I just do gs relative movs for thread and cpu

[stone orbit]

ah yeah that makes sense, i used to do that too

its a decent solution but i feel like this little struct makes it a little bit nicer

[sullen fern]

I just store the current thread

I'm not sure why I even do that but iirc it was simpler for some stuff

[hexed solstice]

i will prepare to move towards a separate struct kcpu_locals in keyronex forthwith

i would like to do freebsd style dpcpu (dynamic per-cpu variables including for kernel modules) and zpcpu (dynamic per-cpu dynamically-allocated memory from the zone (slab) allocator) but i don't fully understand the former yet and the latter i don't know the first thing about yet

[final hatch]

huh

why not put all the cpu stuff into pcb

[stone orbit]

eh, i dont like a self pointer

[final hatch]

anyway other than that i think this is the way to go

?

[stone orbit]

if i wanted to get the cpu pointer i'd need a self pointer, or read from the msr

[final hatch]

i mean like

struct pcb {
    thread* cur_thread;
    // whatever you would have put in struct cpu
};```

[stone orbit]

yeah

how do i get stuff out of the pcb?

[final hatch]

read from the msr?

[stone orbit]

meh

[final hatch]

is that too slow or what

[stone orbit]

serializing

iirc

you probably dont wanna do that in a hot code path

[final hatch]

doesnt sound like too much of a problem to me

i mean go for it i suppose

[stone orbit]

maybe not but still its nicer that way

[final hatch]

but its like 1 extra level of indirection

[stone orbit]

not really

gs:0

gs:8

[final hatch]

instead of reading from the msr and using the offset there

[stone orbit]

to get the cpu and thread, respectively

[final hatch]

you do a gs relative pointer access and then another pointer access

[stone orbit]

also reading the msr then the thread is not atomic

:^)

[hexed solstice]

it may be you won't need a self pointer

[final hatch]

nope, so you can just use a gs relative access for that

[hexed solstice]

it depends what you want to do

[final hatch]

in fact you might be able to do a meme where you declare like __seg_gs KPRCB CurrentPrcb;

[stone orbit]

see, it's a meme, me no like

[final hatch]

or whichever attribute you tell the compiler it's to do a gs relative access to access

[final hatch]

i call it a meme, means not that it's a meme

windows nt actually does an even more "strange" meme on MIPS

[stone orbit]

but you called it a meme

oh yeah but it's NT

the whole thing is kind of a meme anyway

[final hatch]

it maps the last page to the uh

what was it

not prcb

pcr yes

[stone orbit]

thats actually quite nice

i think will did something similar in mintia?

[final hatch]

iirc it does it this way through having a TLB for each processor

i don't know the details of mips so i can't tell you for sure

[stone orbit]

0xfffff000 being the pointer for something?

[final hatch]

yeah

[final hatch]

this sounds similar to me

because like the windows nt mips thing

[stone orbit]

funny 32-bit architecture be like

[final hatch]

you can just KiCurrentPcr->whatever

on mips itd be 0xFFFFF000 + offsetof(KPCR, whatever)

on x86_64 it'd be a gs-relative access to ((KPCR*)gs:0)->whatever

[stone orbit]

you mean gs:whatever?

[final hatch]

yeah

basically

[stone orbit]

yeah funny attribute

but i dont mind the indirection

[final hatch]

i should implement this in boron tbh it's kind of a thing i lack

i read from the msr to get the gs base

except for getting the current thread and IPL of course, i use a gs relative access for those

[stone orbit]

oh wow

apparently rdmsr isnt serializing

the more you know, i guess :')

wrmsr is, i guess

that makes a lot of sense actually

[hexed solstice]

bloody gcc doesn't recognise __seg_gs in objc

[vale pelican]

also doesn't recognize it in c++

which is sad

[hexed solstice]

when i last looked, large parts of the frontend are completely duplicated between c, c++, objc, objc++ so maybe it's an easy fix

i'll have a look one of these days

[vale pelican]

for c++ i think the issue is around extra c++ features?

like global ctors/dtors for things marked __seg_gs

[hexed solstice]

it's understandable then

i don't need-need it, it would be nice to have

but i can do the same job in inline assembly

[hexed solstice]

i am working on introducing a new kcpu_local_data_t structure which will be the new basis for cpu-local data

since __seg_gs is not available to objc it's implemented on amd64 like this

on aarch64 and risc-v it will be implemented like register kcpu_local_data_t *cpu_local asm("x18") and asm ("tp") respectively

should do the job nicely

i am considering extending it for dynamic cpu-local variables as well that can be declared anywhere (go into a special section and load by offset of dynamic cpu local from start of the cpu locals section) and it would be even more fun to support this for modules too

[rain flame]

Use rbp for amd64 :^)

[hexed solstice]

aarch64 port is now converted to use the new cpu local variables stuff, kept in x18 with a backup in tpidr_el1

[hexed solstice]

risc-v (in tp) too and m68k too (a global variable; no SMP is helpful)

[wet apex]

How does this work in aarch64? Ive never really read much into the architecture and I assume x18 is a gpr?

[vale pelican]

the abi reserves x18 for platform use

[stone orbit]

cant userspace just overwrite it though?

or even read it?

[vale pelican]

it can, i assume that's why there's a backup in tpidr_el1

and i assume each task gets it's own value of x18

[hexed solstice]

that's why i do this

and i zero it on entry to userland of course

[stone orbit]

thats just reading it on interrupt entry i assume

ah nice

so its basically like gs on x86, except you reload it every time you enter kernel from userspace

i assume you can still do relative loads, like [x18 + 8]

[hexed solstice]

yes, i use register .... cpu_local asm("x18") and then it generates appropriate code automatically

so it's even less hassle than GS on amd64

[stone orbit]

i know it's not like gs, but you use it as such

oh wow thats nice

[wet apex]

Damn

[stone orbit]

is that universally supported by other compilers?

both by gcc/clang i mean

[valid sequoia]

yes (for x18, not gs)

[hexed solstice]

for which i had to write inline assembly to do gs-relative movs as __seg_gs is not for objc

(and i only bothered for 1, 2, 4, and 8 byte cases, but that covers everything i need it for and more anyway)

[vale pelican]

how much pcpu data do you have in your kernel that's not arch-specific?

[hexed solstice]

cpu number, current thread, scheduler state, dpc stuff, rcu state

when i get around to more sophisticated drivers for nvme, virtio, certain NICs, anything with multiqueue enough to provide a queue per core, it will grow and it will become worthwhile to support either module link time or runtime dynamic allocation of pcpu variables

[stone orbit]

like tls but not limited to stuff you explicitly put into the struct?

[hexed solstice]

yes, and on aarch64 (as well as amd64 and implicitly m68k since there's only one core supported right now) i can also do a relative load/store with the offset stored in another register which will come in handy for this if i want to be able to do it atomically

on risc-v i think will need to go back to the drawing table or only manipulate them with preemption disabled as no suitable addressing mode there

[stone orbit]

there isnt something like mov [tp + xxx], yyy?

[hexed solstice]

only with immediate offset

[stone orbit]

that sucks

just precompile some stubs for accessing the data

arent riscv instructions like 4 byte each?

8 bytes per stub sucks actually

at least you can use an immediate offset with inline assembly

[hexed solstice]

in practice i think it's no great loss as the fixed per-cpu variables are kept in that struct and so long as i don't add too many of them those will work without problem

and getting the current thread is the most important one and the only one that most people will ever do without preemption disabled anyway

[hexed solstice]

i am feeling like sorting out how time and clocks work next

it would be nice to support dynamic ticks

[zenith karma]

i think apple does this

iirc gmp's inline asm just doesn't care though because fuck portability though

or used to anyway

[hexed solstice]

i put it across poorly, i zero it on initial entry to userland but i save it and restore it on interrupts so it can still be used by userland code

gmp should not be using it willy-nilly as the procedure call standard says:

The role of register r18 is platform specific. If a platform ABI has need of a dedicated general-purpose register to carry inter-procedural state (for example, the thread context) then it should use this register for that purpose. If the platform ABI has no such requirements, then it should use r18 as an additional temporary register. The platform ABI specification must document the usage for this register.

[zenith karma]

they used to do that, afaik

because linux is the only platform and therefore cool

[zenith karma]

ah, yeah

[hexed solstice]

for full dynticks operation and availability of high-res timers i might require the platform has a time source that won't ever overflow in a lifetime, and that it has per-CPU deadline triggered interrupts

so with an invariant TSC and TSC-deadline, or with ARM generic timers, or with RISC-V's whatever alphabet soup defines stimecmp, or with qemu m68k virt's goldfish timer, that's met

[vale pelican]

sounds reasonable

[hexed solstice]

it saves a bit of hassle like having to keep one core ticking to timekeep, or having to cross-reference with other timers, and i think the requirement is met by all modern platforms released in the last decade

[earnest salmon]

Proactive writeback for cached files.
I saw this in keyronex' read me, and was wondering what that means

Mapped file cache for coherent cached read/write of files.
I assume this is the page cache?

Bundy resident page-frame allocator with page stealing.
and this is just the PMM + dirty/standby lists I assume?

oh and idk if you checked, but keyronex is 123.9k lines

which is cool

with 73k being code

[hexed solstice]

this will be a daemon or multiple daemons in the kernel that write back data written through the cached write API (which works through the mapped file cache) more eagerly than the dirty page writeback daemon does

[hexed solstice]

it's an area of LRU-replaced mapped file windows

because of the specific character of the area the page fault codepath for it is slightly optimised, while the mapping and unmapping codepaths for that area are even moreso

in keyronex the only way to do cached IO is through a memory mapping which is why this exists

[hexed solstice]

yes, i don't really conceive of them as separate as page stealing is a normal case of allocation

[hexed solstice]

i need to put together a rough todo list for keyronex one of these days. there are a few things i could do:

• move forward on posix support (how will the division of labour between posix server and kernel look? i will need to define the IPC mechanism too. i have most of fork implemented, but i need the posix server for the process model.)
• start the VM locking reform (big!)
• start VM NUMA support (ties in slightly with the above)
• start implementing balance set management in earnest
• setjmp/longjmp for riscv and m68k in mlibc (should be straightforward and let me see whether bash gets as far as a prompt on m68k, as it can on amd64)
• add write support to the dosfs and/or ext2 drivers
• get the kernel module system in good order
• turnstiles
• scheduler enhancements (load balancing, comprehensive priorities with timeshared, fifo, and rt classes; understanding of locality; support for BIG::little and whatever the x86 equivalent is called would be nice)
  the list could go on

[twin haven]

VM locking reform sounds interesting (as do the others tbh), what are you looking to do there?

[hexed solstice]

splitting the big pages lock into (conceptually, and maybe as an initial thing) two - one that locks the state of page tables/the owners of pages, one that locks the page queues

from there it's much easier to move further, to a per-object page mappings state lock. and even further (e.g. per radix tree level lock) is possible on that basis

[twin haven]

thats cool!

[hexed solstice]

an even more exciting possibility is raised of lockless lookup of pages in an object, if the tables are RCU deferred-freed

[hexed solstice]

another thing i can do, because i am really struggling to summon up the energy to actually code anything in keyronex lately: documentation (work on the book, or write some manual pages)

[sinful jetty]

yeah i think thats important as well

is blue logo now the official for keyronex?

i thought it was a bug

[hexed solstice]

the tabloid masthead style logo is white-on-blue and the logo that appears on the graphical console should be near-black but is blue by accident

[sinful jetty]

yeah but the readme now has the bugged logo too

or is that now officially accepted

[hexed solstice]

this one is the tabloid style masthead, it's supposed to be blue

[sinful jetty]

ohh

interesting

[hexed solstice]

it would be red but that's too on the nose and would look completely like a newspaper logo like this one

[sinful jetty]

yeah

[hexed solstice]

it would also bear too much resemblance to

[sinful jetty]

lmao

[hexed solstice]

maybe i should prepare a pretentious Brand Identity document like this one https://brand.systemd.io

[sinful jetty]

that would be cool

[hexed solstice]

The brand colors are systemd green #30D475 and systemd black #201A26.
i mean for goodness sake

The abstract shapes in the brackets symbolize the "OK" from the boot up screen, services running inside systemd, and our overall optimistic outlook.

[sinful jetty]

The only situation where we find it OK to use an uppercase letter in the name (but don't like it either) is if you start a sentence with systemd

lmao

[hexed solstice]

keyronex blue and keyronex black. the blue symbolises little endian the black symbolises big endian. the stylised arrow signals the ever-present, ever-increasing desire to port to more platforms. the box symbolises that it runs on boxen

[sinful jetty]

boxen?

[hexed solstice]

freenode #osdev irc's preferred terminology for computers

[sinful jetty]

oh

[hexed solstice]

i wonder when this turned up on that page incidentally

it's nice that it has some humour now as i always thought the rest of it took itself a bit too seriously

i think i'll start work on the manual pages. there is a lot to describe in the kernel, a lot of which is going to stay true (like the IPL system)

i just need to pick a format- i have used docbook before (it has a manpage schema) but docbook i found a bit of a mess and designed-by-committee; mdoc is thorough, but line-oriented, not much fun

[hexed solstice]

i've started in mdoc

[hexed solstice]

https://manual.keyronex.org/master/intro.9.html

that's the rudiments of the first manual page (on the IPL mechanisms) written

[sinful jetty]

I like it

[hexed solstice]

cheers, i also like it

i might waste some time later adding some style to the rather bare html output it offers

next pages will be spinlock(9) and rcu(9)

[earnest salmon]

Why does it say debian

[hexed solstice]

that's the platform the pages were rendered from mdoc to html on

[earnest salmon]

ah

[hexed solstice]

in the future they will be rendered on a keyronex host

[earnest salmon]

having man pages sounds cool

TODO:

[hexed solstice]

then it'll say Keyronex

[earnest salmon]

well until then

[hexed solstice]

i am finding them not too bad to write, certainly easier than writing the book, since it's more practical documentation and less expansive in scope

[glad sentinel]

It’s a thing for the olds, the type that frequented Slashdot back in the day

[solid hemlock]

_ _

[hexed solstice]

thank you for your informative contribution

now added: spinlock(9), rcu(9), ke_rcu_dereference(9)

mdoc format is obviously an acquired taste but i think i am getting used to it now

[zenith karma]

you know you could write a script that processes them from something more readable

[hexed solstice]

yes, it could benefit from a little style

[wet apex]

true

[hexed solstice]

mandoc has a style=whatever.css option, and even better can elide everything except what's inside <body> so it can be trivial

[wet apex]

and I could render them in astral

[hexed solstice]

i need to decide on a style that suitably embodies the Keyronex Brand Identity

[zenith karma]

i mean from input, not output

write input in normal syntax -> convert to mdoc -> whatever else

[hexed solstice]

oh, that's what i get for not looking at what message you replied to

yes, i agree, it would be an improvement