MINTIA (not vibecoded)

[twilit smelt]

i have no idea how much that helps in practice but i was at least able to come up with some semi contrived examples where it did help

shave off an instruction or two

[icy bridge]

why not just pass return values through tN? considering those get clobbered by calls anyway

i mean it's not an option now because abi but still

[twilit smelt]

cuz im stupid

idk

[lucid umbra]

huh

thats crazy

[twilit smelt]

yeah it evades some like value juggling

some movs

can be avoided

sometimes

its not like a major effect though

and normally only apparent in really little leaf utility functions

that tpye of thing

[lucid umbra]

i wouldn’t expect it to help cause the jackal compiler treats arg registers like they get clobbered regardless

but if it helped 🤷

i think it should pass them reversed when all of them can fit in registers, to match current calling convention

but it should do the regular layout when passing by reference

cause it makes no difference when its in memory, and memory is much harder to shuffle around than registers

[icy bridge]

yeah i think i agree

edited the messages for future ref

[lucid umbra]

cool cool

[icy bridge]

whats your opinion on the whole passing pointer vs implicit destination at sp+0 thing

[lucid umbra]

i think it should be an extra pointer, since the compiler would then be free to put that storage space where it thinks is a good place

[icy bridge]

might also improve codegen on the callee side since setting the return value wouldnt overwrite incoming args

[lucid umbra]

exactly

[icy bridge]

alr edited that as well

[lucid umbra]

probably minimizes stack value shuffling in general

unfortunately i am very busy today but @ me if u have other ideas/concerns/questions

[twilit smelt]

@pastel flume i did turnstile if ur interested https://github.com/xrarch/mintia2/blob/main/OS/Executive/Ke/KeTurnstile.jkl

[pastel flume]

Wowowowo

I was literally just talking about mutexes with @blissful smelt

[twilit smelt]

thats why i linked that

[acoustic sparrow]

btw, using a linked list can degenerate in some cases

its better to use some sort of BST to avoid that

go uses a treap for this

[twilit smelt]

what linked list are you referring to

[acoustic sparrow]

KiTurnstileChains

[twilit smelt]

werent you calling me a poopoo idiot for using AVL tree buckets for my string internment

now youre saying i should use bst buckets for my turnstiles

[acoustic sparrow]

yes because the go runtime has bsts there

[twilit smelt]

the go runtime has turnstiles?

[acoustic sparrow]

with a comment noting they hit terrible perf

[acoustic sparrow]

they have a similar mechanism

[twilit smelt]

why were you looking at the go runtime

[acoustic sparrow]

i was implementing basically the same thing for shkwve

[twilit smelt]

whyd your bootloader need that

[acoustic sparrow]

i ended up doing a naiive linked list because perf is boring

[acoustic sparrow]

because idk

its cool

[twilit smelt]

i did a linked list because thats what illumos and xnu do

[acoustic sparrow]

it has preemptive scheduling

[twilit smelt]

why

[acoustic sparrow]

well ok not preemptive

its cooperative but still

[acoustic sparrow]

i want to be able to run two things at once

[twilit smelt]

reminds me of the SRM firmware on DECstations where they went way overboard and it was basically a shitty unix kernel in rom

including a unixlike shell

with job control

[acoustic sparrow]

interesting

[twilit smelt]

https://github.com/apple-oss-distributions/xnu/blob/main/osfmk/kern/turnstile.c#L277-L282

[digital pivot]

"When two contested mutexes happen to wind up using the same hash table entry, the linear search in semrelease is a significant performance drag" apparently that was the cause?

https://go-review.googlesource.com/c/go/+/37103

[acoustic sparrow]

yeah

[twilit smelt]

Think an AVL tree would work better?

[twilit smelt]

@hybrid condor i see you

lurking

[acoustic sparrow]

i would try a treap like go

[twilit smelt]

why is that good

[acoustic sparrow]

tho a linked list worked for plan9, xnu and illumos

[twilit smelt]

idk much about a treap

[acoustic sparrow]

because if it works for go, it must work for you :^)

[twilit smelt]

well cogolein also reacted so maybe it doesnt actually work so well for xnu and we gave them an epiphany.

[acoustic sparrow]

fair

go has very aggressive concurrency

[twilit smelt]

(probly not)

im unsure i even agree with the change go made

[acoustic sparrow]

they had perf issues from a list

[twilit smelt]

a google program had perf issues from the list

[acoustic sparrow]

im guessing they didnt do avl bcuz its harder

[twilit smelt]

rather than redesigning their program they made the common cases slower for everybody (managing a treap thinger instead of just a linked list)

so that their contrived case is faster

dunno if i agree with that

i mostly did avl tree buckets for the interned strings because i can anticipate a situation where theres like tens of thousands of them and i dont have nearly that many buckets and i dont want performance to disintegrate if that happens

but i dont think thats remotely likely for turnstiles

[digital pivot]

they would probably change it if it were a problem for xnu

[twilit smelt]

probably

sometimes there can be subtle things people dont realize are responsible for stuff though

[acoustic sparrow]

you can have huge numbers of threads in go pretty easily

note that webkit's ParkingLot uses a dynamically resizing hashtable

[twilit smelt]

its not just having a huge number of threads

its also the mutexes have to line up

really unfortuntately

however you spell that word

[acoustic sparrow]

yeah but that can happen

it happened at google because they were the only big go user probably

[acoustic sparrow]

is it even much slower?

[twilit smelt]

its gotta be a little slower than just like

pushing an item to a linked list

[acoustic sparrow]

sure

but i think its not that much worse

and this is a very slow path already

[twilit smelt]

your moms a very slow path

[acoustic sparrow]

also if the treap is empty, its just a couple instructions

and thats the common case

[mortal thunder]

it cant "degenerate" because it's one line

you're aware of the performance impact when using a linked list

[twilit smelt]

wat

[mortal thunder]

its a linear data structure it cant degenerate into something worse

like for example splay trees potentially degenerating into linked lists (?)

[twilit smelt]

A hash table can degenerate from O(1) average to O(n)

The result being that an outer algorithm turns into O(n^2) which is the effect reported

[mortal thunder]

the initial assessment was "btw, using a linked list can degenerate in some cases"

and i'm like, no, it'll always be O(n) lookup

[twilit smelt]

I understood what he meant

[mortal thunder]

are you using hash tables with linked lists? is this what he meant?

[twilit smelt]

yes

[shadow ridge]

how likely is this to actually occur enough to warrant a change though

[twilit smelt]

Not at all likely which is why I didn't wanna change it

[shadow ridge]

makes sense

[icy bridge]

function calls work! (to void functions at least) $ echo 'void bar(...);void foo(){bar(5, 5LL, "Hello");}' | host-pkgs/gcc-bootstrap/usr/local/bin/xr17032-unknown-linux-gnu-gcc -x c -S -o /dev/stdout - -fomit-frame-pointer -O3 .file "<stdin>" .text .section .rodata.str1.1,"aMS",@progbits,1 .LC0: .string "Hello" .text .p2align 2 .globl foo .type foo, @function foo: subi sp,sp,4 mov long [sp],lr lui a3,.LC0 addi a3,a3,.LC0 addi a1,zero,5 add a2,zero,zero add a0,a1,zero jal bar mov lr,long [sp] addi sp,sp,4 jalr zero,lr,0 .size foo, .-foo .ident "GCC: (GNU) 15.2.0"

[sterile frost]

holy based

[shadow ridge]

GNU Station

why not a clang backend

[icy bridge]

because i didn't know llvm provided their own version of libgcc_s.so when i started (which is required by the lsb spec)

and now i've gotten far enough with the gcc backend that finishing it is (probably) less work than writing an llvm backend

[teal trench]

gcc is better anyway :^)

[shadow ridge]

k

[tiny swallow]

do you consider writing a long ahh blog post to sum this progress

[icy bridge]

not really, i abhor writing

[tiny swallow]

😔

[dense vigil]

damn

[icy bridge]

update on gcc progress: i have implemented enough for the compiler to be useful! it's missing atomics, thread-local storage, and PIC support (PIE is supported - the difference is that PIC needs to access globals through the GOT and PLT), but for code that doesn't need those features it's perfectly usable

currently starting work on libgcc

oh and while compiling PIE code works fine actually running it is blocked on the adr instruction being implemented in xremu

[twilit smelt]

https://xrarch.github.io/

I updated the emscripten emulator

demo

even does 2 processors now since the emulator got fast enough to multiplex two CPUs on one host thread

(gh pages doesn't like emscripten pthreads and won't tolerate it)

[twilit smelt]

SICK

#1 is finally done on here

Last step for the emulator polishing was to get the emscripten build working again

2-6 i can probably get done in a couple days and that includes adding the ADR instruction

rn i need to work though

[night needle]

yooo

does gcc = c support only or also c++

[icy bridge]

any gcc frontend

gcc, g++, fortran, ada, etc

[night needle]

[mortal thunder]

i guess though that if you wanted to run gcc apps on mintia youd have to write a tool to export xloff instead right

and add dllimport support similar to pe

[icy bridge]

probably yeah

[twilit smelt]

xlo*

xloff was the old one

[mortal thunder]

mintia1

[twilit smelt]

yeah

[mortal thunder]

not 2

[twilit smelt]

and aisix

[mortal thunder]

oh you changed the format for those

[icy bridge]

although gcc itself is surprisingly object-format-independent so most of the changes would be to binutils instead of to gcc

[twilit smelt]

loff was the format for aisix then xloff added arbitrary named sections for old mintia (aisix was also changed to use it) then xlo was just a new thing for the jackal sdk

[mortal thunder]

yeah okay, but i meant running on mintia1

[twilit smelt]

theres also an alternative

he could write an ELF loader for old mintia

[mortal thunder]

ah yes

[twilit smelt]

program loading is 100% in userspace anyway, the kernel has no knowledge of the executable format aside from simple parsing to locate some symbols in osdll.dll (which is in the init section so that code gets freed after boot)

[mortal thunder]

thats true

[mortal thunder]

thats one thing i have in common with your project

among many others

[icy bridge]

thatd probably be a better solution yeah

[mortal thunder]

my kernel only has knowledge of the executable format of libboron.so

so that i can map in all the sections

but it doesnt even do relocations, those are done in libboron itself in a similar manner to mlibc's ld.so

[twilit smelt]

osdll.dll doesnt need to be relocated its just at a fixed known location

its pre-relocated there

[mortal thunder]

hey what file format does your boot rom firmware load

[twilit smelt]

it loads some sectors off disk and jumps to the start

[mortal thunder]

ah, so it doesnt work like ARC for example

[twilit smelt]

yeah but its at least slightly smarter than bios

it can read a partition table and select a partition and theres information there like the OS name and even a little badge bitmap (for the boot picker)

and it reads a sector which tells it what sector number the boot program starts at and how many sectors long it is

iirc i designed this so you can fit an MBR partition table "around" the partition table sector in such a disk, in the space where x86 bios boot code would go

so you can have both partition tables coexisting within sector 0 (and its automatically compatible with GPT tables as well because those avoid sector 0 entirely)

which will be useful for like, linux @icy bridge

this is all documented in the a4x manual

https://raw.githubusercontent.com/xrarch/books/main/documents/a4xmanual/main.pdf

[icy bridge]

yea that'll be useful

btw is it intended that the firmware doesn't check for installed operating systems on the virtual entire disk "partition" (dksXs8)?

that would allow for xrstation to boot off of non-a3x partitioned disks

[twilit smelt]

i figured it was best to just make people do a dummy APT table at least

[icy bridge]

fair enough

but whats the point of that virtual partition then

[twilit smelt]

it makes the whole disk accessible to boot programs using the a4x API

this isnt currently used though

[icy bridge]

ah

[twilit smelt]

also a4x FwBoot.jkl contains xloff structures

but this is misleading

thats specifically for chain-loading an xloff formatted a3x firmware image (stored at the end of the a4x firmware in rom) when booting old mintia and aisix

cuz their bootloaders both need a3x apis and aisix's kernel even uses them at runtime

(old mintia doesnt because it reclaims the a3x memory, aisix doesnt)

i did a whole new (very simplified) firmware in jackal in order to make the whole ecosystem self-hosting

i also needed the firmware to have SMP awareness

but that necessitated some way to boot the old OSes which relied on the old firmware and i ended up doing that chain-loading thing

kind of quirky

so the boot firmware binary that comes with the emulator contains both a4x and a3x firmware

which takes up almost the entire 128kb of the rom

if you build a fresh a4x firmware binary and try to boot mintia or aisix itll fail with this error message

the way to add an a3x image is literally to just cat ./a4x/build/fre/a4x.rom ./a3x/src/firmware.bin > xremu/bin/boot.bin

just cat'ing them together works (with the a4x image first)

very hi tech

[fleet root]

Some sort of imaginary computer?

[twilit smelt]

yeah im making a self hosted computer ecosystem for fun

theres a custom risc architecture, custom language w/ self-hosted compiler, and an NT-lookin kernel with an SMP scheduler

it has paging and all that

[fleet root]

Have you tried playing around with FPGA?

[twilit smelt]

someone else implemented enough of it on an fpga to run the boot rom

it couldnt boot the OSes (of which there are 3, soon to be 4 because someone is porting linux.) because it didnt implement the mmu or any mass storage

theres a web demo https://xrarch.github.io/

i plan on doing my own full fpga impl when mintia2 (the current OS project that this channel is for and which currently just runs a multithreaded "fireworks" test) is "done"

[fleet root]

Yeah, it's a fun project.

To be honest, I still don't understand how your scheduler works with multiple cores.

I see you are using a postfix language, which I understand to be self-made.

[twilit smelt]

you may be looking at old mintia

which was written in dragonfruit

mintia2 is written in jackal (which is not postfix and is much more readable) and has SMP support

old mintia didnt have smp support

https://github.com/xrarch/mintia2/blob/main/OS/Executive/Ke/KeDispatch.jkl

thats the smp scheduler

[fleet root]

I don't usually use GitHub, so I must have wandered into the wrong depot.

[twilit smelt]

dragonfruit and jackal are both self-made, jackal's compiler is itself written in jackal (as are the assembler and linker) so its self hosting

and jackal is an all around much better language than df lol

[fleet root]

Interesting. The structure is very similar to NT, with many similarities, at least at first glance.

I'm not sure that the scheduler is that similar. But I see that you even have job objects that are hierarchical by default.

[twilit smelt]

i think NT is the most well structured kernel

[twilit smelt]

the scheduler is inspired by freebsd's ule scheduler

old mintia's scheduler was more like 90s NT's scheduler (but without smp support)

i thought ule was a simple way to get better interactivity characteristics and stuff

my implementation of it has some NT-ish characteristics though like the usage of irql (IPL) and the NextThread mechanism

[twilit smelt]

i dont agree with all the finer details in NT though

i think the object manager namespace is a little silly

it probably should have been a dirent cache into which you can insert objects as virtual files anywhere in the fs tree, and the underlying fs driver doesnt need to know anything about it since its represented by a virtual "cached dirent" above its head

with arbitrary mountpoints like unix

i think thats way more flexible and interesting so thats what im planning on doing

havent quite gotten that far though

[digital pivot]

i think it actually makes more sense to have hierarchical jobs, according to reactos implementation of jobs in windows they have some weird limitations like 1 job per process

[fleet root]

I have absolutely no idea how FreeBsd works. I got some of mine from NT as well.

[twilit smelt]

also my job objects arent rly meant to be like NT's jobs, mine are intended for session control, so the root job in a job tree represents a session and is inescapable and atomically killable (you can create sub-jobs freely in the same tree though)

if youve ever heard of DEC MICA its a little bit more like mica's jobs than NT's

also im glad to see you unblocked me lol

[fleet root]

Lukovsky's original implementation is not a masterpiece; it has long since been reworked into something more applicable.

So in Windows, jobs are hierarchical.

[fleet root]

I think it is a sufficiently robust and flexible solution to allow various tricks to be done on it, such as virtualizing name spaces.

[twilit smelt]

once you introduce a common cache for path lookups ("dirent cache" or "name cache") that sits above the fs drivers, it becomes really natural to dissolve the object namespace into it

thats my thinking at least

i did some partial implementation of this but i did it wayyy too early and im unlikely to end up using any of the code thats currently there for it

i think itll turn out really nifty once its done though

[fleet root]

In Windows file systems have a name cache and the cache manager is involved. It doesn't cache sectors but rather files including metadata files.

[digital pivot]

ah i didnt know

[twilit smelt]

by "name cache" i mean like, if you look up \a\b\c, itll create an in-memory representation of this hierarchy with a "cached directory entry" node for \a, for \a\b, and for \a\b\c (linked in a tree or some other lookup structure) and avoid doing the directory search again later

and this is a common component shared by multiple filesystems

windows has this now?

[fleet root]

To be on the safe side, I will tell you that this needs to be clarified.

[twilit smelt]

i understand the fastfat driver sample on github is quite aged but when i looked at it it seemed to have a bespoke solution for this

namely a splay tree linking fcbs together

that it used for path lookups

[fleet root]

Work is underway on NTFS/ReFS; Fastfat is a poor source in any case.

[twilit smelt]

that makes sense

i can see in-memory path lookup caching becoming a common component reused by ntfs and refs and they dont bother to update fastfat (or they do but the fastfat on github is just a decade old)

this one https://github.com/Microsoft/Windows-driver-samples/tree/main/filesys/fastfat

the official fastfat source release

[twilit smelt]

holy moly

[fleet root]

In fact, they are an important part, given today's requirements for operating systems. Instead of a couple of thousand lines of code, it now makes up a fairly big part of the process structure subsystem and is measured in tens of thousands.

[twilit smelt]

thats like a non trivial C program

[lucid umbra]

holy shit

[icy bridge]

if you remove all the boilerplate it's just like 4 printfs but yeah

[twilit smelt]

will it actually correctly generate a 64+64 bit add

i wanna see what it emits for that

[icy bridge]

I wonder what it'll emit for that too given I haven't added any add-with-carry patterns or anything like that

[twilit smelt]

itll emit Internal Compiler Error or segmentation fault

[fleet root]

To be honest, I can't say anything for sure about that. The last thing I knew about work in the cache management area was concentrated in a branch called CcPartition. It was related to a massive overhaul (partitioning) of the cache manager, and I no longer interacted with the file system people.

[icy bridge]

        .file   "<stdin>"
        .text
        .p2align        2
        .globl  foo
        .type   foo, @function
foo:
        add     t0,a0,a2
        slt     a2,t0,a0
        add     a1,a1,a3
        add     a3,t0,zero
        add     a2,a2,a1
        jalr    zero,lr,0
        .size   foo, .-foo
        .ident  "GCC: (GNU) 15.2.0"```

[twilit smelt]

holy frick

idk why thats so amazing to me

lmao

its funny it can figure that out on its own as well

[lucid umbra]

the power of industry compilers ig

[twilit smelt]

i mean i havent stared at that long enough to figure out if it actually works but i imagine it probably does

[twilit smelt]

i have a hunch there are things about the current fs driver architecture (dating back to 1989, no fault of their own) that stop them from doing path lookup caching as generally as linux/bsd/xnu are doing it

but id be fascinated to learn that theyve figured out a way

[queen torrent]

amazing work

how far do you think you are from starting work on getting linux to build?

[icy bridge]

feature wise I'm probably there already but the backend is still way too buggy for that to be realistic

[twilit smelt]

itll probably be a full day of work just to add enough empty stub functions to convince it to emit a linux elf binary that doesnt even kind of work lol

[icy bridge]

yeah

that too

[twilit smelt]

then youll need to figure out my evil paging

and stuff

i mean its not that evil its just a 2-level table (most of the code for 32-bit MIPS page table management will probably be broadly applicable) but its software refill and thats the part thats done a bit evilly and uniquely

and youll get nigh undebuggable hangs if you do it wrong

[icy bridge]

one annoying thing ive noticed is that im pretty sure literally every c atomic memory ordering aside from relaxed will need at least one full mb barrier

[twilit smelt]

i think release only needs wmb though

[dense vigil]

holy

[devout geyser]

based

[icy bridge]

wmb doesn't stop prior reads from being reordered past it though right

[twilit smelt]

it does, if the manual says otherwise i mis-specified it. its meant to be identical to the dec alpha memory model and you only need wmb for release ordering there

for example you will do wmb before releasing a spinlock

[twilit smelt]

and that would be broken if it didnt do this

[mortal thunder]

and then 5

[icy bridge]

ah okay then yeah wmb will work

[twilit smelt]

dont ask why the dec alpha guys decided you only need mb (huge barrier, acq_rel) and wmb (littler barrier, release)

but i stole it

so thats what ur lookin at

[fleet root]

I'll try to find out about that. On my end, I'm tinkering with the scheduler to make it as good as possible. Initially, I was looking at Linux, but I spoke to the person behind the NT scheduler and he told me that it doesn't make sense to make it fair share for the standard scheduling policy. I'll take a look at FreeBSD.

[twilit smelt]

that also means all the weird memory model stuff in linux specific to alpha will need to be that way on xr17032 as well

[icy bridge]

reason I thought it didn't affect loads was because this only says writes have completed and nothing else

[twilit smelt]

yeah i mis-specified it

[icy bridge]

and the only difference between it and mb is that mb does mention reads

[twilit smelt]

when i wrote the manual i had a worse comprehension of memory barriers than i do now

i should have also put

that reads arent reordered past it

to be fair the dec alpha specification of those instructions is also obscenely vague

so maybe they didnt really get it either

[twilit smelt]

basically this is because im terrible at practicing things before theyre relevant to my project and i wrote that manual while in the process of adding smp to my project

so i had literally no smp experience

[icy bridge]

wait so whats the difference between mb and wmb

[twilit smelt]

wmb: stores arent moved after it, loads arent moved before it
mb: nothing is moved in either direction

pretty sure!

[icy bridge]

ahhh

wait then wmb isn't enough for release because release needs to prevent loads moving after it

and doesnt care about anything moving before it

[twilit smelt]

im stupid
wmb: stores and loads arent moved after it.
mb: nothing is moved in either direction

its that.

[icy bridge]

okay yeah that works

thanks for the clarification

[dense vigil]

is there an rmb for acquire?

[twilit smelt]

no but i think the later dec alpha chips did have this

the original ones only had mb and wmb

[dense vigil]

so for acquire u must mb?

[twilit smelt]

probably because for whatever reason this was most natural for their planned uarch

[twilit smelt]

yeah

[dense vigil]

ah

[twilit smelt]

im saying "i think" too much for my liking so im going to go back and re-read the manuals

[digital pivot]

this is funny because linux is changing one fair share scheduler with another

[twilit smelt]

wtf

in the first alpha handbook

theres only mb and imb

memory barrier (acq_rel) and instruction memory barrier (icache flush)

was wmb a later addition??

this is a little cursed

dont tell me they added wmb later and that they added it at the same time as rmb. then ill have egg on my face.

im hoping this section of the manual was just not kept in sync with the instruction listing because i clearly remember there was both mb and wmb (and no rmb) there

[icy bridge]

first google result for "dec alpha manual" lists wmb but not rmb

https://download.majix.org/dec/alpha_arch_ref.pdf

[twilit smelt]

this one lists mb but not wmb or rmb

this is the 21064 manual which was the very first alpha architecture impl

i suppose wmb was a somewhat later addition

so i was right that at one point they had only mb and wmb

wonder what idiot made them have to add this to the manual

[icy bridge]

ah yes, a memory barrier, famous for increasing performance

[twilit smelt]

also i was right that the manual doesnt specify wmb preventing loads from being reordered after it

i have strong reason to believe that it actually would have and for some reason this went under their nose and they didnt write it down

its possible that on the actual alpha chips that existed there was no way for a load to be reordered after a wmb for implicit reasons they never articulated and this being an important part of its function went over their heads

[twilit smelt]

my reasoning for this is that every single lock impl ive ever seen for alpha only does a wmb for release ordering and those would all be broken if this were not the case

@icy bridge also dec alpha could violate causality and xr17032 probably wouldnt lol

it could seem to reorder dependent loads

this was due to a split dcache inside the cpu

the 21064 had a dual issue pipeline and each pipeline had its own dcache

I think it was the 21264, and I have this dim memory of it being due to a partitioned cache: even if the originating CPU did two writes in order (with a wmb in between), the reading CPU might end up having the first write delayed (because the cache partition that it went into was busy with other updates), and would read the second write first. If that second write was the address to the first one, it could then follow that pointer, and without a read barrier to synchronize the cache partitions, it could see the old stale value.

linus is wrong it was the 21064 not 21264

well it might have been both idk

[icy bridge]

that's fucked

[shadow ridge]

One example of this instruction on a uniprocessor system is to ensure that a device has seen a sequence of writes to its registers before asking it to perform a command
isnt this redundant since mmio regions are uncached

[twilit smelt]

no because it would still go through the on-chip write buffer even if it was uncached

xr17032 does have that detail as well

[shadow ridge]

does the emulator simulate it?

[twilit smelt]

the write buffer is actually simulated and i have run into bugs with not correctly wmb'ing to flush it

yes

[shadow ridge]

hmm

[twilit smelt]

nvm im wrong noncached accesses bypass the write buffer on xr17032

the issues i ran into were with other processors not with devices being accessed thru noncached areas

[shadow ridge]

ah i see

do you need WMB if you're doing LL/SC on a lock for example?

[twilit smelt]

you only need to do wmb before releasing it

to ensure that any changes you made in the critical section are committed

cuz itd be bad if you wrote that stuff after releasing the lock from other ppls perspective

then you basically dont have a lock!

and this could really happen due to the write buffer as well

[shadow ridge]

what if another processor acquired the lock and you try to do LL? does the write not being committed not matter since LL will see it's locked anyways?

[twilit smelt]

here are the scenarios

you have a sequence

.retry:
  LL reg, spinlock      // Yoink spinlock value.
  BNE reg, .spin        // If spinlock != 0, its held, so go spin.
  SC reg, spinlock, 1   // Store 1 to the spinlock to acquire it, if nobody
                        // has written to it since the LL.
  BNE reg, .retry       // If reg == 0, the store failed because somebody
                        // else wrote to the spinlock, so retry the sequence.
  MB

and then on two cpus you have these situations:

CPU0: LL  BNE  SC->SUCCEED
CPU1:   LL  BNE  SC->FAIL

CPU0: LL  BNE      SC->FAIL
CPU1:   LL  BNE  SC->SUCCEED

and so on

no matter how you reorder it only one of the SCs will succeed and so you have mutual exclusion

by nature of SC's operation, the hardware has to implement some kind of strict ordering wrt other SCs

one way this can be accomplished is to use the cache coherency protocol

even on systems with insane memory models like alpha, there is full dcache coherency

(numa notwithstanding)

if you have a MESI-like protocol, you can do something like

SC succeeds if the core still has the cache line in its L1 (from the previous LL)

otherwise, the line was taken from it by another cpu and so SC fails

because that means it may have been written to

which caused the line to be taken exclusive by the other cpu

which ripped it out of everybody else's dcache

i actually implement this in my emulator when cache simulation is enabled

[shadow ridge]

does SC make sure all the other CPUs actually see the write

CPU0: LL  BNE  SC->SUCCEED  MB
CPU1:                     LL  BNE  SC->SUCCEED

is this scenario possible

[twilit smelt]

it is not because the SC there takes the cache line exclusive in CPU0's L1

CPU1 def does not have the cache line anymore by the time LL occurs and so itll take the line shared, which requires a writeback of CPU0's exclusive line and for it to be downgraded shared in his L1

[shadow ridge]

ah i see

cache coherency to the rescue

[twilit smelt]

CPU1 reads the correct value from main memory after that (or picks it up directly from the bus when CPU0 does the writeback or whatever else)

thats how it works for me at least

other impls are possible

[shadow ridge]

do you have to MB after you release the spinlock

[twilit smelt]

you have to WMB before you release it

no barrier is necessary after releasing it

[shadow ridge]

isnt the cacheline implementation unoptimal if you have a bunch of spin locks in an array

and a bunch of threads are trying to acquire different ones that are in the same cache line

[twilit smelt]

thats why you dont do that

observe

[icy bridge]

this is terrible for perf on real cpus as well

for the same reason pretty much

[shadow ridge]

why not put the locks in an uncached region if you have a lot of them?

[twilit smelt]

cuz then its incredibly slow

to do an uncontended lock acquire/release

[twilit smelt]

one good thing about this is that uncontended lock acquire/release is literally as fast as any normal memory access

because you do it directly in your L1 and nobody else has to know, beyond the normal cache coherency traffic you already have

which is why some real world uarches implemented ll/sc this way in the 90s at least

mips r4000 for example

[shadow ridge]

for the first time i feel like i understand barriers and spinlocks

[twilit smelt]

forcing spinlocks to be uncached would be hideously slow especially in a system with fine grained spinlocks like mintia2 where im acquiring and releasing them in rapid succession all the time

taking all those penalties on going directly out to main memory each time

eugh

[twilit smelt]

notably, alpha did not implement it this way

because they decided their ll/sc would also be atomic wrt device dma

and dma bypasses the caches so they couldnt (only) use the caches for it

they do actually use ll/sc for atomic communication with devices all the time on alpha systems

so thats not just theoretical either

their ll/sc must work on noncached memory lol

so i think when ll/sc is used on cached memory, they do do the cache thing i talked about

where they use cache line presence in L1 to infer that the data has not been modified by anybody else

but if they use it on noncached memory, theres on-board logic

so, part of the motherboard chipset

which has to maintain a register containing the physical address corresponding to the last LL

and watch all addresses on the bus (so it snoops all bus activity on the system for this purpose)

if any bus addresses match the LL register, it clears it

the next SC goes directly out to the bus (because it is noncached)

the mobo logic sees that this is an SC (because there is special metadata associated with it for this purpose) and checks the SC's destination address against the LL address register (which was cleared)

and reports failure to the processor if it doesnt match (which it doesnt, because the LL address register was cleared, because the chipset saw a bus address that matched it earlier, indicating that another device doing DMA or another processor accessed it)

yeah that

[mortal thunder]

how long does LL last on your arch

like, for how long after an LL do SCs on that address fail

[raven drift]

not even x86 supports atomics on uncached memory

pretty much all modern architectures require cache coherency for synchronization

[dense vigil]

wouldnt be surprised if it just locked the entire bus for that like it does for split lock atomics

[raven drift]

ah actually you're right, it does that

unless uc-lock disable is set

it does not guarantee coherency on uncached memory with wc buffers though

[blissful smelt]

i860 lock instruction lore

[dense vigil]

ah cool

[blissful smelt]

devious

[twilit smelt]

until the cache line is written to by another processor or an rfe instruction is executed on the same processor (ie an exception was taken and returned from)

[twilit smelt]

theyre calling it the worst atomics of all time

i860 asked to go home

[raven drift]

ll/sc is effectively a cache lock as well on many archs

of course, one that is more limited in scope

but still

[twilit smelt]

lock/unlock on the i860 was actually like

it put a lock on the entire system bus

and as a result it only worked for atomic operations on noncached memory

so any like spinlocks or anything else you needed to do atomics with had to be mapped noncached

it was

horrible

i have no idea how they were even planning on doing smp NT OS/2 on that garbage architecture

im sure they were relieved when they just dropped it

[twilit smelt]

ill be able to finish these things up for monkuous this weekend probably

as long as i manage my time wisely

#1 is already done

the remaining items can probably be done by the end of tmrw

depends on how long work and homework end up taking me

[dense vigil]

@icy bridge any progress on your porting work?

[icy bridge]

havent done anything since my last update

[dense vigil]

[icy bridge]

shortly after i posted that i ran into an extremely strange issue while compiling libgcc that i couldn't figure out for the life of me so i decided to take a couple days break

i'll probably start working on it again tomorrow or monday

[dense vigil]

ah damn

[icy bridge]

i ended up thinking of a solution right as i wrote this so there go two hours of sleep, anyway when targeting xr17032-elf libgcc now compiles successfully

xr17032-unknown-linux-gnu hits "unwinding is not supported on this target" so that'll be my next goal ig

[dense vigil]

What was the issue

[icy bridge]

couldn't figure out the root cause but it was fixed by using the default r constraint instead of a (a custom constraint which allows the use of xr17032's inline shifts)

as far as i could tell this hit some kind of untested code path that fucked up instruction selection resulting in an illegal operand being passed to the selected instruction

[twilit smelt]

so gcc doesn't use my inline shifts :(

I didn't invent the inline shifts btw I stole them from 32 bit arm

So you might find precedent for selecting them there

[icy bridge]

ah alr i'll take a look at how it's handled there tomorrow

[dense vigil]

Damn

[icy bridge]

specifically it was collapsing a subi <temp reg>, zero, 1; subi <reg A>, <reg B>, <temp reg> into subi <reg A>, <reg B>, -1 even though the subsi3 instruction template uses a predicate that only allows registers, inline-shifted registers, and 16-bit unsigned immediates

[dense vigil]

But good job figuring it out

[pastel flume]

This guy is a genius debugger

[dense vigil]

Monkuous was able to debug gcc but not astral

Now imagine how fucked it is

[icy bridge]

to be fair figuring this issue out took a total of like 27 hours whereas with astral i only tried for like 2

i put in a lot more effort when it's my own project

[twilit smelt]

you know how hard it is to debug an smp kernel when you wrote the emulator and compiler and linker and they are also buggy

monkuous may be good but put some respect on my name in my own got dam thread

[dense vigil]

Holy shit

[pastel flume]

Yeah yall are something else

[dense vigil]

Like actual 27 hours?

[pastel flume]

I’ll stick to my shitty shite

[icy bridge]

yep

spread over multiple days ofc

but still

[dense vigil]

Damn

[pastel flume]

Crazy

When I patched qemu it took me several hours too

And I don’t even use my patched version anymore

[icy bridge]

honestly i think if i wanted to skip ahead a bit this gcc is ready to use for porting linux, the compiler itself is perfectly fine and the kernel doesn't use libgcc

it's not like libgcc fails to compile due to some bug in the compiler either, it's just that i haven't bothered to implement unwind info stuff yet

[dense vigil]

Yeah who cares about that for now

[pastel flume]

Never said you weren’t a genius for mintia lol

[pastel flume]

Debugging in general fucking sucks and I can’t even imagine that lol

At this point design your own cpu arch

[dense vigil]

Thats kinda what he did

[icy bridge]

might try to write a simple custom kernel first using GCC just as a sanity check that it doesn't have any miscompilations before I start porting linux

[pastel flume]

Well I quit then

[dense vigil]

Eh I'm not sure thats gonna be helpful since Linux has so much shit its bound to run into edge cases or code paths your small kernel won't touch but ig

[icy bridge]

yeah but at least the obvious language features

[dense vigil]

Porting the old hydrogen to it lol

Thatd be fun

Although that relies on HHDM right

But xr has so little memory u can still use the HHDM tbh

[icy bridge]

rn the most advanced thing I've actually verified the code is correct for is a simple hello world app with a basic printf

[pastel flume]

Wouldn’t it be trivial to make it not rely on it? Okay maybe not trivial but

[dense vigil]

Depends

[icy bridge]

would like to have more confidence than that

[dense vigil]

Yeah

[dense vigil]

Btw maybe you could just port some userspace utility and objcpy to mintia format rather than make a kernel?

[icy bridge]

I was thinking of just doing a single address space, no privilege separation, no paging thing

just a basic disk driver, vfs, and executable loader

[dense vigil]

Yeah ig

[icy bridge]

I have no idea how involved it is to add a new binary format to bfd but I'm pretty sure it's much more than adding a new cpu type

I might do the thing will suggested a while ago and add an elf loader to mintia1 though

[dense vigil]

No like, u can output a flat binary and then just add a header to it via like a python script for example

[icy bridge]

Ah yeah

Idk how viable that is but I could look into it

[dense vigil]

But an elf loader in mintia sounds more reasonable

Although that requires dragnofruit code

If you're into that

[icy bridge]

I've spent the last few weeks reading and writing GNU style code, I'm sure it can't be that bad

[dense vigil]

Lol

[pastel flume]

Yeah dragonfruit will definitely be more bearable than that

[twilit smelt]

its not rly that simple

you have to dynamic link it with the system lib and stuff

[dense vigil]

Dragonfruit it is

[twilit smelt]

https://fxtwitter.com/i/status/1961392372022985106

Claire Penis (@ZeroSuitCamus)

The Past was insane, they were showing this shit to three year olds

**💬 40 🔁 83 ❤️ 1.7K 👁️ 97.5K **

▶ Play video

FxTwitter · Aug 29, 2025, 11:35 AM

monkuous after 3 seconds dragonfruit

@icy bridge another alternative is just to write a mintia command that takes a flat binary linked to a specific location as an argument and then it just maps it there and jumps to the beginning

thatd be rly simple

maybe could pass it a table of function pointers to abi thunks you write in assembly so it can do some syscalls

(which youd need to do anyway even if you did an elf loader)

the old mintia (dragonfruit) abi is incompatible with the jackal abi which iiuc is what your compiler target semi-aims for

so youll need to do some register juggling before calling the osdll.dll entrypoints for the syscalls

[icy bridge]

@twilit smelt incorrect pseudocode here, should be Reg[RB] instead of Reg[RA]

[twilit smelt]

Didn't you point that one out already

[icy bridge]

possibly

i forgot which ones i pointed out

but considering my binutils was using the wrong one i probably didn't do that one

[icy bridge]

architecturally speaking is it legal to have multiple tb entries with the same (virpn,asn)?

[twilit smelt]

Why

The answer is no btw

But what's making you think this might come up

[icy bridge]

boot protocol design

basically the boot protocol for this test kernel hands over control to the kernel image with a wired itb entry for the page containing the entrypoint and no other guarantees (undefined eb and remaining itb wired entries have undefined contents)

and i was wondering whether i'd have to make the entrypoint page itb index a constant or also undefined

[twilit smelt]

That's interesting

mintia2 does it in a diff way

Your way should work too though

[icy bridge]

if it's undefined, then for the os to remove the other wired itb entries it's easier if it's able to have multiple entries for the same (virpn,asn)

but it's no big deal i can just say "the entrypoint page itb index is 0"

and then removing any other entries is just addi t0, zero, 1 mtcr itbindex, t0 mtcr itbpte, zero mtcr itbpte, zero mtcr itbpte, zero

[twilit smelt]

Does your bootloader read a filesystem

[icy bridge]

the shitty temp one no (the kernel image is embedded within the bootloader) but the final version (which'll use the same protocol) will

[twilit smelt]

Why not port limine

[icy bridge]

two reasons

• limine is really mainly a uefi bootloader with ifdefs sprinkled all over the place to support bios, it'd be very annoying to port to non-bios non-uefi platforms
• the limine boot protocol is only really suitable for 64 bit platforms

[pastel flume]

The limine protocol supports 32 bit i686

[icy bridge]

the protocol rules i've formulated so far:

• RS: M=1, I=0, U=0, all other bits undefined
• ITB: entry 0 maps the page containing the kernel image's entry point (with the same pte as in the passed page tables), all other entries undefined
• a0: physical address of a page directory which maps all phdrs in the kernel image (using V/W taken from phdr flags, K=1, N=0, G=1, avail=0) and nothing else
• a1: physical address of boot info
  everything not mentioned is undefined

[twilit smelt]

Just write a UEFI wrapper around a4x and port limine to that

Shrimple

[icy bridge]

it does not

the bootloader itself does but the protocol does not

the protocol only support x86_64, aarch64, riscv64, and loongarch64

[pastel flume]

Ohhh yeah

I forgot that lol

[acoustic sparrow]

why do you make everything else undefined?

like, just guarantee zeros there?

[shadow ridge]

if you guarantee zeros then it's harder to add / change stuff

better to just say it's reserved and shouldnt be used than say it's undefined

[acoustic sparrow]

you could also add a real extension mechanism

lol

or require the loader to set zeros, and say it is UB for the kernel to rely on it being zeros

[shadow ridge]

true

[icy bridge]

update on linux port: the bootloader's fdt generation code works! (although i do still have to finish the fw-to-dt translation code, so the generated dtb is incomplete) ```
$ tail -c +3737 bank7.bin | dtc -I dtb -O dts -o /dev/stdout -
/dts-v1/;

/ {
#address-cells = <0x01>;
#size-cells = <0x01>;
compatible = "xrcomputer";
model = "xrstation";

    memory@0 {
            device_type = "memory";
            reg = <0x00 0x800000>;
    };

    memory@2000000 {
            device_type = "memory";
            reg = <0x2000000 0x800000>;
    };

    memory@4000000 {
            device_type = "memory";
            reg = <0x4000000 0x800000>;
    };

    memory@6000000 {
            device_type = "memory";
            reg = <0x6000000 0x800000>;
    };

    memory@8000000 {
            device_type = "memory";
            reg = <0x8000000 0x800000>;
    };

    memory@a000000 {
            device_type = "memory";
            reg = <0xa000000 0x800000>;
    };

    memory@c000000 {
            device_type = "memory";
            reg = <0xc000000 0x800000>;
    };

    memory@e000000 {
            device_type = "memory";
            reg = <0xe000000 0x800000>;
    };

};```

[icy bridge]

also worth mentioning that i haven't run into a single compiler bug while making the bootloader so far so it's more stable than I thought

[dense vigil]

u mean gcc?

[icy bridge]

yeah

[dense vigil]

That's cool

[twilit smelt]

very cool

[acoustic sparrow]

nice!

[acoustic sparrow]

btw "model" should probably be the stylized "XR/station"

for example t8103-j293.dtb has a model of Apple MacBook Pro (13-inch, M1, 2020 not, uh, t8103

[icy bridge]

ah ok

I'll change that then

[acoustic sparrow]

also you should set the compatible to something like xrarch,xrstation

[icy bridge]

well it's the same platform for each model of xrcomputer which is why I made it that but I'll add the xrarch part

[acoustic sparrow]

the xrarch part is the vendor name :^)

but yeah

[twilit smelt]

its cool to see the xrstation memory map represented as a dtb

[mortal thunder]

very cool, let me know when you start actually porting linux

[icy bridge]

finished fw-to-dt translation https://hst.sh/ofemetugih.properties

wasn't sure of the best way to represent xrcomputer's interrupt model so i just collapsed all the lsics into a single device

[dense vigil]

What is fw to dt?

[icy bridge]

translation of the device info provided by firmware to a device tree consumable by linux

[dense vigil]

oh the xr firmware has its own format?

[icy bridge]

yeah https://github.com/monkuous/xrlinux/blob/344e0cedd680273283d22e89e2314059f8b3b97f/bootloader/a4x/a4x.h#L6-L82

[mortal thunder]

how compatible is your C compiler with the jackal abi

[icy bridge]

almost fully

the sole exception is jackal functions that have more than 4 return values (void functions with >4 out parameters or non-void with >3 out parameters)

[mortal thunder]

how are the other 3 OUT values returned

or are they ignored

[icy bridge]

in this case jackal passes the first 4 through registers and the remaining by value on the stack (without an implicit pointer parameter)

C does not have this concept and the closest to it is structs with >4 words

so i decided to bite the bullet on compatibility and just return the whole struct by reference

structs <=4 words are returned in registers by value

[mortal thunder]

An interesting idea

[icy bridge]

so if you had a jackal function FN TestFunction(OUT a: UWORD, OUT b: UWORD, OUT c: UWORD, OUT d: UWORD) you'd declare it in C as c struct { size_t a; size_t b; size_t c; size_t d; } TestFunction(void); and everything would work fine

but FN TestFunction(OUT a: UWORD, OUT b: UWORD, OUT c: UWORD, OUT d: UWORD): UWORD or FN TestFunction(OUT a: UWORD, OUT b: UWORD, OUT c: UWORD, OUT d: UWORD, e: UWORD) are not callable from C

[mortal thunder]

https://github.com/monkuous/xrlinux/blob/main/patches/gcc/jinx-working-patch.patch how is this diff only 2.6k lines

[mortal thunder]

i see

[icy bridge]

i decided on this compromise because maintaining compatibility with that would be needlessly expensive and useless considering there is no known jackal code in existence that uses more than 4 return values

[mortal thunder]

is it really that simple to port gcc to a new architecture

ah theres also binutils and it's 2.9k loc in that case

[icy bridge]

it's also not fully complete

it's missing most atomic ops (only barriers, load, store, and cas are implemented), exception handling (blocker for libgcc), TLS, and some other stuff i can't remember off the top of my head

binutils is also incomplete (attempting to create an ET_DYNAMIC crashes the linker, and the assembler does not do any overflow checking)

[icy bridge]

@twilit smelt in the instruction summary these instructions have the wrong register order (should be ra, rc, rb - listed correctly in the detailed description)

also: do these instructions mask the shift amount with 31, or is that emulator behavior just because my host does so (gcc wants to know)

[icy bridge]

update on the bootloader, i have implemented partition handling (MBR), filesystem reading (ext2), and configuration loading

[dense vigil]

how are you so fast

[twilit smelt]

they mask with 31, adding this to the list of stuff ill revise in the manual

(the list is searching from:monkuous mentions:hyenasky in this discord)

[icy bridge]

alr

worth noting that the emulator rn doesn't explicitly do this and instead relies on the host system doing so

[twilit smelt]

just barely fixed that

none of the guest code relies on this because i was aware of it being a portability hazard and so whenever i do shift by a variable in jackal code, i make sure (in some way or another) that the value in the variable is never >31

[icy bridge]

yeah same, I just noticed that GCC's generated code relies on it sometimes even if shift count <=31

for example when right shifting a 64 bit integer one of the things it does is a right shift by ~count

and if you tell it that shift counts aren't truncated it needs to add a mask manually

[twilit smelt]

some of the guest code does rely on ROR masking with 31 but literally every ROR under the sun does that that i could find

so that doesnt seem like a portability issue

[icy bridge]

update: the bootloader can now load and execute kernel images (https://github.com/monkuous/xrlinux/blob/main/bootloader/PROTOCOL.md)

so i am now able to start porting linux

[twilit smelt]

how come you dont jump to it with paging on

peijing

beijing on

[icy bridge]

decided it'd be nice to give the kernel freedom to set that up however it wants

also this means i don't need to pass a memory map separate from the dtb (because there are only two contiguous regions the kernel needs to be careful not to overwrite, instead of random pages all over the address space)

[pastel flume]

https://tenor.com/view/social-credit-credit-social-уважение-плюс-уважение-gif-1626328442317885176

[twilit smelt]

so the kernel has to relocate itself to the higher half

[shadow ridge]

missed opportunity to call it limnux

[icy bridge]

not really relocation imo since it doesn't involve modifying the kernel image but it has to set up its own page tables yes

[twilit smelt]

mintia2 bootloader loads it in the higher half to begin with after identity mapping itself and enabling paging

[pastel flume]

Ligmanix?

[shadow ridge]

idk what ligmanix is

[twilit smelt]

requires some weird stuff like disabling paging real quick while doing firmware api calls but works

[pastel flume]

https://tenor.com/view/trollface-gif-8242347154758580084

[twilit smelt]

also how come loading the kernel image is done on another processor(?)

[icy bridge]

it's not

[twilit smelt]

sorry im retarded and misread that

[icy bridge]

it's executed on every processor so that i don't have to have any bootloader-reclaimable regions

this is the relevant code in the bootloader

[twilit smelt]

did you do the funny apt+mbr thing

[icy bridge]

i wrote a simple tool to embed an apt with a single partition that just consists of the bootloader image into an mbr image so if that's what you mean yes

the bootloader itself only understands MBR and ext2

[twilit smelt]

thats cool

[icy bridge]

https://github.com/monkuous/xrlinux/blob/main/install-apt/install-apt.c

[twilit smelt]

i wonder if linux will be able to be ported without any architectural revisions

[icy bridge]

i don't see anything preventing it but i'm not the biggest expert on linux internals either

[twilit smelt]

you will be soon!

good luck!

[icy bridge]

ty

also i'm very satisfied that i was able to write an entire bootloader while only encountering 1 compiler bug (varargs handling was broken in situations where a 2-word argument was passed on the stack in a position where a one-word argument would have been put in a3)

[twilit smelt]

this is very good for my project as well because now i can picture a ridiculous hypepost called like "He Made A Computer That Runs Three Custom Operating Systems... And Linux?"

[icy bridge]

i also wonder what linux's minimum memory requirement will be on xrarch

the bootloader needs 560k memory (mostly because i have a statically declared 512k disk cache)

[twilit smelt]

probably way more than you need btw

[icy bridge]

yeah ik

[twilit smelt]

excluding sectors that are only used once (so, the vast majority, such as sectors in the kernel image which are only read once when loading the kernel)

i get a like 90% hit rate for a literal 8 sector cache

in mintia2 bootloader lol

[dense vigil]

Linux does it itself on every architecture

[twilit smelt]

these bypass it completely

[twilit smelt]

inchresting

[icy bridge]

btw where do you do disk caching in the mintia2 bootloader

[twilit smelt]

the aisixfs driver does it itself

actually i think its split in half 4 sectors for metadata and 4 sectors for data

cached

[icy bridge]

i'm pretty sure my disk cache design is Very Weird (it basically mirrors cpu caches where there's a set number of sectors per block and the block number is masked to select a cache line, which is then 2-way associative) so i wanted to take a look at a more normal design

[dense vigil]

Wtf is an apt btw

[twilit smelt]

i mean its not "normal" its extremely simplified

[warm mural]

Why do you need a cache? Is this instead of loading the whole thing in memory?

[icy bridge]

#define BI_BCACHE_SHIFT 12
#define BI_BCACHE_LINES 64
#define BI_BCACHE_SETS 2

#define BI_BCACHE_SIZE (1u << BI_BCACHE_SHIFT)
#define BI_BCACHE_MASK (BI_BCACHE_SIZE - 1)

_Static_assert(BI_BCACHE_SHIFT >= BL_SECTOR_SHIFT, "BI_BCACHE_SHIFT must be larger than BL_SECTOR_SHIFT");
_Static_assert((BI_BCACHE_LINES & (BI_BCACHE_LINES - 1)) == 0, "BI_BCACHE_LINES must be a power of two");

_Alignas(BL_BCACHE_ALIGN) static unsigned char BiBufferCache[BI_BCACHE_LINES][BI_BCACHE_SETS][BI_BCACHE_SIZE];
static uint64_t BiBufferCacheCurrent[BI_BCACHE_LINES][BI_BCACHE_SETS];

static void *BiGetBcacheEntry(uint64_t block) {
    uint64_t id = block + 1;
    size_t line = block & (BI_BCACHE_LINES - 1);
    size_t set = 0;

    for (size_t i = 0; i < BI_BCACHE_SETS; i++) {
        uint64_t currentInSet = BiBufferCacheCurrent[line][i];

        if (currentInSet == id) return BiBufferCache[line][i];
        if (currentInSet == 0 && set == 0) set = i;
    }

    if (!BxReadFromDisk(
            BiBufferCache[line][set],
            block << (BI_BCACHE_SHIFT - BL_SECTOR_SHIFT),
            1ull << (BI_BCACHE_SHIFT - BL_SECTOR_SHIFT)
        )) {
        BlCrash("failed to read from disk");
    }

    BiBufferCacheCurrent[line][set] = id;
    return BiBufferCache[line][set];
}```

[twilit smelt]

to cache like directory sectors and stuff

and fat sectors

and inode table sectors

[warm mural]

yea but u just load the kernel once

[twilit smelt]

i also load other images beside the kernel

[warm mural]

Ah yea right

[twilit smelt]

and will do more complicated stuff later as well like loading driver configuration files

for the iokit-like "plug n play" autoconfig

[warm mural]

Do XML plists

[icy bridge]

also without a bootloader disk cache the filesystem traversal stuff can vastly overshadow the actual image loading in terms of number of read sectors

even in simple bootloaders that only load a config file and a single image

[warm mural]

ah interesting

[icy bridge]

the exact ratio depends on the fs type and fs driver design ofc

but yeah a disk cache is very nice to have

[dense vigil]

Is apt a mintia thing?

[twilit smelt]

its the native partition table of xrstation

its designed to be able to coexist in sector 0 with MBR

which is how monkuous is using it

[dense vigil]

Why not just mbr?

[twilit smelt]

nih

[dense vigil]

Lol

[twilit smelt]

it dates to 2018

when i was 15 and just started

[dense vigil]

Should've invented PTG as a GPT competitor

[twilit smelt]

so its one of the oldest parts of the project now

i remember sketching it in a notebook in the cafeteria before class at like 7am in sophomore year

seems like AGES ago now

[dense vigil]

Does your version of mbr also expect executable code there?

[twilit smelt]

no

[dense vigil]

Lol

This sounds close to how el torito was invented

[twilit smelt]

except my thing is irrelevant and stupid and el torito is cool and ubiquitous

other than that its close

[dense vigil]

Well yeah it is smart I guess

[twilit smelt]

well i guess CDs are gone now

[icy bridge]

oh about apt, i noticed that the documentation seems to be missing how the start of a partition is determined (i had to read a4x code to figure it out: first used partition is at sector 4, and every other used partition immediately follows the previous)

[twilit smelt]

woops

[dense vigil]

yeah but .isos are still ubiquitous

[twilit smelt]

i experienced my first elder moment recently

when

i was watching an ltt video where some of their younger interns are trying a windows xp computer

and theyre given a cd with music on it and are told to play it

and theyre like

"woah ive seen these in like old movies"

[shadow ridge]

WHAT

[twilit smelt]

and i was like shut the fuck up they were told to say that. CDs are not THAT out of the picture.

i look it up

some of them were like, 5 when CDs basically vanished

so yeah

its real

[dense vigil]

lol

[shadow ridge]

dude i have CDs on my desk rn

[twilit smelt]

youre not a normal zoomer

most 18 year olds have never used a CD in their life and now i think of it i havent had to use one since i was a tween myself

[dense vigil]

i mean, ps5/xbox still have cd drives

[twilit smelt]

i had never noticed they disappeared

[dense vigil]

well not cd but

i call bs

[shadow ridge]

ok true i have like 1 walkman (the other one isnt a walkman just remembered) and a lot of cassettes LOL

[icy bridge]

i've never used an actual CD but i do have a bunch of DVDs

[shadow ridge]

ok i just found a mini DVD

still in its og package

says 30 min video, 1.4 gb data

lol

i have a cd rom of a knight rider game (i should also have a cd for a really old train sim game i have no idea where i put this shit)

• the install disks for gta 5 and fallout 4 i think

ok those arent that old

so ltt's kids are just not gaming that much

anyways idk i feel like it's not that abnormal to still have CDs or at least know how to use one

[queen torrent]

Or just being told to say that to generate engagement

[shadow ridge]

we could make a viral video where someone sees HDDs and theyre like "wooaahh ive never seen these before, only in movies!!"

[twilit smelt]

their audience are young enough that nobody was even commenting on that

the commenters were all treating it as a given that someone <=18 yrs old would be unfamiliar with CDs

[shadow ridge]

but how tho

you cant tell me those people never bought a physical game

[twilit smelt]

because theyre gone

[shadow ridge]

and installed it from the cds

[twilit smelt]

they probably havent

18 is young enough now that their earliest memories (at least in america) are of downloading games digitally and CDs would be something theyd see collecting dust in their parents closet or somewhere

[shadow ridge]

cmon are you telling me they never installed gta 5 from cd

that thing has like 7 disks

or 9 i dont really remember

ok i just realized gta 5 is 18+ and most people in america probably actually give a shit about the ratings

or not idk

are you even doing your job as a parent if you didnt buy gta 5 for your 7 year old

[dense vigil]

i mean gta is not the only game with a physical release

[shadow ridge]

ik but it's popular enough where most 18 year olds shouldve at least played it

i cant think about any game that was as popular for such a long time in the 2010s idk

maybe like the last of us or some shit

[warm mural]

Idk I'm 18 and have used CDs plenty

[twilit smelt]

for?

[warm mural]

Maybe it's just because my parents didn't buy stuff digitally

[warm mural]

Games, music

[twilit smelt]

music cds are huuuuugely outdated

[warm mural]

Nah not that much 😭

[twilit smelt]

im 23 and i barely saw music cds being used except by my mom when i was little

they were already going out

because of the rise of the ipod and mp3s

[shadow ridge]

ok even my family didnt use music cds we had a master music USB

[shadow ridge]

with a shit ton of mp3s

i wonder if that thing is still around

[warm mural]

Oh yeah that's true, but when I use CDs for music is when there's no hardware supporting those

[shadow ridge]

is there even a car without a radio that can play from USBs today

[warm mural]

what

are there even cars that can play from usb

What you usually do is you buy a Bluetooth dongle

[shadow ridge]

uuh dont those connect into a usb slot

lol

[warm mural]

No

In the cigarette lighter

[twilit smelt]

do cars still have cigarette lighters

[shadow ridge]

yea

i saw one in a shitty rental car in 2023

[warm mural]

Yea but it's mostly just a standard plug nowadays

Instead of an actual cigarette lighter

[shadow ridge]

@twilit smelt you werent around for floppies right

like even i never saw a floppy in person

tho we had them when i was a kid supposedly

but gave / threw it away since it had 0 use

[twilit smelt]

in theory they still existed when i was a very young child in the early '00s

PCs were still being sold with floppy drives

but i never saw one in "current use" for any purpose

i saw many many many ancient ones from the 80s and 90s in the basement

from my dad's hoard

[warm mural]

I only saw one once

[shadow ridge]

also i wonder if technological literacy went down since computers became much much more convenient to use

[warm mural]

Kinda crazy that something that was so ubiquitous at one point is now a rare sight

[warm mural]

Yes

People don't know how to use computers now

[shadow ridge]

i saw a guy who didnt know how to drag files into folders in the godot dc server

it took him like 10 minutes to figure it out

[warm mural]

There's a blog post from a tech support guy in schools

Eh now I can't find it

But it basically said that younger students didn't know shit about computers

Or technology in general

http://www.coding2learn.org/blog/2013/07/29/kids-cant-use-computers/

[shadow ridge]

that was 2013??

jesus

[warm mural]

You can imagine it's getting worse

Or maybe not as more schools are adopting per-student chromebooks

[shadow ridge]

They click 'OK' in dialogue boxes without reading the message. They choose passwords like qwerty1234. They shut-down by holding in the power button until the monitor goes black. They'll leave themselves logged in on a computer and walk out of the room. If a program is unresponsive, they'll click the same button repeatedly until it crashes altogether.

bro only met the 50% of humans with less than average intelligence in his entire life

like come on, you mustve heard at least once in your life how to make a good password

[warm mural]

No he's right in my experience

[pastel flume]

I STILL USE THEM WHAT

[pastel flume]

Mine does

It’s the best feature™

[twilit smelt]

i860 IN MY COLLEGE HOMEWORK

THIS IS NOT A DRILL

[shadow ridge]

why do they have so weird names

like why is it called "PAGING WITH TLBs"

[twilit smelt]

why did my professor use the i860 of all things as his example

it has nothing to do with the i860 he just needed a die shot and he dug one up of the i860 from the chm's website and used that

what history does this man have with the i860

[teal trench]

i'd guess all the paging logic + the tlbs are in that area?

[earnest zenith]

ask him

[earnest zenith]

I'd guess that a huge part of that area is the TLB

[twilit smelt]

in america youre not allowed to speak to your professor directly. theres an intermediary called the professor whisperer who has to relay all communication. if you talk to the prof directly he has the right to kill you with a hammer

[shadow ridge]

thats obvious

it's just worded weirdly idk

[shadow ridge]

wtf???

where i live, they use serrated blades, a hammer is just too violent

[twilit smelt]

its to make sure your brain is too destroyed for you to ever be resurrected

which is technology we have in america

[shadow ridge]

it would be a really cool distopian concept if you could store consciousness inside magnetic tape

and the gov used that to make templates of loyal employees

like, you need police officers? you just hook some guys up to a brain / tape interface, insert the police officer tape and press play

[twilit smelt]

that's just the premise of Severance with extra steps

[dense vigil]

that memory switching thing really resonated with me because its so close to how context switching works lol

especially with mmu systems

[shadow ridge]

never watched that

[dense vigil]

You should

[icy bridge]

Update: Linux compiles!

Now my terminal's scroll buffer is filled with unresolved symbol errors though

[queen torrent]

hell yeah

you should write a blog post or something about this once this is all done

[icy bridge]

Hell no I hate writing

[queen torrent]

damn

[twilit smelt]

I was considering writing about it as a guest blog post to the virtuallyfun blog

Ik the guy who runs it

[icy bridge]

the patch is already 1.9k lines and it's just headers lol

[twilit smelt]

Shoot over that Linux binary i want to admire it lmao

[icy bridge]

amazingly enough i only encountered 2 gcc crashes (both of which were only a few lines of changes to fix)

well it doesn't link yet

[twilit smelt]

an

ah

Right

[icy bridge]

note the position of the scroll slider

[twilit smelt]

Oh god

[icy bridge]

a lot of these are duplicates though

also apparently xr17032's instruction set is limited enough that gcc feels the need to put 64-bit multiplication in libgcc

so i guess i'll have to implement that builtin myself

[icy bridge]

piped it to a file, it's 855 lines of linker errors

[queen torrent]

damn you've got a lot of stubs to make

[icy bridge]

actually turns out it's only 38 unique symbols

huh

arch_ptrace
__ashldi3
__asm_copy_from_user
__asm_copy_to_user
calibrate_delay
__clear_user
__const_udelay
copy_thread
cpuinfo_op
empty_zero_page
flush_thread
flush_tlb_kernel_range
flush_tlb_mm
flush_tlb_mm_range
flush_tlb_page
flush_tlb_range
__get_wchan
init_IRQ
kernel_map
__lshrdi3
machine_halt
machine_power_off
machine_restart
__muldi3
__one_cmpldi2
ptrace_disable
setup_arch
show_interrupts
show_regs
show_stack
start_thread
swapper_pg_dir
__sw_hweight32
__sw_hweight8
switch_mm
__switch_to
time_init
vm_get_page_prot

this probably has something to do with linker garbage colllection

anyway i'm gonna stop here and probably not make any progress for a while because silksong is coming out in less than an hour

[dense vigil]

What is that?

[icy bridge]

the sequel to hollow knight

[queen torrent]

I still gotta finish hollow knight, started playing it last week

[shadow ridge]

rain world better

[queen torrent]

I love rainworld

[shadow ridge]

i still havent finished the watcher

the invisibility thing is too OP even with the weird things that spawn and chase you if you're in there for too long

love it

[twilit smelt]

can you stub those out and get it to link so i can admire a xr17032 linux kernel

[icy bridge]

sure, have to wait for steam's downtime to end anyway

[twilit smelt]

im surprised theres only those

i think even a new mintia port has more unresolved symbols than that at the start lol

[icy bridge]

i'm guessing that there's way more but most of it has been removed by linker gc

[icy bridge]

I guess I was incorrect because I have a binary now

@twilit smelt

wonder how much more I'd have to add for CONFIG_SMP=y

[twilit smelt]

thats wild

[mortal thunder]

wow

amazing

but where does it break

[icy bridge]

well for one it doesn't have an entrypoint yet

and it's not yet in the format the bootloader expects

so the next thing to do is head.S

[twilit smelt]

Insane to see recognizable xr17032 machine code in a Linux kernel

Surreal

[icy bridge]

btw does your sdk have a disassembler of some kind? because i couldn't find one and therefore did not have any before porting binutils

[twilit smelt]

It does not

[icy bridge]

damn

well for your enjoyment here's what part of it looks like in a disassembler

[twilit smelt]

Cwazy

I like the sign extension on the byte load

I bet that's happening allllllll over

[icy bridge]

no that's a zero extension

[twilit smelt]

True

That's RSH not ASH

Just noticed

[icy bridge]

i haven't bothered to figure out how to tell gcc loads zero extend yet

[twilit smelt]

Won't that be buggy

It's not sign extending stuff that needs to be

[icy bridge]

documentation suggests it's as simple as #define LOAD_EXTEND_OP(MODE) ZERO_EXTEND but i'm using that and it doesn't have any effect

[icy bridge]

seems like if you don't explicitly tell it what's happening it assumes the upper bits are garbage

[twilit smelt]

So it's sign extending or zero extending everything

[icy bridge]

yeah

[twilit smelt]

Sounds inefficient

[icy bridge]

indeed

[icy bridge]

$ echo 'int x(signed char *x){return *x;}' | host-pkgs/gcc-bootstrap/usr/local/bin/xr17032-unknown-linux-gnu-gcc -x c -S -o /dev/stdout - -O3
        .file   "<stdin>"
        .text
        .p2align        2
        .globl  x
        .type   x, @function
x:
        mov     a3,byte [a0]
        add     a3,zero,a3 LSH 24
        add     a3,zero,a3 ASH 24
        jalr    zero,lr,0
        .size   x, .-x
        .ident  "GCC: (GNU) 15.2.0"```

but right now i'm in the "get it working" phase

it's redundant but it works

so i'm leaving it alone for now unless i figure out some extremely simple solution

[icy bridge]

found one $ echo 'unsigned x(unsigned char *x){return *x;}' | host-pkgs/gcc-bootstrap/usr/local/bin/xr17032-unknown-linux-gnu-gcc -x c -S -o /dev/stdout - -O3 .file "<stdin>" .text .p2align 2 .globl x .type x, @function x: mov a3,byte [a0] jalr zero,lr,0 .size x, .-x .ident "GCC: (GNU) 15.2.0"

code size reduction due to this:
before: .text 2.61 MiB, all segments 3.68 MiB
after: .text 2.56 MiB, all segments 3.63 MiB

larger savings than i expected

[icy bridge]

first boot of linux! (the entrypoint is just a halt loop for now so not much to see)

silksong time now

whenever i resume working on this i will try to get it to the point where i can see earlyprintk output

[icy bridge]

nevermind steam payment processing is apparently still down

guess i will work on earlyprintk now then

[queen torrent]

My bf has been trying to buy silksong since it launched and still hasnt been able to

[queen torrent]

I was able to buy it so maybe try again now

[icy bridge]

got it!

@twilit smelt on the nmi masking thing, are the "cycles" mentioned architecturally the same as # of instructions or would a theoretical different xr17032 model be able to execute a lower number of instructions during the masking window

[twilit smelt]

Why'd this come up

[icy bridge]

I want to be able to guarantee I don't get any NMIs between the bootloader starting the bl-to-kernel transition and the kernel's write to eb

So I was thinking of adding something like "the last instruction performed before the jump to the kernel initiated an NMI masking window" to the boot protocol

But I wanted to know first if that actually guarantees anything of value

Linux's code also needs to know whether it's possible to access user memory in an NMI handler which would need a guarantee that the architectural part of the mm switch code cannot be interrupted by an NMI

[heady bobcat]

oh god why does Linux want to access userspace memory within NMI context

[icy bridge]

Beats me

[twilit smelt]

The only usage of the nmi currently is a "programmers key" type thing, mapped to ctrl-alt in the emulator, and the masking is just a "best effort" attempt to make that not just hang if you do it at the wrong moment lol

er

alt tab

[icy bridge]

Ah so in other words no architectural guarantees?

[twilit smelt]

not ctrl alt

[twilit smelt]

That basically just says "your small sequence where you're doing a context switch or switching stacks on exception or something will probably not be broken by receiving an nmi just then"

[icy bridge]

Okay good to know

[twilit smelt]

On an OoO processor there'd probably be a different guarantee than a number of cycles

but xr17032 is in order so you can use that to guesstimate how many instructions you can sneak in

[icy bridge]

Iirc riscvs approach to this is "pretend NMIs don't exist if you're not firmware"