MINTIA (not vibecoded)

[shadow ridge]

im talking about the bootloader

like hyper

or were you talking about like infy adding support to mintia's bootloader

nvm you were

i thought you were talking about adding the support to hyper

i thought it was gonna boot mintia2 or smth

😭

[twilit smelt]

ok u just have been annoying about the fact I haven't been working on mintia as much like 15 times so I thought it was another one of those

[shadow ridge]

no

it would be cool if hyper could boot mintia though

[dense vigil]

nah hyena chose limine for the 64-bit x86 target i think

[mortal thunder]

for fun i looked at the mintia2 fireworks test again and man

it looks so different compared to the original version i contributed

aside from the code style

MmFreeToPoolCache?

a bunch of NUMA stuff?

[tiny swallow]

i told about mintia in our cs club

ankara impressed

[twilit smelt]

i havent even talked about mintia in my own cs club

[tiny swallow]

i said a man in utah is doing cool stuff

they were glazing terry davis

good luck on your numa implementation

[twilit smelt]

all the hard (structural) parts are done

[mortal thunder]

hooray!

[twilit smelt]

i stopped using twitter because 99% of my posts were being critical of people which i thought wasnt a good look to potential employers but i realized that nobody i would want to work for would find me on modern twitter so ive decided to let loose

now i get to post things like this that get like 3 views and 0 likes and it makes me feel GOOD

[twilit smelt]

thank you for telling them about my work im flattered

[mortal thunder]

i stopped using twitter because it's full of right wing propaganda i decided i wasn't going to get ragebaited by

although i do miss the publicity i could get for my projects

it's been a full year break at this point of not logging into my account

[shadow ridge]

i have never used twitter and feel happy about it

[twilit smelt]

did i win

[shadow ridge]

HAHAHAHA

no way he said most people believe theyre better than average

then instantly followed it up

saying he thinks he's the only smart person in a pool of midwits

[twilit ingot]

XD

[raven drift]

tbh i do think that this argument has some merit

There is lots of tech debt that could be cleaned up but isn't because of a lack of resources

And if it's something that just requires mostly mechanical changes, that's a good use case for AI

[twilit smelt]

i wasnt even really critiquing that argument i just compulsively dunk on stemlords on twitter

to my audience of 3 people

and i did think that tweet was satire at first because of the irony between the second and fourth text blocks

i was like wow is this guy based and satirizing stemlord posting before realizing no hes just a caricature of himself

especially because the fourth text block kind of seems unnecessary and weird so it reads as a punch line

[raven drift]

yeah the 4th block is just stupid

And the 2nd as well

[twilit smelt]

stem guys make it really hard not to believe the humanities ppls critique that they never read and are illiterate as a result lol

the fourth text block is a naked expression of solipsism which is something that famously afflicts middle schoolers rather than 62 year old men

ive seen someone point out that these guys all randomly have solipsistic outbursts that indicate being really philosophically immature or insecure

at the same time they like to see themselves as philosophers and like wax poetic about the future of humanity every two seconds

these guys want to shape the future of humanity while simultaneously believing they belong to a class of super special superhumans and openly doubt whether the "lower 99%" are even conscious lol

the total domination of eternal middle schoolers as pillars of that industry is really nuts to me

idk how thatll even be fixed

theres bound to be a cultural reaction one day

[shadow ridge]

these are the "most people are npcs who dont know how to think and im special" people

[twilit smelt]

i think part of it is that the political pendulum made a hard swing into chudville and a lot of these guys are whatever is currently hip, and chuds are into middle school tier thought processes (or lack thereof), so thats what we're getting from them

another part is that for some reason CS has a really high concentration of people who willfully keep themselves ignorant about other fields while simultaneously insisting on a belief that theyre smart enough to comment on anything

well thats also a well known phenomenon in other areas like physics (a number of genius physicists became crackpots in other areas like history after their physics success made them overconfident) and i guess old CS guys just tend to be prominent and rich so we hear it disproportionately from them

well no. thats not exactly true, its not 100% the same.

because prominent-physicists-turned-crackpots are the rare exception and the few guys it happened to are famous for it

prominent cs guys turned philosophy/history/politics/physics crackpots are the NORM

in the field

i think it might just be the sheer amount of glaze received by these guys from not just society but also dedicated groups of sycophants becomes a cognitohazard that very few survive without becoming imbeciles

its the money probably

the money attracts special attention and attracts sycophants who want money and will unconditionally glaze people who have money as being like their ideal role model

couple the wealth-driven sycophantic attention with overconfidence from succeeding in a technical field considered "difficult" and you get a perfect storm for an entire industry worth of prominent figures forgetting how to be smart and becoming pseudointellectuals

i think that might be why

unfortunate these are also the same men who are the most powerful ones in the world right now

im sure this state of affairs will have no lasting repercussions

also this guy is super proud of the fact he has a wikipedia page and links it in his bio

who wants to bet its partially written by himself on a secret account

https://en.wikipedia.org/wiki/Special:Contributions/Neoranga55

this might be him

only edit was adding a bunch of biographical detail to erik meijer's page

https://en.wikipedia.org/wiki/Special:Contributions/Pindakaas

this is more likely to be him

many edits to erik meijer's page over the span of years and the username is dutch

this guy is a massive AI shill and most of his posts are along the lines of "Vibecoding 100% of your code is safe and effective! It works today! Do it today! I'm doing it and it works great!" and then links his own research which says "if we force LLMs to emit formal proofs of whatever they do before they do it, then they're safe" but never actually says this part anywhere alongside the link

im under the impression he was once a highly respected microsoft engineer guy in the 90s/00s so this is a shame

the grift comes for us all

anything to make his stock holdings more valuable

[raven drift]

This would be great if the tooling was actually there to do this at scale, but it isn't

[digital pivot]

@twilit smelt btw have u tried using llms for any patches in your project

[twilit smelt]

that would defeat the purpose of the project so no

[digital pivot]

anyway it probably wouldn't be able to handle your language

[dense vigil]

^

[digital pivot]

just got rejected for an internship position, so i asked them why and a company representative cited a lack of interest in using ai as the reason

[twilit smelt]

wow they actually replied to reject you rather than just leaving it pending for 8 months before it disappears without a trace?

youre dealing with much more polite companies than i have

[digital pivot]

in interviews they now ask about the use of ai tools, which tools do you use etc

[mortal thunder]

😭

that's so cooked

im not gonna lie

[digital pivot]

that's because a friend of mine works there and he asked them in the internal slack chat

[mortal thunder]

ironically i think it should be the opposite!!

as in, you not using AI should show that you're willing to learn and work

[twilit smelt]

what was the company

[mortal thunder]

and not just slack off

[digital pivot]

so i'm wondering whether it's worth lying to companies or if it's time to start using ai tools and learn how they work

[digital pivot]

a bank focused on african countries that is recruiting staff in eastern eu

[mortal thunder]

probably worth lying if you wanna keep your integrity

[hybrid condor]

I believe rejecting LLMs today is roughly the same as rejecting the internet in the 90s

the internet led to the dot com bubble and now we are to some extent going through the same phase of hype

and since it produces some tangible results, it is bound to survive in some form

[twilit smelt]

i dont reject LLMs so much as i reject the type of person who rabidly shills them

as a cure all for humanities ills

when its obviously more likely to cause massive social disruption at least in the near term

which i think a lot of them know and are pretending not to because they have (or think they have) a financial stake

[mortal thunder]

i don't

[hybrid condor]

I have noticed that it saves me some time on tests

[mortal thunder]

frankly the internet is useful

and provides a service that was never seen before

which is widespread, global-ish, communication with almost anywhere in the world

[twilit smelt]

LLMs have use cases and i just rarely discuss that because the mass quantities of reckless grifting is frankly going to be infinitely more impactful in the near term

that is to say their most rabid shills are going to cause 100x more harm than good, im fairly convinced

[hybrid condor]

they're not autonomous agents and I believe it's pointless to blame the tool rather than the operator

[twilit smelt]

its like a fairly unprecedented mass psychosis driven ultimately by ubiquitous greed that i dont think is super comparable to the dotcom bubble just because its on a whole other level of scale

the dotcom bubble never accounted for 50% of the united states' gdp growth

[hybrid condor]

users, hungry for instant fame, believe that large language models can lower the bar to a level they aspire to but don’t fully understand, and as a result, cause 10 times more damage

[twilit smelt]

i think like 1% of the total US GDP and 50% of its growth right now is being generally ascribed to the hypothetical AI bubble by economists

so far

its something truly unprecedented lol

[hybrid condor]

I’ve rarely seen a technology go from fair attraction to magic that fast

(see web3)

[twilit smelt]

i remember in like 2021 or early 2022 i was in a vc where someone was playing with some GPT demo that was circulating at the time

and they had it translate a short snippet of MIPS assembly to x86 assembly and it worked

and they were like "wow thats neat!"

that person is now one of the most violently radical AI haters i know and thats entirely the fault of the massive grifter contingent lol

[dense vigil]

NFT bros switched to AI and its at an even higher scale so its understandable

[mortal thunder]

i don't hate LLMs, i hate the grifters and the billionaires and the waste of energy

[twilit smelt]

this might sound slightly dramatic to some of you right now but i suspect will sound less so in hindsight in like 5-10 years

i earnestly believe some of the top people involved in the current LLM push are being so selfishly reckless that it counts as a crime against humanity and they belong in the hague

[warm mural]

I'm thinking if LLMs were to stay it would be locally

[twilit smelt]

and this includes a large portion of the richest men in the industry

[warm mural]

much cooler and more sensible

[mortal thunder]

"fun" fact: the US is committed to protecting its servicemembers against the hague

they're ready to invade the netherlands just to get them out of ICC detainment

[hybrid condor]

I made a bet with a colleague, we were working on the same thing

I did it on my own, while he used Claude

[twilit smelt]

thats a well known fact because it comes up a lot in arguments that the US in fact has a hegemony over Europe that they are willing to enforce militarily in the event of provincial rebellion lol

[hybrid condor]

I was sure I could do it better myself, but I was actually surprised

Claude produced a pretty decent result

[warm mural]

Claude is really good

Even the base model

[twilit smelt]

there are two common reactions to "this LLM is actually pretty good at this task" which are "😄 omg im so excited for the post-scarcity future where superintelligence liberates us from labor!" and "😨 omg im so terrified of the future where all human labor is rendered irrelevant and fully autonomous machines only work for the 1% and the remainder get slowly starved to death" and both of these camps think the other one is a visionless idiot

im in the latter camp because obviously the wealthy are going to monopolize this shit lol do you really think if it develops to the point where it could enable post-scarcity that theyd just socialize it or whatever

no way

thats not the kind of person who gets that rich

[mortal thunder]

it's so obviously gonna lead to the latter

idk why this is even a debate

[warm mural]

It's just gonna be an excuse to hire less people

And make more money

[twilit smelt]

this is basically the earliest stage of the latter scenario

[mortal thunder]

the trades are more immune from this to be fair

it's way harder to build an AI that can fix your pipes

or build houses

than it is to build an AI that can write code

[mortal thunder]

or even walk btw

[warm mural]

I don't think programming is cooked tho

[twilit smelt]

in the long run this stuff could essentially obliterate the middle class and leave only an uber wealthy 1% living in an insular post-scarcity microcosm, and an utterly destitute 99%

[warm mural]

Not for now at least

[mortal thunder]

is there still a middle class?

[twilit smelt]

frighteningly, many of the AI shills predict that this will occur and rather than advocating for the avoidance of this scenario, they openly say their aim is to "hypergamble" into the "permanent 1%" so that they dont end up in the "permanent underclass"

they think that AI will cause this and they want it to happen and are working for it to happen because they envision themselves as being part of the elite afterward lol

[digital pivot]

sad story tbh

[twilit smelt]

this is a fairly common outlook among tech bros and gen z "founder" types, they just say this stuff out loud on social media and somehow it doesnt make any headlines

a lot of the "founders" of these shitty doomed AI startups will tell a reporter that they started their company because they want to improve humanity and then turn around and tweet about how their startup is their hypergamble to trampoline themselves and their family and closest friends into the 1% before the 99% gets obliterated by the AIpocalypse

only the former ever gets printed lol

your guess is as good as mine as to why these cultural currents in the industry that would make any average person want to rip these people to shreds if it was common knowledge go so underreported

there have been a few articles about these things but they didnt get shared or paid attention to nearly enough

it might be something that people just cant believe because it makes it sound like the tech industry is full of cartoon style supervillains which surely cant be true and is sensationalist garbage

unfortunately that is proven wrong by following a couple prominent tech guys on "X" and then scrolling for 30 minutes

they really are just cartoon style supervillains

very cool stuff

[twilit smelt]

the vast majority of print about these people is that theyre like prodigal supergenius angels of humanity who want to Push Us Forward into the Future

which is pretty clear now to be propaganda of a technocrat oligarchy that wants everybody to believe Kim family style mythology about how cool they all are and have been since birth

which can embarrass them on occasion when one of them is exposed as a fraud like SBF or Elizabeth Holmes or most recently the Delve ppl and everybody can look back at how unanimously they were glazed, evidently completely unfoundedly

somehow this never manages to discredit the guys who glazed them who were obviously talking out of their ass and always are

ANYWAY......................................

[shadow ridge]

the other popular one is "omg im so terrified of the future where the robots somehow kill us all like in the famous movie terminator"

which makes 0 sense

[raven drift]

I also ask this in interviews

Because it also gives you a good way to filter out the vibe coders

I ask them how they use AI and what they do manually etc

and what weaknesses of AI they experience in practice

[dense vigil]

seems like you look for the opposite of what that interviewer wanted

[raven drift]

Well, I'm not looking for zero AI use, but I'm expecting a more critical response than just: yeah i let cursor do all my coding and then click commit + push

[shadow ridge]

ive never found a way to integrate AI into my workflow

prompting takes more time than just reading the docs or literally thinking about it

[brittle locust]

more time? no shot

[dense vigil]

i use it in 3 cases: tiny functions with very limited scope, tests, and scripts

it can write really good tests if u give it examples of existing tests and api headers

for scripts its just a solved problem that there are so many examples of it just one shots any script

anything with parsing shit etc

[shadow ridge]

yeah that sounds useful

id rather not write some python boilerplate for dumb shit

[raven drift]

Claude is really good at completing some skeleton with proper code

I use it like this: I write

struct page {
    // 0 if free, 1 if allocated.
    int state;
    // Order of the free list of the page if the page is free.
    // Order of the allocation if the page is allocated.
    int free_order;
    // Pointer to next page in freelist.
    struct page *next;
};

// Free list for pages of order n.
struct page *head[10];

// List of all pages indexed by PFN.
struct page *pfndb[];

// Dellocates the page.
void dealloc(long pfn) {
    // TODO
}

Then I prompt: Complete the function in @buddy.c

which causes it to:

● The plan implements the standard buddy allocator deallocation: mark free, coalesce with buddy while possible, then insert into the appropriate free list.                                                        
                                                                                                                                                                                                                   
● User approved Claude's plan                                                                                                                                                                                      
  ⎿  Plan saved to: ~/.claude/plans/twinkly-napping-snowflake.md · /plan to edit                                                                                                                                   
     Plan: Implement buddy allocator dealloc function                                                                                                                                                              
                                                        
     Context                                                                                                                                                                                                       
                                                        
     buddy.c has a skeleton buddy allocator with a dealloc(long pfn) function that needs implementation. The data structures are already in place: a page struct with state, free_order, and next fields; an array
     of free list heads by order; and a PFN-indexed page database.

     Implementation                                                                                                                                                                                              
                                                           
     In dealloc(long pfn):
               
     1. Mark page free: Set pfndb[pfn]->state = 0.
     2. Coalesce loop (while order < 9):
       - Compute buddy PFN: buddy_pfn = pfn ^ (1 << order)
       - Check if buddy is free AND has matching free_order == order
       - If not, break              
       - Remove buddy from its free list (head[order] linked list traversal)
       - Increment order; set pfn to the lower of the two PFNs (pfn & buddy_pfn or pfn & ~(1 << order))
     3. Insert into free list: Set free_order on the resulting page, prepend it to head[order].
                                                                        
     File to modify                                                                                
                                                                                           
     - /home/avdgrinten/ToyAround/claude-freelist/buddy.c — implement dealloc function (lines 18-20)
               
     Verification
                                                                                                
     - Read the completed function to verify correctness of buddy calculation and free list manipulation.

And it produces

// Dellocates the page.
void dealloc(long pfn) {
    int order = pfndb[pfn]->free_order;
    pfndb[pfn]->state = 0;

    while (order < 9) {
        long buddy_pfn = pfn ^ (1L << order);
        struct page *buddy = pfndb[buddy_pfn];

        if (buddy->state != 0 || buddy->free_order != order)
            break;

        // Remove buddy from its free list.
        struct page **pp = &head[order];
        while (*pp != buddy)
            pp = &(*pp)->next;
        *pp = buddy->next;

        // Coalesce: keep the lower-addressed page.
        pfn = pfn & buddy_pfn;
        order++;
    }

    pfndb[pfn]->free_order = order;
    pfndb[pfn]->next = head[order];
    head[order] = pfndb[pfn];
}

[warm mural]

I like to ask it about ideas I've had to see if there are any existing similar things

Also to clarify implementation details/concepts that I dont understand

It's obviously not perfect but I can usually complete on my own

Also "I want to do X, what solutions already exist?" Might point me to algorithms or data structures I wouldn't have looked into

[shadow ridge]

thats pretty cool

can it do more complicated algorithms

like generating crypto key pairs or smth idk

[raven drift]

uh, it probably cannot produce a production quality side channel attack free implementation without help

[brittle locust]

just ask claude to generate them, im sure they will be secure

[raven drift]

for complicated stuff you have to nudge it into the right direction

[raven drift]

Also, I don't know enough about production quality crypto implementation to judge if the impl is good, so I wouldn't try to AI generate it

[tame phoenix]

AI can recommend what pre made lib suits your system, but beyond that yeah, wouldn't even trust myself

[acoustic sparrow]

depends on what algorithm

for RSA, even humans cant really do it properly tbh

for eliptic curves, nowadays a lot of the actually timing sensitive operations are implemented by autogenerated (and formally verified, ofc) code in many cases (iirc the go stdlib uses that for their x25519 calculations for example), and using the magic tool is like not THAT hard

for things like chacha20 its also pretty hard to have a side channel

[warm mural]

hey will how are you going to do OOP in your IOKit-like thing? will you add OOP extensions to jackal or do it entirely through function pointers and stuff

[twilit smelt]

i was doing an explicit implementation of vtables lol

i started that but its a stub that hasnt been used for anything so its probably not good

[warm mural]

yeah ok that makes sense, dunno if it will get annoying in the end though

[shadow ridge]

why would it get annoying

it's not like his func pointer syntax sucks

[delicate dome]

manual vtables are fine

It's the lack of generic collections that usually bugs me with languages like C and presumably jackal

[shadow ridge]

thats really only an issue for vectors ime

everything else works fine and even better with intrusive data structures

[warm mural]

inheritance and stuff

gets clumsy to do manually

[delicate dome]

Yeah vectors are the worst for that by far

[shadow ridge]

eh

[twilit smelt]

single inheritance is rly easy

just put the base class at the top of the structure

likewise you put the base class's vtable struct at the top of your vtable struct

[delicate dome]

Inheritance is NOT the problem with C

It's vectors

[warm mural]

Yeah I know but it's still a bit annoying

[warm mural]

Generic typed containers in general

[delicate dome]

no not really

[warm mural]

yes really

[keen stone]

container_of my beloved

[delicate dome]

Hash sets, maps, radix trees, all of these worked fine so long as you accept passing in and getting out void *

[warm mural]

container of is for intrusive containers, I wouldnt consider an intrusive container to be a generic typed container

[warm mural]

You can also do this with vector tho? Just pass in the element size

[delicate dome]

No, the way I made the three I mentioned, they basically just store the pointer

[warm mural]

Also a hashmap is literally like a dynamic array so

[delicate dome]

They don't know what's behind it

Thing is I don't want to have a struct foo ** kinda array

I want a struct foo * kinda array

That's the hard one

[shadow ridge]

intrusive containers are better

most of the time

ok well it kinda depends

[warm mural]

You can't make everything intrusive

[shadow ridge]

but id rather not do struct linked_list_int_node { struct linked_list_int_node *next; int value; }

[warm mural]

And that's not really true either

I agree intrusive for node containers is good

But node containers aren't always good

[shadow ridge]

as i said, it depends

sometimes you really do curse at C

for not having generics

[warm mural]

I like C a lot

[shadow ridge]

right after you do this you have to pull out a gun and shoot the next guy who mentions _Generic though

[warm mural]

_Generic is incredibly useless

[shadow ridge]

honestly id rather implement every different generic container separately instead of resorting to macro hell

or at least use macros to auto generate a common header for common types

[delicate dome]

TBH not having good generic types is the biggest reason I ended up switching to Rust

[shadow ridge]

language design wise i cant really think of a good way to put generics in C

without making it overly complicated

[warm mural]

I think I agree

There's not that many cases in a kernel for example where you need a vector

And if you do you can implement an adhoc one

[shadow ridge]

they benefit from being really simple

[warm mural]

I think the zig way is nice, generics are functions that return a type

[shadow ridge]

it's like 3 lines of code for any op

[warm mural]

tho this would need a major overhaul of C anyway

[shadow ridge]

cant really wrap my head around how this enables you to do generic containers

[delicate dome]

typedef bullshit = generic(innertype) ?

[warm mural]

fn GenericType(T: type) type {
  return struct {
    val: T;
  };
}

[delicate dome]

If I had to guess?

[shadow ridge]

ooh

[warm mural]

sorry i wasnt clear

they take types and return types

[shadow ridge]

well this would kinda require structs to own functions which basically makes C into C++

at that point just do templates

[delicate dome]

Are two invocations with the same T treated as the same type?

[warm mural]

well yes but no

there's no oop per se

everything is a struct

a module is a struct

and as such functions can be defined inside

[shadow ridge]

yeah but that means they belong to a proper namespace

[warm mural]

yes

[shadow ridge]

it also introduces name mangling probably

[warm mural]

no

[shadow ridge]

how do you do it without name mangling

[warm mural]

const lol = struct {
  const Self = @This();
  
  pub fn member_function(self: *Self) {}
};

[warm mural]

why would you need name mangling?

name mangling is for operator overloading mostly afaik

[warm mural]

I think so?

[shadow ridge]

oh nvm then

i thought it would need smth like name mangling

if it didnt return the same type

[delicate dome]

Yeah name mangling for generics is basically mandatory

[warm mural]

No??

[mortal thunder]

you could simply store an element size

[delicate dome]

For this style of actual language-level generic I mean

Because else how are you disambiguating the symbols

[mortal thunder]

and then when you call vec_get(vec, idx) you get vec->start + vec->item_size * idx

[shadow ridge]

tbh id rather have a simply type system where functions dont just become types themselves

how do you represent a struct with a function without representing the function itself as its own type

that just makes no sense to me

it's basically like representing the value of a constant within the struct type

[warm mural]

It's just GenericType(instance).function

That's not mangling

[mortal thunder]

a struct with a function, wdym

like C++ struct methods?

[shadow ridge]

whatever zig does

[delicate dome]

Are you sure there's no mangling happening behind the scenes?

[warm mural]

Yes??

Zig is designed to be as C-like as possible

[delicate dome]

how the fuck does this work then

[warm mural]

??

[delicate dome]

implicit parameters that encode type to the function in question?

Like you have to call the symbols something

[warm mural]

Yes

it's not instance

It's the type

[delicate dome]

And keep in mind that even GenericType_instance_function is still name mangling

[warm mural]

So like GenericType(u32).function

Is it?

[delicate dome]

You can't call symbols that

[warm mural]

Why not

Yes you can

[dense vigil]

just readelf and check what symbol names actually are lol

instead of arguing about it

[warm mural]

They are that

[dense vigil]

[delicate dome]

If this is anything to go by there is name mangling happening, just very simple: https://github.com/ziglang/zig/issues/12751

what the hell

It actually does call symbols stuff like debug.FullPanic((function 'panic')).noreturnReturned

[shadow ridge]

cursed

[mortal thunder]

why wouldn't it be legal

[delicate dome]

crappy old linkers basically

[mortal thunder]

the only barrier to a symbol being called whatever(stuff).something is C

if only you could escape symbols like bash can

[delicate dome]

But I guess I shouldn't be surprised that Zig doesn't care about shitty linkers

[mortal thunder]

void something\ amazing(void* stuff\ of\ dreams);

[delicate dome]

No actually, there are (very old) linkers you can't do shit like put . in symbol names with

[shadow ridge]

you could write an extension that allows you to specify full symbol names using backticks

[mortal thunder]

please add this in C29

[shadow ridge]

void `something amazing`();

[mortal thunder]

zig doesn't care about those

does it?

[delicate dome]

clearly not

[mortal thunder]

i mean zig is NEWER than those old linkers

in the first place

[shadow ridge]

i think this is pretty cool

[mortal thunder]

idk why C doesnt support this actually

fucking bash does

I believe

at least for command line parameters

[delicate dome]

Go tell GNU to add it

[tame phoenix]

void `something hairy`(int `fuck no`)

at least pick a character the language doesn't already use lol

[delicate dome]

void ` main`(int argc, char **argv);
``` ![trl](https://cdn.discordapp.com/emojis/829316641191886888.webp?size=128 "trl")

[shadow ridge]

you could do a pseudo-macro style generics implementation where it's simply preproc style find & replace but generates the definitions per unique expression

struct Vec<T>
{
    size_t capacity;
    size_t length;
    T *members;
};

void Vec<T>insert(Vec<T> *vec, T *element)
{
    ...;
}

since it's find & replace you could probably do fancier stuff

[tame phoenix]

void ` good luck`()

[delicate dome]

Have fun figuring this out with your linker

[delicate dome]

You need too much semantic information for the preproc to be able to do this sadly

[shadow ridge]

you dont need the preproc to do it

you can just do it in the parser

as i said

pseudo-macro

you could have like fixed size radix trees

defind like

RadixTree<512> or some shit

in practice that means you cant just do simple find & replace

but thats not needed

as long as you can parse types or expressions and pass those around

ok maybe it requires more parser gymnastics 💀

[delicate dome]

I'm still gobsmacked Zig effectively just doesn't name mangle

Only stringifies

[shadow ridge]

better than name mangling tbh

who cares about some linker from cave man times

[shadow ridge]

ok hear me out

[delicate dome]

Literally everyone but Zig?

At least the rule to mangle

So like what, Rust is trying to replace C++ and Zig is C or something?

Was that the stereotype

[shadow ridge]

1. a literal macro type
2. a literal macro node in AST
   expand them before semantic analysis and it should work

// let's say <identifier> defines a macro that "binds" the following declaration

<A,B> A add(A x)
{
    return x + B;
}

this gets parsed as

function add
    binding <A> and <B>
    return_type <A>
    argument 0 <A> x
    function body:
      return
        |
        +
       / \
      x  <B>

say we call <int,2>add(3). after we expand for <int,2> we get

function <int,2>add
    return_type int
    argument 0 int x
    function body:
        return
          |
          +
         / \
        x   2

then we do our semantic analysis and stuff and it should work

you could define the AST node as just holding a macro type

you only really need the type here for it to work anyways

note that in this context int and 2 are parsed as actual expressions

there are obvious flaws

like 2+3 and 3+2 being different and expanding differently

i think this could work though

[sterile frost]

my favorite mintia conversation

[shadow ridge]

why

[sterile frost]

if i had to guess

probably because it has nothing to do with mintia

[shadow ridge]

it started from manual vtables in mintia and evolved into a discussion about generics so

id say it does have something to do with mintia

[sterile frost]

i wish it worked this way, but it's clearly off topic, anyway not my thread i was just disappointed that it wasn't anything on-topic

[shadow ridge]

@twilit smelt find a way to tie this back into mintia so it's on topic

you should add generics to jackal

also do you think my macro idea is cool

[warm mural]

Yes

Zig is much more C-like

[twilit smelt]

i dont mind it at all

keeps my thread open

[twilit smelt]

https://www.youtube.com/watch?v=Tk2gNQ1yLeQ

this sounds like rejected setup music from an early 2000s mac os 10.x beta

apple had shit like this playing while u set up ur account

that was before the zeitgeist became "your computer must only speak when spoken to" because everybody collectively decided they hated magic and whimsy

so they stopped doing setup music

https://www.youtube.com/watch?v=ijiI7Z77BiE

[twilit smelt]

^actual example of this

its like they forgot how to be cool

the whole tech industry collectively forgot how to be cool

must be because they aged

[shadow ridge]

you can still install a fitgirl repack to get a similar whimsy

[twilit smelt]

the brief period in the late 90s where operating system UI elements had sound effects was incredible

[shadow ridge]

actually wait do they still come with chiptunes and shit

[twilit smelt]

would have been extremely annoying in practice and i would have disabled it after like a day

but still

classic mac os in particular did that

it made buttons make clicking noises and itd make clicking noises as you navigated through menus and whatever

[shadow ridge]

i honestly think basically all UI sound effects are annoying. i consider myself blessed when i hear bullshit windows sound effects coming from the PCs downstairs

the only sound effect i want is like a clicking noise when im adjusting the volume so i know how loud it is

everything else is just video game shit

im okay with the windows 11 installer using like prokofiev 2 or some shit as background music though to help it reinforce the despair and hopelessness

[dense vigil]

@twilit smelt btw have you ever considered emulating thermals, fans, voltage etc for your system

that would be funny i think

so u could explode the imaginary xr system

and add like graphics to the emulator to depict that

[twilit smelt]

that would be funny but not realistic since there generally wasn't system software control over any of those things on workstations of that period

fans would be directly driven by autonomous circuitry for example

[dense vigil]

oh ok

sad

[twilit smelt]

kernels on desktop systems having awareness of thermals and whatever is an extremely new like early '00s thing

[twilit smelt]

or they would just be pinned to 100% always

some of those old workstations sound like jet engines from the moment you power them on and never really vary in fan volume

[delicate dome]

@twilit smelt do you have a spec for xr17032

and xr/station

[twilit smelt]

yes

[mortal thunder]

should I port my os to your arch (now that monkuous made a C toolchain for it)

[twilit smelt]

If you so desire

I don't see why you would want to though lol

[dense vigil]

Wouldn't you feel proud

To have an arch with more users

[twilit smelt]

I already know it's possible and that it would work and it would behave identically to boron on amd64 so I don't see what the point is

[dense vigil]

Bragging rights

[twilit smelt]

I don't think my brain works like that

[dense vigil]

I mean you already have a Linux port so I guess its not of much value

[twilit smelt]

That was interesting bc it was a proof of concept that the architecture is robust enough to run a kernel widely recognized as not being a toy

With full vmm and stuff

[dense vigil]

Technically it was randomly segfaulting

[twilit smelt]

That is believed to be a TLB invalidation bug I think bc monkuous was able to fix-ish it by flushing the whole TLB on every context switch or something like that

He got bored and didn't track it down any further

[dense vigil]

Ah ok

[delicate dome]

the question was if you could give me it lol

[twilit smelt]

isa handbook: https://raw.githubusercontent.com/xrarch/books/main/xr17032handbook/main.pdf
computer handbook: https://raw.githubusercontent.com/xrarch/books/main/xrcomputerbook/main.pdf

[warm mural]

is this a typo

why is it WHAMI and not WHOAMI

[dense vigil]

Cant believe its already more thought out than risc v

[warm mural]

what

[twilit smelt]

larp

(thats the prism handbook)

[dense vigil]

Cool

[warm mural]

slight nitpick but you should embed the diagrams as svgs

[twilit smelt]

ive been in the process of doing that for like 6 months

i keep doing like

1

every 3 weeks

[warm mural]

also holy shit the patience needed to write all that

nice job

[shadow ridge]

it's all so nicely formatted too

[dense vigil]

What's an MFPR

[twilit smelt]

"move from processor register"

its an instruction mnemonic

[dense vigil]

Ah

[mortal thunder]

In case I get my hands on a mips device, if I already have an xr17032 port I'd have already figured out like half the things I need to get right about the mips port

I guess

Since old 32-bit mips as well as xr17032 both have software refilled TLBs in a similar manner, and I believe some other things are similar enough as well

arm has full hardware page table walking (which is fine, dont get me wrong), so I didn't get the opportunity there

[warm pine]

even very recently it was often problematic

HP c8000 (2005 i think) data sheet boasts of its quietness

[mortal thunder]

worksloads

is this correct in british english

[mortal thunder]

oh, and bragging rights

more ports better🤷‍♂️

[twilit smelt]

the apple openfirmware console famously polls for input without ever halting the cpu, which was unnoticeable on the earlier ppc mac systems

but it remained that way up until the final ones, the powermac g5s

so if you let a powermac g5 sit on the openfirmware console

it gets actually unbelievably loud

doing nothing

[mortal thunder]

i wonder how much they saved by not implementing interrupts in openfirmware

time, money, etc

i know something that also polls for input and that would be the NT kernel debugger but at least that one has a reason to poll for input

[raven drift]

this sounds more like a "we need to focus on something else so ship it now" issue than an intentional cost saving measure

[delicate dome]

i love the cover art

Also I may or may not be interested in adding xr17032 backend to Lily-CC

[twilit smelt]

i uploaded my nt team vaporwave edit to youtube and the singular comment is perfectly targeted ragebait https://www.youtube.com/watch?v=HyyuwhBkY8k

must be someone who knows me very well...

also i was thinking about making more of these with various themes

like unix, powerpc/ibm, macintosh, etc are all possible themes

i know more than enough obscure vaporwave tracks thats not an issue. and i think i know of enough weird archival footage like this too

that i can do lots

with

the nail in this project's coffin would be if i realized i was only ever in it for the vibes and that i can more effectively produce vibes by getting good at video editing and other stuff

[twilit smelt]

my OS course at uni gives me a big head

[mortal thunder]

I'd be interested

[twilit smelt]

"wow this assignment is really easy why do we have 3 weeks for this? all we have to do is modify this code so that instead of mapping 2MB with one huge page, we map 8MB with individual 4KB pages. there thats done. i wonder how everyone else is doing"

then i check the piazza board

[mortal thunder]

my OS course only had the mere basics

[twilit smelt]

total chaos and pandemonium

"HOW THE HELL IS ANYONE SUPPOSED TO FIGURE THIS ASSIGNMENT OUT"

[mortal thunder]

it didn't even involve writing any low level code

the most low level assignment was writing a small libc clone

and an elf loader

[keen stone]

lol our os course is using fork, shmget, socket and semget

[twilit smelt]

the paging unit has been complete intuition for me and ive been doing the assignments in like 30 minutes and almost everybody else is suffering for weeks on each of them, its kind of crazy how much of a leg up experience gives you

[keen stone]

at most it's writing a linked list best fit allocator

[mortal thunder]

then you get slammed in the face when you have to do something you DONT have prior experience since you got comfortable...?

i mean that's how i feel about one of my classes

[twilit smelt]

the hardest part has been x86 architectural details since id never actually gotten a kernel up and running from scratch on x86 before, and the initialization process is a little baroque

and we had an assignment where we had to get a hello world kernel running

but even that only took a couple hours

[mortal thunder]

i have a class this semester which is basically about computer architectures

and rn we're going through a breakdown of SAP-1 if you've heard of it

[twilit smelt]

your OS course sounds like it has the content of our mandatory "computer systems" course

the OS course im taking is mostly a postgrad course that undergrads can sign up for if they dare

[mortal thunder]

SAP-1 is kinda popular in unis from what ive heard

[keen stone]

that sounds cool

[mortal thunder]

ive heard that the final assignment for this class will be implementing some kind of 32-bit cpu?...

[twilit smelt]

if im ever a prof teaching a comp arch class im going to force everybody to learn xr17032

[mortal thunder]

not sure if thats true though

maybe in another group

[mortal thunder]

pity there's no concrete hardware implementation

[keen stone]

nah in order to be a true computer architecture instructor you must teach mips

[twilit smelt]

and trick everybody in the class into thinking its simple by not mentioning anything about the kernel mode architecture

[keen stone]

and branch delay slots :p

[twilit smelt]

which is arguably more evil than x86's

from a kernel writer's perspective

much less evil from a hardware designer's perspective

[keen stone]

as an x86oid mips is insane tbh

it's too simple to be good

[twilit smelt]

the page table walking flow is so funny

and by funny i mean really scary

[keen stone]

yeah i also heard you have to fill the tlb yourself

[twilit smelt]

they do a bottom-up page table walk with an implicit loop caused by tlb miss exceptions

thats the preferred scheme for the software tlb refill

[mortal thunder]

i'd argue arm's kernel mode architecture is more insane

there's like 6 different total modes

system and user which have access to sp and lr

supervisor ("kernel mode") which has its own sp and lr

and abort mode, IRQ mode, fast IRQ mode, and undefined instruction mode, which have their own individual sp and lr

you can have your interrupt vector table at either 0x00000000 or 0xffff0000, no flexibility other than one bit

[twilit smelt]

basically the canonical tlb miss handler tries to load the needed PTE directly from a recursive mapping of the page table (that is, a virtually linear unrolled mapping created by a self-entry in the top level table) and if that page table level is not in the tlb, a nested tlb miss occurs

this repeats until a level that is in the tlb is reached

[mortal thunder]

you get to have TWO!! page table pointers, where you control the size of the other one through TTBCR or whatever its called

[keen stone]

it's kinda funny how at this point x86 is the sanest arch

[mortal thunder]

and the page tables also dont have any capability for self-referencing because the two levels are unbalanced

[twilit smelt]

this has a benefit btw that if the page table containing the pte is already in the tlb, the bottom up walk will terminate immediately and so the upper levels of the page table dont need to be walked

[mortal thunder]

the top level is a 16 kb page table and each second level page table is 1 kb

and you have no software-usable bits in the PTEs

at all

[twilit smelt]

is ths 32 bit arm or aarch64

[mortal thunder]

32-bit arm

i have no experience with 64-bit arm

[twilit smelt]

i know basically for a fact that aarch64 has software-usable bits in the PTEs because NT uses them for stuff

[mortal thunder]

maybe they made a few things more sane but i have no faith in them rectifying any of their issues

[keen stone]

yeah

[mortal thunder]

huh

NT also has an arm32 port

so... how

[twilit smelt]

it doesnt support arm32 anymore i dont think

[mortal thunder]

anymore

but it did

[twilit smelt]

NT only recently started using 3 bits of each PTE to store the page age

0-7

[mortal thunder]

ah

[twilit smelt]

it likely started doing this after arm32 support was dropped

[mortal thunder]

makes sense

[twilit smelt]

also sometimes there are still ways to do a recursive page table even if the page tables are unbalanced or wonky

[mortal thunder]

what were they using before this? the working set list?

[mortal thunder]

I've pulled off a linear mapping of both levels of page tables but it was kinda painful, and definitely not "recursive"

[twilit smelt]

for example you can do it on PAE where the uppermost page table has 4 entries and the lower 2 levels have 512 entries

in a fairly simple way

[mortal thunder]

yeah you probably could just map each of the 4 entries into the last one and then the root page table into another entry

[twilit smelt]

yes, except the root page table is tiny so you probably just have that allocated on the heap somewhere

with the needed alignment

no need to map it explicitly

[mortal thunder]

it doesnt help that on arm the pte formats differ between leaf pt's and non-leaf pt's

[twilit smelt]

i would be very interested to see how windows managed the page tables on arm32

could be worth RE'ing

[mortal thunder]

that would be interesting yeah

maybe i didnt need to use my solution to the problem

[twilit smelt]

the real killer for recursive mapping isnt unbalanced levels its inconsistent pte formats

with unbalanced levels there are often funny tricks you can still do but inconsistent pte formats just kill u

if the pte formats are inconsistent between levels on arm32, then NT likely did explicit mappings of the page tables to create the virtually linear table

which is also what it would need to do on risc-v for the same reason

[mortal thunder]

very inconsistent indeed

[mortal thunder]

yknow what

i think i will download an arm32 build and try and see how it managed ptes

if its not optimized away to hell and back

[mortal thunder]

okay, I took a look at armv7's page tables, because this is armv6 I was looking at, and it seems that armv7 actually has a choice between two formats. one of them seems to be the same as armv6 or similar ("Short-descriptor translation table format"), and the second one seems to be closer to sanity ("Long-descriptor translation table format")

the Long format is similar to PAE in addressing and can be setup with recursive page tables in a similar manner to PAE

meaning, apparently, NT never had to deal with the insanity that is the armv5/armv6 page table format.

(Windows CE did, but it's not as interesting since the CE kernel isn't nearly as complex or documented)

it's so similar to PAE that it even uses 64-bit PTEs!

[twilit smelt]

the CE kernel is a fully fledged kernel in its own right

for the record

it has page swapping and all that

protected memory

memory mapped files

etc

does all of that in its own unique way

shitty way mostly

from what i saw

[mortal thunder]

i see

[marble briar]

this is very cool

[warm pine]

these were fashionable at the time

i wonder if they were directly inspired by m68k MMUs?

the terminology is quite similar, they're also called "long and short descriptors" there

[mortal thunder]

what do 68k page tables look like

[warm pine]

the full feature set varies by model (each successive one was a rough subset of the functionality of the previous)

[warm pine]

A translation tree may be composed of up to five levels of tables requiring the use of five separate index fields to locate a logical-to-physical mapping. Additionally, tables at different levels of the translation tree may all be of the same size or they may each have different sizes.
The MC68851 provides the ability to replace an entry in a page
table with a pointer to an alternate entry.
some highlights

page sizes from 256 bytes to 32k bytes, you can configure how many bits out of a virtual address should be used to index each level of page table, it's quite extensive

there were 8 different permission levels also and each PTE was individually configurable

[mortal thunder]

PA15-PA8? so you had 256 byte pages?

[warm pine]

if you wanted 256 byte you could do that, or any power of 2 from 256byte up to 32kbyte

[mortal thunder]

but what if you opted for 256 bytes, would you need to specify 3 levels of page tables?

would there be a non-canonical region akin to x86_64?

[warm pine]

you could do a single level (linear table) if you want or anything up to 5, for any page size of your choice. you didn't have to use the entire address space but i'm not sure there were ever anything like the canonical hole of x86_64

[mortal thunder]

oh would it just mirror the address space?

and yeah in boron I'd probably use the short descriptor format yeah

[warm pine]

of note as well distinct tables for usermode and supervisor mode

[mortal thunder]

oh that sounds cool

similar to TTBR0 and TTBR1 on arm i guess?

except do you have any control over where the split happens

[warm pine]

one is used for supervisor mode accesses, and one for user mode accesses

[mortal thunder]

but i mean like where the split happens

or is it fixed at 0x80000000

[warm pine]

it's wherever you'd like to put supervisor mode stuff, and indeed you could even have usermode occupy 0x0 to 0x80000000 or whatever and also have the kernel occupy the same span, and it would work since the transition to supervisor mode causes further lookups to be against the supervisor table

[marble briar]

@twilit smelt i'm planning on doing this myself, once my compiler works. any tips or tricks for a newbie?

[mortal thunder]

doing what exactly?

[marble briar]

making my own language, architecture, and os

[tame phoenix]

my advice? dont do all 3 at once

start at the bottom of the stack, with the architecture

then make the language and its compiler, design it for systems work, then make the OS

do the architecture right first time, as you dont want to have to change it to suit your OS or compiler

[twilit smelt]

prepare for it to take a very long time

[tame phoenix]

posted my progress image here rather than my own thread, sorry bout that!

[marble briar]

honestly rn i'm struggling with jumps

[tame phoenix]

well, if you look at something simple like 6502 or Z80 as an example... its easy enough

you need basically absolute jump to address, jump to address indirect (a jump vector table) and branches, where a branch happens if a flag is not or is set, usually one of carry or zero flags

[marble briar]

that's not the part i'm struggling with

[tame phoenix]

carry is set when you add two numbers and they overflow, so this can be used as simple greater/less than check, equality and not-equal are easy

ah ok

[marble briar]

it's specifically the PC interaction

[tame phoenix]

how do you represent system memory?

[marble briar]

do i update PC before execution? after? do i do it in words? bytes?

[marble briar]

flat array of bytes (word and byte addressable)

[tame phoenix]

i dont think it materrers really so long as the outward view of the registers is correct from a machine perspective after the opcode is executed

this is all at the 'microcode' level

also any opcode should be atomic, right?

so if your system is smp, you shouldnt ever be able to see a half updated JMP state on cpu 1 when youre executing on cpu 2

[tame phoenix]

how many bits wide is the arch?

[marble briar]

32

[marble briar]

its just a bit odd tbh

at the moment pc is a hidden register

i might put it in r14

[twilit ingot]

can you even use r14 in 32 bit mode?

[tame phoenix]

if its hidden youll have all kinds of problems later

[marble briar]

yeah?

[twilit ingot]

oh

[marble briar]

i've got r0-15

[twilit ingot]

ah, riscv

[marble briar]

i was thinking about that

[twilit ingot]

or aarch

[tame phoenix]

but it doesnt need to be immediately acessible in a register like r14, you can have an opcode to push pc to the stack

same as flags

[marble briar]

custom but yeah

[twilit ingot]

i was thinking of x86 XD

[marble briar]

lol

[tame phoenix]

do any of r0 to r15 otherwise have special purpose?

[marble briar]

r15 is the stack pointer

but thats it

[tame phoenix]

i would recommend not sticking them in rX

i mean its your architecture youre not forced to take up space in the finite number of user registers, make actual new registrs called sp and ip

[marble briar]

i've only got 4 bits to address registers though

[tame phoenix]

then fix it? 😄

[marble briar]

i cant :/

[tame phoenix]

now's the time, why constrain yourself artificially

course you can you havent made the os and compiler yet

[marble briar]

my instructions are tightly packed

there isn't enough space

https://github.com/gingrspacecadet/Orion/blob/v3/spec.md

[tame phoenix]

only 256 interrupt vectors?

[marble briar]

why would i need more?

also it works to about 32 i think

[twilit ingot]

pci prob?

[tame phoenix]

exactly why x86/x64 has MSI...

something like an nvme drive can register whole ranges of interrupts for increased performance

also what about routing, which irqs go to which cpu in an smp system.... assuming youre going to do smp at some point

[marble briar]

probably not

smp is scary

[tame phoenix]

point is theres no need to artifically constrain yourself unless youre trying to make a tiny 8 bit architecture

this feels like an 8 bit architecture with a 16 bit bus rn

[marble briar]

oh

so what would you change?

[tame phoenix]

i'd make opcodes wider, maybe fixed widths (32 bit?)

more interrupt vectors

and perhaps make it so jumps etc can only be to aligned addresses equal to opcode width

also make a 'save all state to stack' and 'pop all state from stack' opcode

like x64 XSTORE

will be really really useful for an OS to do multitasking

[marble briar]

instructions are 32 bits with an optional 32 bit extended immediate

idk why people keep thinking that they aren't

[marble briar]

they are

[marble briar]

seems like a good idea

i got a bunch of spare opcodes

[tame phoenix]

because of the width of your hex values in your docs

and the fact you dont state how wide they are anywhere except one line: "fixed 32-bit instructions, with optional extended immediates for funsies"

if you made each number in the table 32 bits wide, e.g. 0x00000000 instead of 0x0... it is obvious

also.... my advice? dont make 0x00 a valid opcode

make 0x00 BRK or NOP or something

so if the cpu runs through uninitialised code, its an error not valid code

also you dont have a NOP instruction

[marble briar]

good point

[marble briar]

the opcodes are 6 bits but the instructions are 32

[tame phoenix]

exactly, but the opcode takes up part of the 32 bit space

[marble briar]

yeah

[tame phoenix]

and what do you mean by extra 32 bits for immediate

[marble briar]

if you want a larger immediate

you can use the next 32 bits

[tame phoenix]

hmm?

[marble briar]

instead of having to do top half and lower half

[tame phoenix]

larger how?

[marble briar]

greater than 16 bits

[tame phoenix]

so addresses are 16 bits?

[marble briar]

or 12

[tame phoenix]

i dont understand

[marble briar]

i dont remember

[marble briar]

if you need to encode a number greater than 12/16 bits, you can encode it in the following word

[tame phoenix]

eh

dont put addresses in the 32 bits where your opcode goes

put them in a following separate 32 bits

[marble briar]

what about small addresses

[tame phoenix]

same

[marble briar]

like relative jumps

[marble briar]

is that not a waste?

[tame phoenix]

unless youre optimising for tiny memory, what is the point

aligned reads of 32 bit values will be far easier to understand and likely faster

think about it like this, 12 or 16 bits really isnt that far, for branches and stuff

you end up with an x86 like mess of near/far jumps

and often have to do like : BEQ x : JMP not : .x JMP yes : .not ..... then a few megs later....yes

this was a problem on 6502 too, with branches never being allowed to be more than 128 bytes in either direction from the opcode, but because the memory was so much smaller that compensated for it

[marble briar]

i can do both though

idk

[tame phoenix]

why not keep it simple

[marble briar]

ill think about it

[tame phoenix]

youll thank yourself later

😄

[marble briar]

is it not simple?

[tame phoenix]

embedding 12-16 bits in the opcode but only sometimes, idk not that simple

because then you need a way to indicate if its there or not

basically you need to do some kind of logic to see if an address or offset follows the opcode

thats slower than there just always being an address or offset following said opcode

for things like JMP and branches, theyll be in the hot paths of anything that runs

[marble briar]

its just checking if a bit is high or not

[marble briar]

mmm

[mortal thunder]

this might be interesting https://bryankeller.github.io/2026/04/08/porting-mac-os-x-nintendo-wii.html

to liam, mostly because it explains with a concrete example the way IOKit is structured

[twilit smelt]

been seen it

[warm mural]

This is sick

[twilit smelt]

Unfortunately it had been done before like over a year ago but the first person to do it failed to blog about it and now gets no credit for being first or for doing it at all

[warm mural]

Eh, I don't really care who did it first, I just know this is really cool

[twilit smelt]

This is the second completed port of OSX to the Wii

[twilit smelt]

I just feel bad for the first guy

I watched him do it in a discord server

[warm mural]

Is that the wiitosh thing

[twilit smelt]

yeah something like that

[warm mural]

This

[twilit smelt]

yeah

he got his credit yoinked by a guy who performatively whips out a Wii and laptop on a plane and takes a picture of it for his blog

getting mogged by people with more of a penchant for theatrics is a rite of passage for the computer autist

[warm mural]

I don't think the blog guy wanted to do any wrong

[twilit smelt]

probably not

still black pilling

[twilit smelt]

its completely portable to have an mmu abstraction layer that provides a direct virtual view of the page tables as a linear array where array index = vpn

that can be used by the generic code

this can be implemented like this:
x86: recursive mapping
aarch64: recursive mapping
powerpc: refill hashed page table from simulated multi level page tables, and then do recursive mapping thereof
weird architectures with strange inconsistently sized page tables (m68k) or inconsistent PTE formats (riscv): explicit mapping

then you only need to call into this layer when creating & deleting virtual area, to prepare PTEs in the linear array which you can then directly write into

you can have an API like this:

// After returning, the existence of PTEs to map
// all pages between startVpn (inclusive) and endVpn
// (exclusive) is guaranteed until a matching
// DeletePteRange call.
CreatePteRange(startVpn, endVpn)

// After returning, the existence of PTEs to map
// all pages between startVpn (inclusive) and endVpn
// (exclusive) is no longer guaranteed. When appropriate
// pmap-managed refcounts reach zero, page tables will be
// reclaimed.
DeletePteRange(startVpn, endVpn)

calculating the address of a PTE inside this table is a fully generic macro

aside from accounting for the arch-specific PTE size

[shadow ridge]

and how do you index it like an array

[twilit smelt]

by indexing it

its just an array

[shadow ridge]

wdym

[twilit smelt]

its the real page tables mapped virtually as an array

indexed by virtual page number

which you can do for free with recursive mapping on architectures that support it

and explicit mapping on others

[shadow ridge]

btw vpn is just page address / page size right

[twilit smelt]

yes

[shadow ridge]

hm

i love this ngl

but how useful is it exactly

[twilit smelt]

it allows you to access the page tables without having to walk them

[shadow ridge]

is the overhead worth it

[twilit smelt]

which means you can do certain cases of PTE setting and whatever completely inline with no branches and whatever

[twilit smelt]

what overhead

[shadow ridge]

it takes a lot out of the address space and theres overhead from having to map them in the array when new page table levels are created

[twilit smelt]

the overhead of explicit mappings only exists on LAME architectures that dont allow recursive mapping

with recursive mapping, installing the page table into the appropriate location in the page table hierarchy will cause it to magically materialize in the appropriate location in the array

[shadow ridge]

how tf does that even work

[twilit smelt]

it also doesnt take much out of the address space, its only a total of 4MB with a 32-bit address space

with 64-bit its a much larger area (512GB on amd64 with 4-level paging) but still very tiny relative to the total space (256TB on amd64 with 4-level paging)

[shadow ridge]

32 petabytes if the entire 64 bit address space is usable

but thats just the virtual usage

how much overhead is there when youre actually setting up the mappings

probably not too much since it's recursive right

[twilit smelt]

you only have to set it up one time

by inserting the top level page table as an entry into itself

this creates an unrolled virtually linear mapping of the entire page table as an array

starting at the base address mapped by the entry you chose

this is called a self-entry or auto-entry

[shadow ridge]

you only have like 256 locations to choose from though

[twilit smelt]

512

and yes

thats fine

[shadow ridge]

i was thinking 64 bit

[twilit smelt]

its 512 on 64 bit

there are 512 entries in a page table on 64 bit

[shadow ridge]

yeah but u cant use lower half

[twilit smelt]

there are 1024 on 32 bit

[twilit smelt]

u can

[shadow ridge]

why would you map it in the lower half

[twilit smelt]

idk

u could

its not MIPS, the fact its in the lower half doesnt by itself make it accessible to user code, the PTE bits do and you can set those to only allow kernel access

so

if youre worried that you will run out of spots to put it because youll have more than that many processes

its mapped in the same location in each process

and to access remote page tables you have to temporarily switch to that process's address space

[shadow ridge]

what happens if you try to access unmapped entries

[twilit smelt]

page fault

which can be caught and handled using KeExceptionJacket

[shadow ridge]

it would be cooler if you could use this to map / unmap pages directly

just page_table_array[6] = my_pte

[twilit smelt]

you can

thats exactly how it works

[shadow ridge]

so why not use this over traditional page table walking

[twilit smelt]

wdym

[shadow ridge]

when doing anything virtual memory related you could just use this to map / unmap pages

[twilit smelt]

yes mintia2 already does that

[shadow ridge]

instead of walking the tables level by level

[twilit smelt]

yes

[shadow ridge]

you still have to do walking in the page fault handler right

and when youre freeing the tables etc.

but thats only like 2-3 places where it's necessary

[twilit smelt]

a probable implementation of freeing a table is like

you call DeletePteRange and it deletes all the ptes inside of a page table, the pte count in the page table's pfndb entry drops to zero, which causes it to be deleted and unlinked from the page table tree

which on a recursive mapping implementation might then recursively call DeletePteRange to remove the mapping of that page table from the upper level (by removing the upper level PTE that mapped it)

which might drop its pte count to 0

so on and so forth up to the top most level

[shadow ridge]

wdym when you differentiate between page table and page table tree

[twilit smelt]

a page table is a node in the page table tree

[shadow ridge]

i see

so wait what is a pfndb

[twilit smelt]

array of page structs

describing info about each page frame

indexed by PFN

[shadow ridge]

what was a page frame again

ok at this point i should go read a book about virt memory lol

[twilit smelt]

a page frame is a physical 4kb page of memory

as opposed to just "page" which can (and in my terminology almost always does) mean a virtual page

[shadow ridge]

the page frame doesnt have to exist tho right

it could be like

swapped

[twilit smelt]

its not a page frame then

[shadow ridge]

and then the page struct would hold that info

[twilit smelt]

no it wouldnt

there are only page structs for page frames

its not a page frame if its on disk

[shadow ridge]

where do you hold that info then

[twilit smelt]

inside of memory objects

[dense vigil]

Its an array in memory that contains small structs describing information about each page of usable (real) ram

Its a software concept

Nt calls it pfndb Linux calls it memmap

[mortal thunder]

why couldn't these PTE mappings simply occupy a small part of the address space? like on armv6 and earlier, i've reserved a small amount of memory (like, 8mb of virtual address space) to allow for a virtually contiguous mapping of these regions

and you wouldnt exactly need to add Create/Delete interfaces, just EnsureExists

this is the way i did it in my boron armv6 port and although it's not the cleanest solution (since it always uses the same amount of virtual address space instead of dynamically using it as needed) i think it works

[acoustic sparrow]

i call these reserve_range and unreserve_range respectively

they arent virtually contignuous though

[twilit smelt]

well thats the whole idea!

[acoustic sparrow]

well my whole idea is that it means i can do failure-free mappings

and pagefaults

without allocating

also its kinda annoying to have to switch kernel address space when im switching the user address space

my current primary target is arm64 where they are entirely disjoint

and idk if i like the idea of either

• allocating kernel memory inside of the "userspace" page table
• switching kernel page tables when im switching userspace ones

and it also doesnt quite work with my current memory layout i think, at least without messing with userspace state

plus it kinda breaks kaslr, or makes it a lot less effective anyway

[twilit smelt]

i think the latter is fine idk why youre scared of it

@warm mural explain to me how to do tlb shootdowns good.

[warm mural]

this is what I ended up coming up with, do note that it could probably be improved:
Each CPU has an array of sizeof(size_t) * 8 entries, this array is managed as a lock-free circular queue and contains entries like so:

struct invalidation_state {
  struct address_space *space;
  uintptr_t base_va;
  uint32_t npages;
  bool active;
  uint16_t counter;
  pfn_t phys_head; // Linked list of physical pages to free
};

Each CPU also has a notification bitmask for every other cpu, indicating which states of its local array is relevant to each CPU:
size_t bitmasks[NCPUS].
Additionally, each CPU also has a CPU-local mask, indicating which CPUs sent it a shootdown:
cpumask_t shootdown_senders;

Whenever an address needs freeing:

phys_head = unmap(va, npages);
flush_local_tlb(va, npages);
slot = allocate_slot_for_state() or do_synchronous_ipi();
// figure out how many CPUs we need to flush
// cpu_mask = mask of cpus we flush, ncpus = number of cpus

slot.va_start = va;
slot.npages = npages;
slot.counter = ncpus;
slot.active = true;
slot.phys_head = phys_head;

for (cpu in cpus we flush):
atomic_set_bit(my_cpu.valid_state[cpu], slot);
atomic_set_bit(cpu.shootdown_senders, my_cpu.id);

Then, the idea is that on a transition/quiescent state, such as DPC dispatch (or context switch), you go through every cpu in your shootdown_senders, then go through every relevant shootdown state of that CPU (using instructions to skip clear bits), flush the request appropriately (via either single tlb invalidations or a whole flush) and decrease its counter atomically.

Then you have a per-cpu background thread that is woken up either by an event or periodically (something like every 2 scheduler tick), it goes through its CPU's states and frees the ones with active = true and counter = 0 and marks those states are inactive such that their slots can be reclaimed

If you can do reclamation without blocking then you can also do the reclaiming instantly when noticing the counter reaches 0

if you're worried about memory overhead you can probably skip the N x N bitmasks and put them in the slots instead, and just iterate over all slots in a CPU, the slots arent too big and are contiguous so hardware prefetching helps

[twilit smelt]

new loukeman album on april 24

[twilit smelt]

going to r/osdev

[twilit smelt]

@acoustic sparrow i could just google this but for the sake of conversation what format does the uppermost page table level take on aarch64?

you say there are two uppermost tables, one for lower half and one for upper half

are they both 1 page frame in size

[dense vigil]

they're two separate registers

TTBR0/TTBR1 iirc

[twilit smelt]

i know

thats not really relevant to my question

[dense vigil]

then what are you asking

[twilit smelt]

if theyre both 1 page frame in size

the top levels

the later dec alpha chips acquired split page tables in a similar way, which was intended to better facilitate numa locality for Galaxy VMS (which was their Cellular IRIX rip off)

[acoustic sparrow]

yeah

the virtual address space on arm64 is twice as big as the one on x86

[twilit smelt]

what was your concern with having to switch the kernel one

[acoustic sparrow]

i mean you have to switch twice as much stuff which feels annoying

even if its not like a major cost

[twilit smelt]

its funny doing a uni os course where i have to like, implement a process creation syscall in xv6

after spending like 10 years doing hobby osdev and writing 100 xv6es worth of kernel code

feels like smurfing

this is probably the most advanced ive ever been at something relative to the average person in a class im in

the assignments are considered extremely hard so theres only like 5 of them over the entire semester and you get nearly a month for each of them, and most students really do need the entire time, and i get them done in a couple hours

going to let it get to my head in order to cope for being a 23 year old junior

[digital pivot]

u probably could skip the entire course

[warm mural]

Probably skip a few required classes

It's a shame they don't let you fast track your degree if you already have knowledge

Like I don't need to follow an intro to programming in Python class 😭

[twilit ingot]

same for my java course

i could literally already do the test, but had no option to

[night needle]

this

[twilit smelt]

just to punish me for saying this i made a submission with a bug that was so bad that i think it legitimately broke the entire autograder for everybody

had to make a piazza post of shame... warning the instructors...

[random heath]

does that include the copy-on-write one :D and do they happen to be called "labs" lmao

[twilit smelt]

no i think these are bespoke assignments

i go to u of utah which prides itself on its os stuff so the profs here are less likely to use off the shelf curricula i think

[random heath]

I see

I had someone once come at me with like 4 hours left and all the assignments to complete😭

[twilit smelt]

why did they do this

[twilit smelt]

It's over I literally actually broke their autograder

That runs on their end

Like 4 hours after they released it in the middle of the night too

It's hung I hung it!!!!

[acoustic sparrow]

everyone calls it "labs" i think

[random heath]

😭

[random heath]

adhd n stuff ig

[random heath]

hmm I see

[random heath]

what kinda terrible bug did you put in there😭

and imo that's an autograder skill issue if the user can make it hang lmao

[warm pine]

Americans have a very different system to us

I believe in US undergraduate education exams are not very important (and any way multiple choice) but coursework is considerable and this is marked entirely at the discretion of the tutors

So if they call it something else its not surprising

[mortal thunder]

In my country exams are typically half, ±10%, of the final grade. (I heard that in years 3 and 4 this loosens up a bit, but this has been the rule in years 1 and 2)

And coursework depends on the course but oftentimes it's 25% ±10% of the final grade

[mortal thunder]

I'm sure it does, it felt like smurfing when I finished my first programming class back in year 1 with flying colors

We had an "operating systems" course last semester and we basically had to write lower-level userspace code for Linux, like a small and scope-restricted libc replacement or an ELF loader

I wrote the solution for one of my assignments for this semester (for my communication protocols class) literally last year for fun

[teal trench]

smh my operating systems course a year ago was super underwhelming in comparison

the labs was just writing shell scripts

the lectures also didn't go super in depth on anything

[twilit smelt]

props to Nadia Linux who popped up out of absolutely nowhere and ruined my brain and derailed my plans by talking about how cool LLFree is and then immediately got banned

i saw this paragraph and was like "wow thats a lot of hyphenization. wonder if a german wrote this they have tons of compound words and that might be how theyre translating it to english"

...

[shadow ridge]

banned from where

the mailing list?

[twilit smelt]

from here

[shadow ridge]

oh

[dense vigil]

already unbanned a few days ago

[twilit smelt]

didnt know

[dapper mulch]

that crazy wunderkind girl with small pages, "implementing" all known to humanity computer abbreviations with the pace of a few of them a day. 😁

[dense vigil]

admit it you're just jealous

[twilit smelt]

https://open.spotify.com/track/0wdzoAng9jVm135xlpBWY2?si=c0415c23faee4f51

osdev beatz

[twilit smelt]

my semester is wrapping up and im going to have my first calm summer with lots of free time in like 2 years