#MINTIA (not vibecoded)
1 messages · Page 4 of 1
we are everywhere.
yes we are
i like the two year dark ages at the start where i was struggling with toolchain and firmware shit too hard to have any os stuff going on
no idea how i maintained focus throughout that
i did start some quickly abortive projects during that period that i guess werent worth putting on the timeline bc they never did anything
honestly im surprised at how you manage to pull through all motivation slumps
the compiler making process pulled out the will to live from my will to program
i remember at one point during that i spent 8 months redoing the boot screen over and over
wait what
idk if i have any screenshots of the various designs from then lol
how did you spend 8 months and end up with the most generic ass boot screen ever
at the time i was documenting all my work in the original fantasy consoles discord which was deleted yrs ago after the owner was hacked
so i lost a lot
it sure would be very sad if something similar also happened in here
good thing the MINTIA thread never had years of documentation erased
https://github.com/xrarch/a3x/blob/db5df3d1bb32732ac04ec729a99f39f73a17f72f/src/llfw/POST.s this is what was called limn1k?
limnik
no thats limn2k
all the limn1k code predates the xrarch organization
it was in a repo called limnvm under my account which still exists but is privated
oh, the repo lies then
Boot firmware for the LIMN architecture, written in limn1k assembly and dragonfruit.
well
actually maybe it is?
i dont even remember what it looked like anymore lol
no it seems that is limn1k
i think limn2k existed at that point but it was still kind of experimental or something
limn1k was a retronym
if u look in earlier commits it just says limn and never limn1k
i dont have a ready explanation for why theres "limn1k assembly" in that repo at that point
it seems there was even an interpreted language called AUC in those days
oh yeahhh
there was a df backend to that, it was a bytecode for drivers on device roms lol
you can see the clear influence that unix workstations had on you already with that
dragonfruit/auc is your fcode and you have a device tree and everything
openfirmware eat your heart out
which i dont think i ever actually used for anything
but i certainly got test programs compiled from dragonfruit to auc running and stuff
what did auc stand for
i think it was like architecturally unrestricted code or something like that
idk
actually old dragonfruit had all kinds of weird backends
dragonfruit had a 16 bit target at one point, the R216, a powder toy cpu
i wonder if i actually committed this backend
was it only when carver or chris dorper or harrison or whoever it is (possibly they are all the same person) force fed NT gospel to you in 2020(?) that you started borrowing from that side of things rather than, as at this point, almost solely apparently from unix risc workstations of the late 80s?
i found a commit that has the auc backend and the r216 backend
they were removed in the next commit which also added an early limn2k backend
it seems thats where i got the may 2020 from in the original timeline
but a3x and stuff werent updated to limn2k until months later
so ig there was a bit of development time there
i had essentially zero conception of anything outside unixville until that point
it was a later month of 2020 like maybe september
i read ke.doc in the workbook (which i found on my own and immediately leapfrogged carver in NT understanding as a result lol) and immediately decided id start an experimental project to rewrite the aisix scheduler and stuff to work like that
and that ended up turning into a whole new os which was mintia
actually the history might still be in here?
AHAHAHAHAHAHA
oh how the tables have turned
i think i started doing osdev around that time
very dialectical movement
start with unix, throw away all unix for nt concepts instead, finally synthesise the two
but the "unified cache model" i was conceptualizing just from seeing that meter rebalance
was the NT VMM, particularly the standby list and so on
lol
does reactos actually have a standby list? i have no idea
i know they have a pv map so they aren't doing a properly nt like vmm
you would probably be at mintia 2 userland rn if you hadnt wasted months trying to get boot screens to look right
i legitimately credit that 8 months i spent as like
wiring some minimal graphic design ability into my plastic teenager brain
and its why im like the only programmer on earth who can design things that arent completely ugly looking
^
i guarantee you i spent hours staring at that single line of console text
i can see from the use of ANSI escape codes
the technical aesthetic at this point has been almost completely subsumed by like
DEC
just DEC
found some old boot screen screenshots
chronological order of all(?) the a3x aesthetic changes
well first one is a1x
for some reason i started adding a graphical ui with a cursor around the pink (second) one
and i have an unlisted yt video from ages ago showing that off
this work was just discarded lol
antedecedent
gallery isnt complete without a4x
can u tell what the buttons did in the pink one
this pink design came to me in a dream after i saw this music video https://www.youtube.com/watch?v=EOTebhPy04g
WELCOME TO WHACK WORLD
A Visual And Auditory Project by Tierra Whack
http://smarturl.it/WhackWorld
Music video by Tierra Whack performing Whack World. © 2018 Tierra Whack
due to this music video i briefly got like 50x more gay and then went back to normal
Console, boot, shutdown
winking face
turn right
and uh
not sure what that last one is supposed to be
also those buttons look weirdly 3d
doesnt fit in with the rest of the screen lol
normally the textw ouldnt be there
it would just be the 3 buttons
youd only see the text after hitting the console button
hmm
it would be better if
the console was in its own little 3d looking window too
if you want a fancy boot screen then go all in
yeah 16 or 17
im at the age where you designed that now
what is an antecedent
a thing that comes before
which i thought was a clever name for firmware
but i started thinking it was long and pretentious so it got abbreviated to the "a" in a3x and i never spoke of it again
oh
i guess at first it was like a "version 2" of AISA which was a previous project i did from like mid 2017 to early 2018
which got as far as having firmware with a boot picker, and a shitty DOS
with a hierarchical fs and stuff
damn
also
what prompted you to end the list of block devices with
"none more."
sounds like the ending to a poem
i dont know.
i think i lost something between then and now bc it would never occur to me to terminate a listing with something cute like that
im too professional now
CUTE?
it's like the darkest thing ive read today
anyways i wish you experimented more with cool looking shit like those buttons
you have never made anything as cool looking as the panic screen of Kronos
i have some dms with someone from a month before i started limnstation where i got AISA running in computercraft and these might be the only screenshots left of what it looked like lol
other than this
ope
is that a register dump
grrefef
wait what language did you write it in
yukimi
oh this was the earliest earliest boot screen for limnstation
should go in the gallery
if u notice the background is the bob ross painting thats the icon of the github org and discord
yes
thats where that comes from
i made it and i actually have some laying around
cant be worse than dragonfruit 🤷♂️
aisa dos source code
mostly in yukimi
i was not using any form of source control at this point i was just storing this stuff locally
looks almost normal!
I WAS RIGHT
it actually could not be worse than df
so wait how did you manage to get so much worse
if it evens it out any, there was no elseif
lol
no relation to limnstation aisix
i reused the name like 2-3 yrs later
aisa + ix = aisix
this dos had a relocatable executable format
forgor about that
exec() which would load some kind of old-style flat binary at a fixed location and jump to it, vs chad dexec() which i guess loads these relocatable executables
so even 7 years ago i already had cruft
firmware shell, boot or reset, power off
also the name of this language yukimi came from the name of the lead singer of Little Dragon which was my favorite band of all time in like middle school and high school
i think the process of trying to make my project cooler has also greatly depersonalized it away from me
over the years
personality-wise its like its own entity with nothing to do with me now lol
like something im digging up in the desert that was already there rather than something im creating
anyways i just cant believe you DID actually make a usable language when you were like 17
the name of jackal is probably the only personal splash that remains in the entire thing
yukimi?
i was 14
when i did that one
damn
so this junk is my 14/15 yr old me code
i can't believe it was originally called the AISA Lemon
i wish i had the intelligence of 14/15 yr old you
yeah it had a citrus theme
all the subcomponents were named after citrus fruit and dragonfruit was adjacent to that
some of the citrus names are still in there
like kinnowfb for the framebuffer card and amanatsu for the peripherals bus
like jackal
i think only those two have survived into present day xrstation
that particular fruit i think is a dubious choice
shatkora would be a super zany choice
i was also a way more goofy and zany person based on these ancient dms im crawling through to find these pictures lol
im evil now
you turned from goofy tech enthusiast to bored antichrist making the new satanos
https://www.greatbritishchefs.com/recipes/bangladeshi-beef-shatkora-recipe absolutely delicious btw
looks cool
i probably avoided that one because shat
i went through the "list of citrus fruits" article on wikipedia
and noted down ones that sounded cool
and would pop items off that list whenever i needed names for things
shatkora didnt make it
because shat, presumably
i highly recommend, it might be difficult to find, it's a dish from sylhet and available in the UK mostly because our bangladeshi community is composed in large part of sylhetis - they served in the royal navy in very large numbers as ship's cooks, and their families followed them to the UK after ww2
there are a lot of interestingly named citruses
yeah dont think im gonna find it in turkey
apparently the japanese have one called cumquat
do you use dried limes there?
these things
never seen one
lmao
did it get deleted cuz of the uh
f word
no i just dont want to distract further from the cool old screenshots i dug up
ah ok i thought the mods deleted it
waow my yt account has a 7 yr old AISA vid
shows the emulator
interesting way of showing the serial terminal
i have always lowkey seen my project as like unimaginably cringy
outside of a very specific niche of people
lol
also wtf is that desktop bg
you were always the antichrist
im saying this but most of my videos are also unlisted / private
:\
pictured: gog and magog
i vaguely remember what that was
i did an implementation of sandpiles and i noticed if i initialized the board to the maximum number of grains in each square, and then lowered the maximum number, it would cause very interesting patterns that would eventually stop. the frames of this video are the fixed points at the end of that process from lowering the "toppling" threshold from like 32 grains to 0 grains
plus some really strange compression
artifacts
makes it look a litlte otherworldly
maybe i found satan
in the sandpile
looks like something id see while doing LSD
i have a friend who is deep into like gnostic mysticism and shit to the point of being nearly schizophrenic about it
im going to send him this and see what he makes of it
ngl this could be a great ARG video
also if i allowed the toppling threshold and the number of grains to be like
real numbers
the patterns got way more intricate and indecipherable
before i got completely absorbed by htis project i was a project hopper who would do stuff like that
i probably saw a numberphile video about sandpiles and spent the next 3 weeks investigating sandpiles
and doing zero useful research because i was mostly just trying to make it do weird interesting things
lol
this one has one of the failed early aisix attempts that got scrapped and redone
in it
with ofc yet another unique console design
this is early actual aisix
the system console looked like that at first
wonder if i still have that palm tree test image
OH NEAT!
I DO! but its from a time when the framebuffer was 15bpp and its now 8bpp indexed
i just cat'd it to the framebuffer inside old mintia and it looks psychedelic and interesting
ive never seen color depth mismatch cause glitch art that cool looking
normally its just fucked up in a boring way
extracted for ur viewing pleasure
interesting challenge trying to get this to display properly lol
close...
closer but still strange
this is the original gif of that vid
i found it
fyi its still for quite a while at the start but does start groovin eventually
this disturbs me for some reason
my 14 yr old self summoned a demon with sandpiles
look at that!
horns! a skull mask! a throne! shoulders!
started out trying to imlpement some Ps APIs tonight
and now im freaking out over a demon king in some pixels i generated 7 yrs ago
If you look at it upside down it looks like an alien
you looks like an alien
Flibnort nd
(your real alien name)
totally normal day in the life of the antichrist
tries to implement some Ps API
ends up summoning demons
You know too much
"okay so this is sort of the nasty structure we build which is everybody's pointing this way at each other"
this is uh
why i dont have a complete image of how these things work in nt
that and the fact i took like a 5 month long break
but i do have to say that this presentation is well done
whoa that's crazy
im surprised that the name XR/station came about before XR/17032?
if you had support for single statement if csses you wouldn't need a specific else if
This is true but i hate those so
perazzoli being really self-deprecating about mm's design is interesting lol
i still havent finished it, i should get cozy and do that rn
im like pathologically fascinated by these things
when i first starte dlooking into NT 4 years ago it seemed like a scary black box, never could have imagined how much id ultimately find if i dug for years lol
tons of documentation, books, taped presentations, even had some personal convos with the original designers
people who say its a closed system thats impossible to understand arent trying hard enough !!
the fact they cant do phys->virt backpointers to take a physical page and invalidate all the mappings of it comes up
someone complains it makes their fully coherent network filesystem impossible to implement on NT bc the protocol relies on the ability to remotely shoot down pages on other computers
which ultimately relies on this ability locally
is this really that useful outside of the networking thing you mentioned
he lacked imagination
as it turned out you are looking in the wrong place if you want a pv_list attached to the page for this
when the space efficient solution is really to track where files are mapped
lou also fails to offer an alternative though
he basically goes "this is a deficiency we're aware of and its caused by a desire to save on memory but we might change our minds as systems continue to grow"
i think on the spot for both of them it's easy not to come up with an ideal solution like this
the guy also asked another question
i think it was "if some users are running excel off my network share and i want to update it, can they stop me from overwriting the executable file even though im the administrator, just because theyre running it?"
and lou's answer was like "yes you would not be able to overwrite it"
and the guy seemed mad that two questions in a row the answer was "no NT cant do that"
lol
in particular it was that you cant delete the old excel's name and put a new excel with the same name
that one is a bit of a nasty one
lou said this was because this isnt possible on FAT and they decided to forbid it on the other filesystems as well for consistency
NT itself has nothing against deleting a file name for a file thats still open, leaving the file's allocation on disk intact until the last handle is closed, and you can actually do this with NTFS
the issue is that the default is to open it with flags that forbid this
for this UX consistency reason apparently lol
but that restriction survives to this day
@warm pine tricks for various forms of ipi i just thought of but u might have already figured out
well just one trick
if a cpu is idle you dont need to ipi it at all, it can just check for events to crunch lazily in the idle loop
and for like a tlb shootdown you dont need to spin wait for it to process the ipi
you can just proceed
(so long as you also have a guarantee that before a cpu switches into a non-idle thread itll definitely process any pending shootdowns and stuff)
this helps stop an idle second processor from being a burden on the first one
actually
the idle loop doesnt need to check at all
i think when a processor switches into the idle thread he can set a flag on himself saying "yep im idle"
if another processor wants to do a tlb shootdown on him he can do this:
if target not idle:
ipi him and spinwait for completion;
else:
target pending events |= SHOOTDOWN;
if target not idle:
repeat this whole thing;
and then the exception handler has this at the start:
idle = me.idle;
if idle:
me.idle = false;
if pending events:
process them;
...
me.idle = idle;
i think this works
here the SHOOTDOWN event would do a full tlb flush (you were idle anyway so this probably isnt a big deal) since the sender processor immediately goes off and does other shit and so his information on what specific page(s) you should flush etc goes away with him
you have to do this in the exception handler and not just when you switch away from the idle thread
because otherwise you might, while idle, process a device interrupt that causes you to refer to like a mapped buffer that has stale tlb entries because it was put in place from another processor
so before processing any interrupt ig you need to do this
specifically interrupts i suppose
not really any other exceptions youll be taking while idle
entering some kind of power-reducing mode makes it more problematic
nowadays kernels are looping only for a short time then halting the cpu
what i said after that makes that irrelevant
yes, in practice you need to do certain things as you enter such a mode anyway so you can publish something to say "i'm abandoning the idle loop and going into such a mode as the only way to wake me now is by IPI" and then have a barrier between that and actually halting or whatever it may be
actually this is even better on such a system, because coming out of those low-power states is so slow
and you dont have to spin-wait for it to do that lol
youll just know itll flush its tlb before the next time it might access this virtual address range you flushed
while handling an interrupt i set the idle flag to false in this pseudocode bc an interrupt handler is not that kind of quiescent state, and you need to go back to the normal ipi & spinwait thing
but who wakes the CPU up from the idle loop
because surely the idle loop pauses the CPU execution until an interrupt happens
i still dont understand?
even after the explanation
yes
oh wait this is only for the TLB shootdown right
because yeah the TLB shootdown doesn't really need to be instant
you just need to trust that there will be no code that tries to access the memory region you're shooting down between the TLB shootdown request and the time the actual shootdown is performed
ok i implemented it
high tech test: every time it avoids sending a shootdown IPI it prints the number of the target cpu who was spared one
is this novel
its not right
i always get my hopes up that i just invented something whenever i have a new idea i havent heard of in something else
in this case using the idle loop as a quiescent point to avoid IPIs
im 99.999% sure if its a good idea that linux at a minimum is already doing it though
yeah I'm pretty sure Linux is doing something similar to this with what they call lazy TLB
also wtf are those framebuffer artifacts?
its the fireworks test
they r fireworks
oh lol, that it intentional
it spawns like hundreds of threads
it stresses the scheduler
and each of them pilots a little firework pixel
it also stresses my allocator (as well as my system space range allocator which is why its of use here)
@mortal thunder wrote it for his OS at first and then rewrote it in jackal for mintia2
nice.
i am very proud of the fact that it like
works really smoothly on my 20mhz fantasy computer with hundreds of threads running in the guest os lol
are there any cool concepts you implemented in mintia i can steal for my own kernel? 
i mean like, most of the time the threads are asleep. but at 20mhz its slow enough that a few dozen are ready simultaneously at any given time
it also stresses scalability of the smp scheduler and stuff
and of timer management and whatever
mintia2 is like 80% composed of weird and cool concepts
yeah but it's not like i can just steal all of that, i don't really want to create mintia2 but written in not jackal
unless it's a challenge
but jackal translates quite nicely to C so it would be a lame challenge, except for the part where it's not running on xr17032 or fox32
mintia 2 will run on amd64 too tho
yeah if i get to it
or will beats me to it with his elf->xlo + his own amd64 port
i should get back to the amd64 mintia2 port
with my half assed toolchain shit
i should get back to boron someday
but i don't feel i'm quite ready yet to invest time into studying how memory management works in NT well enough to replicate it
i tried getting back into my project and failed hard
i should pay will to write the backend
yes but im shit at compilers
im not smart enough to program anything including and past the IR stage
@warm pine i no longer send tlb/dcache/icache shootdown ipis to idle processors https://github.com/xrarch/mintia2/commit/66f5299f8d17bd7f9484c362555394cdbe53a1c9
i just stand there staring at the screen tearing flesh off my fingers thinking very very hard about it
lol
why was it calling a function in the first place
idk
shouldnt it have been an assembler macro
its only done in assembly in that spot
on fox32
i also probably wasnt used to the new assembler which lets you do expressions and stuff lol
neither my nor ryfox's old assembler let u do expressions
hmm what reason would you have to flush (specifically) the icache of another processor?
of another processor u mean
yes
ok so basically
on xr17032
the dcache is not coherent with dma
so you need to shoot it down after dma into main memory completes or youll see stale data
the icache is not coherent with ANYTHING
so if you write instructions into a page, you need to flush the icache before you can execute them
you want to do this on all processors in case a thread running that code migrates to a different processor
why not?
would it be hard hardware wise to just make dma invalidate the cacheline
damn
the cache details of xrstation are based on mipsco workstations from the late 80s
back when "mips computer company" existed and mips was their architecture for their workstation computers
ive seen some evidence that really early in development the architecture was formally called Rx and people referred to it informally as "the mips architecture" like in reference to the company mips
and then "microprocessor without interlocked pipeline stages" was a backronym to legitimize that name for the architecture
that could also be completely wrong.
but i have seen the mips architecture referred to as Rx in usenet posts from the mid 80s
Rx because the processors were called R2000, R3000, R4000, etc
so like "x86" being because 8086, 80186, etc
what did the R stand for?
probably RISC.
creative!
yes
nice
this part at least sounds wrong because it implies they called their company "MIPS" for fun
like for no reason they just thought it sounded fun
which is unlikely
so it probably did stand for that originally
also iirc the name is literally wrong
and even the R2000 did have interlocks in the microarchitecture.
but dont ask me where i heard that i probably saw it in a usenet post that ill never see again
with 8 processors running the fireworks test i still average 1000 ipis per second lol
but thats across every like pair of processors
assuming a random distribution you can divide in 2 to get 500 ipis received per second
divide that by 8 to get 62.5 received per second per processor
thats pretty bad still
well
no it makes sense
that just reflects the number of times it goes idle and then quickly get poked by a firework thread waking up and being enqueued to it
with the number of threads waiting on short timers i think 62.5/sec is pretty much what youd expect
Linux maxes out at around 2M I think
i love smp so far
btw
its really fun
im good enough at locking from 3 yrs of doing old mintia vmm that i can say "i like smp" rather than goin glike AGGGGHHHHHHHHH AHHHHHHHH GODDDD WHYYYYYYYYY
but yeah the new opportunities for cool optimizations like the idle loop thing are fun
its also just hilarious this is increasingly the most overengineered fantasy computer of all time
will xrstation get acpi someday?
well you have like a year to get ahead
before i can even compete
so
shouldnt be hard
honestly I'm surprised that my kernel does so well on the uACPI benchmark (when it's not executed on skill-issued AMD virtualization), because I haven't profiled it like at all lol
it will be #1 in not immediately shitting the bed if you mmap and then touch every page in a range as large as physical memory
and like
#4 on the uacpi leaderboard
the uacpi leaderboard is just how fast your cpu is and how much your allocator values speed over memory usage
that leaderboard is poorly put together actually
i was about to mention the fact that it doesnt normalize in any way against varying configs
it lists ppls cpus but like
thats not the only factor
make sure to map everything as uncached for a free 0.33x score multiplier
no way to make it more fair without a centralized runner
it started mostly as a meme anyway
the real benchmark would be how much fps it gets on factorio 
gotta port factorio first
I get 0 fps because it crashes during initialization
basado
i decided earlier that after im satisfied with mintia and xrstation in like 2 yrs ill make a 3d game engine from scratch w/ all tooling from scratch, might even do it in jackal
so watch for that in 2 years.
thatll take me another 5 yrs im sure
lmk when it's better than godot
funny how you're telling that to will
but yeah id rather have the convenience of opengl than deal with vulkan
Will has plenty of will, im sure will can will himself to work with vulkan
but can you will will to will himself to will working with vulkan in the first place
so I found this on the inter-webs...
https://jackcc.sourceforge.net/
"The Optimizing C Compiler for the Jackal 3.0 ISA"
lol
unfortunately i went to take out the trash
and i slipped and fell on my ass
and kinda writhed on the ice for about 30 seconds with the wind knocked out of me
anyway
i have forgotten what mintia is.
so bye.
imagine not having a massive cuchion of fat on your ass
i do
that makes you invulnerable to fall damage
but it magically missed that
and went straight for my tailbone just above that
u see
that sucks, did you break anything?
take care
really.
damn
are u ok
you should sue yourself for not taking care of the ice
My left arm is completely unusable
And I'm left handed
So that's bad
holy shit thats one of my worst nightmares
you should go see a doctor
are u sure it isnt fractured or something
i am building a library on STREAMS, let me know if you are interested in some of the papers
mostly they are about ways to make TCP/IP stack scale with STREAMS
it turns out that the reason the SVR4 one didn't was because the whole tcp/ip processing was multiplexed onto a common stack - and parallelising what was a common stack is not easy
I am
great
https://dn790006.ca.archive.org/0/items/1993-proceedings-winter-san-diego/1993ProceedingsWinterSanDiego.pdf see page 96 for some stuff about the subtleties of locking in a STREAMS stack and why they decided to go for locking the whole of a stack at once in intel SVR4MP, they also deprecate "weightless processes" (a catchall term they use which in our language is "any routine not called in process context")
and these all concern scalable TCP/IP implementation concerns with STREAMS
https://docs.oracle.com/cd/E19457-01/801-6679/801-6679.pdf the STREAMS programmers guide from 1994 will make for good bedtime reading around the exact way the STREAMS subsystem worked in solaris at that time
I broke my elbow
Radial head fracture
My arm is in a splint
This will impede mintia
i slipped on ice while taking out the trash lol
I was half expecting a sport injury like what oberrow did a little while back, but then I thought about it a bit and realised considering your projects, that is quite unlikely. I guess I hope it isn't too bad
Wdym considering his projects lol
I was trying to say it in a non-offensive way, but making an OS in a custom programming language on a fantasy CPU suggests he doesn't have much of a life
i mean yeah thats a common way to cope with someone being way more accomplished than you
"they dont have a life!"
actually im just better but thanks
that sucks
please be more careful
shit like this tends to happen when everything's going alright and you're not paying much attention
were you given a time range of when it might heal?
make no mistake frosty515 guy, you could never have made my project no matter how much of a life u had or didnt have. just fyi
like a month
have u considered what you can do in the meantime
yeah but thats ur dominant
and im left handed
btw do you think first and type fast
or do you type a little, think and repeat
for me my typing speed doesnt even matter because most of the time im looking at the code thinking anyways
unless it's something trivial
you miss the feet pic arc
That sounds easy to miss out on
there are lots of highly capable ppl in the world, who -- given enough time -- can do great projects
that specific guy isn't one
mostly he just annoyed me and I was dunking back though
I don't actually know anything about him lol
STREAMS seems to have a lot of opportunity for glory
Yeah, looking back, what I said might have been a bit offensive. I apologise for that. We all know you are capable regardless of whether you have a life or not.
malding 
u couldnt do it either
now that i have the beginnings of intuition around it, it's clear to me that the bad press it received as far as its compatibility with SMP scalability and zero-copy is completely wrong
i won't argue that it is more efficient to use STREAMS to implement a TCP/IP stack than to do it entirely bespoke - that would be an absurd argument because bespoke always > turnkey for performance with all other things equal, but there's just nothing that stops you doing a good job there
im still more excited for implementing a pty with streams
bc my old duplex driver thing was such a mess
pty and pipes
it will be beautiful there
but i do want awareness of how to do non-bad networking as well
so i dont have to redo it later
have you thought of how to avoid recursion in streams modules
its not as easy as with the IOPs because the streams part is like data-driven rather than request-driven
well idk i guess you can do it basically the same way where you have a per thread list of streams data packets which are annotated with their destination and you just run down that and send them off and repeat until that list is empty
and youd probably represent this whole thing with a one-level IOP
at the level of the stream head
for the put routines it's not so bad since you can have an outer loop and have put routines return e.g. "schedule my service routine", "take this mblk and others attached to it and pass them to the next level's put routine"
if that makes sense
and with service routines you could allow them to make a call into the outer put loop when they want to put to the next queue
tcp/ip stack will be the key director of how i recontextualise STREAMS to my kernel, so watch this space
i think having the streams modules directly deal with IOPs would be inappropriate
bc of what i mentioned
i am codeveloping them
about them being data driven instead of request driven
so IOPs arent really applicable as opposed to a bespoke data packet structure that you pass around instead
i agree, it doesn't fit the concept
STREAMS are streams and you only introduce a level of complexity by pretending that there are discrete requests as such (there might be logically discrete packets from someone's perspective if it's a datagram stream but that's not well-represented by IOPs)
your research into them has gone deeper than mine, i just read about them while racking my brain thinking there has to be SOME way to do the duplex driver thing in a cleaner and more generalized way and found that streams was the way
also a lot of questions were answered by a passage in one of those papers about using a stream message with type READ sent down the write queues when you initiate a read lol
explains when you set up zero copy stuff for devices where its relevant
there are certain things that simply don't make sense for character devices/socket type things like cancelling an individual "request" and that makes life a lot easier in some ways v.s. the IOP world
that one was a revelation
i saw it first in the STREAMS Programming Guide
its still important to be able to cancel an IOP though and youd need a way to dequeue anything in the stream that depends on the user buffer on your way out
yes, it will be fair to do this very roughly though with the FLUSH mblk that should just empty everything out
since with character oriented interface you would never actually want to be able to just cleanly pluck something out of the queue but leave other stuff pending. if you are trying to cancel a write then something has gone horribly wrong and the stream is going to be abandoned
well but like
for reads i think it is slightly more debatable
in old mintia it could happen that u start an async write to a tty over the serial port and then your process is terminated before that finishes
the write request just gets dequeued immediately
and that was convenient
for a situation like that, if i really didn't want to just clear out all writes queued, i might just accept the MDL will persist for a short while
and it's a higher-level duty that the write queue isn't filled up so much that it burdens memory
i was thinking the cancel routine in the streams driver (which provides the one-level iop facade) would lock the stream the iop was targeted toward, and walk a list of buffers dependent upon the mdl and mark them all with a "cancel ASAP" flag. next time a module sees it, it just discards the buffer and decrements a count and completes the IOP with like STATUS_IO_CANCELLED when it hits zero
so you call the cancel routine and then wait for final completion
so you can get completion much earlier than you might otherwise but still leave the buffers in the custody ultimately of the stream
with classic STREAMS it is just sort of assumed that the arrival of some data to the queue is your point to complete read(), they didn't have an asynchronous interface to it so it waited synchronously at the level where the stream head interfaces to the VFS or the STREAMS-specific user API if e.g. flow control was asserted
so for us that is not an acceptable solution, and that's where we would instead have IOPs queued up
disk io cancellation works similarly
and cancelling the IOPs there is easier to do, since it's just the stream head logic that needs to take it off the queue, it's not made it into any of the modules
its easier if you just pin anything you send down the stream since then you can just leave those pages orphaned if the source process terminates early, and they get freed when the refcount finally hits 0 later
i was hoping to sometimes avoid that though
that also has accounting problems
leaving those pages orphaned
id rather hold up the process until this cancellation process is done
you could do it more like
i think it's a bit of an ugly thing but not more ugly than leaving data in a ringbuffer
each streams message has a pointer back to the originating IOP
but your idea for expediting the process is not bad
and thats the only direction you have to look in
you set a "cancelling" flag in the IOP
and the modules check that before they do anything with a message
if they see that they discard it and a count in the IOP is decremented
you could put this in your routine for popping a message block off a queue
so its transparent to each streams driver
and the IOP's existence is guaranteed by the fact theres an associated message block
so no locking required to check that flag
yes, when we take in an iop and construct mblks out of it we could increment that count
and the flag is advisory so yeah
if it turned out absolutely necessary to get robust cancellation, id do a big lock for the entirety of streams
i dont think it is though
but for me personally robustness >>>>>>>>
i thought about it a bit when i was out for a walk and while it would be nice in a very abstract sense i just didn't see what value it would add either to pipes, to sockets, or to ttys to be able to undo writes that, as far as the application is concerned, have happened
again its a memory accounting issue
if you did a 1mb write over the serial port and then bailed out, it sucks to have 1mb completely unaccounted for until the serial port has gotten around to it or whatever
i feel that the accounting there should be done in terms of the receive buffer/send buffer size of the stream
for a 1mib write presumably some of that would be held off at the stream head by the flow control mechanism - the IOPs are queued up at the stream head, and a bit of them at a time is dispatched to the first module's queue, enough to fill to a high watermark, then when the low watermark is reached, a bit more of the IOP is dispatched
and the send buffer of the stream is accounted for as wired memory
this was also important to me because i want to make sure that i allocate a pool of mblks and dynamically size the pool so that there are mblks enough for all the existing streams to not run out of them at an inopportune time
this is easier said than done because with networking in particular you have parameters that can vary around things like the MTU, so knowing how many mblks you need is a dynamic question
also remember that if your mblks can point to user buffers that the user can still modify, and you do a synchronous request, the thread has to be held up (the IOP not completed) until the last such mblk has been consumed
so you need a count of derivative mblks off the IOP anyway for this purpose
shit, i completely overlooked this
but your solution solves that problem
you can avoid this if the data is small though
if its like <256 bytes you can just do a non-derivative mblk backed by kernel heap that you copy the data into, and let the thread carry on
you can complete the IOP right away
as a matter of fact
and leave the mblk orphaned off in there until it gets out into the world
so for an IOP pointing to an MDL which won't be turned into kernel buffer copies, you've got to keep track of it. but it's still data-based keeping track of things, rather than having to know at what is potentially a very low level when a request is really "complete" in some very specific sense, because with the character interface, "request completion" is when you've gotten a request in its entirety onto the first module's queue, but with an MDL the iop completion must be deferred
doign a read is similar
or identical
when you send down the READ message you can attach an MDL to it
to read into
and this message adds to the derivative mblk count of the IOP
maybe
these i'll be able to get very cheaply since i am taking a page from the BSD tcp/ip stack where there is a pool of mbufs (roughly equivalent to mblks) with preallocated data block of something like 150 bytes, and for a lot of the network stuff you do these work well, space can even be programmed to be left at the start of the mbuf, enough space for the tcp and ip and ethernet layers to insert their appropriate headers
there is one interesting concern with reads which is batching up
you can see easily how it would be terrible for performance when there's bulk transfers going on but every time you get a hundred bytes of data you complete an entire IOP
now in some cases you might direct an iop to be completed only when it's completely filled up. but that is not always what you want and some level of intelligent behaviour would be useful. i haven't figured out how this will look yet exactly. there's a very fine line that if you cross you'll introduce unacceptable latency
one of the papers i posted yesterday mentions the problem i think
also youd only send a READ message to begin with if the module at the top of the stack has indicated it wants to receive them probably
otherwise you just leave your IOP set up at the stream head and lazily collect data into it as it comes down the chain
plus reading in whatever is queued up already
this is how like a pty would likely work
at least in cooked mode
(the corollary of this problem, on the transmit side, is what nagle's algorithm tries to deal with - and it's turned off by latency-sensitive programs)
raw mode ptys and also pipes (prob sharing a lot of the same code) would like to receive READ messages so they can set up one-copy stuff prob
so when u write to a pipe thats being read from, that turns into literally just a MmCopyVirtualMemory between the writer process and the reader process inside the pipe streams driver
cooked mode pty will be allocating new buffers for cooking bytes into in both directions
so this isnt applicable there
i got basically one-copy working with cooked mode ptys in old mintia but it was soooooo clusterfucked lol
and probably not even faster cuz it required special copy functions that would cook bytes on the fly
i was impressed by the old duplex driver (as much by what i didn't understand as by what i did) but i am really excited to see the terrible beauty that STREAMSified duplex will birth
it was me learning why the NT guys didnt even attempt an async posix pty driver
this new streams understanding will make it a lot easier though i think
they did have a tty implementation but it was inside the posix subsystem server and operated by synchronous IPC iirc
and not the NtReadFile and so on
thats probably not super healthy
that sounds bad but the splint is keeping it in place while i do it so it doesnt hurt
so its probably fine
and using my left hand isnt in and of itself problematic
true
cuz the problem is all the way at my elbow
i hope it heals fast
thx
i broke my left arm when i was a kid
as do i
the worst part was that i could barely move it after removing the cast
since my muscles basically went offline
during the montth
i hadnt broken anything and was proud of that
then my legs slipped out from in front of me while i was taking out the trash at like 4am
get fucked
and now i have
membership of the special unbroken boy revoked
lesson learned, never take out the trash
no more free pizza party for you
yeah i was surprised when will said he takes out his trash
i just keep it in my room
true, just deprecate it and leave it around
im sitting in a huge pile of garbage rn
What you got? Sprain?
the wind was knocked out of me and i involuntarily went like "awwhhhhh fuuuuuuck" and somehow someone taking a walk witnessed this
he broke his elbow
yes
i didnt even notice i hurt my arm i just got up and kept taking out the trash
i imagine it hurts the most after u break it
it didnt start hurting for about an hour
it wasnt the leg so no cool cane
it was a radial head fracture i think
did they charge u $9999999999 for the visit?
ty the time between the message and the edit placing that comma really made my day
which explains why i didnt get what infy is describing bc its basically a crack in the knob that goes into the elbow joint and so didnt directly affect movement until the swelling set in
Full recovery can take months, take care of your arm
yeah mb lol
a crack in the knob? damn man thought it was only bones you broke
i had to wait a long time to go bc only the ER was open and it would have cost like $4000
but i went to the urgent care instead when it opened and i think that was like $50 out of pocket so.
are u insured?
ah ok
so it is not illegal to not have health insurance?
being poor is illegal 
it is legal not to have health insurance
it is illegal to drive a car on the road w/o car insurance
It hurts a lot, the bone's sensitive periosteum (a membrane covering the bone) is richly innervated by nociceptors
ill keep it in mind for the next time I consider hurting my richly innervated nociceptors
for me it didnt hurt at all until the surrounding flesh swelled up after like an hour so explain THAT.
adrenaline
i guess?
adrenaline for an hour?
idk what kind of drugs youve been using
maybe they combined into a painkiller
LSD?
what
what th efuck
can i tell you guys about a memory from world of warcraft from when i was like 4 or 5 years old
it's your thread afterall
great
lol
i only tell stories if it derails something inappropriately
no it's against the rules
Stress induced analgesia, your body activated descending inhibitory pathways and released endogenous opioids
yeah maybe
me when i use smart words
ohh
is it possible that you happened to swallow one of these when you fell?
@zinc trout fact check pls thanks
it might have slipped in your mouth when you were on the ground
man just give up the bit
in the earliest versions of world of warcraft from the early '00s, before the first expansion, the map for one of the two continents (the Eastern Kingdoms) looked like this (ignore the red lines i got this off a random reddit post)
wtf you played world of warcraft when you were 4 years old
this looks a lot like a map that someone i knew from south korea shared with me about 10 years ago
Its because he's awesoem dw about it
it was called "Hell Joseon"
and wth, doctors sit around and write low level code
if u see the bunny ear looking part in the top right, that part was very significant in the lore from warcraft 3, but hadnt been designed or put into WoW yet, so the bunny ear looking area was just blank and non-interactive on the map. there was a pass from one of the southern zones up north into it, but it was blocked by a fallen tree trunk. so obviously as a like 4 year old i was fascinated and wanted to get up to that big area
gotta grind
Just hop on wrath, then u can visit the ghostlands/the-other-zone-up-there to ur heart's content
i had an idea and went to the northwestern coast of the continent and walked east along the coast for like 30 min and then i looked at the map and it told me i was like right here
even though i was still on a coastline
next to some mountains
this was incredibly creepy to me
especially when i found a totally empty tower along the coast with seemingly no purpose, a 30 min walk from any game content in either direction
you cannot (legitimately) access any of these places after the first expansion (the burning crusade) or later
they are blocked off by invisible walls
and the northeastern part of the continent was totally remodelled when they put something there
the strange thing is, they didnt put it here. if you walk along that coast, this is still exactly like this to this day, just blocked off by invisible walls.
they did some non-euclidean shit where the aforementioned mountain pass blocked off by a tree trunk was unblocked and if you walk down it, you go through a portal that takes you to a totally different place that isnt even on the same continent technically
so this place became even more limbo and strange
its like an alternate reality
a teleport
ah alr
what is the name of this location?
technically "Quel'Thalas" but if you look that up, youll see the new quel'thalas accessible by the portal i just talked about
theres two quel'thalases
one is a like (for lack of a better word which id use if i could come up with it) liminal, backrooms quel'thalas and the other one is the playable, populated quel'thalas
this is backrooms quel'thalas on a detailed map that was official merch during that early period of the game
but when they added it as a playable area, they changed the map
but they didnt change that actual coastline. like i said, they turned that mountain pass into a portal
and the coastline is still fucked up and wrong and they hide it from you with invisible walls
the creepiest part to me, of when i was trying to find this place as a toddler, is that there was no music set for that distant unpolished part of that coastline and so it defaulted to the music for the closest creepy-themed zone which was this https://www.youtube.com/watch?v=E-fMhVtaYVY
The Western Plaguelands are located in northern Lordaeron, wedged between Tirisfal Glades to the west, the Eastern Plaguelands to the east, and the Alterac Mountains to the south. It is filled with a smoky gray mist and the creatures here are blighted and sick, angrily attacking anyone who wanders too close with surprising ferocity. The land is ...
so id just be hugging the coast with some tall mountains walking along a coast for like 30 min with this playing while the map says something "wrong" about where i am
so that was nightmare fuel as a little kid
importantly the continental map is still lying just differently
somehow to this day this is still strangely fascinating to me
sounds like walking around in gm_bigcity
never mind, hell joseon turns out to be different after all
but the style is very similar to these wow maps
thats a map of hellfire peninsula from wow
it's hell joseon
with korean text superimposed
you go through the portal on the right to get there
and technically its another planet
theres space aliens in wow
but theyre orcs so its still fantasy
you still log on to wow?
not relaly
its part of outland.
i still remember this despite not playing the game regularly in literally 12 ish years
this actually ties in to what i was saying
what's happening on this map?
i mentioned how they "added quel'thalas" by putting a portal in that mountain pass and blocking off the fucked up, incorrect coastline with invisible walls
and i said it wasnt even on the same continent
the quel'thalas you get teleported to is actually part of the outland world files
just like
50,000 miles to the southeast
both were added in the same expansion
if you edit the world files to add a land bridge you can walk from one to the other and confuse the shit out of people when you show up in outland as a level 1 blood elf (because the portal to outland is not traversable until a much higher level) and then get your account banned from the game.
there are videos of people doing this.
basically in the lore it was a planet that got blown up and now it looks like that. dont question the physics of it its magical
magic blew up the planet
In this video, we will look at why we still can't fly in Quel'Thalas and the Azuremyst Isles even after the cataclysm revamp where flying was introduced in Azeroth. Enjoy!
=======================================
Music used:
Eversong Woods and Azuremyst Isles Music
Silent Partner - Yard Sale
Links:
Follow ...
this is a video about this topic
if you cant tell i have a like secondary or tertiary autistic fixation on strange video game corner-cutting
which is probably 100% downstream of that experience i described of the map being "wrong" when i was like 4
now its like spiritually fascinating to me forever
for the current year it doesn't look very good
the video is looking at content from 2006 or 2007
people are still sitting in locations this old?
they r still integral to the game
you still quest through them to level up ur new characters and stuff lol
i see
theyve built the content out in layers without ever completely obsoleting old content
if they do, they remove it, and theyve removed very very little since the game's launch in 2004
this is one of the things that makes wow really interesting for data mining and stuff
at this timestamp he flies between hell joseon and new quel thalas https://youtu.be/9rFE8HzIijY?t=224
In this video, we will look at why we still can't fly in Quel'Thalas and the Azuremyst Isles even after the cataclysm revamp where flying was introduced in Azeroth. Enjoy!
=======================================
Music used:
Eversong Woods and Azuremyst Isles Music
Silent Partner - Yard Sale
Links:
Follow ...
dont ask how he did that he probably used some custom world viewing tool that a group of 5 wow data miners keep under lock and key
apparently this guy died of cancer
like a year after making this video
so thats some information
after im satisfied with mintia and xrstation im gonna make a shitty mmo thats extremely shoddily put together on purpose so its fun to explore places youre not supposed to get to.
i will spend 8 years doing this
wow: lotr edition?
Which part
idk what i was referring to at this point lol
probably some of the "why it hurt only after some time part"
and the other stuff that was said
idk you're the medical guy 
yeouch, hope it heals quickly 
thanx
Coulda been worse I could have died!!
hope u will get better will
^^^^ hope u heal quick
anything that does not kill you makes you stronger
omg
tech bro pseud civil war...
the blue check Nazi eugenicists are turning on their master
lmao
theyre desperately trying to sound serious and insulting
but it just ends up being funny af
i want to repeat that now, lest you think it a typo.
its scary the type of idiot who would take this super seriously is like the next ruling class
has the same energy as a middle schooler going "FUUCK YOU. Bastard. Hehehehe. You think i misspoke? FUCK YOU!"
but theyre trying to intentionally lower their voice
not really
its just funny how dumb people talk when theyre trying to sound smart
who gaf 🥺
the fixation on IQ here is a giveaway
it's funny how he's talking as if theres a way to predict someone's IQ from their life history
and thinking it's a good measure
ahhh youre not adding anything stop overanalyzing my shitpost
cta :)