Astral

[hearty fulcrum]

everyone tells me that it's the fastest

it's the fastest one in whole america

[grand shadow]

alright now its as fast as before singlecore

aaaaanddd

using the new queues and interactivity and stuff

I could already see an improvement in how responsive it is now under load

I need to get a before vs after

still missing smp stuff in the scheduler though

that will be the next things that will be done

[analog berry]

[prime mulch]

Are you real

[prime mulch]

Is this ULE

[grand shadow]

ya

[prime mulch]

Siiiick

[grand shadow]

I remember talking to you about it like a year ago and I've been meaning to do that since

finally getting around to it

[prime mulch]

Oh god don't tell me it's been a year

[grand shadow]

also updated uacpi

the uacpi I used was like a year old 💀

[prime mulch]

So is my little cousin that doesn't mean I'm gonna put him in the dumpster

Freak

[analog berry]

Bruh

[grand shadow]

when I first saw mlibc didnt have sys_nice implemented I was surprised that no other os bothered with that, its such a simple syscall

and sudo needed it to at least return an error to work (mlibc outright panics, I have it stubbed out in a local patch)

but I will implement it fully for astral and upstream is soon

[grand shadow]

I also should implement magazines in my slab allocator since I am also doing all this smp work

This will be later tho

Cant do multiple things at once

[analog berry]

Hire hyena to do stuff for you

[prime mulch]

Hirena

[prime mulch]

I'm bad and not useful even if paid money

[analog berry]

Doubt

[supple hatch]

HUGE doubt

[grand shadow]

Worked a bit more on picking a cpu to run a waking thread, should have that done today if I end up having time

[grand shadow]

My ultimate goal with this will probably be a multicore build of gcc

If it does that it would most definetily be stable

[proper meteor]

your weren't building gcc on multicore already?

[grand shadow]

That was single core, multicore made it slower cuz of the scheduler nuking the caches

But with the new scheduler that shouldnt be an issue anymore

[proper meteor]

right, I was implementing SMP as an attempt to be faster precisely for building gcc but I see I'll have to worry about the cache

why was it nuking the cache?

[grand shadow]

Constantly switching a thread back and forth between two cpus

Cuz of single queue

[proper meteor]

you didn't have a per-cpu queue?

[grand shadow]

Nope

That is now being rectified

[proper meteor]

it indeed sounds like an issue

[grand shadow]

My per cpu queues are already done and upstream, I just need to do the cpu migration stuff and I think Im golden

[proper meteor]

cpu migration is the next step for me after tlb shootdown

[grand shadow]

Im doing the "pick what cpu after waking up a thread" part atm and after that Ill make idle cpus try to pull some work from overworked ones and after that I add a load balancing that happers every second and then thsts it I think

What Im doing there is just
Thread has hard target on cpu > walk back topology until root (at some point I can add a flag for the architecture to define the limit) searching for a close idle cpu that can immediatelly run the thread > search for an idle cpu system wide using a bitmap > search for the least loaded cpu that can immediatelly run it > search for the least loaded cpu

From most preferential to least

Aka what ule does lol

[proper meteor]

ule?

sounds good to me btw

[grand shadow]

The freebsd scheduler

[proper meteor]

I'll have to figure out how to explore the CPU topology at some point

[grand shadow]

I just use cpuid to find that

Its in arch/x86-64/cpu.c if you want to take a peek at some point

[proper meteor]

thanks!

[grand shadow]

Its not old astral code so it should be mostly readable

If you have any questions lmk

Oh yeah the code for inserting nodes into the topology tree in the end is a bit wrong but I should push the new fixed code today

[proper meteor]

I'll have a look in approx 3 or 4 hours

[proper meteor]

what does "immediatelly run it" mean? isn't it equivalent to being idle?

[robust geyser]

well that's what they said, an idle cpu

[grand shadow]

If the thread would be able to run there immediatelly, either the cpu being idle or the thread being inserted having a higher priority and being able to preempt the running thread in that cpu

(which is effectively the check since a cpu will always be running a thread, cuz it has the idle thread with idle class and nice 100)

[proper meteor]

thanks

[grand shadow]

That idle shouldnt be there in the walk back I did a brainfart

The only time itd search explicitly for a cpu being idle is using the bitmap

[proper meteor]

well that indeed looks easy to implement

[grand shadow]

Yeah it was mostly just annoying cuz each cpu manufacturer has a different way of doing it

[robust geyser]

there's only two vendors you have to worry about anyway :^)

[gritty shore]

Why do you need a bitmap

cpu being idle = cpu can get preempted immediately

So there's no special treatment for it

Mhm well I guess it's better to favor idle cpus vs ones that have low priority workers, as you can avoid a cache flushing

[grand shadow]

Yeah

[proper meteor]

yeah, I ain't supporting random chinese x86 manufacturers

[proper meteor]

@grand shadow why is rewrite your default branch on github instead of main?

[hearty fulcrum]

probably didn't bother to change it after a rewrite

[grand shadow]

Yeah

[proper meteor]

@grand shadow is there a reason why you have a different structure for CPU topology instead of using the per-cpu structure for that?

[grand shadow]

To allocate the non cpu levels

[proper meteor]

oh yeah

[grand shadow]

alright I can see an improvement in glxgears fps from smp 1 to smp 2

compared to before starting the rework where it actually slowed down

and thats without actually implementing the thing that tells userspace theres more cores

so glxgears still runs single thread, the improvement is just from being able to run more things

[bronze nexus]

nice

[grand shadow]

certainly a big improvement compared to... whatever the fuck that was

[neon crane]

how come?

does glxgears use more than 1 thread

[hearty fulcrum]

other programs use threads too

distributing them on multiple cores improves performance

[neon crane]

fair fair

[proper meteor]

glxgears uses opengl I believe? which likely wants to distribute computation across cores

[neon crane]

depends on the specific implementation

[robust geyser]

llvmpipe will definitely benefit from parallelizing the software rendering

[neon crane]

"llvmpipe" sounds like something that belongs in clang

[robust geyser]

yeah somwehere in there

i believe it's part of mesa

uses llvm for codegen i believe?

[grand shadow]

Theres so many places I can improve scalability in astral that I feel like the scheduler rewrite is just the beggining of it

[grand shadow]

kernel of theseus, if I rewrite enough components is it still the same kernel as before?

[prime mulch]

erm, thats easy

[proper meteor]

Since I started mine in C and it's now in Rust, I have the same question

[bronze nexus]

Hello, I too join the kernel of theseus party

[opal lotus]

food for thought: it never is

[proper meteor]

each new commit is a new kernel

[grand shadow]

currently having a very fun bug where 4 bytes of the stack randomly get set to 0

[prime mulch]

could it be due to corruption of the stack frame due to reception of an exception/interrupt

[pine ledge]

What is int_no in your stack frame when that happenss

Maybe idk

[grand shadow]

yeah I'm thinking of that, it cannot be scheduling related since IPL_DPC

[prime mulch]

confirm you're actually masking off scheduling with this lol

it'd be unfortunate if you're not

and you didn't realize

[proper meteor]

@grand shadow astral running minecraft when

[grand shadow]

its not any interrupt funkiness either, the rip of the last interrupt before dying where it does is in the idle thread

[grand shadow]

when I fight off my demons and come victorious

[fleet osprey]

Just™ port OpenJDK

[proper meteor]

the demons he's talking about:

[fleet osprey]

can't be that hard

[grand shadow]

well there is one

its just runs horribly

and never bothered to figure out why

[fleet osprey]

how horribly is horribly

[proper meteor]

(the only thing working is hello world)

(no idk)

[grand shadow]

qwinci managed to connect to a minecraft server running in astral for like 5 seconds

before it lockede up

[fleet osprey]

oh

how long did it take to connect

[grand shadow]

no idea

[fleet osprey]

and/or to start up

[grand shadow]

@carmine swallow

I think the lockup might be in my tcp code since I know there is a deadlock there somewhere I just havent gotten around to fixing it

[carmine swallow]

maybe

[carmine swallow]

idr exactly but I am pretty sure it wasn't that bad (though it might have taken a while to start)

[grand shadow]

The only thing I can think of is that another core is writing into this core's running thread stack

[grand shadow]

uh what

pushlock_wait_block_t a;
                        memcpy(&a, last, sizeof(pushlock_wait_block_t));
                        __assert(last->funny == 0xdeafbeef);```

this assert trips but when I look at it in gdb both the copy and the original are completely normal

with the right value in funny

[grand shadow]

I have been stuck in this stupid fucking bug for 3 days now and I dont even have a direction on how to fix it

[prime mulch]

pushlock bug?

[grand shadow]

still not sure, that stack corruption made me look to the only place where another thread would be able to change values in another one's stack which would be in the pushlock implementation, and adding a few asserts around does show stupid stuff like this happening but then it has weird behaviour like the assert failing but then I look at both the copy and original pointer and funny (temporary thing for debugging where it starts as 0xdeadbeef and ends up as 0xc0ffee after the thread wakes up) is the value assert expects

I ran without pushlocks and it worked fine and ran into some other bug but now I dont know if the pushlock issue and that are symptoms of another bug or what

[grand shadow]

no like wtf

-559038737 is 0xdeadbeef

[analog berry]

Btw if its really undebuggable maybe consider implementing asan

[prime mulch]

It's not undebuggable

[grand shadow]

I think it might be something tlb related?

always doing a full tlb flush instead of invlpging when below a certain number of pages made it so in 10 tries the bug didnt happen a single time but with the invlpg stuff on 3 bugs on 10 runs

[prime mulch]

how are you doing your full tlb flush

@grand shadow

this is extremely relevant

are you reloading cr3

[grand shadow]

yes

[prime mulch]

thats your problem

are your kernel pages mapped global

[grand shadow]

no

[prime mulch]

why

are you sure they dont have the G bit set

they should!

[grand shadow]

they do not I never bothered to do that

[prime mulch]

nvm then

reloading cr3 doesnt flush global pages so i thought i had ur answer

youre supposed to toggle cr4.pge

to flush all including global

[grand shadow]

considering the problem seems to get better when I do a full flush I'm probably forgetting a invalidate somewhere

though yeah I should make the kernel pages global at some point

I even had a TODO warning me about the cr3 flush thing about global pages

[grand shadow]

it still seems to happen

ok nvm I dont think it makes it any bettetr

https://tenor.com/view/cat-insane-gif-21702784

but the only thing I can see explaining this weird behaviour is that

[grand shadow]

by forcing an invalidate on every mmu_unmap and mmu_remap it seems like the bug is gone?

I dont think thats it

I dont know anymore

[analog berry]

U probably just changed timing

the astral curse

[grand shadow]

cant wait for me to find the bug after like 4 more days of debugging and for it to be the silliest fucjking thing imaginable

[grand shadow]

considering it only affects the upper 32 bits of stuff and x86 is little endian, this might be either a packed struct, a fucked up memset/memcpy or a bitmask thing

since thats not aligned on 8 bytes

[fleet osprey]

just impelement kasan, you'll find a lot more than just this almost certainly

well.... Just™ implement it

it's not quite that easy

[grand shadow]

Ig I can look inti that

Idk how to implement it

At least thatd be something better than banging my head against the wall for hours on end trying to figure out wtf is going on

[flat copper]

have you considered that you are mallocing with the wrong sizeof for something?

[grand shadow]

Its corrupting the stack so I dont think thatd be it

It only seems to happen on multicore too so its probably some super stupid race somewhere

[carmine swallow]

using relacy with the push lock impl from astral gives some data race error when there are 2 shared acquires along with an exclusive one, though I am not sure whether it is relevant to the current bug as it also takes into account possible reorderings + other stuff like that

[grand shadow]

I dont do any shared acquires anywhere so I dont think thatd be relevant, though still good to keep in mind ty

[carmine swallow]

with only exclusive ones it doesn't give any errors even when there are 30 threads (though I don't want to run the full set of reorderings with that, its slow when the number of threads is > 2)

idk how accurate it is though

[grand shadow]

I guess thatd mean the pushlocks are fine and the issue is something else?

[grand shadow]

If I cant find a solution I will just go do something else so I dont burn myself out

[analog berry]

use the monkuous card

[tacit schooner]

@grand shadow sry for ping but can you help me get started in os dev?

[grand shadow]

@sharp carbon

[sharp carbon]

hello

[analog berry]

can u help mathew debug a skill issue

[sharp carbon]

what is the issue

[analog berry]

random variable corruption, right @grand shadow ?

[sharp carbon]

oh god

I can take a look ig

what's the trigger

[analog berry]

lol mathew is gone

[grand shadow]

Random stack corruption

I am at class sorry lmao

Randomly itll just have some value on the stack overwritten

When running with 4 cores

Even with irqs disabled

Most of the time it happens on my tlb invalidate code but thats just a symptom since it happens elsewhere as well

[sharp carbon]

how to reproduce

[grand shadow]

Just run with -smp 4 in qemu many times and itll happen

Using the xorg boot option (first one)

Remove the nosmp argument from the limine.cfg too

Itll take a while for everything to build so take your timme

[sharp carbon]

oh god xorg?

how many hours of compile time am I about to wait

also does it happen during boot or after x amount of idle time or what

[grand shadow]

While its booting up userspace stuff like xorg

[pine ledge]

It’s always the fucking tlb shootdown bruh

I’ve been getting fucked by that too in my kernel

[sharp carbon]

@grand shadow unrelated to the issue but i would recommend changing all ftp.gnu.org links to ftpmirror.gnu.org

this is one of the first times in my life i've seen wget report download speed in bytes per second

[grand shadow]

True

[sharp carbon]

why is it trying to compile a test

[analog berry]

Lol

Does it recursively look for c files

[sharp carbon]

seems so

i just deleted the tests dir

[analog berry]

I guess it worked previously because the tests were in c++

[grand shadow]

Oh I forgot to upstream the fix

[grand shadow]

Yeah

[analog berry]

Any luck?

[sharp carbon]

still compiling

probably gonna have to let it run overnight

[grand shadow]

yeah there are a lot of ports

[sharp carbon]

currently seems to be compiling wine

[grand shadow]

oh maybe it is already nearing the end then

since iirc jinx does alphabetic order

[sharp carbon]

even if it's almost finished i don't have time to debug anymore so that'll have to wait until tomorrow

at least i'll be able to power down my pc instead of leaving it on

does seem to be alphabetical yeah, moved on to xauth immediately after

[grand shadow]

yippee

ty in advance

[sharp carbon]

np

[grand shadow]

I have been banging my head against the wall for the past 4 days

[sharp carbon]

yea build finished

this the issue?

[grand shadow]

seems like so

the top 32 bits are missing

[analog berry]

wait where did kernec go

[grand shadow]

its still there

[sharp carbon]

alr i'll do the actual debugging tomorrow because it's 11pm here

[grand shadow]

this just hit a different step in vmm_fault lol

[analog berry]

i go to bed at 4 am

[grand shadow]

my bedtime is usually around midnight

except when I'm with my bf we stay awake until like 4 am playing games

it should be 10 pm since I wake up 6 am but I cba

[grand shadow]

gn sleep well

[grand shadow]

It has been brought to my attention that I was checking for deafbeef instead of deadbeef here and I feel extremely stupid

https://tenor.com/view/brittlestar-wunkus-gif-418682295799919232

[jovial moat]

deaf beef

https://tenor.com/view/skeleton-gif-13148928981517710530

[fleet osprey]

https://tenor.com/view/anton-chigurh-no-country-for-old-men-javier-bardem-gif-22023128

[sharp carbon]

couldn't figure anything out

[grand shadow]

Damn

You know its fucked when even monkuous cant figure it out

[analog berry]

Indeed

[opal lotus]

time for rewrite

[pine ledge]

Ultimate truth

He’s the ultimate debugger

[grand shadow]

Not even a guess of what it could be

Im gonna try to debug some more today

[pine ledge]

Have you tried using GitHub CI?

It’s good at catching bullshit like that

Like stupid small bugs that are hard to reproduce

[grand shadow]

Its not hard to reproduce, its hard to debug

[analog berry]

Have you tried git bisect to at least find what introduced the bug?

[pine ledge]

Yeah, I would probably do something like Infy said or try to test all versions until the issue comes up and figure it out from there

[grand shadow]

I could yeah its just that other smp stuff might appear first lmao but Ill try it, not much I can do

I still suspect it might be the pushlocks but atp Im not really sure

[pine ledge]

Nt pushlocks?

[grand shadow]

Yeah

[pine ledge]

Damn.. where did you even read up about those

I can’t find any good papers on the topic

[grand shadow]

I saw hyenasky and fadanoid talking about it

Theres not gonna be papers

[pine ledge]

Gotcha

[grand shadow]

the fucked up thing is that like

it only happens on some boots

and when it does boot

it can just work normally without any issues

running 6 glxgears with -smp 4

for like a few minutes now

with no crashes

[grand shadow]

I switched the ext2 dir functions from using vmm_map/unmapto alloc/free temporarily (I gotta rewrite them properly) and the bug is just gone?

cant tell if its because of that or because there are less shutdowns or just the bug doesnt get the timing to happen anymore

[grand shadow]

could it be vmm bug?

[prime mulch]

have you done a bisect to see when the bug appear

[grand shadow]

not yet

[prime mulch]

That's the first thing to do when you run into an evil bug

[grand shadow]

I doubt any commit before the scheduler rewriet is even gonna be able to reproduce it

because of how abysmal the old scheduler was with smp

lol I'm having to fix like 3 different files to get it to not deadlock from fixes I've already done and I made kvm panic

bisecting would be even more annoying than just trying to debug it

[prime mulch]

You might just have to get creative with it

And give in to the annoyingness

[grand shadow]

ok its not reproducible before the scheduler rework started

whether that is because old scheduler too bad or new scheduler caused the buggy is to be discovered

yeah no this is likely added before the scheduler rewrite and smp getting better just made it have the right timining to happen

is there no way of making qemu log like the last 100 writes for every cpu or something

[grand shadow]

https://tenor.com/view/akira-leave-me-alone-leave-me-alone-meme-gif-6420351794930725971

[grand shadow]

With kvm off I was able to get it into a panic with another cpu doing a memset 0 after a vmm_map

It did not write to the same virtual address as the stack of the fucked up core though

So either its the same physical address or its unrelated

Time to delve into the page tables

[grand shadow]

nvm it is not

[pine ledge]

POV menix devlog:

[grand shadow]

hmmmm, making the pushlock wait block be in the thread struct instead of on the stack makes the bug go away

without any big timing changes

[neon crane]

then consider checking if the pushlock wait block's pointer doesn't stick around after the wait exits

[prime mulch]

Didn't you already determine it's not the pushlocks

[neon crane]

im saying this because i had a bug where i was using a timer on the stack and the thread was being terminated yet the timer wasnt being dequeued

so it was using a stale pointer

[grand shadow]

I still am not really sure what it is

[grand shadow]

Ive already checked the impl many times but I can do it again

[prime mulch]

CAN YOU REPRODUCE the bug without pushlock code ever being CALLED

[neon crane]

The impl, sure, but what about all the things that use the impl

The impl of my timers was also fine but what was messed up was how I was managing them

[grand shadow]

No but it could be timing

[prime mulch]

Oh your pushlock is broken

I just looked

The way you traverse the wait blocks is wrong

You should be atomically popping the entire wait block list off the pushlock and resetting the pushlock word to 0 or whatever when you release it and only then do you walk the wait block list

Looks like you're traversing them while they're still linked to by the pushlock which is menacing unless im reading it wrong

You should also be waking all the waiters

That's the easiest way to do it race free

Your way is incredibly racy and could be causing the memory corruption you're seeing

Potentially

[grand shadow]

Is that how other implementations do it

[prime mulch]

Yes

Including windows

[grand shadow]

Oh

Fmk

Fml

[prime mulch]

This part is mandatory to avoid races, popping the entire list off and setting the pushlock free before traversing the wait blocks, and it implies that you have to wake all the waiters since you popped them all off the pushlock at once

If you only woke one and then put the rest back on or something then you have crazy races again and ABA problem

[grand shadow]

I only pop the last one of the back of the list and wake it up

[prime mulch]

Yeah that's bad

[grand shadow]

So you would then do an atomic exchange to set it to 0 and get the old value with the top wait block pointer and go waking these up?

Seems simple enough

[prime mulch]

Yeah

Also turnstiles rule pushlocks drool

[grand shadow]

In the future I might look into them but I currently have bigger issues

Thank you hyena man I will change this later

[prime mulch]

If im right then your bug may disappear

after you change this

This was literally the first thing I thought to check when I decided to take a look so I might have saved you some time if I just looked earlier

soz

Could also be wrong though I just skimmed it on my phone in bed but it's worth trying at least

[grand shadow]

It still happens

Fucked up

[prime mulch]

Did you push

[grand shadow]

not yet I'll send it here

[grand shadow]

                saved_value = __atomic_exchange_n(pushlock, 0, __ATOMIC_SEQ_CST);
                __assert(saved_value & PUSHLOCK_FLAGS_ACQUIRED);
                __assert(saved_value & PUSHLOCK_FLAGS_EXCLUSIVE);
                pushlock_wait_block_t *wait_block = PUSHLOCK_GET_POINTER(saved_value);

                while (wait_block) {
                        pushlock_wait_block_t *to_wake = wait_block;
                        wait_block = wait_block->next;
                        semaphore_signal(&to_wake->semaphore);
                }

this is the exclusive unlock code

wait

WAIT

WAIT

OH

OHHHHH

ohh

there is a race in my semaphore code

when it comes to using it in the stack of another thread

[prime mulch]

I still think the pushlock thing I noticed was racey and broken as well

It just wasn't THE broken thing

[prime mulch]

What is the race

[grand shadow]

it unlocks a semaphore spinlock after waking up the thread, which if it is running on another cpu can do it when it really should not do that

fml

[prime mulch]

I only ever want wingstop at weird moments like when im in bed at 10am after being awake for like 30 hours straight

Can you relate

[grand shadow]

we dont have wingstop here but I do love some wings

thank you hyenaman me rewriting that code gave me an epiphany

mostly cuz I had to think of the placement of that semaphore_signal in the while loop

[prime mulch]

Plus I think you would have noticed weird symptoms from the pushlock thing

I'm like 80% sure there were races where ther could be lost wakeups at least

It just might not have run long enough for you to notice

[grand shadow]

holy fuck yeah it makes sense why the upper 32 bits of stuff would be randomly gone then, it was setting it to 0 and the spinlock is the second entry in the semaphore

[grand shadow]

how does it handle stuff like the shared holder count then, does it store it on the one pointed to by the pushlock always?

[prime mulch]

Looks like it stores it on the final wait block (that is, the first one to be added to the chain)

and uses a "multiple shared" flag in the pushlock for something

the reactos comments aren't helpful because they didn't understand it either

[neon crane]

@grand shadow it also makes sense now why things werent happening if the pushlock waitblock was in the thread struct and not on the stack

the bug will mentioned caused stale pointers to be used

[grand shadow]

How did you do it in mintia

[grand shadow]

It was another bug that caused it but related to that

Semaphore was doing wake up -> unlock when it should be unlock -> wake up to prevent stack corruption

[cosmic crow]

looks like it exists because on contention the pushlock structure changes

and it indicates that multiple threads acquired the pushlock and the releaser needs to traverse the wait blocks

[opal lotus]

real

[prime mulch]

i dint

didnt

i do turnstyles

[grand shadow]

didnyt you do it before taking the turnstyle path

[prime mulch]

yeah i just copied the reactos ones

[grand shadow]

oh I see

[grand shadow]

Well I have been slowly fixing the rough edges of smp and I was able to compile the astral source code with make -j5 on -smp 4 without it crashing

Still a lot of work to do for it to be quicker

[grand shadow]

compiling astral source:
time CC=gcc LD=gcc make with -smp4: real 0m53.783s
time CC=gcc LD=gcc make -j5 with -smp4: real 0m44.550s
time CC=gcc LD=gcc make with -smp 1: 1m58.443s

crazy

that -smp 1 time scares me, maybe I messed up somewhere and singlecore now is slow

hm glxgears is the same speed as before, maybe it was that slow to compile astral and I just forgor

there is still a lot of contention going on but I will sort that out soon enough

happy with the progress :)

before all this work smp would've been slower

LP_NUM_THREADS=4 tyr-glquake is also pretty good

[grand shadow]

no it makes sense, the children gcc spawns can initialize themselves while gcc is still running

and it spawns quite a few of them, so it adds up

[analog berry]

Why is the difference with multiple jobs so tiny?

[grand shadow]

probably crazy contention somewhere

probably the page cache (I implemented it super stupidly like a year ago) and allocators

I plan on redoing it to make it a tree of pages per vnode instead of... whatever the fuck I do right now

its like a hash table of <vnode, offset>

evil

and for allocators I will implement magazines

and at some point in the future rework the vmm as well

me when linked list vmm

[prime mulch]

you say that like thats not what everybody was doing for like 20 yrs and might still be in some cases

[grand shadow]

oh really?

still sounds super not scalable

[prime mulch]

yeah its been a not uncommon way to do it

[prime mulch]

you can do per bucket locking

of the hash table

[grand shadow]

true

[grand shadow]

what is the pros and cons of doing it when compared to something like a balanced bst for every vnode

[prime mulch]

idk i did a per-vnode avl tree

[grand shadow]

well gnu hello configures, compiles and installs with -smp 4 and -j5

pretty quick too

think I have smp finally at an okay state

sooner or later I will throw a nuclear bomb at this (compile gcc

[prime mulch]

can asstroll survive a forkbomb

[grand shadow]

fucked up

oh a page fault

I probably forget alloc somewhere and send a null to free

since it was on free()

[robust geyser]

vmm: out of me-mobrya tso hdo: c opfy oonr wkri:t Out of memory (ENOeM oEn M)a
ddress space (sending SIGBUS)

[grand shadow]

well it doesnt panic on a fork bomb anymore

but it also isnt very usable lmao

[grand shadow]

I'm gonna leave gcc building with 4 cores and j5 while I sleep

alright it is going

its running like 5 configures at the same time if this doesnt kill it idk what will

oh it died for like a stupid reason

out of disk space :^)

oh wait I see these ext2 fails I need to look into that later, seems like its not super safe in low disk space conditions

[pine ledge]

no

[grand shadow]

alright gcc build going again

this time I gave it 16 gbs of disk space compared to the 4 gb from before

oh fuck off I didnt give the vm enough memory

alright giving it 10 gb of memory third time is the charm

makes sense it would die I gave it 2 gb to do a gcc build with 5 jobs 💀

[grand shadow]

Seems to still be going

Although really slow, wonder why

Maybe its just using too much memory and the page cache is empty or something

[grand shadow]

Seems to be going at a ok time, its probably just slow to compile cuz unrelated things

[grand shadow]

if this compiles will I be a part of the cool smp kids

[grand shadow]

still didnt crash but I got tired of waiting

[proper meteor]

@grand shadow why do you lock here? Isn't the tree read only after boot?
https://github.com/Mathewnd/Astral/blob/6050ed1fc66a63d5c3b83602658be870525491e4/kernel-src/sys/topology.c#L93

[grand shadow]

true actually

[proper meteor]

Making Astral slightly faster

[grand shadow]

soon I'm gonna add magazines and break down the page cache lock so it should scale better as well

[grand shadow]

magazines are really simple, damn

I should have done this from the start

the most complicated part is probably dynamically updating the magazine capacity when the depot gets too contended

[grand shadow]

the fast path is so simple I wonder if this will affect the uacpi score in a significant way

in the bonwick paper it said it halved the latency

[gritty shore]

I just skipped that part

[grand shadow]

I was gonna ask you how you did it since ik you did magazines 💀

[gritty shore]

I dont think its crucial

I dont think it's that hard to do either

[proper meteor]

What are those?

[pine ledge]

Ik in a slab allocator it’s a per cpu slab cache

[proper meteor]

Right

[pine ledge]

But for page cache shit idk

I’m assuming it’s a similar concept

[gritty shore]

Small per-cpu list of ready-to-be-allocated slab objects

So the fast path is just popping from the list

[grand shadow]

I got it to boot with magazines and stuff but I still got some more work to do before its upstream

[proper meteor]

I don't even have a slab allocator anymore. I had implemented it a very long time ago when my kernel was in C and when I switched to Rust I didn't translate it out of laziness

[grand shadow]

Ive had slabs since like the beggijing

[proper meteor]

the beijing

[grand shadow]

uacpi score seems to be lower with magazines than without, fucked up

maybe I did somethng horribly wrong

I mena I guess it also makes sense, since theres more ifs for it to go through before falling back into the slab compared to just using the slab, and the mags are gonna be mostly empty

[analog berry]

I think monkuous got a similar result iirc? @sharp carbon right?

[pine ledge]

How do you allocate objects below page size?

[proper meteor]

malloc

[pine ledge]

Do you just create a free list of a certain size and combine them

[pine ledge]

Your own?

[proper meteor]

yes

[pine ledge]

I’m pretty sure @prime mulch does something like this

[grand shadow]

and thus to make my uacpi score better I should preload 32 rounds for every cache right before initing uacpi

[pine ledge]

So like a list of 32 byte objects, and if a 128byte object is allocated, it just combines 4 of the 32 byte objects or something like that

Rather than having a list for each object size

[pine ledge]

Seeing a high resolution troll face here is weird lol

I have eye cancer now, Thank you for that

https://tenor.com/view/trollface-gif-8242347154758580084

[proper meteor]

I think the point of the slab allocator is also to have elements of the same type lying in the same pages so that if you have a tree (for example) of those elements, then they are all in cache

[pine ledge]

Yeah

Idk slab is kinda soemthing you implement once and forget about lol

[grand shadow]

a correct implementation of slabs would use them in a lot of places directly

it is an object cache after all, you can make an object caches of say a vnode

so instead of going through the generic allocator you allocate through the vnode slab

and at this point I am just rewriting my slab implementation pretty much entirely, I wrote it like 2 years ago and it shows its age

[gritty shore]

most times magazines are full

i think

[grand shadow]

I mean like when doing uacpi init

[gritty shore]

ah

[grand shadow]

since its during boot

[gritty shore]

yea

maybe

[analog berry]

i mean yeah monkuous tried doing uacpi init like 5 times in a row and its way faster later on with hot caches and full magazines

like uacpi_init() + uacpi_state_reset() 5 times

[pine ledge]

Yeah I started creating slabs for certain subsystems

[grand shadow]

I reworked the slab list to be more like how bonwick dissertated about and it worked first try no problems somehow

[robust geyser]

are you caching constructed objects? or do you just hand out memory from a bucket by allocation size?

[grand shadow]

caching constructed objects

the old implementation does it wrongly but in this one I am going to do it properly

[bright kernel]

in this example, would the vnode slab also have like multiple caches? or would it just be the cache for the vnode size?

I think that's what I got wrong in my understanding of when to use it

[grand shadow]

in my case you would have many caches for the different filesystems as I keep the vnode struct as the first member of any other filesystem struct to be able to do a pointer cast

[bright kernel]

oh, and you also use that slab to allocate the vnodes for that fs?

so that if u free the fs u also free every node within it

or is that done by another slab

[proper meteor]

that sounds like a good idea

keeping that somewhere in my head

[bright kernel]

thank you!

thats what I might do in my OS, I was thinking about it today while having lunch

maybe having a slab in each fs to allocate everything that fs might need

like vnodes

[proper meteor]

@grand shadow https://github.com/Mathewnd/Astral/blob/8cba07f65ce63ed980fe45943d31f1404935fa98/kernel-src/include/kernel/proc.h#L16
Since your process have only two states, how do you put a process to sleep when you are waiting for a resource? (you don't? )

[sharp carbon]

that's a thread thing, not a process thing

[prime mulch]

https://github.com/Mathewnd/Astral/blob/8cba07f65ce63ed980fe45943d31f1404935fa98/kernel-src/include/kernel/thread.h#L10

[proper meteor]

ah, thanks

[grand shadow]

Sleeping is for kids, we do the grown up thing and just spin

But yeah it is a thread thing

The inly time a while process goes to sleep is a SIGSTOP and even that just tells every thread to do that

[grand shadow]

Think the slab rewrite is almost done, just gotta make some more final touches in the kernel

[grand shadow]

@prime mulch you said it was like super racey to go through the wait blocks when waking up a thread, does the same hold when traversing it to release a shared lock (so decrement the shared count of the last one)

I am rewriting the pushlocks to account for that (and it also allows me to make the fast case faster) and I need to know that to see how I am doing the shared part of the lock

I still dont see how traaversing the list was racey anyways

[gritty shore]

Can't the head change while you're traversing it?

Which is why you should pop it first then traverse it?

idk I'm just guessing from half reading the messages

[grand shadow]

if it does (and that would only matter if its the only wait block in the pushlock), the compare exchange would fail

[gritty shore]

Yea but that was before you added that no?

[grand shadow]

no that was always there

the old version is still pushed

the issue was with unrelated code

[gritty shore]

Where is it

The file

Ah in lib

[grand shadow]

with magazines the make -j5 time has been brought down to 0m36.835s

niice

once I break down the page cache lock it should be better, I imagine that should be the biggest point of contention at the moment

[robust geyser]

just profile it :^)

[analog berry]

@pine ledge dam

Wait why the hell did it ping some random person

Sorry lol

My phone is strange

[grand shadow]

Real

I expect it to go down further once I break down the page cache lock

Astral is no longer shit on smp

[analog berry]

didnt expect it to matter that much

[grand shadow]

I mean it makes sense, it both gives it a faster fast case and reduces the contention on allocators

But it reduces uacpi score which is the only real metric of kernel speed

[analog berry]

yeah but like

it spent 10 minutes in malloc or smth

thats crazy

[grand shadow]

Something uses it a lot ig

Could also be my staged pushlock changes which made the fast case faster than before

[analog berry]

real

[grand shadow]

I still need to implement slab coloring, but that should not be too difficult I dont think

[analog berry]

im kinda curious if it will make any difference at all

[grand shadow]

Probably does on some specific cases

[gritty shore]

This is very easy

You just add a number and increase that number lol

[grand shadow]

Yeah

[gritty shore]

https://github.com/rdmsr/alloc-benchmark/blob/master/kmem%2Fslab.c#L62

Pretty sure there's a bug in there somewhere that i fixed recently and forgot to push tho so be careful

[analog berry]

does it matter on newer cpus?

[gritty shore]

Idk

But can't hurt

[grand shadow]

Oooh thats a nice repo

Is it hard to add another allocator?

Also damn friggs allocator just dies with more threads

[gritty shore]

Nah

It's a very heuristical benchmark tho idk if that makes sense

It's not great

But it's good enough

[gritty shore]

Frigg is better single core iirc

[neon crane]

page cache lock? wdym

was there a global page cache lock that youre breaking down to per-file or was it a per-file lock that youre breaking down further

[grand shadow]

Global lock (I do a intrusive hashtable page cache)

So Im ginna break thst down to per bucket

[neon crane]

shouldn't you have one page cache per vnode?

[grand shadow]

I wanna do a bst for each vnode eventually but considering some other kernels also did a hash table page cache according to will "vmm" hyenasky I will just keep it this way until then

[prime juniper]

Just a subtle way to get more people into the thread, I see what you’re doing

[analog berry]

mathew, im afraid we have been uncovered

[grand shadow]

Just ping everyone

[bronze nexus]

Force new people to join this server while you're at it

[hearty fulcrum]

I don't believe that gets people in the thread

[grand shadow]

Sad

[grand shadow]

Also wouldnt waking everyone at once suffer from a thundering herd problem

I mean ig if too many threads are waiting on a lock then there are huge contention problems or something

[pine ledge]

No it’s okay this was important to me anyways 😭

[grand shadow]

I also probably should enabe global pages in the kernel 😭 idk why I still havent done that

not using that might be impacting the performance significantly when theres a lot of switching around

thats after I finish this page cache stuff

I also might look into redoing how I handle command line arguments, since I want access to parameters early

I might do something like what @analog berry said

section with parameters so I can fill it during boot

like before even doing early init stuff

[opal lotus]

good call

i should do that too

[proper meteor]

I did it a while ago

[grand shadow]

did not do any noticeable changes in performance but still good to have

[grand shadow]

apparently the qemu port is working now, I probably fixed something that made it work

thats nice

its not as slow as I wouldve thought too

wonder if it was the same but that made factorio on wine not work

[analog berry]

Damn

[grand shadow]

probably not

lets see if wine still opens notepad

ignore it screaming in the background

still cant open the save as menu :p

[analog berry]

Damn even notepad calls unimplemented win32?

[grand shadow]

could be astral funnies makng it think something is unimplemented

there is so much going on in that picture

xfishtank from a serial session running as user astral and spiiiin cubes ran from xterm as user root

xfishtank is goated any serious hobby os should have a fishtank application

oh and all of that running with -smp 4

:D

wonder how close astral is to being fully self hosting

[bright kernel]

whats the status of astral on rh?

maybe by then you can do some astral development on astral booted on a physical computer

[fleet osprey]

it boots on most hardware from my experience (new and old), but keyboard support is near-0

somehow it's PS/2 impl is fucked on every single 1 of my machines

[grand shadow]

I have never gotten around to unfucking the ps/2 driver and I have other worries

you would honestly be better off writing a nic driver and sshing into astral

or using serial

kind of funny to say that

[bright kernel]

'least it boots!

better than nothing

[grand shadow]

ps/2 keyboard is one of the most basic things people do but you can ssh into it instad

right now I am trying to do something extremely funny

[fleet osprey]

ooh?

[bright kernel]

he's about to run silksong on astral

[fleet osprey]

imagine

[bright kernel]

I yearn for the days we wont have to imagine it

[fleet osprey]

lol

[grand shadow]

well nvm I wasnt even able to get it to run on linux 💀

[bright kernel]

what were u trying to run?

[grand shadow]

a game

[grand shadow]

I really need to do some userspace maintenance work but I also do not feel like babysitting a build for 5 hours

[grand shadow]

Got the new argument system almost done

Its quite simple you just

DEFINE_KERNEL_ARGUMENT(name, KERNEL_ARGUMENT_TYPE_STR/INT/BOOL)
GET_KERNEL_ARGUMENT_STR(name)

[analog berry]

Use _Generic like me to automatically deduce the type

[grand shadow]

Hm true

Did not know that existed

[bronze nexus]

Just parse the parameters where you use them like I do

[analog berry]

#define PARAM_TYPE_OPS(value) _Generic((value), \
    i8: g_param_i8_ops,                         \
    u8: g_param_u8_ops,                         \
    i16: g_param_i16_ops,                       \
    u16: g_param_u16_ops,                       \
    i32: g_param_i32_ops,                       \
    u32: g_param_u32_ops,                       \
    i64: g_param_i64_ops,                       \
    u64: g_param_u64_ops,                       \
    struct string: g_param_string_ops,          \
    bool: g_param_bool_ops                      \
)

I do this basically

[opal lotus]

huh does this work

[analog berry]

yep

[sharp carbon]

i just make it invoke a function when the arg is passed and that function gets passed the param as a string

[grand shadow]

alright, time to run a make clean and bash my head at the wall until I can make it compile with updated jinx

then with updated mlibc + some more gcc patches

then upstream some stuff I have to do for mlibc

think I will also implement sysinfo to get all of that done in one go

so that userspace can make better use of multicore

I know java checks for the number of online cpus

and I think mesa llvmpipe does too

wtf now it needs to be ran as ./jinx init???

and nothing in the docs talks about that

and wtf is a jinxfile

oh just renamed jinx-config

why tho

what

@sage locust is there any place with like updated jinx documentation or is it just read ironclad

ok so I had to make a new folder and add the jinxfile there and then init it with the jinxfile there

good to know

yeah it is doing stuff now

[sage locust]

basically read Gloire

i did not update the documentation yet

[grand shadow]

fun!

thanks

[sage locust]

np

[grand shadow]

Has anyone compiled mlibc in their os yet

I might port meson and ninja and try that

I think it should work? I have working python

But I need to first get all this userspace bs figured out

[grand shadow]

So my todo list for now is:
Update jinx and do a full rebuild for trsting
Update gcc patch and do a full rebuild for testing
Update mlibc and do a full rebuild for testing
Hook up some more functions related to scheduling into sysdeps and do a basic sysinfo implementation
Upstream it into mlibc alongside some other local patches
Finish pushlock rewrite (related: does anyone know the optimal number of times to spin before going to sleep? My old pushlock didnt have that but I will implement it in the new version)
Change my ipi send so that I do other work while the ipi is sending instead of waiting for that one lapic status register to change, which might save some time on stuff like sending a reschedule ipi

I think these last two might help increase smp performance a bit more as well

[prime juniper]

@grand shadow you have a somewhat stable OS that can run gcc right? So you're the perfect ask for this. Can you try to compile zlib on Astral? Trying to rule out of the ar failure is a skill issue on Managarm's side or if it's a bug that others using mlibc also run into

[grand shadow]

Will do when home

That is if the disk image and iso survived the make clean

Otherwise only after I get jinx to work and do the full build

[prime juniper]

No hurry, thanks for testing!

[grand shadow]

What is the issue, btw?

[prime juniper]

Ar is refusing our stuff

And idk why

[grand shadow]

Is it only failing in zlib or does it fail in general? Gnu hello and gcc builds work and they call ar

[prime juniper]

untested on gnu hello and friends as configure on gnu hello doesn't pass on managarm

[grand shadow]

Damn

Will test when I get the chance

in exchange I want you to port webkitgtk to astral /j

[prime juniper]

How about there’s a small chance it’s a mlibc bug that you can also run into for self hosting. Best I can do

[grand shadow]

What if you wanted to self host but mlibc said no

[prime juniper]

Happens

[grand shadow]

If I can build mlibc in astral I will have built binutils, gcc and the libc in my own kernel

Which would be super damn close to filly self hosting

[prime juniper]

Indeed

[sharp carbon]

Works fine on jinix

[prime juniper]

this is still mlibc under it right?

[sharp carbon]

Yeah

[prime juniper]

alright

Managarm skill issue then

sadge

[analog berry]

what is jinix

[sharp carbon]

Linux with mlibc built with jinx

[analog berry]

interesting

[sharp carbon]

https://github.com/48cf/jinix

[analog berry]

iretq made literally everything it seems

[robust geyser]

lmao

most of the credit here goes to dennis for porting though :^)

[sharp carbon]

Worth noting that jinix was last updated 4 months ago so it could be a recent regression in mlibc

[robust geyser]

mlibc is probably a little bit more behind than 4 months as well

i really want to get back to jinix though :^)

thanks for the reminder that this project exists

[sharp carbon]

Updated mlibc and it's still fine so nope

[grand shadow]

Then it is a managarm skill issue, damn

[sharp carbon]

Well I'm doing a full world rebuild with the updated mlibc now to be sure

But yes most likely

[sage locust]

i will backport as much as i can from Gloire

no one seemed to want to work on Jinix so i kind of just stopped updating it

[sharp carbon]

yeah thats fair

[sharp carbon]

util-linux fails to compile CC lsfd-cmd/lsfd-file.o /base_dir/sources/util-linux/lsfd-cmd/file.c: In function 'get_minor_for_mqueue': /base_dir/sources/util-linux/lsfd-cmd/file.c:664:17: error: implicit declaration of function 'mq_close'; did you mean 'pclose'? [-Wimplicit-function-declaration] 664 | mq_close(mq); | ^~~~~~~~ | pclose /base_dir/sources/util-linux/lsfd-cmd/file.c:664:17: warning: nested extern declaration of 'mq_close' [-Wnested-externs]

ah that's one of the things added by the jinix mlibc patch i disabled

[sharp carbon]

Okay yeah it's a managarm skill issue

[prime juniper]

only so much time and so much to do I'm afraid

(and was on laptop only which slowed work significantly)

[prime juniper]

[grand shadow]

seems like the disk image and iso are still there, will do a double check if it works now

@prime juniper works in astral

so it really is managarm

[prime juniper]

Fuck

Last hope, I see a newer version of zlib (but likely a bug somewhere with us)

Thanks for testing everyone!

[grand shadow]

say version and I will wget it and build it

[proper meteor]

me before the end of the year hopefully

[grand shadow]

honestly its not even that hard you just need to be a bit stable and implement enough syscalls for configure and make to run

[proper meteor]

now that I have SMP implemented, the next step is precisely having configure and make running

[bronze nexus]

You got all the way into making userspace stuff without SMP?

[proper meteor]

yeah

[bronze nexus]

Damn

[proper meteor]

that was a big missing piece, but now it's there 🎉

[grand shadow]

it will be a good test for the stability

[bronze nexus]

Also show progress report @proper meteor

More threads for my collection

[grand shadow]

and then when you know it works you start a gcc build and fix any stability bugts that show up

[proper meteor]

#1191856876787335268 message

[grand shadow]

astral has been... surprisingly stable

[proper meteor]

yeah, I've been waiting so long for this moment

[bronze nexus]

thx

for linking it

[grand shadow]

only deadlock I know exists is somewhere in tcp code I think

happons when downloading a big file randomly

not sure whats up, never looked into it

but will have to at one point

[grand shadow]

it is super exciting

[bronze nexus]

ngl I get kinda jealous of your progress sometimes

[proper meteor]

such progress comes at the cost of long hours of despair and loss of the will to live (in Minecraft)

[bronze nexus]

I know

[grand shadow]

tfw you spend several hours per day over a week debugging a bug and the problem was in code you didnt even check

[bronze nexus]

I have, multiple times, had difficult to find bugs, be single-character typos.

[proper meteor]

It also comes with great moments of joy btw. I remember a few times, after more than 16 hours of debug without any pauses, figuring out where the issue comes from, and starting to dance alone at 4am

and then finally take a shower and have a meal

[bronze nexus]

Definitely. Mounting and at least reading (write is WIP) Ext2 with no manual intervention was a recent milestone I'm really proud of. It required a lot of subsystems to work properly to do so. #1153085124959809616 message

[grand shadow]

frfrfr

[bronze nexus]

The next milestone I'm aiming for is even better: A little demo of BadgerOS installing itself from a USB stick. That requires a bunch of VMM refactors, and syscall re-thinking, and the start of an actual distro for it.

But if you want to hear more we should move that convo to #1153085124959809616

[sharp carbon]

I did 1.2.12, downloaded from the Git tag, so should be the exact same as specified in the managarm recipe

[prime juniper]

Sadge

[prime juniper]

Similar tales here. I distinctly remember spending 2 weeks looking at X11 code and raw Unix socket dumps to figure out why it was shit, we found the bug, stubbed a function and got glxgears. Felt really good after two weeks

[grand shadow]

@sage locust do you know what is going on here
mount: /home/mathewnd/blehh/astral/.jinx-cache/sets/gcc/.image/usr/local: mount point does not exist.

oh you seem to have updated jinx between then, updating to see if that fixes it

still happens

ugh

does it not work with wsl or something

local does not exist in usr

[robust geyser]

rm ~/bleh/astral/.jinx-cache/sets/gcc/.image/.jinx-set-valid

that should make jinx recreate that image set

does sets/gcc/.image contain anything?

oh wait

sorry

scratch that, my bad

[grand shadow]

it does, it just does not have local in /usr

gonna try adding this mkdir to see if that hacky thing fixes it

if ( [ "$JINX_NATIVE_MODE" = "yes" ] && [ "$cross_compile" = "yes" ] ) || ( ! [ "$JINX_NATIVE_MODE" = "yes" ] ); then
        mkdir -p "${imgroot}/sysroot"

        mkdir -p "${imgroot}/usr/local" #<-----------
        container_pkgs_native_mount="mount -o ro --bind '${container_pkgs}' '${imgroot}/usr/local'"
        sysroot_native_mount="mount -o ro --bind '${sysroot_dir}' '${imgroot}/sysroot'"

        run_in_cont_lang=en_US.UTF-8
    else

well at least dies somewhere else

with that mkdir it dies here

/home/mathewnd/blehh/astral/jinx: 341: cd: can't cd to /home/mathewnd/blehh/astral/build-x86_64/host-pkgs/autoconf/usr/local
cp: cannot stat '/home/mathewnd/blehh/astral/build-x86_64/host-pkgs/autoconf/usr/local/.': No such file or directory

I'm not sure if I configured something wrong, or its a wsl skill issue or jinx bug

[robust geyser]

it's either a messed up build, jinx bug or a skill issue (which could be merged with #1)

wsl2 is perfectly capable of running jinx

it's basically a linux vm

so for all intents and purposes it's linux :^)

[grand shadow]

jinx:
        curl https://codeberg.org/Mintsuki/jinx/raw/commit/4bf3b28e2ed064dcfa2a24f7faed3bc5ec2ff9b8/jinx > jinx
        chmod +x jinx

$(JINX_DIR)/.astral_ok: jinx
        mkdir -p $(JINX_DIR)
        cd $(JINX_DIR) && \
        ../jinx init .. ARCH=$(JINX_ARCH) && \
        touch .astral_ok

kernel:
        cd $(JINX_DIR) && \
        rm -f builds/astral.packaged && \
        rm -f builds/astral.built && \
        ../jinx build astral

that is what is ran

JINX_ARCH=x86_64
JINX_DIR=$(shell pwd)/build-$(JINX_ARCH)

[grand shadow]

hmm using a commit before the rootfs changes seems to work

might as well find the last working commit while I commit mass murder in rainworld

[grand shadow]

killing scavs in ||metropolis|| is super fun

[bronze nexus]

Fartificer mod best mod

[grand shadow]

so true

ok bad commit is 5531eeb4152aef05af4171d9a74b0f79f04b494b

might as well use the one before it

aand now the autoconf source is not being found for some reason

oh ok I just had to nuke the sources dir which for some reason was not nuked before

wait but it doesnt even use that sources dir

it uses the one in the subdir

why would that even matter

wait no it does

I am confused

ok gonna try the latest commit again, maybe that was something I fucked up

yeah no even fully clean that commit is still broken

[sage locust]

interesting

[sage locust]

there should be no need to add that mkdir because the Debian rootfs should already have a /usr/local directory

[grand shadow]

well it did not find it somehow

and lsing the folder gave nothing

[grand shadow]

ironclad jumpscare

for some reason libintl refuses to build because of a weird libtool error so I am now trying to update it

maybe its using the linux libtool somehow?

no wait what why does it think it is on libtool 2.4.7 when both the imagedep and hostdep are 2.5.4

insane

[sage locust]

do you have the massive regex vomit in the Jinxfile used to patch in support for YourOS to libtool in each autotools package?

[grand shadow]

I do not

what did even change for that to break

since it used to work just fine

libtool: definition of this LT_INIT comes from libtool 2.5.4.
libtool: You should recreate aclocal.m4 with macros from libtool 2.4.7
libtool: and run autoconf again.```

VERSION=UNKNOWN

Configuring libtool UNKNOWN

insane

oh I fucking see

.tarball-version does not get copied over when copying the source to the build dir!

and it all explodes.

wait no

wait yes

there we fucking go

this was stupid

libintl compiled, time to see what blows up next

xbps-create: ERROR: invalid pkgver! got `base-_1' expected `foo-1.0_1'

version=4.4.36
source_method=tarball```

what is it even yapping about

ohh wait

nvm I'm dumb

wrong package

so much has changed I need to get used to new jinx

I do like that it makes .xbps packages... will make life easier when I need a package manager

I mean I could look into porting it

would be an excuse to fix that tcp bug as well

okayy the minimal distro built with new jinx

and it boots

time for the full build

[prime juniper]

Trust me xbps is an easy port. Just grab OpenSSL and libarchive and you’re basically set (well you need some CA certificates too but details)

[robust geyser]

you need those for curl too, which astral has

i believe there was a demo of fetching and building gnu hello :^)

[prime juniper]

Oh bruh

Then easy

(Technically all are optional if you fetch over insecure connections and curl doesn’t depend on libarchive)

Yeah funny. Astral installs the make-ca program. But not the actual certs

Will fix

[grand shadow]

oh very good to know

[grand shadow]

wait huh

fucked up

[prime juniper]

Yes

[grand shadow]

wait how did it work then wtf

since https worked

[prime juniper]

I don’t see you installing the certs as a port unless you do that somewhere I haven’t looked

[grand shadow]

It all builds fine with new jinx now but xorg breaks cuz /usr/var/log is not getting created

[grand shadow]

I do it in the make-ca recipe

[prime juniper]

Ah no you do

Yeah

[prime juniper]

That path looks scuffed anyway

Shouldn’t that use /var/log?

[grand shadow]

Maybe

No idea why it wants to use /usr/var/log

Maybe its the prefix?

But it used to work fine!!!

[grand shadow]

even removing the --prefix keeps it there

by default localstatedir is prefix/var

so I guess I just set that to /var

[prime juniper]

yeah

[grand shadow]

even setting it to var it still keeps wanting to open /usr/var/log

DEFAULT_LOGDIR='/var/log'

???

ohhh I see I have to do ./jinx REINSTALL now

there we go Ijust gotta mkdir /var/log now somehwere

since xorg does not create that dir for some reason

does jinx not install empty dirs?

ah fuck this I am just mkdiring it on /etc/rc

and now it dies because it cannot find any screen

how did updating jinx fuck everything up so hard 😭

oh right I removed the --prefix

that waws on me

THERE WE GO

xorg with keyboard and mouse working

tonight I will see if no other packages broke

[prime juniper]

xbps does not keep empty dirs around

so you add .keep files

or, if you're managarm and tired of doing that (hicolor-icon-theme installs a fair amount of empty directories, typing that out is pain)

https://github.com/managarm/bootstrap-managarm/blob/master/scripts/keep-empty-directories.py

you're welcome

[grand shadow]

Good to know

[prime juniper]

- args: ['@SOURCE_ROOT@/scripts/keep-empty-directories.py', '@THIS_COLLECT_DIR@'] in xbstrap language is the invocation

so basically same for jinx but shell and collect dir becomes dest dir

[grand shadow]

Probably gonna add a shell version of that to the jinxfile

[prime juniper]

either that or ship the python script in build-support or whatever

[robust geyser]

@sage locust jinx should possibly check for that before calling xbps-create

to prevent creating packages that are half broken

[prime juniper]

One could integrate it yes. So far the only package that might dislike it is hicolor-icon-theme but skill issue install an actual icon theme then

[grand shadow]

@analog berry you wanted the iso so here you go

qemu-system-x86_64 -smp 1 -m 512m -cdrom astral.iso -M q35 -enable-kvm -cpu host,migratable=off -debugcon stdio

[robust geyser]

acpi: [info] successfully loaded 1 AML blob, 1705 ops in 0ms (avg 3730853/s)

on my 13700k

[grand shadow]

damn

I'm not really sure what makes it so slow

[robust geyser]

profile it :^)

[grand shadow]

I implemented slab just like the paper said and my mutex is just a cas

some other time I will

[opal lotus]

this doesn't work for me

huh nvm a reboot fixed

[robust geyser]

well fwiw it doesn't boot for me, it dies on some assert

but i just | grep avg

[opal lotus]

no i mean

it went to seabios

💀

[grand shadow]

well yeah it tries to mount a filesystem on a disk that doesnt exist

[robust geyser]

lol

i see

[grand shadow]

wtf

[robust geyser]

ah

[opal lotus]

acpi: [info] successfully loaded 1 AML blob, 1705 ops in 0ms (avg 3323586/s)

on 265KF

[grand shadow]

yeah damn thats like

embarassingly slow

what the fuck

[opal lotus]

what clock do you use?

kvm?

[robust geyser]

i think that mint did say that ARL was worse at the uacpi benchmark :^)

[grand shadow]

yeah

[opal lotus]

my clock might be fucked because i use tsc

[robust geyser]

use hpet

then

lol

[opal lotus]

[ 0.098841] uacpi: successfully loaded 1 AML blob, 1729 ops in 0ms (avg 6568100/s)

[grand shadow]

ugh

[opal lotus]

this might be qemu being inaccurate for some reason

because before i haven't had scores on this level

only ~4-5M

[grand shadow]

meh this is stupid and doesnt really matter

gotta continue the distro work

ok most ports seem to be working fine

[analog berry]

Arl?

[opal lotus]

arrow lake

[analog berry]

Ah

[grand shadow]

https://cdn.discordapp.com/attachments/971526899757121586/1353851167045587125/glued_to_the_ceiling.gif

[bronze nexus]

Not a real OS without triggering the antivirus

[grand shadow]

now it is time to do a full clean build again with some changes to the gcc patch

[bronze nexus]

Thank you for showing me this img, I will now steal it