Astral | Operating System Development | Page 5

sacred gyro Jul 28, 2024, 4:46 AM

#

not sure how I mixed the two up...

#

neither trackpad or keyboard seem to work tho even though they're both PS/2

fleet osprey Jul 28, 2024, 4:47 AM

#

If I plug a real PS2 mouse in, I imagine it would detect that and probably log something?
If so that could differentiate between emulated and real PS/2 for the Precision

grand shadow Jul 28, 2024, 4:47 AM

#

Damn

#

I reaaaally need to rework this ps/2 code

#

Its almost just straight copied from old astral

grand shadow Jul 28, 2024, 4:48 AM

#

fleet osprey If I plug a real PS2 mouse in, I imagine it would detect that and probably log s...

Probably would work, I dont depend on a keyboard always being present

fleet osprey Jul 28, 2024, 4:49 AM

#

Also, I get that same failing to open /init error when booting from Ventoy on my OptiPlex
So presumably Ventoy is to blame there, only had that issue on that specific method of booting it...
But at least it boots mostly now, which is more than I could say about it before

grand shadow Jul 28, 2024, 4:51 AM

#

Wonder what Ill have first
A working usb + hid driver or a ps/2 driver that almost always works lmaooooooo

sacred gyro Jul 28, 2024, 4:54 AM

#

I wonder if SeshOS PS/2 works on my chromebook

fleet osprey Jul 28, 2024, 4:56 AM

#

I swear I had a PS2 mouse somewhere... Where did I put it....

sacred gyro Jul 28, 2024, 4:57 AM

#

sacred gyro I wonder if SeshOS PS/2 works on my chromebook

Well this is incredibly helpful…

#

#

Of course discord didn’t send the image despite me adding the image to the message

fleet osprey Jul 28, 2024, 4:59 AM

#

well damn
mouse also doesn't come on when using PS/2

#

one of
- the PS/2 ports on this PC are broken (it's old, certainly possible)
- the USB -> PS/2 adapters I'm using are broken (they're even older than the PC, iirc they came with some microsoft thingy from the late 90s)
- the kbd and mouse both happen to just draw too much power for the PS/2 ports to handle

#

but at least the 2 main bugs were fixed

#

also, did you touch anything with PCI? if so, I can check on the latitude again, which hung... here ^^^

#

or wow, that's not what I wanted to reply to

#

🤦

#

I meant this one ^^^

grand shadow Jul 28, 2024, 5:06 AM

#

If you wanna check it out

#

Dont think much is gonna change

fleet osprey Jul 28, 2024, 5:07 AM

#

was anything at all changed with PCI init? if not I probably won't bother, I gtg soon

grand shadow Jul 28, 2024, 5:07 AM

#

I only changed something in msi which likely doesnt matter so not rly

#

Ty for testing

fleet osprey Jul 28, 2024, 5:08 AM

#

well the last line that was printed does mention msi-x, but that looks like it's just listing everything, so I doubt it

#

I'll check when I get back, or maybe tomorrow, for the hell of it

#

oh btw
if you ever plan to expand support to ARM(64 or not), I have a decent amount of hardware I could test on
if you do at all I assume it would likely be far into the future I would assume, but regardless

- Nintendo Switch (it can run UEFI)
- Pi 3B
- Pi 4
- A large variety of Pi 0s
- A bunch of shitty Android devices

craggy tide Jul 28, 2024, 6:36 AM

#

sacred gyro

pitch issue?

sacred gyro Jul 28, 2024, 7:39 AM

#

¯_(ツ)_/¯

neon crane Jul 28, 2024, 7:52 AM

#

thanks, I'll go check it out

#

it seems to boot to a bash prompt, but I can't type in anything

#

for transparency, I have and am using an actual PS/2 keyboard

#

and Limine and Boron work with it

inland cliff Jul 28, 2024, 11:09 AM

#

pro move

grand shadow Jul 28, 2024, 12:33 PM

#

neon crane it seems to boot to a bash prompt, but I can't type in anything

Well good to see that at least it boots

#

I have to get that ps/2 thing sorted sooner or later but its not that much of a priority now since I cant reliably test it myself

neon crane Jul 28, 2024, 12:58 PM

#

grand shadow I have to get that ps/2 thing sorted sooner or later but its not that much of a ...

is there a way to compile just the kernel?

grand shadow Jul 28, 2024, 1:37 PM

#

Yeah the kernel-src subdir is compilable on its own

neon crane Jul 28, 2024, 7:48 PM

#

grand shadow Yeah the kernel-src subdir is compilable on its own

Seems like my compiler has PIC enabled by default or something?

#

I get this:

iprogramincpp@IPROGRAMINCPP2:/mnt/c/Work/x/Astral/kernel-src$ make
# build build.h file
echo "#ifndef _BUILD_H" > include/build.h
echo "#define _BUILD_H" >> include/build.h
echo "  #define BUILD_DATE \"Sun Jul 28 19:47:32 UTC 2024\"" >> include/build.h
echo "  #define BUILD_TARGET \"x86-64\"" >> include/build.h
echo "#endif" >> include/build.h
make astral
make[1]: Entering directory '/mnt/c/Work/x/Astral/kernel-src'
[CC] arch/x86-64/acpi.o
cc -g -ffreestanding -mcmodel=kernel -O2 -mno-red-zone -mgeneral-regs-only -mno-mmx -mno-sse -mno-sse2 -nostdlib -Wall -I "/mnt/c/Work/x/Astral/kernel-src/include" -I "/mnt/c/Work/x/Astral/kernel-src/include/x86-64/" -I "/mnt/c/Work/x/Astral/kernel-src/flanterm" -DPRINTF_DISABLE_SUPPORT_FLOAT  -c -o arch/x86-64/acpi.o arch/x86-64/acpi.c
cc1: error: code model kernel does not support PIC mode
make[1]: *** [Makefile:33: arch/x86-64/acpi.o] Error 1
make[1]: Leaving directory '/mnt/c/Work/x/Astral/kernel-src'
make: *** [Makefile:25: all] Error 2

#

also it wants a <limine.h> in the includes which... okay?

#

why?

#

also youre gonna wanna enable dependency file generation at some point

#

passing -MMD to gcc will generate a .d file alongside the .o, use patsubst on the object file list to turn it into a dependency file list and -include it in the makefile

grand shadow Jul 28, 2024, 7:57 PM

#

neon crane also it wants a <limine.h> in the includes which... okay?

aaaaa I forgot that it downloads stuff

neon crane Jul 28, 2024, 7:57 PM

#

independently compilable my ass, I needed to download flanterm and check it out in kernel-src

#

why not embed it as a submodule?

grand shadow Jul 28, 2024, 7:58 PM

#

I download it when regenerating the source with jinx lmao

#

uhh after jinx regenerates it I think it puts it in sources/astral or something

neon crane Jul 28, 2024, 8:02 PM

#

also this is weird

#

cc -g -ffreestanding -mcmodel=kernel -O2 -mno-red-zone -mgeneral-regs-only -mno-mmx -mno-sse -mno-sse2 -nostdlib -Wall -fno-pie -fno-pic -I "/mnt/c/Work/x/Astral/kernel-src/include" -I "/mnt/c/Work/x/Astral/kernel-src/include/x86-64/" -I "/mnt/c/Work/x/Astral/kernel-src/flanterm" -I "/mnt/c/boron/limine" -DPRINTF_DISABLE_SUPPORT_FLOAT  -c -o sys/timekeeper.o sys/timekeeper.c
sys/timekeeper.c: In function ‘timekeeper_time’:
sys/timekeeper.c:26:20: error: expected identifier or ‘(’ before numeric constant
   26 |         timespec_t unix;
      |                    ^~~~
sys/timekeeper.c:27:13: error: request for member ‘s’ in something not a structure or union
   27 |         unix.s = bootunix;
      |             ^
sys/timekeeper.c:28:13: error: request for member ‘ns’ in something not a structure or union
   28 |         unix.ns = 0;
      |             ^
sys/timekeeper.c:29:29: error: incompatible type for argument 1 of ‘timespec_add’
   29 |         return timespec_add(unix, fromboot);
      |                             ^~~~
      |                             |
      |                             int
In file included from /mnt/c/Work/x/Astral/kernel-src/include/kernel/timekeeper.h:4,
                 from sys/timekeeper.c:1:
/mnt/c/Work/x/Astral/kernel-src/include/time.h:19:50: note: expected ‘timespec_t’ but argument is of type ‘int’
   19 | static inline timespec_t timespec_add(timespec_t a, timespec_t b) {
      |                                       ~~~~~~~~~~~^```

#

no timespec_t 🥺

grand shadow Jul 28, 2024, 8:05 PM

#

wtf

neon crane Jul 28, 2024, 8:27 PM

#

grand shadow wtf

tried -nostdlib but now it complains about printf.h missing

#

damn

#

maybe you should try your build system on multiple machines

#

not trying to insult or anything but i think it needs a refactor

grand shadow Jul 28, 2024, 8:41 PM

#

tbf it expects everything to be run from jinx

#

with the cross compiler

neon crane Jul 28, 2024, 8:41 PM

#

grand shadow tbf it expects everything to be run from jinx

well guess what, jinx doesn't work for me either

#

throws a bunch of "No such file or directory" errors and quits

#

idk I was hoping I'd be able to simply build the kernel so I can fix the ps/2 issue

grand shadow Jul 28, 2024, 8:42 PM

#

neon crane throws a bunch of "No such file or directory" errors and quits

whjat was missing

neon crane Jul 28, 2024, 8:42 PM

#

something starting with root.x86_64

#

i can checkout the repo and try again if you want

#

nvm i didnt delete it

#

oh this

#

📎 message.txt

#

also whys it gotta download 182 mb every time I run jinx

grand shadow Jul 28, 2024, 8:44 PM

#

I think its downloading the arch stuff

#

no ide about those errors, I didn't make jinx

#

also the missing printf.h stuff is cuz I also use an external printf that gets downloaded on the jinx regenerate

#

name=astral
version=0
source_dir="kernel-src"
allow_network="yes"
imagedeps="wget"
flanterm_commit="347d9cf1e3fced64aa20ab278e2fdcc834086035"

regenerate() {
    rm -f include/limine.h
    rm -f io/printf.c
    rm -f include/printf.h
    rm -rf flanterm
    curl https://raw.githubusercontent.com/limine-bootloader/limine/trunk/limine.h > include/limine.h
    pushd /tmp/
    wget https://github.com/mpaland/printf/archive/refs/tags/v4.0.0.tar.gz
    tar -xf v4.0.0.tar.gz
    wget https://github.com/mintsuki/flanterm/archive/${flanterm_commit}.tar.gz
    tar -xf ${flanterm_commit}.tar.gz
    popd
    cp /tmp/printf-4.0.0/printf.c io/printf.c
    cp /tmp/printf-4.0.0/printf.h include/printf.h
    cp -r /tmp/flanterm-${flanterm_commit} flanterm
}

#

t does this

grand shadow Jul 28, 2024, 8:50 PM

#

neon crane not trying to insult or anything but i think it needs a refactor

tho yeah, I do agree the build system is a bit hacked together at places

craggy tide Jul 28, 2024, 8:59 PM

#

neon crane I get this: ``` iprogramincpp@IPROGRAMINCPP2:/mnt/c/Work/x/Astral/kernel-src$ ma...

lol someone did not copy all the template flags

neon crane Jul 28, 2024, 8:59 PM

#

❗

#

I had to add -fno-pie -fno-pic myself

craggy tide Jul 28, 2024, 9:00 PM

#

i mean his kernel probably relies on being built inside jinx

#

which isn't a good idea

#

ideally a kernel should compile anywhere regardless

#

it is handy for things like setting up a language server with bear, for example

grand shadow Jul 28, 2024, 9:24 PM

#

well update on that computer I had

#

it powers on but doesnt work enough even for the display on the back to show anything useful

#

so that wont be useful

grand shadow Jul 28, 2024, 9:41 PM

#

anyways

#

I will try to compile gcc again for shits and giggles

#

configured, building now

#

it tries to redefire errno

#

cuz apparently configure thinks I dont have stdc headers?

#

yeahhhh I don't think this is quite right

#

honestly surprised by how usable this has been, I have been debugging this from inside astral

#

from disk too

analog berry Jul 28, 2024, 10:11 PM

#

nice

grand shadow Jul 28, 2024, 10:21 PM

#

this is whats making it die

#

wtf?

#

I don't know hwo to read bash very well but like its there?

wild marsh Jul 28, 2024, 10:41 PM

#

maybe the grep script is swallowing the exit status?

#

why does it pick egrep over grep -E?

grand shadow Jul 29, 2024, 10:25 AM

#

Ill check when Im home

craggy tide Jul 29, 2024, 12:18 PM

#

it's normal

#

it's just a warning

grand shadow Jul 29, 2024, 7:40 PM

#

well

#

that probably explains some things lol

#

seems like grep wants splice

#

I'll just comment it out I cannot bother

wild marsh Jul 29, 2024, 7:48 PM

#

what does your /usr/bin/egrep actually contain?

grand shadow Jul 29, 2024, 7:48 PM

#

wild marsh what does your /usr/bin/egrep actually contain?

its a script printing that message into stderr and calling grep -E

wild marsh Jul 29, 2024, 7:49 PM

#

i assume it execs grep? if so then that should actually pass the error code correctly

grand shadow Jul 29, 2024, 7:49 PM

#

yeah it does

#

but I think the issue is due to the redirections or something grep wanted splice and since its unimplemented mlibc paniced

#

which causes it to try egrep

wild marsh Jul 29, 2024, 7:52 PM

#

strange that egrep apparently works though?

grand shadow Jul 29, 2024, 7:53 PM

#

commenting out that ifdef for the splice call makes grep work with when called like that now

#

time to see if that changes anything in the configure

#

yup, fixed

#

what

#

anyways time to see if this will fix the other issue

#

while it was compiling with stdout piped like that I saw it also complained about diff missing, so I might port that too

#

yuuup

#

saw the checking for ansi c header files... yes

#

aaand libiberty built

#

now its actually configuring gcc itself I think

#

oh

#

oh no

#

do I have a fucking leak

#

yeahhh

#

shit

prime juniper Jul 29, 2024, 9:25 PM

#

At least you bubble it up

#

We just die

#

meme

#

I think I should focus on self hosting after my vacation

analog berry Jul 29, 2024, 9:30 PM

#

prime juniper We just die

Just buy more ram or we assert your ass

grand shadow Jul 29, 2024, 9:35 PM

#

assertos

#

well I wrote a simple tool to see the number of allocations that haven't been free'd
its quick dirty and sloooow but I can wait

analog berry Jul 29, 2024, 9:37 PM

#

Visual lsan

tawdry edge Jul 29, 2024, 9:41 PM

#

Time is infinite

#

Slowness is eventually nothing

fleet osprey Jul 29, 2024, 10:34 PM

#

yknow you can tell QEMU to just give it 1TB of RAM, and put the RAM as a file on large storage
if you just throw enough storage at the problem eventually it goes away :)

grand shadow Jul 29, 2024, 10:36 PM

#

I ended up finding the problem :p

#

I was forgetting to free() the inode/block bitmap in ext2

#

woopsie!

#

ended up also finding some other leaks so those are fixed

fleet osprey Jul 29, 2024, 10:36 PM

#

nice!

grand shadow Jul 29, 2024, 10:52 PM

#

time to try it again

#

configuring gcc, think it passed where it was before

#

nice

#

time for the real test

#

bruh?

#

this is weird, I can run make again which I couldn't last time

#

wait could it possibly be that I'm building it from the main tree

#

hmm

#

yeah otu're not supposed to do that

grand shadow Jul 29, 2024, 11:33 PM

#

that does not look like an oom

#

out of memolry is when you have over 7gb free apparently

#

so maybe a system call return value is borked

#

oh

#

its a zero sized read

#

I am fucking

#

ugh

#

well posix says zero reads should just return 0 and fuck off so thats easy

#

actually wait

#

fuck off after making sure the fd exists and stuff

#

well at least I discovered a few leaks meme

#

YIPPE

#

alright hope this works now

#

fixincludes now

#

I'll leave this running now

#

still going strong

wild marsh Jul 30, 2024, 12:13 AM

#

nice

warped plinth Jul 30, 2024, 12:15 AM

#

nice nice nice

grand shadow Jul 30, 2024, 12:17 AM

#

it has severely slowed down tho lmao

#

taking minutes to compile some of these files

#

but I think thats just cuz of gcc being heavy

fleet osprey Jul 30, 2024, 12:31 AM

#

tbh, I've seen Linux PCs that compile slower than that

#

honestly quite impressive that you've already gotten Astral that fast

grand shadow Jul 30, 2024, 12:35 AM

#

fleet osprey tbh, I've seen Linux PCs that compile slower than that

its kinda slow now that its at the bulk of the gcc stuff

#

a few seconds per file rn

fleet osprey Jul 30, 2024, 12:37 AM

#

idk, that's still pretty good
I have a machine that takes several hours to compile 50% of xmrig, then fails

grand shadow Jul 30, 2024, 12:37 AM

#

damn

#

I hope this doesn't run into issues in the middle of compiling

#

it'll be hell to debug

#

good time to play factorio while I wait tho meme

grand shadow Jul 30, 2024, 12:58 AM

#

well shit

#

this fucking bug

#

it happens so rarely

#

it compiled gcc for an entire hour straight before dying, thats progress

#

I'll leave qemu open to try to get this shit fixed tmr

winged basin Jul 30, 2024, 5:21 AM

#

compiling GCC is an insane torture test

prime juniper Jul 30, 2024, 9:54 AM

#

analog berry Just buy more ram or we assert your ass

AssertOS!

#

I’ll have to subject managarm to the torture test but I’m pretty sure autotools configure will fuck me on that fcntl.h test (issue open on Managarm)

grand shadow Jul 30, 2024, 10:18 AM

#

I was surprised it lasted for as long as it did, but it is a welcome surprise

#

This virtio block bug has been pestering me for a bit but Ive never been able to get to the end of it

#

It happens rarely enough to not be easily reproducible but often enough to annoy me

#

I do think that seems like the only instability bug when running on a single cpu

grand shadow Jul 30, 2024, 10:52 AM

#

My end goal with this is to eventually do some sort of "astral from scratch", where I will build an astral distro capable of building other software from inside astral itself

analog berry Jul 30, 2024, 11:08 AM

#

@grand shadow im kinda bored so uhhh... do u want acpi support in your kernel?

grand shadow Jul 30, 2024, 11:16 AM

#

I dont use it for anything other than the tables and I dont have any system at all for shutdown atm but if you want to play around with it I wont stop you lol

analog berry Jul 30, 2024, 11:16 AM

#

nor does managarm lol

#

we can hook up the power button ig

grand shadow Jul 30, 2024, 11:17 AM

#

Sure I suppose

#

Gl understanding the code meme

analog berry Jul 30, 2024, 11:17 AM

#

is it that bad?

grand shadow Jul 30, 2024, 11:18 AM

#

My coding style might be a bit weird but its muuuuch better tham old astral

analog berry Jul 30, 2024, 11:18 AM

#

also since u have ported a bunch of software can't we just make a shutdown syscall and make it call that?

grand shadow Jul 30, 2024, 11:18 AM

#

Yeah

#

Shutdown program that just calls sync and then shutdown or something

analog berry Jul 30, 2024, 11:19 AM

#

yeah

#

do u have sync?

grand shadow Jul 30, 2024, 11:20 AM

#

Sync and fsync

analog berry Jul 30, 2024, 11:20 AM

#

damn

#

this man is years ahead of managarm meme

analog berry Jul 30, 2024, 11:21 AM

#

grand shadow My coding style might be a bit weird but its muuuuch better tham old astral

whats weird about it? i see u use tabs but other than that

#

u dont separate words in some places ig

grand shadow Jul 30, 2024, 11:24 AM

#

analog berry u dont separate words in some places ig

Mostly this

analog berry Jul 30, 2024, 11:24 AM

#

any rules to when u dont separate to keep it consistent?

grand shadow Jul 30, 2024, 11:27 AM

#

I do the module_function thing and add an arch_ in front when its arch dependant (like arch_mmu_getphysical)

analog berry Jul 30, 2024, 11:27 AM

#

i see

grand shadow Jul 30, 2024, 11:27 AM

#

For structs I always typedef with a _t so page_t for example

#

And macros usually are LIKE_THIS

analog berry Jul 30, 2024, 11:28 AM

#

fair enough

grand shadow Jul 30, 2024, 11:29 AM

#

Also once everything builds you should probably set the distro type to minimal in the Makefile so you dont have to wait for the full initrd to be loaded

#

I should make it so that it only builds the minimal stuff as well

#

Would make compiling it faster if you just want to test the kernel for example

analog berry Jul 30, 2024, 11:30 AM

#

i see

#

is there currently a placeholder shutdown thing anywhere?

grand shadow Jul 30, 2024, 11:30 AM

#

Nope, completely unsupported

analog berry Jul 30, 2024, 11:31 AM

#

would it live in tools?

grand shadow Jul 30, 2024, 11:32 AM

#

Yeah

#

Tools is for the userland part of the os like init the dhcp client etc

analog berry Jul 30, 2024, 11:32 AM

#

i see

#

makes sense so far

#

and i also appreciate just typing make and getting it to work

#

because i absolutely hate fighting with build systems

grand shadow Jul 30, 2024, 11:34 AM

#

Thank mint for jinx lmao

#

Actually wait I just thought of something which could be an issue
@analog berry does uacpi unmap memory directly?

flat copper Jul 30, 2024, 11:40 AM

#

grand shadow well shit

i am thinking that it has to do with you setting queuewaiting[idx] from one core, and the DPC happens on another

grand shadow Jul 30, 2024, 11:41 AM

#

flat copper i am thinking that it has to do with you setting queuewaiting[idx] from one core...

This was running on a single core

analog berry Jul 30, 2024, 11:41 AM

#

grand shadow Actually wait I just thought of something which could be an issue <@35359394852...

wdym by directly

flat copper Jul 30, 2024, 11:41 AM

#

grand shadow This was running on a single core

ah what the fuck

grand shadow Jul 30, 2024, 11:41 AM

#

analog berry wdym by directly

Like it maps some physical memory and then unmaps it later

flat copper Jul 30, 2024, 11:41 AM

#

compiler reordering?

analog berry Jul 30, 2024, 11:41 AM

#

grand shadow Like it maps some physical memory and then unmaps it later

yeah

flat copper Jul 30, 2024, 11:41 AM

#

maybe you need to do volatile with blkdev, like notify = 0 is set before queuewaiting

grand shadow Jul 30, 2024, 11:42 AM

#

analog berry yeah

I think theres a bug in my vmm that causes physical mappings to try to change some page struct that doesnt exist, I think I can fix that later today

#

When you unmap them

#

For now you can just try running sithout unmapping

analog berry Jul 30, 2024, 11:42 AM

#

grand shadow I think theres a bug in my vmm that causes physical mappings to try to change so...

sure but i dont think im even going to actually map anything, astral does use hhdm anyway right

grand shadow Jul 30, 2024, 11:43 AM

#

analog berry sure but i dont think im even going to actually map anything, astral does use hh...

I only map usable and bootloader pages in the hhdm

#

And module pages too

analog berry Jul 30, 2024, 11:43 AM

#

ohh ok

grand shadow Jul 30, 2024, 11:43 AM

#

Not acpi

grand shadow Jul 30, 2024, 11:43 AM

#

flat copper maybe you need to do volatile with blkdev, like notify = 0 is set before queuewa...

Lemme check

analog berry Jul 30, 2024, 11:43 AM

#

well aml accesses reserved ranges, not acpi ranges

#

so anyways

#

but sure ill just not unmap for now

grand shadow Jul 30, 2024, 11:44 AM

#

grand shadow Lemme check

Its not mmio but I could try so, maybe its compiler optimisation fucking me over

#

When slacking off at work today I will see if I can read through the virtio block code again

flat copper Jul 30, 2024, 11:45 AM

#

grand shadow Its not mmio but I could try so, maybe its compiler optimisation fucking me over

yeah, only thing i can think of is if queuewaiting[idx] is reordered after the volatile

#

is that even legal in c?

analog berry Jul 30, 2024, 11:45 AM

#

i mean i doubt its anything that complex

#

if its single core its probably just a logic bug

grand shadow Jul 30, 2024, 11:45 AM

#

Yeah

analog berry Jul 30, 2024, 11:46 AM

#

lemme know when the unmap bug is fixed

grand shadow Jul 30, 2024, 11:48 AM

#

Wont be for another 10 hours, its 8 am rn

analog berry Jul 30, 2024, 11:48 AM

#

np lol im not that fast

grand shadow Jul 30, 2024, 11:48 AM

#

Oh lmao

analog berry Jul 30, 2024, 11:48 AM

#

would a submodule be ok then?

grand shadow Jul 30, 2024, 11:48 AM

#

Sure

#

I probably should switch out my stuff to submodules anyways

#

Rn it just downloads them in the jinx regenerate

#

Since it depends on flanterm, some printf I forgot the name and limine.h

analog berry Jul 30, 2024, 11:51 AM

#

ohh i see

analog berry Jul 30, 2024, 12:23 PM

#

its still building btw lmao

#

..and just finished looks like

grand shadow Jul 30, 2024, 1:48 PM

#

Yeah it takes a bit, it has to compile gcc twice etc

winged basin Jul 30, 2024, 4:59 PM

#

grand shadow Yeah it takes a bit, it has to compile gcc twice etc

Such a frustration, I really hope to figure out how to not do that some day

inland cliff Jul 30, 2024, 5:30 PM

#

grand shadow do I have a fucking leak

Congratulations

wild marsh Jul 30, 2024, 7:50 PM

#

grand shadow Yeah it takes a bit, it has to compile gcc twice etc

at least its less than the 3 gccs you build during a native build

#

native non-cross-target that is

neon crane Jul 30, 2024, 8:00 PM

#

wild marsh at least its less than the 3 gccs you build during a native build

you what mate?

wild marsh Jul 30, 2024, 8:01 PM

#

if you don't pass --disable-bootstrap, compiling gcc will first build gcc with your current system compiler, then compile gcc with the just compiled gcc, then compile gcc with the new just compiled gcc, and it will finally compare the last 2 gccs to see if there are any differences

winged basin Jul 30, 2024, 8:02 PM

#

wild marsh if you don't pass --disable-bootstrap, compiling gcc will first build gcc with y...

this tortured me when i was trying to re-port modren gcc to hp-ux on hppa

#

in the end i got a gcc that passed the test but perl of all things built a dud miniperl

#

perl of all things

#

called one of the most portable programs to ever exist

neon crane Jul 30, 2024, 8:13 PM

#

wild marsh if you don't pass --disable-bootstrap, compiling gcc will first build gcc with y...

lmfao

grand shadow Jul 30, 2024, 8:13 PM

#

wild marsh if you don't pass --disable-bootstrap, compiling gcc will first build gcc with y...

Oh shit I did not know

#

I will pass that flag when building it on astral lmao

wild marsh Jul 30, 2024, 8:13 PM

#

yeah

grand shadow Jul 30, 2024, 8:20 PM

#

Probably just saved me hours of waiting

prime juniper Jul 30, 2024, 9:23 PM

#

Oh yeah gcc uses Perl during its build step. That’s gonna be annoying

grand shadow Jul 30, 2024, 9:32 PM

#

How difficult is it to port?

eager copper Jul 30, 2024, 9:51 PM

#

halfmemeright if dennis says annoying, I'd take that as not so easy

grand shadow Jul 30, 2024, 9:55 PM

#

actually hm

#

nescessary when regenerating makefile dependencies in libiberty

#

I dont think I have to do that?

#

Perl version 5.6.1 (or later)

    Necessary when targeting Darwin, building ‘libstdc++’, and not using --disable-symvers. Necessary when targeting Solaris with Solaris ld and not using --disable-symvers.

    Necessary when regenerating Makefile dependencies in libiberty. Necessary when regenerating libiberty/functions.texi. Necessary when generating manpages from Texinfo manuals. Used by various scripts to generate some files included in the source repository (mainly Unicode-related and rarely changing) from source tables.

    Used by automake.

    If available, enables parallel testing of ‘libgomp’ in case that flock is not available.

#

idk

wild marsh Jul 30, 2024, 9:57 PM

#

if you don't ./bootstrap it's probably not needed?

grand shadow Jul 30, 2024, 9:59 PM

#

thats what I assume since it was building fine up until that point

#

and I think it built libiberty too

prime juniper Jul 30, 2024, 10:17 PM

#

Thank fuck

#

Perl is notably not portable unless you yeet Perl-cross over it but managarm’s Perl still has issues. Like it’ll run basic scripts fine, including a 2048 game, but autoshit has issues finding modules?

#

That means managarm gcc configure stress test is still on for somewhere in the next 2 weeks

grand shadow Jul 30, 2024, 11:05 PM

#

grand shadow well shit

I have no idea wtf is causing this, the lastusedindex and VIO_QUEUE_DEV_INDEX are correct

#

but the index of the buffer in the queue is 0 somehow?

#

actually wait

#

I'm stupid

#

there is something in buffer 0

#

buuut

#

no threads are waiting there

#

there is a thread waiting on buffer 1

#

wait

#

agh

#

I dont know

#

this is so fking strange

#

I use the same dpc code for vionet

#

and it works there without this random crash

#

yeah, there is something on 1 but not 0

#

could it possibly be some very specific race in the enqueue?

#

thinking about it....

#

size_t driveridx = VIO_QUEUE_DRV_IDX(&blkdev->queue)++;
VIO_QUEUE_DRV_RING(&blkdev->queue)[driveridx % blkdev->queue.size] = idx;

sched_preparesleep(false);
blkdev->queuewaiting[idx] = _cpu()->thread;

*blkdev->queue.notify = 0;

I should increment the driver index right before the next notify, otherwise theres a chance the virtio device will process it while its not fully prepared

#

2 threads want to read from the device, one adds it normally and notifies and goes to sleep, and releases the lock
the other one comes in and increments the driver index but after the driver index is set up and before the buffer index is set the device handles the request and the wrong buffer there is used

#

which explains why it happens so rarely

#

that is like

#

the only fucking thing I can think of

#

welp

#

time to do this and pray that was it

#

because I cannot reproduce this bug easily :D

obsidian pivot Jul 30, 2024, 11:25 PM

#

my strategy for debugging OBOS's weird bugs is guess the wildest things that could've possibly happened and investigate that

#

maybe that explains why developement is so slow....

grand shadow Jul 30, 2024, 11:28 PM

#

grand shadow 2 threads want to read from the device, one adds it normally and notifies and go...

@flat copper since you tried to help earlier, I think this is what is happening, with some other things that lead to it being that there was a thread waiting on buffer index = 1 (which only happens when 2 different threads want to access the device at the same time) and the device ring entry was poiting to buffer 0 (which is the one usually used)

#

time to do the vmm fix for uacpi and then leave it building gcc again so I have an excuse to play factorio

inland cliff Jul 30, 2024, 11:40 PM

#

wild marsh at least its less than the 3 gccs you build during a native build

Why does no one use LLVM / Clang? Is it too complicated? It basically does cross anything no need to compile 100 toolchains, just one for the host you want to use it on.

wild marsh Jul 30, 2024, 11:46 PM

#

you still end up needing either gcc for every target, or at least compiler-rt for every target

#

and you still need to patch and recompile llvm and clang to add your custom os target to it

grand shadow Jul 30, 2024, 11:51 PM

#

alright, put it to build again now, hope it doesn't die

#

w/ disable-bootstrap too so it hopefully doesnt last 5 days

#

I wonder how many hobby oses have managed to build gcc

#

I think sortix might have? I remember seeing on the sortix.org page that their release was built under itself

#

though it could be talking about the kernel lol

#

also @analog berry I've pushed the unmapping fix

grand shadow Jul 31, 2024, 1:09 AM

#

its still going, maybe this time is the one

grand shadow Jul 31, 2024, 1:32 AM

#

its linking cc1

#

oh boy

grand shadow Jul 31, 2024, 1:51 AM

#

jesus its still linking cc1

#

and it hasn't deadlocked so thats good

grand shadow Jul 31, 2024, 2:41 AM

#

Finally lonked cc1

#

NOO

#

Welp that was today's attempt

#

Tmr Ill try again and see what couldve caused this

analog berry Jul 31, 2024, 7:18 AM

#

grand shadow also <@353593948523986945> I've pushed the unmapping fix

Thanks

wild marsh Jul 31, 2024, 7:33 AM

#

grand shadow NOO

rip

analog berry Jul 31, 2024, 9:03 AM

#

wild marsh rip

can managarm compile gcc?

wild marsh Jul 31, 2024, 9:03 AM

#

probably no, we still struggle with configure scripts of smaller projects

analog berry Jul 31, 2024, 9:04 AM

#

which part specifically?

wild marsh Jul 31, 2024, 9:32 AM

#

i mean see last messages in managarm-dev in the managarm server :^)

prime juniper Jul 31, 2024, 9:38 AM

#

fcntl autotools check is being nasty

#

and gcc configure gives a broken Makefile

analog berry Jul 31, 2024, 10:05 AM

#

wild marsh i mean see last messages in managarm-dev in the managarm server :^)

Yeah saw it

grand shadow Jul 31, 2024, 9:29 PM

#

back to the grind

#

I was stupid last night and closed the vm without running sync

#

hopefully everything was automatically synced

#

well shit filesystems fucked meme

#

gcc building again, this time I wont be stupid enough to forget to sync hopefully

grand shadow Jul 31, 2024, 10:16 PM

#

Left it building while visiting family, hopefully it doesnt explode

warped plinth Aug 1, 2024, 12:31 AM

#

grand shadow gcc building again, this time I wont be stupid enough to forget to sync hopefull...

idk, would you be able to add a cronjob or something simple like a shell script that runs sync every half hour or so?

grand shadow Aug 1, 2024, 12:41 AM

#

warped plinth idk, would you be able to add a cronjob or something simple like a shell script ...

The worker thread already does that every once in a while

#

I think I left it at 30 seconds I forgot

#

Still, just got home

#

Its still compiling

#

I think it didn't go very far though

#

my laptop went to sleep cuz I forgot to leave it charging

#

its in the middle of the gcc build still I think

#

I dont think it got to the point of linking cc1 yet

#

wtf

warped plinth Aug 1, 2024, 12:57 AM

#

o h

grand shadow Aug 1, 2024, 12:57 AM

#

what was it trying to do there

#

oop finally got to the cc1 linking

#

this is gonna take a while now

grand shadow Aug 1, 2024, 2:07 AM

#

cc1 linked, soon I will know if this will work

#

died compiling parser.cc again

#

syncing the disk this time

#

the first page fault did happen in a library though, assuming libc

#

apparently died on a frigg rbtree remove half leaf function

wild marsh Aug 1, 2024, 2:16 AM

#

that sounds strange

grand shadow Aug 1, 2024, 2:18 AM

#

addr2line -e sysroot/usr/lib/libc.so 0x23fcb
/sysroot/usr/share/mlibc-build/frigg/include/frg/rbtree.hpp:330

#

disassembly does agree with this being the cause of the fault

#

                if(succ)
                        h(succ)->predecessor = pred;

wild marsh Aug 1, 2024, 2:20 AM

#

could be heap corruption of some sort

#

the only place rbtree is used in mlibc is indirectly via the slab allocator

grand shadow Aug 1, 2024, 2:21 AM

#

I load binaries at 0x1000000 so it does seem strange that it has an address kinda close to it

#

well I'm gonna let the disk sync and then run it again and take a look at the memory map

#

aghh I really need to make my disk sync faster lol

obsidian pivot Aug 1, 2024, 2:33 AM

#

wild marsh could be heap corruption of some sort

the oberrow curse has reached even the likes of astral

#

alas; it has gotten too powerful

grand shadow Aug 1, 2024, 2:34 AM

#

if I can get this disk image to be able to reproduce it with just a make under a minute it'll make my life so so so sooo much easier

#

even if I have to manually switch out libc.so or whatever for testing

grand shadow Aug 1, 2024, 2:36 AM

#

obsidian pivot the oberrow curse has reached even the likes of astral

I really hope debugging this wont involve me having to dig deep into mlibc source code or worse gcc

#

(it probably will)

#

Either way I will only be able to work on this friday, tomorrow Im going out with my coworkers after work

craggy tide Aug 1, 2024, 2:54 AM

#

grand shadow ```cpp if(succ) h(succ)->predecessor = p...

https://tenor.com/bFimU.gif

Tenor

grand shadow Aug 1, 2024, 9:10 AM

#

succ

#

disk synced, e2fsck returns no issues

#

Bruh it gotta link cc1 again

#

Ill leave it doing that

grand shadow Aug 1, 2024, 3:46 PM

#

grand shadow aghh I really need to make my disk sync faster lol

I can probably speed it up a bit by having multiple writer threads as well as optimising my disk and fs stuff which sucks atm

#

That will be after I get/give up on getting gcc to compile though

grand shadow Aug 2, 2024, 1:59 PM

#

Hmmmmmm

#

Do I do sound or fat32 first

analog berry Aug 2, 2024, 2:01 PM

#

sound

fleet osprey Aug 2, 2024, 2:41 PM

#

fix the PS/2 code troll

grand shadow Aug 2, 2024, 2:42 PM

#

fleet osprey fix the PS/2 code <:troll:796858892985434152>

mail me one a machine that has issues troll

fleet osprey Aug 2, 2024, 2:43 PM

#

I can't mail you my entire NAS lmao

#

17" x 7" x 14"

#

and weighs a ton

#

it's a big chonky workstation

grand shadow Aug 2, 2024, 2:46 PM

#

Just teach it to fly

#

Damn I cant access the OSS website at work bruh

obsidian pivot Aug 2, 2024, 3:20 PM

#

grand shadow Damn I cant access the OSS website at work bruh

use tor trl

grand shadow Aug 3, 2024, 1:28 PM

#

after (possibly) fixing a double free in ext2, I'm ready to go after the gcc compilation bug again

#

I say possibly because it happens like 30 minutes into linking cc1 from the disk I have so I will know when cc1 links

grand shadow Aug 3, 2024, 2:13 PM

#

yup, seems like that was fixed

#

now onto debugging this shit

#

yeah, it dies on a malloc

#

I should see if I can get this to happen without having to run make

obsidian pivot Aug 3, 2024, 2:19 PM

#

alas, the oberrow curse seems to be upon you

grand shadow Aug 3, 2024, 2:22 PM

#

00000000124a7000-00000000124de000 0000000000037000 urw
00000000125c0000-00000000125d1000 0000000000011000 urw

#

it tries to access between those two ranges

#

and it does seem to be consistent between invocations that 0x12500050 is the faulting address

#

so first I will see when this address gets mapped and unmapped

#

NOOOO I FORGOT TO SCREENSHOT THE FUCKING COMPILATION COMMANDJGKDGNJFSM,KHSFLHFJK

grand shadow Aug 3, 2024, 4:12 PM

#

ok

#

screenshotted

#

though

#

it does seem the address was never unmapped

#

or I added the wrong check

#

I did, infact, add the wrong check

#

aaaa

#

actually wait no

#

I made both printfs the same

#

I am so stupid

#

manually calling the build makes it not segfault

#

https://tenor.com/view/взгляд-2000-ярдов-война-war-soldier-gif-3632617944134077161

Tenor

#

actual insanity

#

welp this is gonna take a while to debug

warped plinth Aug 3, 2024, 6:08 PM

#

yippie!!! :DDD

fleet osprey Aug 3, 2024, 6:10 PM

#

lovely
that's weird as hell
maybe some of the earlier stuff that happens by make consistently puts the system into a state where the bug can trigger?

grand shadow Aug 3, 2024, 8:26 PM

#

ah whatever I give up I don't know how mlibc's malloc works and it takes an entire hour to get to the point where it matters

#

the memory gets unmapped but seemingly the pointer to there still exists

wild marsh Aug 3, 2024, 8:46 PM

#

mlibc's allocator doesn't unmap memory unless it's was large allocation (larger than largest slab)

grand shadow Aug 3, 2024, 8:52 PM

#

wild marsh mlibc's allocator doesn't unmap memory unless it's was large allocation (larger ...

so far from what I've gathered:
something causes that region to get unmapped (I should get a stack trace or something, but that would mean waiting another hour because I seemingly cant reproduce it without doing so which wtf)
the application calls malloc()
and the rbtree functions try to access that memory on /sysroot/usr/share/mlibc-build/frigg/include/frg/rbtree.hpp:330

#

I am this close to just giving up on gcc for now and go read to try to add audio

#

I'll add a breakpoint to syscall_munmap when the range to unmap contains that address and leave it compiling one last time while I do research on audio stuff

prime juniper Aug 3, 2024, 10:31 PM

#

If you get a tangible issue that’s mlibc related, feel free to report it! I’ve been doing the same trying to configure coreutils and sed, and I have more on the todo list

grand shadow Aug 3, 2024, 10:32 PM

#

prime juniper If you get a tangible issue that’s mlibc related, feel free to report it! I’ve b...

I still don't 100% know if its mlibc related, but once I know whats calling that last munmap I'll open an issue

#

I should open that ppoll issue though

grand shadow Aug 4, 2024, 12:04 AM

#

wild marsh mlibc's allocator doesn't unmap memory unless it's was large allocation (larger ...

is that what VirtualAllocator::unmap does? it seems munmap is being called from there

wild marsh Aug 4, 2024, 12:05 AM

#

yeah

#

map is used to allocate both memory for slabs and large objects, but only the latter ever get unmapped

grand shadow Aug 4, 2024, 12:08 AM

#

this is so weird

#

I can't really tell if this is an mlibc issue or an astral issue

#

and atp I'm only gonna make my headache worse if I try

#

should I open an issue about this

wild marsh Aug 4, 2024, 12:09 AM

#

probably a good idea

#

i suppose what we could do is compile all the necessary programs for linux with mlibc and then try compiling gcc

grand shadow Aug 4, 2024, 12:11 AM

#

that is true

wild marsh Aug 4, 2024, 12:12 AM

#

cc @prime juniper

prime juniper Aug 4, 2024, 7:49 AM

#

Time to revive Linux mlibc

#

I’ll see if I can tackle that soon

grand shadow Aug 4, 2024, 12:29 PM

#

time to sit my ass down and read some of the intel hda spec

grand shadow Aug 4, 2024, 1:47 PM

#

huh what

#

I did *something* that made X.org start logging again

#

I don't know what it was but uh

#

its logging now?

#

like when I ported it, it didn't put anything on stdio, only writing to the log file

#

wtf

#

well maybe it was actually handling nbytes=0 lol

prime juniper Aug 4, 2024, 3:01 PM

#

prime juniper Time to revive Linux mlibc

if only I didn't fucking segfault every fucking time I want to chroot into it

grand shadow Aug 4, 2024, 3:28 PM

#

qemu hda doesnt support msix, I think its time I finally stop being lazy and implement normal msi

grand shadow Aug 4, 2024, 3:28 PM

#

prime juniper if only I didn't fucking segfault every fucking time I want to chroot into it

why's it segfaulting?

prime juniper Aug 4, 2024, 3:29 PM

#

if I knew that I wouldn't be wanting to throw something herre

#

I have no coredump nothing

#

/mnt/linux/managarm/linux-mlibc/scripts/lfs.sh: line 62: 1474491 Segmentation fault chroot $ROOT /usr/bin/env -i HOME=/root TERM="$TERM" PATH=/usr/bin:/usr/sbin /bin/bash --login is all I get

grand shadow Aug 4, 2024, 3:33 PM

#

wtf

prime juniper Aug 4, 2024, 3:34 PM

#

ikr

#

if I don't fix this, I can't try compiling gcc and friends on mlibc linux

#

so this is a big yikes

#

and god damn fucking annoying

robust geyser Aug 4, 2024, 5:18 PM

#

prime juniper `/mnt/linux/managarm/linux-mlibc/scripts/lfs.sh: line 62: 1474491 Segmentation f...

strace it

#

or take a look at dmesg

prime juniper Aug 4, 2024, 5:38 PM

#

dmesg is empty

#

strace a chroot call, I guess that could work

#

strace was a good call @robust geyser

fleet osprey Aug 4, 2024, 5:45 PM

#

I've also gdb'd a chroot call, from one Linux system to another, in the situation where python and gdb were both segfaulting in the target
actually works fairly well

neon crane Aug 4, 2024, 5:45 PM

#

wild marsh map is used to allocate both memory for slabs and large objects, but only the la...

why though

#

why not unmap slabs when theyre empty

wild marsh Aug 4, 2024, 7:59 PM

#

because reusing it for later allocations means theyre done faster?

neon crane Aug 4, 2024, 8:16 PM

#

wild marsh because reusing it for later allocations means theyre done faster?

but what if you run into situations where you're low on memory?

warped plinth Aug 4, 2024, 9:18 PM

#

so, why isn't the playstation 2 code working? /j

wild marsh Aug 4, 2024, 11:19 PM

#

neon crane but what if you run into situations where you're low on memory?

page out the heap to swap meme

#

we probably could do something like madvise(MADV_DONTNEED) if the whole page is no longer used

prime mulch Aug 4, 2024, 11:38 PM

#

wild marsh page out the heap to swap <:meme:575445094589661215>

does managarm have that

wild marsh Aug 4, 2024, 11:38 PM

#

we don't have swap no

obsidian pivot Aug 4, 2024, 11:45 PM

#

come on

#

even OBOS has swap*
*not on disk (yet)

neon crane Aug 5, 2024, 5:33 AM

#

wild marsh we probably could do something like `madvise(MADV_DONTNEED)` if the whole page i...

you should consider adding some kind of signal or something broadcast to every running process that says "hey, running low on memory, free whatever you don't need"

inland cliff Aug 5, 2024, 7:51 AM

#

neon crane you should consider adding some kind of signal or something broadcast to every r...

Well, if something's not needed, it should be freed anyway, but you're probably talking about caches and such?

neon crane Aug 5, 2024, 7:53 AM

#

inland cliff Well, if something's not needed, it should be freed anyway, but you're probably ...

qookie is talking about not freeing slabs immediately

#

I assume to avoid a useless syscall

neon crane Aug 5, 2024, 7:53 AM

#

obsidian pivot even OBOS has swap\* \*not on disk (yet)

what kind of swapping does it have

obsidian pivot Aug 5, 2024, 9:34 AM

#

neon crane what kind of swapping does it have

It swaps into a ram buffer for now

wild marsh Aug 5, 2024, 10:14 AM

#

neon crane I assume to avoid a useless syscall

yeah it's probably likely that the memory will just be reused soon enough by subsequent allocations anyway

grand shadow Aug 6, 2024, 3:41 PM

#

Ive opened an issue on mlibc about the allocator issue

prime juniper Aug 6, 2024, 3:52 PM

#

Thanks for the report

grand shadow Aug 6, 2024, 11:21 PM

#

I made my mmap more posixy, time to se if that will help anything

#

I wasn't handling MAP_FIXED properly and failing if there was already a mapping there

#

dont think that'd help much though

#

if it doesn't I will use the mlibc debug allocator

#

I said I wasn't going to try debugging this but meh I want astral to be self-hosting already

grand shadow Aug 7, 2024, 12:59 AM

#

also I might have to port diffutils to make this faster

grand shadow Aug 8, 2024, 12:09 AM

#

ported diffutils, lets see if this will make it not have to relink cc1

#

YEAHHHH IT OESN'T

#

okokokok

#

this makes life so much easier

obsidian pivot Aug 8, 2024, 12:10 AM

#

||Ctrl+C||

#

you know if you don't like doing all this bs, just port Astral to the m68k

#

many studies show that it calms the heart

#

causes inner peace

grand shadow Aug 8, 2024, 1:21 AM

#

looking deeper into it: it seems like the free_huge_ function is called for the slab but the rbtree still seems to have a pointer to the hook?

grand shadow Aug 8, 2024, 2:13 PM

#

grand shadow looking deeper into it: it seems like the free\_huge\_ function is called for th...

@wild marsh do you have any guesses about what could be happening here so I can try to debug this when I get home later?

wild marsh Aug 8, 2024, 2:27 PM

#

not really honestly

grand shadow Aug 10, 2024, 2:56 PM

#

so enabling FRG_SLAB_TRACK_REGIONS gives me this assert fail

#

grand shadow Aug 10, 2024, 3:18 PM

#

h(grand)->color seems to be color_type::red

#

ugh I am too dumb to understand this shit

wild marsh Aug 10, 2024, 3:48 PM

#

grand shadow h(grand)->color seems to be color_type::red

in frg/rbtree.hpp there's an enable_checking constant that you could set to true to have the rbtree code perform checks at various point (during and after operations)

grand shadow Aug 10, 2024, 3:48 PM

#

alright

#

/sysroot/usr/share/mlibc-build/frigg/include/frg/rbtree.hpp:164: Assertion 'check_invariant()' failed!

#

    void insert_right(T *parent, T *node) {
        FRG_ASSERT(parent);
        FRG_ASSERT(!get_right(parent));

        if(enable_checking)
            FRG_ASSERT(check_invariant());```

wild marsh Aug 10, 2024, 3:52 PM

#

does it happen earlier than the slab crash?

#

kinda hard to tell without a backtrace ig

grand shadow Aug 10, 2024, 4:07 PM

#

wild marsh kinda hard to tell without a backtrace ig

Linked list (successor, forward) violation

/sysroot/usr/share/mlibc-build/frigg/include/frg/slab.hpp:508
/build_dir/builds/mlibc/../../../base_dir/sources/mlibc/options/internal/include/mlibc/global-config.hpp:13
then the rest is in cc1plus I assume

grand shadow Aug 10, 2024, 4:35 PM

#

lemme see what the rbtree checks say when FRG_SLAB_TRACK_REGIONS is disabled

grand shadow Aug 10, 2024, 4:38 PM

#

grand shadow lemme see what the rbtree checks say when FRG_SLAB_TRACK_REGIONS is disabled

still just segfaults

wild marsh Aug 10, 2024, 4:58 PM

#

hm

#

hm, does your anon_allocate return memory that's zeroed out?

grand shadow Aug 10, 2024, 5:15 PM

#

wild marsh hm, does your anon_allocate return memory that's zeroed out?

when a page fault happens on an anon range it maps a page that was zero'd at boot and then on a write fault it does CoW with that page so I'm pretty certain it does return memory thats zero'd out. I could try to check if that page is still all zeros on gdb but I'd assume it is

#

yeah, x/4096c (((uint8_t *)zeropage) + 0xffff800000000000) shows everything as 0

grand shadow Aug 10, 2024, 11:55 PM

#

reads from disk are usually going to be page_sized as it will be read into the page cache
currently reading block by block because lazy
change it to read page by page whenever possible
it gets 8x faster

#

I hate my past self with passion

prime mulch Aug 10, 2024, 11:55 PM

#

wait until u do clustered page-in

fleet osprey Aug 11, 2024, 1:24 AM

#

grand shadow >reads from disk are usually going to be page_sized as it will be read into the ...

a lot of issues in a lot of software that I've looked at follow that same pattern
> x is supposed to be done in y way
> currently is doing it in z way because "not enough time", "too lazy", etc
> changes it to be y
> massive measurable improvement

I swear I've seen this pattern crop up elsewhere like, 10 times already
but regardless, nice job, glad you found it and made it better :)

neon crane Aug 11, 2024, 4:20 AM

#

grand shadow >reads from disk are usually going to be page_sized as it will be read into the ...

are these changes to the nvme driver?

#

seems like yeah

#

im going to change my kernel to use an MDL to perform io operations instead of a bare buffer

grand shadow Aug 11, 2024, 1:20 PM

#

neon crane are these changes to the nvme driver?

both to the nvme and virtio block drivers yeah

grand shadow Aug 11, 2024, 3:20 PM

#

currently implementing named fifos, so I can try out the make jobserver and something else I'm curious about

robust geyser Aug 11, 2024, 3:59 PM

#

took me a while to figure out what a j observer is

grand shadow Aug 11, 2024, 4:58 PM

#

#

now I just need to fix smp meme

prime juniper Aug 11, 2024, 5:16 PM

#

Nice

#

You don’t run into autotools wanting to run?

#

Like on managarm, we do configure (cache a few items), then issue make, but make then invokes autoconf and friends and we don’t have that (nor should it be required)

#

We also build from a tarball

grand shadow Aug 11, 2024, 5:18 PM

#

not really, I dont even have automake or autoconf ported

prime juniper Aug 11, 2024, 5:18 PM

#

Yeah

grand shadow Aug 11, 2024, 5:18 PM

#

prime juniper We also build from a tarball

I wget it directly from the gnu mirror

prime juniper Aug 11, 2024, 5:19 PM

#

The tarball that is right?

grand shadow Aug 11, 2024, 5:19 PM

#

yeah

prime juniper Aug 11, 2024, 5:19 PM

#

Yeah same

grand shadow Aug 11, 2024, 5:19 PM

#

ftp.gnu.org/gnu/hello

prime juniper Aug 11, 2024, 5:19 PM

#

(From the host into the sysroot but that shouldn’t make a difference)

grand shadow Aug 11, 2024, 5:20 PM

#

I do it directly from astral meme

prime juniper Aug 11, 2024, 5:20 PM

#

I could do that too (and have done so)

#

But can’t be bothered

#

And our update image command will nuke whatever I do inside the image unless it’s in the sysroot

#

I guess we can change the rsync command to not do that but meh

lean kettle Aug 11, 2024, 5:22 PM

#

prime juniper I guess we can change the rsync command to not do that but meh

i mean it probably would be a good idea

grand shadow Aug 11, 2024, 5:22 PM

#

the build system only generates the .iso but it leaves a sysroot folder that I have a handy script to use for creating a disk image

lean kettle Aug 11, 2024, 5:22 PM

#

it would amount to maybe removing --delete to the rsync command we use for the /root folder

prime juniper Aug 11, 2024, 5:23 PM

#

Probably

robust geyser Aug 11, 2024, 5:41 PM

#

grand shadow now I just need to fix smp <:meme:575445094589661215>

based

#

good work sir

grand shadow Aug 11, 2024, 5:47 PM

#

tyty

neon crane Aug 11, 2024, 5:50 PM

#

grand shadow now I just need to fix smp <:meme:575445094589661215>

all that for a hello world? /s

#

jokes aside, good job, hopefully you can get to compile gcc on astral very soon 🤞

grand shadow Aug 11, 2024, 7:56 PM

#

speaking of compile gcc, @prime juniper did you ever manage to get mlibc linux to work

prime juniper Aug 11, 2024, 7:59 PM

#

Not yet. I get further on configure but flex is segfaulting

grand shadow Aug 11, 2024, 9:27 PM

#

so now I am faced with the choice of what to do next:

continue debugging gcc
audio
actually support multiple users and stuff
fat filesystem

#

or the 5th option which is continue procrastinating on ftl

analog berry Aug 11, 2024, 9:30 PM

#

i'd say do the first to be officially better than managarm as well as make a meaningful contribution to mlibc by fixing its bugged rb tree

prime juniper Aug 11, 2024, 10:06 PM

#

It’s a miracle we haven’t ran into that rb tree thing

#

So that would be nice

grand shadow Aug 11, 2024, 10:44 PM

#

Failed node: 0xff00000 sucessor(node) is 0xff80000 succ is 0x12500000

#

                        if(successor(node) != succ) {
                                mlibc::infoLogger() << "Linked list (successor, forward) violation" << frg::endlog;
                                mlibc::infoLogger() << "Failed node: " << node << " got sucessor(node) is " << (void *)successor(node) << " succ is " << (void *)succ << frg::endlog;
                                return false;

#

I have no idea what I am doing

wild marsh Aug 12, 2024, 6:32 AM

#

what does _less(*successor(node), *succ) return?

grand shadow Aug 12, 2024, 5:45 PM

#

wild marsh what does `_less(*successor(node), *succ)` return?

/sysroot/usr/share/mlibc-build/frigg/include/frg/rbtree.hpp:627:99: error: ‘_less’ was not declared in this scope
  627 |                                 mlibc::infoLogger() << "_less(*successor(node), *succ): " << _less(*successor(node), *succ) << frg::endlog;

grand shadow Aug 13, 2024, 10:38 PM

#

honestly I think I will just get the multiuser stuff out of the way

#

the longer I wait the more work it will be down the line

#

since I'll have to go through a bunch of stuff to make sure it does the right calls to check for permissions etc

#

while I scribble that out on paper I will leave gcc building with -j2 and the less shit disk rw to see how stable it is on j2 and how faster it willbe now

#

I think the mlibc logging might be starting to slow down the configure a bit

#

and a print I forgot oops

#

still seems much faster than before

#

and I'm leaving it building on an xterm window now rather than the system console

warped plinth Aug 14, 2024, 4:33 AM

#

how'd the gcc build go?

grand shadow Aug 14, 2024, 10:32 AM

#

Had an assert fail but I know whats going on

grand shadow Aug 14, 2024, 8:12 PM

#

I was wondering why everything was running so slow and was starting to think I did some shit in the kernel that slowed everything down
then I remembered I left jinx to build at -O0 so I could debug mlibc better oops

#

gcc build again

#

time to see if it reaches the mlibc bug with j2

grand shadow Aug 14, 2024, 8:38 PM

#

also I should port xscreensaver for the silly

grand shadow Aug 14, 2024, 10:18 PM

#

it did reach the bug

#

much faster this time

grand shadow Aug 14, 2024, 10:58 PM

#

xscreensaver seems to be checking for the libraries on the configure --libdir path this is crazy

#

how does someone fuck autotools up this badly

#

I had to manually remove all the -L$libdir and -I$includedir shit from LDFLAGS and CFLAGS/CXXFLAGS in configure.ac

grand shadow Aug 14, 2024, 11:37 PM

#

sooo cross compilation friendly jesus christ

#

and to top it all off, it uses $(install_prefix) instead of $(DESTDIR)

#

IT KNOWS ITS NOT AVAILABLE
AND IT TRIES TO INSTALL IT ANYWAYS

#

oh my god

obsidian pivot Aug 14, 2024, 11:45 PM

#

lmao

#

gl

grand shadow Aug 15, 2024, 12:02 AM

#

good news! it starts fading the screen out
bad news! it segfaults in some library

#

trying to figure out now

#

it dies in that if (*xlfd != '-') check

grand shadow Aug 15, 2024, 12:47 AM

#

some sscanf seems to be failing and making it pass null to the name

#

time to see if mlibc bug or smth else

#

after I make dinner

grand shadow Aug 15, 2024, 1:41 AM

#

oh huh mlibc doesn't implement %f

#

for scanf

#

it seems

#

guess I'm gonna open an issue about that

#

and just patch it out in xscreensaver to use %d or whatever

#

it seems like it just

#

gets the float and then multiplies it by 10 and casts it to into in a sprintf later

#

I could get to implementing it tomorrow or something but idk if I'll do it in the right way

#

warped plinth Aug 15, 2024, 2:03 AM

#

nice :3

grand shadow Aug 15, 2024, 2:10 AM

#

thats one more motivation to add multiuser support, screensavers

grand shadow Aug 16, 2024, 9:40 PM

#

for today I will implement the system calls for set*id

#

and get*id

grand shadow Aug 16, 2024, 10:34 PM

#

prime juniper Aug 17, 2024, 6:22 AM

#

Nice work

grand shadow Aug 17, 2024, 11:36 PM

#

@prime juniper I'm getting the itch

#

does webkitgtk need any procfs memes?

robust geyser Aug 18, 2024, 1:02 AM

#

grand shadow

did you teach vfs about permissions yet? :^)

grand shadow Aug 18, 2024, 1:02 AM

#

robust geyser did you teach vfs about permissions yet? :^)

I still need to see how I am going to do that

#

I am probably gonna do something similar to kauth(9) per fadanoid's suggestion

#

as tomorrow I wont be as tired (hate that my university does exams on saturdays) I will probably set some actual goals for what I want:

llvm and mesa
ultimately decide how to stop people from doing things they shouldn't
actually go around the kernel and implement all of these checks I haven't

#

the last one I probably wont fit in a day since there are a lot of places and cases

prime juniper Aug 18, 2024, 8:35 AM

#

grand shadow does webkitgtk need any procfs memes?

It really really really wants /proc/self/maps in the Linux code paths

#

But; here’s a good tip. Get LLVM and Mesa with LLVMPIPE up and running

#

You’ll thank me later

prime juniper Aug 18, 2024, 8:37 AM

#

prime juniper It really really really wants /proc/self/maps in the Linux code paths

(It also wants some netlink BS in those code paths)

errant zephyr Aug 18, 2024, 9:23 AM

#

prime juniper It really really really wants /proc/self/maps in the Linux code paths

cant you use the not linux code paths though

prime juniper Aug 18, 2024, 9:30 AM

#

You can, by claiming to be one of the BSDs for example

#

Or Darwin probably

#

I don’t know how painful those are

#

But they’ll require some BSD items likely. Pretty sure I saw kqueue somewhere in the BSD paths

#

(Oh and you’ll want to make sure that whatever you claim to be, you claim the same to glib)

errant zephyr Aug 18, 2024, 9:38 AM

#

prime juniper You can, by claiming to be one of the BSDs for example

or writing your own port

#

for that matter you could try writing your own complete webkit port

prime juniper Aug 18, 2024, 9:58 AM

#

Yep, that can be done too

grand shadow Aug 18, 2024, 2:38 PM

#

I wonder which would be harder, own port or pretending to be someone else

grand shadow Aug 18, 2024, 2:38 PM

#

prime juniper But; here’s a good tip. Get LLVM and Mesa with LLVMPIPE up and running

yeah thats the first thing I'm gonna do today

#

the fact that the first thing is namd wtf is comforting

errant zephyr Aug 18, 2024, 3:28 PM

#

grand shadow I wonder which would be harder, own port or pretending to be someone else

probably the former, especially if you don't have a lot of linuxy things

prime juniper Aug 18, 2024, 6:14 PM

#

grand shadow the fact that the first thing is namd wtf is comforting

Webkit Template Framework (or, if I’m debugging it, which I did, What in The actual Fuck is it doing this time!?)

prime juniper Aug 18, 2024, 6:14 PM

#

errant zephyr probably the former, especially if you don't have a lot of linuxy things

Yes, likely

errant zephyr Aug 18, 2024, 6:15 PM

#

im pretty sure webkit is intended to be really portable so

#

it cant be that bad

prime juniper Aug 18, 2024, 6:30 PM

#

It’s still a bunch of effort tho, and may require a glib backend too. As webkit calls into some glib memes on Linux (likely other platforms too) which I know on Linux will use netlink

grand shadow Aug 18, 2024, 7:38 PM

#

I wonder why the hell is gcc not building libstdc++-v3 with -fPIC

grand shadow Aug 19, 2024, 12:21 AM

#

grand shadow I wonder why the hell is gcc not building libstdc++-v3 with -fPIC

no idea what happened there but adding a patch I found in the gcc mailing list it was fixed lmao

#

llvm takes so long to compile they should've written it in c instead of c++ smh

grand shadow Aug 19, 2024, 1:46 AM

#

yeah no I'm just not building clang with this

#

just llvm so I can use it with mesa

prime juniper Aug 19, 2024, 4:31 AM

#

Understandable

craggy tide Aug 19, 2024, 6:20 AM

#

grand shadow llvm takes so long to compile they should've written it in c instead of c++ smh

llvm+clang with all targets enabled takes 10 minutes on my machine :p

prime juniper Aug 19, 2024, 6:57 AM

#

cries in slow

grand shadow Aug 19, 2024, 9:08 AM

#

My poor thing started swapping like crazy at the end of it and I had to turn down the job count lmao

dim arrow Aug 19, 2024, 11:11 AM

#

llvm for me used to take 5+ hours to compile....

fleet osprey Aug 19, 2024, 1:33 PM

#

I honestly wonder how bad that would be if I tried it on my iBook G3, if it takes fairly good PCs hours.......

robust geyser Aug 19, 2024, 7:04 PM

#

it took around 30 minutes on a 4690k, and that is not decent by todays standards

#

and that was just llvm, no clang

#

and one target

obsidian pivot Aug 19, 2024, 7:08 PM

#

TO(not)DO: Compile LLVM

grand shadow Aug 20, 2024, 12:39 AM

#

no progress today because I had to speedrun an assignment

#

did what was supposed to be a group assignment solo in 6 hours

#

but it was the best snake clone I've ever written :')

#

I do have a bad headache now but worth it for the like 15% on the file grade (I was going to pass that class with or without it)

grand shadow Aug 20, 2024, 1:15 AM

#

now do I want to finish the mesa port today

#

is it supposed to take this much space :(

wild marsh Aug 20, 2024, 1:17 AM

#

did you forget to -DCMAKE_BUILD_TYPE=Release?

grand shadow Aug 20, 2024, 1:17 AM

#

nope

#

it is full of .a files

wild marsh Aug 20, 2024, 1:17 AM

#

oh

grand shadow Aug 20, 2024, 1:17 AM

#

wait does vinix recipes delete them

#

seems like so

#

down to 3.7g now

wild marsh Aug 20, 2024, 1:20 AM

#

what targets are you building it for

grand shadow Aug 20, 2024, 1:20 AM

#

wild marsh what targets are you building it for

        cmake \
        -GNinja \
        -DCMAKE_TOOLCHAIN_FILE=${base_dir}/util/cmake \
        -DCMAKE_INSTALL_PREFIX=${prefix} \
        -DCMAKE_BUILD_TYPE=Release \
        -DLLVM_LINK_LLVM_DYLIB=ON \
        -DLLVM_ENABLE_FFI=ON \
        -DLLVM_ENABLE_EH=ON \
        -DLLVM_ENABLE_RTTI=ON \
        -DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-astral \
        -DLLVM_HOST_TRIPLE=x86_64-astral \
        -DLLVM_TARGETS_TO_BUILD="X86" \
        -Wno-dev \
        ${source_dir}/llvm

        ninja -j${parallelism}
}

package() {
        DESTDIR="${dest_dir}" ninja install

        find ${dest_dir} -name '*.a' -delete
}```

#

only x86 I think

wild marsh Aug 20, 2024, 1:21 AM

#

4GB sounds about right i think

#

for llvm + clang

grand shadow Aug 20, 2024, 1:21 AM

#

I dont think I built clang

#

yeah I disabled that

#

would take too long

wild marsh Aug 20, 2024, 1:22 AM

#

oh right this is for astral, not for the build machine

#

seems a bit large then hm

grand shadow Aug 20, 2024, 1:25 AM

#

most of it is still in libLLVM

#

llvm-exegesis is 1.2g

#

is that nescessary

#

according to the website its a benchmark tool

wild marsh Aug 20, 2024, 1:26 AM

#

strange

#

qookie@selenium ~/projects/managarm/build-x86 λ ls -lh system-root/usr/bin/llvm-exegesis 
-rwxr-xr-x 1 qookie qookie 48M Jan  1  1970 system-root/usr/bin/llvm-exegesis

grand shadow Aug 20, 2024, 1:26 AM

#

wtf

wild marsh Aug 20, 2024, 1:26 AM

#

(don't mind the ctime)

grand shadow Aug 20, 2024, 1:27 AM

#

maybe its cuz I dont strip the executables?

wild marsh Aug 20, 2024, 1:27 AM

#

well -DCMAKE_BUILD_TYPE=Release should build without -g so no debug info should be generated

#

one thing that's different in the managarm recipe is that we don't pass -DLLVM_ENABLE_FFI=ON or -DLLVM_ENABLE_EH=ON

#

we do pass -DLLVM_ENABLE_RTTI=ON though, not sure why we don't pass the other ones

grand shadow Aug 20, 2024, 1:29 AM

#

I just stole the vinix recipes which I assume are supposed to work fine

#

oh my god I AM SO STUPID

#

I left -g in my jinx-config

#

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

wild marsh Aug 20, 2024, 1:30 AM

#

oh lol

#

that'd do it

fleet osprey Aug 20, 2024, 1:32 AM

#

lol, it happens
wonder how different the build time is gonna be now

grand shadow Aug 20, 2024, 1:33 AM

#

hopefully not too long

#

if it does I should just go to sleep I am exhausted

#

either way, a friend of mine is playing around with it and did some stuff

grand shadow Aug 20, 2024, 2:15 AM

#

grand shadow aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa

Went down to 200mb, much better

#

Tmr Ill port mesa

grand shadow Aug 20, 2024, 1:58 PM

#

If anyone feels like trying it out, I have a disk image and boot disk in my server:

first option will boot to fvwm running from disk
second option will boot to a login prompt
third option will boot to the system console running from disk
and fourth option will boot to the system console running from a minimal initrd

to configure networking run netd vionet0
if youre in the system console and want to run xorg run startwm

https://avalios.dev/bootdisk.iso
https://avalios.dev/root.img
qemu-system-x86_64 -enable-kvm -M q35 -cdrom bootdisk.iso -m 4g -debugcon stdio -drive file=root.img,if=none,id=nvme -device nvme,serial=deadc0ff,drive=nvme -boot order=dc -netdev user,id=net0 -device virtio-net,netdev=net0

#

Its a few days old but should still work fine

grand shadow Aug 20, 2024, 6:50 PM

#

anyways porting mesa atm

#

soon 3d graphics will be real

fleet osprey Aug 20, 2024, 8:20 PM

#

get accelerated 3D graphics troll

prime juniper Aug 20, 2024, 8:23 PM

#

yeah about that

#

that's certified pain

#

but, Managarm is working on a library style project to add support for this and other GPU related items (multi display support for example) for at least intel iGPUs

fleet osprey Aug 20, 2024, 8:26 PM

#

ooh

analog berry Aug 20, 2024, 9:07 PM

#

grand shadow anyways porting mesa atm

Daaamn

#

Good luck

grand shadow Aug 20, 2024, 9:21 PM

#

/sysroot/usr/lib/libX11.so /sysroot/usr/lib/libz.so

#

 0x0000000000000001 (NEEDED)             Shared library: [libX11.so.6]
 0x0000000000000001 (NEEDED)             Shared library: [/sysroot/usr/lib/libz.so]```

#

???????

#

hm according to [random github issue I found] this happens because libz doesnt have an SONAME set

#

yeah

#

https://tenor.com/view/william-afton-burning-in-hell-gif-22798912

Tenor

#

time to see if I can fix this in zlib

craggy tide Aug 20, 2024, 9:30 PM

#

grand shadow hm according to [random github issue I found] this happens because libz doesnt h...

something something build-support/checksoname.sh and also backporting changes from vinix

prime juniper Aug 20, 2024, 9:33 PM

#

Yep

grand shadow Aug 20, 2024, 10:53 PM

#

I need to implement mprotect

#

fun

warped plinth Aug 21, 2024, 1:14 AM

#

grand shadow either way, a friend of mine is playing around with it and did some stuff

awh sick

grand shadow Aug 21, 2024, 10:39 AM

#

The good thing about having someone actually play around with your os is that they end up finding bugs you likely wouldnt

obsidian pivot Aug 21, 2024, 12:53 PM

#

wish I could relate

grand shadow Aug 21, 2024, 10:51 PM

#

wonder how long until I hit 100 syscalls

#

currently at 84

#

well glxgears doesnt crash anymore because of the lack of mprotect
but it also doesnt display anything lmao

#

wtf?

#

rip is stuck at 0xebe000

#

what even

#

stack trace time

#

I wonder if

#

its something in llvm

#

or like

#

mesa

#

wait

#

it could be that the shaders are being mapped as executable

#

but something is breaking

#

since it does seem to be handling nonstop page faults from gdb

#

cr2 is ebe

#

that is it

#

well time to see whats up

#

ohhh wait

#

lemme see something

#

its page faulting but the vmm_pagefault function is returning ok (otherwise it'd segfault)

#

seems like so

#

yeah

#

I know whats up

#

the mmuflags change but the ones in the mapping don't

#

I should update those

#

oopsie!

#

yeah

#

it goes like this:
vmm_changemap is called, the range->mmuflag value changes, the ones in the actual PTEs dont, and so on a page fault it checks if everything is okay by checking the range and sees that everything is ok so it returns success

#

but in the eyes of the cpu running that code is not valid

#

so it just keeps page faulting at 0xebe000 trying to run it

prime juniper Aug 21, 2024, 11:23 PM

#

Whoops

grand shadow Aug 21, 2024, 11:23 PM

#

easy fix probs

grand shadow Aug 22, 2024, 12:10 AM

#

hey

#

(the console bleeding into the screen was because of xterm printing to it)

craggy tide Aug 22, 2024, 12:12 AM

#

pretty neat

analog berry Aug 22, 2024, 12:13 AM

#

Big

grand shadow Aug 22, 2024, 12:14 AM

#

doesnt die with 5 so its a win

#

wonder if I could make sdl and tyr-quake use opengl now

#

first I'll upstream this

craggy tide Aug 22, 2024, 12:18 AM

#

grand shadow doesnt die with 5 so its a win

that saddens me as vinix dies with >1

#

but i think it's a scheduler issue of some sort

grand shadow Aug 22, 2024, 12:18 AM

#

craggy tide but i think it's a scheduler issue of some sort

do you have any guesses about what could be happening?

analog berry Aug 22, 2024, 12:19 AM

#

Btw is the rbtree bug still unsolved?

craggy tide Aug 22, 2024, 12:19 AM

#

nah, i haven't looked into it

grand shadow Aug 22, 2024, 12:19 AM

#

you should go back to working on vinix more, its cool to see your progress there

craggy tide Aug 22, 2024, 12:19 AM

#

i am waiting to be paid

grand shadow Aug 22, 2024, 12:20 AM

#

analog berry Btw is the rbtree bug still unsolved?

yeah I have no idea where to even begin (I am severely lacking on some CS fundamentals)

analog berry Aug 22, 2024, 12:20 AM

#

Sadge

craggy tide Aug 22, 2024, 12:20 AM

#

if i worked on it seriously even when i am not paid then who pays me would have no incentive to do so

#

(also i need money for food and shelter)

analog berry Aug 22, 2024, 12:20 AM

#

Thats fair enough

grand shadow Aug 22, 2024, 12:20 AM

#

yeah thats fair

#

hopefully they pay you soon

analog berry Aug 22, 2024, 12:20 AM

#

Are you going to get paid?

lean kettle Aug 22, 2024, 12:20 AM

#

hopefully you get paid soon!

craggy tide Aug 22, 2024, 12:20 AM

#

plus i've been busy with Limine

#

yeah i think maybe sept

analog berry Aug 22, 2024, 12:21 AM

#

Nice

craggy tide Aug 22, 2024, 12:22 AM

#

but yeah i am happy to see all the astral progress

#

honestly my favourite project here

grand shadow Aug 22, 2024, 12:22 AM

#

ty :3

lean kettle Aug 22, 2024, 12:22 AM

#

astral is very impressive, agreed

analog berry Aug 22, 2024, 12:22 AM

#

Yeah it makes porting stuff seem trivial

lean kettle Aug 22, 2024, 12:22 AM

#

although i am obligated to say that managarm is my favourite project halfmemeright

grand shadow Aug 22, 2024, 12:22 AM

#

analog berry Yeah it makes porting stuff seem trivial

even I am surprised at how well it runs

#

I have pretty much removed the initrd option from my local files since I am just running from disk 24/7 now

#

and it runs just fine, you dont even notice that much of a difference after a certain point

#

certainly there is still a lot to be better ~~cough smp cough~~

craggy tide Aug 22, 2024, 12:24 AM

#

funnily enough Vinix runs worse with 1 core than with more than 1

#

whatever scheduler issue it has makes it much worse when it's running with only 1 core

grand shadow Aug 22, 2024, 12:26 AM

#

that is crazy

#

when you find the issue lmk cuz Im interested

craggy tide Aug 22, 2024, 12:26 AM

#

sure

#

(if i remember to)

#

i'll probably post in the vinix thread

grand shadow Aug 22, 2024, 12:27 AM

#

the biggest issues with smp in astral are my slow tlb shootdown (like its insane) and just general instability

#

my tlb shootdown just sends an ipi for every page unmapped

#

which is bad

#

but it was the easiest solution and I am a very lazy person

analog berry Aug 22, 2024, 12:29 AM

#

grand shadow but it was the easiest solution and I am a very lazy person

Aren't we all

grand shadow Aug 22, 2024, 12:29 AM

#

true

#

I do look forward to the day I build gcc with make -jwhatever in astral and it compiles with several cores perfectly stable

craggy tide Aug 22, 2024, 12:34 AM

#

Vinix does not have TLB shootdown

#

i think i may force all threads of a process to run on the same CPU for the time being as a poor man's TLB shootdown

lean kettle Aug 22, 2024, 12:37 AM

#

Maybe I should try adding CPU migration to the Managarm scheduler again

grand shadow Aug 22, 2024, 12:45 AM

#

I just juggle threads around cpus with no care at all

lean kettle Aug 22, 2024, 12:50 AM

#

we dont juggle threads at all

#

when a process is created the CPU that thor puts it on is the one it stays on

grand shadow Aug 22, 2024, 12:51 AM

#

the two ends of the spectrum meme

#

the only places I force a thread to be bound to a cpu is in some per cpu timer handling code and stuff

#

and waiting for timeouts

#

because I use the per cpu timers for almost everything

lean kettle Aug 22, 2024, 12:53 AM

#

i wonder how cancer it would be to implement some sort of cpu performance counter monitoring

grand shadow Aug 22, 2024, 12:54 AM

#

ig you could just measure how much it spends on the idle thread per second or something

lean kettle Aug 22, 2024, 12:54 AM

#

not really even talking about using it for scheduling but just knowing them per thread

#

(at least intel CPUs have perf counters for things like cache misses and pipeline stalls and stuff)

grand shadow Aug 22, 2024, 12:54 AM

#

ohhhhhhh

#

I have no idea

lean kettle Aug 22, 2024, 12:55 AM

#

cache misses might be a nice thing to monitor as well for scheduling

#

if a thread constantly has cache misses due to it being chonking rescheduling it on another core could™️ be less damaging

#

while a thread that has almost no cache misses probably shouldnt be rescheduled if possible

grand shadow Aug 22, 2024, 12:57 AM

#

true

#

I should do something more with priorities in my scheduler some day, I have them implemented but its also barely used outside of just kernel threads having a higher priority

#

doesnt even preempt if a thread with higher priority becomes available yet

#

damn it starts but uh its kinda broken

#

too lazy to debug that

#

and it does seem like the gl version runs worse

#

I guess llvmpipe is slower than whatever the guy who made tyr-quake managed to make

#

<----- coping that something in astral is SLOW

lean kettle Aug 22, 2024, 1:06 AM

#

not impossible that whatever tyr-quake does is faster for tyr-quake specifically ig

robust geyser Aug 22, 2024, 1:06 AM

#

< 60 fps in glxgears is kinda bad ngl

#

i think vinix does like 400

lean kettle Aug 22, 2024, 1:06 AM

#

llvmpipe has to be able to do everything after all

robust geyser Aug 22, 2024, 1:06 AM

#

so you definitely have got some optimizing to do

lean kettle Aug 22, 2024, 1:06 AM

#

robust geyser < 60 fps in glxgears is kinda bad ngl

also that yeah

#

managarm gets quite a bit more than that iirc

grand shadow Aug 22, 2024, 1:07 AM

#

hmmm true

#

gonna leave it running with mint's profiler for a like a minute

#

ringbuffer_reads and ringbuffer_write are the two biggest costs

#

I should try optimising that then

robust geyser Aug 22, 2024, 1:10 AM

#

inline them

grand shadow Aug 22, 2024, 1:10 AM

#

I rw byte by byte

#

I think I should begin by making that better

lean kettle Aug 22, 2024, 1:11 AM

#

https://tenor.com/view/sack-boy-shocked-turn-gif-6205091617477847398

Tenor

robust geyser Aug 22, 2024, 1:11 AM

#

holy balls

grand shadow Aug 22, 2024, 1:11 AM

#

listen I just wrote the ringbuffer routines trying to get it out of the way asap the performance didnt matter until now 😭

#

my memcpy is also byte by byte :')

#

memset too

#

agh

#

first going to make ringbuffer_write/read call memcpy then optimise that

fleet osprey Aug 22, 2024, 1:17 AM

#

grand shadow damn it starts but uh its kinda broken

yknow what that's damn playable

#

it may not be fast for running quake on a modern pc

#

but it's still playable

#

which is still impressive

craggy tide Aug 22, 2024, 1:19 AM

#

robust geyser i think vinix does like 400

it does 1400 for me

robust geyser Aug 22, 2024, 1:21 AM

#

ok yeah i misremembered terribly

grand shadow Aug 22, 2024, 1:31 AM

#

streamlined that, 120 fps now

#

still gotta make it better

#

I assume the slow is in memcpy now

#

yup

#

gonna optimize both memset and memcpy

#

I should make my memcpy just rep mosvb

#

it should be optimised enough for anything in the last decade

grand shadow Aug 22, 2024, 1:43 AM

#

grand shadow I should make my memcpy just rep mosvb

on x86 at least

prime juniper Aug 22, 2024, 1:56 AM

#

lean kettle Maybe I should try adding CPU migration to the Managarm scheduler again

Please do

lean kettle Aug 22, 2024, 1:56 AM

#

prime juniper Please do

am working on lil atm

#

specifically getting geminilake working again

prime juniper Aug 22, 2024, 1:58 AM

#

Also highly based

#

Maybe after lil then

#

That perf improvement would be big I can imagine

grand shadow Aug 22, 2024, 1:59 AM

#

craggy tide it does 1400 for me

with how many cores is this

obsidian pivot Aug 22, 2024, 1:59 AM

#

probably around the same amount

#

(1400 cores)

grand shadow Aug 22, 2024, 2:03 AM

#

after bettering some things it runs at like 190-200fps

#

better than 60 but aghhhh

craggy tide Aug 22, 2024, 2:05 AM

#

grand shadow with how many cores is this

i don't think it matters but it's like 4

grand shadow Aug 22, 2024, 2:06 AM

#

quake is decently playable now

#

this had the side effefct of making it run better on the normal tyr-quake version too

#

the gl one does have weird graphical glitches

#

and seems faster?

#

what

fleet osprey Aug 22, 2024, 2:09 AM

#

that's downright smooth

#

wtf

#

in llvmpipe?

#

I can't even get that much perf through llvmpipe on Linux

grand shadow Aug 22, 2024, 2:14 AM

#

it does seem like llvmpipe uses multiple threads and cpus

#

ah well I am going to blame it on multicore and my cpu being slower meme

#

i5-10300h

#

searching glxgears in the discord I found something funny

wild marsh Aug 22, 2024, 2:16 AM

#

if you're on amd and use -cpu host (so that cpuid is passed through) mesa will even try to use sched_{get,set}affinity to force the opengl thread to be on the same ccd as the main thread :^)

grand shadow Aug 22, 2024, 2:16 AM

#

@craggy tide #voice-0 message I ran into this exact issue

#

turns out mesa wasnt installing a glx.pc

#

so I just added a case like it does with darwin

craggy tide Aug 22, 2024, 2:18 AM

#

ah thanks for letting me know

grand shadow Aug 22, 2024, 2:18 AM

#

I was also confused about it but I decided to dive into the meson.build and discovered that

fleet osprey Aug 22, 2024, 2:20 AM

#

grand shadow i5-10300h

mine is about 1.7x faster - i3-12100f
if you want I can try to run it on there and see how well it compares to some numbers I have with Linux on the same machine from earlier

lean kettle Aug 22, 2024, 2:20 AM

#

The thing with glxgears is that it's so fast it's not a useful benchmark

#

(at least when running on Linux on a pc made in the last 30 years)

fleet osprey Aug 22, 2024, 2:21 AM

#

on my Wii running Linux with llvmpipe I don't think it runs all that well

#

like, sub 10fps

grand shadow Aug 22, 2024, 2:21 AM

#

fleet osprey mine is about 1.7x faster - i3-12100f if you want I can try to run it on there a...

I'll probably update the image on avalios.dev tomorrow after I fix up some more ports and fix a bug or two that my friend told me about

#

would be cool

fleet osprey Aug 22, 2024, 2:22 AM

#

👍 will probably test it over the weekend then

grand shadow Aug 22, 2024, 2:22 AM

#

but yeah for now 200 fps is good enough ™️

grand shadow Aug 22, 2024, 2:52 AM

#

I also figured out how I am gonna handle oom on page in

#

sigbus meme

#

its what linux does when the oom killer is disabled it seems

#

and posix also seems to allow it

#

An implementation may generate SIGBUS signals when a reference would cause an error in the mapped object, such as out-of-space condition.

grand shadow Aug 22, 2024, 3:35 AM

#

grand shadow If anyone feels like trying it out, I have a disk image and boot disk in my serv...

@fleet osprey its on the server with mesa now

fleet osprey Aug 22, 2024, 3:43 AM

#

👍

#

If I get a chance I'll try it today

prime juniper Aug 22, 2024, 8:36 AM

#

grand shadow but yeah for now 200 fps is good enough ™️

it's faster than managarm that's for sure

grand shadow Aug 22, 2024, 10:44 AM

#

I mean it seems fast enough

#

At least it seemed to have increased perfirmance on all things xorg ~~probably because I optimised like the two most used things~~

grand shadow Aug 22, 2024, 11:05 AM

#

Well back on track now. Porting mesa took longer than Id have wanted

#

Today after I get home Ill sort the auth stuff out and start adding checks all around the kernel

robust geyser Aug 22, 2024, 1:16 PM

#

nice work, i wonder if it would scale well with smp

#

unfortunately mouse input kinda broken under wsl so i couldnt test more :(

grand shadow Aug 22, 2024, 1:36 PM

#

robust geyser nice work, i wonder if it would scale well with smp

Damn your cpu seems much better with mine

#

Good to know it was my cpu being bad all along ~~and some of my code~~ meme

#

Llvm pipe does seem to use multiple cores to do rendering if it can so I assume itd get faster

robust geyser Aug 22, 2024, 1:38 PM

#

i did try smp but i couldn't get past a black screen with the cursor

grand shadow Aug 22, 2024, 1:42 PM

#

Smp is a bit borked but tbh it couldve just been the tlb shootdown slowing it heavily

gritty shore Aug 22, 2024, 1:42 PM

#

nice job!

grand shadow Aug 22, 2024, 1:42 PM

#

I think I might just make a struct to describe a shootdown and have it flush the whole tlb of the other cpu if its larger than a few pages and its applicable

gritty shore Aug 22, 2024, 1:42 PM

#

why are some x fonts broken tho

grand shadow Aug 22, 2024, 1:42 PM

#

gritty shore why are some x fonts broken tho

Wdym

#

I might be missing some package

gritty shore Aug 22, 2024, 1:43 PM

#

grand shadow hey

idk look at the top windows

#

DateTime

grand shadow Aug 22, 2024, 1:44 PM

#

I think its just being cut out

#

These windows are fvwm.being broken (I think it wants epoll)

gritty shore Aug 22, 2024, 1:44 PM

#

epoll? doubt it

#

that's a linux API

grand shadow Aug 22, 2024, 1:45 PM

#

I have seen something want epoll in the mlibc logs

neon crane Aug 22, 2024, 3:12 PM

#

robust geyser unfortunately mouse input kinda broken under wsl so i couldnt test more :(

Use qemu for windows ffs

robust geyser Aug 22, 2024, 3:14 PM

#

hell no

obsidian pivot Aug 22, 2024, 3:17 PM

#

qemu for windows is fine

neon crane Aug 22, 2024, 3:23 PM

#

robust geyser hell no

huh

#

qemu for windows is better than qemu for linux on windows wdym?

robust geyser Aug 22, 2024, 3:36 PM

#

i hate the windows version

neon crane Aug 22, 2024, 3:50 PM

#

robust geyser i hate the windows version

perhaps all you dislike is the gtk ripoff, if so, try -display sdl

robust geyser Aug 22, 2024, 4:24 PM

#

actually gtk frontend looks quite okay now

#

the sdl frontend is missing like everything except the framebuffer lol

fleet osprey Aug 22, 2024, 4:44 PM

#

not bad

#

at least it works

#

that's not a given with Windows

neon crane Aug 22, 2024, 6:16 PM

#

robust geyser the sdl frontend is missing like everything except the framebuffer lol

i like the sdl backend actually because it's using native windows

grand shadow Aug 22, 2024, 10:32 PM

#

grand shadow Llvm pipe does seem to use multiple cores to do rendering if it can so I assume ...

an excuse for me to implement sysconf soon

#

first smp bug I fix in like months 🥳

#

one of many

#Astral