Yeahhh
#Astral
robust geyser
thanks🙏
* building package: openjdk17
didn't realize that that's upstream now, cool
grand shadow
Its been since last year
robust geyser
aaaand it failed
grand shadow
🥀
robust geyser
$ btop
ERROR: Exception in runner thread -> Resource temporarily unavailable```Crazy
robust geyser
same errors during build
Compiling 47 files for jdk.jpackage
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
x86_64-astral-g++: fatal error: cannot execute '/usr/local/lib/gcc/x86_64-astral/14.3.0/../../../../x86_64-astral/bin/as': posix_spawn: Resource temporarily unavailable
compilation terminated.
/usr/bin/bash: fork: retry: Resource temporarily unavailable
x86_64-astral-g++: fatal error: cannot execute '/usr/local/libexec/gcc/x86_64-astral/14.3.0/cc1plus': posix_spawn: Resource temporarily unavailable
compilation terminated.
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
x86_64-astral-g++: fatal error: cannot execute '/usr/local/lib/gcc/x86_64-astral/14.3.0/../../../../x86_64-astral/bin/as': posix_spawn: Resource temporarily unavailable
compilation terminated.
/usr/bin/bash: fork: retry: Resource temporarily unavailable
x86_64-astral-g++: fatal error: cannot execute '/usr/local/libexec/gcc/x86_64-astral/14.3.0/cc1plus': posix_spawn: Resource temporarily unavailable
compilation terminated.
/usr/bin/bash: fork: retry: Resource temporarily unavailable
x86_64-astral-g++: fatal error: cannot execute '/usr/local/libexec/gcc/x86_64-astral/14.3.0/cc1plus': posix_spawn: Resource temporarily unavailable
compilation terminated.
/usr/bin/bash: fork: retry: Resource temporarily unavailable
x86_64-astral-g++: fatal error: cannot execute '/usr/local/libexec/gcc/x86_64-astral/14.3.0/cc1plus': posix_spawn: Resource temporarily unavailable
compilation terminated.
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
gmake[3]: *** [lib/CompileJvm.gmk:150: /build_dir/builds/openjdk17/hotspot/variant-server/libjvm/objs/shenandoahSTWMark.o] Error 1
gmake[3]: *** Waiting for unfinished jobs....
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
gmake[3]: *** [lib/CompileJvm.gmk:150: /build_dir/builds/openjdk17/hotspot/variant-server/libjvm/objs/shenandoahStaticHeuristics.o] Error 1
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
/usr/bin/bash: fork: retry: Resource temporarily unavailable
gmake[3]: *** [lib/CompileJvm.gmk:150: /build_dir/builds/openjdk17/hotspot/variant-server/libjvm/objs/shenandoahSupport.o] Error 1
/usr/bin/bash: fork: retry: Resource temporarily unavailable
gmake[4]: /usr/bin/bash: Resource temporarily unavailable
gmake[4]: *** [lib/CompileJvm.gmk:274: /build_dir/builds/openjdk17/buildjdk/hotspot/variant-server/libjvm/objs/jvm_posix.o.op_check] Error 127
gmake[4]: *** Waiting for unfinished jobs....
gmake[3]: *** [lib/CompileJvm.gmk:150: /build_dir/builds/openjdk17/hotspot/variant-server/libjvm/objs/shenandoahTaskqueue.o] Error 1
gmake[3]: *** [lib/CompileJvm.gmk:150: /build_dir/builds/openjdk17/hotspot/variant-server/libjvm/objs/shenandoahUnload.o] Error 1
gmake[3]: *** [lib/CompileJvm.gmk:150: /build_dir/builds/openjdk17/hotspot/variant-server/libjvm/objs/shenandoahStackWatermark.o] Error 1
gmake[3]: *** [Main.gmk:252: hotspot-server-libs] Error 2
gmake[2]: *** [make/Main.gmk:541: create-buildjdk-interim-image] Error 2
gmake[2]: *** Waiting for unfinished jobs....
gmake[2]: *** [make/Main.gmk:252: hotspot-server-libs] Error 2```whoops sorry that's a lof of characters lol
grand shadow
Damn how many jobs did you run it with
robust geyser
why would i when i can just build java lol
i think it's some quota limit that i'm hitting, i wasn't running out of memory
opal lotus
idk bro
robust geyser
me neither lol im just guessing
wild marsh
is this a wsl moment?
opal lotus
you can comfortably compile llvm on all threads with that yea
opal lotus
tried rebooting yet?
robust geyser
although it did swap i think
yes
but i can just build it in 3 parts lol
that's what i did, i just run jinx build openjdk17 3 times, first two it fails and 3rd time it works 🥀
grand shadow
Weird shit
doing a final test drive of the rbtree vmm before tidying up and pushing it
so far everything seems to be working, building mlibc now
oh I have to download linux kernel headers nvm
well meson works so ninja probably does
robust geyser
make: ../genbootimg.sh: No such file or directory
am i not supposed to use make img
grand shadow
I always use make disk, make img was like added by techflash so he could run it on his computers
grand shadow
not really
I just jinx install into a sysroot and generate it from there
otherwise probably would need root to mount and shit
robust geyser
ah true
lol xfishtank is cool
i also didn't know you can have two sessions, a tty and serial one, at the same time
astral has really advanced since last time i touched it
anyway - xhci time 
grand shadow
yeah I've abstracted the whole tty shit properly so now anything that can stream bytes can be a tty
opal lotus
neat
grand shadow
the only two ports I tried which didnt really work is qemu because of a signal deadlock in my todolist and wine because I think of some magazine shit (or it could be my vmm rework, I need to look into it more tomorrow)
robust geyser
i might do the minimal initrd thing so i don't have to wait an hour just to test a small kernel change
grand shadow
if u do it I'll gladly accept it upstream
theres no point in a full intird anymore
rbtree vmm pushed
robust geyser
lol i just forked 🥀
grand shadow
I'll check the wine thing and if its not a easy bug I'll leave it for tomorrow
grand shadow
lol i just forked 🥀
🥀
robust geyser
anyway pr incoming
grand shadow
I mean the only thing it should impact is performance
robust geyser
grand shadow
lgtm, thanks!
grand shadow
I'll check the wine thing and if its not a easy bug I'll leave it for tomorrow
I'll look into this tomorrow I still need to do some stuff irl
if u run into anything that looks like a vmm bug let me know
robust geyser
decent progress, ports 5 and 6 are the keyboard and mouse
submitting an enable slot cmd and getting a command completion works too, very nice
analog berry
crazy
marsh sparrow
nice
robust geyser
crazy
how so lol, it's just a ring buffer :^)
i wrote a grand total of 700 lines of code overnight, not much tbh
now i'm trying to come up with a good abstraction over transfers
i'm not trying to abstract over hcds for now, idek how ehci/uhci works so i'm not gonna bother
analog berry
how so lol, it's just a ring buffer :^)
xhci driver in one sitting is pretty good
robust geyser
eh, it's pretty much guaranteed it won't work on most real hw anyway lol
so fixing that will probably be another 2 "sittings" :(
i also need to figure out how hubs work, i haven't done that last time around and i'm basically clueless as to how they work
but hey maybe we'll be able to use astral on more real hw now :^)
analog berry
yeah considering its ps2 literally doenst work on most
robust geyser
xhci is far less trickier to get to work
at least it's like
standardized
boot from usb is gonna be awesome too, usb msd is really easy
so no need for a 3GB initrd :^)
neon crane
is XHCI USB 3.0 or 2.0?
neon crane
oh yay
i moreso meant that if its USB 3.0 then itll work on my modern real hw
and for my older hw with only USB 2.0 id have to implement EHCI or whatever it was called
neon crane
anyway i guess you need some tiny process or thread to poll every device, right?
robust geyser
but it's hell compared to EHCI/XHCI so i wouldn't bother
neon crane
how do you decide the polling rate
robust geyser
it's all interrupt driven
neon crane
oh, i thought USB was poll driven
robust geyser
well, kinda :^)
neon crane
i guess the controller does the polling?
robust geyser
yes
neon crane
i see
robust geyser
for stuff like keyboard and mouse, you initiate a transfer and the device completes it whenever it wants pretty much
at it's polling rate
neon crane
i see
robust geyser
and you just get an interrupt whenever that happens
neon crane
in 2026 im going to have to implement some kind of usb stack
especially on the wii
or on my laptop
fleet osprey
especially on the wii
on the Wii, USB is actually incredibly cursed
depending on what your goal is with it I'd recommend doing something else first:
- Storage: implement SDHCI support for the SD slot
- Networking: implement SDHCI support for the SDIO wifi
- Input: implement EXI and a USB Gecko driver for a serial console, and/or SI for the GCN ASCII keyboard
- Other USB devices: suffer I guess
wild marsh
pretty sure UHCI is more common
ohci vs uhci i'm pretty sure depends on amd chipset vs intel chipset for the most part
like i have an athlon 64 mobo that has ohci for example
robust geyser
interesting, anyway i'll probably stick to xhci only :^)
much simpler target and i don't need to worry about hcd abstraction
wild marsh
xhci has been in use for at least 12 years now so that's probably not a bad idea
robust geyser
lol that's a good point
robust geyser
well...
usb_xhci_ep_kick slotid 1, epid 1, streamid 0
usb_xhci_fetch_trb addr 0x000000007e603000, TR_SETUP, p 0x0008000001000600, s 0x00000008, c 0x00030841
usb_xhci_fetch_trb addr 0x000000007e603010, TR_DATA, p 0xffff80007e604000, s 0x00000008, c 0x00010c05
usb_xhci_fetch_trb addr 0x000000007e603020, TR_STATUS, p 0x0000000000000000, s 0x00000000, c 0x00001021
usb_xhci_xfer_start 0x725e8c1bf3f0: slotid 1, epid 1, streamid 0
qemu-system-x86_64: ../qemu-10.1.2/hw/usb/core.c:612: usb_packet_copy: Assertion `p->actual_length + bytes <= iov->size' failed.
make: *** [Makefile:159: run-disk-kvm] Aborted (core dumped)```
flat copper
at least on modern high performance chips
DesignWare wants a word with you :^)
robust geyser
it seems i have passed a higher half physical address as buffer.
and it seems like it wasn't the issue.
marsh sparrow
DesignWare wants a word with you :^)
hm?
flat copper
apple uses dwc3, so does qualcomm iirc
robust geyser
dwc3 seems sane-ish, at least from my brief exposure when working with m1
static void control_xfer(ctrl_t *ctrl, slot_t *slot, xfer_t *xfer) {
xfer_setup_t *setup = xfer->setup;
bool has_data_stage = xfer->buffer != NULL && xfer->buffer_len > 0;
bool data_stage_in = has_data_stage && xfer->dir == XFER_TO_HOST;
uint32_t trt = 0;
if (has_data_stage) {
// 0 = no data stage, 2 = data out, 3 = data in
trt = data_stage_in ? 3 : 2;
}
trb_t setup_trb;
setup_trb.dw0 = (uint32_t)setup->type | ((uint32_t)setup->request << 8) | ((uint32_t)setup->value << 16);
setup_trb.dw1 = (uint32_t)setup->index | ((uint32_t)setup->length << 16);
setup_trb.dw2 = 8; // always 8 bytes for setup stage
setup_trb.dw3 = TRB_DW3_TYPE(TRB_TYPE_SETUP_STAGE) | TRB_DW3_IDT | (trt << 16);
trb_t data_trb;
if (xfer->buffer != NULL && xfer->buffer_len > 0) {
data_trb.dw0 = (uint32_t)(uintptr_t)xfer->buffer;
data_trb.dw1 = (uint32_t)((uintptr_t)xfer->buffer >> 32);
data_trb.dw2 = (uint32_t)xfer->buffer_len;
data_trb.dw3 = TRB_DW3_TYPE(TRB_TYPE_DATA_STAGE) | TRB_DW3_ISP | (data_stage_in ? TRB_DW3_DIR : 0);
}
trb_t status_trb;
status_trb.dw0 = status_trb.dw1 = status_trb.dw2 = 0;
status_trb.dw3 = TRB_DW3_TYPE(TRB_TYPE_STATUS_STAGE) | TRB_DW3_IOC | (data_stage_in ? 0 : TRB_DW3_DIR);
// TODO: spinlock around queue push?
queue_push(&slot->ep_queue[0], &setup_trb);
if (has_data_stage)
queue_push(&slot->ep_queue[0], &data_trb);
queue_push(&slot->ep_queue[0], &status_trb);
}```can anybody spot some obvious mistake?
i guess time to dig into qemu source code to figure out what tf im doing wrong
grand shadow
I sleep and bro cooked 🔥
robust geyser
and qemu cooked me
apparently i'm doing something dumb but i'm too tired to know what it is
wild marsh
apple uses dwc3, so does qualcomm iirc
isn't dwc3 xhci with extra shit glued on?
for phy and otg
flat copper
i thought it kept xhci data plane but with different regs but it's been a while
robust geyser
omfg im so dumb
i just set up the setup trb wrong
i mean the code is fine, the values i passed were wrong
i didn't set the to host bit in bRequestType
🥀
wild marsh
i thought it kept xhci data plane but with different regs but it's been a while
/*
* sysdev must point to a device that is known to the system firmware
* or PCI hardware. We handle these three cases here:
* 1. xhci_plat comes from firmware
* 2. xhci_plat is child of a device from firmware (dwc3-plat)
* 3. xhci_plat is grandchild of a pci device (dwc3-pci)
*/
it uses the actual xhci driver for dwc3
flat copper
ah nice
wild marsh
but dwc3 does do stuff like phy etc ```
/*
* Drivers such as dwc3 manages PHYs themself (and rely on driver name
* matching for the xhci platform device).
*/
so you still need extra drivers
marsh sparrow
apple uses dwc3, so does qualcomm iirc
dwc3 is xhci
dwc3 is an xhci controller for host mode bundled with a custom (?) interface for device mode
carmine swallow
its probably a thing on a lot of platforms even tho its optional (the one that intel mobos have has it I am pretty sure and a lot of qcom soc's for an example)
grand shadow
re this: maybe it is really just the cost of taking a page fault? I dont have the exact addresses but I'm gonna assume that is literally just, the cpu spending a bunch of cycles doing the page fault which then triggers the nmi when its done
analog berry
btw since the perf hasnt changed, is there anything sticking out in the profile that u wanna work on next?
grand shadow
well considering I pretty much went from a "70% of the flamegraph is spent on vmm range traversal" to like 20% and nothing changed then idk if its a kernel issue
analog berry
do u think its mlibc?
grand shadow
no idea without profiling or having a second way of testing it out on my hardware on a known fast kernel, as I dont think linux-mlibc has openjdk
could be overhead from running qemu in wsl, could be an actual kernel skill issue but instead of running it spends most of the time sleeping, could be mlibc
but removing all that from the flamegraph and perf being the same weirds me out
I still gotta do one final touch up on the vmm, the range caching thing
though granted a lot of time is still spent on the vmm, mostly from page table transversal when unmapping pages, page fault handling and finding free space
no idea what that rbtree call is in syscall_mmap, maybe my stack trace handling is borked and I'm seeing bad data because of that
could it be just jmping to it or whatever idk
grand shadow
well considering I pretty much went from a "70% of the flamegraph is spent on vm...
like its not even funny
viscid flame
like its not even funny
do you remove elements in your vmm on munmap ?
robust geyser
i thought the red far right flamegraph entry said sigma and i freaked out
if you tell me how to run minecraft and profile it i can try on my pc btw
idk if better cpu is gonna make any difference
grand shadow
if you tell me how to run minecraft and profile it i can try on my pc btw
shove these two into /home/astral and just run it
then once the game is open on another terminal window run astral-prof /dev/com0
u gonna need to compile with -DENABLE_PROFILING and add a serial port to qemu pointing to a file
then u can just ./prof/prof.py profiler_file svg.svg
grand shadow
do you remove elements in your vmm on munmap ?
wdym
oh yeah, after a while astral with profiling enabled will crash because of some bug in my backtrace handling (I probably forgot to shove a null rbp somewhere as a terminator) but it takes a while and the profiler output will still be perfectly valid since it only writes full stack traces out
robust geyser
will do in a few min
grand shadow
based
robust geyser
hm weird, i put the files there i rebuilt distro-files i ran make disk and the files aren't in /home/astral lol
i must have done something wrong
grand shadow
jinx needs u to run ../jinx reinstall sysroot distro-files
it used to reinstall shit automatically but it does not anymore
robust geyser
i thought that ../jinx install sysroot '*' would take care of that?
that's what make disk does
odd lol
grand shadow
same thing but apparently it doesnt install already installed packages
robust geyser
ok, let's see if it works now
i need to figure out a better way to remake a disk image😭
this is so painful
bruh
grand shadow
💀
grand shadow
bruh
robust geyser
i'll just remove the version check lmfao lets see what happens
well, apparently tsc is not fixed.
sharp carbon
does astral not have a logger spinlock or something
robust geyser
it most definitely does
grand shadow
it does I think its just that all of them paniced at the same time and something exploded
sharp carbon
this is why i said that fwiw
robust geyser
you only take the mutex in the printf macro
i think panic bypasses it
so if 4 cpus panic at the same time that happens
sharp carbon
yeah but those logs aren't from panics afaict
sharp carbon
why tf is pmc not exposed on my machine
bios option?
grand shadow
those logs dont get borked on my machine
robust geyser
oh no, i think that happens because multiple cpus write to the flanterm context
grand shadow
so I dunno
robust geyser
flanterm isn't thread safe
sharp carbon
thats what i meant yeah
robust geyser
i'll try that then
i copied all the code and compiled it on the host and i got 5
for pmc version
seems fine
sharp carbon
then maybe it's something to do with how the wsl vm is set up
grand shadow
and on tiny + fast?
robust geyser
80 fps
actually playable holy fuck
i just wish wsl mouse input in qemu wasnt so god damn broken
maybe i should move to linux so i can profile it lol
willow rapids
no such space or space accessed is kerneç 🥀
robust geyser
i still get 50 fps if i full screen it
not bad but it seems like something worth profiling
i'll push my xhci changes to another branch and move to linux
i gotta recompile astral again 😭
oh shit i kinda cheated, i used -smp 1
lemme try with smp 4
analog berry
wait isnt 80 fps actually close to linux with software render
grand shadow
not bad but it seems like something worth profiling
fml I already profiled it and removed half of the time spent on the kernel 🥀 ☠️
analog berry
wasnt the fps way lower before the rb tree patch?
grand shadow
wasnt the fps way lower before the rb tree patch?
I forgor
lemme see if I can find an old screenie
or better yet
git revert
robust geyser
yeah i get 100 fps now, crazy
lemme try bumping graphics settings a bit
honestly, more than playable at 40-50 fps with fast graphics and far render distance
grand shadow
so looking at a specific distance on tiny + fast gives me 30 fps with the rbtree patch
lemme see without the rbtree patch
grand shadow
i5-10300h
robust geyser
ok im rebooting to my arch install
ill have to update it because i hav ent touched it in a while
then ill build astral there
933 packages, total download size 3489.17 MiB, total install size 12791.00 MiB
god help me
we ball
grand shadow
💀
grand shadow
so looking at a specific distance on tiny + fast gives me 30 fps with the rbtree...
25-26 fps without
so there was an improvement but not what I would expect from that
robust geyser
it just works epic, time to build astral
marsh sparrow
well considering I pretty much went from a "70% of the flamegraph is spent on vm...
you could check whether most profiling samples are in the kernel or in userspace
grand shadow
ah thats true
marsh sparrow
but what's your general expectation?
grand shadow
I guess I could add ioctls to get the number of samples taken in kernel and samples taken in user and have the profiler add those to the start of the file
marsh sparrow
you could compare to linux with software rendering in qemu
grand shadow
you could compare to linux with software rendering in qemu
maybe
I can try to spin up a debian or whatever vm and try it out
gritty shore
wait isnt 80 fps actually close to linux with software render
nah mesa should get higher
than 80
On beta Minecraft
robust geyser
keep in mind that this game was, and kind of still is, horribly unoptimized
sharp carbon
keep in mind that this game was, and kind of still is, horribly unoptimized
on my host with that mc build, these settings, and LIBGL_ALWAYS_SOFTWARE=true i get ~380 FPS
marsh sparrow
host vs qemu benchmarks are questionable though (it's possible that there is little difference but also possible that the difference is huge)
robust geyser
i have not yet started building astral btw
the debian snapshots are so god damn fucking slow to download from
wild marsh
The oldest
old mc is using like opengl 2 or something
gritty shore
Not even 1?
wild marsh
it had support for 1.6 up to some release even iirc
i remember seeing a warning on the title screen that ogl 1.6 support would be going away
gritty shore
I say this because I'm writing an OpenGL 1.1 implementation and it would've been very cool to run MC
I'll have to do with glquake I guess
wild marsh
er no 1.6 but 1.2
sharp carbon
i think up to like 1.7 or 1.8 it could still run with opengl 1.2?
wild marsh
afaics up to release 1.7 can do with ogl 1.2
robust geyser
i finally built cross gcc 🙏
gritty shore
afaics up to release 1.7 can do with ogl 1.2
Damn
grand shadow
maybe you could implement opengl 1.2 after you implement 1.1
idk how different they are
gritty shore
Not that much
A few features
I'm not aiming for 100% compat anyway so i can add whatever I want
analog berry
host vs qemu benchmarks are questionable though (it's possible that there is lit...
why
marsh sparrow
Because it's not an apples to apples comparison if there is a VMM involved?
grand shadow
Btw iirc you implemented shmget, right?
iirc monkuous said they did not see any difference
analog berry
Because it's not an apples to apples comparison if there is a VMM involved?
vmm?
gritty shore
And Linux might schedule the VM differently
grand shadow
analog berry
And Linux might schedule the VM differently
theres no difference between a vcpu thread and a minecraft thread
sharp carbon
Btw iirc you implemented shmget, right?
when i was experimenting with things i implemented a rudimentary version of shm* that worked in this specific case but was not commit worthy in the slightest, and it had very little impact
marsh sparrow
of course there is
gritty shore
Isn't the VM at the mercy of the Linux scheduler at the end of the day? That's different from running natively
analog berry
so is java running natively
marsh sparrow
a vcpu thread is all guest OS threads batched into one
analog berry
same scheduler
analog berry
a vcpu thread is all guest OS threads batched into one
wdym, each vcpu gets a thread
gritty shore
so is java running natively
No because you have to schedule the kernel too
Which schedules java
analog berry
wdym, the kernel doesnt run when minecraft runs
only during pagefaults etc, same as native java
marsh sparrow
wdym, each vcpu gets a thread
... which is obviously a difference to each java thread getting a (host) thread?
analog berry
if you pin vcpu threads theres 0 difference
marsh sparrow
with the VMM, the host sees # vcpus many threads and without the VMM, the host sees # native threads many threads?
gritty shore
wdym, the kernel doesnt run when minecraft runs
Well yeah but there's still preemptive scheduling
marsh sparrow
if you pin vcpu threads theres 0 difference
bullshit
I did perf engineering for several years, the difference can be HUGE depending on the workload
analog berry
what type of workload produces this huge difference you're talking about
and how similar is it to minecraft
obviously some things you can do in a vm will be slower, but i cant imagine how minecraft is one of them
marsh sparrow
22,585.19 msec task-clock:G # 2.230 CPUs utilized
80,331 context-switches:G # 3.557 K/sec
3,944 cpu-migrations:G # 174.628 /sec
118,808 page-faults:G # 5.260 K/sec
37,180,238,767 cycles:G # 1.646 GHz
28,047,893,694 instructions:G # 0.75 insn per cycle
4,283,893,446 branches:G # 189.677 M/sec
131,953,788 branch-misses:G # 3.08% of all branches
10.128395164 seconds time elapsed
13.695815000 seconds user
8.984424000 seconds sys
its just basic compute
marsh sparrow
^ see how KVM uses 9 out of 22 seconds in host mode?
analog berry
whats the workload tho
analog berry
booting obviously will produce more vmexits than runtime
also this doesnt really mean much
depends on the reasons it went to host mode
that can be traced on linux easily btw
marsh sparrow
It's just a bad comparison
the proper comparison is KVM to KVM and not KVM to host
unless you want to benchmark KVM performance itself
There is no excuse not to do it, it's very easy to do
analog berry
anyway in a workload like running minecraft u cant expect more than maybe 10-15% of a slowdown compared to host
grand shadow
I cant for the life of me to get prism launcher to run minecraft under debian so astral better than debian question mark
analog berry
assuming u have nested paging on etc
marsh sparrow
there are many variables here
it's very hard to tell w/o doing a proper comparison
analog berry
but mathews cpu is kinda old so it might be lacking a lot of those things that make vms fast
gritty shore
I cant for the life of me to get prism launcher to run minecraft under debian so...
Try an arch VM or something
10th gen isn't that old no?
marsh sparrow
it isn't
gritty shore
Probably has avx512!
gritty shore
ah yeah and avx is newer too
robust geyser
wasn't avx512 briefly introduced in 12th gen cpus?
gritty shore
avx512 was on 10th gen mobile chips apparently
marsh sparrow
there is actually lots of opportunity for inefficiency with the MC workload
even just the non-zero-copy to host vmem
analog berry
which ones are u thinking
marsh sparrow
compared to zero-copy on the host
gritty shore
wasn't avx512 briefly introduced in 12th gen cpus?
Looks like it's 11th gen
analog berry
the qemu framebuffer is zero copy since its just a normal window
gritty shore
Zen 4 for AMD
marsh sparrow
the qemu framebuffer is zero copy since its just a normal window
it is not
there is at least one more copy involved in the VM
analog berry
does it copy it?
gritty shore
I need a new cpu just for avx512 
analog berry
why tho
analog berry
I need a new cpu just for avx512 <:trl:829316641191886888>
get a 9950x3d
gritty shore
I am on zen 3 so that would involve a new motherboard and ram meaning I'd have to sell my liver
analog berry
ram would probably cost more than the cpu
gritty shore
Unironically
robust geyser
me when ram costs as much as the rest of the fucking pc
gritty shore
If I get a new cpu it's gonna be something like 7600X, I'm too broke for fancier CPUs (more like, I have enough money but don't wanna spend that much on a CPU)
analog berry
tbh that might have avx512
gritty shore
me when ram costs as much as the rest of the fucking pc
Unironically a 32gb stick is more expensive than my entire computer
analog berry
a lot of amds do
grand shadow
I am on zen 3 so that would involve a new motherboard and ram meaning I'd have t...
sell half of it, grow a new one, sell it
ezzzzzz
gritty shore
tbh that might have avx512
Yes they do
zen 4 has avx512
Which is all the 7000s and upwards
analog berry
common amd w
gritty shore
But ryzen 5 only has 12 threads
If I upgrade I wanna upgrade for real (I have a 5600x atm)
robust geyser
@grand shadow can you block in dpcs? or is that something that shoudn't be done
analog berry
iirc dpc is like same requirements like irqs so no
grand shadow
nope do not block in a dpc
robust geyser
that's what i thought, sad
i'm trying to figure out how i can build a decent async something for xhci, there is a lot of places where writing code in a blocking manner would be so much nicer
like data transfers, if ONLY i could block until a response arrives..
but it sucks because they are initiated from the dpc ugh
i don't think that's something i'll be able to solve even after some sleep
maybe i should just make it work with callbacks
flat copper
Looks like it's 11th gen
i have one if you're interested
flat copper
There is no excuse not to do it, it's very easy to do
it's probably fairer to compare real hw astral with linux with gpu drivers disabled instead of kvm
marsh sparrow
that's also a fair comparison
native vs native is fine
KVM vs KVM is also fine
but KVM vs native is stupid
flat copper
linux on kvm is probably way more tuned to avoid vmexits etc
analog berry
it's probably fairer to compare real hw astral with linux with gpu drivers disab...
that might be hard because ps2 doesnt work
analog berry
ssh minecraft
grand shadow
that might be hard because ps2 doesnt work <:trl:829316641191886888>
it works on my laptop xd
but also I only know the performance under wsl
flat copper
ssh minecraft
oh thats a pretty good idea
grand shadow
so thats still not good
analog berry
maybe actually try a real hw minecraft test then
grand shadow
dont have a usb stick with me
analog berry
damn
robust geyser
grand shadow
ssh minecraft
unironically would work if I had a real hardware NIC
analog berry
yeah i know astral net stack actually works well right
robust geyser
wait till i implement usb so you can just do usb networking 
grand shadow
so true
using my phone for wifi go brrr
that came in clutch when installing linux before
grand shadow
[this](<https://github.com/48cf/Astral/blob/01466451d689e7188b3e6e4e28e36afd9ea5...
wish I could help but I have exactly zero xhci knowledge
robust geyser
i guess i could technically just do something like
submit and wait
just for the init sequence probably
eh i still dont see a good way to do that, whatever i'll figure it out eventually
i'll wait for astral to build and go to sleep i havent slept in like 20h now
then after i wakje up we ball with xhci
marsh sparrow
linux on kvm is probably way more tuned to avoid vmexits etc
that's a good opportunity to tune astral though ;D
you can get kvm exit counts using perf
tuning to avoid vmexits actually made a huge difference for Managarm overall performance
grand shadow
then after i wakje up we ball with xhci
hell yeah
robust geyser
if i get bored of xhci ill try porting some de to astral
like xfce or something
i havent done that before, ever, so it should be fun
or implement drm and run weston 
marvin got weston to compile without linux option so maybe it could work, who knows
grand shadow
like xfce or something
damn astral running a real de
fvwm has been ok but I barely use any of its customization so a real de might be better atp
I've always thought of porting cde or nscde or w/e but never really did it
update on debian minecraft: not even running it in the same way as astral it works
veredict: astral > debian
robust geyser
lmao wtf
now hear me out, what if - mate
ironclad/gloire had mate before the switch to glibc
soooo.. yoink and make it work on astral?
anyway astral build is finished, enough yapping im going sleep cya
grand shadow
now hear me out, what if - mate
Hmmm
grand shadow
anyway astral build is finished, enough yapping im going sleep cya
Cya gnn
robust geyser
hm i got this when running minecraft
oh💀
nvm
ok, on native linux i get 80-90fps in full screen with tiny render distance fast graphics
gritty shore
What do you get on astral?
gritty shore
ahh lol
robust geyser
previously i ran it under kvm wsl
gritty shore
It seemed low for native linux mesa
grand shadow
hm i got this when running minecraft
Funnily enough this error happens on both managarm and astral when closing the jvm
But yeah it only happens when closing so its not a big issue
grand shadow
update on debian minecraft: not even running it in the same way as astral it wor...
though one thing I noticed is that glxgears perf is like 60% of what it is on wsl on the debian vm
grand shadow
maybe it was a livecd weirdness
grand shadow
hmm today I might try to debug that long standing signal bug from qemu and try to see whats up with that wine crash
prime mulch
@grand shadow another source of vm inefficiency could be the fact you're making only 1 page at a time valid rather than performing clustering, which could cause like 4-32x as many page faults as necessary
idk why I gave a specific range I should have just said a fuckton
grand shadow
yeah I've thought of that as well
I need to see how I would approach doing that properly
which speaking of
how did you do it in mintia or how are you gonna do it in mintia2
also lol I dont think allocating 51 gb for ancillary data is right
weird that my alloc didnt trip an assert from not having any slab cache meeting that though
grand shadow
how did you do it in mintia or how are you gonna do it in mintia2
@prime mulch forgot to ping u
prime mulch
in old mintia I clustered the page cache read-ins to try to read aligned regions of iirc 16 pages at a time from the file
I did not cluster the actual creation of valid PTEs though so I let those be soft faults still
in mintia2 I will and ill just design the page fault path to try to make valid a range of pages rather than just one
and that will allow me to reduce overhead from page fault exceptions and also lookups of mappings bc ill only need to look it up once for a whole range of ptes
and so on
Probably I'd have the backing storage for the mapping (like a file page cache or a pagefile-backed anonymous memory object) always try to perform larger aligned reads
And inform the upper layer how many pages it managed to bring in and then it'll try to make all of those PTEs valid in one loop
Or ill initialize an array of null pointers on the stack and pass a pointer to it to the lower layer and say "pls fill this up with pages in this range, but only initiate a synchronous IO if this particular one (the one the process faulted on) is not resident, fill any other non resident page with null pointers" and then iterate over that array and make valid the corresponding PTEs for each page that it gave you
That'd be a rly efficient implementation probably because it'll let it do a single search of its own structures in whatever way is most efficient to try to fill up that array
grand shadow
also lol I dont think allocating 51 gb for ancillary data is right
apparently there was an unmarked (in the commit) abi break?
not sure if this would count
ah it wouldnt
hm
oor wait would it
hmm yeah there could be some issues arising from that commit
#if __INTPTR_WIDTH__ == 64 && __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
int __pad1;
#endif
socklen_t msg_controllen;
#if __INTPTR_WIDTH__ == 64 && __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
int __pad1;
#endif```in astral that msg_controllen is still a size_t
but in the linux abi its a int
so the upper bits of the thing are just whatever garbage was on the stack
fun stuff
marsh sparrow
which commit is that?
grand shadow
it switches the msg_controllen from size_t to socklen_t (which is an int) and then adds padding to make the whole length stay the same
grand shadow
in astral its still a size_t
marsh sparrow
assuming that the upper bits are zero anyway
grand shadow
and that padding will use whatever is on the stack
I will just switch the socket definitions in astral to match the changes in that commit
marsh sparrow
i'm confused because it looks like the overall struct size + offset of all members stayed the same?
grand shadow
yeah but like
the actual length is now 32 bits
but astral still treats it as 64 bit
even though the 32 bit has padding the upper 32 bits of the number astral gets are still garbage
marsh sparrow
why does that happen on astral but not on linux?
grand shadow
probably because linux truncates it to 32 bits or something?
is it a size_t in linux?
marsh sparrow
CC @formal patio
grand shadow
bruh
could have at least updated it on the other sysdeps which subscribe to the linux abi?
doesnt matter anymore, I am updating it on astral
fixing that wine doesnt crash on that anymore
formal patio
oops
for this I'm wondering a bit where this would be appropriate to fix though
like
sysdep vs syscall handler
for linux we don't really have a choice, hence the sysdep memes
for managarm it didn't matter because IPC
for the rest, unsure
but yeah I fucked up by not checking
grand shadow
probably just by labelling that commit as an abi break would have been enough
since then people can decide by themselves how to fix
grand shadow
I didnt really notice it because I didnt think it was an abi break
marsh sparrow
we should be more conservative with changes like these in the future and treat them as an abi break
formal patio
my line of thought was, the range of values this can take is not being changed, and that you can't rely on a struct to have/not have padding anyway (so memsetting/other way of init would be necessary anyway)
obviously (in hindsight) wrong ^
marsh sparrow
Yeah, because that doesn't take into account that the values are not initialized differently
since there is no default to zero init
formal patio
ig it depends on how the kernel interprets the struct
if it just uses the mlibc struct definition, there should not be a problem (?)
if you have diverging definitions though, shit obv breaks
iirc astral doesn't use mlibc ABI definition mode for the kernel (?)
grand shadow
yeah I dont
grand shadow
ok wine is not getting stuck its just
trying to unmap a really really really big range
because something probably underflowed
grand shadow
update: use after free, trying to figure out how a pointer to it is still being kept
robust geyser
does astral run webkitgtk?
grand shadow
does astral run webkitgtk?
never tried
robust geyser
i see
grand shadow
there arent even recipes for it in astral
grand shadow
if u want to do it go ahead :P
robust geyser
xhci will forever remain in a random branch on a random fork
jk
ill finish it soon hopefully
grand shadow
xhci will forever remain in a random branch on a random fork
hda moment
qwinci's hda fork has been sitting idle for like a year💀
more even
grand shadow
ok I think I know whats happening
basically I take the successor of something before inserting a new regions in my changemap but inserting that range might destroy the successor
since coalescing
so uaf ensues
robust geyser
what if i just use dpcs without blocking.. i guess when i submit something to the controller i could associate a dpc, dpc func + dpc arg with that request and when controller completes that request it can just queue that dpc
grand shadow
Sounds good I think
robust geyser
i made a small change to the dpc system, you now enqueue a dpc without providing fn and arg
you initialize it once
i think that was one thing i found odd when working with it, and i'm pretty sure it'll come in handy when working on usb
especially if i'll have lots of dpcs lol
grand shadow
If u need to change the arg do u just reinit it?
I forgot if theres anywhere I pass a variable argument
robust geyser
oh true, i didn't think about that
goat
arguments are used, yes
nvme dpcs take the queue pair pointer
everything basically passes "itself" as an argument
grand shadow
If you change it to an init sets function then you pass an argument when enqueueing it sounds good to me
Wouldve honestly caught a silly bug I once had
robust geyser
yeah that's what i'm gonna do now
grand shadow
Wouldve honestly caught a silly bug I once had
I was accidentally reusing the same dpc object for the tx and rx on virtio net
robust geyser
ah yes i saw that, almost made the same mistake when migrating it to init+enqueue lol
robust geyser
dpc driven 🔥 although i don't really like the way the code is structured, but it's much better than checking what command was completed and doing something based on that lol
i basically made it so each time you push to a queue you can associate a "submission" with the trb, when a completion comes in it checks if there's a submission for the trb that was completed and if there is it just queues the dpc, pretty simple
robust geyser
here's the device descriptor for the qemu usb keyboard
i guess next step is usb stack so i don't have to write the usb keyboard driver inside the xhci driver
marsh sparrow
xhci will forever remain in a random branch on a random fork
you should finish it such that we have something to compare the Managarm driver against ;D
robust geyser
ugh i need to like rewrite half of the driver
i realized having everything work using dpcs is not feasible
especially across subsystems (xhci driver and the usb stack)
so now i'll create a thread per usb controller that takes care of everything, that way i can block (!!!) while waiting for requests to complete
this should make the code MUCH cleaner and i won't have to allocate ten billion buffers to hold the descriptors, i'll just be able to request them as needed
oh, and i'll take hubs into account this time around lol
analog berry
how does linux handle this?
robust geyser
no idea, i'm not brave enough to dive into linux code lol
i can barely wrap my head around how i want it to work, i am kind of scared what will happen when i lay eyes on linux's usb stack
analog berry
its not that scary lmfao
robust geyser
easy for u to say 💀
i'll give it a try i'm too tired to think anyway
so far i have not yet found where it enumerates ports, probably a skill issue on my side
analog berry
its probably in the generic usb subsystem rather than per driver
robust geyser
seems like linux uses work_struct
or workqueue, i guess, idk what the right terminology here is
robust geyser
but it seems like i'll just spin up a thread and call it a day
analog berry
its the same as a thread
robust geyser
ah, lol
analog berry
just with additional helpers
maybe astral could benefit from such an abstraction on top as well
marsh sparrow
__builtin_coro_suspend :^)
robust geyser
maybe, for now a thread and a semaphore will have to work :D
time to use C++ coroutines in C
robust geyser
is ultra gonna have a similar abstraction?
marsh sparrow
time to use C++ coroutines in C
it does work if you use the builtins
analog berry
is ultra gonna have a similar abstraction?
yeah ofc
im gonna neec workqueues for uacpi work etc
wild marsh
so far i have not yet found where it enumerates ports, probably a skill issue on...
i'm pretty sure the enumeration is done in generic code, linux pretends the root hubs are regular hubs by faking usb transactions with them and returning descriptors and the port status update bits on the hub endpoint
robust geyser
that sounds cursed
robust geyser
holy shit astral doesn't even have a linked list helper or anything 😭
analog berry
lol
robust geyser
everything in the kernel just maintains prev and next pointers on its own
analog berry
but youtube video said linked list slow
robust geyser
i wonder how many more things astral does that could have been nicely stashed in a macro so the rest of the kernel can benefit from doing that
refcount also comes to mind
ive seen a bunch of int refcount in the code
analog berry
based negative refcount
gritty shore
so now i'll create a thread per usb controller that takes care of everything, th...
Make it async
robust geyser
well, idk what async would mean in terms of the astral kernel honestly
ok i was lying im lwk retarded and the only async i know is rust and js async, what would async even mean in the context of like regular programming?
grand shadow
holy shit astral doesn't even have a linked list helper or anything 😭
U have to keep in mind I started working on it like years ago when I was completely clueless about that stuff ☠️
robust geyser
that is fair, but don't you ever look at the code and be like "hmm i really don't like how this thing was made, let me fix that"
grand shadow
I would have to spend like a day fixing every occurence
robust geyser
i'm sure you work with these parts of the code regularly
grand shadow
And Im lazy
grand shadow
i'm sure you work with these parts of the code regularly
Not really tbh
robust geyser
hm, interesting
well if i get a chance then maybe i'll refactor some things
like for example the linked lists, it would be nice to have
even if maintaining 2 pointers isn't much hassle
unless you don't want that of course, i'm not going to try and change things you don't want changed
formal patio
And Im lazy
if you're lazy I don't want to know what effort looks to you
gritty shore
ok i was lying im lwk retarded and the only async i know is rust and js async, w...
Not waiting on it
winged basin
so now i'll create a thread per usb controller that takes care of everything, th...
I also ended up finding this the most sensible way to do certain parts of a usb stack
It's just difficult to unroll a lot of the port management stuff into a continuation based approach
prime mulch
yeah we dont need to be RSX-11/M we live in the future where there are kernel threads
i have moved away from DPCs as being a primary workhorse and they are instead meant to solely be a way to drop down to scheduler IPL in order to kick awake the real workhorse (kernel worker thread pool)
at least, as drivers use them
ive considered abstracting this actually in a way that will completely hide the existence of DPCs from drivers and make them an implementation detail of dispatching work to worker threads from ISR context
grand shadow
anyways that bug is fixed and wine works now
one day I should start working on getting wine to move forward and running cooler shit
right now it crashes if you open the "save as" in notepad
:PPP
grand shadow
anyway, debugging a funny signal meme now
how valid would it be to, instead of having the spinlock value be 1, for it to be the return address of whatever function called spinlock_acquire?
to make figuring out a deadlock easier
I dont think it would have a big perf impact
and even if it does it could be behind a ifdef
since the signal bug that hits both wine and qemu randomly is a funny deadlock in the signal code
prime mulch
how valid would it be to, instead of having the spinlock value be 1, for it to b...
Why would that not be valid
grand shadow
idk was just wondering if there was some big issue I didnt know
worthy mica
anyways that bug is fixed and wine works now
Wow! Congrats.
neon crane
how valid would it be to, instead of having the spinlock value be 1, for it to b...
you'd want to do a compare exchange for that
like if its zero, exchange to the RA
if you've previously used byte sized spinlocks then you might run into some problems if things assume the size of the spinlock is 1 byte
but i dont think 1 byte writes can be atomic anyway?
sharp carbon
they can
analog berry
but i dont think 1 byte writes can be atomic anyway?
any aligned write is atomic
on x86 at least
prime mulch
but i dont think 1 byte writes can be atomic anyway?
they only arent on dec alpha
pseudo burrow
i have moved away from DPCs as being a primary workhorse and they are instead me...
Sorry for not reading earlier messages in case the answer was somewhere above. I'd like to ask if the reason is insufficient discipline/driver bugs leading to unacceptable latency in practice (too long DPC routines)?
grand shadow
ok its just me being stupid
I take a page fault with a spinlock held
and when returing return from that page fault it holds that spinlock again
wait no nvm
that code path should never be run when exiting to kernel
BUT
the thread could still sleep holding that spinlock there
which is bad bad bad
and when another thread tries to acquire that
insanity ensues
blegh insanity fixed
cc @neon crane nanoshell in astral
no idea why the failure though
it does workish
willow rapids
blegh insanity fixed
now run astral inside astral
grand shadow
hey that worked
grand shadow
now run astral inside astral
hmmm sure
loading the initrd might take a small eternity lol
gleaming bay
hey that worked
sick
grand shadow
loading the initrd might take a small eternity lol
it deadlocked somewhere and its a mutex deadlock, annoying to say the least
willow rapids
:(
grand shadow
it got to limine and loaded the kernel but deadlocked when loading the initrd
willow rapids
dang
grand shadow
yeah bro shits fucked idk where to even begin on this one
huh
it never preempts or yields
and is looping userspace code forever
that explains a lot
a breakpoint on dopreempt() is never called neither is one in the beggining of sched_yield()
the preempt dpc is never being called
grand shadow
[
yeah at one point the timer is stopped and never resumed
timer code bug lets go
grand shadow
oooooopsie small overlook in a very specific case
when removing a timer entry there could be a case where it already fired
and I do handle that
but I stop the timer before checking anything
and in the already fired case, I forget to start it back up again
grand shadow
it is
I guess qemu does a system call that removes a timer entry often enough for this specific case to be hit
robust geyser
interesting
so now you can boot astral in astral? :^)
i should implement kvm in astral lol, might be nice to do so i know what to do when i return to kvm in managarm
grand shadow
I think so, it boots into non initrd (forgot to pick the initrd option woops)
grand shadow
i should implement kvm in astral lol, might be nice to do so i know what to do w...
would be cool
qemu is still acting a bit weird though
like, getting faster when I move the mouse
grand shadow
so now you can boot astral in astral? :^)
yes you can
just took a stupidly long time to load the initrd
really need to get that page cache rewrite going some day
robust geyser
holy based
grand shadow
think u have everything to develop the kernel inside itself now
write code in vim/nano, build the kernel, make limine iso
wait fuuuck
I need to port xorriso
apart from xorriso
make limine iso, test it in qemu, git commit git push
gg ez
Productive day, long standing bugs fixed and a better debuggable spinlock implementation
tulip zinc
get go on astral so that there's micro and it'll finally be finished 🙏
pastel cove
astral in astral in astral in astral in astral in astral in astral in astral in astral in astral in astral in astral in astral in astral in astral in astralin astral in astral in astral in astral in astralin astral in astral in astral in astral in astralin astral in astral in astral in astral in astralin astral in astral in astral in astral in astralin astral in astral in astral in astral in astralin astral in astral in astral in astral in astral when?
scenic epoch
run astral on real hardware, and run linux in qemu
robust geyser
i tried porting xorriso but it seems like libburn wants statvfs to create an iso file ;(
also, in the process of debugging what syscall was failing (i could have just enabled e9 output and seen the mlibc logs ) i created a little tool that lets you trace syscalls of a specific program, like systrace ls -la /, so that is probably something i'll upstream
neon crane
hey that worked
holy shit!!
grand shadow
also, in the process of debugging what syscall was failing (i could have just en...
Like does it set a flag that enables the syscall logger for one program or does it send it out to stderr
robust geyser
it enables a flag per process, i didn't really feel like implementing a syscall tracer in userspace :^)
but ptrace would also be cool
so you can use gdb or something
worthy mica
run astral on real hardware, and run linux in qemu
Fire idea.
grand shadow
it enables a flag per process, i didn't really feel like implementing a syscall ...
Sounds nice, better than having the entire system logging it
grand shadow
but ptrace would also be cool
Yeah
robust geyser
it enables a flag per process, i didn't really feel like implementing a syscall ...
i also added another flag which determines whether children should inherit it
so you can do systrace -ff to trace the program and anything else it spawns or systrace to trace just the program
grand shadow
Hmmm wonder what I should work on today
worthy mica
Hmmm wonder what I should work on today
Port webkitgtk?
grand shadow
that probably needs some procfs memes and I dont have a procfs
I should probably update mlibc
and then maybe implement statvfs, since xorriso wants that
opal lotus
i tried porting xorriso but it seems like libburn wants statvfs to create an iso...
what's the issue with that?
opal lotus
@formal patio
grand shadow
xorg doesnt even get to the clearing the screen part lmfao
shits fucked
was there an abi break? there shouldnt be one
opal lotus
there was iirc
grand shadow
this is what xterm does
ah wait it could be not showing the %s
because of that big warning I did not read
but still
xorg is not working and thats scary
robust geyser
what's the issue with that?
not an issue, it just hard depends on that
the function will call statfs and if it returns an error so will the function
opal lotus
ah wait so statfs is implemented using statvfs in mlibc?
robust geyser
it uses it to figure out how much free disk space there is lol
i think so
or maybe it calls statvfs?
idk lol
opal lotus
statvfs is the saner thing since it's posix
grand shadow
☠️
spookier error message
also bruh just noticed I have been running with O0 this whole time
opal lotus
inb4 breaks like Linux
grand shadow
well the new spinlock thing did break lmfao
it inlined into a special function that runs on the per cpu scheduler stack which caused it to fault
ah well doesnt matter I have reverted it
hot
non-page-aligned length
I guess that should be valid
welp that was just my stupidity
I had a check for a non page aligned size but it should have been for a non page aligned address
no idea why it hasnt tripped until now tho
🥀
neofetch doesnt work
maybe I should implement error handling for those wtranscore errors atp
yeah implementing that for mbrlen fixes it
time to see if other ports are working
grand shadow
wine does not seem to work anymor
e
everything else seems to work besidse wine
well nvm it does?
idk what that was
formal patio
maybe I should implement error handling for those wtranscore errors atp
I will gladly review such PRs
grand shadow
I will gladly review such PRs <:meme:575445094589661215>
implementing that rn, was simpler than I thought
just a bit of busy work
grand shadow
I will gladly review such PRs <:meme:575445094589661215>
do u want tests btw
formal patio
would be nice
grand shadow
45/327 mlibc:ansi / sscanf FAIL 0.05s exit status 1
46/327 mlibc:host-libc+ansi / sscanf FAIL 0.05s exit status 1
58/327 mlibc:ansi / sprintf FAIL 0.12s killed by signal 6 SIGABRT
97/327 mlibc:host-libc+ansi / tz FAIL 0.24s exit status 1
100/327 mlibc:host-libc+ansi / sprintf FAIL 0.47s killed by signal 6 SIGABRT
102/327 mlibc:ansi / tz FAIL 0.30s exit status 1
112/327 mlibc:ansi / strftime FAIL 0.46s killed by signal 6 SIGABRT
132/327 mlibc:host-libc+ansi / strftime FAIL 0.78s killed by signal 6 SIGABRT
151/327 mlibc:host-libc+ansi / mbstowcs FAIL 0.80s killed by signal 6 SIGABRT
152/327 mlibc:ansi / locale FAIL 0.87s killed by signal 6 SIGABRT
159/327 mlibc:host-libc+ansi / locale FAIL 1.20s killed by signal 6 SIGABRT
172/327 mlibc:posix / getaddrinfo FAIL 0.63s killed by signal 6 SIGABRT
211/327 mlibc:host-libc+posix / getaddrinfo FAIL 0.91s killed by signal 6 SIGABRT
212/327 mlibc:posix / getnameinfo FAIL 0.84s killed by signal 6 SIGABRT
223/327 mlibc:host-libc+posix / getnameinfo FAIL 0.92s killed by signal 6 SIGABRT
264/327 mlibc:posix / locale FAIL 0.31s killed by signal 6 SIGABRT
266/327 mlibc:host-libc+posix / locale FAIL 0.33s killed by signal 6 SIGABRT
272/327 mlibc:posix / strfmon FAIL 0.08s killed by signal 6 SIGABRT
300/327 mlibc:host-libc+posix / strfmon FAIL 0.39s killed by signal 6 SIGABRT
302/327 mlibc:glibc / error FAIL 0.21s exit status 1
311/327 mlibc:glibc / error_at_line FAIL 0.24s exit status 1
fuck me thats spooky
with master mlibc and no changes btw
setlocale(de_DE.utf8) failed!
how do I get that
robust geyser
here's the patch for selective syscall logging if you want it ;p
includes an mlibc patch
oh shit i commited binary files 💀
that's better
the other 3 patches from the patch set are recipes for libisofs, libburn and libisoburn but statvfs not implemented so they aren't of much use rn
@opal lotus if you're curious as to how the statvfs usage looks like: https://github.com/svn2github/libburn/blob/00ed7dce4f6b3b8cc27fcb9cd7c3a3bf98b6272f/libburn/sg-dummy.c#L252-L322
carmine swallow
does linux have statvfs?
pretty sure it doesn't, there isn't any reason to have it anyway as it can be implemented with statfs
night trellis
everything else seems to work besidse wine
if wine works on astral im gonna run vocaloid on it
miku on astral 🔥
robust geyser
pretty sure it doesn't, there isn't any reason to have it anyway as it can be im...
yeah that's what glibc does internally, just calls statfs
the structs are even the same
opal lotus
if wine works on astral im gonna run vocaloid on it
no audio stack 
marsh sparrow
the structs are even the same
the structs are slightly different
I implemented this recently on Managarm
iirc each struct has one extra member compared to the other one
(and thus also one member missing from the other one)
sharp carbon
both have one field that the other doesn't have: statfs has f_type and statvfs has f_favail
winged basin
freebsd's contemptuous manual page on statvfs: https://man.freebsd.org/cgi/man.cgi?statvfs
The statvfs() and fstatvfs() functions fill the structure pointed to by buf with garbage. This garbage will occasionally bear resemblance to file system statistics, but portable applications must not depend on this.
night trellis
no audio stack <:KEKW:913708372329644042>
as long as vocaloid works
im planning to port alsa to astral anyways smh
marsh sparrow
> The statvfs() and fstatvfs() functions fill the structure pointed to by buf w...
That's a bit overly pessimistic though, isn't it? Sometimes you want to be able to display statistics even if they are not 100% accurate and abstract over the underlying fs
winged basin
That's a bit overly pessimistic though, isn't it? Sometimes you want to be able ...
the manual page for statfs is much less negative
carmine swallow
im planning to port alsa to astral anyways smh
good luck with that, the driver side of alsa is a huge pain lol (and I don't mean the interface but implementing the parameter refining stuff correctly so that the userspace alsa-lib doesn't complain)
gritty shore
That person has said like 5 times they would port something different and didn't do anything so don't really expect concrete ideas and work
carmine swallow
winged basin
consider OSS instead
grand shadow
☝️
viscid flame
good luck with that, the driver side of alsa is a huge pain lol (and I don't mea...
does managarm have implemented alsa
winged basin
good luck with that, the driver side of alsa is a huge pain lol (and I don't mea...
i can't believe they put that in the kernel
carmine swallow
yeah I think its just overcomplicated for no good reason, I don't think any of the other API's there are (like oss and the windows ones) do anything like this
just a simple api where the userspace gives the parameters it wants and then the driver adjusts them as needed + reports them back should be enough for basically all real-world cases
viscid flame
just a simple api where the userspace gives the parameters it wants and then the...
isn't oss +- similar to this
carmine swallow
does managarm have implemented alsa
no, it worked enough to be able to play something using aplay at one point (by just not editing the parameters the userspace gave at all) but that got broken at some point and its not the way its supposed to be implemented anyway
yes
there is even an oss plugin for pipewire (though it's specifically made for freebsd so it'd need some modifications and idk how good it is in general)
grand shadow
I remember ur work getting oss and hda on astral
U went as far as being able to play quake with sound
marsh sparrow
It'd also be great to get the Managarm side merged but I guess that requires fixing the annoying parameter negotiation, right?
carmine swallow
yes
marsh sparrow
is it possible to upstream the driver integration while fixing that later?
carmine swallow
I guess, though it might need to be changed when it can be properly tested (and new stuff added)
marsh sparrow
that's fine imo
night trellis
That person has said like 5 times they would port something different and didn't...
i wanted to like port Qt and ALSA first
and then help with WINE support
and then Wayland
just for the shits and giggles
lmao
not to really make astral more of a desktop i just thought it would be funny
tho they're not really concrete ideas since this is just more for satirical purposes
i am working on it though! trying to port one of Qt's dependencies
grand shadow
I said I would implement statvfs today but I went on a whole ass side quest to solve the wtranscode shit in mlibc and became a mlibc maintainer along the way
carmine swallow
meanwhile I tried to make astral use hzlibc and now it kinda works (at least as far as some of the very basic stuff is concerned, idk about xorg + mesa if they even build lol)
and idk what is that weird shell line caused by (or the other incorrect lines for that matter)
but at least it doesn't segfault in init anymore like it did for the past couple hours because astral didn't supply AT_BASE auxv
marsh sparrow
what's hzlibc?
carmine swallow
my libc experiment that aims for one-way glibc binary compat (as in binaries linked against glibc should work on it, the other way around isn't something I aimed for)
its good enough that you can run factorio on linux using the proprietary nvidia driver (and mesa amdgpu, at least it used to work when I last tried I don't have an amd gpu in my pc anymore)
grand shadow
meanwhile I tried to make astral use hzlibc and now it kinda works (at least as ...
yooooo real
grand shadow
that's better
weirdly this didnt build on my end (missing a stdint.h include) now its merged, thanks! :)
robust geyser
@grand shadow am i dumb or does the identify function in nvme.c leak memory?
you allocate a page and put it in pair.sub.datapointer[0], then you queue it, wait, copy it out and don't free it
grand shadow
lemme see
huh youre right
good catch
btw what are you cooking that you were looking at nvme code
robust geyser
checking if you do anything cool like per cpu io queues
robust geyser
yeah i saw, i was just curious if i could steal some code for my xhci stuff
im still stuck on how i should design the fucking usb stack
if only i had this part thought out in my head as the rest
viscid flame
im still stuck on how i should design the fucking usb stack
on which part you stuck
viscid flame
the apis and how stuff should be abstracted
do you want to support ehci too
robust geyser
i dont, but idk about @ mathewnd
grand shadow
xhci is like, 15 years old at this point I dont really feel the need of supporting early x86-64
robust geyser
thats what i thought, but i still would like to properly layer things
grand shadow
yeah
robust geyser
so you don't end up with a monolithic subsystem that handles everything
i want the driver to implement an interface that the usb subsystem uses to manage usb devices
and then that will talk to input, block, network subsystems
robust geyser
functions with broken printf usages:
arch_profiling_init, arch_ps2_init, ext2_mount, initrd_unpack, _panic (lol), vmm_pagefault 💀, init_run_routine, syscall_mount, syscall_pipe2, syscall_recvmsg, syscall_renameat, syscall_sendmsg, syscall_waitpid
stone oracle
xhci is like, 15 years old at this point I dont really feel the need of supporti...
Heh, some modern arm64 tablets with attachable keyboards, for example Pinetab2, use ... OHCI for the keyboard.
formal patio
interesting info for a x86_64-only OS
stone oracle
Maybe he will want to extend targeted architecture park?
viscid flame
i dont, but idk about @ mathewnd
you can implement it later
ig for now you need only keyboard and mouse which are ~50-70 loc (without hid to ps/2 scancode convert)
carmine swallow
ig for now you need only keyboard and mouse which are ~50-70 loc (without hid to...
doubt you can make a proper (or even not a fully proper) hid driver in ~50-70 loc lol
my (non-complete) hid driver is like 650 lines
robust geyser
if u do boot protocol its probably like 10 lines of code lol
carmine swallow
if u do boot protocol its probably like 10 lines of code lol
that's not proper though
robust geyser
they never said anything about a proper one 
viscid flame
doubt you can make a proper (or even not a fully proper) hid driver in ~50-70 lo...
i meant boot protocol
carmine swallow
ah
robust geyser
but i think id rather shoot myself than write an hid driver
that shit is still scary to me
carmine swallow
debugginr real devices responding with a stall to a set address request is more scary to debug
viscid flame
that shit is still scary to me
btw what is astral xhci driver status
like is this working
carmine swallow
debugginr real devices responding with a stall to a set address request is more ...
or actually not set address, rather get descriptor (with the default mode after you have sent set address with the bit set so it doesn't actually get to the device)
robust geyser
btw what is astral xhci driver status
works
i hardcoded part of the usb stack into the xhci driver for testing and it works but ill probably need to test it on some real hw and fix some bugs
viscid flame
i hardcoded part of the usb stack into the xhci driver for testing and it works ...
the most painful part
robust geyser
but for now i’m trying to write the usb stack and then adapt the xhci driver for it
grand shadow
@robust geyser seeing the keyronex post made me curious, how much glxgears fps do u get on your machine
on my machine I get like 500-600
robust geyser
damn u makijg me get out of my bed
1s
just running it from xterm is fine? no envs?
grand shadow
yeah