i would do it myself but i have no time
#Astral
craggy tide
grand shadow
I'd need to find time too
grand shadow
why not like DISTROTYPE ?= full
I mean it worked for me when setting distrotype in the make cmdline
analog berry
hmm idk maybe its my make skill issue
grand shadow
just set it in the makefile so you dont have to pass it every time
craggy tide
i see your makefile just overwrites it
that's not how it works
if you do WHATEVER = lol or WHATEVER := lol those can still be overridden by user on the command line
analog berry
ohh
craggy tide
if you want to actually assign and not have the user be able to override then you have to do override WHATEVER := lol
(or =)
analog berry
whats the point of ?= then
grand shadow
what changed between the version I'm using (I think its 0.2 something) and new jinx
craggy tide
as per GNU make docs, ?= is basically make ifeq ($(origin FOO), undefined) FOO = bar endif
so it is a recursively expanded variable
idk, i don't see the point in ever using ?= because it's really error prone
craggy tide
what changed between the version I'm using (I think its 0.2 something) and new j...
some stuff has changed names
the container uses Debian now
analog berry
in what way is it recursive
craggy tide
so the imagedeps have potentially changed name
analog berry
whats the diff between = and :=
analog berry
yeah im glad i use cmake
craggy tide
GNU make is fine
as long as you know how to use it
so yeah
for anyone not wanting to waste time, cmake/meson are fine
analog berry
even the shittiest tool can be fine if ur an expert at it
craggy tide
(as in, waste time learning its nuances)
a lot of this shit honestly solely exists for (somewhat bad) compat with other versions of make
analog berry
@grand shadow any way to pipe logs i see at boot to my terminal?
analog berry
yeah
robust geyser
ah nvm sorry
analog berry
yeah like
i want to see whether new uACPI PR correctly handles unmapping tables
but i cant
because no logs
grand shadow
remove the logging_sethook calls in main from anything but the e9
obsidian pivot
remove the logging_sethook calls in main from anything but the e9
I needa implement something like that
analog berry
ngl its very inconvenient
why not multiplex it everywhere
hmm i do this
qemu-system-x86_64 -M q35 -m 4g -smp cpus=1 -no-shutdown -no-reboot -debugcon stdio -netdev user,id=net0 -device virtio-net,netdev=net0 -object filter-dump,id=f1,netdev=net0,file=netdump.dat -cdrom astral.iso -enable-kvm -cpu host
but no output to e9
@@ -53,7 +53,7 @@ void kernel_entry() {
dpc_init();
pmm_init();
term_init();
- logging_sethook(term_putchar);
+ //logging_sethook(term_putchar);
arch_mmu_init();
vmm_init();
alloc_init();
@@ -102,7 +102,7 @@ void kernel_entry() {
nvme_init();
virtio_init();
console_init();
- logging_sethook(console_putc);
+ //logging_sethook(console_putc);
oh wait you also have to enable e9 logging in kernel-src/Makefile
its annoying I need to make this a kernel cmdline option
well that isnt available that early into boot
bruhhh
analog berry
oh wait you also have to enable e9 logging in kernel-src/Makefile
ohh
wait
make clean wipes jinx cache??
analog berry
fuuck, there goes gcc rebuild 9999
grand shadow
@robust geyser can you cpuid | grep TSC on wsl I just need to see if my theory is true
I am still in denial grrr
robust geyser
czapek@raptor-wsl:~$ cpuid | grep TSC
TSC: time stamp counter = true
IA32_TSC_ADJUST MSR supported = true
TSC/clock ratio = 0/0
reference TSC access = true
TSC/APIC frequency MSRs = true
invariant TSC MSR = true
RDTSCP = true```alright yup
seems like hyper-v only passes invtsc to the guest if IA32_TSC_ADJUST is supported
fleet osprey
```
czapek@raptor-wsl:~$ cpuid | grep TSC
TSC: time stamp counter ...
I also get the same, lol
techflash@Michl-Win11ASUS:~$ cpuid | grep TSC
TSC: time stamp counter = true
IA32_TSC_ADJUST MSR supported = true
TSC/clock ratio = 0/0
reference TSC access = true
TSC/APIC frequency MSRs = true
invariant TSC MSR = true
RDTSCP = true
why not just run qemu on host with -accel whpx -cpu host
grand shadow
why not just run qemu on host with `-accel whpx -cpu host`
Wait shit true I think windows lets you run windows shit from wsl
grand shadow
https://github.com/Mathewnd/Astral/pull/12
Merged ty
robust geyser
Wait shit true I think windows lets you run windows shit from wsl
ive had some issues with that, for some reason windows path isnt jn my wsl path
also whpx is basically hyperv, no? wouldnt that mean your vm wont be able to use invariant tsc too?
give it a try tho im curious
grand shadow
thats comforting
think its complaining about the -cdrom
does that have to be power of 2 aligned in windows????
robust geyser
lmfao what
grand shadow
ok I think its just a wsl filesystem skill issue
copying it to windows and calling it there works
robust geyser
pain
so your solution is either copy the iso to the windows drive and run qemu from there, or move astral to the windows drive
keep in mind access to the windows drive is slow as balls, so you will lose a lot of performance
grand shadow
that doesnt work either lmao
grand shadow
gives access denied
I meant like the iso worked
so only way out is to call qemu outside windows and in a windows fs
robust geyser
that is cringe
robust geyser
yeah it seems like the easiest solution lol
grand shadow
actually crazy
grand shadow
qemu-system-x86_64 -cdrom astral.iso -accel whpx
robust geyser
wtf is maxphyaddr then 😭
do you have arch iso?
you could try booting linux and seeing what it is
cat /proc/cpuinfo | grep bits
but... what the hell?
analog berry
Thats fairly normal
grand shadow
cpu host doesnt exist so tryiung qemu64
analog berry
Host only works with accel
grand shadow
same thing with qemu64
grand shadow
Host only works with accel
neon crane
keep in mind access to the windows drive is slow as balls, so you will lose a lo...
not slow enough to be a deterrent IMO
neon crane
just copy an iso to %temp% and thats it?
i've also been doing most of boron's development on wsl1
(but tbcf i am using wsl 1)
analog berry
Maybe with kvm then
robust geyser
copying hundreds of megabytes on every run across their shitty plan9 impl is pain
neon crane
copying hundreds of megabytes on every run across their shitty plan9 impl is pai...
fdym hundreds of megabytes
robust geyser
the other solution would be to move astral to a windows directory and compile and run it from there
but jinx wont work, thats problem number one
and well i wont bother explaining further because thats a deal breaker
grand shadow
or just see wtf is not letting me shrink the partition on windows and fix that
and then install linux
robust geyser
or just see wtf is not letting me shrink the partition on windows and fix that
fragmentation maybe
neon crane
but jinx wont work, thats problem number one
whats picky about jinx
robust geyser
not about jinx
i bet linux is gonna have issues mounting stuff on top of that p9 fs
idk what influences that
jinx is a bit specific
but people have had issues on more "normal" setups
so i wouldnt expect that abomination to not cause issues
plucky crypt
ok I think its just a wsl filesystem skill issue
iirc you have to use wslpath to get it to work
grand shadow
r you shitting me that linux doesnt recognise the disk
think its because of the optaine raid bullshit that came with it
ah fuck it I will just add another fallback for kvm
first hobby os to have kvm clock because windows skill issue
funny
grand shadow
and I might as well have a timekeeper source abstraction as well
the order of priority will be kvmclock -> tsc -> hpet
I think
grand shadow
bruh leaf 0x40000000 is returning all 0s
grand shadow
ok its a skill issue of mine it seems, artix says kvm clock is real
actually wait
looking at __get_cpuid in gcc
static __inline int
__get_cpuid (unsigned int __leaf,
unsigned int *__eax, unsigned int *__ebx,
unsigned int *__ecx, unsigned int *__edx)
{
unsigned int __ext = __leaf & 0x80000000;
unsigned int __maxlevel = __get_cpuid_max (__ext, 0);
if (__maxlevel == 0 || __maxlevel < __leaf)
return 0;
__cpuid (__leaf, *__eax, *__ebx, *__ecx, *__edx);
return 1;
}
if (__maxlevel == 0 || __maxlevel < __leaf)
bruh
bruhhhhhhhhhhhhhhhhh
bruhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
unsigned int __ext = __leaf & 0x80000000;
time to do my own cpuid fuckkkk that
analog berry
thats why u write your own cpuid
its like 1 line of code
struct cpuid_res {
u32 a;
u32 b;
u32 c;
u32 d;
};
void cpuid(u32 function, struct cpuid_res *id)
{
asm volatile("cpuid"
: "=a"(id->a), "=b"(id->b), "=c"(id->c), "=d"(id->d)
: "a"(function), "c"(0));
}
winged basin
```c
static __inline int
__get_cpuid (unsigned int __leaf,
unsigned int...
why is everything a bloody underscore mess
grand shadow
gnu gcc
analog berry
why is everything a bloody underscore mess
its a gcc builtin
winged basin
its a gcc builtin
that's so gcc
analog berry
indeed
winged basin
they love underscores almost as much as linux people do
analog berry
the only difference is they're technically allowed to use it
tribal dust
__underscore
obsidian pivot
robust geyser
very nice
grand shadow
wrmsr(MSR_KVM_SYSTEM_TIME_NEW, (uint64_t)timekeeper_source_info->private | KVM_TIMER_SYSTEM_TIME_ENABLE);
I do tell it to tick
maybe I am doing it wrong
(probably)
robust geyser
whats KVM_TIMER_SYSTEM_TIME_ENABLE?
grand shadow
1
struct pvclock_vcpu_time_info {
u32 version;
u32 pad0;
u64 tsc_timestamp;
u64 system_time;
u32 tsc_to_system_mul;
s8 tsc_shift;
u8 flags;
u8 pad[2];
} __attribute__((__packed__)); /* 32 bytes */
whose data will be filled in by the hypervisor periodically. Only one write, or registration, is needed for each VCPU. The interval between updates of this structure is arbitrary and implementation-dependent. The hypervisor may update this structure at any time it sees fit until anything with bit0 == 0 is written to it.
💀
grand shadow
It filled the buffer I think but the values dont change I need to see if its some caching memes or something but Im not home
grand shadow
seems like it DOES need the TSC, oops!
doesnt need calibration though, which is nice
there was literally a code block in the manual
I really need to rtfm better bruh
grand shadow
ok but the docs also suck
it is ticking now but seems to be a bit faster than real time
was my skill issue, works fine now
calibrate-less ns precision clock nice
tomorrow I will hook up the hpet into the timekeeper source thing with a lower priority to act as a fallback
gritty shore
Damn nice
robust geyser
i am currently fixing up all of my memory manager bullshit because apparently i was less than aware of what i was doing at the time i wrote it, but after that i just might get inspired by your time source abstraction
robust geyser
in my old kernel the lapic timer either went too slow or too fast every boot
i assume its just bad calibration
honestly i didnt even know kvmclock was a thing, ill also implement it in my kernel cause it seems cool as hell
idk why but "stealing" features like that feels wrong, especially right after someone just implemented them in their own kernel
grand shadow
I mean kvm clock is pretty neat, you dont have to deal with calibrating or anything and its fast and linux also deals with most of the computation and gives you just a formula to transform an rdtsc and the structure into a nanosecond monotonic clock
For running in vms Id maybe say its better than an invtsc
Tho idk too much abt timers so
I found kvmclock out of desperation 
robust geyser
exactly
it is a good clock source, especially if linux defaults to it
sad its only available when running under kvm
grand shadow
Only complaint is the horrible doc like bruh
robust geyser
dont worry i won't have to worry about it
grand shadow
You have to do a multiplication and use the 128 bit results
robust geyser
i'll just yoink ur code (jk)
robust geyser
You have to do a multiplication and use the 128 bit results
what the hell
good thing zig supports arbitrary sized integers :^)
grand shadow
uint64_t tsc = rdtsc_serialized(); ticks = tsc - kvm_timer_info->tsc; ticks = (kvm_timer_info->tsc_shift >= 0) ? (ticks << kvm_timer_info->tsc_shift) : (ticks >> -kvm_timer_info->tsc_shift); // we need to pretend ticks is a *128* bit variable! // this will be done in just an inline assembly stub... // mulq will leave the 128 bit results in rdx:rax // shrd will do the 32 bit shift required // we will stay with the low 64 bits and leave the high 32 bits, as they are not nescessary // (the 64 bit nanosecond counter will overflow in centuries...) asm volatile ("mulq %%rdx; shrd $32, %%rdx, %%rax" : "=a"(ticks) : "a"(ticks), "d"(kvm_timer_info->tsc_mul)); ticks = ticks + kvm_timer_info->time;
Yeah idk what's the deal with it
sleeps are 2x too fast
Maybe I should calibrate it like three times and get the average
robust geyser
i assume it's bad calibration because of the code overhead, timer variance or something else
i do an average and it's not very inaccurate but i could sometimes notice it
it was definitely accurate enough for simple time keeping lol
sleeps were pretty accurate as long as you didnt needs like ns precision
gritty shore
i assume it's bad calibration because of the code overhead, timer variance or so...
I use the HPET
grand shadow
hpet is slooooow
gritty shore
sleeps were pretty accurate as long as you didnt needs like ns precision
Yeah no mine is inaccurate on a seconds basis
robust geyser
so do i, but your code still takes some time to execute
and it could vary if you get an irq or something
or if the memory access takes a bit longer
gritty shore
hpet is slooooow
What do you use
grand shadow
What do you use
I used to use the hpet for time keeping and apic timer for everything else (per cpu queue of timer events)
but I'm switching to tsc/kvm timer for time keeping
gritty shore
I just use HPET for calibration
robust geyser
i feel like those tiny changes could affect the timer calibration accuracy noticeably, but i might be overthinking
grand shadow
as I will measure sleep and run time in the scheduler and hpet overhead is too much for modern processors
gritty shore
i feel like those tiny changes could affect the timer calibration accuracy notic...
Idk I do have a bit of overhead
I call timer_sleep which calls hpet_sleep if it is present
I should probs add a PIT fallback
grand shadow
how long do you sleep for
gritty shore
10ms
I tried doing like 1sec and even that didn't work
So in the end I just ended up doubling the speed of the apic so sleeps are accurate
analog berry
seems like it DOES need the TSC, oops!
Wdym
tawny oak
Maybe I should calibrate it like three times and get the average
Fwiw I found doing 5-8 calibration runs, dropping any outliers and taking the average of the remaining runs gets pretty accurate results.
and in terms of calibration, dont do a polled sleep for xyz amount of time, instead sleep for at least that amount of time, and have your calibration results use the actual time slept for. Like if you're using the PIT, you've got a ~838ns resolution, so you cant calibrate for just 10ms, it would be something nearby (and that error adds up over time).
gritty shore
and in terms of calibration, dont do a polled sleep for xyz amount of time, inst...
Not a bad idea, thanks!
grand shadow
Wdym
I thought itd just update the memory but it still relies on tsc cuz I didnt rtfm properly
analog berry
oh so it turns the tsc into that counter?
grand shadow
oh so it turns the tsc into that counter?
More so like, you use a formula to turn the current tsc and the structure data into a monotonic nanosecond clock
Which it updates periodically to account for say sleep time
analog berry
ah
grand shadow
Its pretty neat, you dont need to calibrate it and it can be used as early as you can allocate memory
You allocate some guest memory, write the address into a msr and done
Just have to check you are running in kvm with cpuid
And its supported
It doesnt support interrupts as far as I can see though, so it is only good as time keepinf
grand shadow
pretty cool
now I just have to do the normal init and I'll be done
the normal init is where clocks that need calibration etc will be
or depend on some other feature like the acpi timer
tribal dust
they do tho
the wsl kernel is shared across all wsl instances including internal ones used for WSLg
so the WSLg instance will have gpu paravirtualization
and there's a damn linux build of directx for WSLg to use
for paravirtualization to work the host and guest drivers need to be the same or very similar or compatible versions or something like that i believe
i know a lil bit about WSLg internals because I wanted GPU partitioning/paravirtualization to work in a hyper-v vm
it is possible but turns out my gpu did not support being partitioned or passed through or paravirtualized
any interaction on the gpu in the linux guest would cause very weird issues on the host
black screens blue screens etc.
anyhow but on supported GPUs it's actually very nice
works well
it involved applying the WSL kernel patches to arch's kernel
and copying some loadable kernel modules from the internal WSLg instance
directx on linux is cursed tho
it's basically a directx calls over a /dev/dxgkrnl or something device with ioctls
this is the only time I hacked on the linux kernel btw
the wsl kernel patches were old at that time
I took their 5.7 stuff i think and ported it to 5.11 or 6.2 i don't actually remember
it involved some hacking
gpu stuff didn't work but I got hyper-v tunnels to work!
so I got more responsive RDP because it didn't go through the TCP/IP stack
grand shadow
interesting
alright normal init should be done
I also did some changes to early init, it now takes a us_offset to offset the early clock a bit if nescessary (like during smp startup)
seems like switching the clock source from the hpet to the kvm clock also gave a pretty nice speed increase to glxgears fps
interesting
obsidian pivot
damn hpet always ruining performance
grand shadow
yeah it seems like it was an hpet skill issue
even moving between fvwm pages is faster
the time seems to be properly synced up which is nice
used to be at ~250 fps
deft oak
epic
grand shadow
something I did also seems to have fixed the tyr-glquake visual glitches?
funky
alright pushed the new clock stuff
https://github.com/Mathewnd/Astral/blob/rewrite/kernel-src/arch/x86-64/kvmclock.c @robust geyser if you wanna take a look at how kvmclock is implemented
for when you use it
obsidian pivot
I might look into using that for obos soon™️
grand shadow
its pretty good to use as the main source when running in a vm
astral just passed 1000 commits
nice
obsidian pivot
big
analog berry
astral just passed 1000 commits
Uacpi is 200 behind, damn
obsidian pivot
when will astral have a cmake port
grand shadow
when will astral have a cmake port
how difficult is a cmake port
obsidian pivot
never did one
grand shadow
alright pushed the new clock stuff
I still need to add one more source (inv tsc) for real hw/other vms
grand shadow
how difficult is a cmake port
@prime juniper do you have this knowledge
prime juniper
not difficult
obsidian pivot
I mean surely if windows has support for cmake, and like, windows is windows
prime juniper
managarm has one
you'll need libuv, libarchive and curl
and curl and libarchive are very easy
libuv I did some patching in
see bootstrap-managarm for recipes and patches
grand shadow
hmmmm thanks
I have curl and libarchive would be useful anyways for bsdtar and stuff
obsidian pivot
then I can finally build obos on astral
grand shadow
I'll look into it after I do inv tsc
tribal dust
I have curl and libarchive would be useful anyways for bsdtar and stuff
that's progress for self-hosting yayy
cuz libarchive/bsdtar is used by jinx
grand shadow
jinx cant run on astral and never will with how it currently works since it uses a container
grand shadow
the only thing stopping astral from being self hosting more so is the installing the new kernel to disk part
I've pushed a commit from astral before
obsidian pivot
what do I cat to /dev/acpi to get it to shutdown
grand shadow
but still cant save a new kernel to disk and boot it cuz no fat32 support
obsidian pivot
s?
analog berry
in binary size I mean
Dunno tbh, oberrow knows
grand shadow
p I think
obsidian pivot
ok
obsidian pivot
Dunno tbh, oberrow knows
let me check without ubsan/kasan
tribal dust
i should write an allocator for my kernel and not use the linked_list_allocator crate
obsidian pivot
iirc I got this from your site a week or two back
grand shadow
oh yeah then that one is bugged wrt to the acpi dev
grand shadow
i should write an allocator for my kernel and not use the `linked_list_allocator...
slab is pretty good
oh wait rust
uhhh bleh
idk what a good rust allocater would be
gritty shore
jinx cant run on astral and never will with how it currently works since it uses...
Yeah that's annoying
grand shadow
@obsidian pivot I'll write the invtsc source, port cmake and then upload a new image for you
obsidian pivot
yay
errant zephyr
jinx cant run on astral and never will with how it currently works since it uses...
port everythng to nix :^)
deft oak
idk what a good rust allocater would be
why would the language matter for that
gritty shore
you'd have to switch to a bunch of uncontained shell scripts
obsidian pivot
you'd have to switch to a bunch of uncontained shell scripts
my static lib for uacpi is 1.2M
without ubsan, kasan, or debug info
wait there is debug info
without debug info it's 455K
now I'll see how it is with optimizations
1.7M
prime juniper
I have curl and libarchive would be useful anyways for bsdtar and stuff
libarchive also required for xbps package manager
analog berry
without debug info it's 455K
Smol
obsidian pivot
with optimizations it's 409K
gritty shore
One day I'll switch from lai
gritty shore
Nah too lazy
Shouldn't actually be hard tho, as I abstract acpi stuff in my AcpiPc class
obsidian pivot
imagine abstracting acpi
gritty shore
It's useful for acpi devices
Like
I discover devices through acpi and give them a pointer to the acpi class
grand shadow
imagine doing device discovery and not just calling 0213913021391293219391239129 init() functions in main 
(after I do my scheduler rewrite I will redo how I do devices)
gritty shore
It's clean and object oriented
robust geyser
lmfaoo
grand shadow
my timespec_from_us function was bad, oops
and date shows the correct time nice
perfect
bro the vmware nvme drive still has some debug shit I had for some bug I was trying to catch and it still works lmao
grand shadow
time to real hardware test
and I think this timekeeping stuff is the last thing I needed before the scheduler rewrite
as now on any fastish computer its gonna use the tsc/kvm timer and on any slow computer its gonna use the hpet and the access time to the hpet wont matter as much
grand shadow
ok so on smp its still a bit weird
each core has its own tsc base and when a thread migrates cpu the timing gets weird
either way tsc is working on real hardware
I just have to deal with the timing shit
the clock is still a bit faster
1 minute on windows on my desktop was 1min3sec on my laptop running astral
acpi poweroff still works
grand shadow
time to get cmake ported for oberrow
grand shadow
once I get this scheduler stuff and maybe driver stuff sorted I want to port CDE
I really wanna keep a unix workstation feel to astral yknow
and theres nothing better than cde for that
obsidian pivot
time to get cmake ported for oberrow
based
winged basin
once I get this scheduler stuff and maybe driver stuff sorted I want to port CDE
great idea
my amiga 2000 with it
grand shadow
my amiga 2000 with it
125/128 mib that poor thing
My next step today will be synchronizing the timekeeping of every cpu
obsidian pivot
@grand shadow does astral support linux's fbdev devices (/dev/fb0)
grand shadow
<@215823874615541761> does astral support linux's fbdev devices (/dev/fb0)
Yeah kinda
Enough for xorg
grand shadow
Yeah
obsidian pivot
ok that's all I needed
and you of course have pthread
and can mmap the fbdev
I'm gonna try building my fireworks test for astral
specifically the linux port
the obos one needed many changes
to work on linux
well not too much, but still quite a bit
now where is the astral compiler stored at....
found it
errno.h no such file or directory
obsidian pivot
in native mode
is native mode using the win32 APIs?
like CreateThread instead of pthread
obsidian pivot
oh
@grand shadow
/home/oberrow/Code/Astral/host-pkgs/xgcc/usr/local/bin/x86_64-astral-gcc -c src/main.c -o bin/main.o
src/main.c:1:10: fatal error: errno.h: No such file or directory
1 | #include <errno.h>
| ^~~~~~~~~```NO err NO! everything must SUCCEED!
grand shadow
<@215823874615541761>
```
/home/oberrow/Code/Astral/host-pkgs/xgcc/usr/local/bi...
Its configured to be ran from inside the jinx sysroot......
Youd have better luck wgetting it and compiling inside astral
grand shadow
obsidian pivot
it compiled with little changes
and it is not going well
and it SIGSEGVs when I ctrl+c
I think
running it again causes something to deadlock
and the worst part is, the test doesn't look right
grand shadow
Bruh
obsidian pivot
I'm pushing the fixes I made to the build system rn
then you can see how it is for you
obsidian pivot
I'm pushing the fixes I made to the build system rn
from astral ofc
tawny oak
and it SIGSEGVs when I ctrl+c
Close enough
obsidian pivot
because astral is just like that
@grand shadow if you want to test, clone the repo
to build is just make all
and to run is make run
grand shadow
Wont be home for a bit but sure
obsidian pivot
so it seems like you have some sorta timing issue
when the fireworks test first starts
because there are waaay too many particles
grand shadow
I did fix some timer issues a bit back I think
Yeah it was after the image I uplladed
Uolaoded
Bleh
So we'll see
obsidian pivot
if you want I can join vc for a bit and show you how it looks
also on a second run of the test after the initial one stops
the entire system deadlocks
and with smp it deadlocks on the first run
grand shadow
I am in the middle of class so cannot atm
obsidian pivot
Are you saying obos is more stable than astral
well it doesn't deadlock
running the fireworks test in kernel mode
but if I'm being fair, that test runs fine when it uses timer objects as opposed to the scheduler
but when it uses the scheduler, it doesn't deadlock, it just runs like shit
my next port for the fireworks test will be windows
userspace windows ofc
I couldn't be arsed to do kernel-mode
obsidian pivot
Why would u do it in kernel mode anyway
Idk
analog berry
What hardware?
neon crane
if i port the fireworks test to a windows kernel driver i'd probably manipulate the VGA registers directly
analog berry
This is a multithreaded performance test
neon crane
but it has to be displayed somewhere no?
if you do it in a window you might stutter out the window manager etc
analog berry
The commands to draw stuff are queued from userland
neon crane
im talking about WINDOWS
its probably not as easy to get access to the frame buffer directly
analog berry
I think its the same for windows
analog berry
Otherwise mesa wouldn't work on windows
neon crane
i mean you can open the desktop window and draw on that
but its not the same lol
and it'll probably represent a giant bottleneck
if it even works at all
inland cliff
How do games render then?
obsidian pivot
How do games render then?
Opengl, vulkan, directx
robust geyser
tons of usermode and kernel mode apis put together
but yeah it all in the end is usually just opengl/directx or vulkan
that usually is provided by a usermode library from your graphics card vendor, and there is also something libglvnd but i'm not sure how that works lol
inland cliff
Opengl, vulkan, directx
Was a rhetorical question tho xD
obsidian pivot
<@215823874615541761> if you want to test, clone [the repo](<https://github.com/...
Damn it, I forgot to stage the other changes
To the makefile
When you build it make sure to compile src/x86_64-randseed.asm to the makefile
I just fixed that from school
neon crane
How do games render then?
remember that the fireworks test is massively multithreaded
each particle is one thread
and each thread represents one particle
regular graphics APIs would suck to synchronize and would ruin the point of the test
this is why oberrow's linux port directly uses a framebuffer device
winged basin
My next step today will be synchronizing the timekeeping of every cpu
this is something i was thinking about lately
with invariant TSC (which should be synchronised between cores), ARM generic timers, RISC-V mtime, it should be straightforward since a) the time counter is big, should never overflow, and should be common between cores and b) it's easy to read it
tribal dust
what does synchronizing the timekeeping of every cpu mean?
does it mean like, synchronizing the core-local timers of each core/hart
or does it mean cross-arch compatibility of sorts between them (no idea how this would work or what it would be useful for
neon crane
I thought we had come a long way in terms of graphics
we had
but like
the fireworks test uses single pixels
and the updates ought to be completely independent
i dont think modern graphics APIs can do that
grand shadow
sometimes it doesn't even work on qemu
Wtf
It always works for me
Itd be easier for me find out whats wrong IF I HAD A MACHINE WHERE IT DIDNT WORK ARGHHB
grand shadow
I should get my mlibc sysdeps upstream soon
robust geyser
100% a good idea
grand shadow
alright so on the menu today:
synchronizing the timekeeper between cpus right after smp init
making the mlibc sysdep a bit better and upstreaming it
harsh drift
The image makes me feel good
obsidian pivot
damn astral curse has always been here
grand shadow
I do miss a bit the limine tty because of the wallpaper but it had to go
gritty shore
why'd you remove it
gritty shore
oh right
neon crane
I do miss a bit the limine tty because of the wallpaper but it had to go
you can still use a wallpaper with flanterm
warped plinth
surely stb_image.h has something to do that?
neon crane
surely stb_image.h has something to do that?
you may be thinking of stb_image_resize.h
grand shadow
alright made timekeeping suck much less
alright so now I just have to finish the cpu topology work by writing the architecture agnostic part and then I can create a new branch and get my hands dirty
that'll be tomorrow though today I will tidy up the mlibc patches and throw it upstream
tawny oak
alright made timekeeping suck much less
nice, I might take a peek at your implementation as I'm in the middle of that stuff myself.
grand shadow
Its not the cleanest implementation but at least it supports an arbritary number of sources (even tho the tsc impl would break on systems that have say sockets with different clocks)
winged steppe
i've been using this as reference, thanks 🙏
grand shadow
mlibc sysdep upstream 🎉
time to rebuild the world
to make sure everything is still working
grand shadow
yeah it'd take quite a bit
though if I do like 25 packages per day once I finish all the kernel stuff I plan on doing it could be more manageable
gritty shore
would be nice to see all the recipes updated as well, but it's *so much work*
make a script to do it for you
craggy tide
make a script to do it for you
it's honestly not that crazy
shell lends itself really well for this stuff
the problem is more so validating recipe bugs that have been fixed in Gloire/Vinix
and/or improvements
the raw changing of the recipes shouldn't itself be that complicated to do
gritty shore
reason #38728372 to make a centralized recipes repo
tribal dust
reason #NaN to use #1279525023127830609
with nixstrap, nixpkgs is your centralized recipes repo
with an overlay to apply your patches and stuff ofc
it might be hard to port cppnix if you want nixstrap running on your OS to self-host - there are other nix implementations but idk if they're implementing everything needed for nixstrap
grand shadow
alright
I think I have all I need to start work on the new scheduler
I'm gonna create a new branch and start simple
start with the interactivity stuff and measure interruptible sleep and run time
obsidian pivot
don't forget to test
the fireworks
because they look cool and start many many threads
grand shadow
me trying to understand wtf these hieroglyphs are (I wrote this a while ago and my handwriting sucks)
crude agate
me trying to understand wtf these hieroglyphs are (I wrote this a while ago and ...
more than your writing being bad I'd say you have improperly used the space on the paper, aka you have written random words randomly on it
grand shadow
Ill try doing some more work today if factorio doesnt consume me again
warped plinth
__________
/ \
/ R.I.P. \
| Here lies |
| Mathewnd. |
| Consumed |
| by |
| Factorio |
| |
|____________|
lol
charred sinew
Wait until the expansion releases
lean kettle
when the expansion releases the entire IT world is going down for a week
tribal dust
being a factorio developer must be hard how are they not being consumed by factorio
lean kettle
being a factorio developer must be hard how are they not being consumed by facto...
why else do you think they are fixing tiny bugs and optimizing the hell out of everything
they probably have the biggest megabases the game has ever seen
grand shadow
Sorry for the lack of progress just havent had a lot of motivation lately
Soon™️ I will return
grand shadow
anyways I remember fixing some memory bug. I should try to build gcc again
grand shadow
do you have uni assignments?
very few surprisingly
my uni doesnt like to give assignments out as much as it likes to just make you study a bunch at home
here we go again
analog berry
debugging the rbtree segfault?
grand shadow
I dont have a phd so not really
just seeing if it works because IIRC I fixed some memory issues since last time and also like it built normally on mlibc linux
analog berry
does this mean astral has memory bugs 😲
obsidian pivot
does this mean astral has memory bugs 😲
Who doesn't
grand shadow
does this mean astral has memory bugs 😲
I fixed them 
qemu did a bunch of shit with memory
which let me catch a few
analog berry
wdym
grand shadow
it does some very specific mmaps with replace and mprotect and stuff and iirc there was some bug in some path
lemme find the commit
qemu also helped me find some signal bugs
yeah it was something on my function which handles unmapping and changing memory protection
neon crane
I dont have a phd so not really
i will learn algorithmics and data structures in year 1 2nd semester what do you mean
also Ph.D. means philosophiae doctor, try again
obsidian pivot
doctor of philosophy
neon crane
yup
obsidian pivot
do I need a phd to fix obos memory bugs
neon crane
btw what the hell is "secret private club access" 🤨
neon crane
do I need a phd to fix obos memory bugs
i hope not
obsidian pivot
btw what the hell is "secret private club access" 🤨
dw about it
neon crane
do I need a phd to fix obos memory bugs
nah i think you just need to focus
obsidian pivot
good point
obsidian pivot
@mint can iprogramincpp have the role
grand shadow
i will learn algorithmics and data structures in year 1 2nd semester what do you...
it was a joke about managarm code but yeah
neon crane
anyway I should probably work on boron again at some point. I took a break to develop a small game
neon crane
but I'm like, completely clueless on where to proceed next
well almost completely clueless
obsidian pivot
userspace
neon crane
well i need the memory manager for that
obsidian pivot
nah i think you just need to focus
I will now disappear
Page fault at 0xffffffff80007f65 in kernel-mode while to read page at 0xffffff0020b96000, which is unpresent. Error code: 0```the question is which part of the memory manager should i do first
neon crane
```
Page fault at 0xffffffff80007f65 in kernel-mode while to read page at 0xffff...
show me the code
obsidian pivot
this'll take a bit
obsidian pivot
show me the code
there's the stack trace
static OBOS_NO_KASAN void page_in(uintptr_t base, size_t sz)
{
uintptr_t curr_base = base;
for (size_t i = 0; i < sz; i += OBOS_PAGE_SIZE)
{
volatile char prev = ((char*)curr_base)[i];
((volatile char*)curr_base)[i] = prev;
}
}```volatile char prev = ...```
is the crash location(yes, I know that this is bound to break quickly, since the pages are not locked)
((but I can't lock pages in yet, sooooo))
grand shadow
still hasnt segfaulted
astralbros maybe we are so back
its compiling some gcc/config/i386/i386-features.cc file rn
ASTRALBROS WE ARE SO FUCKING BACK
IT PASSED PARSER.CC
IT WAS THAT STUPID ISSUE
@prime juniper update on gcc
it was my skill issue wrt to vmm stuff
prime juniper
Based
grand shadow
seems to be linking cpp and xgcc
xg++
have there been any other hobby oses which have compiled gcc?
grand shadow
seems like its doing fixinclude stuff
grand shadow
has managarm done it?
mlibc has on linux but not managamarm
afaik
good lord lol
big library
grand shadow
been linking lto-dump for a bit
grand shadow
still hard at work linking lto-dump
obsidian pivot
still hard at work linking lto-dump
here, take cat picture while waiting
grand shadow
cat!!
obsidian pivot
that is indeed her name
grand shadow
maybe a 4gb disk wasnt enough
fucking hell lol
resize2fs to the rescue
its gonna have to restart the lto-dump linkage though
dim arrow
here, take cat picture while waiting
Is that your cat?
grand shadow
linking cc1plus
grand shadow
lto1
grand shadow
Ill leave it compiling and go to sleep
sacred gyro
me when Astral is getting closer to being self hosted
analog berry
how the fuck did astral get to this point in 2 years when managarm couldnt in 10
analog berry
it was my skill issue wrt to vmm stuff
what was the bug
sacred gyro
how the fuck did astral get to this point in 2 years when managarm couldnt in 10
he's just that good
analog berry
unironically
well managarm has drm and more real hw support, but nothing crazy besides that
robust geyser
managarm didn't have all the software ported for them
and they also built mlibc
and a bunch of async stuff for c++
so like yeah, it took 10 years but the scope of the entire project is like 10x bigger too
analog berry
well mlibc is a fair point
prime juniper
mlibc has on linux but not managamarm
No it hasn’t. Build failure in gcov bullshit
And currently Managarm itself runs into some configure memes which result in broken makefiles
grand shadow
FUCK OFF WINDOWS AUTOMATICALLY UPDATED DURING THE NIGHT AAAAAAAAAAAAAA
e2fsck says its clean so hopefully I can recover it
grand shadow
what was the bug
some range bullshit when changing the size of a range
died building libgcc it seems
I'm making a copy of the disk just in case bruh
thank god it had already automatically synced the page cache back to disk when windows decided to troll me
neon crane
FUCK OFF WINDOWS AUTOMATICALLY UPDATED DURING THE NIGHT AAAAAAAAAAAAAA
so it like force rebooted or what
grand shadow
so it like force rebooted or what
yeah I came back to everythinjg closed
its this part of a script thats dying
obsidian pivot
Is that your cat?
Yes
wild marsh
no assembling and linking for you
grand shadow
it does seem to invoke xgcc which is the built gcc and it does seem to at least not immediatelly die
just gotta finish the job
wait what the fuck
this is from the main configure
}?/???
HUH
wait maybe I forgot to pass some shit to mke2fs
wait no
why are there to x86_64-astral-* programs
seems to be the missing tooldir=/usr in make in binutils?
(seeing from the lfs stuff)
do I possibly have to have to build binutils in astral too
crude agate
do I possibly have to have to build binutils in astral too
why not
grand shadow
its just kinda annoying cuz it takes a bit
@prime juniper mr portmaster
binutils on managarm has x86_64-managarm-mlibc-ar?
prime juniper
I think so yes
prime juniper
But I might misremember. Can’t check atm
grand shadow
cuz like aaaaaa
I think the gcc configure script just straght up checks for an x86_64-astral-whatever
but binutils is not installinga ny
prime juniper
Hmm
prime juniper
how do you compile it
Check bootstrap.yml line 925 and down
grand shadow
ty gonna look into it
grand shadow
ah fuck it we ball I'll just build binutils in astral
binutils is building
I assume it'll just work first try
oh
grand shadow
Enjoy porting texinfo
nah
make MAKEINFO=true
also damn I am surprised by how stable astral has been doing this shit
only time it crashed was when WINDOWS UPDATED 
prime juniper
``make MAKEINFO=true``
Works too yeah
prime juniper
also damn I am surprised by how stable astral has been doing this shit
It’s impressive
obsidian pivot
only time it crashed was when WINDOWS UPDATED <a:animated_crying:872130603233980...
grand shadow
epic
installing into a folder in /home/astral just to see if the execs are actually there
ugh
does however work
on wsl its implemented as a symlink
grand shadow
alright managed to find a way to get it work
time to copy the new binutils into the sysroot and then
reconfigure gcc and pray I dont have to build it all over again
grand shadow
thanks
well update on gcc
its not compiling anything new
its fucking linking everything again
including the shit that took hours to link
supple hatch
keep it up, proud of you
when will mint say this about nyaux 
craggy tide
one day
supple hatch
one day indeed
grand shadow
seems to be finishing the lto-dump linking rn
grand shadow
same issue ugh did this file not get regenerated
it didnt so I deleted it and PRAYING PLEASE WORK
ITS COMPOILING LIBGCC
YES
hm
what is it this time
oh some more files I have to remove I think
yup collect-ld still has the old info
I should run a grep and see which files are outdated
what
grand shadow
is it not searching in the location of the file?
what
*something* is wrong
I'll leave it running with a syscall dump even if it takes ages
wait
HUH
WHY IS THE H MISSING IN .H
WHAT
IS MY READLINK BROKEN
wild marsh
trolled?
gritty shore
do you arbitrarily limit paths or something
grand shadow
nope
probably an ext2 inode meme
since if the symlink is like 60 bytes or lower it gets stored in the inode
so I am probably not handling something properly somewhere
considering the link there is exactly 59 bytes large
thats the only explanation idk
grand shadow
the value returned by readlink seems fine though?
as in the filename copied
hm
unless its using the length provided by stat and that is somehow wrong?
grand shadow
if (int e = mlibc::sys_readlink(resolv.data(), path, 512, &sz); e)
return e;
if(debugPathResolution) {
mlibc::infoLogger() << "mlibc realpath(): Symlink resolves to '"
<< frg::string_view{path, static_cast<size_t>(sz)} << "'" << frg::endlog;
}
if (path[0] == '/') {
// Absolute path, replace resolv
resolv.resize(sz);
strncpy(resolv.data(), path, sz - 1);
hmmmmm that sz - 1 there seems sussy
unless mlibc expects readlink to be handled in a different way
from posix:
The following example shows how to read the name of a symbolic link named /modules/pass1.
#include <unistd.h>
char buf[1024];
ssize_t len;
...
if ((len = readlink("/modules/pass1", buf, sizeof(buf)-1)) != -1)
buf[len] = '\0';
readlink() places the contents of the symbolic link pathname in the buffer buf, which has size bufsiz. >>>readlink() does not append a terminating null byte to buf.<<<
On success, these calls return the number of bytes placed in buf.
so that strncpy is just....
completely removing the last character?
hmmmmm
erring on the side of mlibc bug on this, need to test
hm well that made it die on another header file
LOL ok
this one is actually a bug with my ext2 driver
but it seems like mlibc does have a bug wrt to realpath
lol yeah I had
60
instaead of >=60
oops
ughhehahghh I forgot to sync before leaving
shiiit I'll have to leave it linking everything again
ughhhhhhhhhh
at least I found hopefully the last bugs for libstdc++
welp while it relinks I will upstream the mlibc fix
grand shadow
well thats fun
first crash
well huh
its trying to write to the disk and uh
its trying to write past the end of the disk?
read/write
whatever
that does not sound healthy
hm
well I'll finish the mlibc stuff and look at it tomorrow, done a lot today
grand shadow
alright, pr opened
if I had to guess it runs out of space but my ext2 allocation doesnt account for the block group to be smaller or somethiung?
or something like that
supple hatch
well thats fun
Oopsies
obsidian pivot
lies, you said this when all obos had was a gdt
grand shadow
anyways time to see whats up with ext2 and continue the gcc build
I want to slap that self hosting label on astral already
analog berry
lesgo
analog berry
yes
supple hatch
!!!
grand shadow
its nice to see that at least the compiler it built works nicely for the libgcc build
grand shadow
bruh
why is it trying to
bruh
why
is it trying to
literally
read a symlink
I'm probably mishandling AT_SYMLINK_NOFOLLOW on openat
wait no thats not even valid
how is it getting an fd to a symlink
if it even is going there from a read I need to stacktrace
analog berry
astral fs corruption any%
grand shadow
astral fs corruption any%
its not even that, e2fsck hasnt complained in a good while
I'm probably telling the vfs_lookup function somewhere to not follow the symlink
and its writing to the link
not writing
reaidng
hm
oh
count=60
I'm probably
shit
I'm tarted
gritty shore
lots of problems with that 60 lol
analog berry
so it is an fs bug?
grand shadow
yeah
analog berry
i like your magic numbers
mathew before fits_into_inline_storage helper was invented
grand shadow
grand shadow
hmmm
grand shadow
it will also give u your perf numbers
probably on par with reading from a floppy disk 
analog berry
lmao
e.g. on linux u get about a million IOPS for random 4K reads from a good NVMe
and its not linux io stack bound
grand shadow
well I just have to hope uh
I wont have to redo the whole build
because this was a write issue
and if I do well shit
at least I'll test if it will get to that point alone
analog berry
u probably should because who knows what else is tainted by that bug now
grand shadow
yeah true
it does do a bunch of symlinks in libstdc++ include to the main tree
so yeah
well so far, since I fixed the vmm funny:
my binutils wasnt configured properly to add the x86_64-astral-* programs (I think)
mlibc realpath bug
symlink bs
analog berry
Well thats quite decent so far
analog berry
Lets keep our fingers crossed
grand shadow
the good thing about doing this stuff is that it will root out bugs like this whether I want it or not
analog berry
Yeah
grand shadow
bruh
grand shadow
well clear is not working too so
wtf
I think something is broken
was there any abi break pushed to mlibc master
analog berry
The ext fix broke everything
grand shadow
well
reverting mlibc fixes it
was there an abi break or something
guess I'll try to figure out which commit caues that
inb4 its my realpath fix
doesnt seem to be my fix breaking things, that commit works
obsidian pivot
git bisect
grand shadow
yeah mlibc bug
not my commit tho so the blame aint on me 
anyways anyhow I will leave it building
on the commit I know works
obsidian pivot
not my commit tho so the blame aint on me <:halfmemeright:715831702164996096>
git blame
git blame who_did_the_bug
030618d5 (mathewnd 2024-10-14 21:27:54 -0400 1) it was mathewnd
(fun fact, that commit actually exists in obos)
grand shadow
I cursed obos the game was rigged from the start
obsidian pivot
(fun fact, that commit actually exists in obos)
well except for the commit message
and that file does not exist
in obos
grand shadow
well thats fun, that at exit cleanup pr broke everything lol
so I will just leave it a commit before and call it a day
and leave gcc building
grand shadow
Linking cc1plus