Mostly to make it easier to git clone and build it inside astral
#Astral
grand shadow
analog berry
noo i want to spend the next 10 hours after i clone astral watching it build llvm 5 times 
grand shadow
Adding linux namespaces and syscall compat to astral just to force people to go through that
grand shadow
I hit an assert in the page cache yesterday but didnt get around to looking into it very much, qemu is still open for me to fix that later
its probably some race condition
which causes the dirty flag of the page_t to get nuked
never happened before but its good to try to fix it
anyways astral fallen billions must die
I am still flirting with the idea of switching inits, and I need to make my mind at some point
I am still between pure sysvinit and sysvinit + openrc
in one hand just having sysvinit would be simpler and more lightweight
on the other hand having openrc would be fucking cool
and more user friendly than the /etc/rc.d scripts
I have used both before and its kind of a funny story on how I used sysvinit for a while
my (relatively, it was still a fucking celeron) faster laptop had broken and I was then stuck with a 2004 laptop running linux from scratch
for like 2 months
robust geyser
I am still between pure sysvinit and sysvinit + openrc
systemd 
gritty shore
I am still between pure sysvinit and sysvinit + openrc
try runit
grand shadow
I have never used a runit distro so I have no idea how it even works or how to interact with it
gritty shore
afaik it's basically sysvinit but more modern
- it'll fit the theme of stealing stuff from void linux
grand shadow
Hmmmm
feral depot
gritty shore
OpenRC is probably nicer than runit though
It is nicer to use but runit is very fast
marsh sparrow
but it lacks dependencies, for example (other than re-trying into they are met)
prime juniper
But systemd
grand shadow
there might not be any work until after thursday, got a linerar algebra exam to study for
grand shadow
I do think I should rewrite the pagecache soon to use a more standard per node avl tree
Since Im taking a dsa class anyways implementing an avl tree will be good practice and be useful
Same with a rbtree
I would still keep the standby list, but I would switch out the dirty list in favor of a per vnode dirty list and instead have a list of dirty vnodes that would get distributed by the handler between some i/o threads
grand shadow
Exam done, work shall resume
grand shadow
think I fixed it, lazy fix but I'm going to rewrite this soon either way
I think work for today will be adding some ports needed for self hosting
namely rsync (linux headers for mlibc needs this), patch and... perl... automake... autoconf
libtool
pkgconfig
fucking perl
grand shadow
considering perl is gonna block me self hosting I might just try to make it work with mlibc if mlibc was the issue with it on managarm
grand shadow
why do I even bother
grand shadow
-DNO_LOCALE
funni
@prime juniper CFLAGS="${CFLAGS} -DNO_LOCALE":
see if that fixes the perl bullshit in managarm
I think I run into the same meme that fucked up webkit on mlibc
wonder if thats fixed already
seems like so
time for a mlibc and frigg bump
hopefully there were no abi breaks in mlibc
yup, fixed it
funny
and I think that was a libc panic, lets see what made cpan quit
huh
grand shadow
hm there seems to be no way of creating an empty source dir automatically with jinx
sad
there seems to be no reliable way of downloading perl module sources other than using cpan -g (tangent: does anyone know if ckan, the ksp mod manager, is inspired by cpan?)
oh it seems to be
cool
wait
nevermind I am stupi
d
and dumb
sage locust
hm there seems to be no way of creating an empty source dir automatically with j...
why would you want that?
grand shadow
why would you want that?
I did not see the download link for something and I was downloading it from the tool
but I now see it
sage locust
bad UI lol
grand shadow
grand shadow
bad UI lol
yeah cpanmeta ui is questionable
just noticed the install dir 🥀
okay if I copy it to the right dir it is found
and the next one to install after fixing taht is Term::ReadKey
grand shadow
just noticed they are building with the fuckin linux compiler, will finish setting up perl modules tomorrow
prime juniper
Did you use Perl-cross? Cuz you should
I (tried to) disable locale stuff via perl-cross but tbh I marked it as broken a while back as it’s dogshit. I’ll put it on the list of things to fix
formal patio
considering perl is gonna block me self hosting I might just try to make it work...
I mean iirc locale was a problem, which I'm currently working on
I want you to be able to do locale_t caralho = newlocale(LC_ALL_MASK, "pt_BR"); and to have it work properly
grand shadow
Did you use Perl-cross? Cuz you should
Yeah
grand shadow
I want you to be able to do `locale_t caralho = newlocale(LC_ALL_MASK, "pt_BR");...
Hell yeah
grand shadow
perl is actually cancer to port though, holy fuck
still trying to figure out the proper perl Makefile.PL options to pass
prime juniper
It’s pain
grand shadow
okay I think I FINALLY got this working?
wait fuck
that x86_64-linux there
ughhh
what dir did I forget to set
its at least compiliong with the x86_64-astral-gcc which is good
oh right, perllocal.pod should not even be installed
funny
alright, time to see if this works
if it does I do not want to ever hear the words makemaker in my life ever again
why is perl-term-readkey tryign to
run itself under linux
aghfsdhgfsdjkhofkdhjkd
term::readkey is for 2048pl so I could just not do that and instead go straight for stuff autotools would want
but
slim garden
aghfsdhgfsdjkhofkdhjkd
what do you want to do with perl
grand shadow
what do you want to do with perl
automake and autoconf for self hosting
slim garden
automake and autoconf for self hosting
i thought they use c or c++
grand shadow
I wish
vim /usr/bin/automake in your system and see the magic
it is trying to generate itself using itself? what the fuck does ReadKey.pm.PL do
nothing that a light patching of that file wont help
mostly to just hardcode "#no nodelay" (mlibc doesnt seem to implement that) and poll
the fuck?
ah fuck it who needs man pages
wtf?
prime juniper
Skill issue, works on Managarm 
grand shadow
Skill issue, works on Managarm <:meme:575445094589661215>
how did you compile modules
using the cross-perl way or the makemaker way
prime juniper
Idfk it’s been years lmao. But works on whatever I have locally :meme:
grand shadow
grand shadow
I hate perl so fucking much
marsh sparrow
does ironclad package python packages?
grand shadow
I give up on 2048
I will try to see if the automake packages are as regarded
later
I undewrstand your pain now @prime juniper
perl is a mistake
perl-cross seems to have a modules thing, I will take a look at it later
prime juniper
tbf packaging python packages ain't much better either
I might’ve solved the issue
sage locust
does ironclad package python packages?
it has meson and ponysay
prime juniper
(Except for the python module that likes to combine python with rust, that’s still cancer)
See a soon to be opened bootstrap PR on how I do it
grand shadow
When solving perl
prime juniper
Perhaps soon
See https://github.com/managarm/bootstrap-managarm/pull/584 (I just noticed I have GHA failing so it’ll be until next week before I can fix that lmao)
@grand shadow
Also CC: @sage locust, this is how I’ll be packaging python modules
grand shadow
real
grand shadow
grand shadow
No work this weekend, spending time with bf
and I was also extremely frustrated with perl
grand shadow
okay, I have time now
will I erally bother with perl again
there is no escape, automake and autoconf need it
so if I want to be self hosting I need that
analog berry
Power through it
grand shadow
last issue I had was xs key mismatch
which means the perl running the thing =/= the perl it was built with
which is funny cuz
I cant build it using the perl it will run in... because it runs on another os :D
I think I might just try the perl-cross thing
instead of abusing makemaker
grand shadow
perl-cross is recognizing the modules in configure, time hook up the recipes for the module packages and see if that makes a diff
term-readkey doing its stupid fucking thing of trying to run code to check for stuff it should know from its own configure step
need to apply that patch manually now
grand shadow
seems to be working
the perl build system and its consequences
this is not how I wanted to have the perl modules packaged but this is a fight I cannot win (until someone smarter than me figures it out wink wink @prime juniper )
time to port libtool, pkgconfig, automake and autoconf
grand shadow
tomorrow I do the perl scripts
since I will also need to x compile some more perl modules for them
pine ledge
tomorrow I do the perl scripts
all my homies hate perl 🗣️
prime juniper
seems to be working
I see your 2048 and I recognize that mine is wayyyyyy simpler lmao
grand shadow
I see your 2048 and I recognize that mine is wayyyyyy simpler lmao
Oh lmao
I just went with the first 2048pl I found
The acidnik one
grand shadow
Wait holy fuck
Now that I have perl
I can port
The greatest program ever written
Cowsay
pine ledge
Cowsay
I had No idea that was written in Perl lol
Odd choice
Figured it was C or soemthing
grand shadow
After these few ports I will implement an iterative avl tree
Both for studying for my dsa class and for future astral plans
gritty shore
Real challenge is rbtree
AVL tree is pretty easy
Actually the real challenge is B+ tree
grand shadow
AVL tree is pretty easy
yeah
also wtf it doesnt need any third party extensions?
fucked up
formal patio
local man discovers the horrors of autotools
pine ledge
Real challenge is rbtree
Real shit
Implemented it in my own programming language and it was a pain in the asshole
grand shadow
currently running after a rename() deadlock
marsh sparrow
Actually the real challenge is B+ tree
imho a B+ tree is easier than an rbtree
and also, AVL and RB are similarly difficult
the extra implementation complexity of RB is negligible compared to the overall complexity of AVL so you might just go for RB
prime juniper
Yeah m4 ain’t an issue
gritty shore
imho a B+ tree is easier than an rbtree
Ah idk I just named a random thing that looked hard
gritty shore
the extra implementation complexity of RB is negligible compared to the overall ...
I think it's useful to have both
grand shadow
Later today I will fix the rename() deadlock and then continue the self hosting work
After being done with self hosting I will look into implementing more data structures for astral, its really lacking in that department
Which makes sense considering when I started work on it I wasnt even close to having a dsa class
fervent fjord
"Astral more like astrally retarded" - mintsuki, in bed, while cuddling and peeping my phone reading this chat
grand shadow
So true
prime juniper
Lmaooo
pine ledge
"Astral more like astrally retarded" - mintsuki, in bed, while cuddling and peep...
cute
are yall dating?
if you dont mind me asking
me and my girlfreind have been seeing this guy together
like not necessarily a romantic relationship all the way but it kinda is, we cuddle and watch movies and stuff with him
anyways just got super personal in the astral chat 🔥
grand shadow
I already have fvwm, I wonder if getting NsCDE working would be too difficult
An actual cde port seems cancer
proper meteor
I was struggling to figure out how to handle recursive shebangs in execve. So I look at how it's done in Astral and it turns out it's not handled 👍
(or I misread)
hearty fulcrum
lol
grand shadow
I was struggling to figure out how to handle recursive shebangs in execve. So I ...
it is handled
proper meteor
Terrible
grand shadow
there is a reason why syscall_execve calls into execve() :P
execve() is recursive with a max depth
sharp carbon
can't you just keep prepending the path to argv until there's no shebang? no recursion necessary
proper meteor
that's what I ended up doing
grand shadow
yeah thats smarter but I just wanted to get that done with when I made that
opal lotus
that's a good point idek if my stuff is recursive
yeah it is
@proper meteor if you want a reference
proper meteor
imma have a look
proper meteor
I have a doubt on whether I did it right
opal lotus
i probably did it wrong
proper meteor
right. then I'll write some scripts to check the exact result on linux and then I'll add it to my CI I guess
wait I cannot run bash on my kernel in my CI yet, fuck
guess I'll stash the scripts somewhere until I can do that
okay my shitty scripts indicate it works
proper meteor
https://github.com/menix-os/menix/blob/main/kernel/src/generic/vfs/exec/shebang....
My implementation for reference: https://github.com/maestro-os/maestro/blob/6b6118ae7580f49b105c02cde33c14719a9d3939/kernel/src/syscall/execve.rs
grand shadow
this is a very specific edge case involving directory hard links, oops
in which the directory hard link happened because of an improper ENOTEMPTY handling
grand shadow
yeah okay I do rename in a flawedish way I think so I will look into fixing that tomorrow
grand shadow
new rename stuff should be done, I will test them tomorrow and fix any issues that might arise
grand shadow
now debugging some filesystem corruption that only happens when running on qemu tcg
grand shadow
seem to have managed to trace it down to a page not being properly written back to disk
and it gets stuck with a dirty flag forever but never gets written out and so the flag never gets removed
it does still happen with smp disabled so thats less spooky
grand shadow
okay yeah something very spooky is going on
it seems like there was something mixing the standy and dirty page lists (since I use an union)
grand shadow
ok I fixed one bug
when I make the struct page use a union again for the standby and dirty one of the checks I added still fails
I'll go after that one tomorrow
grand shadow
there was no bug
I just put the test in the wrong place in this case :kekw
(talking about the new one
((the old one still was a bug
hearty fulcrum
there is no bug in ba-sing-se
grand shadow
still gotta do some more filesystem work before continuing the autoconf woes
grand shadow
been kinda busy and now I am doing some mlibc security work
grand shadow
okay thats merged now I gotta finish looking at that filesystem stuff, finish autoconf woes and implement AT_SECURE
grand shadow
trolled
I assume thats gnulib autoconf bullshit
might test it out with a better package to do this on
later, gotta do other stuff
grand shadow
I do need to patch config.guess lol
wgetting, autoreconfing, configuring, make -j5ing and make installing works, I am become autotools the gnuer of worlds
I now support all steps of building an autotools application
I support meson + ninja applications
I still need a cmake port
oh boy the writer thread seems to have deadlocked somewhere
aghjgfhjhghgfh
I'll look into that later, already messed with too much annoying filesystem stuff for now
oh fuck, xorg is not starting anymore
kerneç
analog berry
the kernec backtrace
bronze nexus
Error: memory is not kerneç
grand shadow
if I stub out the mlibc backtrace implementation this does not happen
uh oh
welp time to complain in #mlibc-dev
grand shadow
this is probably because I dont enable unwind stuff in libgcc
so I gotta do that later
sage locust
libunwind
see Gloire
actually no that's libatomic i got confused, though maybe it's enabled similarly(?)
neon crane
see this https://github.com/managarm/bootstrap-managarm/blob/master/patches/gcc/...
btw, whats with the September 17th 2001 date
From 59650138ac89368d1de499e79ee46c37ec7d0058 Mon Sep 17 00:00:00 2001
is it some default git date or what
i saw it too
sage locust
yes
it's just a marker date to show it's a git am patch
also, sadly can't enable this in Gloire since we don't have the Linux option
we'd have to write our own custom ironclad-unwind.h
neon crane
it's just a marker date to show it's a git am patch
I see
grand shadow
alright
tomorrow I think I will debug that spooky writer thread deadlock
been busy latel
y
then once that is done I maybe do a full rebuild cuz mlibc abi break and update the gcc patch to add the unwind stuff
grand shadow
@carmine swallow about the openjdk port you upstreamed, did it have jit enabled or not? and if it did how do I disable it? I'm gonna debug it later but it seems to die in mlibc futex stuff and I wanna see if its because its jit or because something else
marsh sparrow
mlibc futex stuff?
grand shadow
spooky backtrace
oh
I dont think it even runs java code
lock cmpxchg %r12d,(%rbx)
according to java rbx is the very frindly valie
0x0101010101010119
I assume 01010101010101 is a poison value?
unique_lock<Mutex> bucket_guard(bkt->bucket_mutex);
I am probably doing a fucky somewhere but where is the quesiton
grand shadow
hmmm
I suppose its time to update mlibc
and rebuild world cuz abi breaks
and yeah looking in the fishos thread it seems he ran into the same issue as me
carmine swallow
<@379250891838193667> about the openjdk port you upstreamed, did it have jit ena...
by default it has the jit enabled yeah, iirc it was something like --with-jvm-variants=zero to build the one that doesn't have it at all (though it might also have been some other options, idr)
grand shadow
by default it has the jit enabled yeah, iirc it was something like `--with-jvm-v...
thanks
deft oak
https://github.com/managarm/mlibc/commit/c32ace0500a6f4453402803abf0ed81187d6d95...
no this crash is the one i ran into after fixing that issue
i assumed i either had a bug in my kernel or the fix i pushed to mlibc caused it somehow
but astral having it proves its something else
(i gave up on debugging this right after so i dont have any idea what it could be)
grand shadow
(i gave up on debugging this right after so i dont have any idea what it could b...
how did you get it to work then
formal patio
it is kind of sus that it happens during matching of this stuff as well tho
deft oak
how did you get it to work then
the time i had openjdk working was with debug mode disabled in which case this doesnt happen somehow
deft oak
it is kind of sus that it happens during matching of this stuff as well tho
i tried replicating the sscanf's it was doing in the mlibc tests and there didnt seem to be any issues with it
formal patio
sounds like UB then
grand shadow
I dont think qwinci ran into this issue a year ago when porting it to astral so it might be a new regression
formal patio
git bisecting that is probably too painful tho
deft oak
I dont think qwinci ran into this issue a year ago when porting it to astral so ...
are you building openjdk with debug mode rn
grand shadow
idk
this is the flags that qwinci used
${source_dir}/configure \
--prefix=${prefix} \
--openjdk-target=x86_64-astral \
--disable-ccache \
--disable-precompiled-headers \
--disable-warnings-as-errors \
--enable-full-docs=no \
--with-boot-jdk="/usr/local" \
--with-freetype=system \
--with-libjpeg=system \
--with-libpng=system \
--with-zlib=bundled```@prime juniper could triplecheck that it is a mlibc issue by running it on managarm if it is upstream to bootstrap-managarm
grand shadow
the time i had openjdk working was with debug mode disabled in which case this d...
how did u do this btw, I tried adding --with-debug-level=release but it still happens
deft oak
how did u do this btw, I tried adding ``--with-debug-level=release`` but it stil...
release is the default already
idk why it didnt happen for me on release mode
i was also using a newer version of openjdk
jdk-17.0.17+7
grand shadow
it dies while it is going to append
mlibc::infoLogger() << "do_scanf: about to append [" << frg::endlog;
if(type == SCANF_TYPE_L)
append_to_wbuffer(c);
else
append_to_buffer(c);
mlibc::infoLogger() << "do_scanf: appended" << frg::endlog;
there must be some stack corruption here
there is no reason for that scanf to be calling into the allocator
on the append_to_buffer
since its just a %[]
bingo
this is the code at fault
awkward
I think fmts bigger than 128 are causing stack corruption?
I forgot if char is signed or unsigned by default
prints -2 on some random compiler I found online
awkward!
wild marsh
i love varargs argument type promotion!
wild marsh
I forgot if char is signed or unsigned by default
depends on the architecture
grand shadow
doing a reinterpret_cast<unsigned char *> fixes it
lol lmao lol lmao
mfw stack corruption in mlibc
can now run /usr/lib/jvm/java-17/bin/javac without that crashing
cc @deft oak
I'm gonna wrap this up and upstream it and run it against the mlibc tests to see if it does not break anything
grand shadow
alright, opened up the PR
grand shadow
hm
grand shadow
this is a symbol defined by libawt_xawt.so and it seems like it is loaded
so I am not sure wtf is happening
like I'm not going insane
I'm not sure I have enough of a phd to debug rtld
grand shadow
I added printfs to my libc log syscall to double check that it is the same process and it is
I am not sure what is going on here, I am not familiar at all with rtld internals
marsh sparrow
maybe it's loaded with RTLD_LOCAL?
deft oak
doing a ``reinterpret_cast<unsigned char *>`` fixes it
crazy
grand shadow
maybe it's loaded with RTLD_LOCAL?
Nope, its loaded with RTLD_GLOBAL
grand shadow
Im not sure if its dlclosed tho, I havent seen any calls to dlclose in the log
I dont see why it would be
Its loaded in the init code of another so iirc
Not by depenencies directly
marsh sparrow
we don't implement dlclose anyway, do we?
not sure why the symbol could be unresolved, maybe adding logs to the symbol resolution logic helps?
symbol resolution is mostly straightforward
grand shadow
Will do when home
grand shadow
update: managarm ran into that as well, so likely mlibc bug
fleet osprey
now run 1.21 or whatever the hell the latest version is these days 
split lance
that's so cool!!!
viscid flame
amazing
grand shadow
neon crane
this is so fking cool
tawny oak
yeah that is super cool, gj
viscid flame
did you tried to run forge with some mods
marsh sparrow
very nice progress
proper meteor
‼️
neon crane
"Runs on 3.000001 million devices"
Pretty sure the java tagline is 3 billion
willow rapids
Pretty sure the java tagline is 3 billion
Ah, then 3.000000001 billion devices
spare sky
niceeeeeeeeeeeeeeeeeeeee
prime juniper
Good stuff indeed. Good collab between Managarm and Astral here
prime juniper
did you tried to run forge with some mods
No lol this is alpha 1.2.0
grand shadow
Good stuff indeed. Good collab between Managarm and Astral here
Yes, minecraft also coming to a managarm near you soon™
viscid flame
No lol this is alpha 1.2.0
a
grand shadow
with 12 cores and LP_NUM_THREADS=12 its still very laggy
I havent been able to enable JIT java, it seems to hang
but I will wait until managarm gets minecraft running to see if they can get it running on JIT
the issue might be the interpreted java, since even with a super small window its still slow
cc @prime juniper
marsh sparrow
non-jit is definitely a problem
grand shadow
I might try to debug why jit hangs then
grand shadow
this was the last log from openjdk
analog berry
with 12 cores and LP_NUM_THREADS=12 its still very laggy
This would blow osdev reddit up
If it ran faster even more so
grand shadow
I think I might have to enable more verbosity
grand shadow
This would blow osdev reddit up
trust I'm gonna cook up a blog post and then post this on r/osdev if I can finish this jit debugging
analog berry
Huge
grand shadow
spooky
every thread is stuck on futex
with one cpu
hm
I wonder if
my futex timeout
is actually working
take tid 141
it has a timespec
but
that futex acquire happens like in the middle of the syscall log
so a lot lot lot happens after its acquired
I gotta figure out if it is actually a long wait or not and if not figure out why its not returning
carmine swallow
that's likely the same thing that I ran into at some point, idr if I figured any workaround for it or was it just that you restart it until it works
it might also be the same thing that caused the mc server to die with a watchdog timeout even in the cases when it initially booted
grand shadow
hmm
well good thing I checked that code anyways, noticed I was missing some usercopy handling there
and also I was supposed to return ETIMEDOUT on a time out but I was not returning that
I could probably rewrite that whole syscall at some point honestly
I dont think it plays nicely with CoW as well
like fork
oh yeah I see what is wrong
me when I want to sleep for one second and accidentally wake up 2 weeks later
crazies nap
wait no
I'm stupid nvm
read that wrong
ok I do think I know what is going on I just gotta fix it and see if that is really it or if I am hallucinating again
grand shadow
oookay
gritty shore
with 12 cores and LP_NUM_THREADS=12 its still very laggy
damn dont think ive ever seen somthing that cool
grand shadow
I am getting into the mojang logo with JIT
gritty shore
what was the fix?
grand shadow
my futex timeout handling was completely borked
grand shadow
so I just changed how the timeout of poll is handled systemwide (yes my futex timeout uses poll fight me)
I mean its on the menu screen but theres an infinite amount of syscall logging so I gotta reboot for perf
gritty shore
so I just changed how the timeout of poll is handled systemwide (yes my futex ti...
kinda cringe ngl 
gritty shore
I mean its on the menu screen but theres an infinite amount of syscall logging s...
inb4 it runs super smoothly
grand shadow
kinda cringe ngl <:nooo:750466870993223750>
my poll is generic enough as an event system that my event stuff is literally just a typedef and #defines to poll stuff
gritty shore
the thing is with a proper futex you can do a bunch of stuff that you cant do with poll
like not having to go into the kernel at all in the fast path
grand shadow
well thats inside the kernel not in mlibc
I mean like
I have a poll abstraction thats used by devices
and since it works well enough I also use it in the futex syscall
gritty shore
yea but futex is in userspace
grand shadow
to handle the event
grand shadow
yea but futex is in userspace
what
gritty shore
like not having to go into the kernel at all in the fast path
so this is handled somewhere in userspace
grand shadow
well yeah mlibc handles it
gritty shore
does it?
grand shadow
yes the sys_futex_wait is just an analogue call to the syscall(SYS_FUTEX, ...) from linux
gritty shore
sad
grand shadow
so JIT still doesnt work
grand shadow
hm futex timeouts STILL seem broken
unless
wait
ohhh
ohh....
I dont think I handle timeout being zero properly
which I assume it might be passing a zero timeout
grand shadow
ok yeah I checked and linux does return ETIMEDOUT immediatelly on a s = 0 ns = 0 timeout
while I didnt properly handle that I cannot think of any use case for something like this (just check the fucking value yourself blegh) so that might not be it
yeah that obv was not it
grand shadow
for some reason one of the futexes want to sleep for..... 24 days
I can only see one place n mlibc that calls sys_futex_wait with a timeout and I dont see how it would be an issue unless like my sys_clock_get is completely borked
and well, it seems that it does call clockget before sleeping
hearty fulcrum
trust I'm gonna cook up a blog post and then post this on r/osdev if I can finis...
I wouldn't be surprised if this got as much attention as Free95
but for good reason this time
grand shadow
if I get jit working blegh
so far the last syscall of threads is mostly just futex wait or nanosleep
it goes past doing xrandr stuff before dying
so lwjgl does get loaded and stuff
well not dying, hanging
I wish it were dying
it also seems to do shader cmpilation
could it be I am missing sysinfo and it would need it somehow? but I dont see why itd impact much since mlibc has fallbacks
I have tried verbose but I have not tried -Xlog yet
grand shadow
🤔
I wonder if it is trying to GC an absurd amount of memory or something and because of that it gets stuck in the gc or soemthing
ohhhh yeah it does a call to sysinfo
and I dont have that
so it is just
using random bullshit from stack there
which results in insane gc data
and all the therads are spinning in a safepoint
waiting for gc to complete
it all makes sense now I think I think
hm sysinfo shouldn't be too hard of a syscall, its just getting a bunch of random shit from around the kernel
and it seems like this code is only used on the hotspot which is the JIT jvm
grand shadow
soon astral will be hosting the osdev minecraft server and playing on it
I mean it probably could
qwinci was able to connect to a minecraft server running on astral at one point
before it died :P
TODO: implement sysinfo and see if that helps the JIT openjdk
got too distracted writing the blog post
analog berry
Were so close
marsh sparrow
did @prime juniper build with or without JIT on Managarm?
prime juniper
Probably without
I don’t see any clear clues in 3 minutes of looking on my phone
So I’ll need to check more in depth
marsh sparrow
i guess disabling the jit is done by --with-jvm-variants=zero, see #1061407633745125397 message
grand shadow
You can also disable it in runtime with -Xint
proper meteor
with 12 cores and LP_NUM_THREADS=12 its still very laggy
now run the latest version and go beat the ender dragon
analog berry
TODO: implement sysinfo and see if that helps the JIT openjdk
Any progress btw
grand shadow
Any progress btw
Been kinda busy
analog berry
grand shadow
I will look a bit more into it after making lunch
grand shadow
Reql
I also gotta do more of the blog post
I want to get it working with JIT and the blog post up before I post it on like r/osdev
viscid flame
proper meteor
Been kinda busy
don't
carmine swallow
I should get back to making my unix microkernel too so one day i can get to the stage astral is, the past month or whatever i have not been that active here and have mostly just worked on multiple rewrites of a compiler for a custom lang (ig if i get it to a decent stage once i could actually write the kernel in it)
grand shadow
would be cool to see it written in your language
grand shadow
I also gotta do more of the blog post
I finished a draft of the blog post, I'm gonna look into the JIT stuff now and finish polishing it up later
grand shadow
it seems to hang earlier with sysinfo() 💀
grand shadow
hm it still dies the same so that is likely not it
grand shadow
I've been mostly focused on the blog post but I'll debug more later
gritty shore
I've been mostly focused on the blog post but I'll debug more later
cmon you cant choke when youre this close
maybe try replicating the sequence of syscalls in another program?
grand shadow
I'm still not sure what it is doing other than its putting all of the threads in safepoints for vm cleanup. maybe some thread deadlocked and it cannot go into a safepoint or something
I know there is a thread holding a futex with a long long long delay which is weird but I'm not sure what it is or if this is the issue
formal patio
btw this page does not include the funni new post
grand shadow
btw this page does not include the funni new post
I know I still need to see if it needs any more polishing
I will post it here when I add it
gritty shore
Well I reread it and I think it is ready
does it work with the JIT?
grand shadow
does it work with the JIT?
Not yet
Im still trying to debug
Its running with trace again
grand shadow
Hmm I think Im looking into this the wrong way
The thread does print it is running
So something else could be at fault
I also need to enable the openjdk debug mode lol lmao oops forgot
regal apex
thats amazing
grand shadow
assert(sizeof(NULL) == sizeof(char*), "NULL must be same size as pointer");
something is defining NULL to 0 instead of (void *)0 or whatever
12 | # define NULL 0
|```
viscid flame
how do yall getting so much stars 😭
neon crane
Making good projects and promoting them in the right places, I guess
But actually I don't know why nanoshell is still getting stars
neon crane
I don't second this. I never posted on r/osdev
What did happen is I was featured in the old showcase channel, back in 2022
grand shadow
post it on r/osdev
I never did
When I do I will post the minecraft stuff and it will farm stars
grand shadow
Damn
viscid flame
When I do I will post the minecraft stuff and it will farm stars
at least astral must have more stars than free95
neon crane
at least astral must have more stars than free95
i cant believe that piece of shit has 720 stars
😭
how
viscid flame
i cant believe that piece of shit has 720 stars
it have half stars of managarm
prime juniper
i guess disabling the jit is done by `--with-jvm-variants=zero`, see https://dis...
We don’t pass that
marsh sparrow
then we're probably using JIT unless we pass -Xint
grand shadow
Indeed
JIT minecraft on astral works until at some point after initializing lwjgl
It does all the xrandr screen discovery stuff since it still had the logs I added from debugging mc
(and I also checked the syscall log to confirm)
analog berry
Btw what happens on newer Minecraft versions
proper meteor
it have half stars of managarm
I have a bit less than twice the amount of stars managarm has and my OS don't do shit 
hackernews-ed
grand shadow
Btw what happens on newer Minecraft versions
Never tried, we wanted to do 1.7.10 but that would need more libraries ported and so we did not do that
Dont see why it wouldnt work
proper meteor
gtnh would be funny af
I was talking to a friend about astral and he suggested I attempt gtnh in particular on my OS
grand shadow
I do see hotspot complaining about some missing sysdeps so I should implement that
more speifically, posix_fallocate and shmget
fallocate doesnt seem that nescessary and jdk doesnt complain about it failing
Hm I am not sure, looking at where they are called they dont seem nescessary for openjdk to work (and failure would cause openjdk to quit)
And no openjdk asserts trip before the bug it seems
wait wait wait it did quick when I enabled infinite logging
oh lol cuz I forgot to set DISPLAY
sharp carbon
Never tried, we wanted to do 1.7.10 but that would need more libraries ported an...
the only native library that was added between alpha 1.2.0 and release 1.7.10 was twitch-platform, and that one is not required for the game to work (and is in fact never even downloaded on linux)
and java-only libraries don't really need porting
lwjgl-platform and jinput-platform were updated between those two versions ofc but for both they're the only required native libraries
grand shadow
I remember trying to run 1.7.10 and it complained about a missing something, maybe I just forgot a lwjgl jar
marsh sparrow
i'm not sure if it applies to MC but java only libs can still require porting
for example, if they check for the current OS
or open files in /dev, /proc, /sys etc
grand shadow
it seems to get stuck here
which likely means some thread somewhere is stuck on something
grand shadow
dear lord I hope this is not common
hotspot seems to really like doing manual syscalls
formal patio
some linux stuff is simply not exposed via glibc, necessitating raw syscalls
although in this case it seems like this should be a POSIX thing?
sharp carbon
gettid isn't in posix
grand shadow
I mean it doesnt seem to call any invalid syscalls yet though
gritty shore
perhaps SYS_gettid is something else on astral tho
like
if it SYS_gettid is 1 and astral's open syscall is 1
then youre fucked
sharp carbon
the SYS_... definition is provided by libc (specifically sys/syscall.h) so if you don't have that header or it wants a syscall you don't have compile will fail
unless hotspot is doing something really stupid like keeping their own copy of syscall.h
gritty shore
Yeah that's true
carmine swallow
hotspot seems to really like doing manual syscalls
its not that many and I am pretty sure I patched them all away in the patch that I made (outside tests and one unreachable code path as astral implements sched_getcpu)
grand shadow
I was able to confirm it is stuck waiting for another thread to go into a safepoint
Just havent been able to confirm why the other thread does not
Cuz I am baking cake
marsh sparrow
why is syscall() not a compile time error on astral? or is it?
grand shadow
it is a compiler error, qwinci is right
he did patch them all away
there is a syscall() function but its locked behind an #include <astral/syscall.h>
I was looking at the openjdk source through github so I did not notice
grand shadow
Cuz I am baking cake
it took me almost 3 hours to bake this cake
hopefully its good
baking a pinneaple cake is a very frustrating process but pineapples are tasty
anyways
the last thing that thread was doing was just running code
not even in a syscall
the last syscall it did was a sys clockget and it was in the middle of the syscall log
now I just gotta figure out what the thread doing
I was gonna try to figure out how to get a log of all threads on openjdk n shit but I think theres a good probability that will be the only thread running on the system lmao
since it seems to be running and not doing syscalls
so I can just attach gdb to qemu
grand shadow
okay I have a pointer
I think its trying to infinitely run the test %eax,(%r10) instruction
r10 points to a memory range not executable, writable nor readable
nor present
I should look into the sigseggv handler to see if it needs siginfo si_addr or some shit
I dont think I implement it
and hotspot could need to use that
wait I do implement si_addr
now I am not sure if si_addr is the address of the instruction or an analogue to cr2
idk is openjdk is insane enough to do something like an mmap on a sigsegv
gold
who are you?
assert(info != NULL && ucVoid != NULL, "sanity");
sanity
yeah si_addr on a sigsegv has the cr2
there was a file mapped on that range
hm
wait nvm the vmm_file flag is not set
grand shadow
accidentally ran rebuild openjdk17 after adding a few printfs
insert openjdk compiling xkcd
grand shadow
bruhhh
this is wrong
YESSS
MINECRAFT IS WORKING ON JIT JAVA
AKSDKDSGLSDL
IM SO FUCKING GOOD
fleet osprey
holy shit 🔥
grand shadow
cc @prime juniper
bright kernel
hopefully its good
was the cake good
grand shadow
it did run into a weird error
when loading in a world
same one as it hit once on non jit minecraft
grand shadow
Cannot load from short array because "sun.awt.FontConfiguration.head" is null
at java.desktop/sun.awt.FontConfiguration.getVersion(FontConfiguration.java:1262)
bright kernel
yeah for me too (🇧🇷)
bright kernel
oh wait you are brazilian
yeah XD
grand shadow
I can kind of see an improvement but its on slowdebug
I gotta enable release again
or at least fastdebug
grand shadow
I can kind of see an improvement but its on slowdebug
(wrt generating the world)
huh
I was able to get in but it doesnt let me get in
and its all weird
I can control the menu just fine
this is weird
reloading the world I got placed in a better spot but something still seems wrong
the fog is all black
and I still cannot leave the menu
while the game seems much faster my work is still not done
this should be null as well
I assume some exception is happening while minecraft wants to close the menu which is causing it to give up on that?
running again the game worked...?
I'm gonna enable fastdebug on openjdk, I do wanna see how fast this can go
surreal wraith
Bro, this is impressive! It looks so cool!
grand shadow
opened first try without issues now, performance still not very good
using this on touchpad sucks
tomorrow I'm gonna do it on mouse
its not COMPLETELY unplayable its just not enjoyable
and its mostly rendering atp, looking up the game gets much faster
even with LP_NUM_THREADS=8 and 8 cores
tomorrow I`m gonna try to record it and do it on mouse and maybe increase the number of cores
and then post it on r/osdev and farm ez karma
I did not survive :(
I wonder if this runs on real hw as well and how faster
pls work
fuuuu
hearty fulcrum
oops
marsh sparrow
how many fps do you get now with JIT?
prime juniper
Nice stuff (taking mental note of what to update in my patch set before merging)
grand shadow
how many fps do you get now with JIT?
At one point I pressed f3 and it showed like 7-13 fps
analog berry
Any idea why its so slow?
willow rapids
what rendering backend are you using? llvmpipe?
wild marsh
even with llvmpipe you should be getting better fps
on my laptop in alpha 1.2.0 i get 20-30 fps with llvmpipe (via LIBGL_ALWAYS_SOFTWARE=true)
with a nearly 1080p window
marsh sparrow
I'm wondering how fast I'll be on Managarm
analog berry
on my laptop in alpha 1.2.0 i get 20-30 fps with llvmpipe (via `LIBGL_ALWAYS_SOF...
thats like only 2x from astral tbh
using very fast syscalls on linux
and i doubt astral has gettimeofday as a vdso syscall etc
or maybe it does idk
deft oak
thats like only 2x from astral tbh
thats a 1080p window though. on my linux desktop in alpha 1.2.0 with llvmpipe using the same default small window size as astral i get 390 fps when using all 12 cores, 130 fps when i let it only use 1 core (with taskset)
deft oak
no it doesnt
analog berry
where does it get the buffer to draw to?
deft oak
it just renders as a regular x11 window
analog berry
for some reason i assumed mesa required at least the bare minimum drm
but guess not which is nice
prime juniper
Time to ping leo on an ETA for 3D accel nvidia graphics and time to go shopping for 16xx series or up cards
formal patio
dunno what the ETA is ngl
the problem with running minecraft on accelerated stuff will be the fact that is uses OpenGL
which is a problem bc running zink on the NVK-on-nvo thing seems to be broken sometimes
prime juniper
:(
gritty shore
using very fast syscalls on linux
Idk if the problem is doing syscalls, I feel like the problem most likely has to do with the implementation of syscalls
analog berry
thats what i meant
gritty shore
I'd strace it and look at the most common syscalls
marsh sparrow
maybe astral doesn't have AVX and similar?
sage locust
llvmpipe should not be that slow
that said at some point not long ago we updated llvm+mesa in Gloire and it did almost like double or quadruple fps in glxgears lol
maybe you're using ancient versions
i think i tried playing minecraft with llvmpipe on my machine once (on Linux obviously) and it worked just fine, it was completely playable
neon crane
opened first try without issues now, performance still not very good
i love how the kernec thing is still there
neon crane
opened first try without issues now, performance still not very good
try reducing the window size again and see if its CPU bound or "GPU" bound
grand shadow
and i doubt astral has gettimeofday as a vdso syscall etc
I do not
grand shadow
try reducing the window size again and see if its CPU bound or "GPU" bound
Even reducing window its still kind of slow
It was getting that fps with a full window
So it likely is cpu bound
grand shadow
that said at some point not long ago we updated llvm+mesa in Gloire and it did a...
I am still running mesa 24.2.0 and llvm 18.1.7
grand shadow
I dont have the time to work on the kernel and port stuff and also update everything 😭
grand shadow
we have the latest of both in Gloire
What version gave that speedup?
I mean it might not even help since its likely something being slow in a syscall
grand shadow
Oh lmao
grand shadow
maybe astral doesn't have AVX and similar?
I dont
analog berry
Its like one register write to enable 💀
marsh sparrow
not really, right? you need some logic for xsave
it's not hard or involved or anything but it's more than one line
analog berry
not really, right? you need some logic for xsave
What sort of logic? U just use xsave
marsh sparrow
you need to enumerate it + enumerate the size of the xsave region
actually allocate it and context switch it
analog berry
I guess, two extra cpuid calls
If u have logic for the fpu save restore u just keep it
Like astral supports fpu atm im pretty sure lol
wraith surge
yeah, it's not that hard to enable it
grand shadow
Would it even have that big of a perf improveme t
wraith surge
yes
sage locust
more like
you should enable it because it's been a thing for many years and software that can take advantage of it will get a speed boost
grand shadow
well added to the todolist then
wraith surge
well added to the todolist then
tbh it's just a matter to allocate the context and the save / load it during ctx switches
grand shadow
its time for a new github image and I think I'm gonna use this as one
sage locust
fastfetch wehn
grand shadow
fastfetch wehn
how easy/difficult is it to port
neofetch is piss easy itos just a huge bash script
sage locust
how easy/difficult is it to port
just check Gloire
it's actually easier than neofetch i'd say
grand shadow
I will get an endless flux of stars, get one million stars and get paid five brazillion dollars to do osdev real
anyway anyhow, I really gotta get going at my infinite todolist of astral fixes and improvements
in it is included updating mlibc which means an entire world rebuild :D :D :D :D :D :D :DDDDDDDDDDDDDDDDDD
and also help dennis get mc on managarm too
and also do uni shit which I have been neglecting for a bit (exam this week 👻)
regal apex
I will get an endless flux of stars, get one million stars and get paid five bra...
5 brazillian dollars?! OMG!1!!1
prime juniper
in it is included updating mlibc which means an entire world rebuild :D :D :D :D...
Again recommending holding on mlibc until newer openjdk drops (and that requires debugging font aids, we keep balling in the evening)
analog berry
https://www.reddit.com/r/osdev/comments/1on20n8/running_minecraft_on_my_hobby_os...
You should repost to the new osdev reddit and also hackernews
It's guaranteed to blow up
viscid flame
https://www.reddit.com/r/osdev/comments/1on20n8/running_minecraft_on_my_hobby_os...
150 upvotes amazinf
hearty fulcrum
You should repost to the new osdev reddit and also hackernews
there is another?
analog berry
kerneldevelopment or smth
prime juniper
It’s already there
grand shadow
You should repost to the new osdev reddit and also hackernews
Idk how to post stuff to hn
proper meteor
Idk how to post stuff to hn
create an account and post
would you like me to post it to https://lobste.rs/ ?
I have an account there
proper meteor
You can post it idm
do you want to write the content or do you want me to do it?
grand shadow
do you want to write the content or do you want me to do it?
I am on my phone on the bus so you can do it :P
proper meteor
will do
grand shadow
Thanks
hearty fulcrum
I went to try to make a hn account and it does not like me
usually a reload fixes that
maybe it's some kind of anti crawler thingy :P
formal patio
Hacker News
prime geyser
its time for a new github image and I think I'm gonna use this as one
oh my
Astral sure looks like a fun thing to ~~steal ~~ get inspiration from
Not as militantly c++ as managarm
proper meteor
do you want to write the content or do you want me to do it?
Terrible website
grand shadow
💀
grand shadow
there seems to have been a regression in fd passing so I gotta debug that
grand shadow
fixed up
next thing will be implementing writev and readv
I already use iovec for all internal i/o so it will be very simple
grand shadow
found a screenshot of astral from like 3 years ago (sep 2022)
sometimes I miss the old limine tty backgrounds
I should add an ioctl or something that gets a raw image loaded by the userspace app and resets flanterm to use it
old astral looked more cozy in a way
probably the warmer colours
also what the fuck is a pfetch
lmao
dapper ember
I have no idea what your talking about, but it's kinda inspiring to see projects like this
neon crane
I should add an ioctl or something that gets a raw image loaded by the userspace...
Or just implement your terminal in userspace
How about that
neon crane
Or just implement your terminal in userspace
And the only thing you leave implemented in the kernel is the TTY connection support
viscid flame
Or just implement your terminal in userspace
userspace tty is my beloved
viscid flame
I should add an ioctl or something that gets a raw image loaded by the userspace...
or just load background image with module or load in initrd and parse it with stb image
prime geyser
Does astral have less performance issues with Minecraft when running headless?
thinking about setting up a Minecraft server with it for silly reasons
giving someone ssh credentials to the server and watching them be like "what the actual fuck"
sage locust
i wanted to add a flanterm request to Limine
that passes all the Flanterm-specific parameters used by the bootloader to the kernel
but i was told "don't, that's stupid" blah blah
it would pass the wallpaper canvas with alpha blending already done and everything plus all the other stuff that Flanterm's init function needs
analog berry
but i was told "don't, that's stupid" blah blah
just add it
if you think it can make the experience better why not
grand shadow
Or just implement your terminal in userspace
Nah I want kernel tty
Kernel shell and kernel gui 🤑
grand shadow
oberrow ssh'd from obos to astral
grand shadow
I really still gotta debug that stupid fuckin writer thread bullshit but I hate debugging that shit
I have a long todolist coming back from like 2024 that I have a lot of stuff to sift through still
which funnily enough
I wanted to look into the sortix os-test thing like a year ago
grand shadow
I do want to do that at some point
but I am busy doing something super importnat
fucking around with random x11 applkications I found in a random website
I should add that xsnow as a package to astral and have it run every 15 minutes for a minute during christmas
gritty shore
fucking around with random x11 applkications I found in a random website
port neko
would be fun
grand shadow
I have xneko but its one thats stuck to a window and doesnt move around the screen
gritty shore
make it so its moving on the screen
gritty shore
yeah but fix it so it actually works
grand shadow
the port I haveis the original source from 1990 💀 the comments are all unreadable cuz japanese
UNIX 4.3BSD
anyone who speaks japanese knows whats writen here? 
gritty shore
it's japanese? damn
I knew neko meant cat in japanese but didnt know it was made by a japanese person
grand shadow
holy fuck thats an insane path
grand shadow
I still have a dream of playing cogmind on wine and jumpscaring the dev with some random fucked up crash reports
and speaking of roguelikes, I wanna port nethack at some point
cant be an unix without a classic unix game
I have brogue but it does not start because of a missing tdelete() in mlibc
maybe I should implement that at some point
hmm yeah the tree functions seem to operate in an avl tree, should be easy to implement
gonna do that tomorrow, still gotta finish studying for that DSA exam
@prime juniper I saw that pr saying that tdelete is needed for vlc so you are probably interested in this
so I will implement tdelete and twalk for you
(complain at me if I dont )
gritty shore
hmm yeah the tree functions seem to operate in an avl tree, should be easy to im...
Where do you see this?
I just think they work on BSTs
Not necessarily avl
Also it's crazy how there are so many random libc features I've never heard of
grand shadow
mlibc implements it as an avl from what I can see
the spec itself does not say if it has to be balanced or unbalanced, just that it has to be a bst iirc
gritty shore
use frigg rbtree 
grand shadow
I think one of the big libcs implement it as a rbtree?
I remember seeing that once but I could be really misremebmering that
gritty shore
Well it would work
I don't see why not
I'd just do a dead simple BST tho considering no one uses these functions
or use my already existing AVL or rbtree implementation
grand shadow
well there is an already existing implementation for most of the tree functions
its just twalk and tdelete that are missing
so I'm just gonna build on top of that
gritty shore
ahh then those will be pretty simple
tdelete not really unless there's AVL delete already implemented
grand shadow
its not but whatever, its not difficult
gritty shore
It's a bit tricky
deletion is trickier than insertion
To me at least, but it's manageable
grand shadow
I think one of the big libcs implement it as a rbtree?
glibc does so yeah
grand shadow
deletion is trickier than insertion
I mean yeah
prime juniper
<@261497806307459072> I saw that pr saying that tdelete is needed for vlc so you...
Pog
grand shadow
from 250 starts to 300 stars in a few days
grand shadow
guess I am just not lucky
or a lot of people who saw it already starred
or somethying
though yeah
1000 upvotes over two posts to 50 stars is kinda weird
sage locust
big osdev keeping you down
grand shadow
alright, I have fixed the avl tree (it was not balancing properly) and added an implementation of twalk and tdelete that passes tests
I will just take a final look at it later before opening a pr
hearty fulcrum
1000 upvotes over two posts to 50 stars is kinda weird
you should have claimed full windows compatibility
gritty shore
1000 upvotes over two posts to 50 stars is kinda weird
Your HN post wasn't good maybe?
Try using Show HN:
So like "Show HN: My hobby OS that runs Minecraft" or something
gritty shore
Your HN post wasn't good maybe?
It barely got attention
analog berry
Nah I think u just got unlucky
