#Astral

1 messages · Page 24 of 1

grand shadow
#

Yeah then I can make a flamegraph with it

bronze nexus
#

flamegraphs are cool

grand shadow
#

unrelated at all to my recent work but I should port an irc client

grand shadow
#

I got the first hex dump out of the profiler :D and its absolute garbage, gotta figure out what is going on

#

garbage as in, not even in the right format

#

much less having any useful data

#

bro wtf the nmi never actually writes to the ringbuffer 💀 where is it getting its data from

#

what was it cooking

#

wait fuuuck I forgot to disable the limine serial

dense valve
#

@grand shadow I have question, do you use serial or framebuffer to log? Both now and when you've just started. Thanks in advance, please answer TvT

grand shadow
#

I do support serial but I dont log to it

neon crane
#

I don't log much to the screen

#

Mostly to e9

prime juniper
#

hexchat too, but unmaintained, deprecated and gtk2 only

#

assuming you want GUI ones

night trellis
#

@grand shadow have you built an iso bexause i am not building astral's gcc

#

i desperately wanna contribute to this

robust geyser
#

that's gonna be hard considering you need gcc to build anything for astral

#

but refer to this, i guess: <#1061407633745125397 message>

#

i don't know how out-of-date this is btw

grand shadow
#

should be mostly up to date

#

But yeah, youll need the astral gcc

grand shadow
#

currently having exams to study for so things will keep being very slow for a bit

#

but after like the 8th (and after I finish one project... its a stupid one to write some software for dos using x86 assembly and masm) we ball

#

summer break will be upon me

proper meteor
prime mulch
#

no its actually just him

#

his hair grew backwards in the womb so he experiences the seasons in reverse

proper meteor
#

bro's heating his house at 35c during winter

night trellis
#

guess like ill have to compile then

grand shadow
#

its not that bad

#

unless you are like on a cpu thats 20 years old

#

which then good luck and I am sorry for your soul

#

#error "Here's a nickel kid. Go buy yourself a real computer."

fleet osprey
#

wtf

#

where does that even come from

grand shadow
fleet osprey
#

of course it's Linux lmao

dense valve
#

Lmao, wtf lol

prime mulch
#

it's a reference to a dilbert strip from the 90s which btw the author Scott Adams went down an insane conservative pipeline triggered by an extreme love for Trump and got his comic strip cancelled by saying a bunch of racist and sexist stuff

#

He's like stonetoss for boomers

neon crane
#

linus torvalds is gen x so kinda sorta checks out

prime mulch
#

I knew that particular strip because it was on the inside of the back cover of a big book about Unix that my dad had that I found when I was little lol

grand shadow
#

I will do a mlibc update either today or tomorrow

#

this will be a bit funky since I'll need to:

  • fix any abi breakages in kernel
  • rebuild the world
  • solve build issues that arise from the ongoing mlibc namespace pollution changes
dense valve
scenic epoch
#

before the gdt was created

#

before the 8086

#

well into the stone age

dense valve
#

We'll be Dr. Stone with this one 🔥

#

No, Dev. Stone. 🔥🪨

scenic epoch
#

make the flintstones know of the mlibc update

night trellis
#

10th hen

#

gen*

#

so not that old ig

#

from 2020 since its a 10th gen

dense valve
#

me too, but mine are i5

grand shadow
#

we flamin

#

I might clean up the work I already have and push it so I can work on updating mlibc

#

no amd pmc but whatevs thats just hooking it up to the infrastructure

#

theres still some things to fix (like that syscall entry thing being separate

#

though there I can already see it spends a LOT of time on a munmap while running glxgears

#

(and thats with one core!)

grand shadow
#

fun thing is you might even be able to run the fucking flamegraph script inside astral

#

since astral has perl

#

and python

marsh sparrow
#

very nice

#

the flamegraph is very cool

#

I should extend Managarm's profiler to do that as well

#

Now that Astral has stolen Managarm's PMC code, I can steal Astral's flamegraph gen meme

grand shadow
#

so true

#

its a python script that just reads the binary file that is in a

number of pointers (1 byte)
pointers (8 bytes each)
...

format
and shoves all of them into addr2line, puts that into the format flamegraph likes and shoves it into flamegraph

#

later I can get a longer flamegraph of glxgears and see if I can see anything (as well as fix some of the weird symbol naming) but for now I gotta get the ball rolling on the mlibc update

scenic epoch
#

#RunTheFlamegraphScriptOnAstral

grand shadow
scenic epoch
grand shadow
#

damn links doesnt support svg 💔

scenic epoch
#

i feel like running this on a real computer for some reason (i did it with some other os already)

#

depends on the system requirements i dont want to sacrifice my actual pc for this

grand shadow
#

beware the spooky ps/2 driver that does not work

scenic epoch
#

what driver does work

grand shadow
#

abandon all hope ye who enter here

scenic epoch
#

none 😔

grand shadow
scenic epoch
grand shadow
#

well wanna know a fun thing

scenic epoch
#

interrupts or smth???

grand shadow
#

on my machines it works

#

its just on LITERALLY EVERYONE ELSE'S THAT IT DOES NOT

#

so I cant debug this shit

scenic epoch
#

buhr

#

does it work on qemu

grand shadow
#

yeah

scenic epoch
#

idk what machines you have but i hope i get lucky bcs im gonna do this on a laptop and i assume its gonna emulate ps/2

#

1.8 gb root image 😔

grand shadow
#

that wont work for booting on real hw

scenic epoch
#

yeah ik

grand shadow
#

you'll need the initrd iso

#

the root is just the filesystem for it to mount into

#

I need to start shipping unified images ☠️

scenic epoch
#

yeah i read the qemu command thing its the hard drive crap

scenic epoch
#

emulation time™

#

UPDATE YOUR LIMINE

grand shadow
scenic epoch
#

it mightve killed itself a little

#

the next line didnt screenshot it says timeout

grand shadow
#

u running without the root disk

#

the initrd minimal thing is minimal

scenic epoch
#

is command in the github evil

scenic epoch
#

im not running the initrd thing tho im running the top one i assume its what im supposed to boot

#

talking about the boot screen

#

did i not mount smth

grand shadow
#

oh right, I forgot that now I literally only ship the minimal distro

#

you'll have to install stuff

#

xbps-install xorg-server etc etc etc

grand shadow
#

xbps-install -S xorg-server fvwm3 xterm etcetcetcetc

#

run netd vionet0 btw

#

to start the networking

scenic epoch
#

ohok

#

sudo doesnt seem to work by default do i have to install it

#

it seems to be there its just not owned by "uid 0" (root?)

grand shadow
#

wait who is it owned by

scenic epoch
#

eror™

grand shadow
#

well that was supposed to be ran by the init hook

grand shadow
#

did you by any chance close the vm without syncing the disk?

#

or running poweroff

scenic epoch
#

yes

grand shadow
scenic epoch
#

ok i guess thats the problem nooo

#

how to fix

grand shadow
#

get a new image

#

or if u dont want just set it to suid but you might get spooky filesystem errors

scenic epoch
#

bugstral

grand shadow
#

since ext2 is very corruption prone on dirty shutdowns

scenic epoch
scenic epoch
#

i redownloaded image and it still dies

grand shadow
#

does it say anything about running the hooks

#

because like, it works on my machine™

scenic epoch
#

ykw im running it from root MUAHAHAHAHAHAAHAHA

#

logout sharted itself

#

acpi shutdown time

grand shadow
#

you sure you not running the same image? it is literally supposed to do that on the first boot

grand shadow
#

I'm downloading the github image to see if that happens on my end

scenic epoch
#

idk its on yo github

willow rapids
#

november 3rd is quite a while ago

grand shadow
#

I think I fucked up and have a bad image uploaded (I think I opened up it for testing and forgot to copy a new one)

scenic epoch
#

oh

grand shadow
grand shadow
#

I think its the same 🥀

#

I can upload a new one rq

scenic epoch
#

tfw developer no test thing 😔

#

(and or dementia)

grand shadow
#

I think I just opened it and closed without syncing

#

which is funny becaues thats stupid

scenic epoch
grand shadow
#

yeah

scenic epoch
grand shadow
scenic epoch
analog berry
grand shadow
# analog berry how did u implement it?

use pmc to get an nmi every set amount of tsc cycles -> record backtrace onto a ringbuffer that is shared between a per-cpu kernel thread and the nmi -> these threads funnel it into a single global ringbuffer which is exposed to userspace in /dev/prof -> the astral-prof program opens it, discards what was there and starts copying data from it into a specified location (be it a file or /dev/com0 or whatever) -> you can now use a python script to generate the flamegraph from that profiler data

analog berry
#

daamn

scenic epoch
#

what???

grand shadow
#

very inspired by the mangarm profiler

grand shadow
#

the image I uploaded is 1.9gb

scenic epoch
#

evil

#

ok now it says 1.99

#

safari crash out

#

it blackscreens

#

no booting lololol

grand shadow
#

works on my end

grand shadow
#

I'll add e9 logging to the bootdisk hold on

scenic epoch
#

k

fleet osprey
scenic epoch
prime geyser
grand shadow
scenic epoch
#

downloading

#

e9 is serial?

grand shadow
#

its debugcon

scenic epoch
#

i forgot thats the qemu one

#

ok so pure silence

#

literally nothing

#

ok im gonna go get my linux pc out

#

the mac gods dont like me rn

grand shadow
#

not even limine?

scenic epoch
#

everything else sharts itself

grand shadow
#

nothing on e9 is crazy tho

#

stuff gets printed on e9 like first thing that happens almost

scenic epoch
#

shrug

grand shadow
#

maybe the version of limine I was on didnt play well with it or something

prime juniper
grand shadow
#

anyway, mlibc bump and world rebuild ongoing

#

hopefully not too much shit I have to fix considering the infinity abi breaks and pollution fixes

grand shadow
#

super close to hitting 30k loc

#

oh fuck I gotta update gcc so I can update mlibc

#

thats fun

fleet osprey
#

Btw how architecture (and notably bootloader)-agnostic is Astral

#

I'll be getting some time soon and I might do some work on that PPC port that you hinted at wanting to do sometime Soon™

#

But if it's still heavily tied to x86_64+Limine then I'll probably hold off until it would be less painful

grand shadow
#

isnt ppc the one with like inverse page tables or some shsit like that

fleet osprey
#

Yes but there's also the BATs that suck less

#

But you get way less of them

grand shadow
#

as much as I have tried to be arch agnostic astral really likes running with an hhdm and there are probably many x86isms that need to be ironed out

#

and theres no bootloader abstraction

fleet osprey
#

Oof

#

I'll probably hold off on it then until some of those are ironed out

grand shadow
#

yeah maybe until I already have it ported to another arch like riscv64 or aarch64

fleet osprey
#

Oh yeah that'll likely entail DT parsing which would be a huge help if it already existed

grand shadow
#

oh god thsi is gonna fucking suuuck

#

gcc 15 using std23 by default is already causing build issues and I am not even 2% done

#

🥀

#

if I keep getting packages broken becauaze of this I am dropping to gcc 14

#

yeah no I am dropping to gcc 14 I cba

#

its either adding -std=gnu17 to almost every package or updating almost every package

grand shadow
#

ok, have gotten a minimal distro to build with gcc 14 and latest mlibc

#

time for the rest of the owl

#

xorg is down

#

codeberg and freedesktop websites when it comes to downtime

midnight stone
#

How long does it take to compiler

robust geyser
#

depends on your computer

#

on a high end machine it'll probably take a couple hours

#

of course if we're talking about building a full distro

night trellis
#

building astral

#

wish me luck

#

i kinda want to make an astral distro

#

like a lightweight one that doesnt take a bazillion years to compile

#

just gcc, kernel and userland

#

and probably pacman for package management

#

damn it only took a few minutes for me what

#

anyways what do i do now>

#

oh yeah

#

oop nvm

#

still needs to compile perl

#

do i have to shell hop again

grand shadow
#

Oh yeah edit the perl recipe to remove that bash call

#

I forgot it there 💀

#

I recommend waiting a bit to do a full world rebuild

#

I am working on updating mlibc

#

And so when I do youll have to rebuild everything

analog berry
#

@grand shadow btw did profiling reveal any bottlenecks

grand shadow
#

Havent taken a look yet, Im gonna do that once I finish updating mlibc

#

But from that first flamegraph u can see that munmap takes a ridiculous amount of time

analog berry
grand shadow
#

mesa probably does tho

#

thats a glxgears flamegraph

analog berry
#

iirc its the glxgears itself spamming it

grand shadow
#

oh bruh

analog berry
#

do u profile across multiple cpus or how does it work btw

#

like is it a kernel wide profile or process wide

grand shadow
#

kernel-wide all cpus

#

I could maybe get away with telling it to profile only a process

#

but for now this works

sharp carbon
#

plus some time accounting and the X event handling

analog berry
#

what does glCallList do? Looks like it references some predefined list

#

anyway i hope whatever minecraft is doing doesnt result in a mmap/munmap per frame

#

like even linux cant do more than ~50k unmaps a second

sharp carbon
analog berry
#

ah so kinda like vbos etc

sharp carbon
#

yea

sharp carbon
analog berry
#

wait really?

sharp carbon
#

yeah

analog berry
#

damn

sharp carbon
#

even with the "use vbos" option enabled, it makes a new vbo each frame, draws it once, then destroys it immediately

analog berry
#

how do they manage to use custom shaders and stuff if they use immediate mode?

grand shadow
sharp carbon
#

but it's been a while since i looked at that part of the old mc codebase so i could be wrong

analog berry
#

thats crazy inefficient

sharp carbon
#

to be clear i'm talking about 1.8 and the like

#

in proper modern versions they use actual decent rendering techniques

analog berry
#

i guess notch wasnt that great at graphics programming KEKW

marsh sparrow
# analog berry like even linux cant do more than ~50k unmaps a second

another Managarm win meme

ci-boot.py[63]: stdout: memory mapping, size = 1 MiB
ci-boot.py[63]: stdout:     134676 iterations per second
ci-boot.py[63]: stdout:     134743 iterations per second
ci-boot.py[63]: stdout:     134529 iterations per second
ci-boot.py[63]: stdout:     135607 iterations per second
ci-boot.py[63]: stdout:     135869 iterations per second
ci-boot.py[63]: stdout:     avg: 135084, std: 544
willow rapids
analog berry
#

Damn

night trellis
#

whats managarm

marsh sparrow
grand shadow
#

new mlibc world build is up

#

cc @night trellis

night trellis
#

tysm

grand shadow
#

flamegraph of running minecraft on astral

#

why the fuck is the page fault handler taking that much even tho vmm_pagefault is a tiny spec

#

a better question would be

#

why is it running after interrupt_set?

#

ah wait

#

the stack contains the thing AFTER the call

#

but still

#

I should fix that and have it point to the call

#

ohhh I dont add the rip into the log

#

me when I am me when me

grand shadow
#

well shit

#

I think its time for me to stop being lazy and give astral an actual non shit vmm datastructure

#

me when linked list vmm

#

guess sooner rather than later I gotta implement rbtrees

#

wait why is vmm_getrange under pfisr

#

think I'm missing a call somewhere

#

still, that is a looong ass time in getrange

#

I need a real data structure for this

#

I could have an avl tree which is piss easy to implement but thats boring and rbtrees are cooler

#

(plus its what everyone uses so its probably better)

#

cc @analog berry since u wanted to know what was bottlenecking mc so much

#

oh yeah I forgot to + 1 the backtrace size

analog berry
#

So basically bad vmm?

grand shadow
#

pretty much

#

it spends most of the time in getrange()

#

which just iterates over the linked list

analog berry
#

Yeah thats pretty bad

grand shadow
#

since I'm gonna be on a big break soon I can probably get a rbtree implementation working before the year ends

#

that should at least double performance maybe triple

bronze nexus
grand shadow
#

getting the range of an address

bronze nexus
#

ah

grand shadow
#

oh it gets inlined that explains it

bronze nexus
#

So you're saying it might not actually be getrange?

grand shadow
#

no it is getrange

#

the stack trace just shows pfisr because its still with the vmm_pagefault rbp

bronze nexus
#

Ah, the second graph would be inlining disabled.

grand shadow
#

but its not because it gets inlined

bronze nexus
#

I'm gonna try astral at some point, lmk if you want to see it running off a USB on my framework laptop ;)

grand shadow
#

bet, maybe at one point when I get ps/2 fixed for real

bronze nexus
#

I guess that's a good way to check your trackpad support

bronze nexus
grand shadow
#

oh

#

that will take a while

#

I do not have anything usb at all

bronze nexus
#

lol

#

I can do you one of my old PCs then, will take me a bit to find PS2 mouse and keeb though

#

I'd try running it on my main if I didn't have the principle of never running a hobby OS on something I don't want filesystems corrupted on :P

#

I'm off now. Keep up the good work.

gritty shore
#

But rbtrees are better in general

grand shadow
#

yeah

grand shadow
deft oak
robust geyser
#

or… try using a data structure better suited for frequent lookups

#

better than caching linked list lookup results

marsh sparrow
#

AVL vs rb is not that important

robust geyser
#

anything but a linked list tho

flat copper
#

left-leaning rb trees are much simpler than avl

deft oak
#

but caching should still help

#

since its quite common to lookup the same range over and over

grand shadow
#

I plan on writing a red-black tree implementation

dense valve
dense valve
wraith surge
grand shadow
# wraith surge how did you get this flamegraph?

use PMC to generate a nmi every X tsc cycles
that nmi takes a backtrace and stores it in a per cpu ringbuffer
a per cpu thread copies the data from that per cpu ringbuffer into a global ringbuffer
that global ringbuffer is exposed to userspace in /dev/prof
there is a userspace utility that is used to copy the data, discard old data etc etc and copy it into a file (could be a file or say /dev/com0)
then there is a python script that takes that file, puts it in a format flamegraph wants and generates a flamegraph out of it

wraith surge
#

oh ok

#

thx

neon crane
wraith surge
#

i'm using a linked list too for vmm

#

lol

grand shadow
opal lotus
neon crane
#

VA allocation I guess?

opal lotus
#

yes

neon crane
#

i see

#

if you have LA57 support thats probably fine

winged basin
#

Its surprisingly common to use a linked list for map entries

#

Mach vmm did so and it was inherited by its descendents

#

With at least a last map entry faulted on cache however

#

So for the typical case just test if that's the one you faulted on. This approach worked well for a long time

grand shadow
#

interesting

opal lotus
grand shadow
#

I assume address spaces were smaller + less fragmented back then

proper meteor
#

I use a rbtree for vmm

hearty fulcrum
#

same

#

an interval tree

worthy mica
#

@grand shadow how does astrals init route work? I'm thinking about making one as im kind of tired of having 10k init calls in my main func

grand shadow
grand shadow
#

currently trying out os-test stuff on astral

#

it git cloned from gitlab and is compiling now

grand shadow
#

compiled, getting it to run

#

so far only one crash (syscall_accept() code did not check for an implemented socket op and thus call to zero)

#

and a hang on "udp/bind-socket-shutdown-r-sendto-first-poll"

dense atlas
#

I might wanna adapt it to support modules depending on modules, although I don't need that yet because modules can't link to other modules currently

opal lotus
#

but i don't use extra code to encode those dependencies

#

i just have the dependencies in the dynamic section of the modules

#

the compiler/linker creates them for me

dense atlas
opal lotus
dense atlas
#

do you not have dynamically loadable modules?

#

that's what I meant

opal lotus
#

i do

#

i specify the modules i want to link against in the build system, and the dependencies get saved like regular shared objects in userspace

#

the kernel doesn't do automatic dependency resolving though, it's up to the user to load the modules in the correct order

#

the kernel will simply refuse to load a module if a dependency is not loaded

dense atlas
#

omg sorry I confused you with mathew 😭😭

I'm sleepy I see

opal lotus
#

💀

dense atlas
opal lotus
#

yeah

dense atlas
#

since it doesn't know where they are

dense atlas
#

in fact I am using Kconfig :3

opal lotus
#

i did that before

#

but rust is kinda annoying in this regard

#

i can't use a build system of my choice

dense atlas
#

oh yeah, rust really does want you to use their stuff

#

I had to deal with it with my Makefile mess, as I have support for rust

opal lotus
#

i want to use meson but i'm being held up by stupid issues

dense atlas
grand shadow
#

I had done an oopsie when compiling the os-test and now after recompiling it I run into more issues, so I will do more debugging of it later

sharp carbon
dense valve
dense atlas
#

sometime my ESP8266 port will get somewhere..

grand shadow
#

okay so schedule is:
finish uni project
do some os-test work
then write out an rbtree implementation

grand shadow
#

negative refcount javyBoom

#

happened because I forgot to check if the fifo vnode was already bound to a pipe before unbinding it

#

woo that seems to be the only remaining hard issue on os-test

#

time to shove those results into linux and see how astral did

#

astral better than macos real troll

#

(this is missing the locale stuff by leo)

#

cc @timid jetty

#

wrong leo ☠️

#

cc @formal patio

#

hm most of the udp tests fail because connect return ENOTSUP on udp

#

that sounds like a easy way of bumping up the score

#

0% on pty is crazy tho

#

all ENOSYS

#

tf am I missing

#

oh wait

#

wasnt openpty turned into a sysdep or something

#

well thats another place that probably can get more % on

#

seems like a lot of the sys_socket code did not work because ENETUNREACH

#

huh it does seem like I have most of the infrastructure needed in udp to support udp connect() its just that I dont have udp_connect() proper

bronze nexus
#

UDP connect is kinda an oxymoron if you think about it

#

What with a protocol that doesn't handshake at all

grand shadow
grand shadow
bronze nexus
#

Sure

grand shadow
#

#define TIOCSPTLCK 0x13376167

#

actually hm, I'm seeing that unlockpt() is a noop on like bsds

#

ah whtaver I can make it a no-op ioctl kernel side

#

or just check that its the master pty

grand shadow
#

<---- forgot to add POLLHUP to his ptys

dense valve
worthy mica
dense valve
#

ty

worthy mica
opal lotus
#

what about _IOC meme

proper meteor
# grand shadow

I feel like if I try os-test on maestro it would be a disaster

proper meteor
#

maybe

viscid flame
proper meteor
opal lotus
proper meteor
opal lotus
#

at least it says on the website

#

@formal patio confirm

formal patio
#

not necessarily

#

ssh is only preferred if you want sortie to update os-test results on the website

#

you can run it without ssh locally

#

or you can do the managarm/redox strategy of allowing sortie to download os-test result tarballs from CI

opal lotus
#

well i'd just use os-test for my own purposes

#

is it hard to run locally?

#

i remember you talking about some memes with html or something

grand shadow
#

Well the tests are fine to compile and run

#

The big issue is generating the html

opal lotus
#

why?

grand shadow
#

You have to do that on the host

#

On astral with 12gbs of ram it still SIGBUS'd the html process

opal lotus
#

wth?

grand shadow
#

Too much memory

opal lotus
#

what is it doing that needs that much memory?

#

surely this can be fixed meme

grand shadow
#

No idea

#

inb4 every character is in a linked list or something

formal patio
#

html generation is fine on managarm tho

#

there's just no point to doing it on managarm when I can do it on my host, where I also have other results available, and where I can view the results, too

opal lotus
blazing atlas
#

imagine acpi microwave turn on support

grand shadow
#

I have a leak somewhere then

#

or it does some allocation on loop because it never seems to print anything

formal patio
opal lotus
#

ah that's easy enough

#

i'll add ext2 support very soon anyways

proper meteor
grand shadow
#

considering doing a switch over to xbstrap at some point

#

since newer jinx does NOT work on my environment for some reason

#

theres a bug on jinx that has been happening for the past 3 months

opal lotus
#

just being able to do meson_configure is pretty neat

grand shadow
#

I dont use that so I'd not miss out on much

opal lotus
#

it saves about 10-20 lines per recipe

gritty shore
#

you could do your own stuff as well

#

shouldnt be that hard right? meme

grand shadow
#

I do wonder how difficult it would be to use ebuilds

#

it would be very fucking funny if I did that

#

switch out xbps for portage

#

port openrc

#

and become gentoo 2.0

#

and in like 5 years when astral is actually fast I can mog on the haters by using portage without binary packages

#

but for now, I'm gonna finish the linux-mlibc-os-test project using xbstrap to get a feel for it and later I can see if I can do an ebuild mlibc linux thing to also get a feel for that

marsh sparrow
#

Using ebuilds would probably not be that difficult

#

Using nix may also be an interesting option

#

iirc @tribal dust was doing that basically

sage locust
#

i do not have the time to effectively juggle all the projects i work on, sorry

#

all you did was ping me with an error message that i have never seen, without instructions for me to be able to reproduce it

#

idk wtf you wanted me to do about it lol

#

not even so much as to tell me basics about your environment like what your distro is, etc.

#

i know you're way smarter than this Mathew, so if this is just about looking for an excuse to fully bite into the managarm ecosystem, for whatever reason, do so

#

you don't really need to give me an excuse, unless you're trying to give yourself an excuse

opal lotus
prime mulch
grand shadow
#

I'll try actually debugging it then

#

I still prefer jinx to xbstrap because yesterday I realized writing yaml sucks 💀

marsh sparrow
#

because it's yaml or because of something else?

grand shadow
#

it has the same issues with python wrt identation and like shell scripts are just more convenient

opal lotus
#

build: |

grand shadow
#

which is why I was gonna start looking into ebuilds but thats probably another kanker into itself

#

the last known good commit is 0.6.7 javyBoom

#

I dont see any big changes other than how debootstrap is called so hm

#

dawg I read my own message wrong

#

jinx 6.7 is not the good commit its the first bad commit

#

running on 3 hours of sleep 🔥

prime mulch
#

will astral ever tackle numa

carmine swallow
#

or well apparently I can't because of some m4 meme (might need a bit older snapshot idk) but like at least it started building troll

grand shadow
#

I think its some wsl meme

#

since on anything else it works

grand shadow
#

idk

carmine swallow
#

also what would be nice would be having some kind of minimal desktop package set

grand shadow
#

yeah I need to do that

#

probably just figure out some minimal set that can boot into fvwm with xterm

#

or maybe that has all the xorg shit + fonts + some other packages

#

yeah debootstrap is not creating /usr/local

gritty shore
grand shadow
#

bro FUCK ME I'm trying to open the debian wiki about deboostrap but wiki.debian.org is down

#

🥀

robust geyser
#

so i highly doubt it's anything to do with that, especially considering it works for me™

#

but i never used the weird ass ubuntu distro

elfin turtle
#

are u running it on a mounted ntfs

robust geyser
#

no

elfin turtle
#

cuz for me those have been fucky asf

grand shadow
#

I'm not that insane

robust geyser
#

you can see the path is ~/...

elfin turtle
#

good

robust geyser
#

and the errors are /home/mathew/...

#

so yeah

grand shadow
#

ok I have a clue

#

somehow the second stage bootstrap is not being run

#

theres no errors logged which sucks

robust geyser
#

you can try using set -x

#

to see all the commands being ran

#

but it might be a little verbose

grand shadow
#

can just set -x before the run_in_container1 "$JINX_CACHE_DIR/sets/.image" /debootstrap/debootstrap --second-stage and set +x after

robust geyser
#

that's not gonna do much considering it unshares and executes a new shell instance

#

can you probably try doing sh -c 'set -x; /debootstrap/debootstrap --second-stage'

grand shadow
#
            set -e
            set -x

            ld_preload=/nochown.so             home=/root             lang=en_us.utf-8             lc_collate=c             path=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin             ld_library_path=/usr/local/lib64:/usr/local/lib:/usr/lib64:/usr/lib             debian_frontend=noninteractive             "/debootstrap/debootstrap" \
"--second-stage" \
            # keep this whitespace
        ' > dbstraplog &> dbstraperr```
this runs but dbstraplog and dbstraperr are empty
#

env -i path= chroot
chroot not found

#

fym

#

that path= does not look like it should be empty

#

tho nothing in jinx sets ${path}

prime mulch
grand shadow
#

wait why is everything inside this string lowercase

#

what the fuck is this some curl bullshit

#

@sage locust sorry for bothering you again but I think found what was the issue: I forgot to pass --raw to curl I am actually stupid ☠️

#

TIL that curl does http shit if you dont disable that

viscid flame
grand shadow
#

wait wtf that was not it??? what

viscid flame
#

nothing ever works

prime juniper
robust geyser
#

nice green role bro

marsh sparrow
#

why would jinx care about what flags you pass to curl?

prime juniper
robust geyser
grand shadow
grand shadow
marsh sparrow
#

lol

robust geyser
#

http does not do any processing btw

grand shadow
#

then idfk why curl made everything lowercase but it did

marsh sparrow
#

Maybe it's doing processing based on the mime type

robust geyser
#

iirc jinx distros now switched to git cloning the jinx repo and copying out the script

prime juniper
grand shadow
#

smh

gritty shore
#

port system using lua when galaxybrain

robust geyser
#

you either do that or ship a copy of the jinx script in your repo

prime juniper
robust geyser
#

i think that's what gloire does

gritty shore
#

nah bash is pretty much ideal for this but whatever lua is cooler

robust geyser
gritty shore
#

or python

marsh sparrow
gritty shore
#

my biggest gripe with xbstrap is yaml

#

i hate it

#

i think ebuilds may be a good alternative though

#

and I will try them in the future

#
  • I get free gentoo recipes
prime mulch
#

xrbuildtool

marsh sparrow
#

Since xbstrap now supports preprocessing you can in principle use any language that can emit json as a recipe lang for xbstrap :^)

robust geyser
gritty shore
#

wait until you realize most gentoo packages run on freebsd too

robust geyser
#

Why the fuck

#

but still

gritty shore
#

gentoo freebsd is a thing

robust geyser
#

again

#

Why the Fuck

gritty shore
#

ah its not anymore

#

anyway

robust geyser
#

thank god.

gritty shore
#

but that shows its portable

robust geyser
#

yeah, i guess, overlays are a thing

#

i think arsen made an mlibc overlay or something lol

gritty shore
#

overlays are repos

robust geyser
#

i wouldn't know

#

sounds fancy

gritty shore
#

no it's literally just a name for a repo

robust geyser
#

yeah i know, but for the people who dont know it sounds like it does more than just provide packages

gritty shore
#

but anyway if the portage repos were ported to freebsd I'm pretty sure its doable to do it for your own OS

grand shadow
#

cant u remove the gentoo main overlay and use your own instead or something

gritty shore
#

yes

robust geyser
#

but then it's not gentoo anymore

#

it's just portage

gritty shore
#

yes

#

but you can still steal ebuilds from the gentoo repos

robust geyser
#

boooring

#

go big or go home

gritty shore
#

you literally do that with managarm tho

marsh sparrow
#

It's not surprising that ebuilds run on FreeBSD, is it?

#

It's just shell scripts

gritty shore
#

yea I know its not

gritty shore
marsh sparrow
#

Whether the recipes actually work is a different story:^)

grand shadow
#
  1. xbstrap and steal from managarm
  2. jinx and steal from gloire
  3. ebuild and steal from gentoo
  4. pkgbuild and steal from arch troll
gritty shore
#

unless you wanna run linux specific software

gritty shore
#

xbps and steal from void

marsh sparrow
#

I wanna see if nix is feasible for a hobby OS lol

robust geyser
#

xbps-src is dogshit

viscid flame
robust geyser
#

it's like worse pkgbuild

#

and pkgbuild is already questionable

gritty shore
#

afaik ebuilds are pretty much the best

#

for that kinda stuff

#

another option would be to use netbsd's port tree

marsh sparrow
#

ebuilds still have the disadvantage that they are shell scripts

gritty shore
#

how is that a disadvantage?

robust geyser
#

how is that a disadvantage

marsh sparrow
#

nix is side effect free halfmemeleft

robust geyser
#

just run it in a container bro

marsh sparrow
#

xbstrap is also side effect free

robust geyser
#

then it's side effect free*

gritty shore
#

nix is an overengineered way to do shell + container

gritty shore
marsh sparrow
marsh sparrow
gritty shore
#

I don't understand the issue

#

like, jinx is pretty much deterministic

#

and it's a glorified shell script in a container

marsh sparrow
#

That's why it's really easy to support distributed builds, for example

#

which is what xbbs does

gritty shore
#

phd level script

#

I still don't get why shell scripts dont work

#

is it because you have finer control over when commands get executed?

grand shadow
#

brother

#

    unshare --user --map-root-user --mount --pid --fork -- sh -c "
        set -e

        mount --rbind --make-rslave /dev '${run_in_cont1_root}/dev'
        mount -t proc proc '${run_in_cont1_root}/proc'
        mount --rbind --make-rslave /sys '${run_in_cont1_root}/sys'
        mount -t tmpfs run '${run_in_cont1_root}/run'
        mount -t tmpfs tmpfs '${run_in_cont1_root}/tmp'
        mount -t tmpfs tmpfs '${run_in_cont1_root}/var/tmp'

        mount --bind '${apt_cache}' '${run_in_cont1_root}/var/cache/apt/archives'

        env -i PATH="${PATH}" chroot '${run_in_cont1_root}' /bin/sh -c '
            set -e

            LD_PRELOAD=/nochown.so \
            HOME=/root \
            LANG=en_US.UTF-8 \
            LC_COLLATE=C \
            PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin \
            LD_LIBRARY_PATH=/usr/local/lib64:/usr/local/lib:/usr/lib64:/usr/lib \
            DEBIAN_FRONTEND=noninteractive \
            $(for arg in "$@"; do echo '"'"$(echo "${arg}" | sed 's/"/\\"/g')"'" \'; done)
            # keep this whitespace
        '
    "
#

see that "${PATH}" there?

marsh sparrow
#

Shell scripts work in principle if you have the discipline that everything is 100% deterministic

grand shadow
#

see those double quotes?

#

${PATH} is being expanded outside that string

#

thats fucking everything up

#

if thats the issue its gonna be so funny

marsh sparrow
#

do you have spaces in PATH or in cwd?

grand shadow
#

yes wsl does

robust geyser
#

wsl path includes windows path too

#

lmao

#

L

#

fix your windows PATH bro 🙏

grand shadow
marsh sparrow
#

lol

grand shadow
robust geyser
#

oh, mine also does that

#

/mnt/c/Program Files/NVIDIA Corporation/NVIDIA App/NvDLISR

#

crazy

#

hm then that shouldn't be an issue i guess?

grand shadow
#

running jinx for real now lets see

#

DEBOOTSTRAP SECOND STAGE RAN

#

guess I gotta create a codeberg account now

feral depot
#

i like to look at this channel at random at times and just be surprised at how random it is

#

true random

sage locust
#

thanks for looking into it <3

grand shadow
#

np lol

#

I already made an account so I'm gonna open the pr when I'm done testing it

#

ya ok its working

sage locust
#

merged it, tyvm

#

space in $PATH is evil 💀

flat copper
#

yeah it's a wsl2 issu

marsh sparrow
flat copper
#

i've worked around it a while back by disabling the feature

#

because i've had other issues with it in the past

#

i think with pip or dune

marsh sparrow
#

I'd expect many of the make based build systems to break

prime juniper
prime juniper
grand shadow
sage locust
#

i make it succeed once by stubbing everything

#

aka comment out all steps

#

then it should not do that anymore

grand shadow
#

I see, thanks

sage locust
#

np

grand shadow
#

cooking up an rbtree insert implementation while running on 3 hours of sleep at 11 pm, what could go wrong

gritty shore
#

wdym cooking up

#

you guys arent translating the "introduction to algorithms" pseudocode to C meme?

grand shadow
#

Tfw you actually want to understand the code you wrote 🥀

#

My dsa class did not give any pseudocode for rbtrees

#

At most we got the trees

#

And even then the slides arent complete and fail to mention some stuff ☠️

#

Plus it makes me feel smarter knowing I was able to actually implement that

gritty shore
#

well the pseudocode is far from complete

grand shadow
#

In my class my professor always gave out implementations for shit like linked lists in the labs 💀

#

Like if u cannot write a linked list u not built for this

gritty shore
#

stuff like rbtrees aren't conceptually complicated

#

So they're not hard to understand

grand shadow
#

Yeah

gritty shore
#

The pseudocode just makes it easier to implement

#

imo

grand shadow
#

Once u understand them doing it without any pseudocode is also easy

gritty shore
#

yeah that's how I did AVL

#

rbtree in particular I implemented a while back and didn't bother

grand shadow
#

Do u use both in your os or did you swing towards one over the other

gritty shore
#

both

#

The funniest data structure I have is HAMT

#

That's conceptually very simple but the implementation is a bit tricky to get right

grand shadow
#

I think I remember you mentioning that before

gritty shore
#

I haven't gotten to the point of using it yet

#

Nor AVL or rbtree trl

grand shadow
#

Inb4 30k line hello world like infy

gritty shore
#

nah it's around like 8k total atm and it has a scheduler

grand shadow
#

Oh nice

#

Ule I assume?

gritty shore
#

yeah it's a fixed version of my fancy scheduler

#

Seems to work pretty well

#

now I haven't worked on stuff for like two weeks because of finals but I'll get back to it soon (I was in the middle of implementing turnstiles)

grand shadow
#

still working on that rbtree

#

had a busy weekend but now I'll have a bunch of free time

#

wonder how big of a speedup will rbtree + caching the last accessed range per thread would be

#

especially on mc where like 70% of the time is spent on range search halfmemeleft

limpid trail
#

Because that’s hella smart + simple

grand shadow
#

probably storing it in a per thread variable

#

since each thread will be faulting different addresses

limpid trail
#

Oh true

#

But same idea

#

Can I yoink that idea for my vmm

#

I still use a linked list for my ranges and I don’t feel like switching to something better rn so I kind of need the free speed where I can get it

grand shadow
#

its not like I am the owner of ideas or anything

limpid trail
#

Bet

#

Also I’ve said this before I think but your project is so sick

#

2030 year of the Astral desktop?

grand shadow
#

thansk

#

more like 2100 :p

grand shadow
#

seems like I will have to add a container_of macro to astral finally Kekw

grand shadow
#

first test run of the rbtree (with 100 nodes):

#

I triple checked the insert function to make sure everything is ok so maybe I made an error in the check function where I did not triple check it

#

thats an issue for tomorrow me

#

its 3 am

grand shadow
#

looking at it its only after 2 inserts which is not enough for a black height violation

#

so my check is broken

grand shadow
#

ok, that was a check bug

#

now I think I actually hit a bug in the insert

#

I think its a bug with the left rotation

#

something is causing a right pointer to become null

#

when it should not

grand shadow
#

update: rbtree insertion works

#

can insert 100 values and look them up and they all pass the check

prime mulch
#

the 101st value is what gets you

analog berry
#

Yeah id add test cases for every rebalance case at least

marsh sparrow
#

If you insert a few few 1000 random elements, you'll hit each case anyway

analog berry
#

also removal is easy to fuck up

proper meteor
wild marsh
#

replace removed element with tombstone :^)

elfin turtle
pine ledge
scenic epoch
dense atlas
#

@grand shadowI am currently taking "inspiration" from your VFS code to maybe possibly finally come up with something somewhat sane for my kernel and I wonder if you can break the devfs

unfortunately I'm at school and the wifi is too slow to download the astral image, so I can't test it rn

what happens if you

  • mount a devfs on at /dev
  • mount devfs again (which is the points to the same files as the first one right?) at e.g. /mnt
  • mount a tmpfs on /mnt (on top of the devfs)

Now if you look at /dev you should see the tmpfs instead I think???? until you unmount /mnt again

Just guessing, I may also just be completely wrong as all this is from me just studying the code

grand shadow
#

I dont really support bind mounts and traditionally you cant mount the same filesystem twice

#

So this is more so a hacky way of supporting mounting devfs to multiple places

dense atlas
grand shadow
#

2025 is almost over and so I need a proper 2026 goals list for astral
hmmmm

#
  • compiling a distro inside itself without crashing, even if a minimal distro
  • procfs, maybe
  • audio stack
  • finish the installation guide or write a install program
#

maybe throw in a webkitgtk there, but I feel like thats way too ambitious

fleet osprey
#

Debian GNU/Astral when trl

#

it works on Hurd and the BSDs....

#

(or at least, did, at various points in the past)

#

honestly not completely far fetched

grand shadow
#

💀

#

anyhow, havent worked on this these past few days

prime mulch
#

numa

#

astral numa 2026

#

its calling you

proper meteor
prime mulch
proper meteor
prime mulch
#

each vertex on those hypercubes is 2 numa nodes

#

4 processors

#

in an SGI Origin 2000 from 1998

#

thats the network topology between them for cache coherency messages and so on

#

1 computer

neon crane
#

its so funny

dense valve
#

WOW chefs_kiss

dense valve
dense valve
neon crane
#

thus, the "Numa Numa" song

neon crane
#

and thus the connection I keep drawing unintentionally with NUMA (non-uniform memory access)

dense valve
#

I only connect it with MY YAHOO

#

Lmao

#

MY YAHOO HO (Ma-ia-hoo, ma-ia-haa)

night trellis
#

honestly i wanna make an astral distro called Tidal Axis with its own package management system called axpkg

#

and make wayland support for the astral kernel

grand shadow
#

If u want I can make the kernel build standalone from the rest

grand shadow
#

gyatt rizzmas or however the cool kids say nowadays

willow rapids
#

i can't do that yet :(

dense valve
viscid flame
sage locust
#

fvwm

elfin turtle
#

istg u can take any random 1-3 letters and postfix wm and it probably exists

grand shadow
#

67wm

elfin turtle
#

kill me

#

rizzwm

scenic epoch
#

wmde trl

#

dewm trl

#

except it would be like a desktop environment instead of a window manager

grand shadow
#

rbtree update: inserting 1000 nodes and removing them does not break the rbtree properties (tested after each operation)

#

I will do more specific tests like checking if succession is ok and then maybe do a test with 10000 or 100'000 nodes

grand shadow
#

ok predecessor and successor checks seem to be working

#

iterative rbtree is almost done :D just gotta tidy up the code

grand shadow
#

50k worked, 100k didnt work because rand is not random enough to not repeat stuff :P

prime mulch
#

if its a 32 bit rng

#

its not LIKELY but if you saw it every once in a long while it shouldnt be surprising

grand shadow
#

time to begin converting the vmm to using the new rbtree

grand shadow
#

one last function remaining, changemap

grand shadow
#

(and rewriting the worst function there while at it)

grand shadow
#

oh thats a bug I never noticed

night trellis
#

ohh

#

lmao

grand shadow
#

when splitting a mapping and in the remap case I dont set the middle file offset properly

night trellis
#

do you mind making the package manager my own instead of xbps

#

axpkg and axports

grand shadow
#

should've been new_offset = range->offset + range->size

night trellis
#

astral is unix-like iirc, no?

grand shadow
grand shadow
night trellis
#

in tidal axis development

grand shadow
#

ah then you can just use whichever package manager you would want

night trellis
#

or at least for axpkg

grand shadow
#

@sharp carbon when you added the arch_extracontext_detect() function, did you try it on tcg as well? it seems like tcg really does not like the mov to cr4

sharp carbon
grand shadow
#

currently working on the rbtree stuff

#

got to a bash prompt using it

#

went thru all the whole init and first boot process too

#

so it seems to at least be working for most cases

#

fvwm boots

#

time for the trvth nvke

#

oops, didnt work but thats because of a lazy assert I added for a very specific case (unmap on an empty space)

#

damn java doesnt work

#

🥀

#

spooky backtrace

grand shadow
#

meinkraft got into the main menu, creating a world to see if it works

#

too many debugging logs to get an idea of improvement tho

#

ran into the font meme I dont have the patch for

#

actually I forgot if thats a patch I need or just an unclean shutdown after handling fonts which causes a failure on the next shutdown

grand shadow
#

new minecraft flamegraph

#

the lookup funkyness has been eliminated

#

didnt notice a lot of perf improvement in mc though

#

I also need to figure out that isr_14

#

because it only happens in the brief time the very start of the page fault isr is being executed

#

but its taking up so much time that idk if its the cpu taking too many cycles to handle a page fault because kvm in wsl or what

#

I still gotta do the range caching thing

#

oh wait my laptop also isnt connected to power

#

💀

robust geyser
#

building astral rn, i wanna see if i can write an xhci driver

grand shadow
#

Yooooooo

robust geyser
#

i don't promise anything but if it works out it'd be pretty sick :^)

#

i hope you have msi(-x) support, otherwise i'll be very sad

grand shadow
#

I do

#

I dont have any support for the acpi pci interrupt pin thing

#

But I do have uacpi :P

robust geyser
#

or i might be wrong, who knows, not me since i haven't looked at the spec in god knows how long

#

but even then, having a shared interrupt handler is probably fine

opal lotus
robust geyser
#
bashline.c:65:10: fatal error: builtins/builtext.h: No such file or directory
   65 | #include "builtins/builtext.h"          /* for read_builtin */
      |          ^~~~~~~~~~~~~~~~~~~~~
compilation terminated.```
#

lol what i just reran make and bash built just fine, wild

grand shadow
#

Wtf

#

Bro got bit flipped

robust geyser
#

yeah i'm not sure what that was about lol

#

but it's all good now

#

i wrote a qucik gen_compile_db.py script to create a compile_commands.json, should help writing any code considering i don't know the codebase at all

#

i'll start working as soon as everything is built, i think i'm building a full distro so that might take a second

formal patio
#

wait astral doesn't use meson/cmake?

#

I don't know enough about jinx to figure out how astral gets built meme

grand shadow
#

Just a plaon makefile

grand shadow
formal patio
#

bruh

formal patio
robust geyser
#

if it aint broke...

formal patio
#

handwritten Makefiles are always broken

#

at best, it's just (yet) undiscovered brokenness ultrameme

robust geyser
#

meson has it's own set of issues

robust geyser