#Astral
1 messages · Page 9 of 1
you aren't supposed to pass -j to the makefile
and I don't quite have forever
jinx handles parallel builds
thats a very interesting way of wording "you arent supposed to pass the -j flag"
saves me a command
happy to help
how do I make jinx build
theres also vmmcache.c which is the page cache
make DISTROTYPE=minimal
yeah -j${parallelism} is in the recipes
the vmm is some early astral code and is no place of honour so gl
@grand shadow btw u dont have tests of any type right
yea
i think having some sort of userspace stress tester would be nice
that could run for every commit
otherwise things will always regress
uStressTester
more like
wget https://ftp.gnu.org/gnu/gcc/gccblahblahblah tar-xf gccblahblahblah ....... 
OberrowUnstabilizer
even a simple bash script that does this
then checks the checksum
executes sync
u know
could be one of the test cases
ok but that was only fourth kernel
lol
this kernel only died on uacpi init twice
and it wasn't a two-week long debug session like last time either
void *start;
size_t size;```
.
......
I moved uacpi init after init of the disk swapping thingy temporarily, and it didn't even crash
and I thought it would because it was completely untested
I mean I would have to figure out how I would even do something like that
its probably safe to swap out the entirety of uacpi except for the interrupt handler
I know nothing about git ci and the limits and all
technically irq handlers don't need to non-pageable, it's just not quite efficient probably
git ci is just a yaml file with steps and a bash script embedded in those steps
idk
I should do that ngl
only memory allocated by uacpi is pageable
the rest is non-pageable because I forgor
to
build astral, run vm with some option to make init run the tests, look at the logs to see if it succeded
which is what apple does aggressively
are there any limits on github ci or something
a macbook with 16gb of ram will cache way more than a 8gb one
nah
or yes but you wont run into them
there's also caching helpers so u can cache the gcc and other shit
instead of building it 10 trillion times
there is basically unlimited linux machine time for public repos
you could run it straight up for almost a month every month
yup
bruh damn
I thought there was some limit or something, good to know
I should set it up to at least build astral so I know clean builds arent broken
@grand shadow if u make the tests i could help u with the yaml for running them
you can run qemu in the ci?
hopefully doesnt end up being too slow if I end up wanting to test like "get a program configure compile and install"
u could make small and large tests if that becomes a problem
and only run large tests in some cases
for large prs for example
to be fair being able to boot astral and ssh into with some simple command is already a very good test
yeah tests user stuff, memory, netstack, etc
example of a test:
- boot vm
- scp hello.c (just prints hello world)
- compile hello.c on astral
- run ./hello
- make sure it printed out what u expected
- sync
- poweroff
- fsck astral.disk
- PASS!
trivial test but huge coverage
that wont work, dw
i did send a compile_flags.txt somewhere
its not as good as compile_commands.json but it works fine
-g
-ffreestanding
-mcmodel=kernel
-O2
-mno-red-zone
-mgeneral-regs-only
-mno-mmx
-mno-sse
-mno-sse2
-nostdlib
-Wall
-I
kernel-src/include
-I
kernel-src/include/x86-64
-I
kernel-src/flanterm
-I
kernel-src/io/acpi/uacpi/include
-DPRINTF_DISABLE_SUPPORT_FLOAT
there, thanks
ty
oh yeah i want one
send too
maybe clion would work with it
tru
sounds good
for the ssh I could just gen a key and copy it into /home/astral/.ssh and then scp should work fine
something like
# - compile everything
# - generate ssh key
# - copy key to /home/astral/.ssh
# - create the disk with proper chowns (I have a local script here for that which could be used)
# - run qemu with port 12345 on host mapped to port 22 on astral
scp -P 12435 hello.c [email protected]:/home/astral/hello.c
ssh -p 12345 [email protected] "gcc hello.c -o hello"
ssh -p 12345 [email protected] "./hello"
ssh -p 12345 [email protected] "poweroff" # syncs disks already
# - wait for qemu to close
e2fsck -f root.img
like speaking from experience having like a ./run_tests.sh with a decent coverage is an absolutely HUGE UX experience boost, it gives u confidence and makes your life soo much better as a developer, you're no longer scared to make big changes as any regression will be detected instantly etc
best part is
it's less likely for me to bring over the oberrow curse
and it gets merged
perfect yeah
i dont think astral handles oom very well?
wait until I add the kill(2) checks
so u could dos it fairly quickly 
it doesn't assertOS funnily enough
thats fine
it actually passes down ENOMEMs and stuff
managarm left in shambles
the only places I assertos memory allocations are in like initialisation things that get called once
if a memory allocation fails there its over
that makes sense even linux does it
there's like a "kernel deadlocked on memory" oom error or whatever
now I have two options here:
- only page out regions (more lazy option)
- change the vmm to have page-granularity
vmmrange_tstructs
what if someone mmaps a 1tb range
who the fuck mmaps a 1 tb range
lemme calculate the memory usage for that
any database
they invented 5 level paging for a reason
it'd take around ~31250000000 bytes of ram
thats why u swap it out
assuming vmmrange_t is 32-bit
(for the page structures)
what
well it's more like me renaming the struct
what other operating systems even have a vmm with page granularity ranges
me
:^)
I really dont think its a good idea
just realized why that might be a bad idea
anyone can just mmap a huge range and bam memory gone
(TODO: Fix)
ok I know what I'll do instead of that
I'll have vmmrange_ts of paged out regions
in some different list
why do u even need it like
u have the page address in the page table
u have the range in the page table
i always thought that this "paging out" thing is done at the page level
not vmm range level
u can literally have a start and an end pointer
its not i think
yeah it sounds wrong
like u have ranges of mapped user memory
and saying stuff outta my ass
like an interval tree
you should brainstorm this while like not tired 
yeah considering I have no idea how astral vmm works that'd be a good idea
probably not exactly rocket science 
but
it's called astral
!
is mathewnd just throwing a giant explanation of his vmm rn
he's been typing for like 8 minutes
I do for anon objects, but they are allocated on-demand
yeah I need to fix my vmm
wait what do you do
hold on
maybe
maybe not
(oops)
whaaat am i looking at
a very big struct
jesus why
how many bits is it
my vmm isn't the thing I'm most proud of
over 64 is shit
well that part is pretty bad
i dont think thats how a page struct should be done
128 bytes
it's not for phys. pages
why is it named like that then
i havent looked at it deep but why
wait your struct page isnt for physical pages
because like virtual page
what
waait
ngl terrible naming 💀
no comment
on top of the real struct page?
kinda cringe ngl
I have no such thing for physical pages
should atleast do it on-demand
you know you can have all that attached to a start and end pointer
how do u lookup slabs
and when you mprot/munmap you just split it
and keep info about if its swapped out or not on the actual physical page
multiple people can map the same physical page so it makes sense to keep that info there
yeah that might require an entire rewrite of the vmm...
everything in the vmm uses that struct
do that before rewriting the astral one pls
yeah
@obsidian pivot
basicalloc_node* n = ((basicalloc_node*)base - 1);
it's stored right before the start of the region
what if its a small slab
basically like
there are contexts, spaces, ranges and pages (from pmm.h)
contexts have a space and a page table pointer for mmu
spaces have a lock and ranges
ranges describe individual ranges of pages, doing fragmentation checks on mapping etc
pages get handled by the pmm but its basically like this:
there are several sections (like <1mb, <4gb and rest)
these sections each have a free list and a standby list
priority is to take pages from the freelist but if that isnt possible theyre reclaimed from the standby list (which is currently only page cache pages with refcount = 0)
they can be backed or not (which determines if they'll go to the standby or free list when refcount = 0), and have an offset within that backing (which is a vnode)
on mapping the vmm inserts a range into the space, checks if it can merge with its neighbor and does more things if nescessary (such as instantly allocate the memory if needed)
on a page fault the vmm checks if the faulting address is inside a range, then checks if the action is valid (like trying to write to a read only page), then if its still valid it checks what it needs to do (get page from page cache, map 0 page, copy on write, etc) based on the current state of the PTE by calling into arch_mmu
on unmapping the vmm chops whatever ranges are needed, cleans up the already mapped pages, dirties them in the page cache if needed and then unrefs them
the page cache is basically just a hash table with a linked list that holds file pages, and keeps dirty pages in a dirty list with a refcount of >= 1 until either some time passes or a sync is called, which it then begins writing them back to disk and unrefing them into standby
when someone wants to get a page from the cache, it calls into it asking for a specific offset in a vnode and if its found it increases the refcount of the page by 1 and its taken out of the standby list if nescessary, otherwise it just allocates a new physical page and VOP_PUTPAGEs into it
there are only two spaces accessible for a user process, the specific user space for the vmmcontext and the kernel space, each with its own locking
honestly slabs are kinda OP
was gonna write more but since you have more troubling things to do I think I'm gonna save some time 💀
if u have a struct page yes definitely
might end up rewriting the kernel by accident
(not)
otherwise u have to enginner some sort of
idk
the initial slab paper talks about a hash table
u need to know the freelist it came from
kmem_cache_free(&obj, &cache)
can astral block on memory allocation?
wdym
like wait for free pages
yes but u need to know which actual slab it was allocated from to free it
which is why you pass the slab... ?
but if you have a random pointer you dont have the slab
which is the question
its in the top of the page bruh
how d you go from a random pointer to a slab
thats not even the problem
(which is from the paper)
then i misunderstood too
unless u have a struct page which links to slab_t
lets say your 8 byte slabs start at 0xFF..DEAD0000
then u can just look at those bits
and know the slab type
yeah i just allocated a single page for my slabs with the actual storage inline
or u use a struct page
making out first 12 bits gets me the slab pointer
this is what I do
which has so many other usages
actually goated
the hashtable isnt real btw
nice linked list hash table
yeah so u scan it right
yes
ngl linked lists are based
page->bufctl
dunno how well that would work for sub-page-size objects
oh yeah nvm
u can just round down
yeah u keep track of the struct page in the slab then u can figure it out
i thought “bufctls” were a multiple of the page size
yea
u can link bufctl pages
or just allocate contiguously
linux does contiguous by default with a fallback
it pains me how bad astral pmm is for non-single page allocations
would be nice if someone rewrote it 
in my understanding they're just contiguous pages for slab objects?
the bufctls for large caches are allocated out-of-line (from a bufctl cache) and linked onto a hashtable so you can swiftly resolve an address to its associated bufctl
and xhci is boring because im stupid
for small caches bufctls are inline and only used for the freelist
do u actually use a hash table for them?
i thought u had a struct page
we'll see if im able to work on astral even
struct kmem_bufctl {
/*!
* Linkage either for free list (only case for small slab); or for large
* slabs, kmem_zone::bufctllist
*/
SLIST_ENTRY(kmem_bufctl) entrylist;
/*! slab to which bufctl belongs */
struct kmem_slab *slab;
/*! absolute address of entry */
char *base;
};
bufctllist should be bufctl hashtable
they are needed either way for large slabs
or rather i guess its metadata of which slab a page belongs to
for large objects that span the entire page u have to place it out-of-line
for smaller ones u embed it
for the smaller ones only the entrylist field exists, and it doesn't exist at all when you actually allocate something
god damn astral pmm is kinda poop actually, i never realized that 
the page struct list is basically allocated out of the biggest memory entry
I should rewrite tinyvmem and my kmem thing (which now I realize was basically scal-ux's but slightly changed) in C++ and adopt magazines
would be nice
and i dont think its sliced, its just the biggest memory entry
since
slab = (struct kmem_slab *)SMALL_SLAB_HDR(PGROUNDDOWN(ptr));
but i might be misunderstanding the code
its not
(I was lazy)
soon youll come to realize it isnt just the pmm
i might port over my simple buddy allocator that was partially stolen inspired by keyronex
i am however thinking of migrating the slab header into the page struct (in small slabs it's the last few bytes of the slab; that's alright but means you can't fit a nice even number of elements into the slab even if it's naturally aligned)
Elixir Cross Referencer - source file of Linux (version v6.10.7). Browsed file: /mm/slab.h
linux has this
how'll that work with the page struct and standby list and stuff
basically struct page becomes a struct slab
i think u can avoid any hash tables entirely?
i am not going to touch any of that, what changes is the way the page list is allocated and how pmm_alloc/pmm_free works
i am also going to add a few flags to the page struct
but thats it
this has pointers to literally anything u might need
yeah yeah i know
but eventually that's what returns the page to the freelist
oh wait i think i see the problem
or rather my lack of knowledge
maybe if you allocate the large slabs on naturally aligned boundaries so you can figure out the base of it in the same way as for small slabs
i will read the current pmm and figure out what i can do about it
I mean why do you need to allocate a physical range over PAGE_SIZE during runtime
good point, u can probably guarantee that for 64-bit address spaces?
during init it should be fine
while im at it i might look into making the page array virtually contiguous without stealing a bunch of physical memory
yeah that was on my todo list but you can do it 
when you plug in a network adapter for example
usb one
you need to allocate rx/tx descriptors
then if you want to keep the large slabs' headers out-of-line a hashtable from slab base to slab struct instead, or even just resolve the physical address the first page of the large slab points to, if you store the slab data in the page strucgt
just thought about this, wouldnt you still need to iterate over the page
i guess you can use scatter gather or whatever
for example 8 byte objects
and still get away with single page allocations
it'd be a much smaller iteration but still considerable
wdym?
sorry its very late im kinda
wait I may have forgotten what bufctls are
freeing them is easy, just let the content of that 8 bytes point to the previous freelist head, and point the freelist head to that 8 bytes
maybe there actually isnt a legit reason to do non single page allocations at all..?
lets say u have a slab of 12k objects
a lot of kernels are getting by with very limited suport for that
and u have slab_t of 10 objects
someone frees the second object
u can find the base of that slab because you can have the struct page point to the base page of that slab_t can you not?
outside of system init I would make an assert to not allow it if I could bother
because there is just very little reason
yeah but to find the proper bufctl, you need to iterate over the page's bufctls and find the one you need, don't you
if its just system init then i would completely get rid of it and rewrite the code that does use it
i did add the buddy allocator for storport actually because some storport drivers wanted larger contig allocations and i just didn't have the heart to do freelist sorting and scanning, but that would have worked too, what the buddy will be really useful for is to allocate large pages
since you said was you being lazy that led to that
the bufctl is that 8 bytes
or rather it will become a bufctl after you free it
vionet being example iirc?
ok yeah and since it's a freelist you don't need to put it anywhere "special" (anywhere in the list will do)
I was confused mb
anyway what i realized from this convo is that i have no idea what im talking about, i need to actually write a good slab allocator to get it properly
for small slabs the bufctl just decomposes into a freelist link
that would only be called during init as I dont really support hotplug for like anything
oh yeah, with vio-nic you can really get away with page only allocations
why the heck is higher half mapped with the user bit set
doesnt NT basically do that
@grand shadow
WHAT
for (int i = 256; i < 512; ++i) {
uint64_t *entry = pmm_allocpage(PMM_SECTION_DEFAULT);
__assert(entry);
memset(MAKE_HHDM(entry), 0, PAGE_SIZE);
template[i] = (uint64_t)entry | INTERMEDIATE_FLAGS;
}```
#define INTERMEDIATE_FLAGS ARCH_MMU_FLAGS_WRITE | ARCH_MMU_FLAGS_READ | ARCH_MMU_FLAGS_USER
it does or at least it did as of the book i read on it
i guess thats true, but you could probably still get rid of that since its useless :^)
looking at my e1000 driver it seems it also gets by with no contiguous allocs
im assuming the intermediate flags thing is used for all page table mappings then
p sure yeah
looks like i was angry about it at some point
what language is this
Instead of calling like panic or something lol
Objective c
hm
Its pretty neat
there's an assertion but i think i added that when i was bothered by the debugger
either index was optimised out even with -Og, m_rxNextTail was inaccessible without GDB doing a gratuitous malloc (it does that sometimes), or both
What does +, - and : mean?
Lol
- are factory methods, - are instance methods, : makes keyword components of method selectors (method selectors like
object doX: x withY: yare ultimately compiled down to something likeobjc_msgSend(object, "doX:withY:", x, y);)
Wow this is more involved than id thought
factory methods are roughly equivalent to C++ static methods but classes are object instances of their metaclass so they are still truly member functions, just defined by the metaclass not the class
how is the metaclass they belong to determined? everything after @implementation foo belongs to foo?
Does It like parse that string at runtime or something?
yeah, so + whatever belongs to foo class (the metaclass) and - whatever to foo (the class)
not by default, the typical route is to look it up in some data structure that class objects (and metaclass objects are also class objects) have to map selector strings to method implementations
it's a legacy of smalltalk's early form
I see
alan kay liked a feature of lisp called fexprs which preceded lisp macros, and were basically a way to invoke a function with arguments being unevaluated
Yeah I know it’s coming from there, im just not very familiar with the idea of smalltalk or how it worked
what
so
(function '(params))
and early smalltalk worked exactly like that, objects were functions that evaluated their arguments
Hmm
a macro?
to for token step stop var start exp (
:☛var. (∢←⇒(:start.)☛start←1).
(∢to⇒(:stop.)☛stop←start.)
(∢by⇒(:step.)☛step←1.)
∢do. :#exp. CODE 24)
'
``` that's how `for` was defined in early smalltalk - it's a class = it's a function
wouldnt it be something like a list of quoted values
unless I'm misunderstanding what unevaluated means
no you do understand, it gives you the raw form and not it's value
like it would in a macro
damn i hit the easter egg
later it was decided that this was a bit too flexible (and unoptimisable) so the common patterns of syntax were explicitly supported instead: unary messages, binary messages (plus, minus, multiply, etc), and keyword messages (x doSomethingFunTo: y with: z), and they were dispatched by an efficient lookup in a hashtable of message selectors -> method implementations
yeah okay
but the body of the fexpr is not made to produce a list that is then evaluated again
what happened here
it's just a regular function
im trying to map hhdm and kernel using 2MB pages
Wait it wasn't previously?
Bruh
// TODO use 2mb pages
// populate hhdm
https://smalltalk76.com/0011-IFS-File-System.st.html that's how the syntax became regular like this instead of everything being totally freeform according to however you parsed every arugment you got
Me when astral destroys my tlb
so yeah for some reason this isnt working...c for (uint64_t off = 0; off < e->length;) { uint64_t phys = e->base + off; uint64_t entry = (phys & ADDRMASK) | ARCH_MMU_FLAGS_READ | ARCH_MMU_FLAGS_WRITE | ARCH_MMU_FLAGS_NOEXEC; if ((phys & (LARGE_PAGE_SIZE - 1)) == 0) { __assert(add_page(FROM_HHDM(template), MAKE_HHDM((void *)phys), entry | LARGE_PAGE_FLAGS, DEPTH_PT)); off += LARGE_PAGE_SIZE; } else { __assert(add_page(FROM_HHDM(template), MAKE_HHDM((void *)phys), entry, 0)); off += PAGE_SIZE; } }
hm maybe i should mark it present 
however - you can still dynamically do whatever you want as if there's no hashtable entry for a selector, it's dispatched to a specific method with the arguments extracted from the stack and registers and put into a structure
Nahh
wait no read means present
in x86-64 read is that
In your screenshot it faulted on 380
Not some hh memory
yeah i see that, thats why im confused
because i didnt change anything other than that
interesting fault address
addr2line to the rescue
it's low enough i might blame a null dereference
Oberrow curse
what line did it fault at
lemme addr2line
one sec
static void gpfisr(isr_t *self, context_t *ctx) {
thread_t *thread = _cpu()->thread;
if (thread->usercopyctx) {
memcpy(ctx, thread->usercopyctx, sizeof(context_t));
thread->usercopyctx = NULL;
CTX_RET(ctx) = EFAULT;
} else if (ARCH_CONTEXT_ISUSER(ctx)) {
signal_signalthread(thread, SIGSEGV, true);
} else {
_panic("General Protection Fault", ctx);
}
}```
the if
thread->usercopyctx one
dear lord
i added a null check to that
and now im crashing in czapek@raptor-wsl:~/new-sources/Astral$ addr2line -e builds/astral/astral ffffffff800039d1 /build_dir/builds/astral/include/spinlock.h:12
Lol
sooo yeah, not useful
Whats on that line
while(!__sync_bool_compare_and_swap(lock, 0, 1)) asm("pause");
Huh
yeah ok I read a bit about them and they're basically lazy eval
Indeed
($define! $if
($vau (x y z) env
($cond ((eval x env) (eval y env))
(#t (eval z env)))))
from what I understand this saves evaluation so if x is true, only then y is evaluated
(define (if condition true-val false-val)
(cond ((eval condition) (eval true-val))
(else (eval false-val)))
(if '(< 3 4) '7 '3) ; Returns 7
not sure how that differs from that
just fakeroot
I have no idea how to do that (and I need to mount an image to generate it)
agh idk
I have zero idea about this stuf
hmm
guestfs?
or whatevr its called
you can pass -d to mkfs.ext4
to have it copy files from a directory
czapek@raptor-wsl:~/new-sources/Astral$ truncate -s 4G minimal.ext4
czapek@raptor-wsl:~/new-sources/Astral$ /usr/sbin/mkfs.ext4 -d minimalsysroot minimal.ext4
mke2fs 1.47.1 (20-May-2024)
Discarding device blocks: done
Creating filesystem with 1048576 4k blocks and 262144 inodes
Filesystem UUID: d7a9fa14-c15a-4101-9cb1-3304e4228548
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736
Allocating group tables: done
Writing inode tables: done
Creating journal (16384 blocks): done
Copying files into the device: done
Writing superblocks and filesystem accounting information: done
czapek@raptor-wsl:~/new-sources/Astral$ mkdir minimal-mounted
czapek@raptor-wsl:~/new-sources/Astral$ sudo mount minimal.ext4 minimal-mounted
czapek@raptor-wsl:~/new-sources/Astral$ ls minimal-mounted/
bin etc init lost+found root usr```
czapek@raptor-wsl:~/new-sources/Astral$ df -h minimal-mounted/
Filesystem Size Used Avail Use% Mounted on
/dev/loop0 3.9G 116M 3.6G 4% /home/czapek/new-sources/Astral/minimal-mounted
czapek@raptor-wsl:~/new-sources/Astral$ du -sh minimal.ext4
184M minimal.ext4```
thats nice too
I still need to like chown and stuff
fakeroot
does it allow you to chown to root?
yeah ofc
it hooks into filesystem libc functions
and intercepts everything
and creates like a mini vfs in memory
for the duration of what youre doing
czapek@raptor-wsl:~/new-sources/Astral$ fakeroot << EOF
> id -u
> EOF
0```
nice
you can write multiline stuff that way
so you'd just chown that, create an image (truncate -s len)
ohhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhhh
holy shit
ty
I will see if I can
make a simple ./makedisk.sh script
that uses it
that would be based
you can also use it for initramfs
so it has proper permissions
someone suggested it already
in here
like last week i think
if I add a ./makedisk.sh command I think I might even phase out the usage of the initrd a bit and only ship a minimal one
because you'd be running from disk
not a bad idea tbf
#!/usr/bin/bash
fakeroot << EOF
truncate nvme.img -s 4g
find sysroot/ ! -type l -perm -04000 -exec chown -h 0:0 {} + \
-exec chmod u+s {} +
find sysroot/ ! -type l ! -perm -04000 -exec chown 0:0 -h {} +
find sysroot/home/astral ! -type l -perm -04000 -exec chown -h 1000:1000 {} + \
-exec chmod u+s {} +
find sysroot/home/astral ! -type l ! -perm -04000 -exec chown -h 1000:1000 {} +
mkfs.ext2 -O ^dir_index nvme.img -d sysroot
EOF
hoooly shit
it works
based
@robust geyser can you see if that script works for you with this qemu cmdline
qemu-system-x86_64 -enable-kvm -M q35 -cdrom astral.iso -m 4g -smp cpus=1 -monitor stdio -debugcon file:/dev/stdout -serial file:/dev/stdout -drive file=nvme.img,if=none,id=nvme -device nvme,serial=deadc0ff,drive=nvme -boot order=dc -netdev user,id=net0,hostfwd=tcp::1234-:22 -device virtio-net,netdev=net0
sure one sec
czapek@raptor-wsl:~/new-sources/Astral$ ./makediskimg.sh
find: ‘sysroot/’: No such file or directory
find: ‘sysroot/’: No such file or directory
find: ‘sysroot/home/astral’: No such file or directory
find: ‘sysroot/home/astral’: No such file or directory
/bin/bash: line 9: mkfs.ext2: command not found```
oh wait you're running with the minimal sysroot
hmm ill have to change a few things
yeah lol
might want to create the sysroot in the script
also dumb debian /usr/sbin isnt in path
idk if thats a proper solution but you might want to prepend /usr/sbin to path
almost
idk what happened
also dont mind the font i was bored lol
ooh its trying to mount vioblk0 but its running as nvme
uhh change nvme in the qemu to virtio-block or whatever it is
ill try nvme
or oyu can just change the entry in limine.cfg too
yaay
i want to do a simple procfs for uptime and cpuinfo
so neofetch isnt as sad as it is right now
alright I added an option to pass the sysroot to the script
so it could theoretically be called from the Makefile
I'll add the option for the disk size
can you also```diff
- fakeroot << EOF
- PATH="/usr/sbin:$PATH" fakeroot << EOF
or something similar
so it doesnt fail on debian and derivatives
I will
thanks
also i noticed that booting off nvme is so much faster than booting off initrd lol
i like that
yeah booting off disk isnt that slow (thanks pagecache)
theres a reason why I've been doing that for months now
also I think I shouldn't regenerate the initrd and disk image on every run of make
so I will just add a message telling them to run a make target for it
jinx uses rsync to sync the sysroot
so it shouldnt be that slow
unless you nuke it between runs 
but yeah creating the image might be tiny bit slow
cant work on anything
i literally stare at my screen blankly
dont know what to do
if you need me to test anything im around
but im not able to code
I'm just redoing a bit of the main makefile
alright so regenerating the initrd is a manual command now
and make only creates it on the first activation for each distrotype
are there any more packages that could be interesting to put here
neofetch
alright I think the main makefile is done
added
based
what's going on?
i wanted to do 2 things later today, though i think i only have time for half
those being adding post-install scripts and packaging to Jinx and bumping all of the Astral packages, but frankly it sounds like the latter is a lot of work that i have to allocate like, a day, for
I've just been redoing a bit of the main makefile and adding targets to generate an ext2 disk image so now anyone can just run it from there
that'd be neat
i dont think we even need post install hooks
just make the packaging process use fakeroot
and the generated package will have the right permissions
when installing without root you cant do proper permissions/ownership anyway
and when installing on astral it will just work™️
pls use xbps for packaging 


yes
yes to what
YEEEEEEESS
will you do so name bullshit? or do you just assume the person writing recipes isnt stupid
then i guess i'd make some sort of xbbs equivalent for jinx, but idk if i'm gonna handle a web view beyond like, fucking apache/nginx showing the list of files in a directory lol

soname bullshit is annoying
you can probably take a look at how xbps-src is doing it
after that's done jinx is basically better xbps-src
don't they have a literal list of libraries maintained manually
i have no idea lol
i know that xbps-src before calling xbps-create looks over all the binaries and collects the shared libs its using
the deal with sonames is that say you have library X with 100 dependencies, you only wanna bump the dependencies revision if there is a change in the library that warrants doing so
its more so you can prevent breaking packages when installing/updating/removing
because each package knows what libraries it needs
so if you break that xbps detects that and warns you/prevents you from doing that
no that's silly, just use normal dependencies for that
@robust geyser I pushed the new makefile and scripts
the main issue with sonames is ^
see if it works for you
sure
like sonames aren't even an issue if you like
have a jinx command to automatically bump the revision of all dependencies of package X
then all you have to do, as packager, is make sure you run said command if there is a breaking change of the library
at least as far as i understand the issue, here
damn based
they worked it seems?
it also has networking by default now so if you run netd vionet0 it should set everything up
yup it did indeed work
epic
also @grand shadow distro-files should probably create like $dest_dir/usr/{bin,lib} and symlink $dest_dir/{bin,lib} -> usr/{bin,lib}
currently you have a broken filesystem layout
a few files in /bin
and the rest in /usr/bin lol
i'll fix all those things once i get around to do an overhaul as i mentioned above
no its just vim and bash
alrighty
ah fair mint
czapek@raptor-wsl:~/new-sources/Astral$ ls minimalsysroot/bin
getsubids login sh su vi
but yeah if mint fixes it that's fine 😄
@analog berry @carmine swallow revamped the makefile, theres also a disk target now to run from disk
should make testing much faster
damn thats actually so cool
why do you have to type in the password every time?
do you have any idea?
probably some pam shit or whatever
probably, I have no idea how sudo does that
iirc pam yeah
i may just go for a custom package format instead of xbps tbh

i've taken a look and i think tighter integration between jinx and a jpm is better
where we have more control over the packaging format and whatnot
it will be such a pain when you decide to randomly break all packages lol
because you have to go and repackage everything
im talking about the package format
not a big deal, it's literally just 1 rebuild
... of the entire world
yeah
oh boo hoo
talking of 1h of my life
1h that could've been spent in bed
distro world rebuild and chill
plus ideally it'd run on a server somewhere
hot
i think jinix had a thing that literally checked if installed packages have any files in /{bin,lib}
did you backport that to gloire/vinix/whatever is the template now?
boot up factorio in the managarm vm and play it while the world builds smh
worth it for glxgears
also i understand that "tighter integration" between jinx and package manager, i guess i cant do much to change your mind
but xbps is pretty flexible when it comes to packaging
i just wish mouse wasnt broken in qemu on wsl
wdym broken
ill show when astral builds
basically i cant move my mouse inside the qemu window
it only moves when
uhhh
so this is a crazy theory and idk how that works at all
but i assume somewhere in qemu source code it does guest_mouse += mouse_delta
whenever it gets the mouse move event from wayland/sdl/whatever
the mouse only moves if i click on the window to capture it
wait what qemu version are you using
and it moves by (presumably) the mouse delta at that time lol
iirc there was a ps2 bug at one point in qemu
because i can move it by quickly moving and capturing
bro is using the ps2 mouse that only polls when you click
hnn it should have been fixed by then
and qemu 9.0.2, nice
I remember the old wsl version I used to test astral had that bug and I had to build qemu from source
I dont build clang so it should be more bearable
llvm for me it's like 10 minutes
with all targets and clang and lld
idk if astral has those enabled
I dont
build() {
cmake \
-GNinja \
-DCMAKE_TOOLCHAIN_FILE=${base_dir}/util/cmake \
-DCMAKE_INSTALL_PREFIX=${prefix} \
-DCMAKE_BUILD_TYPE=Release \
-DLLVM_LINK_LLVM_DYLIB=ON \
-DLLVM_ENABLE_FFI=ON \
-DLLVM_ENABLE_EH=ON \
-DLLVM_ENABLE_RTTI=ON \
-DLLVM_DEFAULT_TARGET_TRIPLE=x86_64-astral \
-DLLVM_HOST_TRIPLE=x86_64-astral \
-DLLVM_TARGETS_TO_BUILD="X86" \
-Wno-dev \
${source_dir}/llvm
ninja -j${parallelism}
}```
I dont use them and my poor computer 😭
yeah that's gonna be like 5 min
if only my cooler was able to cool down this 300W space heater
bro
just run it with 253W power limits
update the BIOS and enable intel default settings/extreme
it still struggles and is constantly at ~100c
2nd most likely
i have a 13900KS and at 253W power limits the cooler i have is mostly able to handle it
it does get close to 100c but not like, throttling all the time
it's otherwise around like 95c
all core?
that's not bad but also that means you have MCE on
my 13900KS does 5.0p/4.0e all core
i do wonder how the hell that happens tho
possibly bad cooler mouting too
😵💫
LMAO how
good software
well considering that i get good scores on benchmarks but llvm build is a bit slower than expected im gonna blame it on the fact its running in a vm (wsl)
but its done now, around 10 minutes
czapek@raptor-wsl:~/new-sources/Astral$ stat builds/llvm/build.ninja | grep Birth
Birth: 2024-09-04 05:04:12.768382830 +0200
czapek@raptor-wsl:~/new-sources/Astral$ stat pkgs/llvm/usr/bin/llvm-ar | grep Birth
Birth: 2024-09-04 05:11:03.328264255 +0200```
6:51 i guess
i guess if it wasnt throttling as much it would be faster
also if it was ran natively
i am laughing like an idiot at 5 am because i remembered this
when was the font changed btw
ah
i have that locally
it looks nice, what is it? it looks like terminus
oh wtf
ISO.F16 from https://github.com/viler-int10h/vga-text-mode-fonts
its under ibm pc something iirc
also the mouse bug + ext2 panic
I think my ext2 driver might not like the new mke2fs too much
I'll debug it tmr its 00:24
no worries, gn!
Hmm maybe I am the weird one for staying up until 3-5am debugging stuff....
are you handling block number 0?
block number 0 is just always all 0s, and when writing you replace it with a real block
and looking at the code it looks like you got a block number 0 and just bail out on it
(ig you could also create build.h somewhere and add the dir its in as an include dir or add the actual build dir as an include dir) but I didn't bother because its only used within sys/syscalls/uname.c anyway
why
oh right because Astral can't build on its own for some reason
not a Jinx issue, but an Astral build system one
Ohhhhhh I am not
Yeah that is probably it
about sudo: i think its available without a password, yes
Wont be needed anymore anyways lol iretq opened my eyes about fakeroot
it runs in a vm anyway that gets destroyed afterwards so there is nothing secure in there
the alsa stuff is really annoying
like why does the kernel have to have a constraint solver for the different parameters like buffer size, sample rate, channels, chunk size, times in us between irqs and whatever else, why couldn't it just report the supported ranges and then it would be up to userspace to choose whatever exact values it wants from the actual hw params and then calculate the rest of the parameters based on the chosen ones
that code alone is almost the size of the hda driver now lol
what if there is more than one userspace program running?
also its not like having userspace do it is easier you are just moving complexity somewhere else
alsa is rarted
the good audio systems are OSS as it evolved on BSD and Boomer as it evolved on Solaris
alsa only allows one program to access it anyway at a time when using the kernel impl (and not proxied through eg. pulseaudio or whatever)
and why do i need to have a use case for my audio to work
ah lol
ALSA is NIH CADT
ig I could just implement the oss interface instead, I know that its simple enough because I did that (partially) for vinix
My plan originally when I was looking into audio stuff was oss too
only annoying thing about it is the build system, but ig it can be fixed
I can look into having it ported
Ive had to deal with quite a few stupid build systems already
Though I wont be home for a few more hours
most audio-capable apps are gonna want pulseaudio, and pulse probably wants ALSA
it may have support for OSS, but I've never tried
I just remembered I have an xedit port
Does that even work
I dont think I ever tested it
it does
oh really? interesting
it runs on at least four BSDs without problem
pipewire >
ofc
but pipewire seems like it would be even harder to port
now I got the queue stuff kinda implemented, I am not sure what that pause at the start is caused by tho, either its some kind of skill issue within the code or alternatively the thread that's supposed to fill the buffer isn't being scheduled fast enough after being unblocked by the irq handler
well there is supposed to be a little delay before it starts but after it played a little there is a period with nothing in it that then gets filled after that ig
I can try, Ill just have to first switch the sample rate manually because you are supposed to use ioctls to do that
yes
there wasn't that weird delay either, I wonder if it has something to do with /dev/urandom
absolutely not
Holy shit thats so cool
also one thing which I didn't yet figure out is how do I create a symlink in the devfs? I guess with vfs_link but what am I supposed to pass as the linkref vnode?
Its in the comments in the impl
In vfs.c
I dont think devfs supports symlinks tho
yeah but idk what this means in both cases, linkref and linkpath describe the location of where to create the new link I mean the linkpath is self-expanatory ig, is linkref the folder where its created in?
Linkref is like, start the lookup from this vnode
So ../../etc/link from the reference vnode
ah so it would be the devfs root in this case
Yeah
(assuming that it supports links)
Type = V_TYPE_LINK
Linkref = devfs root
linkpath = viovlk0 or pts/0 for example
destparh = contents of symlink
looks like it indeed doesn't, the link functions just point to devfs_enodev
yes, looks like it works (I copied the functions from tmpfs and changed devfs_create to accept link type, added the link member to devnode_t and added the cleanup logic from tmpfs to devfs_devfs_inactive, I assume that the dir cleanup logic should also be there?)
Wait did I forget to destroy the table
In inactive for dirs
Dang seems like I did
now all of this should in theory™️ work with multiple outputs too, it switches the output to wherever you write data to (and if there is currently a thread writing data to some output it will wait until that write is done before switching)
idk if this is actually what oss should do in case there are multiple exclusive outputs (and not all of the outputs necessarily are exclusive, I just didn't write the code to check that yet at least)
Im gonna be home in a bit, will you need the userspace oss stuff soon?
it would be nice to test the ioctls yeah
What exactly is the package you need and what it does so I can have some idea of what to expect
there is literally a package that's called oss that contains some utils like ossplay and the oss header but the build system is kinda cursed for cross compiling (though it can be made work like it was on vinix)
and maybe something like sdl compiled with oss support could be nice because it likely uses more of them (and some game or whatever that uses sdl, I think you do have something ported already)
Ahhh
Ill look into ir
And yeah I have tyrquake
Tho audio there was weird
Ill look into it further
I think it kept segfaulting wanting some mlibc shm stuff?
No idea
But maybe with sdl itll be better lol
oh shit I still have to fix the ext2 bug
or more like
implementing sparse file support
if I even want to bother
I probably do
yes yes u do
I don't think its that hard either? at least in my old kernel when I implemented ext4 (without journaling or whatever) it was literally just one if to check if the block is more than the extent's block + length and if it is then memset zero
for read that is, idk for write
yeah I think the only place I'll have to change anything for now is in my rwblocks() and maybe in the resize when I want to support something like that
just a
if (block == 0 && read) then memset 0 else if (block == 0 && write) allocateblock()
oh its literally just pretty much what my vmm does when mmaping anon pages lol
map a zero'd page and then copy on write if needed
Basically yes
pushed the fix (aka r/w of sparse files)
now onto building oss
I built sdl2 with oss support already
it needs sys/soundcard.h tho, idk how would it properly enable the support if that's not present
I'll base it out of the vinix recipe and see what I can do
thanks 🙏
bruh how the fuck does this build script manage to be worse than the likes of tyr-quake or even xscreensaver
probably it wouldn't be completely unreasonable to just write a new better build script for it that would install the header and build the utils (or even just the header for now)
seems like it generates the makefiles in srcconf.c, I think I will do a funny and just pass some envs down to the conf program
idk this build system fucking sucks
that was fucking painful
this is held together with invisible duct tape
I now know why mint just nuked it out of the vinix tree
i think mint nuked it because the build system is shit and we had troubles building it on aarch64
at least thats the only "bad" thing i remember with this package
I had to do some bullshit like this
build() {
ASTRAL_CC=x86_64-astral-gcc \
ASTRAL_HOSTCC=cc \
ASTRAL_CPLUSPLUS=x86_64-astral-g++ \
ASTRAL_CFLAGS="${CFLAGS}" \
ASTRAL_LDFLAGS="${LDFLAGS}" \
ASTRAL_ENDIAN=LITTLE \
ASTRAL_OS=Astral \
ASTRAL_ARCH=x86_64 \
${source_dir}/configure --enable-libsalsa=NO --only-drv= --target=astral-x86_64
CC=x86_64-astral-gcc make -j${parallelism}
}
// ASTRAL NOTE:
// EVIL HACK EVIL HACK
// due to the build system being HORRIBLE, we will just manually define here the variables that will get passed down to the Makefiles.
// these will get defined in the jinx recipe
fprintf (f, "CC=%s\n", getenv("ASTRAL_CC"));
// fprintf (f, "LD=ld\n");
fprintf (f, "HOSTCC=%s\n", getenv("ASTRAL_HOSTCC"));
fprintf (f, "CPLUSPLUS=%s\n", getenv("ASTRAL_CPLUSPLUS"));
#ifdef VXWORKS
vxworks_genheader (f, path);
#endif
fprintf (f, "CFLAGS=%s\n", getenv("ASTRAL_CFLAGS"));
if (*conf.ldflags != 0)
fprintf (f, "LDFLAGS=%s\n", strcat(strcat(conf.ldflags, " "), getenv("ASTRAL_LDFLAGS")));
sprintf (tmp_endian, " -DOSS_%s_ENDIAN", getenv("ASTRAL_ENDIAN"));
fprintf (f, "OSFLAGS=%s%s\n", conf.OSflags, tmp_endian);
fprintf (f, "OS=%s\n", getenv("ASTRAL_OS"));
fprintf (f, "ARCH=%s\n", getenv("ASTRAL_ARCH"));
why cant you just defined those directly
to be able to do it from the jinx recipe
you could have just CC=x86_64-astral-gcc make
it didn't get passed down
idfk it generates the makefile in a super fucked up way
it runs a C script to create every makefile
fprintf (f, "CC=%s\n", conf.ccomp);
it works this way and at least gives me some more control over wtf is happening
it's not a hack if the build system itself is a hack
😎
that is true
and considering how like
the makefiles are just
# Makefile for Open Sound System module ossinfo
CC=x86_64-astral-gcc
HOSTCC=cc
CPLUSPLUS=x86_64-astral-g++
CFLAGS=-O2 -march=x86-64 -pipe -mtune=generic
OSFLAGS=-Wall -DOSS_LITTLE_ENDIAN
OS=Astral
ARCH=x86_64
TOPDIR=../..
OBJDIR=$(TOPDIR)/target/objects
TMPDIR=.
MODDIR=$(TOPDIR)/target/modules
BINDIR=$(TOPDIR)/target/bin
LIBDIR=$(TOPDIR)/target/lib
SBINDIR=$(TOPDIR)/target/sbin
OSSLIBDIR="/usr/lib/oss"
THISOS=kernel/OS/Linux
CFLAGS+=-O
OBJECTS=$(TMPDIR)/ossinfo.o
TARGETS=$(BINDIR)/ossinfo
DEPDIR=$(BINDIR)/
CSOURCES=ossinfo.c
all: $(TARGETS)
lint: lint_sources
dep: $(AUTOGEN_SOURCES) dep_local
include $(TOPDIR)/make.defs
ossinfo: $(BINDIR)/ossinfo
$(BINDIR)/ossinfo: $(OBJECTS)
$(CC) $(CFLAGS) -s -o $(BINDIR)/ossinfo $(OBJECTS) $(LIBRARIES) $(LDFLAGS)
$(TMPDIR)/ossinfo.o: ossinfo.c
$(CC) -c $(CFLAGS) $(OSFLAGS) $(LIBRARIES) $(INCLUDES) ossinfo.c -o $(TMPDIR)/ossinfo.o
clean: clean_local
this lets me have DIRECT control over how it builds
which is epic I think
the only thing I dont have control is the os.h part
yeah it's not that bad
I'm not going down that rabbit hole
@carmine swallow I pushed oss
tell me if youre able to build it
builds for me