Astral

[grand shadow]

I will better the tlb shootdown soon

[grand shadow]

hm nevermind this is something I will have to sit down and think about and probably rewrite a function or two in the vmm for

and I am not in the mood today to think that much

well hopefully I think of something that doesnt suck soon

because smp is pretty much unusable from how much my shootdowns suck 💀

[robust geyser]

i wonder if what i experienced was due to tlb shootdowns

i tried running with 8 cores

[grand shadow]

the black screen was related to tlb shootdowns but wasnt because it was slow

[robust geyser]

i could move the mouse around but fvwm did not launch

[grand shadow]

it was actually a bug in one of the timer abstractions

[robust geyser]

ohh

[grand shadow]

it kept trying to schedule into a specific cpu to access its apic timer

but that cpu was waiting for the cpu that thread was running on to do the shootdown

[gritty shore]

why dont you just pin threads to CPUs

[grand shadow]

aka skill issue

[robust geyser]

interesting interaction

[gritty shore]

then do thread migration if needed

[robust geyser]

did you manage to fix it?

sounds non trivial

[grand shadow]

yeah it got to fvwm

[grand shadow]

idrk

[robust geyser]

ah nice

is there a test iso available? :^)

[grand shadow]

I can update the bootdisk, the nice thing about separating the root and kernel images is that I dont have to copy a 4 gb file every kernel update

[robust geyser]

that would be nice

i kinda want to try running glxgears with 16 cores or some shit lol

[grand shadow]

I tried glxgears and even then its sloooooooow and the system locked up after a bit

but I will upload the new image

hopefully once I fix the HORRIBLE shootdowns I can go after the rest of the smp bugs

-j8 gcc build 🥹

[grand shadow]

scp finished running it should be there

same link

just need to update the bootdisk.iso

[robust geyser]

yup

yeah it is quite slow with 16 cores 😅

[grand shadow]

sending an ipi for every page unmapped does that

[robust geyser]

even with 4 it's significantly slower than 1 core

ohh

[grand shadow]

being lazy early on is hitting me hard now lmao

[robust geyser]

yeah that makes sense lol

[grand shadow]

I want to hopefully make it go from one shootdown per page unmapped to one shootdown per vmm_unmap or vmm_changemmuflags

idk how feasible that'd be with my current design but it'd be a hell of a speedup

I mean I managed to fit a page cache into my shitty vmm design I can probably get at least something faster going

[robust geyser]

something that wouldn't require too much change is probably one page per shootdown -> one address range per shootdown

[grand shadow]

yeah I was thinking of that

since I do have specific functions to destroy or change the mmu flags of a specific range

[gritty shore]

how hard was llvmpipe to port btw?

[robust geyser]

yeah that's perfect, but keep in mind that the functions to unmap/remap single page still need to shootdown, and if your functions to unmap/remap a range call out to the single page variant dont forget to have some sort of flag to omit shootdown 😅

[grand shadow]

with some help from vinix (I wouldnt even know where to being to patch it) the biggest things I ran into were a bug(?) in gcc's build system, mesa not installing some .pc file and having to implement mprotect which I didnt have implemented

the hardest part was waiting for llvm to compile

and having to do it several times because I did oopsies

[gritty shore]

oh yeah fuck

compiling llvm

[grand shadow]

yeah I think this weekend I might look into that further

having smp be faster than single core would be a cool start even if unstable as shit

[gritty shore]

what alternative is there any way

[grand shadow]

the non llvm renderer

[gritty shore]

softpipe I guess

[robust geyser]

i might try and implement an xhci driver (again)

for astral

that would be cool too

[grand shadow]

oh and zlib not setting SONAME by default unless you patch the configure script

[grand shadow]

true that'd be cool

it'd probably work better on real hardware than ps2 if you write a HID thing lmao

[robust geyser]

i only ever implemented the hid boot protocol

and my keyboard doesnt even implement that properly

if you havent looked into usb hid, boot protocol is basically a standard that is the bare minimum for input devices used during boot

[grand shadow]

lets see how long glxgears lasts with 8 cores

[robust geyser]

mainly for the firmware setup and stuff like that

really easy to implement if you dont need all the bells and whistles

[grand shadow]

15 seconds, pretty cool

thats 15 seconds more than 0

[robust geyser]

damn that was fast

for me it lasted at least 30

i just ctrl-c because it was slow as balls

[grand shadow]

I dont even implement a lot for keyboard/mouse anyways

[robust geyser]

40 fps or so

[grand shadow]

if I moved the mouse it ran faster which ??????????

smp is that weird right now

[obsidian pivot]

What happens, does it crash or something?

[grand shadow]

deadlock

[obsidian pivot]

Of the entire kernel?

Damn

OBOS usually only deadlocks on tlb shootdowns

[grand shadow]

well the kernel still works its just that someone is waiting on some mutex and that deadlocks

like all cpus are on the idle thread

[grand shadow]

shootdowns my behated

what do you do for shootdowns?

[obsidian pivot]

Setup packet with cr3 and page

Send IPI to all cpus but self

[gritty shore]

to avoid shootdowns

couldnt you

have a pagetable per core

[obsidian pivot]

They are one of the highest priority irqs

[gritty shore]

and ensure all threads of the same process run on the same cpu

bam

no shootdowns

I have solved osdev

[obsidian pivot]

What about idle threads

[gritty shore]

what about them

[obsidian pivot]

For the kernel context

Do they all just run on the same cpu

[gritty shore]

you have a kernel pagetable per core

wait no that wouldnt work

yeah youd need kernel threads all on a single core

which sucks

[obsidian pivot]

just keep the cr3 of each core in its cpu_local struct

then loop over each one and for each cr3 that matches the one in which you modified the page table

send an IPI to it

[gritty shore]

yeah that's what I thought of, but you'd still need shootdowns because if a kernel thread remaps memory you need to propagate it

[obsidian pivot]

otherwise go onto the next cpu

[gritty shore]

yeah so you still need shootdowns atleast for kernel threads

and having all threads of a task on the same core could cause issue with load-balancing

[grand shadow]

ah

[grand shadow]

I have solved load balancing (any thread can run on any cpus at any time)

[gritty shore]

yeah I suppose you could avoid many tlb flushes if you dont share the same pagemap

[obsidian pivot]

https://github.com/OBOS-dev/obos/blob/master/src/oboskrnl/arch/x86_64/map.c#L234

are you guys saying that each time you do a TLB shootdown, you mindlessly reload cr3 to flush the tlb

or smth

or is this just a terrible misunderstanding on my end

[obsidian pivot]

I really gotta refactor this file

contains all mmu stuff

[grand shadow]

I do the invlpg on all other cores

its just that mine sucks cuz I do that for EVERY unmapped page

[gritty shore]

nah I just do invlpg

[obsidian pivot]

I do it each time a page's prot is modified

whether that be the page is unmapped, or the prot bits change

but it's one page at a time

which truely sucks

obvious solution is to stub munmap and mprot so that you never have to do a TLB shootdown

[grand shadow]

REAL

[obsidian pivot]

didn't you not have mprot until now

like a day or two ago

[grand shadow]

yeah lmao

never needed that

until llvmpipe started to run code like that

[obsidian pivot]

I needed it for my driver interface

in the driver loader

in the elf laoder

*loader

[grand shadow]

I think thats a TODO in my elf loader lmao

or I just manually remap calling the arch_mmu_remap directly

I forgor

my elf loader just uses mmap anyways

demand page in ftw

[gritty shore]

my todo for the elf loader is mmap the file direclty

because the ubsan messages annoy me (I write at address 0, which is mapped)

[obsidian pivot]

my driver loader's elf loader just takes a buffer with the file directly

so you can mmap the file if you'd like

[robust geyser]

did you enable mlibc to mmap the shared libraries?

[grand shadow]

I did yeah

[robust geyser]

nice

thats my long time goal of forever

i need to succeed

[grand shadow]

• internal_conf.set10('MLIBC_MAP_DSO_SEGMENTS', true)
• internal_conf.set10('MLIBC_MAP_FILE_WINDOWS', true)

[gritty shore]

what do you need for that anywyay

[grand shadow]

idk if I'm missing any

[robust geyser]

vfs too hard

[gritty shore]

mmap file?

[robust geyser]

vmm too hard

osdev too hard

[grand shadow]

yeah

[gritty shore]

ok not bad

will be trivial with my tmpfs

[grand shadow]

bonus points if page cache

[robust geyser]

if you dont set those options, basically, mlibc just mmaps anon memory then reads the file in

[gritty shore]

no need, my fs is a page cache

[grand shadow]

in astral tmpfs IS the page cache lmao

all tmpfs data stays in the page cache

[gritty shore]

interesting

I think I'll do that too

eventually

[grand shadow]

yeah I noticed that poking at the meson.build

[grand shadow]

there seems to be another one in managarm's config

internal_conf.set10('MLIBC_MAP_DSO_SEGMENTS', true)
internal_conf.set10('MLIBC_MMAP_ALLOCATE_DSO', true)
internal_conf.set10('MLIBC_MAP_FILE_WINDOWS', true)

wonder what that one does

[proper meteor]

this project progresses so fast

[grand shadow]

I think for the shootdown I wont have to change too much code and would be able to make it work once per unmap if I do it something like this:

Make mappings inacessible but keep the PTEs with the pages, and have the change function return a list of pages of sorts that has either the pages to shootdown or says to shootdown the whole tlb if its too big of a range
Do the tlb shootdown
Go through the PTEs again and release the pages like I would normally, knowing that if any other core tried to access it they would have to go through the page fault handler first and get a segfault instead of doing a use after free

It could still be optimised further (requires going through the range twice) but for an initial thing that wont require that much change its good enough

[grand shadow]

Also ty

A mesa port was long overdue anyways

[wild marsh]

MAP_DSO_SEGMENTS means file mmap is used to map in shared library phdrs (instead of anon mmap + memcpy), MMAP_ALLOCATE_DSO means mmap is used to allocate the address space that shared libraries use (instead of a fixed counter), MAP_FILE_WINDOWS means mmap is used to map files internally (currently only for /etc/localtime iirc)

[obsidian pivot]

what if a page gets mproted from RW to RO

then it won't pf on other cores

[grand shadow]

Wdym

[obsidian pivot]

nvm

I think I misunderstood

[grand shadow]

Does MMAP_ALLOCATE_DSO still have a library base or so I have to pass that too

I might turn it on later, I already have the other two on

[wild marsh]

MMAP_ALLOCATE_DSO does: ```
max_addr = 0
for each phdr with p_type == PT_LOAD:
max_addr = max(max_addr, p_vaddr + p_memsz)

max_addr = round_up(max_addr, page_size)

library_base = mmap(NULL, max_addr, PROT_NONE, MAP_ANONYMOUS, -1, 0)

it's only for figuring out where to place the library (and reserving the address space)

you most likely want MMAP_ALLOCATE_DSO in conjunction with MAP_DSO_SEGMENTS, but it should work on it's own as well

the main benefits are aslr and that libraries won't collide with anything that might just happen to be in the way

[grand shadow]

Ohhh

I see

[neon crane]

Yeah you need to do a TLB shootdown everytime you constrict the permissions of a page

[grand shadow]

Yeah

That was just for the unmap

[fleet osprey]

If I'm lucky I might test out that build today

[grand shadow]

and so it begins

and it does work for files owned by the euid

now I just need to draw the rest of the owl

[fleet osprey]

Nice!

[grand shadow]

su - doesn't run cuz of this hm

after I finish implementing all those different permission checks in astral I'll implement {set,get,end}usershell in mlibc

[grand shadow]

implemented checks for a bunch of stuff but still got a while to go

[grand shadow]

lmao oops I had forgotten about the permissions for /dev /tmp /dev/shm

so my /etc/rc created them with the wrong ones and the user couldnt write to /tmp for example

now they have the right ones (755, 1777, 1777)

[grand shadow]

you can have ptys now as an user too

[neon crane]

very cool

[grand shadow]

porting startx would be nice eventually, so you wouldnt have to be root to start an X server

I actually got it to run X.org and xeyes but it complained about euid not being 0

and twm didnt work idk why

[robust geyser]

can you type fg to make cat foreground again?

[grand shadow]

yeah

[robust geyser]

damn thats cool

[grand shadow]

job control works well enough

best benefit of implementing signals

I have some 6 system calls left to look into, then I will implement some checks for a few ioctls, then I will implement the mlibc stuff that stops su - from working

[grand shadow]

doing login -p mathewnd on an xterm allows for me to run stuff like tyrquake just fine

these are the only remaining ones for me to look at

-rw-r--r-- 1 mathewnd mathewnd  917 Aug 21 19:49 mprotect.c
-rw-r--r-- 1 mathewnd mathewnd 2196 Jul 17 21:05 renameat.c
-rw-r--r-- 1 mathewnd mathewnd 1235 Jul 17 21:05 unlinkat.c

and theyre the more complicated ones

after I'm done with credentials and the mlibc thing its gonna be fat filesystem and then an installer

actually idk too much about the installer yet

cuz either it gets from the internet or from the initrd

[analog berry]

does mesa work now?

[grand shadow]

ya

speaking of mesa, I should compile xscreensaver with glx support now

[analog berry]

niice

[grand shadow]

pfft

[obsidian pivot]

assertos strikes again

[grand shadow]

__ensure(!"Not implemented") is the ultimate punishment

I think mlibc missing the %f on scanf is fucking me over

your os is cool but does it have SCREEN SAVERS??

[regal stone]

is that dynamically rendered?

cause holy shit

that’s awesome

[prime juniper]

Nice work

[grand shadow]

yeah

mesa + llvmpipe

[regal stone]

wowzers

[proper meteor]

‼️

[obsidian pivot]

soon™️

[grand shadow]

redoing the vfs locking scheme so the locking is done at the vnode layer instead of the filesystem layer (I will have to spend 10 days debugging deadlocks)

[fleet osprey]

nice

[grand shadow]

boots to xorg finally

still has some rough edges though

[robust geyser]

u should add /proc/meminfo and /proc/cpuinfo

so neofetch can display those 😁

[grand shadow]

maybe I can add procfs one day....

also holy shit I just noticed

GPU: llvmpipe (LLVM 18.1.7, 128 bits)

thats news

[fleet osprey]

Lol

[grand shadow]

Got it to run configure but it deadlocked on make -j8 (no smp just trying to see how many processes at the same time goes)

Tmr Ill deal with that

[neon crane]

gotta love the Low Level Virtual Machine pipe.........

[wild marsh]

beats having a soft( )pipe

[tawdry edge]

I think you can get blue pills for that

[fleet osprey]

Btw, just tried to do a build of Astral to try to implement cpuinfo and memonfo just for the hell of it, and the latest code on GH doesn't seem to actually build
Will post the error (and try to investigate in a sec)

In file included from arch/x86-64/main.c:22:
/build_dir/builds/astral/include/x86-64/arch/ps2.h:17: note: this is the location of the previous definition
   17 | #define ECHO 0xee
      | 
[CC] arch/x86-64/smp.o
[CC] arch/x86-64/syscalllog.o
[CC] flanterm/backends/fb.o
[CC] flanterm/flanterm.o
[CC] fs/devfs.o
[CC] fs/ext2.o
[CC] fs/file.o
[CC] fs/initrd.o
[CC] fs/pipefs.o
[CC] fs/sockfs.o
[CC] fs/tmpfs.o
[CC] fs/vfs.o
[CC] io/audio/hda.o
io/audio/hda.c:3:10: fatal error: kernel/hda.h: No such file or directory
    3 | #include <kernel/hda.h>
      |          ^~~~~~~~~~~~~~
compilation terminated.
[CC] io/block/block.o
make[1]: *** [Makefile:33: io/audio/hda.o] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/build_dir/builds/astral'
make: *** [Makefile:25: all] Error 2
make[1]: *** [Makefile:40: kernel] Error 2
make[1]: Leaving directory '/home/techflash/src/astral'
make: *** [Makefile:12: all] Error 2

fully stock clone from https://github.com/mathewnd/astral

[grand shadow]

wait

OOPS

Did I accidentally fucking upstream my nonexistant hda driver 😭

You can just remove the include

[fleet osprey]

👍 alright lol
I figured something had to be up

[grand shadow]

I think at one point I just did git add kernel-src/io and that got added

Might as well just up the hda.h later lmao

[fleet osprey]

and now it craps out trying to build some userspace (I think?)

Making all in lib
make[1]: Entering directory '/build_dir/builds/ace-of-penguins/lib'
gcc -O2 -march=x86-64 -pipe -mtune=generic   /base_dir/sources/ace-of-penguins/lib/make-imglib.c -o make-imglib -lpng -lz
gcc -O2 -march=x86-64 -pipe -mtune=generic   /base_dir/sources/ace-of-penguins/lib/text2c.c -o text2c -lpng -lz
/base_dir/sources/ace-of-penguins/lib/make-imglib.c: In function ‘tokenize’:
/base_dir/sources/ace-of-penguins/lib/make-imglib.c:205:5: error: ‘return’ with no value, in function returning non-void [-Wreturn-mismatch]
  205 |     return;
      |     ^~~~~~
/base_dir/sources/ace-of-penguins/lib/make-imglib.c:199:1: note: declared here
  199 | tokenize(char *string)
      | ^~~~~~~~
/base_dir/sources/ace-of-penguins/lib/make-imglib.c:207:20: error: implicit declaration of function ‘isgraph’ [-Wimplicit-function-declaration]
  207 |   while (*next && !isgraph(*next)) next++;
      |                    ^~~~~~~
/base_dir/sources/ace-of-penguins/lib/make-imglib.c:10:1: note: include ‘<ctype.h>’ or provide a declaration of ‘isgraph’
    9 | #include <png.h>
  +++ |+#include <ctype.h>
   10 | 
make[1]: *** [Makefile:694: make-imglib] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/build_dir/builds/ace-of-penguins/lib'
make: *** [Makefile:385: all-recursive] Error 1
make: *** [Makefile:13: all] Error 2

[robust geyser]

astral audio stack before managarmis actually crazy

[grand shadow]

Wait why is it using the host compiler wtf

Seeing as that is a non nescessary package you can just rm recipes/ace-of-penguins or whagever the name os

[grand shadow]

If you consider audio stack to be just initialising some pci stuff for the controller

I should do a clean build soon so I can fix these

[supple hatch]

Astral is a star and this os deserves a huge star!

[fleet osprey]

oh boy
now it's gonna build LLVM 💀
at least this is one-time, once this is done I can keep recompiling semi-quickly until cpuinfo and meminfo work

[grand shadow]

If you feel like boot times are too high due to the initrd you can change DISTROTYPE to minimal in the makefile and thatll only build some things

Well

Itll build everything

But only some times will be in the initrd

[robust geyser]

i just manually invoked jinx to build the minimal distro

and then make astral.iso run or whatever

or actually make DISTROTYPE=minimal sysroot astral.iso run

i think that was it

[grand shadow]

That works too

[robust geyser]

i feel like you could make it a bit less painful

but it is what it is

[grand shadow]

I really should

[robust geyser]

its a good project, just a bit rough around the edges

i need some time to get used to the code then i can take a look at xhci

[grand shadow]

If you have any questions lmk

[robust geyser]

the code is too densely packed for me to be able to read it comfortably

so i take extra time

[grand shadow]

Or you can just make an ext2 image with the sysroot and tell qemu to use it as nvme or virtio :')

[fleet osprey]

btw you can make this boot way faster on QEMU lmfao

-cdrom emulates an IDE cd

[robust geyser]

i believe its not with -M q35

[grand shadow]

I thought I had passed -M q35

[fleet osprey]

you can do some fancy shit to do virtio cd, and EFI / seabios will boot from it at the speed of light

at least, that's my experience from it

[grand shadow]

Maybe its just in my local machine lemme check

[fleet osprey]

oh nah I see q35
does that change that -cdrom emulates? I haven't used it very often

[grand shadow]

Yeah its much faster

[grand shadow]

If you really want it to be as fast as it can do this and change the limine.cfg

I have a handy script at home

[fleet osprey]

well I still don't even know if it's gonna be slow or not yet, I thought from your comment that you were implying it's going to be painfully slow to boot

probably just misread then, sorry for the confusion

[prime juniper]

Not really. No one is tackling that over at Managarm because everyone has something else they want to do

[grand shadow]

The initrd would still take a tiny bit to load and it just annoyed me when I wanted to make small changes

[fleet osprey]

ah

[robust geyser]

@grand shadow do you use a language server when working on astral?

[grand shadow]

I just use whatever vim version came with the distro and nothing more

[robust geyser]

damn kinda based

[grand shadow]

Idk what a language server is

[robust geyser]

thats the programmer i always wanted to be

but never became

lmao

[neon crane]

yes

if you don't use q35 then by default it'll use IDE hard drives & cd-rom drives

[grand shadow]

Ive mostly only been writing code with vim at home for the past few years

At uni I try to use vim when I can too

[robust geyser]

i wish i could write code without a language server

[grand shadow]

But at work we have to use windows and use online platforms so I cant really

[grand shadow]

What is that

[robust geyser]

it indexes your code

provides autocomplete

live warnings/errors

[grand shadow]

Ohhhhh

That sounds helpful but Ive just trained myself to quickly look at the header files

When I used codeblocks a whiiile back that helped me a bit

And now with the convenience of doing everything from the terminal idk if I can go back to a gui application for osdev

This reminds me

I should write a serial driver soon and hook it to the tty subsystem

[fleet osprey]

that would be neat

DOOM serial match over 2 computers running Astral ?

hmm, dead link, or is my internet being weird? I get NXDOMAIN here if I try to manually resolve it with dig, so unless cloudflare's DNS is having a heart attack, I would imagine the former

make[1]: Leaving directory '/home/techflash/src/astral'
./jinx build-all
* building package: pixman
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:--  0:00:02 --:--:--     0curl: (6) Could not resolve host: www.cairographics.org
make: *** [Makefile:13: all] Error 6 

and it just went away? weird, nvm them ¯_(ツ)_/¯

[robust geyser]

This site can’t be reachedwww.cairographics.org’s server IP address could not be found.
Try:

Checking the connection
Checking the proxy, firewall, and Secure DNS configuration
Running Windows Network Diagnostics
ERR_NAME_NOT_RESOLVED

also using 1.1.1.1

[grand shadow]

Works on my end

Not using 1.1.1.1

[fleet osprey]

yeah try clearing your DNS cache and try again
I just tried again and it started working

guess Cloudflare just had a stroke for a minute or 2 there

[grand shadow]

If I get a source port with multiplayer you can probably play it over the internet

I tried to make tyr-quake multiplayer but it needs more interface memes

[fleet osprey]

oh does doomgeneric strip it out? rip

[grand shadow]

Idk if it works

Maybe it does

Id need.to check

[winged basin]

try doom legacy

that one is famed for its netplay

[fleet osprey]

ooh I'm getting deep into Xorg stuff
if I had to guess, it should be done building soon, and I can finally try to poke at procfs to try to add cpuinfo and meminfo

[grand shadow]

I dont have a procfs lmao

[robust geyser]

does astral have an easy-ish way to hook up input devices?

like mouse/keyboard

[grand shadow]

Yeah

/dev/mouseX and /dev/keyboardX

Mouse.c keyboard.c in io

You can look at ps2kbd.c and ps2mouse.c for an example

[robust geyser]

so i just call newmouse and i can push packets using mouse_packet

[grand shadow]

Yeah

[robust geyser]

easy enough

keyboard seems easy too

[grand shadow]

No support for stuff like unplugging yet but I have support for that in the devfs (becahse of ptys) so implementing it shouldnt be too hard if needed

[robust geyser]

its probably going to be useful down the line

usb is built for hot plug

[fleet osprey]

ah, might try make a tiny stub implementation off tmpfs then that has a partially complete cpuinfo and meminfo
I don't have the time to implement the entire thing

[grand shadow]

Yeah, but I think thatd just end up being a call to like mouse_unregister to remove it from the tree and wait for a mouse_inactive function to call when all references to the devfs node are done

[grand shadow]

Like ctrl c ctrl v the tmpfs?

[fleet osprey]

¯_(ツ)_/¯

I don't really have a plan yet

kinda making this up as i go along

first I need to actually have a working stock build lol

[grand shadow]

Ah

[fleet osprey]

alright, so I now have a working stock build, and am looking at the fs code
I feel like duplicating the tmpfs code isn't really the best idea, probably would be better to try to let it sit on top of the existing tmpfs codebase, since most of the actual fs would be the same
only differences (that I can think of) from tmpfs are that it needs to create some nodes on mount (and then in the final implementation also create nodes for PIDs, but I'm too lazy to go implement that right now), and forbid deleting anything, return EPERM if attempted, and forbid userspace from making new nodes

I feel like making an entire duplicate of the tmpfs codebase just for that is a bit extreme

although actually, now that I think about it

does any of this really need to be "stored"

would probably be cleaner to implement it as a minimal thing on it's own with some nodes that aren't actually stored in RAM, but generated on the fly when read
which would actually be more ideal for meminfo and cpuinfo especially since they have the opportunity to change

[grand shadow]

I do imagine yeah

Just be aware of the refcount and stuff

[grand shadow]

polished a bit more the new vfs locking, configuring and making gnu hello with -j8 doesnt deadlock anymore and e2fsck doesnt scream at me

<----- spent 1h going after a bug that was literally nothing it was just mkfs.ext2 making the filesystem with dir_index without me actually supporting it so it complained

going to finish adding the last few checks to the vfs_unlink and vfs_rename functions as these are more annoying because they need to also check the ownership of whatever they are working with

then I will add an /etc/shells to astral, add the *loginshell functions in mlibc and add an /etc/mke2fs.conf

almost forgot, after adding the mke2fs file I will also add the check for mprotect and signal

[grand shadow]

sticky

oops I need to add a ? EACCES : 0

[grand shadow]

and rename too

[grand shadow]

setuid programs now work too

though I do run into the issue that uh

the sysroot jinx puts out is under my host uid and gid

[robust geyser]

lmao i was gonna ask you what you did about that

[grand shadow]

@craggy tide is there any good fix for that or do I just chown -R root:root sysroot and pray

[robust geyser]

literally that iirc

though

keep in mind

[grand shadow]

I chowned it from astral

[robust geyser]

that some packages might want to install files under different uid:gid

[grand shadow]

yeah

like sshd

[robust geyser]

or some web servers

so yeah idk how to solve that

i wonder if there is some hack to set the uid of installed files to some sentinel value

[grand shadow]

ah shit just remembered chown also clears the setuid bit if youre not root

hm

[robust geyser]

i think there might be no way to solve this without root access at least when installing packages to a sysroot

[fleet osprey]

fakeroot is pretty nice

[robust geyser]

but the problem is that what's root inside the container is the user uid outside

and THAT is the problem

that uid preserves into the initrd and into astral

and there is no way to fix that without root

[fleet osprey]

yeah, so create, and pack the initrd all inside of fakeroot

then the faked root-owned files persist in the final initrd image

they're never actually owned by root on the real fs, but whatever makes the initrd is fooled into believing that they are, and thus, in the image, they are owned by root

[robust geyser]

the only problem now is that you have no way to package stuff which wants a specific gid:uid

and idk how that can be solved

[fleet osprey]

yes you do
you just install it with the regular uid:gid

[robust geyser]

but that pack initrd inside container thing is a good idea

[robust geyser]

ok, then what?

[fleet osprey]

fakeroot won't touch them

it fakes that you are uid 0, any files you create are owned by uid 0, and when read back, are still owned by uid 0 as long as you're in the container

[robust geyser]

no but like... you cant change from 0:0 to anything else inside the container

[fleet osprey]

yes you can, you just run chown on them

[robust geyser]

no, you cant

[fleet osprey]

and then it becomes non-0

[robust geyser]

you cant do that as a regular user

which you are

[fleet osprey]

yeah, but fakeroot fakes that

[robust geyser]

but the filesystem you are running this on does not know that

[fleet osprey]

it hooks libc to essentially fake all operations that require root

[fleet osprey]

correct

[robust geyser]

for all it cares you are still 1000:1000

[fleet osprey]

nothing is ever saved to disk

[robust geyser]

so you cant chown

to anything else

can you?

[fleet osprey]

if you do everything in fakeroot, you can

[robust geyser]

there is this thing currently in jinx

apt tried to chown files

but it fails

exactly because of that

and prints a warning

W: chown to root:adm of file /var/log/apt/term.log failed - OpenLog (22: Invalid argument)

exactly that

[fleet osprey]

oh? interesting

[robust geyser]

you can't chown the file from 1000:1000 to anything else

it appears as being owned as 0:0 inside the container

[fleet osprey]

because fakeroot is specifically designed for debian packaging iirc

so it's weird that apt would blow up on it

[robust geyser]

maybe they have some funky shit going on

it doesnt, its just a warning

but yeah

thats how fakeroot works

nothing you can do about it iirc

maybe debian pckages have metadata that tells the pckage manager how to properly install the pckage

which includes fixing uid:gid

idk

[fleet osprey]

hmm

techflash in ~ at Michael-ArchT14 ➜ fakeroot     
root in ~ at Michael-ArchT14 ➜ mkdir a      
root in ~ at Michael-ArchT14 ➜ cd a                                    
root in ~/a at Michael-ArchT14 ➜ touch balls
root in ~/a at Michael-ArchT14 ➜ chown root:adm balls
root in ~/a at Michael-ArchT14 ➜ ls -la      
total 0
drwxr-xr-x 1 root root   10 Aug 27 18:52 .
drwxr-xr-x 1 root root 1942 Aug 27 18:52 ..
-rw-r--r-- 1 root adm     0 Aug 27 18:52 balls
root in ~/a at Michael-ArchT14 ➜
techflash in ~ at Michael-ArchT14 ➜ ls -l a
total 0
-rw-r--r-- 1 techflash techflash 0 Aug 27 18:52 balls

interesting

works on my machine

[robust geyser]

maybe fakeroot works a bit differently

plain unshare calls dont seem to allow for that

https://github.com/mintsuki/jinx/blob/80b6fbda678735078eccce77d003cda11615ae8b/jinx#L1308-L1708

there's this piece of software that i wrote

that's used for containerization in jinx

maybe it would be a good idea to replace it with bwrap/fakeroot

its essentially homemade bwrap lol

[fleet osprey]

yeah
fakeroot hooks into libc's file related functions and keeps track of all of the spoofed data in RAM until it exits
not sure how unshare does it, never used it

[robust geyser]

it doesnt

unshare is a linux syscall

user namespaces stuff

[fleet osprey]

ah, so that's the difference then

[robust geyser]

yup

that uid:gid mapping to root is also a linux thing

there's /proc/self/uid_map and /proc/self/gid_map

yeah, fakeroot just lets you perform actions as if you were root without actually touching the underlying filesystem

this wouldn't work with jinx where packages are installed into a per-package destination directory

it would require a rewrite of jinx where each package is actually packaged and installed into a temporary sysroot as needed

packaging it inside of fakeroot would work well, and then when building you could install it in fakeroot again which would presumably preserve permissions it was packaged with

[craggy tide]

the way to do this is to have post install hooks

this = have non-root owned files and whatever

the post install hooks would change the owner/perms of files

the hooks should run on the OS itself

[robust geyser]

thats exactly what managarm does, i think

they use xbps

and i believe packages are reconfigured or some shit during boot

rare xbstrap W

[craggy tide]

yes

that's how it has to be done

i still don't know how i wanna do packaging in Jinx

[robust geyser]

xbps!!

it's really nice

you should really look into it

[craggy tide]

yeah

or be the change you want to see :^)

[robust geyser]

eh jinx is too big of a shell script for me to even attempt to look at

i'll pass sorry lol

[fleet osprey]

eh, xbps package templates are kinda pain imo
but when using working packages, it works great

[robust geyser]

what xbps package templates

are you talking about void packages?

https://github.com/void-linux/void-packages/blob/master/srcpkgs/bash/template

like this thing?

[fleet osprey]

yea, with xbps-src
that's like, the main way that you build xbps packages, no?

[robust geyser]

main way to build void packages for sure

this is very void specific

at least i think so

the script is literally in their package repository

it does not come with xbps itself

[fleet osprey]

I'm well aware, I have to maintain custom packages for it, for an architecture that's no longer officially supported

but, in case you weren't aware, xbps is a Void project lmfao
https://github.com/void-linux/xbps

[robust geyser]

yeah i know that as well

[fleet osprey]

of course it's gonna be void-specific

[robust geyser]

xbps itself is not very void specific

but the xbps-src script seems to be

especially the way its lodged in their repository containing all the packages

the only real way to get it is to fetch the raw file out of the repo

which sucks

[fleet osprey]

yes, that's what I'm talking about
I'm not aware of any way other way to make xbps packages though without doing it fully manually

[robust geyser]

its not that hard

except if you want all the soname/dependency tracking things

but ive done it

its like 20 lines of bash

[fleet osprey]

idk, if you have an alternative way to make packages for it then sure

but xbps-src templates are honestly kinda depressing to work with
especially if your package has a non-standard build method

[grand shadow]

:3

[robust geyser]

holy

port sudo now

:D

[grand shadow]

gonna open a pr for *usershell in mlibc now

[grand shadow]

I want to do that soon

[robust geyser]

or doas, might be easier

[grand shadow]

true

[robust geyser]

anything bsd is usually easier lol

[grand shadow]

that will be uh, after I rebuild the world

kinda fucked up an rm command while trying to clear the mlibc source

[robust geyser]

lmao

big rip

[fleet osprey]

yikes

[robust geyser]

is astral using the great new debian based jinx version?

[grand shadow]

still on the old arch version

[robust geyser]

might want to look into that then, while you're at it :p

if you do, make sure to back up your mlibc patches so you dont accidentaly nuke them!

[grand shadow]

thats on git lmao I would explode if I nuked it and didn't have a fallback

I'll leave it building while I go to sleep and deal with it tomorrow, uni break starts soon so I will have more time

[robust geyser]

oh yeah, talking about git

did you try running git on astral?

:^)

[grand shadow]

[robust geyser]

ah, almost forgot

nice job

[grand shadow]

I'll leave the world rebuiling overnight and tomorrow I will finish the last few checks for mprotect and signal

then uh, I think thats it?

add a login option to init and add an astral user too I guess

and see if I can port sudo/doas

oh and add the appropriate chowns to my gendisk.sh script

[prime juniper]

Nice nice

[supple hatch]

oh my gosh

oh my gilly gosh omg

real?

compile ur own os on astral?

[craggy tide]

almost enough to build Limine on astral

you probably only need to port mtools and nasm if you don't have those

[grand shadow]

I do have nasm since I can build the kernel on astral, its just mtools then

[supple hatch]

guys I will use astral on my main machine for a week if you port neovim

[grand shadow]

If you have the drivers xd

For disk theres only nvme

And theres no installer etc etc

Thats coming after fat

[supple hatch]

who uses anything other then nvme these days

[winged basin]

the AHCI is very popular

[grand shadow]

I do wish do write an ahci driver soon, my main computer has an unused ssd connected via sata

And dual booting astral would be very very cool me thinks

[obsidian pivot]

OBOS

[main moon]

obos supremacy

[robust geyser]

@grand shadow it's just stuck like that after i pulled and rebuilt the kernel, any idea? lol

i also rebuilt mlibc, bash and init

nuked the mlibc sources of course

[obsidian pivot]

why is the scheduler loading /init

[robust geyser]

why would it not

[obsidian pivot]

because it's the scheduler

not program loader

[robust geyser]

boo

hoo

:^)

void sched_runinit();

it makes perfect sense to me actually

yaaay sourceware.org is down!

[grand shadow]

No idea

How did you revuild the kernel

Theres a syscall logger you can enable too in kernel-src/Makefile

[robust geyser]

make kernel

i will do that

[grand shadow]

Oh you have to make clean-kernel

I dont have header dependencies

[robust geyser]

☠️

wow

its like it worked

[grand shadow]

I really need to revisit the build system lmao

[robust geyser]

oh yeah, you do

thanks for the help!

[grand shadow]

Like switch to submodules from the jinx regenerate for the kernel stuff and other QoL things

[grand shadow]

Np :3

[robust geyser]

proper kernel build system too, i suppose?

so i dont have to clean-kernel to make sure it doesnt break lmao

[grand shadow]

Yeah lmao

[robust geyser]

thank god

[grand shadow]

It annoys me sometimes too, I change a header file and forget to clean and wonder why its breaking and then remember a second later

But yeah, for today after I rebuild the world (left it building when I left for work, hopefully it didnt crash and burn), I will look into the remaining functions I have to implement the access stuff for and port sudo

[obsidian pivot]

does astral work on any real hw?

it'd be interesting seeing it compile itself while running itself on real hw

too bad it cannot do so with SMP

[grand shadow]

Worked on my main computer and laptop

[obsidian pivot]

interesting

[obsidian pivot]

I'd try it, but compiling it is too much work

(I'm not up to compile gcc twice)

[grand shadow]

I have a bootdisk that has a minimal initrd in my server

[robust geyser]

gcc doesn't even take long to compile

also that

[grand shadow]

https://avalios.dev/bootdisk.iso

It was meant to be used with an nvme image in qemu but it should boot in real hw if you pick the initrd option in limine

[grand shadow]

The main thing stopping me from going after the smp bugs is how slow my tlb shootdown is

But once thats done I can look into it further

8 core gcc compilation 🥺

And once I have sysconf() llvmpipe could also use the other cores

[analog berry]

this reminds me i promised uacpi support for astral

also if it doesnt die before uGPU maybe a drm subsystem as well

since it has mesa ported over now

anyways soon ™️

[grand shadow]

Yeah
I fixed that bug so it should be perfectly fine now

[grand shadow]

Dont think itll die anytime soon I am still pretty hyped about it actually not sucking that much

[analog berry]

thats great

im confident I could get mesa to execute stuff on igpus

since ive done it freestanding before

id just need to implement drm for your kernel

[robust geyser]

that would be giga based

[analog berry]

at least like a dirty POC thing that could run a game

without any isolation or security

[grand shadow]

Hw accel glquake

[analog berry]

yeah as an example

[prime juniper]

If you want to try it, managarm has DRM ready to go (and soon™️ modesetting on some iGPU)

[analog berry]

Does it have mesa ported tho

[grand shadow]

Yeah managarm does

Its even upstream lol

[prime juniper]

Bro we’re in upstream Mesa

We’ve had Mesa for at least 5 years

[analog berry]

Oh lol

I think I want to support both full on drm drop in subsystem as well as an extension to an already existing DRM systems so I need both

[prime juniper]

Excellent

Then I think you have your testbeds

[analog berry]

Ye

[neon crane]

tack on -MMD to gcc and itll generate *.d files alongside your *.o files

then

DEPS = $(patsubst $(OBJS),*.o,*.d)
-include $(DEPS)

and like clockwork

[grand shadow]

Seems easy enough

[neon crane]

I hope it's as easy as I portrayed it 😛

[robust geyser]

it is very easy actually

i am suprised you didnt have that 😭

[prime juniper]

Managarm doesn’t do that either? Unless meson does it for me

[robust geyser]

well no Shit it does bruh

u guys are on easy mode

[obsidian pivot]

I'd rather do userspace developement for windows than use make tbh

I just don't wanna learn it

[robust geyser]

sorry if i come off as rude lol my bad

[obsidian pivot]

nothing personal against make

[robust geyser]

i half dont mean it

[analog berry]

Ofc it does, thats the entire point lol

[prime juniper]

I mean it makes sense

But I never gave it more than 1 second of thought. So thanks for answering that, TIL that those options are already done if you use meson

[prime juniper]

No offence taken, thanks for the answer

[analog berry]

Yeah I mean make is like assembly and meson is like idk, python

[prime juniper]

In terms of things you need to do yourself? Yeah that’s accurate

[craggy tide]

honestly just yoink it from the limine-c-template

[neon crane]

^

[errant zephyr]

meson does some other stuff

because it uses ninja aka the superior build system

[robust geyser]

does ninja itself do dependency tracking? i didnt know that

i thought meson generates a bunch of rules

[analog berry]

Nah I think meson generates it that way

[grand shadow]

why tf did I not

allow for jobctl_setctty to return an error

(it should fail with EPERM if there is already a controlling tty and its not stealing)

[analog berry]

Assert better, prove me wrong

[grand shadow]

welp, went ahead and implemented ctty stealing

since login likes using that

I will probably have to do startwm as a setuid program

since xorg doesnt like it if you run it with euid != 0

[grand shadow]

also pretty much rewriting init from scratch and doing some improvements

well not from scratch but most of it

[grand shadow]

I will want to run most things as an user when I have this working fully

just finishing having startwm work as an user

[grand shadow]

went ahead and reworked a bit the distro files

and I'm done

rootless xorg works

[wild marsh]

very nice

[grand shadow]

I think I am going to add those tests to mlibc now to get the usershell stuff upstream and then try to port sudo or something

[craggy tide]

i wonder how much work it would be to bump all of your packages to sync them with vinix/gloire and use jinx 0.3

[grand shadow]

122 files in recipes/, 126 files in source-recipes/, 15 files in host-recipes/

[wild marsh]

hm on that topic, i wonder whether comments in /etc/shells have to be handled by the user of the function?

the /etc/shells on my system opens with ```

/etc/shells: valid login shells

/bin/bash
...

the manpage doesn't mention anything about that

[grand shadow]

maybe its in the mlibc

yeah

lemme see what su does

[wild marsh]

i guess the first line would not be a valid field value in passwd anyway?

because of the : in the middle

[wild marsh]

interesting

[grand shadow]

but in another file it only does an strcmp

[wild marsh]

glibc seems to parse all the shells when opening the file for the first time, and seems to specifically ignore comments

https://github.com/bminor/glibc/blob/3fc063dee01da4f80920a14b7db637c8501d6fd4/misc/getusershell.c#L133

[grand shadow]

hm

I mean, it wouldn't hurt to do it anyways

[wild marsh]

i think the impl in mlibc should be fine though? musl does not do that for example

[grand shadow]

I love functions that aren't standardized!

[wild marsh]

the musl impl is basically the mlibc impl but more code golfed :^)

[grand shadow]

so what do you think

do I check for them or nah

[wild marsh]

i think it should be finee

can passwd even have non-absolute paths to shells

[grand shadow]

probably not, that'd be bad

[wild marsh]

if someone can write a bad entry they can just create a malicious user anyway

[grand shadow]

yeah

[wild marsh]

so i think the impl in the pr is fine

[craggy tide]

damn you still have separate sources

this sounds like a lot of effort

[grand shadow]

yeah its like, gonna take an entire day of full time effort to do that

[errant zephyr]

nop

meson just tells ninja that the depfile will land at <path>

and thats the end of ther involvement

basically you do this https://git.sr.ht/~pitust/wwce/tree/8426d162165fa1834a3ef3e7c69ee831b6d04f71/item/x#L152

[analog berry]

ah i see

good to know

[grand shadow]

I think today I will port sudo/doas and also implement putgrent and fgetgrent in mlibc

Because useradd dies if you dont pass -N without those

And without them you have to manually create the groups for each user anyways

[grand shadow]

I do wonder how difficult itd be to port something in the likes of wine

Didnt managarm already run it

[robust geyser]

i think it did, to some extent

qwinci worked on that

[grand shadow]

@carmine swallow is there anything super weird or specific needed for wine? I might try my hand at that some time in the future

[carmine swallow]

at least I didn't run into anything weird when porting it to managarm other than needing to add a new os in signals_x86_64.c (or whatever file it is, I don't remember the name of it) which likely just consists of copying the linux code and adding your own call to a syscall that sets the gsbase

well there were funny mlibc signal memes but they are fixed by now

[grand shadow]

That sounds too easy wtf

I already have a syscall for that

[carmine swallow]

also its a good stress test for your filesystem drivers because it writes a decent amount of stuff that's very likely the reason why it takes ages to start on managarm (and sometimes even times out)

[grand shadow]

Could be an interesting way to test my new vfs locking stuff then

Configuring and compiling gcc already weeded out some disk and ext2 bugs

Gonna look into it after finishing the stuff I have planned, ty

[analog berry]

@grand shadow what IDE do u use and how do u set it up

[grand shadow]

vim and I dont

[analog berry]

f

[grand shadow]

I literally just use whatever camr with wsl

[analog berry]

lol ur on wsl too

i thought of booting into linux so i could use a proper ide at least

with astral

but idk maybe over wsl is possible too

[obsidian pivot]

wdym visual studio very normal ide

nvm

[analog berry]

@grand shadow is there like a build lightweight or whatever

so i dont have to build 10 copies of gcc etc

[grand shadow]

Uhhh @robust geyser had some commands he used

[robust geyser]

make DISTROTYPE=minimal clean-kernel kernel initrd astral.iso run-kvm

[analog berry]

lol

❯ make DISTROTYPE=minimal clean-kernel kernel initrd astral.iso run-kvm
find builds/astral/ -name *.o -delete
find: ‘builds/astral/’: No such file or directory
make: *** [Makefile:43: clean-kernel] Error 1

[robust geyser]

lol nice

also before that

[grand shadow]

Dont need clean kernel

[robust geyser]

you need to ./jinx build minimalsysroot mlibc bash coreutils init distro-files vim nano mount netd

[grand shadow]

In thr first run

[grand shadow]

Yeaj

[analog berry]

where did u get jinx

[grand shadow]

make jinx

[robust geyser]

make jinx

[neon crane]

jinx

[robust geyser]

also @grand shadow what is the stance on timeouts in driver init code

[analog berry]

thnaks

[robust geyser]

xhci controller halt should happen within 16ms but i dont want to a) spin b) block the main thread

[obsidian pivot]

omg is infy starting uGPU now on astral

[analog berry]

no im not

[obsidian pivot]

this is a moment for history

nvm

[analog berry]

im just porting uacpi to astral

[robust geyser]

BASED

[analog berry]

lmao

[neon crane]

why dont you want to block the main thread

[robust geyser]

idk it sounds stupid

[analog berry]

are u writing something for astral as well

[robust geyser]

to block everything cuz xhci is initializing

yes

[grand shadow]

Meh it rly doesnt matter

[robust geyser]

so sleep in main thread?

[neon crane]

create a new thread for that I guess?

[grand shadow]

Sure

Nvme spins

[robust geyser]

lets goooo

[analog berry]

/home/infy/Astral/jinx: 865: .: cannot open /home/infy/Astral/recipes/minimalsysroot: No such file

[robust geyser]

🤔

[grand shadow]

Theres no minimalsysroot recipe

Its distro-files

[robust geyser]

did you change that recently?

[grand shadow]

Nope

Always been distro files

[robust geyser]

oh my bad lol

[analog berry]

an xhci driver?

[robust geyser]

i had a brain fart

[analog berry]

damn

[grand shadow]

xhci will be very useful (my ps2 code sucks)

[robust geyser]

also, how to sleep?

lol

sorry

[analog berry]

so i replace that with distrofiles?

[grand shadow]

look at sched.h

[grand shadow]

Yeah

[robust geyser]

its already in the command so

[grand shadow]

Should also add astral there

[robust geyser]

i just copied one elements to the left accidentally

[analog berry]

ah ok

[robust geyser]

void sched_sleepus(size_t us); this what i want?

i assume yes

[grand shadow]

Iyeah

[robust geyser]

perfect thank u

[analog berry]

are there mutexes/events/work queues whatever?

[grand shadow]

Mutexes and events yes

Work queues not rly I think

[analog berry]

i guess i can emulate it with events

and a separate thread

can u pin threads to a core?

[grand shadow]

Yeah

[analog berry]

nice

[grand shadow]

sched_targetcpu then sched_yield

[analog berry]

damn bro astral goated

[grand shadow]

To untarget run targetcpu with null

[robust geyser]

i just wish the code was a little bit cleaner

[analog berry]

and u can make kernel threads right?

[robust geyser]

but its so far pretty good

sched_newthread im p-retty surer

[analog berry]

based

[robust geyser]

that then sched_queue it seems

if (shouldntstart == false)
    sched_queue(thread);```

crazy code

[analog berry]

i find lack of capitalization or underscores a bit hard to read but whatever

[robust geyser]

same

[grand shadow]

I never thought itd reach the point where other people would contribute lmao

[robust geyser]

that is fair honestly

[grand shadow]

Trust me it is better now than old astral
In old astral everything was much more disorganised

Working on lyre for like a month helped fix me

[analog berry]

i belive you

i get this

[grand shadow]

Uhhh you should run on the console

Its one of the limine entries

[neon crane]

I hope I can get to a similar point to your OS at some point

[neon crane]

there's a lot I still need to do to reach that point

[grand shadow]

Its trying to get nvme

[robust geyser]

use the 2nd boot option

[grand shadow]

Is there no initrd option for a console

[analog berry]

ohhh

i used option 4

[grand shadow]

Nit nvme virtio block 💀

[grand shadow]

It seems like youre focusing more on kernel design so when you do reach itll be more robust

[obsidian pivot]

I too, wishO OBOS could get to the point of astral

stupid slowfat driver is slowing me down

[analog berry]

its doing a thing

[robust geyser]

depending on how good resharper is with this project you might need to use clangd + this

compile_flags.txt

[analog berry]

resharper? im using clion

[robust geyser]

yeah

its their language engine bullshit

"resharper c++ engine"

[analog berry]

oh

what else do i need besides compile_flags?

[robust geyser]

provides auto complete, errors and whatever

nothing, it just works as it for me in vscode

though i should try clion

[analog berry]

i see

[neon crane]

I am focusing more on kernel design

[robust geyser]

i got a license so i shouldnt let it go to waste :^)

[obsidian pivot]

can I have it

[neon crane]

Well my plan is roughly like this: first, the kernel, then my own userspace ecosystem, then getting mlibc and the "unix-like" world running in a protected subsystem / compatibility layer / whatever

[robust geyser]

if it wasnt in my name

[neon crane]

I think it'd be pretty cool

[robust geyser]

@grand shadow no way to allocate a dma buffer?

[neon crane]

what kind of buffer do you want

[robust geyser]

xhci wants a bunch of small things, didnt want to have to waste whole pages for it

i guess i could calculate how much physicala memory i need for all buffers and allocate all pages at once, then bump

or throw a bunch of pages at it and call it a day

[analog berry]

@robust geyser so what exactly did you do for vscode configuration?

[robust geyser]

just that one file

[analog berry]

like i opened the project dir, now what

[robust geyser]

along with the clangd extension

i placed the file at project root (next to Makefile)

its not proper language support but it gets the job done

[analog berry]

oh yeah that works i think

[robust geyser]

most notably jump to implementation doesn't really work unless you have the corresponding .c file open

i just takes you to definition instead

its because there is no compile db

its usually generated by cmake/meson

the compile_commands.json file

make astral use meson

[obsidian pivot]

I found some tool that can convert a makefile to a compile_commands.json

https://github.com/rizsotto/Bear

[robust geyser]

bear yeah

but astral is built inside jinx

and i dont want to know, or even try

i have a feeling this would fail terribly

[analog berry]

strange because go to definition does work

[robust geyser]

czapek@raptor-wsl:~/new-sources/Astral$ bear -- make DISTROTYPE=minimal clean-kernel kernel initrd astral.iso run-kvm
find builds/astral/ -name *.o -delete
find builds/astral/ -name *.asmo -delete
rm -f builds/astral.packaged
rm -f builds/astral.built
./jinx build astral
* building package: astral
/home/czapek/new-sources/Astral/.jinx-cache/rbrt: execvp() failure at line 351: No such file or directory
make: *** [Makefile:40: kernel] Error 1```

lmao

[robust geyser]

yep, beacuse they are in headers which it can index (it has the include paths)

but it doesnt know about all the source files, except the ones that are open

[analog berry]

ah ok

[robust geyser]

so jump to implementation will only work with those

[analog berry]

well clion refuses to properly connect to my wsl soo

ill have to use this

[robust geyser]

sadge

but this works okay too

[analog berry]

yeah thanks

@grand shadow soo your acpi stuff currently lives under arch/x86_64 for some reason, mind if i move it out of there?

or at least most things releated to acpi

[grand shadow]

I do that in the virtio net driver

Multiple page allocations are to be avoided BUT during system init Id say its ok

[grand shadow]

Yeah sure idk why I put it there either

[analog berry]

also like

would a drivers/ directory be ok?

maybe iretq can put their stuff there as well

[grand shadow]

put it in io/acpi/

[analog berry]

sure

is io like the drivers?

[grand shadow]

Yeah

Block net etc lives there

[analog berry]

intreresting

[robust geyser]

im just going to allocate single pages for now, simplifies init a lot

[grand shadow]

My pmm is a linked list of struct pages

But I can iterate through them to find ones that are adjacent in physical memory

[robust geyser]

maybe ill come up with a small dma allocator that can be generalized for the rest of the system

[grand shadow]

Could be helpful

[robust geyser]

i need to allocate a bunch of small structures in physical memory, like pointer arrays

so yeah it would cut down memory usage a bit

but its not the end of the world

[robust geyser]

ill look into that maybe too, i like going and improving random things and this could be improved :D

one thing at a time tho

i want to do everything but if i do then i end up not doing much

[grand shadow]

I already do that

Its just not recommended outside of driver init code as I dont really support taking pages from the page cache while doing it

[grand shadow]

seems like sudo wants some mlibc stuff stubbed

[analog berry]

@grand shadow is there no way to sleep from a kernel thread?

[grand shadow]

There is?

What r you trying to do

[analog berry]

i only found this

[grand shadow]

You prepare the sleep then yield to actually sleep

[analog berry]

so i have to call 4 functions to sleep?

[grand shadow]

Wdym 4

Its only 2

[analog berry]

prepare, target, insert, yield

[grand shadow]

insert?

[analog berry]

timer_insert

in the screenshot above

[grand shadow]

Thats for a timeout but yeah

[robust geyser]

i just called sched_sleepus

[analog berry]

otherwise its just infinite sleep?

[robust geyser]

it does everything for u

:D

[analog berry]

oh

[grand shadow]

Target is for the timeout too

[analog berry]

thats a thing?

[grand shadow]

Yeah

[analog berry]

nvm

[grand shadow]

I should prooobably write some docs sooner or later lmao

[analog berry]

did u also forget it existed?

[grand shadow]

I didnt know what you were trying to do

I thiught you wanted to wait on an event with a timeout or something

[analog berry]

i mean idk how u can explain it better lol

[analog berry]

oh

[grand shadow]

sched_preparesleep prepares the thread to sleep
And sched_yield() blocks it

Anything else is just support for stuff like timeouts

[analog berry]

yeah i get that

[robust geyser]

i really like the way it works

[grand shadow]

Useful if you need to release locks and stuff before sleeping but these locks also protect the object youre sleeping on (like a semaphore/mutex)

[analog berry]

that makes sense

[analog berry]

@grand shadow whats the reason that you currently map tables using vmm_map and not MAKE_HHDM?

[grand shadow]

I don't map them when mapping the hhdm

since IIRC their entries aren't guaranteed to be page aligned

so can cause issues

in real hw

[analog berry]

cant u just round down to nearest page

[grand shadow]

I mean multiple entries could be on the same page and I assert the vmm_map that maps the hhdm

vmm_map with VMM_FLAGS_PHYSICAL shouldn't even take that much space anyways

[analog berry]

does vmm_map support non-page-aligned physical offset?

[grand shadow]

kind of

[analog berry]

because i see map_table code doing mental gymanstics

that make it impossible to unmap that table

[grand shadow]

wdym

that function was supposed to be temporary anyways