in obos
#OBOS (not vibecoded)
flint idol
I have an idea
rn it's possible for the kernel to completely go to shit if usermode passes in a dead handle
it'd overwrite the handle table treating it as whatever object used to represent
and since kernel objects are always aligned to 0x10
actually this is stupid
I can just check if the entry is within arr...arr+size
flint idol
actually this is stupid
I was gonna set bit 0 to indicate that this entry is free
but a range check is easier and more reliable
flint idol
and since kernel objects are always aligned to 0x10
in case this ever changes
first syscalls using handles I will make are thread-related syscalls
specifically spawing threads
blocking/readying threads
yeah
signals too
flint idol
I have implemented one half of handle cloning
and have also defined some thread-related syscalls
flint idol
I have implemented one half of handle cloning
as well as one half of handle closing
the other half is whatever the underlying object actually needs to do to get cloned
which is in a per-handle type callback
+extern void(*OBOS_HandleCloneCallbacks[LAST_VALID_HANDLE_TYPE])(handle_desc *hnd, handle_desc *new);
+extern void(*OBOS_HandleCloseCallbacks[LAST_VALID_HANDLE_TYPE])(handle_desc *hnd);
+
+obos_status Sys_HandleClone(handle hnd, handle* new);
+obos_status Sys_HandleClose(handle hnd);```#define thread_object_from_handle(hnd, return_status) \
({\
struct thread* result__ = nullptr;\
if (HANDLE_TYPE(hnd) == HANDLE_TYPE_CURRENT)\
result__ = Core_GetCurrentThread();\
else {\
obos_status status = OBOS_STATUS_SUCCESS;\
handle_desc* _thr = OBOS_HandleLookup(OBOS_CurrentHandleTable(), (hnd), HANDLE_TYPE_THREAD, false, &status);\
if (obos_is_error(status))\
return return_status ? status : HANDLE_INVALID;\
result__ = _thr->un.thread;\
}\
result__;\
})```was fun writing this macro
(I have no idea if it will work as intended, if at all)
it compiles
limber wave
congrats on 21k messages
flint idol
oh wow
flint idol
but now I can do stuff like
obos_status Sys_ThreadReady(handle thread)
{
struct thread* thr = thread_object_from_handle(thread, true);
return CoreH_ThreadReady(thr);
}```instead of repeating the same code
over
and over
again
limber wave
macros are convienient
flint idol
macros are convienient
that is, until you crash within one
xD
I have implemented all thread-related syscalls
as in, ones that manipulate thread objects
flint idol
@real pecan i found something that causes uacpi to allocate zero bytes
I can send you the blob
So you can test it on your end
My kernel api does not seem to be printing the size wrong
This is the same computer with the broken aml on linux
Btw
flint idol
@real pecan
ornate ginkgo
I think it's UB or ID
flint idol
Idk why uacpi would do that
ornate ginkgo
I just return null
flint idol
I just return null
So do I
ornate ginkgo
Since if you access it that's your issue
real pecan
i dont think uacpi allocates 0 bytes anywhere
but thats well defined
just return anything
real pecan
ill investigate if astral tries to allocate 0 bytes as well
flint idol
Discord for mobile is stoobid
Can you send a link to the message with the iso?
In the astral thread
lean glen
I think it's UB or ID
I think it's supposed to return something that can be freed
real pecan
U need a fresh astral iso
flint idol
Frick
lean glen
so whatever malloc(0) returns, free must accept
ornate ginkgo
I think it's supposed to return something that can be freed
Yeah, so null should be fine
real pecan
even if uacpi allocates 0 bytes (which would be a bug and i have guarded against that previously), it would treat null as OOM
which it does in oberrows image
lean glen
Yeah, so null should be fine
yes
real pecan
thats why malloc implementations usually return some placeholder value that is not 0
or do max(1, size)
flint idol
I'd try and debug this but I don't quite have a dev environment on me
real pecan
what happened
flint idol
what happened
I am at my friend's house
lean glen
infy C++ uacpi bindings when
real pecan
btw did your kernel hang after that?
lean glen
or should I make them 
real pecan
you would be very welcome to make first class c++ bindings
flint idol
btw did your kernel hang after that?
No comment.
real pecan
lol
lean glen
you would be very welcome to make first class c++ bindings
alright
I shall
then I will finally port uacpi to my kernel
real pecan
because i see that uacpi did finish loading all of the tables successfully
ornate ginkgo
thats why malloc implementations usually return some placeholder value that is n...
Interesting, good to know.
vale nymph
<@215823874615541761> where can we get the latest astral image
pinned message inthread has one from a few weeks ago
vale nymph
Can you send a link to the message with the iso?
https://avalios.dev/bootdisk.iso initrd option
wait
lemme upload a more recent one
real pecan
well i did try it in userspace but since im using host malloc it didnt break, i havent added a 0 size check tho
i will do that if astral breaks
vale nymph
I never intended for alloc(0) to be used but I think it does work
iso updated @flint idol
vale nymph
I need to redo my alloc.c bruh its nasty
real pecan
#5 0x000055555571adc1 in uacpi_kernel_alloc (size=0) at /mnt/d/uACPI/tests/runner/interface_impl.cpp:225
#6 0x000055555571b346 in uacpi_kernel_calloc (count=0, size=8) at /mnt/d/uACPI/tests/runner/interface_impl.cpp:262
#7 0x00005555557970d0 in uacpi_package_fill (pkg=0x6030003ff3d0, num_elements=0) at /mnt/d/uACPI/source/types.c:125
#8 0x00005555557b9021 in handle_package (ctx=0x624000002100) at /mnt/d/uACPI/source/interpreter.c:799
#9 0x00005555557ea67e in exec_op (ctx=0x624000002100) at /mnt/d/uACPI/source/interpreter.c:5307
it was one of the first things I did when I started new astral and its bad
real pecan
this aml creates an empty package for whatever reason, and its not like its mutable so its literally a useless object
i guess i need to guard against that to not confuse malloc implementations
flint idol
this aml creates an empty package for whatever reason, and its not like its muta...
Isn't this aml restarted
real pecan
yes
vale nymph
ps2?
vale nymph
normal
real pecan
at least it boots
vale nymph
I will write a driver for every NIC on earth so astral will be ssh only
flint idol
normal <:meme:575445094589661215>
It hangs at ps2 init
real pecan
f
flint idol
This computer seems to not have smi
Well not that
But like
The fan starting spinning like crazy
real pecan
wtf is astral doing
flint idol
wtf is astral doing <:meme:575445094589661215>
Probably busy loop
vale nymph
yeah
(totally not mining crypto)
((trsut me))
@real pecan when infy ps/2 driver for astral 🥺
real pecan
soon ™️
real pecan
Isn't this aml restarted
btw it then returns this package as the thermal relationship table
lean glen
<@353593948523986945> when infy ps/2 driver for astral 🥺
does vinix work
real pecan
thats why u saw linux complain
lean glen
just use vinix's ps2 code honestly
flint idol
btw it then returns this package as the thermal relationship table
I like your funny words, magic man
real pecan
this one trick saved obos from dying
flint idol
this one trick saved obos from dying
Ty
real pecan
ill push a bit later, need to make sure it hasnt regressed anything
also add a test case for 0 size package
also add an assertion for alloc(0) since its not something im allowing i guess
flint idol
also add an assertion for alloc(0) since its not something im allowing i guess
Will do
flint idol
Still will do that
real pecan
lol
flint idol
What are the other acpi errors btw? Just obos curse?
real pecan
it aborted DSDT load mid way, so all SSDTs referencing shit from it will not load properly
so its just a consequence of the earlier abort
flint idol
Why doesn't a dsdt abort abort the entire namespace load or init or whatever
real pecan
no reason to do that really
u see even with that abort it loaded most blobs without any errors
flint idol
u see even with that abort it loaded most blobs without any errors
And is there any reason that there are ssdts instead of just compiling all of the blobs into the dsdt
lean glen
this one trick saved obos from dying
i wonder if [[unlikely]] is actually significant
ruby condorBOT
real pecan
based on components available
lean glen
bruh
real pecan
i wonder if [[unlikely]] is actually significant
it does reorder the blocks so that the likely case is favored, so its lighter on the icache and stuff
flint idol
Makes sense
real pecan
e.g. your nvidia gpu provides an ssdt
flint idol
The only microoptimization I do to save icache
Is memzero instead of memseting zero
Saves a couple bytes
real pecan
lol
flint idol
And in fact you won't find a memset zero in obos except in outside libs
real pecan
also i still believe in the idea that unlikely/likely specifiers help with understanding program flow especially if u forget how something works or never seen it before
so its both a programmer and a compiler hint
real pecan
like i read some random linux code and it does random shit
i see unlikely()
and i instantly go ok i can skip that branch for now
lean glen
I use it in my new library
flint idol
like i read some random linux code and it does random shit
Warning: Viewing linux code can cause cancer, birth defects, or other reproductive issues.
real pecan
part of the $job
flint idol
part of the $job
Sue for the reasons above
lean glen
just provide OOP interfaces?
real pecan
yeah i guess
lean glen
I'll take a look
real pecan
for interfaces u find too Cfied
real pecan
The OOP api?
lean glen
yeah well C++ bindings
real pecan
ah sure
real pecan
sounds great
flint idol
time to implement Sys_WaitOnObjects
which will be annoying
OBOS_EXPORT obos_status Core_WaitOnObjectsPtr(size_t nObjects, size_t stride, struct waitable_header* objs);
because that's the underlying implementation
wait this won't work at all
what was my intention for this
why isn't it an array of struct waitable_header*
now it's less annoying
I wish clangd could read my mind
so when I type OBOS_ST
it defaults to OBOS_STATUS_SUCCESS when I autocomplete
all syscalls that are scheduler related are completed
handle Sys_ThreadContextCreate(uintptr_t entry, uintptr_t arg1, void* stack, size_t stack_size, handle vmm_context);
obos_status Sys_ThreadContextRead(handle thread_context, struct thread_context_info* out);
// scheduler/thread.h
handle Sys_ThreadOpen(handle proc, uint64_t tid);
handle Sys_ThreadCreate(thread_priority priority, thread_affinity affinity, handle thread_context);
obos_status Sys_ThreadReady(handle thread);
obos_status Sys_ThreadBlock(handle thread);
obos_status Sys_ThreadBoostPriority(handle thread, int reserved /* ignored as of now */);
obos_status Sys_ThreadPriority(handle thread, const thread_priority *new, thread_priority* old);
obos_status Sys_ThreadAffinity(handle thread, const thread_affinity *new, thread_affinity* old);
// Can only be called once per thread-object, and must be called before readying a thread.
obos_status Sys_ThreadSetOwner(handle thr, handle process);
uint64_t Sys_ThreadGetTid(handle thr);
// locks/wait.h
obos_status Sys_WaitOnObject(handle object /* must be a waitable handle */);
obos_status Sys_WaitOnObjects(handle *objects, size_t nObjects);
// scheduler/process.h
handle Sys_ProcessOpen(uint64_t pid);
handle Sys_ProcessStart(handle mainThread /* optional, set to HANDLE_INVALID if unwanted */, handle vmm_context);
obos_status Sys_ProcessKill(handle process, bool force);```note that fork.2 will be something like
handle new_context = Sys_CreateEmptyContext();
Sys_Fork(CURRENT_CONTEXT_HANDLE /* fork this */, new_context); // Make the vmm do its magic
handle new_proc = Sys_ProcessStart(main_thread, new_context);
doing this instead is pretty good since userspace can make an empty context and map whatever it wants in it for the new process
without much trouble at all
flint idol
```c
handle new_context = Sys_CreateEmptyContext();
Sys_Fork(CURRENT_CONTEXT_HAN...
instead of this, one could do
handle new_context = Sys_CreateEmptyContext();
// Loads a elf file into a vmm context
load_elf_file(new_context, "/usr/bin/bash");
handle new_proc = Sys_ProcessStart(main_thread, new_context);```or to implement exec, I might (possibly incorrectly?) do something like that, but I kill the current process
handle new_context = Sys_CreateEmptyContext();
// Loads a elf file into a vmm context
load_elf_file(new_context, exec_path);
handle new_proc = Sys_ProcessStart(main_thread, new_context);
Sys_Exit();```I could add an option to Sys_ProcessStart to replace the current process' data with the new data
in case the pid needs to be the same for exec
wait, exec is provided by the stdlib
and is not a syscall
man fork.2 will be quite benefical for me
I could've sworn madvise wasn't posix
whatever
btw I heard somewhere that fork does not copy all mappings of the parent
I know it doesn't take memory that is madvised to MADV_DONTFORK
That depends on flags passed to
mmap. If MAP_SHARED is specified then mapped pages would be shared. If MAP_PRIVATE, mapped pages would not be shared - each forked process would get its own copy.
I will never get why one needs fork
why don't you just spawn a new process
don't you usually use a combination of fork and exec anyway to start a new process
assuming you don't have the spawn syscall
ok posix does specifiy spawn
flint idol
don't you usually use a combination of `fork` and `exec` anyway to start a new p...
considering exec would wipe most mappings anyway
hmm well it is most mappings...
so you could save memory big on shared libs
assuming they all have the same bases
instead of having 17 copies of mlibc for 17 processes that you spawned which use mlibc
you only have one since it all got forked from one process (init)
shy forge
Is memzero instead of memseting zero
what's memzero
shy forge
Ain't that just memsetting zero
flint idol
yes
BUT
it copies code from memset, but it can assume that the value being passed is zero
so instead of mov eax, edi
which is very big instruction
it can xor eax,eax
shy forge
Oh I see
flint idol
+ (uintptr_t)Sys_HandleClose,
+ (uintptr_t)Sys_HandleClone,
+ (uintptr_t)Sys_ThreadContextCreate,
+ (uintptr_t)Sys_ThreadContextRead,
+ (uintptr_t)Sys_ThreadOpen,
+ (uintptr_t)Sys_ThreadCreate,
+ (uintptr_t)Sys_ThreadReady,
+ (uintptr_t)Sys_ThreadBlock,
+ (uintptr_t)Sys_ThreadBoostPriority,
+ (uintptr_t)Sys_ThreadPriority,
+ (uintptr_t)Sys_ThreadAffinity,
+ (uintptr_t)Sys_ThreadSetOwner,
+ (uintptr_t)Sys_ThreadGetTid,
+ (uintptr_t)Sys_WaitOnObject,
+ (uintptr_t)Sys_WaitOnObjects,
+ (uintptr_t)Sys_ProcessOpen, // Unimplemented
+ (uintptr_t)Sys_ProcessStart,
+ (uintptr_t)Sys_ProcessKill, // Unimplemented```new syscalls!!!
thick jolt
less goo
inland radish
it can `xor eax,eax`
i love micro optimizations that actually serve nearly no purpose!!!
honest citrus
you should save those cycles because what if someone's running your OS on an Intel Quark D1000, a 0.025W x86 microcontroller, then it'll be possibly slightly measurably faster :)
flint idol
i love micro optimizations that actually serve nearly no purpose!!!
Same!
limber wave
you should save those cycles because what if someone's running your OS on an Int...
that shit was crazy
intel made a few dev boards for fucking with that stuff (Edison, Galileo)
the intel Curie was something similar to that, designed for wearable uses, and it was tiny
I'd've never thought someone could fit an x86 core in a space that small
the intel edison was initially designed to be the same form factor as an SD card
anyways, enough of #hardware creeping in
flint idol
time to fix these bugs with syscalls
well probably handles
bruh
I switch to debug mode
and it double faults
and it boots so slooooooooow
flint idol
omg ur the same as me !!!!
Except I fixed the bug
flint idol
bruh
why is my syscall handler
getting
the wrong
values
everything after the 3rd parameter
looks cursed
except that's only the 4th parameter
wait
rcx is taken by the syscall instruction
I'm boutta go insane
hold on
affinity
is a 128-bit parameter
which takes two registers
I can finally start user threads
in user mode
asm (
".intel_syntax noprefix;"
".global test_program;"
"\
test_program:;\
push rbp;\
mov rbp, rsp;\
sub rsp, 0x8;\
lea rdi, [rip+test_thread];\
mov rsi, 0;\
mov rdx, 0x40000;\
mov r8, 0x4000;\
mov r9, 0xfe000000;\
mov eax, 6;\
syscall;\
mov [rbp-4], eax;\
mov rdi, 2;\
mov rsi, 0;\
mov rdx, 0;\
mov r8, [rbp-4];\
mov eax, 9;\
syscall;\
mov [rbp-8], eax;\
mov edi, [rbp-8];\
mov esi, 0xfe000000;\
mov eax, 15;\
syscall;\
mov edi, [rbp-8];\
mov eax, 10;\
syscall;\
mov edi, [rbp-8];\
mov eax, 4;\
syscall;\
mov edi, [rbp-4];\
mov eax, 4;\
syscall;\
call Sys_ExitCurrentThread;\
test_thread:;\
call Sys_Yield;\
call Sys_ExitCurrentThread;\
Sys_ExitCurrentThread:;\
mov eax, 0;\
syscall;\
ret;\
Sys_Yield:;\
mov eax, 1;\
syscall;\
ret;\
Sys_Shutdown:;\
mov eax, 3;\
syscall; ret;\
Sys_Reboot:;\
mov eax, 2;\
syscall; ret;\
.att_syntax prefix;\
.global test_program_end; test_program_end:\
"
);
and my beautiful code to do so
now if only I could write comments in inline assembly
flint idol
```c
asm (
".intel_syntax noprefix;"
".global test_program;"
"\
test_pro...
in C this is like:
// test_program
handle thread_ctx = Sys_ThreadContextCreate(test_thread, 0, 0x40000, 0x4000, HANDLE_CURRENT);
handle thread = Sys_ThreadCreate(PRIORITY_NORMAL, 0, thread_ctx);
Sys_ThreadSetOwner(thread, HANDLE_CURRENT);
Sys_ThreadReady(thread);
Sys_CloseHandle(thread);
Sys_CloseHandle(thread_ctx);
Sys_ExitThread();
// test_thread
Sys_Yield();
Sys_ExitThread();
I made it sigsegv before to make sure it actually reached that point
in the test thread
all these other syscalls should work
because they have no reason not to, as they were all copy pasted one after an other
all other syscalls need other syscalls to be implemented
waitable object syscalls need event/whatever syscalls
and process ones need vmm syscalls
is it worth it to make a Sys_ProcessOpen syscall
it'd be slow as hell because it'd need to traverse the entire process tree
nvm I need it
to terminate processes
just committed these brand new syscalls
@vale nymph are u proud
oops
forgot to stage some files
yes so tomorrow I will be making vmm syscalls
and implementing fork after that
and adding that syscall
then vfs syscalls
then some sync primitives
then some driver interface syscalls
then I will finally merge this branch
bruh this'll be pain to solve the merge conflicts for
actually it's not too bad
less than I expected
this is on a test clone of obos
I am scared to see what automerge will do though
which is why I'll need github ci
lean glen
not sure what the use of a yield syscall is
flint idol
not sure what the use of a yield syscall is
To yield
flint idol
I just realized the handle system has a race condition
If it's expanded and a syscall is in the midst of using a handle descriptor, it is possible for a use after free
I can solve this in two ways, lock the handle table for as long as a handle descriptor is being used
Or to have the handle table be an array of pointers to handle tables
handle_desc** table
As opposed to handle_desc*
white mulch
If it's expanded and a syscall is in the midst of using a handle descriptor, it ...
I just have my handle as a variant with atomic shared pointer inside
lean glen
Just lock the handle table when it's modified?
flint idol
Just lock the handle table when it's modified?
Yes I do
lean glen
So what's the problem
flint idol
On lookup it returns a pointer inside the array
And then it's used
And if something expands the handle table that pointer gets invalidated
white mulch
make the array contain reference counted pointers instead
lean glen
Yea
vale nymph
make a fursona like managarm 
honest citrus
Also obos logo is beautiful
Techflash OS logo is better 
real pecan
No obos progress updates = sad day
flint idol
No obos progress updates = sad day
Dw I'm gonna be back in 5 minutes
flint idol
5 minutes happened
anyway
idk why it takes so long for the kernel to remap the framebuffer
for (size_t y = 0; y < Arch_Framebuffer->height; y++)
for (size_t x = 0; x < Arch_Framebuffer->width; x++)
OBOS_PlotPixel(OBOS_TEXT_BACKGROUND, &((uint8_t*)OBOS_TextRendererState.fb.backbuffer_base)[y*Arch_Framebuffer->pitch+x*Arch_Framebuffer->bpp/8], OBOS_TextRendererState.fb.format);```I wonder...
flint idol
obos' now has colorful text on the console output thing
obos best os, has colorful logs
@vale nymph take that
vale nymph
needs more rainbow smh
flint idol
obos' now has colorful text on the console output thing
ignore the random address being printed in between all those logs
flint idol
make the array contain reference counted pointers instead
you see the problem with that is how I define the actual array for the handle table
I do struct handle_desc* arr
and reallocate that array
and return elements of that array
to lookup
flint idol
and reallocate that array
to expand the table
so the problem here was that
if I had code like:
handle_desc* desc = handle_lookup(...);
// there is a slight possibility that *desc becomes freed if the table expands in between the lookup and access
thread* thr = desc->un.thread;```I hate IDEs
I do {
DON'T ADD A }
F8UDSFP[VGKDSFOSWOIF\
idk what genius
thought it's a good idea to define OBOS_NO_KASAN as nothing
oh right that was me
ok so I made it so that
the syscalls that use handles (all of them but 4)
lock the handle table while using a handle_desc
which isn't too long
as most you can do with that is fetch/set the address of the kernel object that the kernel handle is
all I need to do now is implement proper freeing of thread and thread context handles
which means I need to implement a handle close callback
flint idol
I am honestly surprised with how little time it takes obos to boot with optimizations
it takes ~2 seconds
possibly less
except the fat driver is slow af
truely worth of the title slowfat
a 512M FAT32 partition takes >5 seconds to load
for (pagecache_dirty_region* curr = LIST_GET_HEAD(dirty_pc_list, &pc->dirty_regions); curr; )
{
if (off >= curr->fileoff && off < (curr->fileoff + curr->sz))
{
Core_MutexRelease(&pc->dirty_list_lock);
return curr;
}
curr = LIST_GET_NEXT(dirty_pc_list, &pc->dirty_regions, curr);
}```according to qemu-prof.sh
the kernel spends a lot of time in this loop
qemu-prof.sh x86_64 0.00001 '-drive file=../out/obos.iso,format=raw -M q35 -drive file=../disk.img,format=raw -accel kvm -cpu host -debugcon file:log.txt' ../out/oboskrnl```perhaps it would be faster to use a different data structure
perhaps a RB-tree would be good ||(I only know four data structures)||
int compare_dirty_regions(pagecache_dirty_region* left, pagecache_dirty_region* right)
{
if (in_range(right->fileoff, right->fileoff+right->sz, left->fileoff))
return 0;
if (left->fileoff < right->fileoff)
return -1;
if (left->fileoff > right->fileoff)
return 1;
return 0;
}```to lookup a dirty region I would simply RB_FIND the thing I want
to create one, I simply lookup one, if it exists, return it, otherwise, if there is a region that is contiguous with the region that I want to create, I expand the contiguous region
and finally if none of that is true
then I make a new one
lean glen
I am honestly surprised with how little time it takes obos to boot with optimiza...
here's mine
you cant see it on the video because its too fast but there's logs before the graphical console
flint idol
but still impressive
I just need to fix my page cache
idk if this took longer or less time
it took the same amount of time*
*I was counting on my fingers
why does it take a lot less time in qemu-prof.sh
no smp
damn you tlb shootdowns
TODO for tomorrow:
VMM syscalls
TLB shootdowns need to be unfucked
lean glen
ok i made it faster
flint idol
TLB shootdowns need to be unfucked
I need a fork syscall
lean glen
aka stripped the initrd by half
flint idol
life if we didn't need to do TLB shootdowns would be amazing
flint idol
why is time wrong
lean glen
it's UTC
flint idol
at least I can swap stuff out to RAM
trust me, I'll reimplement swapping out to disk work soon™️
flint idol
and it's not like <2 seconds to boot is a lot
lean glen
i sped up boot times a lot recently
flint idol
well to boot into nothing
lean glen
because i wanted it to be bearable on my non-native mac
flint idol
obos uses state of the art technology to do absolutely nothing
lean glen
i could probs make the kernel part of the booting a bit faster
flint idol
but hey who needs bash anyway
just make some dumb program to play with threads in a preallocated stack by the kernel when the program started
(what I did to test thread syscalls)
lean glen
few easy optimizations:
- better path parsing algorithm (I've already made a better one which is 2x faster)
- dont use a linked list for anon pages (array would be better)
- minimize allocations
flint idol
which reminds me, I need to implement a generic elf loader soon
lean glen
the anon thing idk if its relevant
but i do like 5000 linked list lookups (O(n)) at boot
flint idol
lean glen
which is not that much but still
flint idol
few easy optimizations:
1. better path parsing algorithm (I've already made a be...
which reminds me, I need to fix something with dirent lookups
basically
static namecache_ent* namecache_lookup_internal(namecache* nc, const char* path)
{
namecache_ent what = { };
OBOS_StringSetAllocator(&what.path, Vfs_Allocator);
OBOS_InitString(&what.path, path);
namecache_ent* hit = RB_FIND(namecache, nc, &what);
OBOS_FreeString(&what.path);
return hit;
}```wait do you do string interning
flint idol
wdym interning
lean glen
not sure what that means
flint idol
struct string constructor
which might have been the worst thing I have devised in my kernel
lean glen
what is it
flint idol
a string
lean glen
a dynamic string?
flint idol
dynamic string
lean glen
meanwhile
String str{"hello"};
``` omg
static void namecache_insert(namecache* nc, dirent* what, const char* path, size_t pathlen)
{
namecache_ent* ent = Vfs_Calloc(1, sizeof(namecache_ent));
ent->ent = what;
ent->ref = what->vnode;
ent->ref->refs++;
OBOS_StringSetAllocator(&ent->path, Vfs_Allocator);
OBOS_InitStringLen(&ent->path, path, pathlen);
if (!namecache_lookup_internal(nc, OBOS_GetStringCPtr(&ent->path)))
RB_INSERT(namecache, nc, ent);
else
{
OBOS_FreeString(&ent->path);
Vfs_Free(ent);
}
}```in C++
lean glen
yeah i do a bunch of shit like that
flint idol
funny
this could be easily solved by just
brain fart
shi I forgot
at least the vfs path lookup isn't a crucial path at boot
so it doesn't slow boot times
I must implement UID stuffs soon
I technically have permissions in my vfs
idk if they work properly or not
might be a TOCTOU bug in there
flint idol
I need a fork syscall
I also need page locking
sharp pike
cynix is gonna beat all of you in speeeeeed
i need to assume chemicals and work on it
flint idol
I assume this is your new kernel?
sharp pike
ye
sharp pike
-Ofast doesn't help kernels
also -Ofast is kinda stupid anyways
it's -O3 + standards breaking floats optimisations
clang is removing it
sharp pike
the other day I and infy fixed a couple of Limine bugs that triggered on -O0
sharp pike
now it works at all optimisation levels
flint idol
cynix is gonna beat all of you in speeeeeed
if it doesn't have colorful logs, then it's a bad kernel /j
sharp pike
how do you know it doesn't?
flint idol
I'm just saying if it doesn't it's bad
which is why astral's kernel is bad, no colors
flint idol
it's np
colorful logs are fun
and serve exactly zero purpose
except if you set the color on panic to red
ornate ginkgo
DON'T ADD A }
Yeah it's incredibly annoying - it's almost always configurable though.
real pecan
the other day I and infy fixed a couple of Limine bugs that triggered on -O0
I mean thanks for the credit but I dont think I did anything there? Lol
real pecan
Yup
vale nymph
which is why astral's kernel is bad, no colors
He says this after borrowing the background color I use
flint idol
ok vmm syscalls
lesgo
let's see
what kernel functions will I expose
Mm_VirtualMemory*
so Mm_VirtualMemoryAlloc Mm_VirtualMemoryFree and Mm_VirtualMemoryProtect
a syscall to create a new (empty) vmm context
Mm_VirtualMemoryLock and Mm_VirtualMemoryUnlock when I implement those
what am I missing
surely there's more to it
considering I have a 3k loc vmm
which isn't much
but I'd defintiely think I'd have more than 6 functions to expose...
a function to set a context's working-set capacity
is one thing I'd expose
flint idol
a function to set a context's working-set capacity
speaking of that, I should implement expanding ws sizes automatically
I want a syscall to get a context's statistics
(memory in use and stuff)
one to get a count of physical memory in use at the moment
as well as a fork syscall when I implement that
flint idol
so `Mm_VirtualMemoryAlloc` `Mm_VirtualMemoryFree` and `Mm_VirtualMemoryProtect`
note that my virtual memory alloc will deviate from posix' mmap a bit
but I'll make up for that in the mlibc sysdep
maybe I will also expose MmS_QueryPageInfo
rather directly
or I just use the range (in the tree) for the address
I made the chagnes to the pagecache dirty regions
to use an rb-tree
but idk whether they're really worth it
by that I mean whether it actually provides a performance boost
I think it'll be fine
flint idol
as well as a fork syscall when I implement that
technically that's just cloning the current context
so I can abstract that behind Sys_HandleClone
void* Sys_VirtualMemoryAlloc(handle ctx, void* base, size_t size, prot_flags prot, vma_flags flags, handle file, obos_status* status);
obos_status Sys_VirtualMemoryFree(handle ctx, void* base, size_t size);
obos_status Sys_VirtualMemoryProtect(handle ctx, void* base, size_t size, prot_flags newProt);
obos_status Sys_VirtualMemoryLock(handle ctx, void* base, size_t size);
obos_status Sys_VirtualMemoryUnlock(handle ctx, void* base, size_t size);
handle Sys_MakeNewContext(size_t ws_capacity);
obos_status Sys_ContextExpandWSCapacity(handle ctx, size_t ws_capacity);
obos_status Sys_ContextGetStat(handle ctx, memstat* stat);
size_t Sys_GetUsedPhysicalMemoryCount();
obos_status Sys_QueryPageInfo(handle ctx, void* base, page_info* info);
should be all the syscalls I need + fork
lean glen
these syscalls seem way too low-level
flint idol
``Sys_VirtualMemoryFree(ctx, (void *)0xffffffff80000000, 0x10000000)``
access denied, cannot open a handle to the kernel context
flint idol
these syscalls seem way too low-level
I mean
not really
I mean kinda
the context-related ones
flint idol
these syscalls seem way too low-level
most syscalls directly translate to native calls to a kernel function
after verifying handles, copying memory, etc.
flint idol
I have implemented all vmm syscalls
I will now test some of them
wait
Sys_VirtualMemoryAlloc has 7 parameters
but my syscall abi can only take 5
struct vma_alloc_userspace_args
{
prot_flags prot;
vma_flags flags;
handle file;
};```so apparently rn
is the best time
for user memcpy to break\
wtf
does memcpy have severe brain damage rn
fuck you gcc
I'm tryna debug
and you're optimizing out everything
fuck inline asm
fuck att syntax
HOW DO IDEFINE A DWORD
ghregdreger
grneoneovgjdnfvojkdfnbvljvn
.int
@real pecan is there any requirement to which CPU uacpi_prepare_for_sleep_state/ uacpi_enter_sleep_state/uacpi_reboot run on, or can they run on any arbitrary cpu?
ornate ginkgo
Check the spec, the section on sleeping is fairly concise
I don't think so, but then again ACPI likes its baked in assumptions, so maybe it has to be the BSP.
real pecan
<@353593948523986945> is there any requirement to which CPU `uacpi_prepare_for_s...
Not really, but ideally u halt all other CPUs
Some hw might not shutdown otherwise probably
real pecan
I don't think so, but then again ACPI likes its baked in assumptions, so maybe i...
Maybe, I didn't see Linux try to do it on the bsp, but maybe I missed it
flint idol
Not really, but ideally u halt all other CPUs
I do that
flint idol
@real pecan I see that linux disables GPEs in preperation for acpi shutdown
and was wondering if I should too
Oh master of ACPI
anyway, time to implement fork
obos_status Mm_ForkContext(context* into, const context* toFork);
Uacpi already does that internally
flint idol
ok
real pecan
Or well it does leave only wake ones on
I dont think it matters honestly
U can call that if u want
flint idol
already done for me, so I won't
flint idol
god damn it
there's like
huge bug
with CoW
basically
the other pages aren't mapped as RO
anyway
I now go play minecraft
and will be back in an indefinite amount of time
but before I go, I have one thing to say:
FUCK COW
vale nymph
The curse is real :onepiece:
flint idol
I back
flint idol
If I am not missing anything, I should be able to fork a vmm context now
obos has been cursed since the beginning
I really need to start making more kernel code pageable
there is a total of 12 non-paged pool allocations
12
yet 10M memory is non-pageable
which is kinda absurd
[ LOG ] Currently at 10500 KiB of committed memory (1200 KiB pageable), 4 KiB paged out, 10052 KiB non-paged, and 2504 KiB uncommitted. 17388 KiB of physical memory in use. Page faulted 905 times (0 hard, 905 soft).```before I sleep I will leave that message here
flint idol
fnuy things are happening
OVMF simply breaks the kernel with KVM
it hangs while waiting for the BSP to initialize its timer
but the handler for vector 254 "never sent an EOI" (don't know what's up with that)
on TCG it boots
but seems to triple fault when I attempt to shutdown
nope
it just triple faults after booting
also don't know what's up with that
real pecan
F
flint idol
but the handler for vector 254 "never sent an EOI" (don't know what's up with th...
basically vector 254 doesn't have a handler registered
so it was ignored
how?
idk
why?
idk
fixed
was a race condition
that only happened on qemu KVM with OVMF
flint idol
was a race condition
basically as soon as I set LINT0 to 0x7fe on the bsp
it fired
but since I didn't have vector 0xfe registered yet, as that happened at the end of the function
it didn't work
flint idol
damn you past me
thick jolt
played mc all day
bru
😭
at least u can do things
ive been
in my bed
all fucking day
just coughing and
man being sick sucks ass
i didnt even touch my laptop til like 1am
flint idol
hope you get better
thick jolt
hope you get better
thank you !
real pecan
Understandable
flint idol
I now shall work on this
a bit
so where was I
I think forking vmm contexts
obos_status Mm_ForkContext(context* into, context* toFork)
{
if (!toFork || !into)
return OBOS_STATUS_INVALID_ARGUMENT;
irql oldIrql = Core_SpinlockAcquire(&toFork->lock);
page_range* curr = nullptr;
RB_FOREACH(curr, page_tree, &((context*)toFork)->pages)
{
if (curr->can_fork)
{
if (curr->mapped_here && !curr->cow)
{
// If shared, then simply copy the range and continue.
remap_page_range(clone_page_range(into, curr));
continue;
}
if (!curr->cow)
{
// Make pages be Symmetric CoW.
curr->cow = true;
curr->un.cow_type = COW_SYMMETRIC;
if (!curr->prot.ro)
{
curr->prot.rw = false;
remap_page_range(curr);
}
}
(void)clone_page_range(into, curr);
}
}
Core_SpinlockRelease(&toFork->lock, oldIrql);
return OBOS_STATUS_SUCCESS;
}```this function seems pretty... small
to be complete
but at the same time
idk what else to put
I guess I might need to handle asym cow
nvm, idk what I'd need to handle in the first place
yeah ig forking contexts is complete until I find that it is completely broken while porting stuff
implementing sys_fork for mlibc is gonna be interesting
specifically finding the return value
maybe if I just implement a kernel wrapper it'd be easier....
flint idol
okie
time to commit this
anyway
I must now implement some handle cloning/closing callbacks
I currently need ones for:
thread context handles
threads handles
which shouldn't be too hard, right?
well thread handles should be easy
but thread context handles are harder
because if a thread is currently running
and you try to read its context
you get stale data
I didn't think this through
thread context handles should not be readable from userspace
since that'd just leak kernel stuffs
flint idol
cmp [r11+rax*8], 0
; Basically a call if zero
; Maybe we should just do something normal?
push .finished
jz Sys_InvalidSyscall
call [r11+rax*8]
add rsp, 8
.finished:
; shit
just wrote this code
it's kinda cursed
flint idol
I will be expanding my driver interface a bit rn
if you care I talked about them with infy here
#1217009725711847465 message
flint idol
if the ps/2 kb driver depends on an exported symbol from the ps2 controller
the kernel would just abort the load
since it doesn't see that symbol anywhere
real pecan
will it retry it later
real pecan
so how would it work now
flint idol
you load ps/2 kb driver before ps/2 controller driver
it returns a status of symbol unresolved
then you can load the ps/2 controller driver
and it loads it
and continues on
real pecan
no but
flint idol
you would need to explicitly load the ps/2 kb driver again
weak kestrel
obervisor
flint idol
you can literally just slap a valid driver header onto any ET_DYN elf that matches the expected arch stuff
and obos will load it as a driver
real pecan
unless I misunderstood
Well u have to solve it now right
real pecan
Do u have a script or something
flint idol
for what?
weak kestrel
just load the ps/2 controller driver before the ps/2 keyboard driver
why not make it a single driver
flint idol
ps/2 controllers differ between arches
weak kestrel
ifdef __x86_64__
flint idol
also so I can save some memory
you don't have a ps/2 mouse?
just don't load the driver
weak kestrel
a few kilobytes max
flint idol
a few KBs is a lot
real pecan
Lol
flint idol
my init system is probs gonna not do much
since the kernel loads most drivers needed
via kernel parameters
and drivers it finds in the initrd
OBOSKRNL usage:
NOTE: Any amount of dashes ('-') can be used at the beginning of the option or flag.
--enable-kdbg: Enables the kernel debugger at boot. Not all architectures support this.
--initrd-module=name: The name or path of the initrd module.
--initrd-driver-module=name: The name or path of the initrd driver module.
--load-modules=name[,name]: If an initrd driver is specified, then 'name' is an absolute path
in the initrd, otherwise it is the name of a module to load as a driver.
--mount-initrd=pathspec: Mounts the InitRD at pathspec if specified, otherwise the initrd is left unmounted
when 'init' is called.
--root-fs-uuid=uuid: Specifies the partition to mount as root. If set to 'initrd', the initrd
is used as root.
--root-fs-partid=partid: Specifies the partition to mount as root. If set to 'initrd', the initrd
--working-set-cap=bytes: Specifies the kernel's working-set size in bytes.
--initial-swap-size=bytes: Specifies the size (in bytes) of the initial, in-ram swap.
--help: Displays this help message.```weird how info mem doesn't work with la57 paging
on qemu
info tlb does work
oh yeah I should implement SMAP and SMEP soon
as well as stuff to do with whatsitcalled
sse
bru
how is my 3rd kernel's init code only 400 lines
but this kernel's is ~1000 lines
something cursed is happening
something has kernel code as user
causing SMEP to complain
nvm
I forgot to put an asm function in the .text section 
I really need to make more of my driver loading code arch agnostic
flint idol
@real pecan am I missing anything, or does uacpi not have some sort of public api to get the current init level
I am using g_uacpi_rt_ctx.init_level in the meantime
uhhh
nullptr access
the lone uacpi symbol in use
I should probably check if it's a uacpi stdlib symbol before failing
flint idol
I will be making my driver loader more cross-platform soon
rn it's a mess
I have disabled strict aliasing in my kernel
and in the drivers
and while building uacpi
for everything
apparently it is evil
uh oh
it is evil
real pecan
<@353593948523986945> am I missing anything, or does uacpi not have some sort of...
get_current_init_level?
thick jolt
oh i see
flint idol
that symbol turned out to be uacpi_strnlen iirc
which caused me to make it check the symbol is a uacpi stdlib symbol
thick jolt
which caused me to make it check the symbol is a uacpi stdlib symbol
whats the problem
im a little confused lol
thick jolt
```
OBOSKRNL usage:
NOTE: Any amount of dashes ('-') can be used at the beginnin...
what in what context am i looking at
lol
flint idol
whats the problem
uacpi stdlib symbols have not much to do with uacpi itself
and I still want to allow load of a driver if it uses one (and uacpi init level isn't what the driver wants)
thick jolt
oh i see
flint idol
what in what context am i looking at
context was me explaining how the kernel is kinda like an init system in and of itself
thick jolt
context was me explaining how the kernel is kinda like an init system in and of ...
oooo i seeee
flint idol
note how I said kinda
its job after all subsystem initializations is to load some drivers using pnp and drivers passed to the kernel
which must all be in the initrd
and then to call the init program after finalizing vfs init (which means mounting root to the partition specified on the cmdline)
flint idol
After this driver interface stuff, I want to look into implementing suspend
real pecan
do it
i need a kernel that can test uacpi wake api
and right now there isnt a single hobby kernel that has suspend
flint idol
I will be doing that
and will also be adding some driver interface functions to make a driver tell its devices that we are going rather to a deeper sleep state 'n', or a more shallow sleep state 'n'
or just one to change the devices' sleep state
void(*change_sleep_state)(uacpi_sleep_state new_sleep_state);```I think if I just set all devices that can wake me from a GPE
to wake me
I'll be good
torpid wigeon
```c
void(*change_sleep_state)(uacpi_sleep_state new_sleep_state);```
maybe you should abstract the sleep state concept
(if you ever plan on doing something other than acpi)
flint idol
PCI devices have P sleep states
flint idol
(if you ever plan on doing something other than acpi)
those can have a special case
torpid wigeon
k
flint idol
PCI devices have P sleep states
which I think correspond to acpi sleep states
(@real pecan fact check?)
real pecan
D states
torpid wigeon
d3
flint idol
like if I were going to S3, would some driver want to tell its device to go to D3
real pecan
nope they're unrelated to acpi states
acpi does have a way to see which d state can wake you at which s state
that u use to decide how deep to put the device
real pecan
flint idol
I assume the acpi namespace has a node for each pci device that supports power management
real pecan
indeed
flint idol
what part of the namespace?
real pecan
find the pci controller and look for devices underneath
but u dont need that
just enumerate the entire namespace and see if the device is wake capable
just read the goddamn linux source i linked lol
torpid wigeon
reading linux source
real pecan
>reading linux source
idk if u think its bad or something
torpid wigeon
no it just takes time because there's layers upon layers of abstraction
real pecan
in this case its rather simple
torpid wigeon
every time i'm doing a deep dive i have like 20 open files
real pecan
true but with bootlin its easy
real pecan
its really nice for finding implementations and usages
torpid wigeon
and cross referencing on past releases
real pecan
yup
torpid wigeon
iirc they also have llvm source
flint idol
when will they have astral code
flint idol
I get that code, but how would one go about finding the acpi device from some place on the pci bus
real pecan
ez
find the pci controller
that has _SEG and _BBN
so u know XXXX:XX part of the PCI address
then scan underneath
devices have _ADR
so u get the XX:XX part of the address
now u sum the two together and u have XXXX:XX:XX:XX
flint idol
ok
real pecan
for examples look at pci_region_attach in uacpi default_handlers.c
real pecan
and tahts using the older api, there's now uacpi_eval_integer etc so its even easier
flint idol
and the pcie bus is HID PNP0A08
real pecan
indeed
flint idol
I just found this random file online
many moons ago
is _BBN for the like
number of the bus
real pecan
base bus number
flint idol
if I had say, multiple PCI buses
there would be multiple PCI devices
each with BBN returning something unique
?
real pecan
of course
on an unrelated note, apparently i have 15 commits in QEMU and only 11 in Linux
real pecan
unlucky
white mulch
then you just assume that it is zero
real pecan
thats the case for both _BBN and _SEG ^
flint idol
okie
oh wow what's the HPET doing here
what's the 8259 PIC doing here
even a x87 compatible fp processing unit
flint idol
damn, even a 'IBM Enhanced Keyboard'
which has a PRW thingy
I wonder what GPRW does
and something equivalent for PS2 mice
which has a _PSW
I am done extending my driver interface code
(which adds yet one more thing I need to port to the m68k)
real pecan
I wonder what `GPRW` does
Probably gets the respective power resource
flint idol
kinda
real pecan
Does it have an actual ps2 controller
flint idol
nope
real pecan
Like no ps2 ports?
flint idol
no ps2 ports
flint idol
makes me wonder how linux is waking from suspend, from the ps/2 keyboard wake event, or from the actual usb keyboard
is there some sorta command to see
real pecan
The actual keyboard ofc, it disables emulation as soon as u acquire USB ownership from bios
real pecan
is there some sorta command to see
Probably something in sysfs idk
flint idol
real pecan
/proc/sys
flint idol
oh
flint idol
I am starting on suspend code
I think by default I will try to go in the deepest state supported that is shallower than S4
white mulch
afaik S3 is always supported as I am pretty sure that's what windows sleep uses
flint idol
S2 and S1 never hurt anyone
real pecan
Yup
flint idol
unless they are literally never used
real pecan
Just do S3 as the rest are weirder I think
flint idol
Just do S3 as the rest are weirder I think
when I get thrown in my wakeup vector
is it only the bsp
real pecan
But I dont know how they differ exactly on the lower level. I know they resume at the same rip
real pecan
is it only the bsp
Yes
flint idol
so I need to wake the other processors again?
white mulch
yes
flint idol
bruh
real pecan
Yes
flint idol
through my smp trampoline?
white mulch
yes
real pecan
Yes??
flint idol
bruh
white mulch
and basically the same thing for the bsp as you likely start in real mode
well without the apic wake ofc
flint idol
I think I can just make my wakeup vector my smp trampoline then
real pecan
There's S4BIOS btw, where u just write to one reg and it magically puts u into S4 with everything preserved
But I think barely anything supports it
flint idol
sounds hacky
real pecan
I think I can just make my wakeup vector my smp trampoline then
Sure, but keep in mind that all ram contents are preserved so u dont have to reinit from scratch
white mulch
and you have to think about where you run the sleep enter aml from, what I did for now is make a throwaway thread that isn't even put into the sched queue, I manually switch to that when entering sleep and execute the aml there and then when the original thread that initialized the sleep resumes I destroy the thread
real pecan
Oberrow can just run it on the normal uacpi gpe worker thread