will
#Nyaux
molten grotto
idk who else
I try to for the most part (as in using guarded reads/writes etc) but there are other things that I don't do that you can use to oom crash the kernel like if you supply huge buffer sizes 
and I haven't bothered implementing any kind of permission checking because idk, there are just more important things to do
median goblet
I don't implement permission checks either because I have more fundamental things to do
snow wolf
my opinion is that lack of care for security is not a valid argument to intentionally write vulnerable code.
median goblet
But I do, as far as I know, thorough range size checks, and also opt for pinning physical pages in memory if the memory is required for a prolonged operation (like I/O)
snow wolf
not permitting userspace to do whatever it wants, including borking your kernel, has the advantage that there is one less thing that can bork your kernel.
median goblet
it's got that insurance that if your kernel crashes, it's your fault as the kernel's developer
molten grotto
I try to for the most part (as in using guarded reads/writes etc) but there are ...
actually I just found invalid range checks too 
I assume unsigned values for some enums where I haven't explicitly specified them to use unsigned base types
median goblet
oh
that reminds me
i should validate the start address of a thread on creation
not because you can execute kernel code in ring 3 lmao
but because it'll be helpful if the kernel just refuses then and there rather than allowing creation of a thread which immediately segfaults
ebon needle
I just realized that I have this bug too 
kind root
well to be fair userspace might do stupid things not out of bad intent but simply because it has a bug, so crashing the entire kernel if you have a bug in your wip program is probably not ideal
ebon needle
well to be fair userspace might do stupid things not out of bad intent but simpl...
"help me some calculator overwrited my kernel 😭"
ionic musk
I purposefully have a kernel-api function to overwrite the IDT so it triple faults
I give the user the freedom to do as they wish
tender gorge
I purposefully have a kernel-api function to overwrite the IDT so it triple faul...
make sure any user can use it regardless of perms
flat nymph
syscall_tripple_fault
crystal scarab
Beautiful
tender gorge
fuck me chat, i need to implement poll and im doing jackshit again
sorry im ace
surreal path
~~sorry im ace~~
laura has corrupted you 😭
surreal path
i didnt even try to make it sound dirty 😭
tender gorge
i just thought it would be funny 🥺
tender gorge
🥺 🫂
surreal path
tender gorge
surreal path
i need to implement poll but ive done nothing all day and have just been watching freddy fazball videos
tender gorge
i need to work on like a billion projects and things but ive done nothing all day because its all overwhelming
rigid fable
<:hug:647834498758737921>
Hows the porting coming along 👀
surreal path
Hows the porting coming along 👀
procastinating on implementing poll() cause bash needs it to determine if fd 0 is non blocking or what not
rigid fable
Ohh I see; Bash always seems to want to be difficult lol
surreal path
bash and ncurses
rigid fable
After seeing how difficult of a time you had with ncurses I think I may avoid trying to port it in the future lol
rigid fable
Once I finally get into userspace I may give it a go... I did finally get a read-only EXT2 driver going, but it's incredibly simple. I also am rewokring some DMA allocation stuff for my virtio driver...
surreal path
may have been sleeping in
tender gorge
may have been sleeping in
is okie 🥺 💙
surreal path
again i dont know if im in the mood for nyaux dev today
i feel even worse compared to yesterday
im not sick anymore im just
depression
its hard to make yourself do anything
when u feel really really depressed
especially after ur doctor trolling u by not giving u new meds
tender gorge
when u feel really really depressed
i understand, ive been there 🫂
my advice of like, doing everything possible to get another doctor / considering going back to hospital has not changed 🥺
surreal path
should i make poll a vfs op or how is that typically implemented
quick quiver
yeah
surreal path
okay
median goblet
should i make poll a vfs op or how is that typically implemented
I would make the VFS return a waitable object (mutex, event, etc) for each vnode referenced, instead of calling poll for each FD
The point is that multiple FDs are waited on at the same time and one of them will wake up the thread
surreal path
I would make the VFS return a waitable object (mutex, event, etc) for each vnode...
im probs not gonna do that consideringthat sounds complicated
median goblet
The idea is poll() is used to poll multiple FDs instead of just one
surreal path
i know
i just probs cannot implement that
consdering how stupid i am
i have no fucking idea about waitable objects
edgy pilot
very polite poll()
kind root
progress ig
will i ever catch up to nyaux
surreal path
u will
edgy pilot
very sophisticated log ring
kind root
That we'll have to see lmao
surreal path
too depressed to do anything today
kind root
too depressed to do anything today
Did literally nothing as well
Never catching up to nyaux like I said
Well I did have more work and less sleep so
surreal path
everyone i apologise for the long hautius
tmrw i will implement a poll vfs op
which is cringe yes ik
but its the simpler impl so
yea
and hopefully bash and ncurses will work nicely
and we can move on to implmenitngin fbdev
then get doom working
tender gorge
everyone i apologise for the long hautius
its okie 🫂 💙
edgy pilot
everyone i apologise for the long hautius
you are in now way required by anyone to work on your project
surreal path
you are in now way required by anyone to work on your project
i know
but i want to
tmrw
edgy pilot
what's the apology for then?
tender gorge
i just want to apologise
its okie 🥺 🫂
median goblet
tmrw i will implement a poll vfs op
what is your implementation going to look like, considering you can poll from multiple file systems at the same time and you can also wait on sockets?
perhaps i misunderstood your intent last time i asked
surreal path
ok we gamer we start coding now
surreal path
what is your implementation going to look like, considering you can poll from mu...
its just gonna be a vfs poll op
is this okay?
@median goblet i know the perfomance will be shit but is this okay?
surreal path
all requested fds*
thorn bramble
i think you're supposed to call each fd's vnode poll op, if any one returns something that matches the criteria (readable, writable, etc) just return that, otherwise you block on something and when any one of the fds becomes readable/writable/etc you unblock the thread and return that
surreal path
i think you're supposed to call each fd's vnode poll op, if any one returns some...
or i can just spin and yield instead of properly blocking
but yk
what should a poll op look like
?
thorn bramble
returns information about whether the node can read/write/whatever without blocking i guess
i'm not sure
look at existing kernels and how they do it
surreal path
kk
surreal path
tender gorge
going to eep now
good eep :3
median goblet
the poll op is in the vfs
does that mean you are literally calling poll for each vnode or what
surreal path
Yes
molten grotto
the way I implemented poll was that the vfs op just returns a bitmask of events (without taking in any kind of pollfd struct or whatever) + in the poll syscall I loop through the fd list calling the poll vfs op updating the pollfd struct with it and if any of them returned a non-zero bitmask then just return
then for the case where you block for events the vnode has an event that gets signaled when the poll status would change that I collect into a list of waitable events in the poll syscall + do Event::wait_any with the list (which is slightly annoying to implement but its not that bad)
surreal path
LETS FUCKING GOOOO
@kind root @tender gorge
and i can quit nano too
to be brought back to bash
which WORKS
kind root
daamn
surreal path
no bugs other then a weird vfs bug i discovered when trying to save a file
surreal path
bru 😭
snow wolf
I guess serial is usable for that
median goblet
now get networking so we can make CTF challenges out of this
ctf difficulty: trivial
snow wolf
we do security in userspace :^)
snow wolf
ctf difficulty: trivial
nahh, because you have to live long enough to actually print the flag.
median goblet
nahh, because you have to live long enough to actually print the flag. <:meme:5...
you dont need to crash the kernel
you just need to overwrite the right addresses
so f.ex. overwrite the PTE of the "check ownership" function to mark it RWX, then overwrite the "check ownership" function to say "yep you own this"
thorn bramble
you realize you can just use read and write syscalls and memcpy primitives right?
in nyaux, that is :^)
modify kernel PTEs by writing to a file and reading it back into the HHDM
and then you can just write+read your own shellcode into syscall handlers or smth
crystal scarab
modify kernel PTEs by writing to a file and reading it back into the HHDM <:trl:...
Very secure
rigid fable
Ooooo did you finally get it going?? Hell yeah man!
surreal path
hi guys
orchid dawn
surreal path
im currently taking a piss !!
edgy pilot
im currently taking a piss !!
is that really an appropriate thing to announce to public?
edgy pilot
it's a public server
surreal path
fair
edgy pilot
please refrain from doing so in the future
surreal path
alright
surreal path
Do you have a loicence for that?
i cooked the most disgusting
thing
imaginable todah
i vommited
from it
dont cook brownies without eggs
anyways nyaux development yesyes
okay so
the vfs bug is with typing any one line character
strangerly crashes it
tender gorge
dont cook brownies without eggs
Will note lol
worn rain
please refrain from doing so in the future
common bro
its a fucking piss
some countries have public urinals
ebon needle
strangerly crashes it
what's wrong with your commits 😭
surreal path
very eeoy chat
median goblet
what's wrong with your commits 😭
it just says "nano works" and then "how how" in romanian
or "with with" in latin
ebon needle
very eeoy chat
mr. nyaux, to better describe your commits, there are missing emojis of eggplant, poop and droplets, please pay attention to this suggestion
tardy hull
ebon needle
they give prizes to british people on tv for cooking
cooking in what meaning
surreal path
making beans or smt
ebon needle
making beans or smt
I can make mashed potatoes 😃
rigid fable
im currently taking a piss !!
Thanks for the info D:
surreal path
i will work on nyaux tmrw
im fucking
bro iretq gave me brainrot\
from the instagram reels
holy shit
coral dove
I will
When I finish this scheduler
I’m so close to being done I swear
“New sched cum cum”
In nyaux we trust 
surreal path
nearly finished rewatching nick and morty
SORRY rick
and morty
then i can work on nyaux
im nearly on the last episode
surreal path
I THOUGHT MY THREAD GOT DELETED
😭
i got scared
anyways this vfs bug seems kinda weird
it might be a bug with openat itself
considering tmpfs lookup is never called
or devfs lookup
frigid cliff
That would be sacrilege
flat nymph
nah but seriously tho i really thought
Nyaux's becoming a bigger threat every day
surreal path
i fell asleep
mb
ok so the issue is that its accessing the root vnode therefore "finding" something
surreal path
im trying to think of ways to solve it but i keep sleeping all day
which is
um yea
edgy pilot
im trying to think of ways to solve it but i keep sleeping all day
do you sleep at night as well?
surreal path
thats crazy
surreal path
i fixed the vfs bug
:)
holy shit is the fix scuffed tho
very scuffed fix but wtv
fbdev next ? :3
surreal path
real
surreal path
its good lol im being really slow with nyaux
as u can see
mostly due to depression
but wtv
we will have doom soon on nyaux
surreal path
for now ™️
broken depot
you see thing is
you have more time for OSDEV than I do
We'll see what happens I guess
I think I, too, want to use DOOM as a demo now
broken depot
legendary
broken depot
BadgerOS has 'em
surreal path
see ur ahead :)
broken depot
Yeah but BadgerOS also doesn't even have a libc for userspace to use
lots of stuff yet to do for both of us
surreal path
true
surreal path
probs signals
before signals probs ps2 driver
as im using serial for input
and that isnt
fun yk
broken depot
oh ye
surreal path
as im using serial for input
for the tty
surreal path
~~if all u care about is qemu~~
ik the ps2 curse lol
surreal path
ps2 is piss easy
ik its just reading from some port
surreal path
yea that too
surreal path
oh im not
thorn bramble
then why avoid it like it's fire
surreal path
cause implementing fbdev is fun
thorn bramble
that's what u seem to be doing rn
but maybe ur just doing some other shit
and u dont mind using serial
surreal path
ill implment ps2 after
i implement fbdev
reecompiling nano is not working
thats crazy
surreal path
nah its working
median goblet
reecompiling nano is not working
cs.gmo? more like ts.pmo
kind root
I will work on catching up with nyaux today (I won't anyway )
surreal path
I will work on catching up with nyaux today (I won't anyway <:trl:82931664119188...
yes work on catching up with the nyaux daily news
i got coreutils to compile on nyaux
i am able to run cat /dev/zero successfully
kind root
Ah
surreal path
runs nicely
kind root
You should try running a fork bomb for 10 hours
kind root
but its pain
It is
kind root
i will do so
surreal path
11!1!!
kind root
lol
surreal path
holy shit gcc will u just fucking install into the sysroot
holy shit
i personally will knock on the gcc devs door
holy fucking shit they need to be killed (in minecraft ofc)
keep urself safe
im going to eat bro
there we go
:c
why does gcc die
huh
kind root
0x6 is invalid opcode no?
surreal path
yes?
kind root
dump memory at rip i guess
surreal path
:c
this is wrong loaded elf at 0xffff
it shouldnt be doing that
it makes sense why it dies in the linker
because its loading at this garbage address
seriously how
uvm_region_alloc is returning 0x101000
this could be a bug with the elf loader itself maybe
the calculated lowest address is wrong
i guess @glossy walrus was right when he said this code would be a problem in the future
brisk zenith
the calculated lowest address is wrong
nothing looks wrong here and according to the output lowest_address is 0x400000 which sounds sane
surreal path
nothing looks wrong here and according to the output `lowest_address` is `0x4000...
elf should not be loaded at 0xffffffffd01000 tho
that is wrong
and since uvm region alloc returned 0x101000 it causes the number to wrap around
cause ur negating an unsigned number and theres no -1 garbage because its unsigned
so an number overflow basically
brisk zenith
that isn't printing the allocated address but the offset that needs to get added to vaddrs found in the image to get to the real address, there's nothing wrong with said offset being negative
surreal path
that isn't printing the allocated address but the offset that needs to get added...
yes but again, negating more then the allocated address i.e. causing it to wrap around
brisk zenith
so? unsigned arithmetic still works fine, e.g. 0x400000 + 0xffffffffffd01000 using unsigned 64 bit arithmetic is 0x101000
surreal path
so? unsigned arithmetic still works fine, e.g. `0x400000 + 0xffffffffffd01000` u...
yes but the linker gets this entry point
surreal path
e_entry
brisk zenith
oh hm
brisk zenith
yea this is below the lowestaddress so that's expected
what does readelf say about the phdrs of gcc?
like does it agree with your calculation of lowestaddress being 0x400000
surreal path
output of readelf
brisk zenith
seems like lowestaddress is right
i'm confused
does readelf agree with your e_entry? if so gcc has an entrypoint that isn't loaded which is just ???
surreal path
does readelf agree with your e_entry? if so gcc has an entrypoint that isn't loa...
how do i check
brisk zenith
readelf -Wh prints the entrypoint
surreal path
entry point is 0x408331
brisk zenith
okay yeah that sounds right
okay so the e_entry you printed already had the offset applied
brisk zenith
oh yes then that's correct
surreal path
yea
surreal path
oh yes then that's correct
it dies here if further context helps
when it tries to init the executable
brisk zenith
what does addr2line say that rip is?
surreal path
what does addr2line say that rip is?
it doesnt know
brisk zenith
it's probably an address in ld.so, and you should subtract the load offset of whatever object it's from first
surreal path
0xb3 is the subtracted address
brisk zenith
thats worryingly low but maybe its fine
surreal path
it still doesnt know
molten grotto
its likely within in an init array function in some object
surreal path
it doesnt know
molten grotto
you could enable the mlibc logging for object base addresses
surreal path
where
molten grotto
logBaseAddresses in options/rtld/generic/linker.cpp
brisk zenith
surreal path
that is enabled
brisk zenith
in any case id just set a breakpoint in mlibc where that Running DT_INIT_ARRAY functions gets printed and step until you see something that doesnt look right
surreal path
i cant really do that
brisk zenith
why not?
brisk zenith
add-symbol-file in gdb with the right offset, and set a breakpoint like normal?
surreal path
ill try ig
i couldnt catch it
okay
lets see it do shit ig
@brisk zenith weirdly it hangs on tcg
and i cannot figure out why
gdb is being no help too
so it hangs for 10 seconds then 0 dies like this
i do not know why its doing this
edgy pilot
nyaux running gcc and I don't even have mmap
surreal path
nah im struggling
kind root
i dont even have.. literally anything
surreal path
so it hangs for 10 seconds then 0 dies like this
yea i have absolutely no idea whats going on and gdb is not telling me much
help
brisk zenith
yeah idk
brisk zenith
so it hangs for 10 seconds then 0 dies like this
do you mean it hangs for 10s on tcg then dies or that it hangs forever on tcg and does that on kvm
surreal path
do you mean it hangs for 10s on tcg then dies or that it hangs forever on tcg an...
hangs for 10 seconds then dies
on tcg
brisk zenith
then it's probably just because of how much slower tcg is than kvm
brisk zenith
oh so it's reading instead of writing
surreal path
mhm
brisk zenith
that might just be an implementation detail? like for example if it's crashing on an instruction that both reads and writes to an address your cpu says "the fault was due to a write" and tcg says "the fault was due to a read"
surreal path
ig
it might be worth for u to investigate urself maybe when u have time ofc 🥺 👉 👈 , u can remove coreutils from base in recipes so u dont have to compile coreutils
brisk zenith
i'm finishing up some stuff in proxima rn but after that yea i'll try and investigate
surreal path
i'm finishing up some stuff in proxima rn but after that yea i'll try and invest...
okay epicc :)
kind root
i'm finishing up some stuff in proxima rn but after that yea i'll try and invest...
what are you working on atm
brisk zenith
syscall entry and vdso mapping
it'll be the first userspace code executed by new proxima
surreal path
lesgoooo
kind root
why are you doing vdso so soon?
brisk zenith
i dislike having stuff like syscall numbers being part of the kernel abi, so the only way to call into the kernel is via the vdso
kind root
i dislike having stuff like syscall numbers being part of the kernel abi, so the...
so its like ntdll for u
brisk zenith
plus i have a Very Large Amount of userspace memory management code written but untested, and i dislike having that much untested code in the kernel
and since i don't have a vfs yet a vdso is the way to go
brisk zenith
vdso?
essentially, it's a small shared library that's bundled with the kernel and loaded alongside every executable
its base address is passed in the auxiliary vector and ld.so uses it during symbol resolution
it can be used for e.g. implementing syscalls in a way that doesn't do an actual user-to-kernel transition
that's what linux uses it for
surreal path
i see
brisk zenith
so its like ntdll for u
yep
also allows me to do stuff like this ```c
static uint64_t get_ns_since_boot_syscall(void) {
return SYSCALL0(SYSCALL_GET_NANOSECONDS_SINCE_BOOT).integer;
}
static uint64_t get_ns_since_boot_kvmclock(void) {
return x86_64_read_kvmclock(&vdso_info.arch.kvmclock);
}
static uint64_t get_ns_since_boot_tsc(void) {
return timeconv_apply(vdso_info.arch.tsc2ns_conv, x86_64_read_tsc());
}
static uint64_t (*resolve_ns_since_boot(void))(void) {
switch (vdso_info.arch.time_source) {
case X86_64_TIME_SYSCALL: return get_ns_since_boot_syscall;
case X86_64_TIME_KVMCLOCK: return get_ns_since_boot_kvmclock;
case X86_64_TIME_TSC: return get_ns_since_boot_tsc;
default: UNREACHABLE();
}
}
attribute((ifunc("resolve_ns_since_boot"))) EXPORT uint64_t hydrogen_get_nanoseconds_since_boot(void);
pretty cool
surreal path
also allows me to do stuff like this ```c
static uint64_t get_ns_since_boot_sysc...
did u finish finishing up stuff for proxima yet?
brisk zenith
i just pushed the commit a few seconds ago actually
surreal path
oh nice
brisk zenith
well, commits, because it ended up uncovering a few bugs and a memory leak
surreal path
well, commits, because it ended up uncovering a few bugs and a memory leak
yea i get that
brisk zenith
cloning and building nyaux now
surreal path
kk
kind root
Lol monkuous casually working on multiple osses at the same time
brisk zenith
well i also have sirius, and i think andromeda qualifies as an os considering what it's capable of
but yeah
kind root
Sirius?
brisk zenith
another on-and-off project, it's a 32 bit x86 kernel
brisk zenith
nah it's pretty much the most generic kernel you can think of
kind root
Do you enjoy working on it?
brisk zenith
it's enjoyable enough, but I mostly just use it as something to switch things up with when I've been working on one project for too long
a lot of core components are adapted from proxima so it's also a way to trick my brain into indirectly working on proxima when motivation for directly doing so is low
brisk zenith
no recursive mappings, it just uses hhdm and ignores >1G
struct page is allocated physically and virtually contiguously from the highest memory region that's large enough
as I said, nothing super interesting about it
kind root
Lol yeah fair
brisk zenith
new proxima is kind of superseding it because I'm writing it with 32 bit platforms in mind (although i do intend on using hhdm even on said 32 bit platforms)
and the platform bitness is kind of the only thing making Sirius even remotely interesting to work on
uhh @surreal path
surreal path
uhh <@787341689333350450>
that is very weird
brisk zenith
esp since jinx runs in a container
brisk zenith
what repos does the jinx rootfs use for packages? the package isn't named aclocal
surreal path
what repos does the jinx rootfs use for packages? the package isn't named acloca...
ubuntu
debian*
brisk zenith
oh wait it's just automake ofc
yeah that worked
weird that it got removed from build-essential (?)
surreal path
yea
brisk zenith
ncurses
jinix updated to some snapshot when updating to gcc 15 so i'm trying that one now
surreal path
okay
brisk zenith
patch rejected
brisk zenith
okay the part of the patch that was rejected was just whitespace changes so i removed that part
that worked so i guess it was another "c23 became the default" thing
surreal path
yep
brisk zenith
how do you regenerate a patch with jinx?
broken depot
i am able to run cat /dev/zero successfully
yooo
brisk zenith
still fighting with gcc15
specifically i just got doomgeneric to compile
and i am hoping that was the last broken package
surreal path
u dont need doomgeneric
go into base recipes and remove coreutils
forget about coreutils
just remove it from the recipe
and ./jinx build base
brisk zenith
if coreutils fails to build i'll do that but rn it's busy
surreal path
if coreutils fails to build i'll do that but rn it's busy
compiling?
brisk zenith
yeah
surreal path
fair
brisk zenith
why the fuck is the coreutils download speed 100 kb/s
brisk zenith
this is why i use ftpmirror.gnu.org for all my tarball urls
broken depot
why the fuck is the coreutils download speed 100 kb/s
Non-piracy reason to torrent exhibit 1
brisk zenith
nah i'm not doing that
surreal path
lol
brisk zenith
yea now it's maxing out my bandwidth, that's better
broken depot
Damn I really do have my work cut out for me if I want to catch up to Nyaux
broken depot
I don't look for a day and BOOM! Another thing Nyaux has I don't.
broken depot
u have signals
I do
broken depot
lol
Pretty sure that in the grand scheme of things, nyaux has way more stuff than BadgerOS does though
surreal path
like what
broken depot
BadgerOS has like 17.7kLOC, I imaging Nyaux has more?
broken depot
like what
Userspace software basically at all
Userspace libc included
x86 support (lol)
surreal path
i dont have riscv support
lol
also amd shits itself on nyaux atm
for some unknown reason
broken depot
Do you have mass storage devices (SATA/NVMe/etc)? Because BadgerOS doesn't yet.
broken depot
BadgerOS has a way uglier thing
I have a script that makes code that copies files into a RAMFS
It's literally just a function that goes "haha file go brrrrr" with constants
How does Nyaux deal with devices and their drivers btw?
Since that's what I'm currently banging my head against with BadgerOS.
brisk zenith
@surreal path got everything to compile, here's a patch that fixes all gcc15 compile errors
gonna investigate the actual issue now
surreal path
okayy
brisk zenith
did i manage to break my host gdb??
surreal path
😭
kind root
extern char *video_driver;
brisk zenith
right so mlibc tries to call a function from gcc's .init_array at 0x4083f0, but that's the unrelocated static address (i've verified this by looking at the static data with objdump) - the address it should be calling is 0x1093f0
now to figure out why this happens
(it smells like mlibc bug)
kind root
nyaux is a bugless system
brisk zenith
ohhhh wait
yeah nevermind not an mlibc bug i should've known better
the issue is that you're loading all executables as if they are PIEs
but gcc isn't a PIE
it doesn't contain any relocations for itself, just for resolving external references, so it breaks if not loaded at the linked address
surreal path
how would i fix this then
brisk zenith
while loading elfs, if the object isn't relocatable (i.e. e_type is ET_EXEC instead of ET_DYN), simply load all PT_LOAD segments at their p_vaddr without adding any offset
and ofc reserving those areas so that nothing gets allocated there later
surreal path
while loading elfs, if the object isn't relocatable (i.e. e_type is ET_EXEC inst...
could u add a quick check for that into the elf loader and see if that solves the issue
brisk zenith
2s
surreal path
theres a function called ualloc_region_fixed
ull be able to use to alloc at fixed virtual addresses
surreal path
```
$ grep -R "ualloc_region_fixed" *
$ ```
sorry uvmm_region_alloc_fixed
brisk zenith
diff --git a/kernel/src/elf/elf.c b/kernel/src/elf/elf.c
index d10755e..8d54b86 100644
--- a/kernel/src/elf/elf.c
+++ b/kernel/src/elf/elf.c
@@ -7,6 +7,7 @@
#include <limine.h>
#include <mem/kmem.h>
#include <stdint.h>
+#define ET_DYN 3
#define PT_LOAD 1
#define PT_INTERP 3
#define PT_PHDR 6
@@ -61,8 +62,10 @@ void load_elf_pie(pagemap *usrmap, Elf64_Ehdr *hdr, struct ElfInfo *out) {
highest_address = t;
}
}
- size_t sizeofexecutable = highest_address + lowest_address;
- void *feet = uvmm_region_alloc(usrmap, sizeofexecutable, 0);
+ size_t sizeofexecutable = highest_address - lowest_address;
+ void *feet;
+ if (hdr->e_type == ET_DYN) feet = uvmm_region_alloc(usrmap, sizeofexecutable, 0);
+ else feet = uvmm_region_alloc_fixed(usrmap, lowest_address, sizeofexecutable, false);
kprintf("elf: loaded elf at %p, with entry point %p\r\n",
(void *)((uint64_t)feet - lowest_address), (void*)((uint64_t)feet - lowest_address + hdr->e_entry));
out->entrypoint = (uint64_t)feet - lowest_address + hdr->e_entry;
thanks :)
brisk zenith
it seems gcc functioning is blocked on the ability to create files
it tries to create one of the intermediary files in /tmp which fails with enoent
brisk zenith
i might try fixing the amd kvm issue
surreal path
i might try fixing the amd kvm issue
do u have an amd device?
brisk zenith
my main dev pc is amd, and it happens when running in kvm
brisk zenith
the faulting instruction is an iretq and the error code is has rpl0
is this the fucking sysret behavior difference issue
i agonized over this for hours in proxima
surreal path
wdym
brisk zenith
on intel cpus, sysret forces the rpl of both descriptors to 3
on amd, it doesn't
so you get into user mode with dpl3 rpl0
that's an illegal state but sysret doesn't actually do those checks so it doesn't care
iretq does though so the next interrupt you get you're fucked
brisk zenith
yep
fixed with this patch ```patch
diff --git a/kernel/src/arch/x86_64/syscalls/syscall.c b/kernel/src/arch/x86_64/syscalls/syscall.c
index 17cbfcd..e4aea38 100644
--- a/kernel/src/arch/x86_64/syscalls/syscall.c
+++ b/kernel/src/arch/x86_64/syscalls/syscall.c
@@ -369,7 +369,7 @@ void syscall_init() {
if (edx & (1 << 11)) {
uint64_t IA_32_STAR = 0;
IA_32_STAR |= ((uint64_t)0x28 << 32);
- IA_32_STAR |= ((uint64_t)0x30 << 48);
- IA_32_STAR |= ((uint64_t)0x33 << 48);
wrmsr(0xC0000081, IA_32_STAR);
wrmsr(0xC0000082, (uint64_t)syscall_entry);
wrmsr(0xC0000084, (1 << 9));```
brisk zenith
indeed
brisk zenith
amd cpus don't force rpl to 3, so the fix is to set it to 3
surreal path
what does ur name mean anyway
like are u a really intelligent sentiant monkey or smt
brisk zenith
what does ur name mean anyway
monkeys are my spirit animal
surreal path
what does it mean
brisk zenith
nyaux on real hardware
surreal path
fair
brisk zenith
doesn't work on my Ryzen dev machine :(
brisk zenith
and the cr2 is in hhdm
surreal path
and the cr2 is in hhdm
that is very weird
brisk zenith
man nyaux init is slow when the vm has 128G ram (and your host only has 64G with the rest being swap)
(trying to see if it's related to the amount of memory in the system)
surreal path
coujld be
brisk zenith
nope. tried 64gb and 128gb, both with 16 cores, got to userspace fine in both cases
(the real hw it crashed on is 64gb 16 cores)
surreal path
weird
surreal path
@brisk zenith did u figure it out?
brisk zenith
nah
surreal path
understandable
brisk zenith
can't recreate it in a vm or on hardware other than my dev machine so I'd have to reboot every time I try something, I wouldn't have any debugging tools, and it takes a full minute to load the initramfs each boot
so I didn't feel like continuing
kind root
nyaux on real hardware
Damn
Monkuous really is the goat
What does DPL stand for again I forgor
brisk zenith
descriptor privilege level
it's the one that you specify in the gdt
rpl is the one that you specify in the selector, and cpl is cs's rpl
kind root
Makes sense
edgy pilot
Hi @brisk zenith, I wrote a physical frame allocator which appears to have come out as some kind of a tiered free list, but the initialisation takes a few seconds when the system has multiple gigabytes of memory. would you happen to know how I could speed up the init?: https://gist.github.com/ilobilo/f134b3e64a8887466f917aa9b5907718#file-pmm-cpp-L223-L251
edgy pilot
one thought was to get pages from individual memmaps whenever they were needed, but that just makes the problem less noticeable, instead of fixing it
desert haven
Hi <@938223886439436320>, I wrote a physical frame allocator which appears to ha...
i'm not monkuous, but for my freelist pmm I store a count of consecutive pages as well as linkage to the next page/region. It's an extra 2-3 loc when allocating but it meansduring init you only have 1 freelist entry per region, instead of per page.
edgy pilot
i'm not monkuous, but for my freelist pmm I store a count of consecutive pages a...
mine is not really a typical freelist, it looks more like a buddy so I don't know how I would adapt that solution to it
brisk zenith
Hi <@938223886439436320>, I wrote a physical frame allocator which appears to ha...
The logic looks fine, what's max_order? You could try increasing it
edgy pilot
hmm good idea
I think I saw a few regions larger than it
brisk zenith
There's not much else you can do to improve init performance with something buddy like unfortunately
At least afaik
molten grotto
my kernel takes like 1-2s on a debug build with order 14 on tcg qemu with 32gb ram (and that also includes other stuff like mapping the hhdm)
edgy pilot
The logic looks fine, what's max_order? You could try increasing it
it went from 3-4 seconds to near instantaneous wow
kind root
it went from 3-4 seconds to near instantaneous wow
kinda strange because linuxes buddy is usually max order 10 but works instantly
edgy pilot
I'm not a linux level developer :P
kind root
im just thinking what they could be doing differently
edgy pilot
hmmmm
wait, I changed max_order back to 10 and it's still fast
wth
I mean, it's good, but why was it slow before
because I reduced memory size from 8G to 512M
nvm me
larger max_order is the way to go
edgy pilot
I think that's pretty much what I do as well
surreal path
ebon needle
tender gorge
https://tenor.com/view/pixel-trombone-playing-musical-instrument-prank-joke-gif-15407543308447681253
surreal path
sans
tender gorge
holy shit is that sans undertale from the hit game undertale
surreal path
yes
surreal path
ebon needle
https://www.youtube.com/watch?v=iwtkiJNuTGE
tawdry mirage
https://www.youtube.com/watch?v=iwtkiJNuTGE
surreal path
why is nano only taking a portion of the screen now????
im reporting the cols and rows correctly?
doesnt matter what i give back nano stays in this crunchy resolution???
@brisk zenith did u change anything with the tty at all? with the patches yesterday?
brisk zenith
nothing
surreal path
nothing
then this is really weird behavior
could nano be going to a fallback size??
brisk zenith
that looks like 80x25 yeah
or well i guess i updated ncurses? the new version might be broken in upstream ig
surreal path
or well i guess i updated ncurses? the new version might be broken in upstream i...
maybe
brisk zenith
actually looks like you didn't adopt that patch so that can't be it
brisk zenith
oh then you haven't pushed it yet
surreal path
yea i havent
brisk zenith
i got the ncurses revision from jinix, i wonder if nano's broken there too
surreal path
i got the ncurses revision from jinix, i wonder if nano's broken there too
they dont have nano in their recipes
wait they do
they are on 8.4
nyaux is using 8.3
maybe updating nano might solve this
interesting
brisk zenith
yea nano 8.4 needs gl_cv_func_strcasecmp_works=yes in configure
brisk zenith
does the window size ioctl ever even get called?
surreal path
yes
and it gets reported fine
struct winsize {
unsigned short ws_row;
unsigned short ws_col;
unsigned short ws_xpixel; /* unused */
unsigned short ws_ypixel; /* unused */
};
i do this
brisk zenith
yea that looks fine
surreal path
yea so its weird its falling back to some default size
i will try printing cols and lines in nano source code
in regenerate_screen()
brisk zenith
it looks like it's getting the size from this (in ncurses src)
but it tries the ioctl above that in the same function
so that shouldve worked
surreal path
werid?
brisk zenith
okay wtf
i added a few debug printfs in ncurses, changed nothing else, and now it works
brisk zenith
yea ok if i remove those printfs it doesn't work anymore, and even with them added it only works on tcg (not on kvm)
i smell UB somewhere
surreal path
yes it is probs UB
brisk zenith
with the printf on kvm the ioctl fails with an unknown error code
surreal path
this issue gets more weirdier
brisk zenith
brisk zenith
figured it out
the tty device's ioctl function does not set result, only return success
syscall_ioctl then uses that uninitialized result variable and returns it to userspace
the value in that uninitialized variable happens to be negative
mlibc notices that the error code is 0 and returns the result - which, again, is negative
and when ioctl (not sys_ioctl) returns a negative value it's an error
so ncurses thinks the ioctl fails
diff --git a/kernel/src/dev/tty/tty.c b/kernel/src/dev/tty/tty.c
index 6241929..35a7e1b 100644
--- a/kernel/src/dev/tty/tty.c
+++ b/kernel/src/dev/tty/tty.c
@@ -78,6 +78,7 @@ static size_t rw(struct vnode *curvnode, void *data, size_t offset, size_t size,
static int ioctl(struct vnode *curvnode, void *data, unsigned long request,
void *arg, void *result) {
sprintf("tty(): request is 0x%lx\r\n", request);
+ *(void **)result = NULL;
switch (request) {
case TIOCGWINSZ:
// usermode is requesting to get the window size of the tty``` this patch fixes it for memonkuous can i personally tell you how based you are
holy shit
local man finds every nyaux bug in a couple of minutes
brisk zenith
honestly i'm surprised ubsan didn't pick this one up
surreal path
nano is happy as a cow now
lesgo
now today i can either implement fbdev orrrrrrrrrr implmenet ".." handling in the vfs lookup function
brisk zenith
i think the latter is somewhat more important
surreal path
yea most likely
ill have to think of how to implement that cause that can get complicated
brisk zenith
i haven't looked at the nyaux vfs code too much but with a classic vnode system your generic lookup code would only check .. when you're at the root of a filesystem (in which case you need to delegate to the bottom of the mount stack, unless that is the root of the vfs, in which case you just treat it as .), the actual filesystem implementations are responsible for the getting-the-parent-vnode part
surreal path
i haven't looked at the nyaux vfs code too much but with a classic vnode system ...
what would that function be called?
"getting-the-parent-vnode"
surreal path
oh makes sense
broken depot
BadgerOS does it the way monkous says too; the FS finds .. for itself and the generic FS code only handles the .. of a mountpoint. The latter is simply a variable added to the file object since BadgerOS needs it opened anyway for a mount.
surreal path
ill have to have some kind of way to keep track of the parent vnode in tmpfs and devfs
shouldnt be TOO hard
broken depot
ill have to have some kind of way to keep track of the parent vnode in tmpfs and...
Most filesystems literally have .. and . entries in all directories.
brisk zenith
andromeda does it a bit differently: the dentry cache holds the parent of every dentry - filesystem implementations never see lookup requests for .. and .
broken depot
Including BadgerOS' RAMFS
surreal path
Most filesystems *literally* have `..` and `.` entries in all directories.
oh that could be easy then
broken depot
Indeed
That's how BadgerOS does it
It just silently assumes they exist and only actually intervenes if you do .. at the root of a mountpoint that isn't /
surreal path
i actually have .. already implemented??? at least in tmpfs
and devfs too???
okay so
broken depot
well that's very easy then :P
surreal path
vfs_lookup is calling lookup without any care
so
i need to figure out whats going on?
broken depot
BTW this is literally all that is needed for BadgerOS's mountpoint .. behaviour
wait wrong screenshot
Wait no correct screenshot
brisk zenith
does that handle stacked mounts correctly? or does badgeros not support those
brisk zenith
mounting multiple filesystems on the same directory
broken depot
Then no
It does support mounting filesystems recursively but not stacked, because of this.
brisk zenith
makes sense
surreal path
i think i know whats going on maybe
im not properly updating the current working directory
.. and . and friends work fine tho
it seems
broken depot
Also don't forget this snippet, which makes something like the following:
Mounts at / and /mnt: Path is /mnt/dir/../..
Work properly.
broken depot
.. and . and friends work fine tho
nice
surreal path
yea it works
surreal path
Also don't forget this snippet, which makes something like the following:
Mounts...
i dont rlly need to do this
im able to traverse file systems fine
accross both devfs and tmpfs
nvm
broken depot
i dont rlly need to do this
In BadgerOS, without this snippet, doing .. at a directory that lives in the root of some mountpoint would give you that directory handle, which can't get to the parent FS
broken depot
So if you were to look up .., you'd just get the mountpoint again, which is incorrect
Unless you have some extra logic that causes .. at a filesystem root to be directed to the parent filesystem, you will also have this issue.
surreal path
it should
broken depot
So it's just another case of debugging then
kind root
mounting multiple filesystems on the same directory
how does that work?
brisk zenith
the most recently mounted filesystem is the one that's visible, and if you unmount the most recently mounted filesystem gets unmounted
kind root
waiiit thats a thing u can do??
brisk zenith
it's basically just changing if (dentry->mounted_fs) dentry = dentry->mounted_fs->root to while (dentry->mounted_fs) dentry = dentry->mounted_fs->root in a few places
brisk zenith
waiiit thats a thing u can do??
yep
kind root
it's basically just changing `if (dentry->mounted_fs) dentry = dentry->mounted_f...
wait can u elaborate
brisk zenith
right so during lookup usually you check if the dentry/vnode/whatever is a mountpoint and if so continue with the mounted fs's root
adding support for stacked mounts basically amounts to just doing that in a loop until it's no longer a mountpoint
and the reverse for when you're dealing with ..
surreal path
idek how mounts work properly
lol
im just manually um, setting up file systems atm
broken depot
idek how mounts work properly
In BadgerOS, to mount a filesystem the kernel will ask the FS implementation to mount, then open its root directory.
Then, it registers it in a table and inserts a reference to the file object of the new filesystem's root at the mountpoint directory.
kind root
right so during lookup usually you check if the dentry/vnode/whatever is a mount...
hmm, but would all mounted fses have the same parent and thus u can just dentry->parent? why wouldnt that work
brisk zenith
hmm, but would all mounted fses have the same parent and thus u can just dentry-...
usually dentry trees are filesystem-specific i.e. do not cross mount barriers
kind root
usually dentry trees are filesystem-specific i.e. do not cross mount barriers
would root in this case point to the parent directory of the mount or the actual root
surreal path
In BadgerOS, to mount a filesystem the kernel will ask the FS implementation to ...
no idea what any of this means but ok
brisk zenith
would root in this case point to the parent directory of the mount or the actual...
root is the root dentry of the filesystem
broken depot
no idea what any of this means but ok
Do you know what a file object is?
surreal path
brisk zenith
i.e. the dentry that has parent set to null
kind root
but then your cd .. would cd into / no?
surreal path
Do you know what a file object is?
file object? i have file descriptions but idk what a file object is, usually its fs specific in how a file is represented
brisk zenith
.. is handled by the generic vfs lookup code, the fs implementations themselves never see ..
and the generic vfs lookup code knows about mounts
kind root
hmm okay ill have to think about this more, i know jack about VFSes apparently
surreal path
hmm okay ill have to think about this more, i know jack about VFSes apparently
relatable!
brisk zenith
hmm okay ill have to think about this more, i know jack about VFSes apparently
if you want to look at a dentry-based vfs impl you can use andromeda's as reference
kind root
thanks
broken depot
file object? i have file descriptions but idk what a file object is, usually its...
In POSIX operating systems, you can have multiple references to the same logical file. Either through hard links or simply opening it multiple times. Each such unique file has a file object in the kernel to store all info required to do the usual file operations. Then, a file descriptor is each handle created by an fopen call for example, adding open mode information (read, write, etc.) and current file offset (at least, this is the case for BadgerOS).
brisk zenith
the vfs code itself is capable of maintaining a dentry cache (but in andromeda this is disabled because its kfree never returns any memory to the pmm)
kind root
is it a bump allocator also?
surreal path
In POSIX operating systems, you can have multiple references to the same logical...
i have a file description which stores the seek offset and such
brisk zenith
nah it's a slab allocator it just doesn't keep track of whether the slabs are empty
kind root
ah
brisk zenith
kfree'd memory can be kalloc'd again but it can never be pmem_alloc'd again
kind root
why not?
brisk zenith
i don't remember
broken depot
@surreal path I can do VC if you want a better explanation
brisk zenith
i think it was because the state necessary to do that would've pushed page_t over 32 bytes and i didn't want that
surreal path
<@787341689333350450> I can do VC if you want a better explanation
yea might help
brisk zenith
but i'm not sure
kind root
even your bootloader has struct page?
brisk zenith
also note that in a more normal kernel the vfs code would be quite a bit more complicated than andromeda's because of locking - andromeda is single-cpu and non-preemptive so it can get away with not having any locking anywhere
brisk zenith
even your bootloader has struct page?
it has userspace, demand paging, copy on write, file memory mapping, futexes, signals, pipes, etc
kind root
bruh lol
brisk zenith
it's more of a kernel than a bootloader
brisk zenith
isnt that overkill
i wasn't really trying to write a practical bootloader but for a practical bootloader it would be overkill yes
and yes it runs boot protocols in userspace
kind root
what was your motivation for this project?
brisk zenith
i thought it'd be funny
that's really all
kind of ballooned into being able to run bash gcc etc
brisk zenith
how do u do handover and stuff
/dev/cpu has an ioctl for it
kind root
least overengineered bootloader
brisk zenith
said ioctl just sets the cpu registers (including gdtr and segregs) to arbitrary values
the linux impl uses it to start the kernel's 32 bit entry point, the limine impl uses it to start a prekernel that handles smp
kind root
can u show the actual impl of limine for example
brisk zenith
least overengineered bootloader <:LETSFUCKINGGOOOOOO:1003833035688513606>
unfortunately the overengineering makes it kind of impractical: the fact it has a proper page cache (that actually caches stuff that isn't being used, and evicts on memory pressure instead of on dereference, unlike the dentry "cache") means physical memory gets insanely fragmented as soon as you start userspace, which limits your initrd size greatly
brisk zenith
can u show the actual impl of limine for example
the limine impl is kinda huge but https://github.com/monkuous/andromeda/tree/main/sources/andromeda-boot/limine
loader is the part that runs in userspace
kind root
how long did this take you?
brisk zenith
the entire project from start to where it is now was about 2 months of active dev i think?
kind root
like full time active dev?
brisk zenith
basically yes
kind root
how do u do it so fast
brisk zenith
dozens of old abandoned projects that got to various stages of dev to steal code from
kind root
you're a student right
brisk zenith
i live with my parents
brisk zenith
nah just high school, i'm only 19
brisk zenith
uhh i got into it when i was like 12 but the first project that was more than just a kernel-mode shell i started shortly after my 14th birthday
brisk zenith
the limine impl is kinda huge but <https://github.com/monkuous/andromeda/tree/ma...
the page table creation part of the limine impl is kinda interesting actually, since you're in userspace you can't do any of the usual physical memory access stuff, and as a security feature (yes i actually bothered to implement this in what amounts to a joke kernel) the kernel heavily restricts access to kernel-managed memory via /dev/mem (when you mmap kernel-managed memory through the /dev/mem fd and try to access it you just get a sigbus, the only way to access actual ram is to allocate it through an ioctl which gives you an fd and mmap that) which makes it impractical to use temporary mappings
what i ended up going with was a "shadow" page table tree which contained the virtual addresses of the tables
kind root
bruh
couldnt you just pre-construct the page table and then map it later in the prekernel?
brisk zenith
that's what this is necessary for yes
kind root
ig i misunderstood the problem or something
do u know the physical address of the thing u have allocated?
brisk zenith
you get a physical address and an fd, and you can mmap the fd
kind root
so can't u just use that to make an in-memory page table to be loaded after handover, why do u need a shadow one?
brisk zenith
the problem is that the process of constructing said page table requires accessing page tables created in prior steps
and you can't access physical memory purely by its physical address
so you need to keep track of both the paddr (stored in the page tables themselves) and the vaddr
brisk zenith
typedef struct pte {
uint64_t *real;
struct pte *shadow;
} pte_t;
static paddr_t alloc_page_table(pte_t *entry) {
paddr_t paddr = UINT32_MAX; // the prekernel needs to be able to access the page tables
entry->real = alloc_pages(&paddr, 0x1000, 0x1000, LIMINE_MEMORY_LOADER);
entry->shadow = malloc(sizeof(*entry->shadow) * 512);
memset(entry->real, 0, 0x1000);
// don't need to clear entry->shadow, entry->real is authoritative for whether the corresponding shadow entry exists
return paddr;
}``` some of the relevant codemakes sense
brisk zenith
oh and here's limine's handover code since you were interested in that c andromeda_cpu_regs_t regs = { .eax = kernel_efer_value, .ebx = kernel_cr4_value, .ecx = top_page_table_phys, .edx = kernel_efer_value >> 32, .esi = boot_info_phys, .cs = 0x18, .ds = 0x20, .es = 0x20, .fs = 0x20, .gs = 0x20, .ss = 0x20, .esp = stack_addr, .eip = prekernel, .gdtr.limit = sizeof(boot_info.gdt) - 1, .gdtr.base = boot_info_phys + offsetof(boot_info_t, gdt), }; libboot_handover(®s);
libboot is a shared library that has some helpers that are shared between boot protocols (it includes stuff like memory map tracking, since the kernel only provides a raw memory map)
surreal path
yea i think i know whats going on and its quite dumb, literarly trying to use tmpfs to create the dev directory right? and ur trying to insert .. and ., tmpfsnodes into a directory that is using the devfs lookup function
so basically weird shit happens
and i dont even know how devfs is even functioning
holy shit
kind root
oh and here's limine's handover code since you were interested in that ```c
...
Yeah I saw that and looked at libboot as well, pretty cool
surreal path
i will fix the bug after i beat sans undertale
holy shit sans undertale wont die
die sans undertale
kind root
Don't feel bad, you're still years ahead of me 
silver yarrow
surreal path
vfs bug fixed
surreal path
kind root
make / a text file
tawdry mirage
hex edit the file system so that the root inode is a file :^)
(linux won't mount it :^)
surreal path
i need to add support for hard links but
