#Nyaux
1 messages ยท Page 12 of 1
no
it is aligned
its still faulting

ok new problem
when mlibc tries to log via a syscall
the value of the msg is null
so i page fault
is that allowed
wtf
sounds like your sysdep is broken?
FUCKING LOOK
CHAT LOOK
WE WON
MLIBC WORKS
LOADED FROM AN ELF
OH MY GILLY GOSH
!!!!
MINTSUKI
SEE THIS

good progress
thanks
setting ldShowAuxv to true isnt printing anything

so far implmentated 2 syscalls
sys_log and sys_vm_map
i need sys_open
then these syscalls
and then i will have bash
๐
Are you telling me bash doesn't need to start processes or wait on them
uhhh no just for like
a basic prompt
like without executing processes lol
to continue tho
i need to implment file decsriptors
or whatever tf

Trivial
also with me now officially getting into userland and stuff
i wanna start thinking of how i should design my kernel
currently my kernel is a monolithic kernel
but i am a little interested in the benefits of a microkernel or hybrid kernel
im not saying im making nyaux into a hybrid kernel or heck even a microkernel
but im just saying that like
when i get a chance to breath and we have a good amount of syscalls implmented into the kernel
i might start thinking about splitting some things off into modules or smthin maybe make the kernel more modular idk
we'll see
also maybe one day ill rewrite the scheduler to support smp
later down the line
also 23d is pretty fast for proper user mode and elf loading
do i get server wr any%

oh and also we just hit 11k messages in ze thread
lol
well if you make a hybrid kernel
you'll likely need module loading
and modules will likely need kernel symbols or symbols from other modules
which pretty much means you need some sort of dynamic linker in the kernel
"only 700 lines"
๐
yeah and my entire vmm is 533 lines
im not doing a dynamic linker u cannot make me!!! ๐
old kernel is only 300 lines
bru
oh
but it supports almost all relocations for some reason
you apparently can get away with only a couple
im sorry but i got into userspace in one night with zigux :^) i was loading statically linked binaries but whatever, i had some syscalls and i was able to run an assembler, i was also learning zig at the same time!!
bruh

thats not fair, wasnt your first
but it was my first that got past creating a page table
23d was about when i had things like signals and poll()
then i lost all my motivation
Doesnt count not your first kernel ๐ค

a hybrid kernel has all the benefits of a monolithic kernel and none of the downsides of a microkernel
for what it's worth nothing that people call a "hybrid kernel" actually looks like that
all actually-existing things called "hybrid kernels" are just monolithic kernels with more features
that graph is also nonsense generally
i think whoever made the bit for hybrid kernels in particular was literally guessing, but ironically it's basically what mach 3 - which is called a microkernel - does
also, on another note, it seems you have K&R-style function definitions in your code
register ๐
so a monolithic kernel?
no, no, you are not understanding me
it's a special sort of kernel which has all the benefits of a monolithic kernel
but none of the downsides of a microkernel
how is that different from a monolithic kernel?
because it's a hybrid kernel
other than that you can pretend to be a microkernel
monolithic might be a cool word but hybrid is cooler
okay yes but in terms of functionality
it has all the benefits of a monolithic kernel but none of the downsides of a microkernel
okay but like
do you have drivers in usermode
that one is true of monolithic kernels too
stolen lol
i stole it

from 4.4bsd by the looks of things
yea
I'd recommend fixing it, and maybe not stealing code in the future
especially considering licensing
cuz the license on your repo says BSD0
oh yeah you cant do that
wait wha
and I don't think you can redistribute 4.4BSD code under BSD0
3-clause BSD, possibly 2
3-clause BSD iirc
lemme read about it
nah its okay ill just modify my license or whatever
i dont want bsd to sue me
nah, you want to get rid of that code anyways
it's easy to follow the licence, you just copy the notice
^
yea thats what i was gonna do
that's true
lol
but it's a K&R style definition
so just get rid of it
just a handful of lines
you can write it yourself
its probs gonna be shitty code so im just gonna copy the notice
no
the code you currently have is shitty
you'll improve it by rewriting it yourself
also you don't have to change the license of your entire project if you use some code licensed under a different license unless its like gpl or whatever, you just add a notice somewhere saying what part is licensed differently and the corresponding copyright/license notice
^
okay
at least if you're going to keep it
I urge you
rewrite it to ANSI-style declaration
please
also while you are at it fix all the compiler warnings (well except for unused things) :p
dunno where to steal an ansi like strncmp function
and im not touching string functions i fucking hate working on them
just rewrite the one you have
you don't need to steal you can just move the parameter types into the brackets
sek
if u do that make it a pr or somthin
turn
int strncmp(s1, s2, n)
register const char *s1, *s2;
register size_t n;
{
if (n == 0)
return (0);
do {
if (*s1 != *s2++)
return (*(unsigned char *)s1 - *(unsigned char *)--s2);
if (*s1++ == 0)
break;
} while (--n != 0);
return (0);
};
into
int strncmp(const char *s1, const char *s2, size_t n)
{
if (n == 0)
return (0);
do {
if (*s1 != *s2++)
return (*(unsigned char *)s1 - *(unsigned char *)--s2);
if (*s1++ == 0)
break;
} while (--n != 0);
return (0);
};
that's all you need to do
also fix the declarations
in the header
from:
int strncmp(s1, s2, n);
to:
int strncmp(const char *s1, const char *s2, size_t n);
same goes for every other K&R style function you have in your codebase
i believe you only have to return -1, 0, 1
i.e. strcpy etc ...
int strncmp(const char* a, const char* b, size_t len) {
for (; len; --len, ++a, ++b) {
int ret = *a - *b;
if (ret != 0 || !*a || !*b) {
return ret;
}
}
return 0;
}
add -Wall -Wextra -Werror to your makefile
well maybe not -Werror
no add -Werror
but he already has -Wall and -Wextra
bruh
this is uacpi code
im not fuckin messing with that
go ask @kind root to fix the warnings in his code
I guess it's complaining cuz the default case in that switch doesn't initalize res
yeah
uacpi_error("dead code\n");
return UACPI_STATUS_INTERNAL_ERROR;
``` add this instead of the break after default
tell him to merge that into uacpi or smthin
what
its a false positive
no its not
that case is unreachable
i dont do that
or return internal error or whatever
yeah this probably
then go do that
you should build with -Wall -Wextra -Werror tbh
I'd ignore that for the time being if I were you
its not gonna build with -werror cause uacpi
maybe nyaux is using some new gcc
i am
the official fix will be return UACPI_STATUS_INVALID_ARGUMENT
well new warnings with new compilers is always an expected thing
the vmm alone has 30 warnings
fucking pain

burh
frick am i supposed to fix these type of warnings
yes
bruh
just use zero in place of NULL if its an integer
ill just cast null to uint64
bruh

its not even a pointer that you are comparing so why do you want to compare it with NULL
yes but it doesn't really make any sense to do that
๐ฆ
unless you purposefully want to make bad code
also what on earth is that line
checks if theres a page table entry at that index
anyways fixed all vmm.c warnings
the heck is this warning for
it doesn't make any sense to have volatile on a return type
at the very least when its void
right
probably not otherwise either
okay fixed a bunch of warnings
be happy now
๐
anyways
i need to work on making file descriptors or whatever tf those are
okay so a file descriptor is like
an integer
wdym
struct file_descriptor
{
struct vnode *ptr;
}
struct file_descriptor descriptors[MAX_FILE_DESCRIPTORS];
or maybe im
misunderstanding
am i misunderstanding?
i think you want another level of indirection between a file descriptor and a vnode?
wdym
fd table -> open_file * -> vnode *
and what is this open file structure consists of
things like the current position in the file for an example
why would u store that
and mode its opened with
wdym mode?
because when you read one char then read another, you arent reading from the start again
huhhhh
with a file with contents hello, you would expect these results getc() -> 'h' getc() -> 'e'
which means you need to store the position somewhere
actually you need even more indirection? fd table -> fd -> open file -> vnode
fd has flags like CLOEXEC, open file has offset
well akshually the second one is called a file description
(also abbreviated fd)
fd is just an index into fdtable
ah
yeah
no i mean a struct fd that the fd table contains
this is getting confusing
close on exec
that it says on the tin
execute what
when you exec file descriptors with CLOEXEC set are automatically closed
closes the file when you call exec
bro doesnt know what exec is
bru
when you exec, file descriptors with cloexec set are automatically closed
the most common unix primitive
bru im confused (i barely know shit about unix internals, so its difficult to make my kernel unix like if i dont know unix internals in ze first place
)
im gonna watch a few videos on unix internals ig
this playlist seems cool
GUYS OD U SEE THAT
S-S-S-S-S-SUS????
THE FUNNY WORD????
#amongusreferencefoundinunix
so funny.
i know right
the sus was first.
yes i thought you meant file descriptor
not description
i guess this is one of the mistakes of writing a kernel too fast, you miss key aspects about basic os design (in this case UNIX) and you spend way more time then u should learning about them cause you didnt take the time to learn about them
dunno if this made sense but yea
well to be fair
this is my first real os afterall, i never rlly got far with neptune or photon cause those fucking sucked and i didnt understood mostly of what i was doing
to which i do now
and the faact that i got this
far
into userland?
with elf loading and mlibc
im rlly proud of myself
and in just under 30 days too
im happy that all the late night hours working on this project paid off to something
the kernel may be spaghetti code, and i literarly only have 2 syscalls but still
it paid off to something
kitty ๐ฑ
whats with the marks on the kitty
what happened
???
are those kiss marks?
poor kitty
also @surreal path can you commit the .gitmodules file when you do your next commit?
without it being there you have to manually clone uacpi and the submodule is useless
i dont know how to set that up

its set up by git
you should have it if you added a submodule with git submodule add
I wonder what exactly is the uACPI thing you have in your repo then
uacpi
I mean obviously but it just results in an empty folder when you clone it and it shows it the same way as submodules in github ui
i dont know how to fix that
cause i didnt use git submodule add
for uacpi
i just cloned it
ah yeah that probably explains it
[submodule "uACPI"]
path = kernel/src/ACPI/uACPI
url = https://github.com/UltraOS/uACPI.git
``` just put this into a file named .gitmodules in your repo root
you mean files like platform/stdlib.h?
well actually no you don't need to modify that as you can just create a file named uacpi_stdlib.h somewhere in the include paths and define UACPI_OVERRIDE_STDLIB
that would recursively include it, assuming you do have src/ACPI added to the include paths
you don't want to include platform/stdlib.h in your uacpi_stdlib.h
ok
and you add the define in the makefile using -D
cppflags
did you add -I src/ACPI to cppflags too
assuming you didn't modify anything else within uacpi then yeah that should be enough
looks like you also should add -D UACPI_SIZED_FREES to cppflags as you are using the sized variant of uacpi_kernel_free, include lib/nanoprintf.h in uacpi_stdlib.h and define PRIu64 to "lu" there too because uacpi still looks to be using that in a few places instead of UACPI_PRIu64
after those changes it builds straight from git (after either cloning it with git clone --recurse-submodules or running git submodule update --init after cloning it)
ok done
ok works now
anyways i have a question, so when loading ld.so
what does ld.so do
what does mlibc do
im curious
go read the code
ld.so is the dynamic linker
it looks at the executable
loads all the needed libraries
relocates them
resolves symbols
jumps
so why does ld.so call sys_openat
or sys_open
is it trying to get some libraries
to load
?
lol
also another question, about sse and stuff
i heard i need to do fxstore
and fxload
after every context switch
or smthin
is this correct?
do i have to store the value of these registers in the thread struct somewhere?
or am i misunderstanding
yes, sort of
you do need to save and restore them
at the same time you save and restore other registers
except the fpu state is variable size
that's if you're using xsave
oh
well with the legacy fxsave its not
oh fair enough
why not
it seems quite complicated and i dont wanna dive into the SDM head down for days trying to setup xsave

bruh
also some cpuid for the save of the xsave region
and some writing to some fpu register
to enable it
xcr0
or somethin
i dont think its a big deal if my kernel supports xsave or not
it kinda is
why
no
you lose a lot of sse
you need it for avx
stuff
whats avx and why do i need it
which user space might want to use
advanced vector extensions
if you compile userspace stuff with march=whatever then userspace might use it
but fpu state can have variable region size
if you use xsave
which you might want to do
you use cpuid to get the size of xsave region
oh
sorry subleaf?
sdm says subleaf is ecx
i think
so i need to put 0xd in eax
and 0 in ecx
and then run the cpuid instruction
correct?
indeed
then the output ecx should have the size of xsave
note you should probably check if xsave is supported beforehand
and maybe fallback to fxsave if it's unsupported
anyway gtg
in bytes?
yes
yes
okay
get_xsave_region_size:
push rbp
push rbx
mov rbp, rsp ; stack frame lol
mov eax, 0xd
mov ecx, 0
cpuid
mov rax, ecx ; put return value
pop rbx
pop rbp
ret
then i can call it in C
you dont need the first 2 instructions and pop rbp if you dont touch the stack
you have to preserve rbx
also just make a cpuid function in c that does cpuid using inline asm and returns a struct
its more useful
cpuid touches rbx tho
okay updated
xsave region is of 2696 bytes
how big can a xsave region be?

that sounds reasonable
wait until you realize that avx512 exists
it might grow even bigger if you enable that
now should i just store a pointer to the xsave region allocated on thread creation for each thread struct?
yea
struct Thread
{
...
void *fpu_state;
...
}
fpu_state
fine fpu_state
probs should store this in cpu_context
to be fair
doesnt matter
okay and what else do i need to do
on context switch just do the instruction "xsave"?
and xstor?
yeah first save the old thread's context using xsave and then load the new thread's context using xrstor
also you need to setup the xcr0 register once at the same time when you get the xsave region size for an example
the two first bits you can always enable yeah
oka
for the rest you should query cpuid to see if they are supported (mostly just avx and avx512) before enabling
why care about avx512
well if you don't compile userspace programs with it then you don't need to care
just steal my cpuid inline asm function or smth
where
its gpl according to the repo license but I don't care if you just steal it, its so small anyway
u sure?
yes
thanks qwinci
and I mean you have to modify it a little to work with c anyway 
yea 
also one thing with the xsave area is that it has to be 64-byte aligned so you probably want to just align the size up to page size and then allocate that amount of virtual pages (unless your kmalloc gives you addresses that are aligned to that)
๐
ill align to page size then np
bru what
xcr0 is a register
tf
you use xsetbv to write to there
oh
xsetbv %0
no wait i need to put the value in ecx
this is where my inline asm knowledge dies
asm ("" : outputs : inputs (and for outputs you prefix the modifier with =), for ecx you use "c", for eax "a", for edx "d"
oh
or actually there is a way which might be easier to understand, c void wrxcr(uint32_t index, uint64_t value) { register uint32_t ecx asm("ecx") = index; register uint32_t eax asm("eax") = value & 0xFFFFFFFF; register uint32_t edx asm("edx") = value >> 32 & 0xFFFFFFFF; asm volatile("xsetbv" : : "r"(ecx), "r"(eax), "r"(edx)); } "r" normally specifies that it can be any register but if you define a variable like that with register and asm(<reg name>) it uses the register specified in that asm
yeah thats fine, I just made this up as I don't do it using register/asm like this, but imo this is easier to understand if you don't know the inline asm register modifiers
I think you can choose what states (like avx etc) from the ones you have enabled in xcr0 you want to save, you can just pass all ones to it to save everything you have enabled
same with xrstor
all ones
I meant like all bits as ones
also you want to use the 64 bit versions of xsave/xrstor which means postfixing them with q
I don't think you need it for that
ok
you need it for xsave/xrstor because they use a different state format depending on whether you use the 32-bit or 64-bit one
okay these are my functions
i think they look okay
xsetbv causes invalid opcode?

there is a cr4 bit for it
oh
yea bit 18
ok enabled
now on thread creation just create fpu_state
allocate size of xsave_region
and whenever we switch threads
use xsave store and xrstor
?
that it
should kernel threads also have fpu state
if your kernel doesn't use sse or whatever else then they don't need to
yes
you might need to make xsave/xrstore static inline instead of plain inline though because inline sucks in c
or just make them normal non-inline functions in a c file and then put decls to a header
probably because you need to put parens around the %0 as they expect memory operands
oh also there is some special values you want to put in fcw/mxcsr parts of the fpu state that you can look up in the sysv amd64 spec when you create the state (and other than that it should be zeroed out)
that was actually rlly easy
fcw?
mxcsr?
(and you find fcw/mxcsr registers in the fxsave structure that is at the beginning of the xsave region)
wha
just look at the sysv abi spec and sdm
umm sysv abi and sdm spec dont say much about it
i dont have a fxsave structure
i just
void *fpu_state
well you can just create it and then cast your fpu state to it
"initial stack and register state" in sysv abi tells you what the values should be and you can find the structure of the fxsave area in the sdm (and also what the various bits in the registers are)
i cant find the fxsave structure

but i do see what im supposed to set fcw and mxcsr to
sdm is pain to search through
is this it?
look up fxsave in the instruction listing in vol2 chapter3 and then look at the ia32e part (with REX=1)
yes that
what are the numbers on the top
bytes from the offset on the right
okay so mxcsr and fcw is all i care about
struct fpu_state_t
{
uint16_t fcw;
...
uint32_t mxcsr;
}
fcw is two bytes
okay
thats not correct
wdym?
look at the image you sent
yes
last one should be 6 bytes
instead of 8
ig
okay there
i should also make the struct packed ig
first rev should be 6 bytes
yes
I don't think the values you put there are correct though, the sysv abi just tells you the bit values of the individual fields, they are not in that order in the actual control word/mxcsr
oh

then what is the values that are correct
8.1.5 and 10.2.3 tell you the positions
I mean yeah you could just steal those, it might be easier 
MIT
do i have to put a copyright notice or
it already says stolen from managarm
yea but like i need to put the actual copyright notice
ok
don't think constants like that are even copyrightable, considering headers aren't (iirc?)
shhh stop exposing my plan
that would be unfortunate
enabled
no, no it does not
well then you'd also need to save fpu state on syscall entry and restore on exit
and same for interrupts
sounds like pain
and context switch
pain
save user - restore kernel - save kernel - restore user
also mega slow
4 fpu reloads per switch
yea sounds mega slow
anyways pushed fpu changes to git
time to crawl back into that unix playlist
til bro talks about file descriptors
so i can flipping implment them
cause i wanna make SOME progress to bash
im gonna make this a challenge
every day for the next week i must implement at least 2 syscalls
pins this
keeping myself accountable with this
what was your end goal
both should be easy
don't really need to restore kernel for just syscall/irq entry, since it's a fresh function call
if you know what you're doing you can speedrun those in 24 hours
fair
maybe ill do a speedrun to xorg in another project when i reach my end goal with nyaux whatever it is
idk
yeah you need to slow run it before you can speed run it
fr
yo someone pin this whoever has mod powers or whate
thanks qwinci
anyways i have 2 hours to implment 2 syscalls cause my dad forcing me to sleep early
so
lets go
you're 17 right?
ye
interesting
why u askin lol
last time my parents forced me to sleep I was like 14
lol
they are just plain integers, you just have some sort of way to get a struct out of one of them
like a dynamic array or whatever that you can index with it to get the corresponding struct
no they arent cause what qookie said earlier
there are things called file descriptions
and
stuff
yes but it still works the same
you map an integer to a struct
you could probably just ignore the second layer of indirection for now and just store the access offset + mode + vnode within that struct
whats the purpose of the second layer of indirection
idk tbh
and what does it mean to "open" a file and to "close" a file in this situation, like the file is right there in memory why do i need to "open" it
does that just return a fd
or what
stream reading/writing
it might not be in memory if it hasn't been opened yet and it's from an on-disk fs
yes, you allocate an fd and then make that fd's entry within the fd->struct map point to a new struct that you make and then return that fd
allocate an fd? isnt a fd just an integer?
yes
so why do we need to "allocate" an fd
I mean allocating them can mean just incrementing a variable you have inside the process
but you likely want to reuse closed fds too
so you have some sort of vector of freed fds you can look at first before creating a completely new fd by incrementing the integer inside the process or smth
can threads have separate file descriptor tables
I do that in my kernel
if you want to follow posix then I don't think
ok
struct FileDescriptor
{
struct vnode *ptr;
uint64_t offset;
}
struct ProcessInfo
{
struct FileDescriptor Descriptors[MAX_FILE_DESCRIPTORS];
uint8_t descriptor_bitmap[MAX_FILE_DESCRIPTORS];
}
somethin like this?
or misunderstanding?
something like that yeah
okay
how many file descriptors should a
process have
whats the max
value
what value should i set that
to be
is 256 too little?
I think linux has 1024 by default
oko
i made mine like 64
or you can just dynamically resize the array
do you not have some kind of struct for a dynamic set of objects
wha
why not
so on sys_open
the kernel should
find the vnode from the path
find an avabile fd
get the empty entry from within the array
set the ptr to the vnode found from the path
and whenever the process tries to read with an offset or write
update the offset of the fd
correct?
you update the offset every time a thread reads or writes by the count bytes
or when the thread seeks then you set the offset to the value where it wants to seek at
you might want the mode as well
read/write
why do we need modes
well for an example you may only have read permission to a file
makes sense
idc about permissions!!!
so im not gonna worry
๐
#ihatesecurity
#alwaysassumetheuserisright
can directories have file descriptors?
?
yes
the open operation from userspace should be distinguished from an underlying vnode operation
the open syscall will probably include code like this
vnode = cwd_vnode->lookup("whatever/path/asked/for");
if (vnode != NULL)
if (vnode->open() didn't return an error)
then do the rest of the work
the open vnode op on the other hand may do next to nothing, as in e.g.:
https://github.com/NetBSD/src/blob/trunk/sys/ufs/ext2fs/ext2fs_vnops.c#L174
the amount of bytes read
why
because it failed to open the file
okay fixed bug in my vnode lookup
lol
now we gotta give mr sys_seek
what is whence
the heck does that mean
๐ญ
where to seek from
man 2 lseek
specifically
whether to seek from the current file pos
the end of the file
or the beginning
because the offset can be negative?
oh right
so you can do things like "-10 bytes from the end"
something includes cstdio
generic.cpp
yes
and
im not modifying anything

@tawdry mirage do u know whats going on??
are you sure clangd didn't help you by adding the cstdio include
idk
lemem check
ok works now lol
me when clangd
bru
bro what
u gotta be kidding me
๐
oh
flags are wrong values
what ze fuc
oh right
brub
what is bro trying to do
map memory for phdrs
probs do something other then map_annoymous
it specifies MAP_ANONYMOUS | MAP_FIXED
right
Bro just switched on a flags field
so bro is trying to map fixed
Why would bash want a fixed mapping?
ld.so mapping in library phdrs
Oh
yes
what flags are you enabling in meson.build for your os?
uhhh
hm if you copied vinix then none
yea
oh it's MAP_PRIVATE | MAP_FIXED | MAP_ANONYMOUS
right
Whats unclear
blud is trying to memcpy
everything
epic mapping failure
Although I think its doing a -1 for the fd
someone is clearly trying to memcpy this
So I dont know what thats supposed to do
MAP_ANONYMOUS allocates fresh memory instead of using a file as the backing memory
sys_read?
So what's the purpose of PRIVATE?
bru
so it's not shared on fork
Wait anon memory is shared on fork?
so clearly someone is giving me invalid memory to whatever
And not cowed?
could it be buf? 
nope
it is
buf
for some reason buf is invalid???
like sorry
what
if you use MAP_SHARED

