#Nyaux
1 messages ยท Page 24 of 1
my dad gonna be hella mad if i dont get a good grade, the exam is like tomorrow as well
๐ญ
also while i was at school i was thinking about how the memory corruption is happening
i think its cause of my memset, like either i dont memset at all or i memset the wrong length writing zero's to locations i shouldnt
i would test this but
again
exams


isn't his name mohamad
no
Emily
w h a t
iykyk
"Arab dad gets robbed"
and no it's not a name of that kind of vid on a sketchy site
idk why I thought of that but whatever
oh that
@finite summit fixed memory corruption bug but my iterator not working

causing infinite loop and pmm to die
go to SLEEP!!!
ur in Big Trouble young Man

what even is the problem
iterator not working
iterators Do Work
cause i didnt memset
seems like mine isnt
ur just doing something wrong
also wrong offsets and some stuff
u know how iterators work right?
struct holder_type2(Option<NonNull<slab_header>>);
impl Iterator for holder_type2 {
type Item = NonNull<slab_header>;
fn next(&mut self) -> Option<Self::Item> {
let mut start = self.0;
unsafe {
if start.is_none() {
return None;
}
if (*start.unwrap().as_ptr()).next_slab.is_some() {
start = (*start.unwrap().as_ptr()).next_slab;
}
return Some(start.unwrap())
}
}
}
you implemented your own iterator?
how
ur returning the wrong thing
ur doing start=next
and returning start
so ur returning next
u need to do self=next
yk
oh
new issue
someone is overwriting my slab thing
i feel like its how i calculate the objects
this is what i currently do
let obj_amount = (4096 - size_of::<slab_header>()) / size;
slab header just looks like this
#[derive(Debug)]
#[repr(C)]
struct slab_header {
obj_size: usize,
next_slab: holder_type2,
freelist: holder_type,
}
holder type is simply a tuple struct
struct holder_type2(Option<NonNull<slab_header>>);
struct holder_type(Option<NonNull<um>>);
WHATS WRONG WITH HOLDER THYPE

i dont have a choice
i cant type alias
i cant do anything, iterator wants a type that came from the crate
and im not reimplementing option its just easier to have a holder type
anyways i will pull out debugger
after i take a big shit
just so yall know
ive been trying to work on the slab allocator in userspace
this has been true hell
and even in userspace
its still
hell
even with ALL THE TOOLS
THE DEBUGGERS
even the fucking disassembly
i am
dying
of actual obos
curse
expect worse
@finite summit this is the nyaux curse
if u say skill issue rn
Award you get when starting your own curse
real
why would I? I am so lazy I do not even have the tools to code normal apps on my pc rn
like my env is a mess
so consequently I do not make anything useful, thus no right to say skill issue (it is ๐)
jk
so close
cr2=18 so
null deref ig
@finite summit at least we are closer
welp
im going sleep
i got closer
today
but
thats yk
i may spend months getting a non buggy allocator to work lmao
but i dont care
i dont care if this project takes months or years
i dont intend on abandoning it ever again
fn pop(&mut self) -> Option<NonNull<um>> {
match core::mem::replace(&mut self.freelist.0, None) {
Some(t) => {
self.freelist.0 = unsafe {(*t.as_ptr()).next.0};
Some(t)
},
None => {
None
}
}
}
core::mem::replace is goated btw
no one diffs me
debugger giving me no dice
it doesnt see the rip as something idfk
i dont think its my slab allocator tbh
i ran a for loop 10000's of times in userspace and it ran just fine
if anythings its prob the vmm
but maybe im wrong
who knows
i should sleep
but i cant stop thinking
about the issue
its 2am
my sleep schedule is so fucked
time to rewrite vmm in userspace am i right or am i right!!!
nah that wont solve anything
i actually have
zero idea
whats going on
thats the full code of my userspace slab allocator, its the same as the kernel one expect well. userspace
woahhh no way
nyauxmaster using lifetimes
???
@thorn bramble isnt that crazy???
nyauxmaster using iterators and lifetimes????
core::mem::replace???
crazy.
had the longest 2 hour back and fourth with the rust community lmao
for me to actually
implement a sane allocator
anyways gn see yall tmrw
How about rewriting the slab allocator from the ground?
again?

but im so close
@kind root
does it look like memory corruption to you
yes or no
its a null deref
im unsure whats going
no
im crashing
whats on those lines
wdym
zeroing memory you shouldn't be zeroing
fair
then it does thing->member
is ite because on alloc()
and alloc_zeroed()
i always zero my slab memory
?
as well as my vmm memory
is that bad
hmm
probably because your allocator gets corrupted somehow
i think this should be fine
so
this as well
this is fine
as well as this
so
i dont know
whats going on

thats all the memsets
thats not the bug
oh
not the zeroing itself
is just that your allocator returns random garbage or previously allocated address
so u zero it as you should and as a side effect overwrite something
YEA
YOUR RIGHT
whoops
caps lock
okay ima
figure this out
i could do this in userspace contain a hashmap of all allocated addresses, if the returned allocated address has the address in the hashmap itll panic
@kind root good idea?
i think userspace debugging is so useful
yup
i do this for uacpi sized frees mode debugging
aka making sure that uacpi frees the exact number of bytes that it allocates
gothca
@kind root
i wrote the tet
test
it succeded
both in
release and dev mode
thats the entire test
if self.hashmap.get(&y.as_ptr().addr()).is_some() {
panic!("What the Hell.");
}
self.hashmap.insert(y.as_ptr().addr(), j.obj_size);
return Some(y.as_ptr().cast::<u8>());
if let Some(t) = self.hashmap.get(&addr.addr()) {
assert_eq!(*t, self.size);
self.hashmap.remove(&addr.addr());
}
this is on free
@kind root
so it cannot be my slab allocator
may it potienally be my
vmm region allocator?
^
on alloc
and dealloc
@kind root rewrite vmm region allocator in userspace 
fun
sure, why not?
yea lets do it
yea u should
it works it just leaks memory 
can i get the page table manager part of my vmm to work
in userspace
or is that impossible
memmap
or
mmap
yea i think there would be no point
but ill do it anyway
@kind root we found are issue
***our
OUR
issue
okay
figured it out
its llvm
optizing out
my shit
yea the vmm region allocator using a Vector isnt a great idea
i will rewrite that part i suppose
FINALLY
LOOK AT THIS SHIT
FINALLY FUCKING FINALLY
@finite summit
LOOK AT THIS SHIT
FINALLY
i rewrote the entire
vmm allocator
that was hell
@kind root
WHY IS UD2
BEING CALLED
CAUSING AN INVALID OPCODE EXCEPTION
ITS NOT EVEN MY MEMORY
LIKE
MY ALLOCATORS FINE
WHY IS IT
Ud2
ing

NVM
PRO RUST DEVS
TOLD ME
to use
with_exposed_addr
to get the provence from the ptr back
ez
done
we are offically
out.
of nyaux hell
fr this time.



the day after that is gnome
maybe gnome before chrome?
im gonna
do SMP first surprisingly, then maybe the scheduler -> then timers -> then a driver abstraction interface maybe
@elder shoal u should've done this with astral 
im going to design nyaux in a good way
no more spagthati code
I did I just never did proper debugging and it got too annoying to run with smp cuz of my bad tlb shootdowns
but now after I rewrite the scheduler smp should be a net positive (wow incredible)
wheres per cpu queues
huh
๐ก
behind you
im trying to think of a good way to abstract SMP using rust
and store data about the CPU's without using dumb ah static variables
well tbh does the CPU struct rlly need to be mutable?

and how does this fit in with um
scheduing
tons of questions
im gonna have a think about this
no
why
why would it
im gonna work on parsing the kernels symbol table first
i think this would be useful
oberrow and me were chatting abt it
about having a dynamic driver loading thingy
i think this is a good idea to implement
i reintroduced a mutex into the slab allocator
and it deadlocks here
because uacpi holds the lock
towards this
and well
yea
i dont think it matters if it has a lock or no
to be honest
rust ๐ฆ will make sure the code is blazingly ๐ฅ fast ๐ and safe
for you
๐
well deadlocking is safe :^)
:)
little weird
yea ik
also have you considered using a library for doing elf parsing
it is 100% guaranteed to not crash and burn
nah im good tbh
i mean you are blind casting a pointer to an elf header so
just wanted to point that out
ik
im trusting limine isnt giving me a wrong ptr to the kernel file
cause i love limine !
what is this
reading kernel elf to get symbol table and string table, for stack trace and debugging and shi
dont u love it!
i even stored it in a hashmap
and i used the Once<T> primitive for storing it, no static mut muheheheh
no need for a hash map thats a giant overkill
store in flat array and use binary search to find address
nah hashmap easy ๐
i COULD store it in a vec
and do that
but
nah
im just
built different like that
anyways now this is done
stack trace ig !
truly 
i might use btreemap
actually
for .range
then i can find out the function
easy
make sure you don't hard rely on it in case someone stripped the kernel
(i invite you to try and see what happens by stripping the kernel yourself)
yea i know ive put it as an Option<T>
lol
(my kernel's driver loader needs kernel symbols)
sounds incredible
i used btreemap for functions like .range
while (!addr in hashmap) addr -= 1;
or array
does it matter
and get like a 1 bajillion percent performance gain
literally what i suggested
I use an rb-tree since symbol lookup by name is used a lot in the driver loader
for that a hashmap is fine too
but also loading drivers is pretty cold so a bruteforce search isnt bad either
by what name if ur doing a backtrace
if I'm doing a backtrace I do a linear search
I think that's what it's called
I start at the first symbol in the rb-tree up until the last
until I find it

not fast
@haughty kite @kind root check the spede on btreemap
oh ur doing some sort of combined approach
that sounds like a microbenchmark
(read: a massive lie)
against go?
but anyway getting tracebacks is cold
yea idk ๐ญ
which reminds me
populating the symbols is slow af
I'm not doing anything
I don't lookup symbols
like it takes 2+ seconds
i might abstract the elf loading a bit
no dynamic modules?
this just shows array is faster lmao
used to, but no
lmaooo
having no dynamic modules is an L
lit ^^
nyaux is getting dynamic drivers
just like OBOS
no working userspace is L
:(
okay mister
hey! I have two syscalls
is it?
exactly !!!!
macos doesnt have them
that work half the time
/hj
cause its a microkernel
why
whats the macos appraoch
its nice
and security
yea
link modules + kernel into a single blob
in userspace
i'll also have my own plist file format 
then sign it all together
chicken egg?
make sure to have at least 3 xml parsers in the kernel
how do i load at runtime
you dont
no it'll either be json or a sexpr thing
then thats just a normal static module?
cant use json, that's way too concretely specified
you can relink them from blobs
true
but yeah is json a good idea
or bson
like you can go (kernel + kexts) -> kernelcache
maybe
wdym
authors write json files then "compile" them
you can change the set of modules you link
you just cant do it at runtime
now if you'll excuse me, I must fix syscalls
thats bs
i see
wdym
yeah thats bs
TIL having a posix layer in userspace == hybrid
you should make a working one
okay yea true
and not tear down more code
make it to userspace
okay
with an actual kernel
ill just add driver loading
thats not a speedrun
okay
then smp
i need to redo my iokit thing
and then
devkit i called it I think
and make sure the kernel isn't cursed
yep
will do
im gonna abstract a lot of things
and
make them actually usable
wait could u run x11 on macos
is there a way to like
remove aqua
and carbon and coco
idfk
there's xquartz
oh i see
and i believe
linux kernel is a monolitic kernel
wait
i have an idea
like ill give nyaux dyanmic driver loading
okay this is way off in the future but
buuuuuut
i could have like
a package manager to get drivers
and then
yea yk what i mean
don't forget that any sort of nyaux installer using the nyaux kernel would need to have a minimal set of drivers installed
by default
yea of course
but waittttttt ugh
noooo
cause like
idfk thats for future me to figure out
lets just work on THE STACK TRACES !!!!!!!!!
whats bson
i see !
lol this is so cute
yes
better to have multiple CPUs and have your scheduler support them from the beginning
okay good point
so you don't have to modify it later
u typed it yourself and then said that its cute?
yes
:3
okay anyways ill be live streaming
if anyone wants to join
:c
oh I just noticed VCs are numbered from 0 :D
yea :)
why is uacpi::reboot not working infy
@kind root
just asking
skill issue
bru
did u implement io write
yes
it probably works for me
are u using -M q35
yea
then yeah skill issue
what
whats the status code
huh
yea
didnt give how
it returns a uacpi_status
lmao
in C
then your io write is broken
@kind root
it says in uacpi_prepare_for_sleep_state
Must be caled with interrupts ENABLED.
do they need to be enabled? or is it just preferable
so GPEs should be unmasked
it might do shit like Wait(EVENT, 0xFFFF)
unsafe fn raw_io_read(&self, addr: uacpi::IOAddr, byte_width: u8) -> Result<u64, uacpi::Status> {
if byte_width.is_power_of_two() == false {return Err(uacpi::Status::InvalidArgument);};
match byte_width {
1 => {
let value: u8;
core::arch::asm!("in al, dx", out("al") value, in("dx") addr.as_u64());
Ok(value as u64)
},
2 => {
let value: u16;
core::arch::asm!("in ax, dx", out("ax") value, in("dx") addr.as_u64());
Ok(value as u64)
}
4 => {
let value: u32;
core::arch::asm!("in eax, dx", out("eax") value, in("dx") addr.as_u64());
Ok(value as u64)
},
_ => {
return Err(uacpi::Status::InvalidArgument);
}
}
}
thats what my raw_io_read is
ok
maybe not 99% but like 95
since syscall handlers run at IRQL_DISPATCH
reboot writes not reads
and I'm making a Sys_Shutdown
oh yeah
unsafe fn raw_io_write(&self, addr: uacpi::IOAddr, byte_width: u8, val: u64) -> Result<(), uacpi::Status> {
if !byte_width.is_power_of_two() {return Err(uacpi::Status::InvalidArgument);};
match byte_width {
1 => {
core::arch::asm!("out dx, al", in("al") val as u8, in("dx") addr.as_u64() as u16, options(nomem, nostack, preserves_flags));
return Ok(());
},
2 => {
core::arch::asm!("out dx, al", in("ax") val as u16, in("dx") addr.as_u64() as u16, options(nomem, nostack, preserves_flags));
return Ok(());
},
4 => {
core::arch::asm!("out dx, eax", in("dx") addr.as_u64() as u16, in("eax") val as u32, options(nomem, nostack, preserves_flags));
return Ok(());
},
8 => {
return Err(uacpi::Status::InvalidArgument);
}
_ => {
return Err(uacpi::Status::InvalidArgument);
}
}
}
thats what i do
unsafe fn io_write(
&self,
handle: uacpi::Handle,
offset: usize,
byte_width: u8,
val: u64,
) -> Result<(), uacpi::Status> {
let ok = handle.as_u64();
let ok = core::ptr::with_exposed_provenance_mut::<IOthingy>(ok as usize);
println!("{:?}", *ok);
self.raw_io_write((*ok).base, byte_width, val)
}
okay
0x0
this is the init
its not printing
anything
after uacpi succed init
its writing to io port 0?
ill add more printf
in init yea
yeah thats already bogus
sec
yea
is len an offset
offset is offset
thats just bounds
ohhh i see right
one sec
i think i fixed it
well now its writing
still no reboot tho
oh wait
i see qemu to
--no-reboot
YEP
@kind root it reboots
fun!




if you dont care about real hardware just yet (and plan on only testing on kvm) you can do kvm clock
oh
"high presicion" my ass
my fireworks test on real hw using the hpet was running at <1 fps
as opposed to qemu+hpet running at an acceptable pace
well u still need to calibrate it
u calibrate against hpet or acpi timer or pit
but isnt the hpet inaccurate
accurate enough to calibrate stuff
okay
but not accurate enough to use for like. anything else.
and its slow as shit
on real hw it might take a huge amount of time to respond or return random garbage
depending on bus load
yea i get you
and im guessing
that after u calibrate invarient tsc
u can use it to calibrate the lapic timer?
lapic has a mode where it works with tsc as the counter
wdym
no
you calibrate the invariant tsc and/or lapic timer using the hpet/acpi timer -> pit if neither of those exist
then you use the lapic timer in invariant tsc mode to get timer IRQs
Elixir Cross Referencer - source file of Linux (version v6.11.3). Browsed file: /arch/x86/kernel/hpet.c
some notes on hpet
and the tsc as your timer tick count
rdtsc
- Legacy horrors and sins from the past.
HPETs are a complete disaster.
* That's perfectly fine. HPET is a piece of hardware designed by committee
* and the only reasons why it is still in use on modern systems is the
* fact that it is impossible to reliably query TSC and CPU frequency via
* CPUID or firmware.
*
* If HPET is functional it is useful for calibrating TSC, but this can be
* done via PMTIMER as well which seems to be the last remaining timer on
* X86/INTEL platforms that has not been completely wreckaged by feature
* creep.
*
* In theory HPET support should be removed altogether, but there are older
* systems out there which depend on it because TSC and APIC timer are
* dysfunctional in deeper C-states.
*
* It's only 20 years now that hardware people have been asked to provide
* reliable and discoverable facilities which can be used for timekeeping
* and per CPU timer interrupts.
*
* The probability that this problem is going to be solved in the
* foreseeable future is close to zero, so the kernel has to be cluttered
* with heuristics to keep up with the ever growing amount of hardware and
* firmware trainwrecks. Hopefully some day hardware people will understand
* that the approach of "This can be fixed in software" is not sustainable.
* Hope dies last...
*/
but before doing that, you check cpuid 0x15 for the frequency
if it exists
which it only does on newer hardware
and if it doesnt use lapic timer
got you
Back to the same issue 
hhow?
infy thats not the hpet itself am i correct
the variable h
#1230349543623757845 message
im not tho
im doing .read()
lol
i fixed it
i did 1 << 13
thats wrong
lmao
๐ญ
Why
What's the correct way
1 << 12
Oh
yea
okay so the hpet is actually not 64 bit
1010100010001010101000001001000
count from 0
even on kvm
idk what qemu youre using becauseo n mine its 64 bit
QEMU emulator version 8.2.2 (Debian 1:8.2.2+ds-0ubuntu1.2)
Copyright (c) 2003-2023 Fabrice Bellard and the QEMU Project developers
i dont think its a bug either
address is aligned
9.0.2
everything is correct
it was 64 bit on older versions I used so youre doing sdomething wrong
how
i lit just printed the address
not the address
the deref of the register
count from 0
its disabled
i foudn something
its for 7.2.0
tho
it was closed 6 months ago tho
This bug has been fixed in qemu 8.2.1.
oh then
wtf
its not a bug
kill me
i just figured it out
and its the most
STUPID
bug
ever fucking
made
do yall know what it is?
IT POINTED ME TO THE ACTUAL HPET TABLE
NOT THE HPET ITSELF
๐
letss gooo
okay nanoseconds is 8
or nvm
thats wrong
yea that is wrong
okay its right now i think
10 nanosecond
ill just use the lapic timer to be fair
lol
is easy
okay time to calibrate the lapic timer
okay probs my biggest commit yet
anyways
its time for scheduling @finite summit
!!!
yay
I mean you can make a smp scheduler without smp
you can add smp later on
but do smp first
gl
thanks
inb4 nyaux rewrite 3
this time in C
nopers
C? In assembly you mean
okay lapic is done and smp is kinda done lemme just add 3 cpus
yea
works fine
cool
now i just make scheduler
ez i hope
wait
a bit
gotta make some cpu structs right @finite summit
indeed
have it store the current thread
possible a pointer to the CPU's idle thread
as well as any other per-cpu scheduler stuff
like local run queues
yea i already have that
setup
i just need to make the thread struct
and process struct
and yea
then the cpu struct itself
so i can store the cpu run queues inside of the cpu local struct?

oh and how do i randomly decide when i wanna give threads to another cpu queue or smthin
like smthin rlly basic
it's terrible
threads starve
work is balanced tho
but that doesn't matter
since threads starve
there is world hunger in my kernel
I have them stored in a linear array
they're only allocated once so you can do that
yea fair
From Oberrow curse to Oberrow famine
the next one is gonna be very bad
very bad
the world if every computer ran nyaux
no, thats obos

hey you are not the one with the long standing curse
jfc (does this exist?) which part of the mm was hard to make
what fucking problems does discors have? It blacks out every now and then for no reason
idk ๐ญ
I mean which part of the memory management? The vmm?
slab allocator, vmm
@finite summit i have an idea
one sec lemme pull up microshit paint
going through space time in your obos-powered time machine
then
Kernel Panic on CPU 1 in thread 10, owned by process 2. Reason: OBOS_PANIC_EXCEPTION. Information on the crash is below:
Page fault at 0xffffffff8000dd88 in kernel-mode while to write page at 0x0000000000000000, which is unpresent. Error code: 2
Register dump:
RDI: 0xffffff0000023ea0, RSI: 0x0000000000000000, RBP: 0xffffffff80328ff0
RSP: 0xffffffff80328f68, RBX: 0xffffffff80028a92, RDX: 0x0000000000000000
RCX: 0x0000000000000000, RAX: 0x0000000000000000, RIP: 0xffffffff8000dd88
R8: 0x0000000000000000, R9: 0x0000000000000000, R10: 0x27235311250fd7ed
R11: 0x0000000400000000, R12: 0x0000000000000000, R13: 0x0000000000000000
R14: 0x0000000000000000, R15: 0x0000000000000000, RFL: 0x0000000000210282
SS: 0x0000000000000010, DS: 0x0000000000000000, CS: 0x0000000000000008
CR0: 0x0000000080010011, CR2: 0x0000000000000000, CR3: 0x0000000000006000
CR4: 0x0000000000000020, CR8: 0x0000000000000000, EFER: 0x0000000000000d01```
