wym
#Nyaux
surreal path
surreal path
i already do that
surreal path
okay
flat nymph
Yeah probably Arc if you don't have RCU
surreal path
#[repr(C, packed)]
pub struct perthreadcpuinfo {
kernel_stack_ptr: *mut u8,
user_stack_ptr: *mut u8,
current_thread: *mut Thread
}
something like this?
surreal path
Yeah probably Arc if you don't have RCU
what
flat nymph
Arc<Mutex<Box<dyn
flat nymph
Or a bunch of unsafe code
surreal path
```rs
#[repr(C, packed)]
pub struct perthreadcpuinfo {
kernel_stack_ptr: *mu...
can i even store an arc in a repr c struct
in gs
flat nymph
Why can't you?
surreal path
oh okay
flat nymph
can i even store an arc in a repr c struct
Also it doesn't have to be repr c
Just store a pointer to normal struct
surreal path
okay, does it have to be packed?
flat nymph
CPU loads it for you already
surreal path
no
flat nymph
From tss
surreal path
in syscall
flat nymph
Oh
flat nymph
Also it doesn't have to be repr c
Ignore this probably (I need sleep)
surreal path
pub fn scheduletask(mut regs: *mut Registers_Exception) {
// pick a queue
let que: &mut Vec<cpu_queue> = unsafe {
&mut ITIS.as_mut().unwrap().queue
};
let mut got_you = que.pop();
if got_you.is_none() {
return;
}
let mut r = got_you.unwrap().queue;
if r.is_none() {
return;
}
let mut new_process = r.as_mut().unwrap().pop();
if new_process.is_none() {
return;
}
let mut new_thread = new_process.as_mut().unwrap().lock().threads.pop();
if new_thread.is_none() {
return;
}
let info = unsafe {
let e = rdmsr(0xC0000102) as *mut perthreadcpuinfo;
e
};
// get current thread
let thr = unsafe {
(*info).current_thread.clone()
};
save_context(regs, thr.lock().context);
let fs = rdmsr(0xC0000100);
thr.lock().fs = fs;
switch_context(regs, new_thread.as_mut().unwrap().lock().context);
wrmsr(0xC0000100, new_thread.as_ref().unwrap().lock().fs);
wrmsr(0xC0000102, new_thread.as_ref().unwrap().lock().gs_base as u64);
let o = thr;
new_process.as_mut().unwrap().lock().threads.insert(0, o);
r.as_mut().unwrap().insert(0, unsafe {(*info).current_thread.lock().process.clone()});
}
i think this is good
i was forced to um
use Arc<Mutex>>
its fine whatever
finite summit
20002th message
elder shoal
20003th message
finite summit
20004th message
flat nymph
20005th message
plush hearth
20006th message
finite summit
200007th message
edgy pilot
2000008th message
surreal path
hard at work on scheduler
also lol
okay im having issues
trying to create the queues and shit its proving difficult
using arcs and mutexes everywhere doesnt help
i might just have to go with raw pointers
unless yall have a better idea
??
chat i do not know how to implment the scheduler in safe rust with SMP 
molten grotto
just do it™️
surreal path
just do it™️
bro i did
wanna see my first shitty implmentation that page faulted INSTANTLY
:3
molten grotto
wanna see my first shitty implmentation that page faulted INSTANTLY
I mean you are even using a physical address directly in it again 
surreal path
I mean you are even using a physical address directly in it again <:meme:5754450...
yea but I had ABSTRACTION over it
like a ton of shit
boxes and what not
surreal path
no like
theres only one time
i used a physical address
how does that cause insta page fault
😭
molten grotto
well it would cause a page fault when you'd switch to that new thread
and then it would also fault after that thread has printed hi
molten grotto
the first one because you are using a physical address as the sp for the new thread
second because there is nothing that the thread would return to
surreal path
the first one because you are using a physical address as the sp for the new thr...
right
surreal path
second because there is nothing that the thread would return to
wdym
molten grotto
extern "C" fn lol()
{
println!("hi")
}
what do you think will happen when it will return from that
molten grotto
```rust
extern "C" fn lol()
{
println!("hi")
}
```
well in that case yeah but the gist literally had this
surreal path
i see
surreal path
the first one because you are using a physical address as the sp for the new thr...
SO CLOSE
rayan@pop-os:~/NyauxKT$ addr2line -e kernel/kernel ffffffff8001b9c2
/rustc/9c01301c52df5d2d7b6fe337707a74e011d68d6f/library/core/src/ptr/non_null.rs:366
rayan@pop-os:~/NyauxKT$
what does that mean
null deref?
CR2=0000000000000010
cr2 is 10
i added the hhdm offset
so why is it page fault
hmmmmmmmmmmmmmmmmmm
molten grotto
gdb
surreal path
gdb
yeaye ik
pulling up gdb now
btw qwinci do you think i rlly needed all that arc mutex bull shit
on the vecs
molten grotto
well like on the cpu frame struct
*cpu_context
no need for it to be a raw pointer inside a box
molten grotto
either store it as frame: Registers_Exception or make new return a non boxed cpu_context
surreal path
either store it as `frame: Registers_Exception` or make new return a non boxed c...
okay fine
ill make non boxed cpu_context
molten grotto
I mean not that this really matters, its just weird
same kinda goes for the queues being in a vector instead of being in your struct cpu or whatever
but eh
surreal path
same kinda goes for the queues being in a vector instead of being in your struct...
okay
what else
molten grotto
idk
surreal path
alr
molten grotto
I could go over it and tell you all the non idiomatic things that it does but I don't feel like that rn 
surreal path
okay i fixed that
still page faults
but gdb time ig
yall want a stream in #voice-0 or nah
guess not lol
#0 core::ptr::non_null::NonNull<alloc::sync::ArcInner<spin::mutex::Mutex<NyauxKT::fs::devfs::devfsfile, spin::relax::Spin>>>::as_ref<alloc::sync::ArcInner<spin::mutex::Mutex<NyauxKT::fs::devfs::devfsfile, spin::relax::Spin>>> (
self=0xffff80007bba0358) at /rustc/9c01301c52df5d2d7b6fe337707a74e011d68d6f/library/core/src/ptr/non_null.rs:366
#1 alloc::sync::Arc<spin::mutex::Mutex<NyauxKT::fs::devfs::devfsfile, spin::relax::Spin>, alloc::alloc::Global>::inner<spin::mutex::Mutex<NyauxKT::fs::devfs::devfsfile, spin::relax::Spin>, alloc::alloc::Global> (self=0xffff80007bba0358)
at /rustc/9c01301c52df5d2d7b6fe337707a74e011d68d6f/library/alloc/src/sync.rs:1823
#2 alloc::sync::{impl#28}::clone<spin::mutex::Mutex<NyauxKT::fs::devfs::devfsfile, spin::relax::Spin>, alloc::alloc::Global> (self=0xffff80007bba0358) at /rustc/9c01301c52df5d2d7b6fe337707a74e011d68d6f/library/alloc/src/sync.rs:2102
#3 0xffffffff8000dba1 in NyauxKT::fs::devfs::{impl#2}::create (self=0xffff80007ff0c2c0, name=...)
at src/fs/devfs/mod.rs:55
#4 0xffffffff8000e29a in NyauxKT::fs::devfs::{impl#4}::dev_create (self=0xffff80007ff0c2c0, name=..., ops=...)
at src/fs/devfs/mod.rs:122
#5 0xffffffff8000eecf in NyauxKT::fs::devfs::devfs_init () at src/fs/devfs/mod.rs:153
#6 0xffffffff80000a36 in NyauxKT::kmain () at src/main.rs:90
huh???
devfs is panicking
wha
lemme comment out sched init
WTF IS GOING ON
AAAAAAAA
surreal path
i cannot debug wtf is going on
it doenst fault
when sched init is not happening
when sched init happens
boom
fault
page fault !!!!
can anyone help im kinda stuck
wait lemme debug sched init
one sec
GALAXY BRAIN IDEA
surreal path
20003th message
LUCKY
chat does that make my thread
the most messages
????
😭
haughty kite
```rs
#0 core::ptr::non_null::NonNull<alloc::sync::ArcInner<spin::mutex::Mutex<...
are you building your kernel in debug mode?
also
miriiiii
tawdry mirage
i think miri would have a stroke on nyaux :^)
surreal path
are you building your kernel in debug mode?
right miri
rayan@pop-os:~/NyauxKT/kernel$ cargo miri build
fatal error: cargo miri supports the following subcommands: run, test, nextest, clean, and setup.
rayan@pop-os:~/NyauxKT/kernel$
i cant
surreal path
o
haughty kite
or cargo miri test or whatever
surreal path
or cargo miri test or whatever
its only complaining about the tests
haughty kite
its only complaining about the tests
miri is dynamic analysis so duh
surreal path
miri is dynamic analysis so duh
what about my other code
like the scheduler code
its not complaining about that
test fs::USTAR::populate_vfs ... error: unsupported operation: `open` not available when isolation is enabled
--> /home/rayan/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/std/src/sys/pal/unix/fs.rs:1175:36
|
1175 | let fd = cvt_r(|| unsafe { open64(path.as_ptr(), flags, opts.mode as c_int) })?;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ `open` not available when isolatio
theres an env var to turn it off lol
surreal path
whats the env var
surreal path
Running `/home/rayan/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/bin/cargo-miri runner target/miri/x86_64-unknown-linux-gnu/debug/NyauxKT`
error: no global memory allocator found but one is required; link to std or add `#[global_allocator]` to a static item that implements the GlobalAlloc trait
error: unwinding panics are not supported without std
|
= help: using nightly cargo, use -Zbuild-std with panic="abort" to avoid unwinding
= note: since the core library is usually precompiled with panic="unwind", rebuilding your crate with panic="abort" may not be enough to fix the problem
error: aborting due to 2 previous errors
ummm
its using x86_64-unknown-linux-gnu????
ummm
trying to do the ocmmand gives this
rayan@pop-os:~/NyauxKT/kernel$ cargo -Zbuild-std with panic="abort" miri run
error: no such command: `with`
Did you mean `fetch`?
View all installed commands with `cargo --list`
Find a package to install `with` with `cargo search cargo-with`
rayan@pop-os:~/NyauxKT/kernel$ cargo -Zbuild-std with panic="abort" miri run
oh my fucking god
surreal path
oh my fucking god
i did MIRIFLAGS=-Zmiri-disable-isolation on cargo miri test
all passed
surreal path
okay
flat nymph
btw qwinci do you think i rlly needed all that arc mutex bull shit
You would typically use intrusive lists (?)
I don't know how that would look with Rust
There is a crate for it...
haughty kite
I don't know how that would look with Rust
they work incorrectly
flat nymph
Alternatively, at some point I just had sched queues with intrusive lists with shared_ptr in my kernel
surreal path
they work incorrectly
pitust how to use mmap
in rust
wihtout any crate
???
otherwise i cannot write this test
without 1 SINGULAR
page
haughty kite
you can just extern "C" fn mmap lol
flat nymph
Typical Rust
surreal path
you can just extern "C" fn mmap lol
the test run just fine
the sched init
ran fine under miri too
molten grotto
do you still have the same issue
surreal path
yea
molten grotto
what did gdb say
surreal path
```rs
#0 core::ptr::non_null::NonNull<alloc::sync::ArcInner<spin::mutex::Mutex<...
^
and not having sched init it runs fine
surreal path
what did gdb say
.
molten grotto
well then ig you do something weird in the sched init, can't really tell what
molten grotto
what more does the real thing do
surreal path
i literarly let the cpus schedule their shit
molten grotto
hmm
surreal path
@molten grotto
fn sched_queuetest() {
extern crate libc;
use std::io;
use std::ptr;
use libc::{mmap, munmap, sysconf, _SC_PAGESIZE, PROT_READ, PROT_WRITE, MAP_PRIVATE, MAP_ANONYMOUS, MAP_FAILED};
use std::println;
use std::thread;
let mut fake = cpu_queue {queue: Some(Vec::new()), lapic_id: 0};
let mut process = Arc::new(Mutex::new(process::new(None, unsafe {None}, "hi".to_string())));
// Map one page of memory into the process (anonymous memory, not backed by a file)
let addr = unsafe {mmap(
ptr::null_mut(), // Let the OS choose the address
4096, // Size of the mapping (one page)
PROT_READ | PROT_WRITE, // Read/write access
MAP_PRIVATE | MAP_ANONYMOUS, // Private anonymous mapping (not backed by a file)
-1, // No file descriptor (since this is anonymous memory)
0, // Offset (ignored for anonymous mapping)
)};
// Check if mmap failed
if addr == MAP_FAILED {
assert_eq!(1, 2);
}
// Cast the returned address to a mutable u8 pointer
let data = addr as *mut u8;
let kstack = data;
let mut context = cpu_context::new(lol as u64, false, unsafe {kstack.offset(4096) as u64});
let mut thre = Arc::new(Mutex::new(Thread::new(process.clone(), context, "kthread".to_string())));
let per = perthreadcpuinfo::new(unsafe {kstack.offset(4096)}, 0 as *mut u8, thre.clone());
thre.lock().gs_base = Box::into_raw(per);
unsafe {
let o = fake.queue.unwrap();
let mut j = o;
process.lock().threads.insert(0, thre);
j.push(process);
}
unsafe {munmap(
addr as *mut _,
4096
)};
}
what about you try switching to the some other thread in the test
though miri probably doesn't like that to be fair
surreal path
what about you try switching to the some other thread in the test
how tf am i supposed to do that
its in an std enviroment
molten grotto
you can still switch contexts around in the same priv ring
as long as you don't use iret or smth to do it
surreal path
how
molten grotto
the same way as in the kernel?
surreal path
just dont use iret?
molten grotto
yeah you'd have to do it with normal ret
molten grotto
yes
surreal path
and push them
molten grotto
yes
flat nymph
they work incorrectly
Who do Rust kernels do though?
I know Linux just ships stuff like RCU to C
haughty kite
Who do Rust kernels do though?
I know Linux just ships stuff like RCU to C
none of them are particularly great afaik
but also you can write linked lists that, while incorrect, do always produce correct code
all hail rust btw
flat nymph
But then "who is the owner"
molten grotto
well yeah it would be unsafe
unless the objects are actually owned by the list
which ig would work as long as an object is part of only one list
flat nymph
If you pin threads to CPUs that may work...
molten grotto
well ofc you could still take them out of the list and move to other but like at the same time
flat nymph
But like it's inevitable to be referencing them on other processors
surreal path
yes
im gonna breakpoint the sched tick function
foudn the issue @molten grotto
let thr = unsafe {
(*info).current_thread.clone()
};
dereference info
cause a page fault
molten grotto
nice
surreal path
yep
null deref
let info = unsafe {
let e = rdmsr(0xC0000102) as *mut perthreadcpuinfo;
e
};
hmmm
yea so
theres nothing in the kernel base msr
molten grotto
that's the inactive one
the kernel base msr always contains the currently inactive gsbase (and the currently used one is in the gsbase msr)
surreal path
gotcha
surreal path
that's the inactive one
okay it fixed that but
now a new issue
the entry function isnt executing
and it panics on non kvm
lovely
page faults*
yea i dont know whats going on
info is still 0x0 lol
molten grotto
do you actually write something to the msr
surreal path
yes
#[no_mangle]
pub fn scheduletask(regs: *mut Registers_Exception) -> Option<*mut Registers_Exception>{
// pick a queue
let que = unsafe {
&mut ITIS.as_mut().unwrap().queue
};
let mut got_you = que.pop();
if got_you.is_none() {
return None;
}
let mut r = got_you;
if r.is_none() {
return None;
}
let mut new_process = r.as_mut().unwrap().lock().queue.as_mut().unwrap().pop();
if new_process.is_none() {
return None;
}
let mut new_thread = new_process.as_mut().unwrap().lock().threads.pop();
if new_thread.is_none() {
return None;
}
let info = unsafe {
let e = rdmsr(0xC0000101) as *mut perthreadcpuinfo;
e
};
// get current thread
let thr = unsafe {
(*info).current_thread.clone()
};
save_context(regs, &mut thr.lock().context);
let fs = rdmsr(0xC0000100);
thr.lock().fs = fs;
let cc = switch_context(regs, &mut new_thread.as_mut().unwrap().lock().context);
wrmsr(0xC0000100, new_thread.as_ref().unwrap().lock().fs);
wrmsr(0xC0000102, new_thread.as_ref().unwrap().lock().gs_base as u64);
let o = thr;
new_process.as_mut().unwrap().lock().threads.insert(0, o.clone());
r.as_mut().unwrap().lock().queue.as_mut().unwrap().insert(0, o.lock().process.clone());
Some(cc)
}
oh wait i havent written gs_base into
the msr on init
right
okay @molten grotto i do this now
wrmsr(0xC0000102, thre.lock().gs_base as u64);
yet it still pages faults fun
molten grotto
you write to the inactive msr
surreal path
oh
surreal path
the kernel base msr always contains the currently inactive gsbase (and the curre...
no i think im getting it confused now
on init
should i write to which msr
the kernel
or the gs base
cause clearly
somethings not working
molten grotto
very likely to the gsbase
molten grotto
no you read both from the same gsbase msr
molten grotto
they don't change around unless you swapgs
molten grotto
to start with no if you write to the gsbase
surreal path
let info = unsafe {
let e = rdmsr(0xC0000101) as *mut perthreadcpuinfo;
e
};
i read from gs_base
0xC0000101
its 0x0
gdb tells me
molten grotto
is it also zero if you read it back directly after you write it there
surreal path
is it also zero if you read it back directly after you write it there
let me check
no its not
where is
wtf
where is gs base
bro wtf
molten grotto
you added the code into a test
surreal path
you added the code into a test
yea something funky is going on @molten grotto
i think i know whats going on
still crashes
im unsure whats going on
gdb didnt give me much
WHY IS SMP SO DIFFICULT TO DEBUG
AAA
molten grotto
just do it on a single core to start with?
surreal path
okay
surreal path
just do it on a single core to start with?
yea so
on one core
it doesnt crash but rather
doesnt execute the thread
like at all
wtf
i give up debugging this
surreal path
hi im back from school
will work on scheduler later
im tired atm i have homework
dev fs created!
from rip 0xffffffff80000be4
switching context to 0xffffffff80000be4
returning it at rip 0xffffffff80000be4
from rip 0xffffffff80000be4
yea thats not right
0xffffffff80000be4 um
is the lol function at that address?
lets check with object dump
rayan@pop-os:~/NyauxKT$ objdump -d kernel/kernel --demangle | grep ffffffff80000be4
ffffffff80000be4: eb fd jmp ffffffff80000be3 <NyauxKT::hcf+0x3>
rayan@pop-os:~/NyauxKT$ objdump -d kernel/kernel --demangle | grep ffffffff80000be4
what????
bro what
yea something is very wrong
either the compiler hates me or my code is SO bad that
it makes rust want to kill itself
i say the second option
nyauxmaster try not to make shitty code
im dying chat
chat i might just rewrite the entire scheduler
good plan?
what do yall think
i think the scheduler is just too shitty
surreal path
i have changed the design of the structs
#[repr(C, packed(8))]
pub struct perthreadcpuinfo {
kernel_stack_ptr: *mut u8,
user_stack_ptr: *mut u8,
current_thread: Box<Thread>
}
pub struct process {
vfs: Option<*mut vfs>,
pagemap: Option<*mut PageMap>,
cwd: Option<Arc<Mutex<dyn vnode>>>,
pid: u64,
handles: HashMap<usize, HandleType>,
name: String,
threads: Option<Box<Thread>>
}
struct Thread {
name: String,
tid: u64,
gs_base: Box<perthreadcpuinfo>,
fs: u64,
context: cpu_context,
process: Arc<Mutex<process>>,
next: Option<Box<Thread>>
}
struct cpu_queue {
next: Option<Box<cpu_queue>>,
lapic_id: u32,
process: Option<Box<process>>
}
pub struct CpuSQueue {
queue: Option<Box<cpu_queue>>
}
using less raw pointers
and less arc mutex bullshit
and not using vectors
and using a linked list instead
tell me if theres any issues with it
is there any issues with it?
i really wanna make sure i get this right before proceeding
molten grotto
i have changed the design of the structs
```rs
#[repr(C, packed(8))]
pub struct...
the real question is why does the scheduler have to deal with processes? the scheduler queue should just be a list of threads
and each cpu can have its own queue of threads
surreal path
the real question is why does the scheduler have to deal with processes? the sch...
buttttt we need to store the pagemap PER process
molten grotto
yes but other than that
molten grotto
like don't modify the thread list inside the process struct in the scheduler
molten grotto
like at least in the old scheduler code you got the next thread to run from the thread list inside the process
surreal path
like at least in the old scheduler code you got the next thread to run from the ...
yes?
so what abt that
im so confused lol
???
molten grotto
that its not a good idea
molten grotto
the scheduler queue would just be a list of threads and then it accesses the process using the thread's field when it needs to swap the pagemap
surreal path
the scheduler queue would just be a list of threads and then it accesses the pro...
i see
so like the thread would point to the process struct, the queue is just a list of threads
makes sense
molten grotto
yeah
surreal path
#[repr(C, packed(8))]
pub struct perthreadcpuinfo {
kernel_stack_ptr: *mut u8,
user_stack_ptr: *mut u8,
current_thread: Box<Thread>
}
pub struct process {
vfs: Option<*mut vfs>,
pagemap: Option<*mut PageMap>,
cwd: Option<Arc<Mutex<dyn vnode>>>,
pid: u64,
handles: HashMap<usize, HandleType>,
name: String,
}
struct Thread {
name: String,
tid: u64,
gs_base: Box<perthreadcpuinfo>,
fs: u64,
context: cpu_context,
process: Arc<Mutex<process>>,
next: Option<Box<Thread>>
}
struct cpu_queue {
lapic_id: u32,
thread_list: Option<Box<Thread>>
}
pub struct CpuSQueue {
queue: Vec<cpu_queue>
}
impl CpuSQueue {
pub fn create_queue(&mut self, lapic_id: u32) {
let queu = cpu_queue { lapic_id: lapic_id , thread_list: None};
serial_println!("created queue with lapic id {}", queu.lapic_id);
self.queue.push(queu);
}
}
improved version
the thread is a linked list
or actually should it be a vec
would that be easier or slower
surreal path
okay
so
the lapic timer
is firing when it shouldnt
i cli'd the boot cpu
and the other cpus
idk whats fucking going on
does anyone kno
w
edgy pilot
speedruning osdev was a bad idea from the beginning
surreal path
speedruning osdev was a bad idea from the beginning
im not speedrunning
im taking my time
edgy pilot
this is speedrunning
you're just trying to implement things without fully understanding them or the language you are using first
surreal path
this is speedrunning
you're just trying to implement things without fully unders...
i understand what im implmenting
maybe i could learn about the language more
but
i understand what im doing
edgy pilot
idk whats fucking going on
^
surreal path
^
it shouldnt happen
the cli instruction disabled ALL external interrupts
i understand what im doing
im gonna have to do the hacky solution of masking the lapic timer manually
by writing to the register
elder shoal
Or find out why the interrupts arent being actually disabled. Hint: flags register is what has the interrupt flag
surreal path
Or find out why the interrupts arent being actually disabled. Hint: flags regist...
okay then
surreal path
Or find out why the interrupts arent being actually disabled. Hint: flags regist...
WHY IS IT STILL ENABLED
WTF
WHY ISNT THIS INSTRUCTION EXECUTING
@elder shoal i think the compiler is optimizing out the asm function
thorn bramble
well gg
surreal path
oh my fucking god
thorn bramble
you should only sti if the interrupts were originally disabled
kind root
dont forget ur print does cli; print; sti
Lmao
kind root
A scheduler queue shouldn't be aware of arch implementation details like lapic id
surreal path
A scheduler queue shouldn't be aware of arch implementation details like lapic i...
i mean yea but
i plan on supporting just x86
kind root
Still incorrect level of abstraction
surreal path
check_exception old: 0xffffffff new 0xe
160: v=0e e=0002 i=0 cpl=0 IP=0028:ffffffff8000ba93 pc=ffffffff8000ba93 SP=0030:ffff80007a24ddd0 CR2=ffff80049077b358
RAX=ffffffff80068068 RBX=0000000000000000 RCX=0000000000000011 RDX=ffff80049077b330
RSI=000000000000000d RDI=ffff80007a24de68 RBP=0000000000000000 RSP=ffff80007a24ddd0
R8 =0000000000000000 R9 =0000000000000002 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
yo wtf
the rip is just garbage
ummm
CR2=ffff80049077b358
finite summit
nyuax might just be more cursed than obos
surreal path
yea i dont know whats fucking going on
thorn bramble
i uncovered at least 10 memory corruption bugs by writing an xhci driver
surreal path
time to try and get a backtrace
thorn bramble
from missing ists through (accidentally) shared kernel stacks
thorn bramble
told ya
surreal path
told ya
it was smp
.rs
new issue tho
my ENTRY FUNCTION ISNT EXECUTING
what
0xffff80007febb438
0xffffffff80011e70
wtf
surreal path
we got threads but
its only from cpu 0???
huh??????
wtf
i have 10 cpus
and ive inited the lapic for all of them
its coming from only one cpu huh???
wtf
i did info lapic on the other cpus
and the all work fine so wtffff
surreal path
dying chat
nvm forgot to switch page maps
okay deadlocking issues now
!!!
lovely
yea this isnt going well
helppp
i hate smp
i hate smp
cinder plinth
you know you don't HAVE to do smp
surreal path
you know you don't HAVE to do smp
i gotta have it
surreal path
you know you don't HAVE to do smp
also btw i think its because i have a mutex around my thread and process
pub struct Thread {
name: String,
tid: u64,
gs_base: u64,
fs: u64,
context: cpu_context,
process: Arc<Mutex<process>>,
}
pub struct cpu_queue {
lapic_id: u32,
thread_list: Vec<Arc<Mutex<Thread>>>,
cur_idx: Option<usize>
}
i think this is causing a deadlock issue
so maybe i should remove the mutex all together in favour for a refcell<T>
what do yall think
either that or i should put a mutex on the cpu queues
i have no idea why
it deadlocks
i thought rust prevents these things lmao
surreal path
yea i do not have a mutex on the global queue
molten grotto
i thought rust prevents these things lmao
there is no way it can prevent eg. you locking a mutex in normal code and then again in an interrupt handler that happens to occur when the mutex is locked in the normal code with interrupts enabled
flat nymph
Wrap it in Mutex<>
surreal path
Wrap it in Mutex<>
will do
surreal path
there is no way it can prevent eg. you locking a mutex in normal code and then a...
fair
molten grotto
its eg. why you should disable interrupts when holding a spinlock that can be taken from an interrupt handler too
flat nymph
If you drop the references as you're supposed to, you shouldn't have deadlocks
surreal path
its eg. why you should disable interrupts when holding a spinlock that can be ta...
fair
ill just disable interrupts on entry to my scheduler interrupt
and reenable them on exit
molten grotto
they are already disabled if its within an interrupt
so no point in doing that manually
surreal path
oh i see
flat nymph
or have a nonpreemptive microkernel and don't worry about interrupts
flat nymph
I have a counter in my spinlocks
Which counts to like 100 million
And prints a stack trace when that is reached
To diagnose deadlocks
surreal path
yea im having a lot of weird issues now
surreal path
they are already disabled if its within an interrupt
yea its still only cpu id 0
idk whats fucking going on
lol
but i mean
theres only one thread
maybe if i spawn another thread or smthin
its still only cpu id 0
...
bro
elder shoal
Are you sure theyre even getting scheduled on the other cpu and put on the other cpu's queue?
surreal path
Are you sure theyre even getting scheduled on the other cpu and put on the other...
wdym
elder shoal
You have per cpu queues, you could start by checking if the thread is actually put there lol
surreal path
i just have it so on schedule task a cpu picks a random queue to execute its thread
elder shoal
A cpu looks in another cpu's queue to execute it?
thorn bramble
WHA
surreal path
is that bad 😭 ??
thorn bramble
did it not cross your mind
elder shoal
That beats the whole point of having per cpu queues
thorn bramble
it's called a per cpu queue for a reason?
surreal path
right
thorn bramble
yea right man
surreal path
yea right man
dont be mean i just got confused 😭
thorn bramble
man don't get confused like THAT
thorn bramble
well just don't 😭
surreal path
😭
surreal path
What the fuck
😭 😭
okay it works now!!!!!
we have scheduler chat
!!!!!
i need a rand function now
edgy pilot
based
surreal path
to randomly pick a cpu queue to assign a thread to
kind root
edgy pilot
i need a rand function now
int rand()
{
return 4;
}
lmao
edgy pilot
it's random
surreal path
very random
kind root
i need a rand function now
Rdrand
edgy pilot
does qemu support rdrand?
kind root
Ofc
surreal path
uhhh idk
kind root
Qemu supports everything with the right flags
surreal path
theres a good crate that implments this functionality
i can use this and have it work with the rand crate
giving me random numbers
edgy pilot
there's truly a crate for everything
surreal path
yea i get u
edgy pilot
second good thing about rust
thorn bramble
there is a reason there is a crate registry...
kind root
theres a good crate that implments this functionality
U know its one x86 instruction right
surreal path
lmao
surreal path
arent u a rustie
kind root
Urdrand incoming
thorn bramble
yeah but im not LIKE that
surreal path
okay fine ill implment my own rdrand
thorn bramble
bro its one (1) instruction
surreal path
i know
thorn bramble
3 lines of rust code
edgy pilot
you can even fit it on one single line
thorn bramble
no im talking 3 lines of normal rust code
one line for the out variable, one line for the core::arch::asm! and one line for the return
edgy pilot
who needs \n
surreal path
why rdseed
whats the difference
surreal path
im using kvm
surreal path
okay it works now
thorn bramble
cool
surreal path
rand seed 653978260621011362
pro
just gotta stick this into rand and
we got random numbers
😎
thorn bramble
yea
nice
is this actually random or just happens to be a random number
reboot and try
thorn bramble
nice
surreal path
LETS GO IT WORKS NOW
😎
okay scheduler is done
whats next chat
userland time?
ITS USERLAND TIME
WEEEEEEEE
ITS USERLAND TIME
shit i do not have mlibc
uhhhhh
what do i do hmmm
also what happens when a thread returns
what will the scheduler do
lol
rigid fable
```c
int rand()
{
return 4;
}
```
lmao
surreal path
oh right thats a syscall
exit()
to which i do not have syscalls
i should probs set that up
i do not know what to start with
elf loader or get build system working or what
or syscalls or what
xsave? fpu setup?
idk
rigid fable
I was just going to say "hah this looks like the Cell architecture (the PPE coordinates the SPEs) and then ilobilo literally put the ECDSA breach meme for the PS3 here
rigid fable
i just have it so on schedule task a cpu picks a random queue to execute its thr...
about this
thorn bramble
i think that's a reference to xkcd/221
chosen by fair dice roll, guaranteed to be random
surreal path
idk
yall know what i should do first?
syscall, setup fpu, elf loader, or build system?
thorn bramble
I was just going to say "hah this looks like the Cell architecture (the PPE coor...
was that an actual thing tho?
if so funny thanks for telling me
rigid fable
I had this on my phone for some reason
let's gooo
my discord bugged out
I was seeing multiple images of this in the chat
molten grotto
those kinds of things are nice tbh (not for sony tho, fuck sony)
rigid fable
those kinds of things are nice tbh (not for sony tho, fuck sony)
well now we can install Linux on PS3 again
and that because Sony removed that feature in the first place
which lead to the previous problem
molten grotto
also reminded me that I should try ps4linux again now that there is a jb for 11.00
or ps4managarm 
would be funny
surreal path
or ps4managarm <:trolol:836730473061154832> would be funny
ps4nyaux when qwinci
rigid fable
molten grotto
I mean why not, though idk if you'd have to use the terrible linux boot protocol
rigid fable
fyi the PS4 is an 8 core machine sooo
molten grotto
the way you execute linux on it is with a kexec like thing anyway so eh
though ig you could still make some kind of modified bootloader that would be booted that way
surreal path
elder shoal
or ps4managarm <:trolol:836730473061154832> would be funny
Assertion failed: '!"wtf are you doing"'
kind root
though ig you could still make some kind of modified bootloader that would be bo...
Same idea as uefi lol
rigid fable
though ig you could still make some kind of modified bootloader that would be bo...
now with custom firmware patches you can automatically boot in otherOS on the PS3
rigid fable
Assertion failed: '!"wtf are you doing"'
Astral on ps4 when?
surreal path
Assertion failed: '!"wtf are you doing"'
assertation os
elder shoal
Astral on ps4 when?
When astral itself works good enough
rigid fable
When astral itself works good enough <:meme:575445094589661215>
don't worry. The PS4 OS itself does not work properly anyways
surreal path
When astral itself works good enough <:meme:575445094589661215>
is nyaux better then astral
elder shoal
don't worry. The PS4 OS itself does not work properly anyways
Then astral should be good enough
elder shoal
is nyaux better then astral <:memedown:1267962938132533248>
You have per cpu queues so instantly better than astral
finite summit
kernel toes
thorn bramble
cwd: unsafe {(*vf).vnode.clone()},
dear lord
why is this man passing around raw pointers everywehre
surreal path
i have a problem
rigid fable
why is this man passing around raw pointers everywehre
because rust is **
thorn bramble
also Option<*mut ?> lets me pass a Some(core::ptr::null_mut()) lol
rigid fable
thorn bramble
use Option<NonNull<?>> instead
elder shoal
I like my pointers medium rare
surreal path
^
molten grotto
shit i do not have mlibc
use hzlibc and implement the ~200 sysdep fns
thorn bramble
use hzlibc <:trolol:836730473061154832> and implement the ~200 sysdep fns
dont give him ideas
you'll be stuck here answering questions for the next 20h
molten grotto
and see how much unsafe i really need
yeah I don't think you actually need like a ton (at least not as much as nyauxkt with its c like code)
molten grotto
and linked lists + other things like that probably
molten grotto
lol
thorn bramble
im definitely not writing my own pairing heap
yay
insertion and popping is O(log(n))
i dont think thats bad, but idk data structures so
tawdry mirage
insertion and popping is O(log(n))
well with a pairing heap popping from the top is O(1) afaik
surreal path
well with a pairing heap popping from the top is O(1) afaik
qookie hiii
tawdry mirage
hi
thorn bramble
ok so i probably want to find something better than that
unkempt relic
pairing heap isn't so hard to implement
surreal path
x86-64 crate
unkempt relic
what's it to be used for? a timer queue?
surreal path
tawdry mirage
well with a pairing heap popping from the top is O(1) afaik
i suppose it's probably not that bad if you have some global value that stores the nearest timestamp for something in the heap that you check before checking the heap
thorn bramble
i found this
Operation Time Complexity
append O(1)
peek O(1)
pop O(log n) (amortized)
push O(1)
push_pop O(log n) (amortized)
replace O(log n) (amortized)
this looks much better
molten grotto
i mean hes not wrong
I am in that you shouldn't use my libc because I don't even know whether things like xorg work on it without potentially crappy sysdep impls, that's on my todo list (and to actually write the posix compat syscalls in my kernel), and I know that cmake doesn't properly work when using it (as in when using the libc on the build system)
thorn bramble
cats > dogs
molten grotto
^
surreal path
and if you disagree i will terminate you instead
surreal path
I am in that you shouldn't use my libc because I don't even know whether things ...
lol
unkempt relic
yeaaah but this is rust
are you making a kernel all your own or refactoring nyaux kernel?
thorn bramble
i wouldnt dare touch that code
one wrong move and it all falls apart
i dont think nyaux even has timers lol
surreal path
i have the hpet
thorn bramble
that is not a timer in the sense we are talking about
surreal path
what timer are u talking abt then
unkempt relic
lapic timer continously firing interrupts
what, like a ticker?
thorn bramble
yeah iirc nyaux just sets up the lapic timer to fire continously
every 10ms or whatever
thorn bramble
and that drives the scheduler
molten grotto
what timer are u talking abt then
eg. having a thread sleep for 2s or having a function run after x ns
surreal path
yea
surreal path
eg. having a thread sleep for 2s or having a function run after x ns
i have the hpet for that
thorn bramble
thats how everyone starts
unkempt relic
it's not unreasonable, it's how most kernels were until fairly recently
surreal path
if i need to use it
thorn bramble
i have the hpet for that
have something happen after eg. 2s without having the cpu spin
thorn bramble
while hpet.read() < deadline { wait(); } is what you're trying to avoid
tawdry mirage
solution: base your whole kernel around coroutines
finite summit
solution: make event objects, profit
molten grotto
what should i use then
you eg. keep a list of timer objects and then in your timer irq you check the list for the ones that expired and execute their actions
finite summit
that's what I do
surreal path
you eg. keep a list of timer objects and then in your timer irq you check the li...
oh easy i have the lapic timer
easy
😎
and i can reuse my handle system for this
finite summit
I look for expired timers and call their handlers
surreal path
thats already in the process struct
thorn bramble
im still unsure how that interacts with the scheduler, i dont have a mental map of that just yet
finite summit
im still unsure how that interacts with the scheduler, i dont have a mental map ...
you could have it wait on an event
to be set
as in the thread
blocked on the timer object
finite summit
basically:
timer_irq userdata:
set_event(userdata);```we still far off lol
i still have a bunch of shit to do
like
an elf loader
or
finite summit
then you:
.... setup timer object ....
wait_on_object(event);```getting ze fpu working
finite summit
in the thread
elder shoal
yeah iirc nyaux just sets up the lapic timer to fire continously
Do tickless smh
finite summit
thorn bramble
i forgot what the difference between tickless vs tickful even is lol
surreal path
solution: dont worry about this until its needed later
thorn bramble
someone explained it to me but im dumb
surreal path
lmao
thorn bramble
no ive been there already
i need better now
all my efforts end with my code being shit and me wanting it to be better
tawdry mirage
i forgot what the difference between tickless vs tickful even is lol
with tickless if you have an event that should happen in 10ms, you set the timer to fire in 10ms (assuming there are no events that should happen earlier)
thorn bramble
so i want to write good stuff once and be done with it
elder shoal
with tickless if you have an event that should happen in 10ms, you set the timer...
This
tawdry mirage
without the timer ticking regularly
thorn bramble
with tickless if you have an event that should happen in 10ms, you set the timer...
ah so basically have the scheduler be driven by the "timer subsystem"
elder shoal
I do tickless in astral even if it has some rough edges™️
surreal path
i will do timer objects later
im not having nyaux be tickless cause im lazy
lol
ill just have working timer object,
finite summit
i will do timer objects later
it's like < 300 lines
surreal path
struct Thread {
handles: HashMap<usize, HandleType>
}
enum HandleType {
...
TimerEvent(struct TimerEventObject)
...
}
struct TimerEventObject {
handler: Box<dyn timereventhandler>,
time_to_expire: usize
}
tickless for scheduling is nice since the scheduler just says "fire this event in N ms" where N is the current thread's quantum
thorn bramble
what if there isnt another thread to run
tawdry mirage
assuming no other timed events are pending, no timer ticks will happen
you're waiting for some other device in the system to raise an irq
tawdry mirage
e.g. the hard drive or usb controller
thorn bramble
what if my userspace program is a 1: jmp 1b
surreal path
get build system workin
thorn bramble
and no other threads are running
surreal path
so i actually have elfs to run
thorn bramble
would the cpu just spin until an irq happens?
that brings it to the isr, it does its thing, and returns back to the scheduler
which schedules my jmp 1b again?
surreal path
anyways im gonna go distro hop rq brb
thorn bramble
since that isnt blocked on anything or whatever
tawdry mirage
you could probably avoid entering the scheduler if there is only one running thread
and any scheduler timer ticks
thorn bramble
yeah i was kind of tryihng to get a mental map of how it all works together
so when i go and implement it i dont end up being confused
edgy pilot
i think that's a reference to xkcd/221
yep that was it lol
tawdry mirage
you could probably avoid entering the scheduler if there is only one running thr...
you could also not treat the single thread case in a special way and just schedule the timer to fire in N ms anyway
and just find out no other threads are runnable
unkempt relic
tickless for scheduling is nice since the scheduler just says "fire this event i...
i think with modern timers that are 64-bits wide or near-enough (lapic with tsc deadline, stime* on riscv, aarch64 generic timers) it's even the easiest thing to do to go tickless, or dynamic tick
thorn bramble
i think if no other threads are running i shouldnt queue the timer
unkempt relic
and for older timers without that probably worth emulating
thorn bramble
i think if no other threads are running i shouldnt queue the timer
shouldnt be that much additional code
tawdry mirage
i think with modern timers that are 64-bits wide or near-enough (lapic with tsc ...
yeah with timers that only do deadline it's extra effort to have periodic ticks instead of tickless
and it seems that that's a trend with modern timers
thorn bramble
literally if (new_thread is null) { return; }
else save current thread context, load new thread context, arm timer and return
also put old thread back on run queue probably
surreal path
i think with modern timers that are 64-bits wide or near-enough (lapic with tsc ...
fadanoid whats ur opinion on progress wit nyaux kernel
unkempt relic
fadanoid whats ur opinion on progress wit nyaux kernel
i am pleased with the progress and look forward to seeing it go further
i see no reason why you shouldn't have bash running in due course
thorn bramble
also put old thread back on run queue probably
i dont think i will be rescheduling a thread that is blocked, that would go through a different route probably called yield, which saves the thread context, picks a new thread (or the idle thread) and switches to that
then when whatever the thread was blocked on is signaled i can reschedule again to that thread?
idk
i shouldnt talk about it there ig
unkempt relic
then when whatever the thread was blocked on is signaled i can reschedule again ...
one thing that helps with latency, giving a thread newly made runnable by (for example) it's reading from the keyboard device and there's now data available to read, give the thread a little help to run soon
so e.g. preempt a currently running thread on some core to get it immediately going, if its priority agrees
thorn bramble
i have no idea how thread priorities work (yet) but i dont think that would be too hard
i guess every thread has a baseline priority it starts with
then when a thread is woken up, because of io completion for example, it gets a priority boost?
and that allows it to preempt a thread on another core
or the core it has affinity for
unkempt relic
then when a thread is woken up, because of io completion for example, it gets a ...
it can, there are a few schools of thought
thorn bramble
i guess i should try the core it has affinity for first, then other cores
in my book, the core it has affinity for is the core it was last run on, so idk if that its right, just clarifying
unkempt relic
one school is that the scheduler itself should include heuristics to identify what threads are I/O bound v.s. compute bound, usually by considering how much of its timeslice the thread uses up v.s. how much remains (lots used = compute bound little used = I/O bound)
and then the scheduler dynamically adjusts the priority based on that
the other school is explicitly temporarily boosting the priority of a thread woken up by I/O completion
unkempt relic
in my book, the core it has affinity for is the core it was last run on, so idk ...
that's the best way to go, migrating between cores is expensive to varying degrees
thorn bramble
i think i could do both, or start with the 2nd and move to 1st at some point in time when im confident that my scheduler needs to be improved
unkempt relic
with SMT you can fairly move a thread between two hyperthreads, hyperthreads share a lot; on the other extreme, moving a thread from one NUMA node to another is a hateful business
thorn bramble
its about caches, tlb and all that i assume?
unkempt relic
its about caches, tlb and all that i assume?
yeah, and numa then adds on top inequality in latency of memory access to other ndoes
molten grotto
What I do is that I lower the priority of a thread if it uses its full timeslice and if it blocks then I raise its priority (and the lower the priority the bigger the timeslice is)
and it has worked mostly fine™️ so far
thorn bramble
you mean the higher the priority?
if a thread uses it's timeslice fully i dont think it should get even more cpu time
unkempt relic
for low priority paradoxically it is often better to have a larger timeslice
thorn bramble
really? damn
unkempt relic
as the low priority threads are hopefully compute-bound, and the high priority are I/O-bound, and when a high priority thread becomes runnable it should preempt the low priority one
so while the i/o-bound threads are not ready, let the compute-bound ones grill
thorn bramble
ah right, preemption exists
is preemption literally just "get out of here, its my time to run"
unkempt relic
is preemption literally just "get out of here, its my time to run"
yes, if someone higher priority turns up, then the lower priority one typically gets punted
molten grotto
you mean the higher the priority?
the scheduler always takes the next thread to run from the high priority queue (with the smallest timeslice) and only if there are no runnable threads in there it considers the lower priority ones with bigger slices
thorn bramble
so would low priority threads never run if there were more than cores available to scheduler threads on?
but i assume scheduler would bring their priority down if that happened
to let low priority threads run
assuming they turned from io-bound to compute-bound ones
otherwise they would do io and let lower priority threads run again
that makes sense
molten grotto
so would low priority threads never run if there were more than cores available ...
low priority threads don't run until there are no more runnable high priority threads (which is guaranteed to happen as they either block or get turned into low priority ones themself)
molten grotto
ah yeah
thorn bramble
gives me confidence in actually being able to implement it properly
i think im going to do temporary priority boosts for now because i dont feel like trying to figure out how to do automatic priority stuff just yet, mainly because im not sure i can do it properly lol
surreal path
im just playing minecraft, ill work on the elf loader later
finite summit
minecraft: the procrastiation tool
flat nymph
also reminded me that I should try ps4linux again now that there is a jb for 11....
I have not updated my PS4 in a long time
flat nymph
fyi the PS4 is an 8 core machine sooo
Wasn't it 4 core bulldozer?
rigid fable
Wasn't it 4 core bulldozer?
I checked rn and it says 8 cores, with 2 4 core modules
flat nymph
Also, PS4 isn't a PC (?)
finite summit
I need to get obos running on a ps4
flat nymph
I've heard it doesn't have BIOS/UEFI, south bridge and a lot of stuff like that
So it's x86, but not a PC
thorn bramble
the most reputable source of information
quora
i mean even if they arent half wrong
its still not a deal breaker
you can write drivers around this whole thing no problem
didnt people get linux to run on the ps4?
rigid fable
didnt people get linux to run on the ps4?
yes
thorn bramble
whats stopping you from getting your os to run on the ps4 too?
well, not you
but anyone in general
rigid fable
PS3 might be simpler
not at all
thorn bramble
even if its not a standard pc its not any harder
rigid fable
even if its not a standard pc its not any harder
security features
finite summit
even if its not a standard pc its not any harder
Fair
@kind root when hyper for ps4