#OBOS (not vibecoded)

1 messages · Page 11 of 1

inland radish
#

Generally there shouldn't

#

But it's fine if something that owns a spin lock is interrupted by something at higher IRQL

ashen goblet
#

Well yeah but you should only lock the same lock from within the same irql right

inland radish
#

Correct

#

Although there's an exception which is that it's acceptable to acquire spinlocks from <DISPATCH_LEVEL because the spinlock will raise/lower the IRQL to DISPATCH_LEVEL if needed

flint idol
#

I hate this

#

The stack is corrupted

#

I cannot figure out why

main girder
flint idol
#

it's all over the place

#

one boot it faults there because something returned to some random kernel address

#

another, it's at some random address in the kernel heap

main girder
#

probably a result of you not testing something well enough

#

could have been avoided.

flint idol
#

I currently suspect that the problem is somehow in the thread context switch code

#

maybe the rsp was set incorrectly somehow

# 
lea rax, [rsp+0x10] ; skip the return address
mov [rdi+thread_ctx.frame+0xC8], rax ; rsp```
#

The ret address + some value were pushed at the beginning of the function

#

which means rsp before the call would be the current rsp+0x10

#

right

#

for a second I thought I'd found the bug

#

but I was wrong

#

time to look at the switch to thread context function

#

that looks clear

#

found a subtle bug in the timer dispatcher

#

two variables were named oldIrql

#

amazing

#

it still crashes

flint idol
#

I think the bug has to be with my yielding function

#

the only difference between when these tests run and when they don't is that there are multiple threads yielding often

#

ok now wtf is going on

# 
Kernel Panic on CPU 2 in thread 6, owned by process -1. Reason: OBOS_PANIC_EXCEPTION. Information on the crash is below:
Page fault at 0x0000000000000000 in user-mode while to read page at 0x0000000000000000, which is unpresent. Error code: 0
Register dump:
        RDI: 0x0000000000000000, RSI: 0x0000000000000000, RBP: 0x0000000000000000
        RSP: 0x0000000000000000, RBX: 0x0000000000000000, RDX: 0x0000000000000000
        RCX: 0x0000000000000000, RAX: 0x0000000000000000, RIP: 0x0000000000000000
         R8: 0x0000000000000000,  R9: 0x0000000000000000, R10: 0x0000000000000000
        R11: 0x0000000000000000, R12: 0x0000000000000000, R13: 0x0000000000000000
        R14: 0x0000000000000000, R15: 0x0000000000000000, RFL: 0x0000000000000000
         SS: 0x0000000000000000,  DS: 0x0000000000000000,  CS: 0x0000000000000000
        CR0: 0x0000000080010011, CR2: 0x0000000000000000, CR3: 0xffffff000008ce88
        CR4: 0xffffffff800b92ea, CR8: 0x0000000000000001, EFER: 0xffffffff8009fea7```
#

see anything wrong with this crash log

sharp pike
#

nice EFER

flint idol
#

ikr

#

cr3 is beautiful as well

#

well that crash was a result of a thread's context not being initialized

#

was easy to fix

#

needed to change a jz to a jnz

#

I shall continue debugging tomorrow

#

studies have shown that I don't enjoy debugging this shit

lean glen
#

What bug are you having now

inland radish
#

@flint idol I notice something

#

what this for

#

oh nvm

#

how dis work

#

~~ that digs into the stack~~ nvm

#

kinda weird that youre hardcoding the stack size in asm

#

also @flint idol do you ever call this recursively/reentrantly

#

like does a function called by CoreS_CallFunctionOnStack call that function CoreS_CallFunctionOnStack

#

anyways the scheduler code is kind of a mess

main girder
#

and then look at mine if you want to know how to do it better than that

inland radish
#

Especially now that I don't use self IPIs anymore

#

Ok maybe the next thread picker and stuff

#

But I was talking about the switching logic itself

main girder
#

fine grained locking!

inland radish
#

in that regard its not much better as it's basically the same

#

boron does lack fine grained locking but I'm totally fine with that

flint idol
inland radish
flint idol
#

Now that I'm at my computer I'll search for it

#

Nope, it's impossible for it to get called recursively

#

As the only other reference of it is under the exit current thread function

#

But it masks all IRQs before calling that function

flint idol
flint idol
#

This seems to be linked to SMP

#

As without SMP nothing crashes

#

something very weird has happened

#

basically has something to do with 0x8000 bytes of stack being used in the idle thread

flint idol
#

my brain's starting to hurt

#

what would happen if I theoretically rewrote ||the scheduler||

flint idol
#

very funny

#

Nevermind, I don't

flint idol
#

After commenting out the TLB shootdown code

#

nothing crashes

#

now time to figure out why

#

I'll first check if it's a bug with the isr return code somehow

#

If it is, then it should always crash on the first IPI shootdown

flint idol
#

nope

#

that's not the case

#

well don't ask why, but the IRQ for TLB shootdowns is an NMI

#

idk if that'll be a problem

flint idol
flint idol
#

I have found the problem

#

I think

#

The NMI handler lives off the cpu temp stack

#

therefore on return there's a chance that it goes in the scheduler

#

which uses the cpu temp stack

#

Solution: Throw the NMI handler on another stack

#

OH AND OF COURSE WHEN I TURN ON THE TIMER INTERFACE IT BREAKS AGAIN

#

AHHHHHHHHHFDVHFXDSVGHXDRFHVGXDRFSGHVS'

# 
Kernel Panic on CPU 3 in thread 6, owned by process 1. Reason: OBOS_PANIC_FATAL_ERROR. Information on the crash is below:
Core_LowerIrqlNoThread: IRQL 2 is greater than the current IRQL.```
#

I think I found the bug

#

yup seems like it

#

I'd forgotten to lower the irql at a return statement

flint idol
#

and was somehow right

#

ok so what was I doing

#

ah yes page fault handler + page replacement algo

#

Just pushed the fixes

flint idol
#

I just pushed some PF handling code

flint idol
#

what do you guys think of my explanation of the page replacement algorithm I'm going to implement

// Basically how this works is:
// The page replacement algorithm goes through each page in the working set and the referenced list.
// For each page in there, it runs the "aging," PR algorithm.
// This algorithm basically states that:
// 'u' is a string of bits, in our case a string of 8 bits.
// Each time a page reference occurs 1 -> u0.
// At the end of each sampling interval or, the bit pattern contained in u0,u1, ... ,
// uk is shifted one position, a 0 enters u0, and uk is discarded
// The sampling interval in our case, is the time between two calls of this function.
// Any page in the working-set with an "age" of 0 is removed, and is replaced with one in the referenced list.
// The working set shall never exceed it's set size in the context structure.```
#

should be pretty good

#

if anyone spots any mistakes in my understanding/wording, please let me know

#

anyway, I now go to sleep

flint idol
#

as in some implementation details that might not exist in other kernels

#

idk though

#

tbh I never looked at other kernels

#

I mean I think that algorithm is good

#

but I should go to sleep and not implement it rn

flint idol
#

I am now awake

#

and shall implement this*
*I started half an hour ago in my sleep

#

Just finished the implementation

#

Time to test it

flint idol
#

fuck this shit

#

the swap decides to fall down

flint idol
#

why\

#

tf

#

is kernel code getting corrupted

flint idol
#

but that's marked to not be swapped out

#

and if it's never swapped out it's never swapped in either

#

I'm going to go play mc

#

I'll be back in

#

time

flint idol
#

fuck

#

just died in my hardcore world

#

to a stupid

#

fucking

#

creeper

vale nymph
#

Smh shouldve carried a shield

flint idol
#

I had one

#

it came out of nowhere

flint idol
#

After almost 10 days

#

I'm back

#

time to go back to debugging this bug

inland radish
#

basically the wait function in my kernel went (no longer goes) like this:

  • setup the wait
  • unlock dispatcher and lower IPL
  • yield
#

during the lowered IPL phase DPCs might come in which might actually wake up the current thread

#

since its "waiting" its placed "back" on the run queue, and since it just got a priority boost the current thread will yield, leading to a curious situation where the current thread will switch to itself

#

due to the nature of how I wrote the thread switch code, this causes a crash

#

the solution was to unlock the dispatcher lock but not lower IPL, and in fact never lower IPL until after the thread was completely switched from

flint idol
#

So I was dealing with kernel code corruption

flint idol
#

The only thing I can think is happening rn is a bug in the pmm

#

where it returns two identical physical addresses

#

doesn't look like it

ashen goblet
#

Another memory corruption? nooo

#

What is it this time?

flint idol
#

this time it's worse

#

a lot worse

#

it's of kernel code

ashen goblet
#

modules/something else you are loading into memory or your actual kernel binary loaded by hyper

flint idol
#

the actual kernel

#

specifically __asan_load1_noabort and functions in the same page

ashen goblet
#

Hmm good luck figuring that out

flint idol
#

actually, it seems to only be some functions

#

this reminds me of the reason I decided to play mc instead

#

the page looks like it was used as a stack

#

I see addresses from the kernel

#

the page is mapped as Read, Execute

#

I think I'll check if that phys address is used anywhere else

#

weirdly there are pages that are mapped to phys address 0 while being RO

#

(possible TLB shootdown bug?)

flint idol
#

wth

#

the stack pointer is set to the page of the function

#

that's corrupted

#

this happens in the kernel main thread

flint idol
#

It's a stack overflow

#

which happens in the pf handler

#

because one specific function wasn't marked as non-pageable

#

(starts to regret making functions opt-out of being able to get paged out instead of opt-in to getting paged out)

#

idk why the processor was allowed to write the page that was marked as RO

#

I'll check cr0

#

the supervisor protection bit thing

#

well would ya look at that

#

it's off

#

chances are it's off on each cpu

flint idol
#

I just made being paged-out opt-in instead of opt-out

#

and everything boots

flint idol
#

I just pushed the page replacement algorithm and this fix

#

All that should be left are allocation functions for the vmm

#

It should not have taken two months for the VMM

#

considering how simple it is

#

but you know the bugs and stuff take a lot of time

#

and there was a rewrite

#

of the vmm

#

so whatever

flint idol
#

this math isn't mathing

#

0x1000-0xffffff0000083000 is apparently 0xfffff7e000

#

and also apparently 0x1000-0xffffff00001000 is zero

flint idol
#

I don't think my RB Tree is RB treeing

#

If I call RB_MIN on an rb tree (freebsd's implementation), it should return the node with the smallest address

#

well it doesn't

# 
inline static int pg_cmp_pages(const page* left, const page* right)
{
    if (left->addr == right->addr)
        return 0;
    return (left->addr < right->addr) ? -1 : 1;
    // return (intptr_t)left->addr - (intptr_t)right->addr;
}```
#

nvm a rebuild fixed it

#

probably some weird dependency thingy

flint idol
#

All that's left for me rn is to add the vma's free function

#

then I can merge my two month old branch for the vmm

inland radish
#

OBOS return to greatness

#

and I'm all for it

flint idol
#

after this I'm making the driver interface (for some reason I like doing this)

#

I find it fun making driver interfaces, idk why

#

I get to port uACPI for the driver interface

#

and for the vmm tests

#

I get to practically rewrite relocation code

#

for loadable kernel modules

flint idol
#

and I don't wanna port that to C

inland radish
#

for Boron the entire kernel is available and can be linked against (except for static functions)

#

I don't know if I can, should, and want to change that

lean glen
#

Nah that's not a bad way of doing it

#

Isn't that what Linux does

flint idol
#

technically that's what obos rewrite 3 did

#

except if you wanted most symbols you needed to do some linker tricks*
*it was literally just marking the symbols as weak so that the dynamic linker had to resolve that symbol

lean glen
#

You could, however, probably restrict what modules can use

#

If you have aslr

#

Else you could do some paging memes probs

flint idol
inland radish
#

the downside is that you can call internal functions such as KiSetEvent etc which aren't defined as static

flint idol
#

instead of marking the symbol as weak for drivers

#

bruh today I told myself I'd sleep early

#

*earilier than the last week or two

#

that went wrong

#

osdev > sleep /j

#

maybe this mental health break from osdev made me better

#

note: my mental health wasn't really affected by osdev

#

but the break was nice

#

and ig without it I would still have lost my hard drive

#

I basically had a two week mc grind in the time I was gone

#

like I've pretty much maxed out my stuff

#

my world was on a server on which I played on with my friends

#

we had duos

#

and I ensued chaos after deciding I wanted to piss off a certain duo

#

I blew them up

#

didn't even deny it

#

then continued to plan with one of them to trap and kill the other

#

we ended up making the other guy rage quit

#

now it's just us two on the server

#

server had a max of 5 people on it

#

we had beaten the ender dragon by the 2nd (irl) day

#

I got an elytra before doing that

#

we had some mod that let you teleport randomly

#

and to places

#

I got fireworks and used my elytra to destroy the end crystals

#

but anyway that's enough off-topic stuff

#

when I wake up I'll implement the vma's free function

#

and tests

#

make the allocator free properly once again

#

then finally merge the branch

#

anyway gn all

ornate ginkgo
#

It's easy to do too

#

With elf you can use the symbol visibility, and you have control over the linker so you can error out if a module tries to link against something it's not supposed to.

lean glen
flint idol
#

I shall now make the vmm free function

#

(after going to the bathroom)

real pecan
flint idol
#

ok I am now back after a couple detours

#

The free function needs to:

  • Unmap the pages
  • Remove the pages from any VMM data structures (working set, page tree, referenced list)
  • Exit
#

and most importantly, not have some dumb off-by-one bug that causes a week or twos worth of debugging

#

it also needs to account for any guard pages

#

added during allocation

#

well the question is how

#

I could just add a flag to the page struct

flint idol
#

I just finished implementing the function

#

Time to test it

#

So it works

#

which is too good to be true

#

as the only fix I needed was changing RB_RIGHT to RB_NEXT

#

I'll test it to see if it can detect guard pages

#

then use my allocator test that runs the allocator multithreaded

#

while it seems the guard page was found

#

it doesn't seem to have the flag set

#

that's because there's a bug in the virtual allocate function

#

I forgot to make it offset the base when there's a guard page added.

#

wait why didn't it page fault then thonk

#

waiiiiiit

#

why are these pages already mapped

#

whoops

#

why aren't these pages in the rb tree

#

that's because they were never meant to be

#

I accidentally rounded up somewhere

#

where it was supposed to be rounded down

#

which caused the pages to appear

#

which isn't the case

real pecan
#

Damn this rewrite is going at a slower pace

#

Last time u speedran it very quick

flint idol
#

it's because of my two-week mc phase

#

in that time there was no developement

real pecan
#

I see

#

Did you enjoy it

flint idol
#

yes

real pecan
#

Nice

flint idol
#

now I need to figure out who's dreaming up this new page, and isn't registering it

real pecan
#

Gl

flint idol
#

found it

#

it was in the initialization of the timer interface

#

it used the basicmm instead of the vma

#

to allocate a stack

#

but that memory was never recognized

#

the solution is to rather use the vma or initialize the timer interface earlier

#

I think it's time to add a back trace to the panic

#

too much work

# 
        OBOS_Panic(OBOS_PANIC_FATAL_ERROR, "%s: Timer interface cannot be initialized before the VMM. Status: %d.\n", __func__, OBOS_STATUS_INVALID_INIT_PHASE);
#

I wonder why I put this

#

what could possibly go wrong

#

nothing blows up so I'm good

#

and it doesn't give me an already mapped page

#

still some bugs with guard page allocation/freeing

#

which I think I fixed

#

they're fixed

#

the vma free function is finished

#

now time for tests

#

time to search the git history for my multithreaded allocator tests

#

oh nvm it found a way to crash

#

I probably accidentally removed a line in preparation to commit

#

I forgot to check if the allocated pages had a guard page

#

before offsetting a page to the returned base

#

adding the check fixed that

#

so I'll now commit this code

#

that code is committed and pushed

#

lmao adding the test causes an immediate crash

#

the threads don't even get the chance to start

#

after rebooting, it double faults

#

damn it I got to stop making functions in the allocator pageable

#

I don't think the vma is ready for those bigger test yet, so I'll test having two allocated blocks from the vma at once

#

a smaller test

#

gives a KASAN violation

#

wait I forgot to remove pages from the vmm referenced list and working set on free

#

which is why it did a use after free

#

after fixing that, there is no use after free

#

bruh how is there stack corruption in a thread

#

before it even gets a chance to start

#

like it's right as it starts, before it even gets the chance to say it started

#

it's in KASAN

#

in __asan_storeN_noabort

#

but... how?

#

it doesn't make any writes to the stack but the stack canary (at least that's what it seems like)

#

probably some other thread sharing the same thread

#

which probably explains why sometimes it just double faults

#

gdb be acting kinda suspicious rn

#

I do print stack and it says unavaliable

#

or maybe I shouldn't be allocating the thread stacks pageable

#

bruh I added the flag VMA_FLAGS_KERNEL_STACK just for it to continue to do nothing

#

after fixing that, it page faults

#

but the thread at least gets the chance to say it started

#

oh no the crashes are random

#

The VMA returns a base address that isn't mapped

#

which causes a crash

#

now three things could be happening:

  • The pf handler is having a smooth brain moment
  • It just plain doesn't do anything and returns some random base
  • It doesn't register some page node, causing the pf handler to not recognize the page
#

well it isn't a pf handler smooth brain moment

#

at least that's not what it seems like

#

I don't lock the free virtual memory function

#

I forgot to

#

so I'll do that and see if that fixes anything

#

my friend sent me this

image.png
#

OBOS is now pronounced "OBOE+SSS"

hollow geyser
#

yeaaaa

flint idol
elder hornet
flint idol
#

good

flint idol
#

I don't get why the allocator would just not allocate

#

if only I had a way to reverse process execution

#

the page faults, no matter the faulting address, always happen at the first reference to blk in the allocator

#

(note: blk is allocated by the vma)

elder hornet
#

which allocator are you using?

flint idol
#

mine

elder hornet
#

yeah but like which algorithm

#

bitmap? buddy?

flint idol
#

those are pmm

#

allocators

elder hornet
#

ohhh you're doing a userspace allocator?

#

yeah okay sorry

flint idol
#

what no

elder hornet
#

sorry now I'm confused

flint idol
#

like malloc as in allocator?

elder hornet
#

my brain is a bit smooth xD

elder hornet
flint idol
#

Update: This seems to be a problem with multi-threading

#

Using 16 threads causes the kernel to crash eventually

#

however, using only 4 threads causes the tests to pass

#

actually nvm, it just crashed

#

it only seems to crash in two cases:

  • The thread count > cpu count (scheduler bug?)
  • on any reboots (unzeroed memory?)
#

actually no, I think my kernel just hates me

#

so it decides to crash occasionally

#

With 8 threads, the kernel sometimes crashes with an allocator failure

#

(well the test threads do)

#

OBOS_STATUS_IN_USE

#

which can only be coming from the VMA

# 
if (exists)
{
    if (flags & VMA_FLAGS_HINT)
    {
        base = 0;
        goto top;
    }
    else
    {
        set_statusp(ustatus, OBOS_STATUS_IN_USE);
        return nullptr;
    }
}```
#

when the pages requested are already in memory, it returns this error

#

except the base is (indirectly) found using MmH_FindAvaliableAddress

#

which can only mean that it returns an address already in-use

#

OBOS_ASSERT(!(oldIrql & ~0xf));

#

has failed

#

I think I fixed it

#

TIL the allocator is slow(er) multithreaded

#

if I were to make it lockless then it could be faster

#

but I can't

#

(mainly because skill issue)

#

4/8 threads have exited successfully

#

so far

#

maybe this would be faster if I don't just lock unconditionally at the beginning of Allocate()

#

I'm just going to dump where the allocator needs to have the lock held in here

#
  • When allocating a new region
#
  • While appending a node
#
  • While searching for a free node in a region
#
  • Possibly while looking for a region to use
#

maybe I could optimize the allocator by using an rb tree somehow for the regions

#

or some sort of tree

#

instead of a linked list

#

as the time to search a region for the allocator is O(n) worst case

#

and if that ends up failing because there is no region big enough

#

it just ends up allocating a new one

#

and this is a lot of time wasted

flint idol
#

so I was testing with 7 threads

#

then it deadlocked...

#

||while crashing||

flint idol
#

I can finally merge the vmm

#

I've resolved the conflicts (it was only in the readme)

flint idol
#

bruh now I have to write non-trivial code

#

time to design a driver interfac

#

I'll just dump everything here

#

each driver has it's own header

#

this header identifies the driver

#

as in how it should be detected in PnP

#

what it is

#

possibly the address of any callbacks like read,write,ioctl

#

drivers are going to be position independant executables

#

drivers will be able to access any kernel function, unless the kernel explicitly blocks it

#

yeah that seems good

#

after all, what could a driver interface possibly ever need other than that

#

also I'll need to implement DPCs of some form

#

since this all pretty non-trivial and boring

#

I'll port uACPI (@real pecan)

real pecan
real pecan
flint idol
#

oh yeah that as well

#

this time I won't rewrite in the middle of writing the driver interface

#

(no promises)

real pecan
#

Lol

flint idol
#

I wonder how much kernel api I can copy+paste without major changes

#

only one way to find out

#

I wouldn't be surprised if some headers still exist in this rewrite

#

well it uses lambdas because the previous rewrite was in C++

flint idol
#

bruh

#

I have to deal with the IRQ code and deferred execution code

#

I like how most of the old kernel's code for the uacpi kernel api has mostly stayed untouched

#

deferred execution code ported

flint idol
#

last time I checked uACPI wasn't this annoying to port

#

ok I got it to compile

# 
Page fault at 0xffffffff80001cc5 in kernel-mode while to read page at 0x0000000000000000, which is unpresent. Error code: 0```
#

👍

#

bro

#

all that testing

#

just for it to hang in the idle task

#

on all CPUs

#

oh wait qemu was just being slow

lean glen
flint idol
#

I was doing something in the wrong order in the kernel api

# 
uACPI, ERROR: unable to create FADT GPE block 0: invalid argument```
#

it was because my kernel api did something stupid

#

@real pecan

#

It works

#

idk why I decided to ping infy there

#

but it works

#

shutting down works

#

so does rebooting

#

so I'll just assume everything else works

#

unless uACPI is broken

#

I think I'll test OBOS on real hw

flint idol
#

test subject #1 works

#

test subject #2 works

#

changes have been pushed

flint idol
real pecan
#

Is obos actually stable now

real pecan
flint idol
#

And it worked on real hw first try

flint idol
#

And speaking of nothing changing, the kernel ported my old kernel's impl of uACPI

#

Barely anything changed

#

Anyway I'll be back in a couple hours

real pecan
flint idol
#

I think I've figured out how driver dependencies will work. At driver load, the kernel will keep a list of each driver that the driver imports symbols from, and save that somewhere in memory. This way, I won't have to worry about drivers providing an invalid list of dependencies (and I don't have to worry about how the list would be stored).

#

If a driver is requested to be unloaded, but other drivers depend on it, the unload will fail with OBOS_STATUS_IN_USE

#

I'll just give each driver a ref count

#

idk whether to count the kernel as a module

#

for dependencies

#

that wouldn't make make sense

#

but the kernel can have dependencies on driver symbols. Meaning it can be counted in a driver's ref count.

flint idol
#

TODO: Figure out what the "required" symbols actually will be

#

just a screenshot of it not crashing

#

what should I call the directory that contains the code for the driver interface

#

'driver_interface' is a bit long

#

'di' makes no sense

flint idol
#

my previous kernel had a special type of driver that was basically just for pnp

#

that would be used to detect pci(e) devices, as an example

#

there would be a seperate driver for each bus supported

#

and a driver would simply specify which driver it wanted to detect it

#

then the kernel would do the work

flint idol
#

my third kernel was the same, but simplified

#

the driver would specify what bus it was on (ACPI and PCI were the only choices at the time)

#

and it would specify some stuff about the device it was representing

#

for PCI that would be the class(es), subclass(es), and progIf(s)

#

(I just realized it should've also optionally had a vendor id)

#

ACPI would just have the HID and CIDs

#

I don't want to load drivers for the sole purpose of detecting whether said drivers should be loaded or not

real pecan
#

U can keep the supported device list in a separate section or something if u want

#

But usually dynamic drivers are loaded by userspace

flint idol
#

in this case it's for pnp

real pecan
#

Statically compiled ones always register themselves at init and provide the supported device list

flint idol
#

(not just ACPI PnP, but also loading drivers for PCI)

#

shit

#

I just realized I'm going to have to implement fork eventually

#

for user space

#

that sounds like a problem for future me

flint idol
flint idol
#

I did that

#

and added some macros to export symbols from both the kernel and drivers

flint idol
#

I got the ACPI powerdown button to work

#

it hangs on real hw with that code, but I'm not too worried about that, as I have no EC driver

real pecan
real pecan
#

You can also disable osi above windows xp and it wont touch ec

flint idol
#

doing that doesn't sound like an amazing idea

#

I worded that bad

#

nvm

real pecan
#

Lol

flint idol
#

anyway time to stop doing random stuff

#

and get to the driver interface

real pecan
#

Yes

chilly osprey
flint idol
#

lol

#

I think the format of the header looks good

chilly osprey
#

how does the header look?

flint idol
# 

enum { OBOS_DRIVER_MAGIC = 0x00116d868ac84e59 };

typedef enum driver_header_flags
{
    /// <summary>
    /// Should the driver be detected through ACPI?
    /// Cannot be specified if DETECT_VIA_PCI is specified.
    /// </summary>
    DRIVER_HEADER_FLAGS_DETECT_VIA_ACPI = 0x1,
    /// <summary>
    /// Should the driver be detected through PCI?
    /// Cannot be specified if DETECT_VIA_ACPI is specified.
    /// </summary>
    DRIVER_HEADER_FLAGS_DETECT_VIA_PCI = 0x2,
    /// <summary>
    /// If the driver does not have an entry point, specify this flag.
    /// </summary>
    DRIVER_HEADER_FLAGS_NO_ENTRY = 0x4,
    /// <summary>
    /// If set, the driver chooses its entry point's stack size.
    /// Ignored if DRIVER_HEADER_FLAGS_NO_ENTRY is set.
    /// </summary>
    DRIVER_HEADER_FLAGS_REQUEST_STACK_SIZE = 0x8,
} driver_header_flags;
typedef struct driver_header
{
    uint64_t magic;
    uint32_t flags;
    struct
    {
        uint32_t classCode;
        /// <summary>
        /// If a bit is set, the bit number will be the value.
        /// <para></para>
        /// This bitfield can have more than bit set (for multiple values).
        /// </summary>
        __uint128_t subclass;
        /// <summary>
        /// If a bit is set, the bit number will be the value.
        /// <para></para>
        /// This bitfield can have more than bit set (for multiple values).
        /// <para></para>
        /// If no bit is set any prog if is assumed.
        /// </summary>
        __uint128_t progIf;
    } pciId;
    struct
    {
        // There must be an HID, but CIDs are not required.
        // These strings are not null-terminated.
        char hid[8];
        char cids[16][8];
    } acpiId;
    size_t stackSize; // If DRIVER_HEADER_FLAGS_REQUEST_STACK_SIZE is set.
} driver_header;```
#

some lines have been copy+pasted (tm) from the previous kernels

chilly osprey
#

and does this go at the start of the file?

#

i assume

flint idol
#

it's more like metadata for the driver

chilly osprey
#

yeah i have something similar

flint idol
#

it can be anywhere in the file

chilly osprey
#

just a section in the elf file

#

ah i see

flint idol
#

maybe I'll make a section where it checks first

chilly osprey
#

mine is a lot more basic than this currently

flint idol
#

trust me most of that is just docs

chilly osprey
#

just a function pointer to the get_kernel_func and stuff like ID and stack size right now

flint idol
# 

enum { OBOS_DRIVER_MAGIC = 0x00116d868ac84e59 };

typedef enum driver_header_flags
{
    DRIVER_HEADER_FLAGS_DETECT_VIA_ACPI = 0x1,
    DRIVER_HEADER_FLAGS_DETECT_VIA_PCI = 0x2,
    DRIVER_HEADER_FLAGS_NO_ENTRY = 0x4,
    DRIVER_HEADER_FLAGS_REQUEST_STACK_SIZE = 0x8,
} driver_header_flags;
typedef struct driver_header
{
    uint64_t magic;
    uint32_t flags;
    struct
    {
        uint32_t classCode;
        __uint128_t subclass;
        __uint128_t progIf;
    } pciId;
    struct
    {
        char hid[8];
        char cids[16][8];
    } acpiId;
    size_t stackSize; // If DRIVER_HEADER_FLAGS_REQUEST_STACK_SIZE is set.
} driver_header;```
#

that's without any documentation

chilly osprey
#

yeah, still a lot more than mine

flint idol
chilly osprey
#

i plan on that yes

flint idol
#

I need to rewrite relocation code (again)

chilly osprey
#

otherwise it will be quite annoying to load them at the correct places

#

i dont have any kind of elf loading right now, still very early on in dev

#

started like 3 days ago

flint idol
#

I have elf loading, but only for remapping the kernel

#

at page table init

chilly osprey
#

are you having yours be in their own threads or anything?

#

or own processes even

flint idol
#

Drivers will have their own main thread that they can do anything they want with

#

at one point they were processes

#

but that design was cursed

chilly osprey
#

so just like a "driver process" that has threads for each one?

flint idol
#

in some of my previous designs, yes

#

my previous kernels' drivers mainly use their main thread for initialization of the device/fs/whatever it controls

#

then just exits

chilly osprey
#

why not make it just block until it gets a request to do something?

#

that way you dont need to create extra threads later on

flint idol
#

requests from the kernel were handled via a function table

#

for example, if the kernel needed to read from an AHCI disk, it would call the read function in the driver's function table

chilly osprey
#

right

#

on the active thread?

flint idol
#

yes

chilly osprey
#

so just sync IO then?

flint idol
#

ye

chilly osprey
#

i guess thats fine yeah

ornate ginkgo
chilly osprey
#

although then tbh i dont know if you even need a thread for that

#

since its just one init func

real pecan
ornate ginkgo
#

You can always create one later if you need it

real pecan
#

Just have all supported pnp ids

#

Compare against both his and cid

flint idol
ornate ginkgo
#

Also reminder to both of you to not directly use elf sections.

flint idol
#

yeah I know

#

the kernel will try an elf section

chilly osprey
flint idol
#

then it goes on to search for the header magic

flint idol
ornate ginkgo
chilly osprey
#

ah

#

i did not know that

#

perhaps i can just throw the header at the start of the file after linking it

inland radish
chilly osprey
#

then when the kernel parses the elf, it just skips past the header

ornate ginkgo
#

Yeah I didn't know for a while either. I recommend a custom phdr or just a big magic number

flint idol
ornate ginkgo
chilly osprey
ornate ginkgo
chilly osprey
#

i guess it makes readelf slightly more annoying

inland radish
#

it seems i had to specifically hunt out the GOT section to add the load base to everything inside that section

flint idol
#

I forgot what

#

wait a sec

#

DT_PLTGOT

#

my previous kernels' dynamic loader just assumed it was a GOT section

#

and said:

// TODO: Find out whether this is the PLT or GOT (if possible).```
chilly osprey
#

are you doing proper dynamic linking in yours?

#

like SO

flint idol
#

basically

chilly osprey
#

ah i see

inland radish
#

not sure if I can call it proper

chilly osprey
#

i planned on just providing a get_kernel_func that you call to get a function from its name

flint idol
#

I had support for almost every elf relocation in x86_64

inland radish
#

but I've only had one issue which was unimportant to me so I didn't address it -- that references to variables inside the kernel arent resolved properly

#

so like size_t MmTotalAvailablePages; isnt accessible by drivers

chilly osprey
#

getMmTotalAvailablePages()

inland radish
#

exactly what I'm doing

flint idol
inland radish
#

thats why i called it unimportant

inland radish
flint idol
#

where is the loader

#

in the kernel

#

?

#

nope

inland radish
#

that

#

thats the elf loader

#

ok so take MmTotalFreePages

ornate ginkgo
#

I appreciate the comment 🙏

inland radish
#

the address reported by test.sys is 0xFFFFF00000027000

#

which it isn't

flint idol
#

ok nvm idk how the hell that'd happen

#

gl with that

inland radish
#

its actually just the load base of the test.sys driver

image.png
inland radish
#

thanks for that

flint idol
#

did r4 write your relocation code by chance

inland radish
#

he didn't write any code for boron, I just adapted the Northport version

#

anyway MmTotalFreePages is defined in the GOT

image.png
#

IDA correctly identifies it as an external reference somehow

ornate ginkgo
#

You shouldn't need to worry about the got directly, unless you're doing lazy load shenanigans

flint idol
#

if it weren't for this, I would be cooked in terms of dynamic linking

Object File Format - Linker and Libraries Guide

This book describes the operations of the Solaris Operating System (Solaris OS) link-editor and runtime linker, and the objects on which these link-editors operate. The book covers the Link-Editor: ld(1), the Runtime Linker: ld.so.1(1), Shared Objects (sometimes referred to as Shared Libraries), and the ELF object file format.

inland radish
#

in my case I just add the load base to every got entry

#

which is wrong it seems

chilly osprey
inland radish
#

but I don't know of a better way

chilly osprey
#

i wrote a userspace dynamic linker purely by examining elf files a while ago

flint idol
chilly osprey
#

readelf and objdump were my best friends at that time

inland radish
#

but well I've been able to get by without such external references so it's probably ok

ornate ginkgo
#

Yeah you do

flint idol
#

oh yeah and does anyone know how to stop clangd from telling me this:

Unsupported option '-fsanitize=kernel-address' for target 'x86_64-unknown-unknown-elf'clang(drv_unsupported_opt_for_target)```
#

clangd 18.1.8

inland radish
flint idol
#

I have all the elf structs I need for relocation code

#

now I'll just write the base of the loader

#

but before that I need to write some code to find each kernel symbol and stick it in a symbol table

#

because I'm lazy, the symbol table will be an rb tree

flint idol
#

why are you in here and not in polls

hollow geyser
#

Idk

ornate ginkgo
#

Yeah, not the place for this?

chilly osprey
#

or would it just be the var that is required

flint idol
#

just the var

chilly osprey
#

hm thats annoying

flint idol
#

I've written the driver verification code

flint idol
flint idol
#

I've made a resolve symbol helper function

#

now time to get to the loading part

#

basically it first searches the kernel for the symbol

#

if that fails, it searches the loaded drivers

#

if that fails, it returns nullptr

flint idol
#

The VMA needs a memprotect function

#

or something similar

#

for the driver loader

#

I don't want to work around not having one

#

which I don't wanna explain how, but it's not very good

#

anyway I'll do that in about an hour, I gotta do something now

flint idol
#

I'm back

#

I now make thing

#

the memprotect function will just take in a base and size

#

and new prot flags

#

sounds good enough

#

ok I'm done

#

I'll leave it completely untested

#

it's basically just a bunch of code stolen from the free function (verifying the pages' existence + looping over them)

#

and some code stolen from the allocate function (protection flags to the page's protection bitfield)

flint idol
# 
struct relocation
{
    uint32_t symbolTableOffset;
    uintptr_t virtualAddress;
    uint16_t relocationType;
    int64_t addend;
};
struct relocation_array
{
    struct relocation* buf;
    size_t nRelocations;
};
void append_relocation(struct relocation_array* arr, struct relocation* what)
{
    arr->buf = OBOS_KernelAllocator->Reallocate(OBOS_KernelAllocator, arr->buf, ++arr->nRelocations, nullptr);
    OBOS_ASSERT(arr->buf); // oopsies
    arr->buf[arr->nRelocations - 1] = *what; 
}```
#

This looks pretty good so far

#

(none of this is relocation code yet)

weary hound
#

what file format are you using for your drivers?

flint idol
#

ELF

weary hound
#

then why not just have a pointer to Elf64_Rela from the file instead of copying that info into an array?

flint idol
#

that is a good question

#

which I will have an answer for in a sec

#

iirc it was because there can be multiple Elf64_Rela things

#

something like that

#

yeah that is why

#

also I wanted a unified way to represent the relocations so I didn't have to copy a bunch of code

#

I just discovered a bug dating all the way back to the 3rd kernel

#

it would treat DT_REL dynamic headers as DT_RELA

weary hound
flint idol
#

idk how I hadn't found that eariler

weary hound
#

also DT_REL you're unlikely to find anywhere these days

flint idol
#

which I assume why that bug hadn't showed up

weary hound
#

maybe it's more likely on 32-bit architectures

flint idol
#

until I saw it while copy+pasting the code from my third kernel

#

maybe it'd be more memory efficient to store pointers to the Rela tables

#

in the array

#

instead of copying the entire thing

flint idol
weary hound
#

i mean there's only 2 arrays that you only need to look at once

#

it's not like you're gonna keep referring to them all throughout the runtime of th driver

flint idol
#

yeah true

#

I'll just have the array store pointers to the tables

# 
struct relocation_array
{
    Elf64_Dyn** buf;
    size_t nRelocations;
};
static void append_relocation_table(struct relocation_array* arr, Elf64_Dyn* what)
{
    arr->buf = OBOS_KernelAllocator->Reallocate(OBOS_KernelAllocator, arr->buf, ++arr->nRelocations, nullptr);
    OBOS_ASSERT(arr->buf); // oopsies
    arr->buf[arr->nRelocations - 1] = what; 
}```
#

with the help of copy+paste from previous kernel where I did all the work, I think I'll be able to get this done in <20 minutes

flint idol
#

first thing I'm going to do as soon as I have a stable driver interface is make an AHCI driver

#

then make swap be on disk

flint idol
#

I think I'm done the driver loader

#

I'll test it

#

inb4 everything just breaks

#

like straight up doesn't work

#

because I'm too lazy to make a proper driver to test with

#

I'm going to make some makeshift driver with some dumb header

#

basically only bare minimum to get it loaded

#

done

#

I added in some dumb header'

#

after some bug fixes

#

it finds the driver header

#

but it never loads the elf object

#

||because I forgot to add in the call to the function that does so||

flint idol
#

it almost works

#

I just need to solve some paging shenanigans

#

done

#

it was something to do with the tlb

#

it now has yet another cryptic log message

#

now if anyone was wondering, the test driver doesn't actually exist on the git repo

#

it's a < 100 line file that just has some dumb driver header and imports OBOS_Log

#

that's compiled into a PIE

#

that's turned into a c header file

#

which is loaded by the kernel

#

I'll test further tomorrow

#

and make this an actually proper driver

#

but until then

#

so yeah tomorrow I'll test dependency counting

#

which is actually not finished

stone gyro
#

_ _

inland radish
#

also limine can load multiple modules

flint idol
inland radish
flint idol
#

The test driver isn't even built alongside the kernel

#

This "driver" was purely there to test relocation

flint idol
flint idol
#

so the thing broke

#

I think I told the linker to do something stupid

#

but I can't quite figure out what

#

one of the relocation tables is just zeroes

#

unless I'm parsing this wrong

#

readelf seems to understand it

#

I don't get it

#

how does 0x1178 become 0x2178

#

for readelf

#

well it's using the section which has that offset

#

I think I wrote my linker script wrong then

#

Can anyone make sure I didn't write anything that would mess up the dynamic header

real pecan
#

i'd ask mint

flint idol
#

I link using:

-nostdlib
-Wl,--export-dynamic
-Wl,-shared
-Wl,-pic
-T driver_link.ld```
flint idol
sharp pike
#

what?

#

oh god

#

you're trying to use custom phdr types

#

may god have mercy

flint idol
#

is there anything that would mess up the dynamic header though

sharp pike
#

what is the loader?

flint idol
#

my kernel's dynamic loader

#

for drivers

#

if that's what you meant

sharp pike
#

yeah alright

#

do you apply all relocations as needed?

celest acorn
#

OBOS

flint idol
#

when it finds the PT_JMPREL header

#

it gets the offset 0x1178

#

which is the same as reported by readelf

#

except, there's only zeroes at that offset

# 
Dynamic section at offset 0x4000 contains 15 entries:
  Tag        Type                         Name/Value
 0x0000000000000004 (HASH)               0x1148
 0x0000000000000005 (STRTAB)             0x10f8
 0x0000000000000006 (SYMTAB)             0x1050
 0x000000000000000a (STRSZ)              79 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x2158
 0x0000000000000002 (PLTRELSZ)           72 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x1178
 0x0000000000000007 (RELA)               0x1178
 0x0000000000000008 (RELASZ)             24 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffb (FLAGS_1)            Flags: PIE
 0x0000000000000000 (NULL)               0x0

Relocation section '.rela.dyn' at offset 0x2178 contains 3 entries:
  Offset          Info           Type           Sym. Value    Sym. Name + Addend
000000002150  000100000006 R_X86_64_GLOB_DAT 0000000000000000 __stack_chk_guard + 0
000000002170  000300000007 R_X86_64_JUMP_SLO 0000000000000000 __stack_chk_fail + 0
000000002178  000400000007 R_X86_64_JUMP_SLO 0000000000000000 OBOS_Log + 0```
white mulch
flint idol
#

wdym

#

like in the program's binary

white mulch
#

as in the base you load the driver at + dt_jmprel_offset

flint idol
#

wait that's what I was supposed to do

#

I thought it was an offset into the program binary

#

How are all these bugs in my code (since my third kernel!) only showing up now

#

lol

sharp pike
#

it says it's a PIE in FLAGS_1

#

but you don't seem to want a PIE

flint idol
#

what would I want then?

sharp pike
#

well idk that's up to you, i am saying that because you're passing -shared -pic to the linker

#

that's usually done for shared objects, not PIEs

flint idol
#

the only reason I passed -shared was because I didn't really know how to get the linker to understand that the symbols that were unresolved were from the kernel

sharp pike
#

for PIE you link with -pie, and as a matter of fact i am surprised the final object is marked as PIE, unless the C compiler driver is passing -pie on its own

#

PIEs can still link to outside symbols

flint idol
#

how do I get the linker to assume that any unresolved symbols are somewhere else

#

like undefined reference to `OBOS_Log'

white mulch
#

one option would be to make a shared library that contains stubs for all the exported kernel symbols and link the exe against that

inland radish
#

I use -shared in boron

flint idol
white mulch
#

or just make the driver be a shared library and pass --allow-shlib-undefined to the linker

inland radish
#

here are my flags

override CFLAGS += -fno-omit-frame-pointer -std=c11 -ffreestanding -fno-stack-protector -fno-stack-check -fno-lto -fPIC -m64 -march=x86-64 -mabi=sysv -mno-80387 -mno-mmx -mno-sse -mno-sse2 -mno-red-zone -MMD -MP -I.
override CXXFLAGS += -fno-omit-frame-pointer -std=c++17 -ffreestanding -fno-stack-protector -fno-stack-check -fno-lto -fPIC -m64 -march=x86-64 -mabi=sysv -mno-80387 -mno-mmx -mno-sse -mno-sse2 -mno-red-zone -MMD -MP -fno-exceptions -fno-rtti -I.
override LDFLAGSBASE += -nostdlib -shared -e $(DRIVER_ENTRY) -m elf_x86_64 -z max-page-size=0x1000
sharp pike
#

it's fine to make a PIE for anything that does not need to be linked against

#

if drivers are meant to use symbols from each other, then shared objects is the way to go instead

flint idol
#

ok

sharp pike
#

so yeah you can pass --unresolved-symbols=ignore-all i guess, even when making a PIE

#

additionally, you should probably add the headers to your PHDRs

flint idol
#

which headers?

sharp pike
#

like something like PHDRS { headers PT_PHDR PHDRS; text PT_LOAD FILEHDR PHDRS;

flint idol
# 
not enough room for program headers, try linking with -N```
#

wait

#

ok I fixed it

#

now time to continue with the driver loader

#

all that's left for me to do is the dependency thing

#

which will be pretty simple

flint idol
#

also simple

sharp pike
#

ld doesn't seem happy unless you make a shared object instead of a PIE

#

with unresolved symbols

#

so it's either shared object or what qwinci said

#

afaik

flint idol
#

well shared object is a lot more convinient

#

so I'll go with that

#

btw, how do I set the visibility of a symbol defined in a linker script

sharp pike
#

i'm not sure you can within the linker script, but i'd check the documentation

flint idol
#

well some stack overflow answer says you cannot

flint idol
# 
[ DEBUG ] Loading test driver.
[  LOG  ] OBOS_DriverEntry: Hello from a test driver. Driver base: ffffff00004a4000.
[  LOG  ] Test!```
#

it works again

#

I needed to implement some hashtable code

#

for accessing symbols internal to the shared object

#

and by implement, I mean copy+paste from my previous kernel

flint idol
#

my thought is to just iterate over symbols in the dynamic symbol table, and all symbols that aren't marked as undefined will be added to the list of driver symbols

#

then there's one more thing I'd need to do, and that's make all phdrs without the flag PF_OBOS_PAGEABLE be non-pageable

flint idol
#

the spaghetti code that is the driver loader is completed

white mulch
flint idol
#

yeah that's what I meant

#

ty

flint idol
#

I'm getting a bogus address in the GOT

flint idol
#

uACPI init takes way to long in tcg+debugger attached

#

my vmm is probably the bottleneck

#

or maybe dozens of breakpoints within critical kernel api wasn't a good idea

#

oh wait

#

the bug with the loader

#

wasn't a bug with the loader

#

it was because the imported symbol was hidden

#

upon making the api for finding symbols better it errors out on load

#

instead of crashing during driver execution

#

now that I've exported that symbol\

#

the driver loads successfully

#

that's like, the majority of the driver interface done

#

the other part is pnp

#

and determining what functions a driver might have in some function table

#

then I'm making an AHCI driver

#

probably

#

then I'm making swap run off of the AHCI driver

#

now I'm going to test whether drivers can import functions from others

#

for that, I'll need yet another test driver

#

the code is pushed

#

the symbol resolver does not work properly

#

but I will debug that later

flint idol
#

I fixed it

#

I forgot to insert the symbol into the symbol table lol

#

just pushed the bug fix

flint idol
#

an example of the former is an AHCI driver

#

an example of the latter is a gpu driver

#

however, each driver will have an ioctl callback

#

no matter the type

#

for example, maybe the AHCI driver would have an ioctl for power management of the drives

#

since most devices aren't just read and written from

#

and have some additional configuration you might need to do

#

or some other stuff you might want to do for a performance boost

flint idol
#

the kernel compiles a driver list

#

then the pnp thing will seperate those drivers into ACPI drivers and PCI drivers (a driver can be in both of these lists)

#

then, the kernel will iterate over the ACPI devices using uACPI, and for each one, it will search the ACPI list for a driver that has said device's HID

#

or maybe I'll have a map of HIDs->Driver(s)

#

that'd be faster

#

as for PCI, the kernel will simply enumerate the bus and for each device, the kernel will search for a driver supporting the class code, subclass, prog if, and vendor id

#

perhaps also with a map

#

and maybe I'll throw PnP into a kernel module

#

in case someone wants to just have a config file of drivers to load at init

empty kernel
#

what happens if there is a device which is both in PCI and ACPI?

flint idol
#

oh wait I thought you meant driver

#

I'll probably just make the driver header have a member which specifies its preferred interface

#

to be detected

#

holy

#

the driver interface so far has added 2000 loc

#

to the kernel

flint idol
#

but before that

#

I'm going to compile the new qemu version

#

(debian is stuck on 7.2.11)

flint idol
#

this is taking a lot longer than I thought

#

6000 more files left

#

bruh

#

it was compiling all targets

#

I think I'll just do x86_64-softmmu for now

#

then when I port obos I'll compile more targets

#

I'll just leave it to build overnight

flint idol
#

nvm it took less than overnight

#

after I changed it to that single target

flint idol
#

but I was having problems with configure

# 
QEMU 9.0.50 monitor - type 'help' for more information```
#

unfortunately, tcg isn't much faster

#

@real pecan Your uACPI broke

#

on real hw

#

hold on let me turn on trace logging

#

oooo UACPI_LOG_DEBUG

#

nvm my tty is too slow

#

maybe the scheduler did something dumb

#

it did

#

what it did?

#

I have no idea

#

debugging a scheduler bug present only on real hardware without a debugger

#

yeah no

#

why in the world would it just hang

#

without any trace

#

randomizing qemu's memory doesn't reveal any problems

#

since I don't care enough about that at the moment

#

I tried compiling as release

#

for it to pf in find avaliable address

#

lastNode is nullptr

#

idk why

# 
if ((currentNodeAddr - lastAddress) >= (size + pgSize))
#

isn't supposed to be true at the first iteration of the loop

#

now why is the kernel just not adding pages to the rb tree

#

you know what fuck this

#

I'm too tired for this shit

real pecan
flint idol
vale nymph
#

Rename uACPI to uStressTester

real pecan
#

Lol

flint idol
#

it wasn't uACPI that was causing problems with the scheduler

#

when I masked out the scheduler during uACPI init, it went through init just fine

#

but as soon as the scheduler was unmasked everything hung

flint idol