OBOS (not vibecoded) | Operating System Development | Page 21

real pecan Oct 5, 2024, 11:01 PM

#

They just spin?

flint idol Oct 5, 2024, 11:01 PM

#

nope

#

they exit their thread

#

then each driver has a function table

#

kinda like a vtable

#

that the kernel calls to request something from the driver

real pecan Oct 5, 2024, 11:02 PM

#

Yeah but like

#

Cant u make this struct like an out parameter of the driver entrypoint

#

And have the trampoline call this exit thing

flint idol Oct 5, 2024, 11:03 PM

#

drivers just store it in their header

#

because it's simple for one

#

and also drivers can request to not have an entry point

real pecan Oct 5, 2024, 11:03 PM

#

I mean the status struct

flint idol Oct 5, 2024, 11:03 PM

#

oh the status struct

#

no that'd break compatibility with the test driver I use for fireworks

#

because that's on master, but these changes are going in userspace-work

real pecan Oct 5, 2024, 11:04 PM

#

Cant u just uhhh... Change it?

flint idol Oct 5, 2024, 11:04 PM

#

I mean

#

the test driver's fireworks test is only on master

#

as of now

#

these changes are not on master

real pecan Oct 5, 2024, 11:05 PM

#

F

flint idol Oct 5, 2024, 11:05 PM

#

and are instead on userspace-work

real pecan Oct 5, 2024, 11:05 PM

#

Your wip branch is behind master somehow?

flint idol Oct 5, 2024, 11:05 PM

#

yes

real pecan Oct 5, 2024, 11:05 PM

#

💀

#

Wtf

flint idol Oct 5, 2024, 11:05 PM

#

I should maybe bump it sometime soon

#

I fear merge conflicts though

#

because like

real pecan Oct 5, 2024, 11:06 PM

#

git rebase master

flint idol Oct 5, 2024, 11:06 PM

#

this

real pecan Oct 5, 2024, 11:06 PM

#

How does that even happen is beyond me

flint idol Oct 5, 2024, 11:06 PM

#

2k additions

#

2k deletions

#

this is a lot better than panicking on driver failure KEKW

#

which I unknowingly did

#

or rather I forgot that I did

flint idol Oct 5, 2024, 11:39 PM

#

yeah something fnuy is goin on with my ahci driver

#

lmao

#

I forgot to remove a forever loop

#

when testing timeouts

#

the exit driver function works

#

it's quite simple

#

simply prints the info if there is an error status

#

it exits the thread if the fatal == false or if status==SUCCESS

#

otherwise

#

it initializes a DPC to unload the driver

#

and exits the thread

#

void Drv_ExitDriver(struct driver_id* id, const driver_init_status* status)
{
    if (!id || !status)
        return;
    if (id->main_thread != Core_GetCurrentThread())
        return;
    irql oldIrql = Core_RaiseIrql(IRQL_DISPATCH);
    if (obos_is_error(status->status))
    {
        if (OBOS_GetLogLevel() <= LOG_LEVEL_WARNING)
        {
            OBOS_Warning("Initialization of driver %d (%s) failed with status %d.\n", 
                id->id, 
                uacpi_strnlen(id->header.driverName, 64) ? id->header.driverName : "Unknown",
                status->status
            );
            if (status->context)
                printf("Note: %s\n", status->context);
            if (status->fatal)
                printf("Note: Fatal error. Unloading the driver.\n");
        }
    }
    if (!status->fatal || obos_is_success(status->status))
        Core_ExitCurrentThread();
    dpc* dpc = CoreH_AllocateDPC(nullptr);
    OBOS_ASSERT(dpc);
    dpc->userdata = id;
    CoreH_InitializeDPC(dpc, unload_driver_dpc, CoreH_CPUIdToAffinity(CoreS_GetCPULocalPtr()->id) /* make sure this runs on this CPU. */);
    Core_ExitCurrentThread();
    OBOS_UNREACHABLE;
    OBOS_UNUSED(oldIrql);
}```

#

only a wall of text

#

if anyone's wondering why I only allow the dpc to run on the current CPU it's because there can be a race condition where the dpc runs before the thread is exitted

#

causing unload to fail saying the main thread didn't exit

#

and I raise the irql to make sure I run the dpc on a separate thread stack

flint idol Oct 6, 2024, 12:19 AM

#

ok time to get back

#

to torture

#

mirror mirror on the wall

#

who has the buggiest code of them all

#

♫ 99 little bugs in the code ♫
♫ 99 little bugs ♫
♫ take one down, patch it around ♫
♫ 137 little bugs in the code ♫

#

klange, 2024

#

excuse me why the fuck does the ahci driver map ~4M of memory

#

hmm what could this bug be

#

region corruption

#

where could that happen

#

if (block->prev)
    block->prev->next = block->next;
if (block->next)
    block->next->prev = block->prev;
if (This->regionHead == block)
    This->regionHead = block->next;
if (This->regionTail == block)
    This->regionTail = block->prev;

#

there's no way this list remove code is wrong, right?

vale nymph Oct 6, 2024, 12:39 AM

#

flint idol ```c void Drv_ExitDriver(struct driver_id* id, const driver_init_status* status)...

why are you calling into the allocator with dispatch irql that sounds bad

flint idol Oct 6, 2024, 12:40 AM

#

I don't think it's a problem

#

it acquires a spinlock in the allocator, but that's about it

#

@devout niche is it ok to call into the allocator at IRQL_DISPATCH?

#

fadanoid the wise

flint idol Oct 6, 2024, 12:41 AM

#

flint idol ```c if (block->prev) block->prev->next = block->next; if (block->next) ...

nope

#

all I can think of is

#

mismatched allocators

#

but that'd have panicked

#

because kasan violation

devout niche Oct 6, 2024, 12:46 AM

#

flint idol <@391791686881247234> is it ok to call into the allocator at IRQL_DISPATCH?

it depends on your allocator and what you're allocating

#

if the call would have to wait for memory to become available no

#

if it acquires a sleeping mutex, no

flint idol Oct 6, 2024, 12:47 AM

#

I see

#

thanks

#

I feel like this is an off-by-one in the vmm free function

#

like it was last time

#

I have written allocator test

#

allocator_info* alloc = OBOS_KernelAllocator;
void* to_free = nullptr;
while (true)
{
    size_t sz = random_number() % 4096 + 256;
    char* ret = alloc->ZeroAllocate(alloc, sz, 1, nullptr);
    ret[0] = random_number8();
    ret[sz-1] = random_number8();
    if (random_number() % 2)
    {
        alloc->Free(alloc, to_free, 0);
        to_free = ret;
    }
}```

#

I made that multithreaded

#

and it caught a bug

#

seemingly

#

well when I make it so freeRegion does nothing

#

it "works"

#

other than the fact that no memory is getting freed

#

which sounds like a bug with munmap

#

perhaps it accidentally frees past the region bounds

#

you know that's it

#

I'm going full verbose log mode

#

it seems to be freeing 0x5000 bytes at the faulting address

#

before it faults

#

derefing physical page 000000003cca0000 representing ffffff0000ed7000
unmapping ffffff0000ed7000
derefing physical page 000000003cbb9000 representing ffffff0000ed8000
unmapping ffffff0000ed8000
derefing physical page 000000003cc2e000 representing ffffff0000ed9000
unmapping ffffff0000ed9000
derefing physical page 000000003cc8e000 representing ffffff0000eda000
unmapping ffffff0000eda000
derefing physical page 000000003cc9f000 representing ffffff0000edb000
unmapping ffffff0000edb000```

#

Page fault at 0xffffffff8001aaa3 in kernel-mode while to write page at 0xffffff0000ed7090, which is unpresent. Error code: 2

#

and it seems to be a different size each size

#

now it's 0x10000 bytes

#

tf

#

it freed that in vma stack free

#

and sometimes its freed in freeRegion

#

if (block->prev)
        OBOS_ASSERT(block->prev->next != block);
    if (block->next)
        OBOS_ASSERT(block->next->prev != block);
    OBOS_ASSERT(This->regionHead != block);
    OBOS_ASSERT(This->regionTail != block);```

#

I added these assertions

#

tf

#

blk->prev ffff80003ccdd000
blk->next 0000000000000000
linked ffffff0000ed7000```

#

0xffff80003ccdd000 is an address in the vmm allocator

#

yet this crash happens in the non paged pool allocator

#

and that block was allocated in the non paged-pool allocator

#

sometimes it's not

#

printf my beloved

#

HOLD ON

#

printf > gdb

#

freeing 20480 at ffffff0000913000. called from ffffffff8001c0f7```

#

then later in my log file

#

blk->prev ffffff0000913000
blk->next 0000000000000000
linked ffffff000050a000```

#

without allocation of that address in between

flint idol Oct 6, 2024, 12:05 PM

#

aaand it's an allocator mismatch

#

mapped 65536 bytes at ffffff0000ec8000
mapped 65536 bytes at ffffff0000ed8000
mapped 20480 bytes at ffffff0000ee8000
linked ffffff0000ee8000
blk->prev ffffff0000d6e000
blk->next 0000000000000000
This ffffffff800dc840```

#

alloc: freeing blk ffffff0000ee8000
This: ffffffff805032e0
blk->This ffffffff800dc840```

#

I have no idea why this isn't failing

if ((allocator_info*)r->This != This_)
    OBOS_ASANReport((uintptr_t)__builtin_return_address(0), (uintptr_t)base, nBytes, ASAN_AllocatorMismatch, false);```

#

FUCK

#

I forgot to handle allocator mismatch

#

in asan_report

#

LiPerformDelisquesceUponObjectW

flint idol Oct 6, 2024, 12:36 PM

#

rather my vmm committed memory counter is broken

#

or something is terribly wrong

#

since there is ~20M of committed memory

#

yet the allocators report a lot less in use

#

only 5M + 82K

flint idol Oct 6, 2024, 12:37 PM

#

flint idol since there is ~20M of committed memory

27M at the sample time

#

well ~8M is the framebuffer+backbuffer

#

and for some reason

#

the fat driver is using 3M

#

I wonder if I can optimize the fat driver

#

to directly use the page cache

#

instead of using the VFS to read/write from a volume

#

iirc microsoft's fat driver does something like that?

#

idk if it does

flint idol Oct 6, 2024, 12:49 PM

#

flint idol to directly use the page cache

I mean this sounds like a good idea

#

I just pushed the code

flint idol Oct 6, 2024, 1:02 PM

#

flint idol I mean this sounds like a good idea

I'll be doing that

flint idol Oct 6, 2024, 1:02 PM

#

flint idol the fat driver is using 3M

nvm

#

FATAllocator = OBOS_KernelAllocator;

#

my FAT manipulation code now uses the page cache

#

directly

#

the goal is to make anything that uses Vfs_FdRead in the FAT driver use the pagecache manipulation functions instead

#

for (size_t i = 0; i < szClusters; i++)
{
    Vfs_FdSeek(cache->volume, ClusterToSector(cache, cluster+i)*cache->blkSize, SEEK_SET);
    Vfs_FdRead(cache->volume, buf, bytesPerCluster, nullptr);
    Vfs_FdSeek(cache->volume, ClusterToSector(cache, newCluster+i)*cache->blkSize, SEEK_SET);
    Vfs_FdWrite(cache->volume, buf, bytesPerCluster, nullptr);
}
if (cluster)
    FreeClusters(cache, cluster, szClusters);```

#

I wish I had commented this code

#

I think it is supposed to be copying sectors?

#

it is

#

and in looking at that code I remembered that I need to mark the pc regions dirty

flint idol Oct 6, 2024, 2:54 PM

#

I hate my fat driver

#

the code is so dirty

#

and unpleasant

#

but it kinda works

#

which is good enough

#

if (inRoot && cache->fatType != FAT32_VOLUME)
{
    Vfs_FdSeek(cache->volume, fileoff, SEEK_SET);
    Vfs_FdWrite(cache->volume, buf, blkSize, nullptr);
}
else
{
    Vfs_FdSeek(cache->volume, fileoff, SEEK_SET);
    Vfs_FdWrite(cache->volume, buf, blkSize, nullptr);
}```

#

what is teh point of this if statement

#

lol

#

while (!wroteback)
{
    if (inRoot && cache->fatType != FAT32_VOLUME)
    {
        Vfs_FdSeek(cache->volume, fileoff, SEEK_SET);
        Vfs_FdWrite(cache->volume, buf, blkSize, nullptr);
    }
    else
    {
        Vfs_FdSeek(cache->volume, fileoff, SEEK_SET);
        Vfs_FdWrite(cache->volume, buf, blkSize, nullptr);
    }
    wroteback = true;
}```

#

tf is the point of the loop

#

something tells me I was really tired writing that code..

#

suspiciously the fat driver still works

#

when I changed it to use pagecache

flint idol Oct 6, 2024, 3:21 PM

#

flint idol suspiciously the fat driver still works

first try

#

wow my ahci driver does not know how to scatter gather properly does it...

#

ahci_dma_prepare_buf ahci(0x5567b57622b0)[1]: prepare buf limit=16384 prepared=12288

inland radish Oct 6, 2024, 3:28 PM

#

this is incredibly irresponsible tbh

flint idol Oct 6, 2024, 3:28 PM

#

I removed that code

inland radish Oct 6, 2024, 3:28 PM

#

if the driver hangs it's a problem in the driver

flint idol Oct 6, 2024, 3:41 PM

#

I think I'm rewriting my scatter gather algorithm

#

to not be shit

inland radish Oct 6, 2024, 3:48 PM

#

the only use i know of so far for scatter gathering is for readv/writev support

flint idol Oct 6, 2024, 3:48 PM

#

AHCI uses it

#

it's called the PRDT iirc

inland radish Oct 6, 2024, 3:48 PM

#

perhaps i should add support for it in my NVME driver

#

for now I only support the PRP method

flint idol Oct 6, 2024, 3:49 PM

#

I assume it you just give it a physical address and size

#

and it DMAs from there

inland radish Oct 6, 2024, 3:50 PM

#

You give it several physical page addresses

#

Not just one

#

The first physical page in the PRP list may have an offset in the page

#

This allows for zero copy IO up until the file system layer

flint idol Oct 6, 2024, 3:53 PM

#

qemu-system-x86_64: hw/ide/core.c:934: ide_dma_cb: Assertion `prep_size >= 0 && prep_size <= n * 512' failed.```

#

bruh

flint idol Oct 6, 2024, 4:29 PM

#

ok the prdt population code is rewritten

#

and works with this current test

#

flint idol Oct 6, 2024, 10:21 PM

#

after a small break

#

I will now utilize the page cache within read/write

#

but I'll optimize that further

#

actually

#

nvm, my 2nd optimization strategy was dumb

#

I was going to see if the clusters within the FAT were contiguous then do the IO in chunks

#

maybe that's a good idea...

#

but that makes this two-pass for what gains...

#

well I guess the io would be done in less fragments

#

instead of reading sector n then sector n+1 then sector n+2

real pecan Oct 6, 2024, 10:25 PM

#

congrats btw

flint idol Oct 6, 2024, 10:25 PM

#

this is an honour

#

and

flint idol Oct 6, 2024, 10:26 PM

#

flint idol instead of reading sector n then sector n+1 then sector n+2

as in, in separate commands

#

I would read them all in one command

#

or write

#

I also want to implement extending of cluster chains

#

and allocating uncontiguous clusters

flint idol Oct 6, 2024, 10:41 PM

#

real pecan congrats btw

wait how do you see that

#

Damn it my computer died on me

lean glen Oct 6, 2024, 10:44 PM

#

flint idol and

"systems software engineer" kinda sounds like thats his job

flint idol Oct 6, 2024, 10:45 PM

#

ok my computer is back

#

it just quite literally died

real pecan Oct 6, 2024, 10:45 PM

#

flint idol wait how do you see that

lol

#

because i follow fadanoid

flint idol Oct 6, 2024, 10:46 PM

#

flint idol ok my computer is back

when it came back

#

it started spamming 4

#

and the ctrl+alt+delete screen was up on only one display

real pecan Oct 6, 2024, 10:48 PM

#

lean glen "systems software engineer" kinda sounds like thats his job

Well thats his dream

flint idol Oct 6, 2024, 10:50 PM

#

well vscodium died on me

#

guess I'm using nano now

lean glen Oct 6, 2024, 10:50 PM

#

or use a proper text editor

#

like

#

emacs

flint idol Oct 6, 2024, 10:51 PM

#

kate meme

#

(the kde text editor)

lean glen Oct 6, 2024, 10:51 PM

#

I'm aware of it

#

never tried it

#

heard its pretty good tho

flint idol Oct 6, 2024, 10:51 PM

#

it seems to support LSP (language server protocol I think?)

#

and git

#

ngl kate is actually kinda good....

flint idol Oct 6, 2024, 11:34 PM

#

the fuck is this allocator bug

#

in Reallocate

#

it's giving me a region starting in the shadow space

#

meh whatever

#

I think I'm going to add colours to my logs

#

enum colour
{
         COLOR_BLACK = 0,
    COLOR_BLUE = 1,
    COLOR_GREEN = 2,
    COLOR_CYAN = 3,
    COLOR_RED = 4,
    COLOR_MAGENTA = 5,
    COLOR_BROWN = 6,
    COLOR_LIGHT_GREY = 7,
    COLOR_DARK_GREY = 8,
    COLOR_LIGHT_BLUE = 9,
    COLOR_LIGHT_GREEN = 10,
    COLOR_LIGHT_CYAN = 11,
    COLOR_LIGHT_RED = 12,
    COLOR_LIGHT_MAGENTA = 13,
    COLOR_LIGHT_BROWN = 14,
    COLOR_WHITE = 15,
};
struct log_backend
{
        void(*write)(char* buf, size_t sz);
// Can be nullptr.
        void(*set_colour)(colour c);
// Can be nullptr.
        void(*reset_colour)();
};```

lean glen Oct 6, 2024, 11:45 PM

#

COLOR_COLOR

#

why

#

😭

flint idol Oct 6, 2024, 11:45 PM

#

I was copy pasting from somewhere

#

typedef enum {
    COLOR_BLACK = 0,
    COLOR_BLUE = 1,
    COLOR_GREEN = 2,
    COLOR_CYAN = 3,
    COLOR_RED = 4,
    COLOR_MAGENTA = 5,
    COLOR_BROWN = 6,
    COLOR_LIGHT_GREY = 7,
    COLOR_DARK_GREY = 8,
    COLOR_LIGHT_BLUE = 9,
    COLOR_LIGHT_GREEN = 10,
    COLOR_LIGHT_CYAN = 11,
    COLOR_LIGHT_RED = 12,
    COLOR_LIGHT_MAGENTA = 13,
    COLOR_LIGHT_BROWN = 14,
    COLOR_WHITE = 15,
} color;
typedef struct log_backend {
    void* userdata;
    void(*write)(const char* buf, size_t sz, void* userdata);
    void(*set_color)(color c, void* userdata);
    void(*reset_color)(void* userdata);
} log_backend;```

#

my final thingy

vale nymph Oct 7, 2024, 12:33 AM

#

vga ass colors

#

did you copy it from the barebones thing in the wiki lmao

flint idol Oct 7, 2024, 12:34 AM

#

vale nymph did you copy it from the barebones thing in the wiki lmao

probably

#

I copied them from my 1st kernel

#

lmao

#

I just need simple colours ok

#

and I also need to fix an allocator mismatch

#

before smp init?

#

weird

#

it's while parsing the kernel command line

#

ofc only in release mode

#

it was because I was checking for that in Free regardless of kasan being enabled

#

but in the allocate region function it was only setting the region's owner pointer if kasan is enabled

#

causing r->This to be nullptr

#

#

I now have pretty logs

#

:-)

#

infy can you not change the uacpi log levels any time soon

#

OBOS_SetColor(OBOS_LogLevelToColor[level-1]);
printf("[uACPI][%s]: ", prefix);
vprintf(format, list);
OBOS_ResetColor();```

#

in my kernel's uacpi_kernel_vlog

#

ty

ornate ginkgo Oct 7, 2024, 12:52 AM

#

use a switch and a default case?

flint idol Oct 7, 2024, 12:52 AM

#

too much work

#

yk what I mean

#

if they change that's when I do that

#

speaking of uacpi...

#

I need to update my kernel api

flint idol Oct 7, 2024, 1:12 AM

#

which was as soon as I updated uacpi...

#

since ttys are stupid

#

I will implement the next thing on the list

#

which is...

#

pipes

#

there's no way they're that complicated...

#

isn't it just a fifo buffer?

#

that's a named pipe

#

I think anon pipes are the same

#

i.e., pipes made with pipe.2

#

except they aren't named

#

but no, one fd is wrote to

#

and the other is read from

#

so fifo pipes are basically just temporary files in ram?

#

seems like it

flint idol Oct 7, 2024, 1:39 AM

#

I'll be starting with fifo pipes

#

since they're more intuitive

#

for me

#

ahhhhhhhhhh

#

vnode initialization

vale nymph Oct 7, 2024, 1:58 AM

#

they have some annoying rules but theyre ok

flint idol Oct 7, 2024, 1:58 AM

#

vale nymph they have some annoying rules but theyre ok

fifo pipes?

#

to man fifo I go...

#

oh yeah the atomic thing

#

and to man 7 pipe I go...

flint idol Oct 7, 2024, 2:17 AM

#

@vale nymph is this the pipe atomic funniness you were talking about before?

#

The only difference between pipes and FIFOs is the manner in which they are created and opened. Once these tasks have been accomplished, I/O on pipes and FIFOs has exactly the same seman‐
tics.

#

just a note to self

vale nymph Oct 7, 2024, 2:47 AM

#

yeah

flint idol Oct 7, 2024, 2:47 AM

#

ok

#

// !O_NONBLOCK, n <= PIPE_BUF: Atomic writes, block if no room
//  O_NONBLOCK, n <= PIPE_BUF: Atomic writes, return OBOS_STATUS_TRY_AGAIN if no room.
// !O_NONBLOCK,  n > PIPE_BUF: Unatomic writes, blocks until data is written (which includes blocking until the pipe is full).
//  O_NONBLOCK,  n > PIPE_BUF: Unatomic writes, return OBOS_STATUS_TRY_AGAIN if no room. Note: Partial writes are possible (check nWritten).```

#

I think I understood that correctly

#

what does it mean by atomic write?

#

take a rw lock until the write is done?

#

sometime tomorrow I will continue implementing pipes

#

and hopefully will be done pipes tomorrow or the day after

#

and can start syscalls after

#

pipes are at least ez pz

#

I'll implement TTYs in the near future™️

vale nymph Oct 7, 2024, 2:53 AM

#

flint idol what does it mean by atomic write?

Write the whole thing in one go

#

So if its above pipe_buf the pipe data might have shit from other processes inbetween

flint idol Oct 7, 2024, 2:54 AM

#

ah

#

so something like (when n <= PIPE_BUF)

PushlockAcquire(&pipe->lock, WRITER);
memcpy(pipe->buffer+pipe->out_off, buf, szWrite);
PushlockRelease(&pipe->lock, WRITER);```

#

assuming you have room in the pipe

#

and unatomic would mean I just copy into the thing?

#

but what if

#

there is an atomic write going on

ornate ginkgo Oct 7, 2024, 2:58 AM

#

then you wait lol

#

its atomic

flint idol Oct 7, 2024, 2:59 AM

#

so like

ornate ginkgo Oct 7, 2024, 2:59 AM

#

or you dont wait, depending on flags

flint idol Oct 7, 2024, 2:59 AM

#

I'll just figure this out tomorrow

#

even though this isn't complicated or anything

#

I'm just gonna say fuck unix

real pecan Oct 7, 2024, 6:34 AM

#

flint idol infy can you not change the uacpi log levels any time soon

I dont have any plans to do so if you're on latest master

flint idol Oct 7, 2024, 10:48 AM

#

ok thanks

vale nymph Oct 7, 2024, 10:59 AM

#

flint idol and unatomic would mean I just copy into the thing?

Unatomic just means it wont guarantee it to be in one entire chunk but instead itll possibly have the data written from other writers inbetween

#

So if you write less than pipe buf itll be guaranteed to arrive in an entire contiguous chunk in the pipe but if its above you can have some other data in the middle

#

If, say, theres no space in the pipe to complete the whole write at once

flint idol Oct 7, 2024, 11:11 AM

#

Ohhhh that makes sense

flint idol Oct 7, 2024, 9:48 PM

#

ok pipes time

#

probably

#

so I think I can set PIPE_BUF to some random number

#

#define PIPE_BUF (512 /* minimum specified by POSIX */)

#

idk what it's supposed to define

#

seems like it's only the limit to change atomic thingies or not

#

flint idol Oct 8, 2024, 10:40 PM

#

idk what's up with my kernel rn

#

but

#

when there is a partition that is not FAT

#

it just hangs while probing it

#

I also need to make my pagecache faster

flint idol Oct 9, 2024, 9:08 PM

#

I'm gonna use clang-format for my os

#

now

flint idol Oct 9, 2024, 10:50 PM

#

I have settled on using the ms style

#

from clang-formating this

#

I don't think it's a good idea to commit a 17k commit

vale nymph Oct 9, 2024, 10:59 PM

#

coward

flint idol Oct 9, 2024, 10:59 PM

#

especially with another commit in the works

#

I'll finish the other one

#

then format

#

now I would continue what I was doing before

#

but I have no idea

#

what I was actually doing

#

yeah what

#

why is it hanging

#

bro wtf

flint idol Oct 9, 2024, 11:24 PM

#

wait what

#

I need to fix a bug

#

slightly related to this

torpid wigeon Oct 10, 2024, 4:57 PM

#

flint idol I have settled on using the ms style

what's the ms style

#

like c#?

thick jolt Oct 11, 2024, 12:44 AM

#

flint idol I have settled on using the ms style

clang moment

flint idol Oct 11, 2024, 2:18 AM

#

torpid wigeon what's the ms style

idk

#

it's microsoft's code style

#

or at least how clang-format sees the ms code style

#

let's see what did I do today for the kernel

#

nothing

#

actually

#

I made it so that

#

I meant

#

I fixed the git history

#

since my commits had their name as Linus Torvalds

#

luckily for me no one but me has a local copy of obos' repo

ornate ginkgo Oct 11, 2024, 4:18 AM

#

As far as you know

opaque pulsar Oct 11, 2024, 4:53 PM

#

ultrameme

flint idol Oct 11, 2024, 9:29 PM

#

ornate ginkgo As far as you know

Well don't forget to update the repo

flint idol Oct 11, 2024, 10:06 PM

#

flint idol Well don't forget to update the repo

For all I know obos might not even work on someone else's qemu

#

Lol

flint idol Oct 11, 2024, 10:27 PM

#

flint idol I need to fix a bug

I was making pagecache for a partition (e.g., sda1) use the pagecache of its drive (e.g., sda)

#

instead of having its own

#

it did make a bug

#

when I did that

#

as a wise klange said

#

#1141057599584878645 message

flint idol Oct 11, 2024, 10:36 PM

#

flint idol it did make a bug

where bug

#

there it is

#

reading pc entry at ffffff0020b95000 for vn ffffff0000537e50```

#

and right after that

#

Page fault at 0xffffffff80007f65 in kernel-mode while to read page at 0xffffff0020b96000, which is unpresent. Error code: 0```

#

and a rebuild causes a hang

#

beautiful

#

after a bit

#

it crashes

#

why is it trying to read 9 sectors

#

page_size / sector_size != 9

#

page_size is 4096
sector_size is 512

#

obos_status status = driver->ftable.read_sync(vn->desc, (void*)addr, OBOS_PAGE_SIZE/blkSize, current_offset, nullptr);

#

the most stupid thing I have done in my kernel

#

is definitely not adding a vnode* owner to the pagecache

#

I made it have that

#

because otherwise I do questionable stuff

#

a jinn is playin' around with my computer

#

uhhh, what? (1050624 + 8) > port->nSectors```

#

printf("uhhh, what? (%d + %d) > port->nSectors", blkOffset, blkCount);

#

blkCount = (blkOffset + blkCount) - port->nSectors;

#

I might be delulu

#

if ((blkOffset + blkCount) > port->nSectors)
{
    printf("uhhh, what? (%d + %d) > port->nSectors\n", blkOffset, blkCount);
    blkCount = (blkOffset + blkCount) - port->nSectors;
}```

#

well I almost fixed that

flint idol Oct 11, 2024, 11:42 PM

#

I add some range checks

#

for block and file offsets

#

in the pagecache, vfs, and ahci driver

#

and now, file mapping has mysteriously stopped working

#

fixed

#

now time to wait

#

to see if the previous bug was fixed

#

it isn't

#

because a disk was being accessed out of range

#

and now the pc get entry function returns nullptr when that happens

#

fixed

flint idol Oct 12, 2024, 12:06 AM

#

I will finish almost everything else I need for pipes today

#

wait

#

a pipe can have multiple readers and writers, right?

#

typedef struct pipe_desc
{
    struct waitable_header wait_hdr;
    vnode *vn;
    void* buf;
    size_t pipe_size;
    uoff_t write_offset;
    uoff_t read_offset;
} pipe_desc;```

#

wouldn't that mean that *_offset need to be

#

per-each reader/writer

#

actually

#

I don't know

#

I think that

#

it's supposed to be global to the pipe

#

once read_offset == write_offset, the read would return EPIPE I think?

#

or some sorta error status saying the pipe is empty

#

and once write_offset becomes equal to the pipe size, the pipe is full?

#

I suck at ipc don't I

flint idol Oct 12, 2024, 12:13 AM

#

flint idol and once write_offset becomes equal to the pipe size, the pipe is full?

or is it just one offset

#

that gets increased on write

#

and decreased on read

#

typedef struct pipe_desc
{
    struct waitable_header wait_hdr;
    vnode *vn;
    void* buf;
    size_t pipe_size;
    uoff_t offset; // on read, subtract from offset, on write, add to offset
} pipe_desc;```

#

@vale nymph if I guarantee atomic writes no matter the size of the write in my pipe implementation, will it come back to bite me?

vale nymph Oct 12, 2024, 12:20 AM

#

flint idol <@215823874615541761> if I guarantee atomic writes no matter the size of the wri...

you are deviating from posix so likely

#

and it'll be that fun type of bug where you dont know why its happening :')

flint idol Oct 12, 2024, 12:20 AM

#

bruh ok

#

I hate u posix

#

I'll just make PIPE_BUF be UINT64_MAX

#

meme

#

I need to fix a bug with pushlocks

#

it is currently possible for there to be a reader while there is a writer

#

I think I fixed that

#

before there was a race condition

#

where when a reader lock could be acquried on the pushlock

#

while there is a writer

#

now, acquire contains this:

if (reader)
    {
        if (lock->currWriter)
        {
            lock->nWaitingReaders++;
            while(lock->currWriter);
            lock->nWaitingReaders--;
        }
        lock->nReaders++;
        return OBOS_STATUS_SUCCESS;
    }``` (forgive the formatting, copy paste is funny)

#

and if you release a writer lock:

lock->currWriter = nullptr;
return lock->nWaitingReaders ? OBOS_STATUS_SUCCESS : CoreH_SignalWaitingThreads(&lock->hdr, false, true);```

#

if there are waiting readers, then it simply continues

#

otherwise it wakes a thread that wants to be a writer

#

I will use a pushlock for 'atomic' writes

#

typedef struct pipe_desc
{
    struct waitable_header wait_hdr;
    vnode *vn;
    void* buf;
    size_t pipe_size;
    uoff_t offset;
    pushlock lock;
} pipe_desc;```

#

current structure of a

#

pipe descriptor

#

something tells me that only having one offset for each opened pipe sounds wrong

#

well

#

one way to find out is to see whether this comes back to bite me

#

static obos_status read_sync(dev_desc desc, void* buf, size_t blkCount, size_t blkOffset, size_t* nBlkRead)
{
    if (!desc)
        return OBOS_STATUS_INVALID_ARGUMENT;
    pipe_desc *pipe = (void*)desc;
    if (!pipe->offset)
        return OBOS_STATUS_EOF; // nothing to read...
    if (blkCount > pipe->pipe_size)
        blkCount = pipe->pipe_size - pipe->offset;
    Core_PushlockAcquire(&pipe->lock, true);
    memcpy(buf, (char*)pipe->buf + pipe->offset, blkCount);
    Core_PushlockRelease(&pipe->lock, true);
    pipe->offset -= blkCount;
    if (nBlkRead)
        *nBlkRead = blkCount;
    return OBOS_STATUS_SUCCESS;
}```

#

what happens

#

if a write has

#

a size that is greater than the pipe size

#

shit

#

I'm dumb

#

I think I'd need to split the writes

#

into smaller writes

#

so something like:

#

while bytes_left_to_write
  write(...)
  bytes_left_to_write - bytes_written```

#

since I made it so pipe_size has to be >= PIPE_BUF

#

writes greater than pipe size should be able to be done unatomically

#

without consequence

#

static obos_status write_sync(dev_desc desc, const void* buf, size_t blkCount, size_t blkOffset, size_t* nBlkWritten)
{
    OBOS_UNUSED(blkOffset);
    if (!desc)
        return OBOS_STATUS_INVALID_ARGUMENT;
    pipe_desc *pipe = (void*)desc;
    if (blkCount > pipe->pipe_size)
    {
        size_t written_count = 0;
        size_t curr_written_count = 0;
        off_t current_offset = 0;
        long bytes_left = blkCount;
        while (bytes_left > 0)
        {
            obos_status status = write_sync(desc, buf + current_offset, bytes_left > pipe->pipe_size ? pipe->pipe_size : bytes_left, 0, &curr_written_count);
            if (obos_is_error(status))
                return status;
            written_count += curr_written_count;
            bytes_left -= curr_written_count;
            current_offset += curr_written_count;
        }
        if (nBlkWritten)
            *nBlkWritten = written_count;
        return OBOS_STATUS_SUCCESS;
    }
    bool atomic = blkCount <= PIPE_BUF;
    Core_PushlockAcquire(&pipe->lock, !atomic /* if we want to do an unatomic write, then we can be a reader, otherwise, we must be a writer */);
    memcpy((char*)pipe->buf + pipe->offset, buf, blkCount);
    pipe->offset += blkCount;
    Core_PushlockRelease(&pipe->lock, !atomic);
    if (nBlkWritten)
        *nBlkWritten = blkCount;
    return OBOS_STATUS_SUCCESS;
}```

#

write should be done

#

I do not plan on testing this.

ornate ginkgo Oct 12, 2024, 5:35 AM

#

flint idol I do not plan on testing this.

real pecan Oct 12, 2024, 9:10 AM

#

wait

#

this shit recurses into itself?

flint idol Oct 12, 2024, 12:51 PM

#

real pecan this shit recurses into itself?

Only once

#

If it needs to do big writes

#

It splits it into smaller writes

flint idol Oct 12, 2024, 2:08 PM

#

I just need to implement create pipe (equivalent to pipe.2, but it takes an optional pipe size)

flint idol Oct 12, 2024, 2:29 PM

#

done

#

probably

#

OBOS_ASSERT(!"untested") in my pipe impl

#

for educational purposes

vale nymph Oct 12, 2024, 2:33 PM

#

https://cdn.discordapp.com/attachments/1287456798407790684/1290809675943776298/assert.gif

flint idol Oct 12, 2024, 2:35 PM

#

I will commit this code

#

and then run clang-format

#

on everything

#

just pushed pipes

#

now time to clang-format this shit

#

 215 files changed, 17962 insertions(+), 17093 deletions(-)```

#

perfect

#

still boots

#

time to commit

#

Misc: Small changes```

#

lol

sharp pike Oct 12, 2024, 2:50 PM

#

nothing biggie

flint idol Oct 12, 2024, 2:51 PM

#

it even formatted r4's m68k loader thing

#

the limine stub

#

but I think I will undo that

#

I don't know why, but my m68k entry code

#

is 100x cleaner than the x86_64 entry code

#

350 lines vs 1000 lines

#

I will actually undo that commit

#

and format everything once I merge this branch with master

#

that way I don't get as much merge conflicts

flint idol Oct 12, 2024, 3:16 PM

#

before I start making syscalls

#

I want to make some final changes to the mm

#

to make entering user space possible

#

or rather, starting user programs possible

#

basicalyl all it is

#

is create_context

#

void Mm_ConstructContext(context* ctx); // userspace-only
// Allocates a page table. Must have enough of the kernel mapped for a userspace -> kernel mode switch to work.
page_table MmS_AllocatePageTable();

#

void Mm_ConstructContext(context* ctx)
{
    OBOS_ASSERT(ctx);
    memzero(ctx, sizeof(*ctx));
    ctx->pt = MmS_AllocatePageTable();
    ctx->lock = Core_SpinlockCreate();
}```

#

simple enough

#

the hard part lies in allocate pt

#

I only need two symbols to be there to allow a user->kernel transition to happen

#

so those two symbols will be the onyl ones allocated

#

actually

#

it's more like an entire file

#

which is isr.asm

#

MmS_AllocatePageTable turned out to be pretty simple

#

I do need to map the kernel stacks in the tss tho

#

my syscalls will run at IRQL_DISPATCH

#

ok, I have implemented that

#

time to see if I can start a userspace-thread

#

that just jmp $s

#

three things can happen:

it triple faults
it page faults
it hangs as intended

#

    context* new_ctx = Mm_Allocator->ZeroAllocate(Mm_Allocator, 1, sizeof(context), nullptr);
    Mm_ConstructContext(new_ctx);
    void* mem = Mm_VirtualMemoryAlloc(new_ctx, nullptr, 0x1000, OBOS_PROTECTION_EXECUTABLE, VMA_FLAGS_NON_PAGED, nullptr, nullptr);
    uintptr_t mem_phys = 0;
    MmS_QueryPageInfo(new_ctx->pt, (uintptr_t)mem, nullptr, &mem_phys);
    char* mem_kern = MmS_MapVirtFromPhys(mem_phys);
    // jmp $
    mem_kern[0] = 0xeb;
    mem_kern[1] = 0xfe;
    thread* thr = CoreH_ThreadAllocate(nullptr);
    thread_ctx thr_ctx = {};
    void* stack =  Mm_VirtualMemoryAlloc(new_ctx, nullptr, 0x4000, OBOS_PROTECTION_EXECUTABLE, VMA_FLAGS_NON_PAGED|VMA_FLAGS_GUARD_PAGE, nullptr, nullptr);
    CoreS_SetupThreadContext(&thr_ctx, (uintptr_t)mem, 0, true, stack, 0x4000);
    CoreH_ThreadInitialize(thr, THREAD_PRIORITY_NORMAL, Core_DefaultThreadAffinity, &thr_ctx);
    process* new = Core_ProcessAllocate(nullptr);
    new->ctx = new_ctx;
    Core_ProcessStart(new, thr);```

flint idol Oct 12, 2024, 4:09 PM

#

amazing

#

it triple faulted

flint idol Oct 12, 2024, 4:10 PM

#

flint idol ```c context* new_ctx = Mm_Allocator->ZeroAllocate(Mm_Allocator, 1, sizeof(c...

perhaps it would be helpful to add OBOS_PROTECTION_USER_PAGE to the protection of the memory

#

oops

#

I forgor to put switch to thread context in the user cr3

#

now I only have one teensy weensy problem (it's a big problem)

#

how do I access a thread's context

#

in CoreS_SwitchToThreadContext

#

after switching to the user's cr3

#

what I could do is copy the context to the stack which is an ist stack (and therefore, mapped)

#

then use that

#

what I'll do is

#

before switching cr3

#

I'll copy whatever is left of the thread_ctx struct to the stack

#

then switch rdi to rsp

#

mov rdi, rsp

#

since it switches to the context pointed to by rdi

flint idol Oct 12, 2024, 4:24 PM

#

flint idol I'll copy whatever is left of the thread_ctx struct to the stack

but maybe it'd be better to only do this if the new cr3 != the kernel cr3

#

to speed stuff up

flint idol Oct 12, 2024, 5:04 PM

#

it almost gets to user mode execution

#

in fact, it triple faults in user-code

#

I have a 1.8M qemu log lmfao

flint idol Oct 12, 2024, 5:08 PM

#

flint idol in fact, it triple faults in user-code

nvm, it triple faults in iretq

#

@steep ravine why do you triple fault?

#

 82968: v=0e e=0000 i=0 cpl=0 IP=0008:ffffffff8000dab3 pc=ffffffff8000dab3 SP=0010:ffffffff802a6f28 CR2=ffffffff80262f18
RAX=0000000000000000 RBX=0000000000000000 RCX=0000000000000000 RDX=0000000000000000
RSI=0000000000000000 RDI=0000000000000000 RBP=0000000000000000 RSP=ffffffff802a6f28
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff8000dab3 RFL=00200086 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 ffffffff 00af9b00 DPL=0 CS64 [-RA]
SS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 ffffffff80262b20 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0028 ffffffff80262f30 00000070 00408900 DPL=0 TSS64-avl
GDT=     ffffffff80262ef8 00000037
IDT=     ffffffff8025d900 00000fff
CR0=80010011 CR2=ffffffff80262f18 CR3=000000003d090000 CR4=00000020
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000018 CCD=ffffffff802a6f28 CCO=ADDQ
EFER=0000000000000d00```

#

w h a t

#

what is this PFing on

#

gdt isn't mapped probably

#

fixed that

#

and now it seems like my gdt is broken

#

as I get a GPF

#

info->arch_specific.gdtEntries[0] = 0;
info->arch_specific.gdtEntries[1] = 0x00af9b000000ffff; // 64-bit code
info->arch_specific.gdtEntries[2] = 0x00cf93000000ffff; // 64-bit data
info->arch_specific.gdtEntries[3] = 0x00aff3000000ffff; // 64-bit user-mode data
info->arch_specific.gdtEntries[4] = 0x00cff3000000ffff; // 64-bit user-mode code```

#

that's my gdt

#

and it gpf's on 0x20 which is user code

#

I seem to have switched around user code and data

#

check_exception old: 0xffffffff new 0xd
 83278: v=0d e=0020 i=0 cpl=0 IP=0008:ffffffff8000dab3 pc=ffffffff8000dab3 SP=0010:ffffffff802a6f28 env->regs[R_EAX]=0000000000000000
RAX=0000000000000000 RBX=0000000000000000 RCX=0000000000000000 RDX=0000000000000000
RSI=0000000000000000 RDI=0000000000000000 RBP=0000000000000000 RSP=ffffffff802a6f28
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff8000dab3 RFL=00200086 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 ffffffff 00af9b00 DPL=0 CS64 [-RA]
SS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 ffffffff80262b20 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0028 ffffffff80262f30 00000070 00408900 DPL=0 TSS64-avl
GDT=     ffffffff80262ef8 00000037
IDT=     ffffffff8025d900 00000fff
CR0=80010011 CR2=ffffff00007d4df0 CR3=000000003d095000 CR4=00000020
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000018 CCD=ffffffff802a6f28 CCO=ADDQ
EFER=0000000000000d00```

#

it finally gets into user code

#

well it triple faults immediately

#

check_exception old: 0xffffffff new 0xe
 81919: v=0e e=0000 i=0 cpl=3 IP=0023:0000000000001000 pc=0000000000001000 SP=001b:0000000000007000 CR2=ffffffff8025db00
RAX=0000000000000000 RBX=0000000000000000 RCX=0000000000000000 RDX=0000000000000000
RSI=0000000000000000 RDI=0000000000000000 RBP=0000000000000000 RSP=0000000000007000
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=0000000000001000 RFL=00200202 [-------] CPL=3 II=0 A20=1 SMM=0 HLT=0
ES =0000 0000000000000000 ffffffff 00cf1300
CS =0023 0000000000000000 ffffffff 00affb00 DPL=3 CS64 [-RA]
SS =001b 0000000000000000 ffffffff 00cff300 DPL=3 DS   [-WA]
DS =0000 0000000000000000 ffffffff 00cf1300
FS =0000 0000000000000000 ffffffff 00cf1300
GS =0000 ffffffff80262b20 ffffffff 00cf1300
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0028 ffffffff80262f30 00000070 00408900 DPL=0 TSS64-avl
GDT=     ffffffff80262ef8 00000037
IDT=     ffffffff8025d900 00000fff
CR0=80010011 CR2=ffffffff8025db00 CR3=000000003d095000 CR4=00000020
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000000 CCD=ffffffff802a6f28 CCO=EFLAGS
EFER=0000000000000d00```

#

I also need to map the idt

#

forogr about that\

#

check_exception old: 0xffffffff new 0xd
 86482: v=0d e=0008 i=0 cpl=0 IP=0008:ffffffff8000805f pc=ffffffff8000805f SP=0010:ffffffff8035afd0 env->regs[R_EAX]=0000000000000000
RAX=0000000000000000 RBX=ffffffff80263648 RCX=0000000000000000 RDX=0000000000000001
RSI=000000000000001a RDI=ffffffff802639d2 RBP=ffffffff8035aff8 RSP=ffffffff8035afd0
R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000
R12=0000000000000000 R13=0000000000000000 R14=0000000000000000 R15=0000000000000000
RIP=ffffffff8000805f RFL=00200297 [--S-APC] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
CS =0008 0000000000000000 ffffffff 00af9b00 DPL=0 CS64 [-RA]
SS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
DS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
FS =0010 0000000000000000 ffffffff 00cf9300 DPL=0 DS   [-WA]
GS =0010 ffffffff80263648 ffffffff 00cf9300 DPL=0 DS   [-WA]
LDT=0000 0000000000000000 0000ffff 00008200 DPL=0 LDT
TR =0028 ffffffff802636a0 00000070 00408900 DPL=0 TSS64-avl
GDT=     ffffffff80263668 00000037
IDT=     ffffffff8025d900 00000fff
CR0=80010011 CR2=0000000000000000 CR3=0000000000006000 CR4=00000020
DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 
DR6=00000000ffff0ff0 DR7=0000000000000400
CCS=0000000000000095 CCD=ffffffffffffffff CCO=EFLAGS
EFER=0000000000000d00```

#

mapped the idt

#

and now it GPFs

#

in the idle task

#

uhhh

#

wtf

#

the gdt is fucked

lean glen Oct 12, 2024, 5:37 PM

#

e=0008

#

it faults on your gdt

flint idol Oct 12, 2024, 5:37 PM

#

I know

#

gdt is fucked

#

so is the entrie cpu local struct?

#

*entire

real pecan Oct 12, 2024, 6:17 PM

#

what

vale nymph Oct 12, 2024, 6:42 PM

#

rust rewrite when

thick jolt Oct 12, 2024, 6:59 PM

#

^^

#

meme

#

halfmemeright

#

dont rewrite in rust

#

seriously tho

#

dont.

#

if u dont know rust

#

do not

#

and thats coming from someone who still has a LOT to learn

#

shy forge Oct 12, 2024, 7:10 PM

#

thick jolt if u dont know rust

Nah fuck it

#

Do whatever you want

#

try and fail hard

flint idol Oct 12, 2024, 7:11 PM

#

it's been in C for like

#

5-6 months

#

also because I'd rather do win32 api dev using assembly than use C++ for a kernel again

lean glen Oct 12, 2024, 7:12 PM

#

C++ is nice

thick jolt Oct 12, 2024, 7:31 PM

#

shy forge Nah fuck it

bro doesnt know

#

bro

#

go to #1230349543623757845

#

see the past

#

fucking month

#

actually

#

3 months

#

💀

shy forge Oct 12, 2024, 7:32 PM

#

Brother I'm telling you failing hard ain't a bad thing unless you're like on a time or money budget

#

Life's a learning experience bro

#

I learnt C through osdev

#

My first malloc was literally char arr[size]; return arr;

flint idol Oct 12, 2024, 10:05 PM

#

broooooooooo

#

where tf is my cpu_local struct corrupted beyond recognition

flint idol Oct 12, 2024, 10:06 PM

#

real pecan what

dw, I didn't rewrite

#

I think his message lagged

#

since it's been

#

5

#

months

#

since I started this rewrite

#

so one CPU's gs_base is zero (swapgs memes?)

#

another CPU's cpu local struct might be corrupted

#

*is

#

I see a problem

#

gs base is set to some random cpu's gs_base in the thread context

#

and is restored

#

while on another cpu

#

and kernel gs base is also zero

#

so what I should do on switch to a userspace context

#

is swapgs

#

first

#

then restore the thread's gs_base

#

and now it doesn't crash!

#

who'd have thought it'd take all day to get to usermode

#

[  LOG  ] User thread 10 SIGSEGV```

#

when I add

mov rax, 0xdeadbeef
mov qword ptr [rax], 5```

#

!!!!

#

but then ofc, since signals are completely untested until now

#

it crashes

#

#

oopsie

vale nymph Oct 12, 2024, 10:34 PM

#

0xdeadc0fee

#

or coffe

#

i dont speak engrish

flint idol Oct 12, 2024, 10:36 PM

#

vale nymph 0xdeadc0fee

dead beef

#

wait

#

what

#

signals

#

worked first try

#

this cannot be

#

now I rather implemented a way to kill arbitrary threads

#

or I did not

#

idr

#

I did not make such a thing

flint idol Oct 12, 2024, 11:13 PM

#

flint idol now I rather implemented a way to kill arbitrary threads

actually who needed that

#

for signals who's default action is to kill the thread

#

I'll just exit thread in the default signal handler

#

as for SIGKILL

#

I'll always force the default signal handler

#

hmm

#

what's "with additional actions"

#

implementation-defined abnormal termination actions, such as creation of a core file, may also occur.

#

in my case

#

idgaf about core files

vale nymph Oct 12, 2024, 11:26 PM

#

dont say that about core files you'll make them sad

real pecan Oct 12, 2024, 11:33 PM

#

flint idol dw, I didn't rewrite

didnt know there was a c++ obos

flint idol Oct 13, 2024, 12:02 AM

#

real pecan didnt know there was a c++ obos

rewrite 2,3, and 4 were all C++

#

anyway

#

after charging myself with noodles

#

and burning my tongue in the process

#

I will finalize my signal implementation

real pecan Oct 13, 2024, 12:05 AM

#

flint idol rewrite 2,3, and 4 were all C++

Oh

real pecan Oct 13, 2024, 12:05 AM

#

flint idol after charging myself with noodles

Were they good noodles at least?

flint idol Oct 13, 2024, 12:05 AM

#

yes

real pecan Oct 13, 2024, 12:06 AM

#

Nice

flint idol Oct 13, 2024, 12:22 AM

#

signal default actions now work

#

which means my signal implementation should be complete

#

uhhh

#

that can't be good

#

vale nymph Oct 13, 2024, 12:23 AM

#

I said the same thing months ago

#

https://tenor.com/view/взгляд-2000-ярдов-война-war-soldier-gif-3632617944134077161

Tenor

flint idol Oct 13, 2024, 12:24 AM

#

O_O

vale nymph Oct 13, 2024, 12:24 AM

#

then I saw I was doing like 5 things wrong and forgot to implement some other things

flint idol Oct 13, 2024, 12:24 AM

#

until then

#

I will be committing this

flint idol Oct 13, 2024, 1:13 AM

#

syscall interface

#

how shall I abstract syscalls....

#

since a lot of syscalls are the same across all archs

#

I think I should reserve many syscalls for arch-agnostic stuff

#

and the rest are arch-specific

flint idol Oct 13, 2024, 1:14 AM

#

flint idol I think I should reserve many syscalls for arch-agnostic stuff

like syscalls from 0-0x8000'0000 will be syscalls that are arch-independant

#

and the rest are arch-specific

flint idol Oct 13, 2024, 1:15 AM

#

flint idol like syscalls from 0-0x8000'0000 will be syscalls that are arch-independant

but, instead of having a table that big

#

uintptr_t syscalls[UINT32_MAX]; // obviously a bad idea```

flint idol Oct 13, 2024, 1:15 AM

#

flint idol like syscalls from 0-0x8000'0000 will be syscalls that are arch-independant

actually

#

make this 0-0x10000

#

nvm

flint idol Oct 13, 2024, 1:17 AM

#

flint idol but, instead of having a table that big

I'll have something like

// increase when needed
#define ARCH_SYSCALL_TABLE_SIZE 0x10000
uintptr_t arch_syscall_table[ARCH_SYSCALL_TABLE_SIZE];
#define SYSCALL_TABLE_SIZE 0x10000
uintptr_t syscall_table[SYSCALL_TABLE_SIZE];
#define get_syscall_from_num(n) \
  (((n) >= 0x80000000) ? \
    arch_syscall_table[n-0x80000000] : \
    syscall_table[n])

#

then the arch's syscall trap handler will convert the syscall abi to the kernel abi (whatever that may be)

#

I think on x86-64, the syscall abi will be equivalent to the systemv abi (in terms of calling convention), except with eax being the syscall number

#

to make it easy to abstract it

#

because speed

#

I will implement the syscall instruction

#

and have no other trap method

#

bro

#

why did I make my syscall init function

#

in ASSEMBLY

#

in my 3rd kernel

flint idol Oct 13, 2024, 1:55 AM

#

flint idol I think on x86-64, the syscall abi will be equivalent to the systemv abi (in ter...

and r10 is guaranteed to be clobberred

#

SIGILL thinkong

#

when testing syscalls

#

fixing the SIGILL causes a triple fault

flint idol Oct 13, 2024, 2:41 AM

#

syscalls almost work

#

returning causes a SIGSEGV

#

idk if that's the syscall's fault

#

[ DEBUG ] hai```

#

syscall zero is currently a test syscall

#

that prints 'hai' and returns

#

I made it take a parameter

#

fixed a bug

#

and now it takes in a string from usermode and prints that

#

ORG 0x1000
mov eax, 0
mov rdi, 0x1100
syscall
jmp $
ORG 0x1100
db "hai from usermode", 0

#

that is the current code

#

tomorrow I will implement syscalls for everything

#

meaning

#

vfs, vmm, scheduler, sync primitives, ipc, and driver interface syscalls

#

then I will implement fork()

#

in the vmm

#

and expose that

#

then I'll make sure to get this all working on the m68k port

#

and finally merge userspace-work

#

then I port stuff

#

I just committed that

#

code

#

that's enough osdev for the day

#

I'm headed to sleep

#

wake up
shit
get out of bed
osdev
sleep (optional)

flint idol Oct 13, 2024, 12:09 PM

#

time to implement a bunch of syscalls

#

I feel like it would be a good idea to add a context argument to read/write of drivers

#

since usermode reads and writes from drivers are possible

#

I will also implement locking pages into memory

#

I'll start with page locking

flint idol Oct 13, 2024, 12:19 PM

#

flint idol I feel like it would be a good idea to add a context argument to read/write of d...

then this

#

actually, that's unneeded

#

if I just use user memcpy

#

along with currentContext

#

but there is one thing I must do

#

that is, when entering an interrupt from kernel mode

#

I need to switch the current context to the kernel ctx

#

I think I just found the longest single instruction in my kernel

#

of 13 bytes

#

flint idol Oct 13, 2024, 1:09 PM

#

my vmm is ~3K loc

#

which ig is cool

#

my kernel is 27k loc

#

without drivers

#

my kernel in total is 33.3K loc

#

including drivers

flint idol Oct 13, 2024, 1:28 PM

#

anyway, time to start making some syscalls

#

I will start with scheduling related syscalls

#

yield and exitcurrentthread

#

I'll implement a handle system after those two syscalls

#

should I sync pending signals on return from syscall

#

I think so

#

because it is a return from kernel->user

#

I'd need some funny logic

flint idol Oct 13, 2024, 1:47 PM

#

flint idol I'd need some funny logic

which I don't want to implement

#

I think it's sufficient

#

to just sync signals on return to usermode in an irq

flint idol Oct 13, 2024, 2:52 PM

#

ok there is a bug

#

user stacks cannot be swapped out

#

without it breaking

#

specifically

#

it page faults twice

#

then recursively locks

#

it's a bug in cow

#

memcpy(MmS_MapVirtFromPhys(new->phys), (void*)addr, info->prot.huge_page ? OBOS_HUGE_PAGE_SIZE : OBOS_PAGE_SIZE);

#

specifically here

#

fixed

vale nymph Oct 13, 2024, 3:02 PM

#

flint idol ``` memcpy(MmS_MapVirtFromPhys(new->phys), (void*)addr, info->prot.huge_page ? O...

does MmS_MapVirtFromPhys just return an address in the hhdm or does it actually allocate a virtual address

flint idol Oct 13, 2024, 3:02 PM

#

arch-specific

vale nymph Oct 13, 2024, 3:03 PM

#

would that not leak memory then

flint idol Oct 13, 2024, 3:03 PM

#

but on x86-64 it returns an hhdm address

flint idol Oct 13, 2024, 3:03 PM

#

vale nymph would that not leak memory then

until it actually allocates a virtual address

vale nymph Oct 13, 2024, 3:04 PM

#

sounds leaky to map a virtual address and not unmap it later lol but sure

flint idol Oct 13, 2024, 3:04 PM

#

well it doesn't

#

in any implementation

#

yet

#

and it's used very rarely

#

meaning 30 times

lean glen Oct 13, 2024, 3:10 PM

#

Chances are you're going to identity map everything anyway

#

So don't do any allocation in that function

flint idol Oct 13, 2024, 4:17 PM

#

I need a logo for obos

#

but I don't have any sort of artistic skills or an ai to do it for me

thick jolt Oct 13, 2024, 5:38 PM

#

shy forge I learnt C through osdev

💀

thick jolt Oct 13, 2024, 5:39 PM

#

flint idol but I don't have any sort of artistic skills or an ai to do it for me

i will make one for u

#

@flint idol

#

meme

#

oberrow use my logo

#

its the best logo

flint idol Oct 13, 2024, 5:49 PM

#

thick jolt <@841762704766140486>

amazin

#

g

thick jolt Oct 13, 2024, 5:49 PM

#

flint idol amazin

use it

#

meme

flint idol Oct 13, 2024, 5:50 PM

#

hold on I just need to crop it a bit

#

#

amazing

#

bruh fuck gimp

thick jolt Oct 13, 2024, 6:01 PM

#

amazing indeed

flint idol Oct 13, 2024, 8:19 PM

#

    mov rax, [rdi]
    mov cr8, rax
    push rdi
    call Core_GetIRQLVar
    pop rdi
    mov rcx, [rdi]
    mov [rax], rcx ; crashes here becuz nullptr
    add rdi, 8```

#

and I have no idea how it crashes

#

for reference

#

this is in CoreS_SwitchToThreadContext

#

hold on wtf

#

Kernel Panic on CPU 1 in thread 10, owned by process 2. Reason: OBOS_PANIC_EXCEPTION. Information on the crash is below:
Page fault at 0xffffffff8000dd88 in kernel-mode while to write page at 0x0000000000000000, which is unpresent. Error code: 2
Register dump:
        RDI: 0xffffff0000023ea0, RSI: 0x0000000000000000, RBP: 0xffffffff80328ff0
        RSP: 0xffffffff80328f68, RBX: 0xffffffff80028a92, RDX: 0x0000000000000000
        RCX: 0x0000000000000000, RAX: 0x0000000000000000, RIP: 0xffffffff8000dd88
         R8: 0x0000000000000000,  R9: 0x0000000000000000, R10: 0x27235311250fd7ed
        R11: 0x0000000400000000, R12: 0x0000000000000000, R13: 0x0000000000000000
        R14: 0x0000000000000000, R15: 0x0000000000000000, RFL: 0x0000000000210282
         SS: 0x0000000000000010,  DS: 0x0000000000000000,  CS: 0x0000000000000008
        CR0: 0x0000000080010011, CR2: 0x0000000000000000, CR3: 0x0000000000006000
        CR4: 0x0000000000000020, CR8: 0x0000000000000000, EFER: 0x0000000000000d01```

#

this function clis at the very beginning

#

and rflags.if=1

#

fuck

#

may god have mercy on me.

#

this is stack corruption at the best

flint idol Oct 13, 2024, 8:52 PM

#

#

hmm

#

it seems to have returned from a page fault handler

flint idol Oct 13, 2024, 8:53 PM

#

flint idol ```x86asm mov rax, [rdi] mov cr8, rax push rdi call Core_GetIRQL...

to here

#

#

looks kinda weird

#

#

better

flint idol Oct 13, 2024, 9:39 PM

#

after procrastination through football clips

#

I will continue debugging

flint idol Oct 13, 2024, 10:48 PM

#

I added the syscalls Sys_Rebootand Sys_Shutdown

#

that reboot and shutdown (surprising right?)

#

and it doesn't break

#

it seems to be a bug with yield

#

since

#

when I replace the yield syscall

#

with one that does nothing

#

it doesn't crash

#

mov rdi, Core_Schedule
xor rsi,rsi
call CoreS_CallFunctionOnStack```

#

in yield

#

it uses the same stack

#

that the syscall handler uses

#

CoreS_CallFunctionOnStack this function

flint idol Oct 13, 2024, 11:26 PM

#

fixed the bug

#

this PR is at

#

changes

#

I need to merge this soon

real pecan Oct 13, 2024, 11:30 PM

#

flint idol I added the syscalls `Sys_Reboot`and `Sys_Shutdown`

letsgo

#

time for real hw tests

flint idol Oct 13, 2024, 11:30 PM

#

forgot about those

#

will do

#

just need to remove the ahci driver from the initrd, it's problematic

real pecan Oct 13, 2024, 11:31 PM

#

lol

flint idol Oct 13, 2024, 11:36 PM

#

It works on test subject 1

real pecan Oct 13, 2024, 11:47 PM

#

Which one did astral hang on

flint idol Oct 13, 2024, 11:47 PM

#

#1

real pecan Oct 13, 2024, 11:47 PM

#

Ah

flint idol Oct 13, 2024, 11:47 PM

#

Which reminds me to test it on the other one

#

Which does not have native ps2

real pecan Oct 13, 2024, 11:47 PM

#

Is it emulation

flint idol Oct 13, 2024, 11:48 PM

#

It's the one with the dumb x component in the video thing

flint idol Oct 13, 2024, 11:48 PM

#

real pecan Is it emulation

Yes

#

I will test a laptop I have with native ps2 soon

real pecan Oct 13, 2024, 11:48 PM

#

Ooh that one

flint idol Oct 13, 2024, 11:48 PM

#

AMD retarbios

infy, 2024

#

AMD ATOMBIOS

#

#

This is obos running on test subject 1

#

The test program does sys shutdown

real pecan Oct 13, 2024, 11:51 PM

#

Nice

#

Does that take so long u manage to take a photo of it?

flint idol Oct 13, 2024, 11:52 PM

#

real pecan Does that take so long u manage to take a photo of it?

I was able to take a photo within a split second

#

Before it shutdown

#

It takes like 2 or 3 seconds to shutdown on this hw

#

Quite dumb

real pecan Oct 13, 2024, 11:52 PM

#

I see

flint idol Oct 13, 2024, 11:53 PM

#

Test subject 2 seems to hang in the uart driver

#

Bruh

#

Idk how I messed that up

real pecan Oct 13, 2024, 11:53 PM

#

Lmao

#

Out of all things u fucked up the serial driver

flint idol Oct 13, 2024, 11:54 PM

#

Will be rebooting without com

flint idol Oct 13, 2024, 11:54 PM

#

real pecan Out of all things u fucked up the serial driver

😭

#

Upon disabling the serial driver

#

It works

#

#

Credits to @vale nymph for the background color

flint idol Oct 14, 2024, 1:29 AM

#

ok

#

handle interface

#

basically

#

a handle is an opaque integer

#

user-facing

#

that represents a certain kernel object

#

threads, timers, etc.

#

in the kernel

#

they are keys into a hashmap in the process structure

#

enum handle_type
{
    // ...
};
struct handle_desc
{
    enum handle_type type;
    union {
        // ...
    } un;
};
typedef uint64_t handle;```

#

the actual integer type of handle is undecided

#

rather int to stay compliant-ish with unix FDs

#

or uint64_t to give more handle values (you never know when your handles will go over INT_MAX)

#

I mean it's easily changeable, especially considering nothing in userspace would be using the handle type directly but my mlibc sysdeps

#

however I think I will go with int

#

or unsigned int

#

typedef unsigned int handle; // we use unsigned int instead of uint64_t to stay compliant-ish with unix```

#

allocating handles will be shrimple

handle_freelist freelist;
handle last_handle; // when the free list is empty.```

#

someone had a good idea

#

and I will be taking it since it sounds like a good idea

#

flint idol Oct 14, 2024, 1:43 AM

#

flint idol

this is how my handle will be laid out

lean glen Oct 14, 2024, 1:45 AM

#

Hashmap might suck here

#

Just use an array

flint idol Oct 14, 2024, 1:46 AM

#

but what if the handle value gets really high

#

then the array would have to be that big

#

and like

#

that's bad

lean glen Oct 14, 2024, 1:46 AM

#

that's why you have a limit

flint idol Oct 14, 2024, 1:47 AM

#

if I limit handle values to UINT16_MAX it takes 1M of memory

#

for the entire handle table

flint idol Oct 14, 2024, 1:48 AM

#

flint idol if I limit handle values to UINT16_MAX it takes 1M of memory

if I did my math correct

#

I can split that in half if I remove the handle type

#

from handle_desc

lean glen Oct 14, 2024, 1:48 AM

#

That's a shitton of handles

flint idol Oct 14, 2024, 1:48 AM

#

well yes

#

idk how many handles are usually in use at a time

#

for like bash for example

lean glen Oct 14, 2024, 1:49 AM

#

Less than 65535

lean glen Oct 14, 2024, 1:49 AM

#

flint idol for like bash for example

I have an arbitrary limit of 64

#

And it runs

flint idol Oct 14, 2024, 1:50 AM

#

but handles are like each kernel object for me which includes mutexes

#

meh whatever

lean glen Oct 14, 2024, 1:50 AM

#

Why tho

flint idol Oct 14, 2024, 1:50 AM

#

I'll increase it when I need to

flint idol Oct 14, 2024, 1:50 AM

#

lean glen Why tho

because how else

#

would I represent a mutex in userspace

#

for example

lean glen Oct 14, 2024, 1:51 AM

#

Are mutexes even implemented by the kernel

vale nymph Oct 14, 2024, 1:51 AM

#

no

#

not on linux at least

#

and on mlibc

flint idol Oct 14, 2024, 1:51 AM

#

well frick you, I implement them in the kernel

lean glen Oct 14, 2024, 1:51 AM

#

yeah

#

Futexes are tho I think?

flint idol Oct 14, 2024, 1:52 AM

#

well

#

so I don't need mutex syscalls

#

probably

#

I mean it could be useful for any native programs I make

lean glen Oct 14, 2024, 1:53 AM

#

I don't think you understand why you want handles

flint idol Oct 14, 2024, 1:53 AM

#

to represent a kernel object in usermode, no?

#

like a file

#

or thread

#

or any sort of kernel object that the kernel wants to expose to userspace through syscalls

lean glen Oct 14, 2024, 1:54 AM

#

Yeah I guess

flint idol Oct 14, 2024, 1:55 AM

#

handle Sys_OpenFile(const char* file, int open_mode);
obos_status Sys_ReadFile(handle file, char* buf, size_t sz, size_t* nRead);

#

for example

lean glen Oct 14, 2024, 2:04 AM

#

Yeah NT seems to use an array

#

For handles

flint idol Oct 14, 2024, 2:04 AM

#

how big is it?

#

I'll just search it

#

for some reason chrome takes a minute to start

lean glen Oct 14, 2024, 2:05 AM

#

This is according to the design workbook btw so it might not be true nowadays

flint idol Oct 14, 2024, 2:05 AM

#

I have windows internals downloaded

lean glen Oct 14, 2024, 2:05 AM

#

Yeah use that

flint idol Oct 14, 2024, 2:06 AM

#

damn

#

only have part 1

#

Objects and handles
(references to instances of an object) are discussed in more detail in Chapter 8 in Part 2.

#

I'll just ask the NT nerds

flint idol Oct 14, 2024, 2:31 AM

#

typedef struct handle_table {
    handle_desc* arr;
    handle_desc* head;
    handle last_handle;
    size_t size;
    pushlock lock;
} handle_table;```

#

that'll be my handle table

#

a dynamically expanding array

flint idol Oct 14, 2024, 3:11 AM

#

obos with optimizations is very fast

#

it take <2s to boot

#

1s

#

while (Arch_LAPICAddress->interruptCommand0_31 & (1 << 12))
    pause();

#

according to qemu-prof.sh

#

my kernel spends many time in here

#

while (invlpg_ipi_packet.active && ++spin <= 10000)
    pause();```

#

as well as here

#

unsurprisingly it spends a lot of time

#

in here

    for (basicalloc_region* r = start; r; )
    {
        if (!r->free.nNodes || !r->biggestFreeNode)
            goto end;
        if (r->biggestFreeNode->size >= size)
        {
            from = r;
            break;
        }

    end:
        r = r->next;
    }```

#

(TODO: Write a better allocator)

flint idol Oct 14, 2024, 3:35 AM

#

flint idol ```c typedef struct handle_table { handle_desc* arr; handle_desc* head; ...

this will actually be the first time I use a singly linked list

#OBOS (not vibecoded)

O_O