OBOS (not vibecoded) | Operating System Development | Page 2

flint idol Dec 25, 2023, 9:24 PM

#

I tried that, it fails to keep the thread from being run on multiple cores

#

The only think I could think of was affinity

pastel sparrow Dec 25, 2023, 9:36 PM

#

flint idol I tried that, it fails to keep the thread from being run on multiple cores

well you can have two approach combined

#

if you want to run on multiple cores you want some type of atomicity

flint idol Dec 25, 2023, 9:37 PM

#

I still use it

flint idol Dec 25, 2023, 9:37 PM

#

pastel sparrow if you want to run on multiple cores you want some type of atomicity

The same thread running on different cores at the same time

#

I don't want that

#

I'm trying the affinity approach to see if it does any good

pastel sparrow Dec 25, 2023, 9:38 PM

#

no

#

i'm not saying that

#

actually atomicity disallows that type of bugs to happen

#

if you are changing thing on memory from different cores at the same time you will have bugs if they are not handled correctly

flint idol Dec 25, 2023, 9:46 PM

#

I have a scheduler lock don't worry. The only way it could be broken is if my mutex implementation doesn't work

pastel sparrow Dec 25, 2023, 10:37 PM

#

if you lock the thread then you won't have a problem

flint idol Dec 26, 2023, 12:00 AM

#

pastel sparrow if you lock the thread then you won't have a problem

the thing is, I basically do, if the thread's status is not THREAD_STATUS_CAN_RUN, it can't be running, basically meaning it's locked

#

All operations on the thread object are done by the thread itself

#

The only thing I can think is the block callback changing while it's being called by another cpu

#

But that shouldn't be possible, as the scheduler only calls the block callback when thread->status & THREAD_STATUS_BLOCKED, and the block callback is set before the transition from thread->status=THREAD_STATUS_RUNNING to thread->status=THREAD_STATUS_CAN_RUN | THREAD_STATUS_BLOCKED

#

The block callback is something that threads set when they want to block themselves. It returns a boolean saying whether to continue running the thread

#

Anyway, with the affinity approach, I seem to have fixed the problem

#

I still have a problem with stack corruption in callBlockCallbacksOnList

#

That's likely because the kernel changes to the cpu's temporary stack before calling the scheduler, and then while calling a block callback, it uses that same temporary stack

#

My scheduler lock is broken

#

That probably explains a lot of problems

#

This is my mutex lock code:
c++ bool Mutex::Lock(uint64_t timeout, bool block) { if (!block && Locked()) { SetLastError(OBOS_ERROR_MUTEX_LOCKED); return false; } if (thread::g_initialized && m_canUseMultitasking) if (m_ownerThread == thread::GetCurrentCpuLocalPtr()->currentThread) return true; // Compare m_locked with zero, and if it is zero, then set it to true and return true, otherwise return false and keep m_locked intact. uint64_t wakeupTime = thread::g_timerTicks + timeout; if (timeout == 0) wakeupTime = 0xffffffffffffffff /* UINT64_MAX */; while ((atomic_cmpxchg(&m_locked, false, true) || m_wake) && thread::g_timerTicks < wakeupTime); if (thread::g_timerTicks >= wakeupTime) { SetLastError(OBOS_ERROR_TIMEOUT); return false; } if (thread::g_initialized && m_canUseMultitasking) m_ownerThread = (thread::Thread*)thread::GetCurrentCpuLocalPtr()->currentThread; return true; }

weary hound Dec 26, 2023, 12:17 AM

#

why is your code bold

flint idol Dec 26, 2023, 12:17 AM

#

atomic_cmpxchg:

atomic_cmpxchg:
    mov al, sil
    lock cmpxchg byte [rdi], dl
    setz al
    ret```

flint idol Dec 26, 2023, 12:18 AM

#

weary hound why is your code bold

I surrounded it with **

weary hound Dec 26, 2023, 12:18 AM

#

okay, question still remains

flint idol Dec 26, 2023, 12:18 AM

#

I thought it would look good

weary hound Dec 26, 2023, 12:18 AM

#

fair

#

also why not use the __atomic_compare_exchange builtin

flint idol Dec 26, 2023, 12:20 AM

#

Can I use it without including stdatomic?

#

nvm

#

it's builtin, so that wouldn't make sense

flint idol Dec 26, 2023, 12:25 AM

#

weary hound also why not use the `__atomic_compare_exchange` builtin

my function's easier to use, and does the same thing

#

(it's only easier because I made it)

weary hound Dec 26, 2023, 12:26 AM

#

easier how

flint idol Dec 26, 2023, 12:26 AM

#

I understand it better

#

and they are the same code

#

or rather, the compiler emits the same code with the builtin

#

https://godbolt.org/z/83jYs488G

Compiler Explorer - C++ (x86-64 gcc 13.2)

bool atomic_cmpxchg(bool* val, bool valToCompare, bool src)
{
return __atomic_compare_exchange(val, &valToCompare, &src, false, 0,0);
}

weary hound Dec 26, 2023, 12:32 AM

#

sure but the builtin is inlined (and also you can use a bit weaker memory orders)

#

but whatever

weary hound Dec 26, 2023, 12:37 AM

#

flint idol This is my mutex lock code: **```c++ bool Mutex::Lock(uint64_t timeout, bool blo...

at a glance i don't see the issue? what about the other methods

flint idol Dec 26, 2023, 12:37 AM

#

https://github.com/oberrow/obos/blob/master/src/oboskrnl/multitasking/locks/mutex.cpp

GitHub

obos/src/oboskrnl/multitasking/locks/mutex.cpp at master · oberrow/...

I was bored... Contribute to oberrow/obos development by creating an account on GitHub.

#

Ignore Mutex::LockBlockCallback

#

That was from my old mutex implementation

weary hound Dec 26, 2023, 12:38 AM

#

Mutex::Locked is theoretically broken

#

because it's not an atomic load

flint idol Dec 26, 2023, 12:40 AM

#

Let me push new code

#

Done

#

The only thing is my atomic_test is just a normal load

#

I think thread::g_timerTicks < wakeupTime is the problem

#

I'll change it to greater and see what happens

#

Same thing

#

I'll change the scheduler's lock to something that checks if other cpus are running the scheduler

#

See what that does

weary hound Dec 26, 2023, 12:58 AM

#

i'm suspicious of the fact your lock is recursive

#

(same thread can acquire it many times without releasing)

flint idol Dec 26, 2023, 12:59 AM

#

m_canUseMultitasking is false for the scheduler's lock

weary hound Dec 26, 2023, 12:59 AM

#

i mean in general

#

but in that case that's probably not the issue

flint idol Dec 26, 2023, 1:03 AM

#

I'll put the block parameter to false

#

See what that does

#

The only downside is that threads will get slightly more time to run before being preempted

#

It runs until it gets to mounting the initrd, then the assertion to check if only one cpu in running the scheduler fails

#

How about instead of crashing, I just go back to locking the mutex

#

I have a goto statement or something

flint idol Dec 26, 2023, 1:13 AM

#

flint idol I'll put the block parameter to false

One problem with that is if a thread blocks itself and then the scheduler returns, it will cause the blocked thread to continue running

#

Easy fix is to switch to the idle thread

#

or to try locking again

#

Trying to lock again is better because the function to switch to a thread's context isn't guarenteed to not return to the caller (eg: if the implementation for an architecture just sets *interrupt_frame = *thread_context and returns from the interrupt)

#

Why does trying to lock again kill qemu

#

Maybe by default I'll return from the scheduler, but if the thread blocks itself then it switches to the idle thread

#

The scheduler is now in a deadlock

#

A cpu forgot to unlock the scheduler lock

#

How about I add a condition to detect that and unlock the scheduler lock

flint idol Dec 26, 2023, 5:11 PM

#

I just realized my way to interface with drivers is kind of retarded

#

IPC is nice but I'm not planning on making a modular kernel

#

After I fix the scheduler bugs I'll make it use functions

#

and remake the module concept

#

Right now a driver is it's own process

#

with it's own context and everything

#

Context switches are apparently expensive so what I'll do is make drivers use position independent code so I load them in the kernel's context

#

So no context switches

#

I just realized a dumb bug (or should I say oversight)

#

The current bug is every core idling even with running threads

#

Those threads were created with the thread handle interface

#

and when setting the affinity:

thread->affinity = affinity;```

#

But before I said I'd add the original affinity variable

#

I implemented that and only fixed the kernel main thread and the idle threads

#

So when the scheduler went to set the thread's affinity to currentThread->ogAffinity it got set to zero, making the thread not being able to run on any cpu

#

the kernel main thread was blocked waiting for these threads

#

Now I've yapped too much let me test it

#

Kernel heap corruption

#

How fun

flint idol Dec 26, 2023, 5:39 PM

#

I'm pretty sure most scheduler bugs are fixed

#

time to get to remaking the module interface

#

How fun

#

(it's not like I'll need to delete a lot of the code I've made over the past 2 months)

#

but hey that's why git exists

#

to undo deletion

#

This will be pain.

#

It's fine. It's not like I came into osdev because I thought it would be easy.

inland radish Dec 26, 2023, 6:00 PM

#

flint idol I just realized my way to interface with drivers is kind of retarded

You mean it's behind in terms of time?

flint idol Dec 26, 2023, 6:02 PM

#

uhh sure

flint idol Dec 26, 2023, 9:38 PM

#

I'm implementing relocations for modules

#

Just I have no idea how to test them

flint idol Dec 27, 2023, 12:57 PM

#

1108th message

flint idol Dec 27, 2023, 1:54 PM

#

I found out a way to test relocations

#

I use weak symbols, then it gets put in the relocation table

#

This might be UB though, and the linker decides to put the symbol in the relocation table

#

and on other platforms it doesn't

flint idol Dec 28, 2023, 5:02 PM

#

I got relocations working

flint idol Dec 29, 2023, 12:40 AM

#

Is it a problem I don't even understand my vfs code anymore

#

meh it'll be fine as long as it doesn't break

#

I fixed locks*

#

*it will probably not work in the worst times

#

all I did was add this line:

while (m_locked);```
before the cmpxchg line

#

https://stackoverflow.com/questions/1485924/how-are-mutexes-implemented
I just read this page

Stack Overflow

How are mutexes implemented?

Are some implementations better than others for specific applications? Is there anything to earn by rolling out your own?

#

then I realized my mistake (assuming he's right)

flint idol Dec 29, 2023, 12:54 AM

#

flint idol I fixed locks*

oh and I also got mount to work with the new driver interface

limber wave Dec 29, 2023, 7:32 AM

#

flint idol Is it a problem I don't even understand my vfs code anymore

lol
didnt you comment anything?

flint idol Dec 29, 2023, 12:24 PM

#

I almost commented nothing

flint idol Dec 29, 2023, 1:32 PM

#

Read now works with the new driver interface

#

Now time to work more on userspace

#

services in the kernel

#

||(you really thought I was going to work on userspace programs lol)||

#

I can:
a) Add more syscalls
b) Make some sort of signal interface
c) Improve handles on the user side

flint idol Dec 29, 2023, 2:18 PM

#

I think I'll do b then c then a then I'll see what I'll do

flint idol Dec 29, 2023, 2:34 PM

#

I'm going to define what a signal is, and all the signal ids and what each one means:

Definition: A callback (much like an interrupt) called by the kernel on a certain condition (ex: page fault).
Signal IDs:
  SIGPF   (0x00): A page fault occurred that the kernel couldn't resolve (not related to demand paging or any VMM feature).
  SIGPM   (0x01): Permission error. This occurs when the program tries to modify something other than memory that it is not allowed to modify.
  SIGOF   (0x02): Integer overflow.
  SIGME   (0x03): A math error occurred. This can occur on division by zero or any operation that involves math (like anything to do with floating point). This should not occur on integer overflow (see SIGOF).
  SIGDG   (0x04): A debug exception occurred.
  SIGTIME (0x05): A timer set by the thread finished.
  SIGTERM (0x06): A program made a request to terminate the program. This is provided so that the program can properly free any resources opened.
  SIGINT  (0x07): A program wishes to interrupt the program.
  SIGUDOC (0x08): An undefined opcode exception occurred.

flint idol Dec 29, 2023, 3:18 PM

#

I have an urge to make an ahci driver, Idk why. After signals, I'm working on an ahci driver.

flint idol Dec 30, 2023, 4:13 PM

#

I've made a function to iterate over page tables and free them

#

In case anyone's wondering why it's so a process's virtual address space can be freed

#

Of course doing it with a single loop iterating over virtual addresses and VirtualFreeing them is slowwwww.

#

I don't have any sort of data on pages mapped in process structures except for the pml4 pointer

#

and the VirtualAllocator object

flint idol Dec 30, 2023, 4:53 PM

#

flint idol I have an urge to make an ahci driver, Idk why. After signals, I'm working on an...

before I do that I need to do one last thing with modules

#

and that is loading modules if they were detected

#

Meh too much work

#

I'll do it later*

#

*I hope

flint idol Dec 30, 2023, 7:22 PM

#

Debugging is so annoying

#

I'm on one line of code, suddenly it's on some line in the scheduler

flint idol Jan 1, 2024, 12:59 AM

#

YEAH

#

I GOT READ TO WORK

#

WITH THE AHCI DRIVER

#

I just need to fix some bugs

#

Like cmdheader->ctba being null because I didn't setup that right

#

or well at all

#

And I should identify the device with IDENTIFY_DEVICE to find out whether it's ATA or ATAPI

flint idol Jan 1, 2024, 2:11 AM

#

flint idol or well at all

I fixed that (kinda, I needed continuous pages)

#

uintptr_t allocateCLB(size_t maxTries = 10)
{
    uintptr_t firstPage = memory::allocatePhysicalPage();
    uintptr_t secondPage = memory::allocatePhysicalPage();
    size_t i = 0;
    for (; i < maxTries && (firstPage + 0x1000) != secondPage; i++)
    {
        memory::freePhysicalPage(firstPage);
        memory::freePhysicalPage(secondPage);
        firstPage = memory::allocatePhysicalPage();
        secondPage = memory::allocatePhysicalPage();
    }
    if (i == (maxTries - 1))
    {
        memory::freePhysicalPage(firstPage);
        memory::freePhysicalPage(secondPage);
        return 0;
    }
    return firstPage;
}```

#

Not the best algorithm to get continuous pages, but it works

#

(I hope)

#

Now I'm going to get back on task and send IDENTIFY_DEVICE instead of playing around with ATA_READ_DMA_EXT

ashen goblet Jan 1, 2024, 3:06 AM

#

flint idol Not the best algorithm to get continuous pages, but it works

How are you storing physical pages

#

Bitmap?

flint idol Jan 1, 2024, 3:06 AM

#

freelist

ashen goblet Jan 1, 2024, 3:07 AM

#

Couldnt you just hand out a range from there

#

My pmm allocator just looks if it can find [count] contiguous pages in my bitmap

#

And hands out the first one

flint idol Jan 1, 2024, 3:17 AM

#

I only need continuous pages early in boot

#

It's not worth it considering I don't plan on supporting big pages

#

If there comes a time where I need them, then I'll support them

ashen goblet Jan 1, 2024, 3:20 AM

#

fair

flint idol Jan 1, 2024, 2:50 PM

#

I got ATA_WRITE to work

#

So I can now officially Write and Read to disk

flint idol Jan 1, 2024, 5:46 PM

#

I am going to now implement automated module loading

#

For the one device driver I have

#

After that I'm implementing a FAT driver

limber wave Jan 1, 2024, 5:59 PM

#

FAT32?

flint idol Jan 1, 2024, 5:59 PM

#

Yes

#

But last time I checked a FAT driver should support FAT12, FAT16, and FAT32

limber wave Jan 1, 2024, 6:00 PM

#

oh?

#

thought FAT12 and FAT16 were obsolete

flint idol Jan 1, 2024, 6:01 PM

#

Maybe I'm just remembering that wrong though

#

I know I'll at least support FAT32

limber wave Jan 1, 2024, 6:02 PM

#

yea FAT32 is the easy and most universal

flint idol Jan 1, 2024, 6:03 PM

#

Anyway I should start automated module loading

limber wave Jan 1, 2024, 6:04 PM

#

good luck

flint idol Jan 1, 2024, 6:04 PM

#

It shouldn't be too hard*
*ok maybe that was a lie

flint idol Jan 1, 2024, 10:18 PM

#

I need to make a directory entry iterator in my VFS

#

for automated module loading

flint idol Jan 2, 2024, 12:56 AM

#

I just implemented a hashmap structure for the kernel

#

then forgot what I needed it for.

flint idol Jan 2, 2024, 1:27 AM

#

Nvm I remember now

flint idol Jan 2, 2024, 3:12 PM

#

Y'know what, I'm making continuous allocation for my PMM

#

I'll need it sooner or later so might as well do it now

ashen goblet Jan 2, 2024, 3:14 PM

#

ashen goblet Couldnt you just hand out a range from there

Hehe

flint idol Jan 2, 2024, 3:15 PM

#

It might also be faster to initialize the pmm as it doesn't need to make a node for each page and instead just a range of pages

flint idol Jan 3, 2024, 3:12 PM

#

#

I made a hexdump for the first of a disk

#

I tested this on real hardware and it triple faulted while initializing the scheduler

#

I need to fix that

limber wave Jan 3, 2024, 3:14 PM

#

flint idol

why does each port x2 have no drive

flint idol Jan 3, 2024, 3:14 PM

#

If anyone is wondering why it says "Hello, blocked size allocations!" it's because I was testing if it could read multiple chunks of 4 mib blocks from the disk without breaking anything

flint idol Jan 3, 2024, 3:15 PM

#

limber wave why does each port x2 have no drive

Probably just a coincidence

limber wave Jan 3, 2024, 3:16 PM

#

why do you enable fpu and SSE?

flint idol Jan 3, 2024, 3:16 PM

#

So eventually user programs can use it?

limber wave Jan 3, 2024, 3:16 PM

#

ah ok

flint idol Jan 3, 2024, 3:16 PM

#

Just because it can't be used in the kernel, it doesn't mean it should be forbidden for everyone

limber wave Jan 3, 2024, 3:16 PM

#

i was just wondering if you were using it in the kernel lul

flint idol Jan 3, 2024, 3:17 PM

#

Well I think it's possible that some drivers might accidentally use it

#

whoops

limber wave Jan 3, 2024, 3:17 PM

#

?

flint idol Jan 3, 2024, 3:17 PM

#

I'll put -mgeneral-regs-only for the drivers

limber wave Jan 3, 2024, 3:17 PM

#

wdym accidentally?

flint idol Jan 3, 2024, 3:18 PM

#

The compiler might generate SSE memcpy for example

#

as an optimization

limber wave Jan 3, 2024, 3:18 PM

#

ohh

flint idol Jan 3, 2024, 3:18 PM

#

or I use floating point math somewhere and don't notice because I didn't put -mgeneral-regs-only

#

idk why I'd do that though

limber wave Jan 3, 2024, 3:19 PM

#

when youre tired

#

or drunk

flint idol Jan 3, 2024, 3:19 PM

#

In my religon you can't intoxicate so being drunk shouldn't be a problem unless I do it by accident

limber wave Jan 3, 2024, 3:20 PM

#

flint idol In my religon you can't intoxicate so being drunk shouldn't be a problem unless ...

yep

#

although accidental seems unlikely

flint idol Jan 3, 2024, 3:24 PM

#

flint idol Probably just a coincidence

It was, PI is 0x3F, so when looping over ports, the code found that some ports had no underlying port, so it told the user and continued scanning

#

and also a bug

#

whoops

#

logger::warning("AHCI: No drive found on port %d, even though it is implemented.\n");```

#

See the problem?

#

fixed

limber wave Jan 3, 2024, 3:28 PM

#

flint idol ```c++ logger::warning("AHCI: No drive found on port %d, even though it is imple...

no variable after the char* ?

flint idol Jan 3, 2024, 3:28 PM

#

yes

limber wave Jan 3, 2024, 3:28 PM

#

so it was printing random values?

flint idol Jan 3, 2024, 3:28 PM

#

no variable for the format specifier

flint idol Jan 3, 2024, 3:28 PM

#

limber wave so it was printing random values?

yes

#

it just so happened they were the next power of two

limber wave Jan 3, 2024, 3:29 PM

#

huh

#

cool

flint idol Jan 3, 2024, 10:13 PM

#

I hate templates

#

and references.

flint idol Jan 4, 2024, 12:50 AM

#

gdb doesn't work properly on windows when using elf

#

I tried using x86_64-elf-gdb but the g packet was too long

#

and no xml packet descriptions

#

So I need to compile gdb for windows pain

unkempt pawn Jan 4, 2024, 1:07 AM

#

hey, have you tested OBOS on real hw?

#

its looking cool so far!

flint idol Jan 4, 2024, 1:07 AM

#

Recently, yes

#

It triple faults

#

While starting the cores

#

I need to fix that

unkempt pawn Jan 4, 2024, 1:08 AM

#

oh, that sucks

flint idol Jan 4, 2024, 1:08 AM

#

I'll just try fixing it now

#

gdb can wait

#

It triple faults right after sending the SIPI

#

So that suggests it faults in the trampoline code

#

I'm just spamming jmp $ everywhere

unkempt pawn Jan 4, 2024, 1:22 AM

#

cli
hlt

is cooler 😎

flint idol Jan 4, 2024, 1:23 AM

#

nah

db 0xEB, 0xFE```

#

I found out where it faults

#

mov eax, 0x80010001
mov cr0, eax

#

So as soon as it entered long mode and turned on paging, it faults

#

thonk

#

Maybe it's a problem with XD

#

But then it'd happen on qemu

#

I'll try disabling support for it

#

Not the problem

#

thonk

#

SMEP

#

I'll try disabling that and see what happens

#

Nope

#

Maybe the page isn't mapped in the first place

#

ie. the page containing the trampoline

#

Well, I checked the page mappings and all of them have XD cleared

#

and they're all present

#

It's something else

#

It faults on wrmsr

#

I removed the code for setting bit 11 (NXE) of EFER if it's supported

#

still faults

#

The only thing it could be is a GP because If the value in EDX:EAX sets bits that are reserved in the MSR specified by ECX.

#

But that shouldn't be the case

#

Because the only bits that (should be) set are bit 8,10, and 11

#

All of which are supported by the two computers I've tested on

#

I tried not setting bit 10, no effect

#

The only thing it could be is junk in edx

unkempt pawn Jan 4, 2024, 1:50 AM

#

Real hardware is beautiful to debug, isn't it? 😅

flint idol Jan 4, 2024, 1:51 AM

#

I think it's because edx had junk from cpuid

#

And qemu failed to recognize that

#

in fact, now that I put test instead of and for edx, I'll test it on qemu

#

Doesn't fault, I think it's a bug with qemu's wrmsr

#

After zeroing edx it doesn't fault

#

Now I'll just let it run and see what happens

#

I fixed it

#

Now it gets to the same point as on qemu

#

Now I'm going to resume debugging my automated driver loader

flint idol Jan 4, 2024, 2:03 AM

#

flint idol Now it gets to the same point as on qemu

AKA Doing nothing because of a bug with my automated driver loader

#

Well better than triple faulting

#

It's a bug with my hashmap's iterators

#

I think it was because of a single missing = in >=

#

I'll test that theory out

flint idol Jan 4, 2024, 5:23 PM

#

flint idol and references.

and structs local to a function

#

obos::utils::Hashmap<obos::driverInterface::LoadableDriver, obos::driverInterface::ScanPCIBus(obos::utils::Vector<obos::driverInterface::LoadableDriver> const&)::pciDevice, &(bool obos::utils::defaultEquals<obos::driverInterface::LoadableDriver>(obos::driverInterface::LoadableDriver const&, obos::driverInterface::LoadableDriver const&)), &(unsigned long obos::utils::defaultHasher<obos::driverInterface::LoadableDriver>(obos::driverInterface::LoadableDriver const&))>::HashmapIter<obos::driverInterface::LoadableDriver, obos::driverInterface::ScanPCIBus(obos::utils::Vector<obos::driverInterface::LoadableDriver> const&)::pciDevice, &(bool obos::utils::defaultEquals<obos::driverInterface::LoadableDriver>(obos::driverInterface::LoadableDriver const&, obos::driverInterface::LoadableDriver const&)), &(unsigned long obos::utils::defaultHasher<obos::driverInterface::LoadableDriver>(obos::driverInterface::LoadableDriver const&))>::operator++(int)

#

Like what the hell is this

flint idol Jan 4, 2024, 7:53 PM

#

There seems to be a bug with the AHCI driver exclusively on real hardware

#

All the port descriptors (the things the driver uses for keeping track of drives) get zeroed, idk when, how or why, but they just do

flint idol Jan 4, 2024, 8:09 PM

#

Maybe it's not that...

#

The address of the port descriptor is not the same when it got printed out after initializing the drive then when it was printed out when the driver needed to refind it

#

Hmmmm...

#

So it's probably the GetPortDescriptorFromKernelDriveID function not working, even though it's literally one of the most simple functions ever

#

Port* GetPortDescriptorFromKernelDriveID(uint32_t id)
{
    Port* portDescriptor = &g_ports[0];
    for (uint8_t i = 0; i < 32; i++, portDescriptor++)
    {
        if (portDescriptor->driveType == Port::DRIVE_TYPE_INVALID)
            continue;
        if (portDescriptor->kernelID == id)
            return portDescriptor;
        continue;
    }
    return nullptr;
}```

#

g_ports being Port[32]

#

Maybe if I change Port::DRIVE_TYPE_INVALID to be zero instead of negative one

#

That fixes it

#

I can now read from a drive on real hardware (or at least the one computer the kernel was tested on)

#

and I can now say that this OS works on real hardware (or well, what I have so far)

#

I just need to clean up the logger::debug statements

#

Next thing to do is make a way, preferably in the vfs, to read from a drive

unkempt pawn Jan 4, 2024, 8:24 PM

#

flint idol and I can now say that this OS works on real hardware (or well, what I have so f...

nice dude, must feel amazing!

flint idol Jan 4, 2024, 8:25 PM

#

If it weren't for the winter break I would still be in the stone age (in terms of this kernel)

#

I'm actually not quite sure if the READ command read everything correctly

unkempt pawn Jan 4, 2024, 8:30 PM

#

flint idol ```c++ Port* GetPortDescriptorFromKernelDriveID(uint32_t id) { Port* portDes...

Love the continue at the end of the for loop lol

flint idol Jan 4, 2024, 8:30 PM

#

Yeah I was going to remove that

#

But it looks good

#

So why not

#

keep it

#

For reference this is the hexdump of the first sector now:

#

I'll reboot into obos and see if they're the same

#

The hexdump is the same

#

It won't be though after I change to a WRITE command troll

#

For obvious reasons though, I won't

flint idol Jan 4, 2024, 9:21 PM

#

flint idol Next thing to do is make a way, preferably in the vfs, to read from a drive

This is roughly how it'll work:

Disk Driver registers drive: VFS detects that and adds a new drive node "n" (with n being the kernel drive id).
To access a drive node you would open it like any other file, but with the prefix Dn:/ instead of m:/ (with m being a mount point id), and you'd use DriveHandle instead of FileHandle.
You read by calling handle.Read(buff, lba, nSectors);
You write the same way but with Handle.Write
To query the amount of sectors or bytes/sector for a drive you'd call handle.QueryDiskInfo(&nSectors, &sectorSize)

#

and accessing partitions directly would use the same interface, except the path is DdPp (with lowercase d being the drive id, and the lowercase p being the partition id)

vernal chasm Jan 5, 2024, 12:39 AM

#

I was sent here by an external force

flint idol Jan 5, 2024, 12:39 AM

#

Nice, so give star

#

(please)

vernal chasm Jan 5, 2024, 12:39 AM

#

kekw

loud badge Jan 5, 2024, 12:47 AM

#

flint idol Nice, so give star

i did too 🤣

flint idol Jan 5, 2024, 12:47 AM

#

Two people sent by an external force. Hmmmmm........

#

Anyway finally broke 10 stars 🎉

flint idol Jan 5, 2024, 12:57 AM

#

unkempt pawn Love the continue at the end of the for loop lol

I sometimes also add a return to the end of a void function

flint idol Jan 5, 2024, 1:30 AM

#

I'm making a vectorized memcpy for the drive interface.

#

Why? Because it'd be slowwwwww copying sectors on a one byte granularity to a user buffer otherwise.

#

I'll save the SSE state don't worry

vernal chasm Jan 5, 2024, 1:36 AM

#

flint idol I'm making a vectorized memcpy for the drive interface.

Dont

flint idol Jan 5, 2024, 1:36 AM

#

why

vernal chasm Jan 5, 2024, 1:36 AM

#

https://www.felixcloutier.com/x86/stos:stosb:stosw:stosd:stosq
These kind of instructions lead to the highest throughput

flint idol Jan 5, 2024, 1:37 AM

#

there was a whole discussion about this in #x86

#

#x86 message

vernal chasm Jan 5, 2024, 1:39 AM

#

Well on my r9 7950x REP MOVSB was the fastest

#

Literally went as fast as my memory could go

flint idol Jan 5, 2024, 1:39 AM

#

~~well I could check the processor and determine whether to rep movsb or to vectorize~~

vernal chasm Jan 5, 2024, 1:39 AM

#

Initially at 70 GBps iirc and then dropping to 50 GBps after cache

vernal chasm Jan 5, 2024, 1:40 AM

#

flint idol ~~well I could check the processor and determine whether to rep movsb or to vect...

zmm :>

flint idol Jan 5, 2024, 1:40 AM

#

How much data was copied?

vernal chasm Jan 5, 2024, 1:40 AM

#

Think I did 120 megs

flint idol Jan 5, 2024, 1:40 AM

#

Well that's more data my kernel will come across in a long time

vernal chasm Jan 5, 2024, 1:41 AM

#

Thing is tho, REP MOVSB does the unaligned copy much better than what you could do yourself

flint idol Jan 5, 2024, 1:41 AM

#

Frick

vernal chasm Jan 5, 2024, 1:41 AM

#

So implementing memcpy as a single REP MOVSB is gonna give max speed

flint idol Jan 5, 2024, 1:41 AM

#

I forgot about alignment

vernal chasm Jan 5, 2024, 1:42 AM

#

But obviously it's not exactly like it's supported by literally every cpu, not to mention the fact that some lower end intel cpus had rather low perf using it.
However AMD is just better, so not an issue imho :>

flint idol Jan 5, 2024, 1:42 AM

#

intel sdm better though

vernal chasm Jan 5, 2024, 1:47 AM

#

You should just use the byte version

#

It picks the proper version to use and takes the same amount of startup time

flint idol Jan 5, 2024, 1:47 AM

#

Ok

flint idol Jan 5, 2024, 2:03 PM

#

idk how but I have a bug with panic which causes an infinite page fault loop (because the page fault was in kernel mode, so it panics, causing an infinite loop of page faults)

#

It seems to be specifically a bug while clearing the console

#

Again, don't know how I mess up that bad

#

It was because memset wrongly jumped to memcpy to do it's work

#

whoops

flint idol Jan 5, 2024, 2:30 PM

#

Ok I'm done making the DriveHandle interface

#

Time to ~~get to work on the fat driver~~ make the AHCI's driver send WRITE commands on a 4mib granularity instead of a page granularity

#

Then I'll start the FAT driver.

flint idol Jan 5, 2024, 2:53 PM

#

Now it's time to start a FAT driver.

#

But before that, I need a way to interact with partitions.

#

Not to interact, to initialize.

flint idol Jan 5, 2024, 3:14 PM

#

Not to initialize, to recognize them in the kernel

#

So for that, I need to get back to the automated driver loader

#

After loading PCI drivers, it should look for partitions and find a module that supports the filesystem on the partition

#

But for that, I need to add a new section in the driver header for identifying whether the driver can load a certain filesystem on a partition.

#

So uhhh how do I do that

#

Without loading the driver so it can check whether that filesystem exists on any disks or not

#

By always loading filesystem drivers, and letting them find themselves galaxybrain

#

and if that filesystem doesn't exist, ah well that sucks, as I don't have UnloadDriver or anything like that implemented

#

or just add a parameter to the kernel while booting saying what file system drivers it should load

#

I think I'll do the latter approach, as random "file system" drivers aren't being loaded

#

Downside: user will have to edit limine.cfg if they want to boot from another drive

#

That's fine though

#

Downside: more work as I need to now parse the kernel command line

#

That's also fine, I'd probably need to do it sooner or later anyway

flint idol Jan 5, 2024, 3:52 PM

#

I don't think limine can pass a command line to the kernel

#

Not even for multiboot, which is weird

#

I'll just make a config file in the initrd instead

flint idol Jan 5, 2024, 4:17 PM

#

Now, I need to make a config file parser

#

or, I could find a portable one that's compatible with the MIT license

#

Yeah I doubt that exists

flint idol Jan 5, 2024, 5:17 PM

#

I'm done the lexer

#

or whatever it's called

#

Now it should be pretty easy to put all the tokens into a map.

vernal chasm Jan 5, 2024, 6:28 PM

#

flint idol I'm done the lexer

What you making now?

#

Ah I see a config thing

sharp pike Jan 5, 2024, 7:24 PM

#

flint idol I don't think limine can pass a command line to the kernel

of course it can

#

both in Limine and mb

#

and Linux of course

#

all protocols really

flint idol Jan 6, 2024, 3:44 PM

#

flint idol Now it should be pretty easy to put all the tokens into a map.

That was easy, now time to find out why the fuck it page faults

#

in a destructor

#

Why is the compiler passing a Element** to ~Element as the this parameter?

#

😮‍💨

weary hound Jan 6, 2024, 3:55 PM

#

that sounds wrong?

flint idol Jan 6, 2024, 3:55 PM

#

I know

weary hound Jan 6, 2024, 3:56 PM

#

how did you come to that conclusion

flint idol Jan 6, 2024, 3:56 PM

#

looking at the disassembly and looking at rbp

#

#

#

Now I'm debugging the disassembly before the call to the destructor and it's 0xfffffffbf00063b0?

#

I've mistaken rbx with rax

#

I rebuilt it and it works now

#

(ie. doesn't page fault)

#

The second time it page faults?

#

What is this?

flint idol Jan 6, 2024, 5:09 PM

#

Fixed it

flint idol Jan 6, 2024, 6:07 PM

#

Nvm, it was one of those cases when it decided not to triple fault

#

*page fault

flint idol Jan 6, 2024, 7:30 PM

#

Use-after-free bug

#

How fun

#

there's a bug with my lexer on incorrect syntax, but whatever

#

It doesn't report an error like I think it should

#

There is a bug with my lexer

#

FS_DRIVERS = [ 0:/fat32_driver ]

The lexer seems to ignore that statement

#

Because there are spaces between the =

#

It doesn't ignore the statement, the variable name has a trailing space

flint idol Jan 6, 2024, 9:29 PM

#

Ok the config file parser is done

#

Time to ~~start the FAT driver~~ detect partitions on a disk and register them

#

I'm thinking it should be a module

#

If there is a mbr on a disk, load the mbr driver

#

If there is a gpt on a disk, load the gpt driver.

#

Current roadmap (in order):
Make partition detecting->FAT32 driver->Implementing FileHandle::Write->Usermode services?

flint idol Jan 7, 2024, 12:18 AM

#

Done the "MBR driver"

flint idol Jan 7, 2024, 3:40 PM

#

Made a new branch for drivers related to disk access and stuff like that

#

https://github.com/oberrow/obos/tree/disk-related-driver-dev

GitHub

GitHub - oberrow/obos at disk-related-driver-dev

I was bored... Contribute to oberrow/obos development by creating an account on GitHub.

flint idol Jan 7, 2024, 8:30 PM

#

I just found a funny bug in the hashmap class

#

in erase delete (node*)_node->data;

#

the problem is, _node->data is allocated in the node table

#

so at one point it gets to the node which has the base of the node table and frees it

#

then this:
kfree(m_nodeTable); m_nodeTable = nullptr;

#

tries to free the node table, but it's already freed

flint idol Jan 7, 2024, 9:39 PM

#

I tested the gpt module

#

and it doesn't work

#

I fixed that bug, just to find out it still doesn't work

#

Because the crc32 instruction isn't supported

#

unfortunately on real hardware you can't just make instructions exist

#

so I have to implement it

#

TEMP1[7-0] := BIT_REFLECT8(SRC[7-0])
TEMP2[31-0] := BIT_REFLECT32 (DEST[31-0])
TEMP3[39-0] := TEMP1[7-0] « 32
TEMP4[39-0] := TEMP2[31-0] « 8
TEMP5[39-0] := TEMP3[39-0] XOR TEMP4[39-0]
TEMP6[31-0] := TEMP5[39-0] MOD2 11EDC6F41H
DEST[31-0] := BIT_REFLECT (TEMP6[31-0])```

#

I mean that can't be that bad, right

#

But what's BIT_REFLECT

weary hound Jan 7, 2024, 9:46 PM

#

bit flip

#

as in out[0] = in[width-1], out[1] = in[width-2] etc

flint idol Jan 7, 2024, 9:55 PM

#

what is MOD2

#

Is it just modulo

flint idol Jan 7, 2024, 10:57 PM

#

This algorithm for CRC32 doesn't work, does anyone know what I did wrong**

uint32_t crc32_byte(char ch, uint32_t previousChecksum)
{
  uint64_t dest = previousChecksum;
  uint64_t temp1 = ~(ch);
  uint64_t temp2 = ~(dest);
  uint64_t temp3 = (temp1 << 32) & 0xff'ffff'ffff;
  uint64_t temp4 = (temp2 << 8) & 0xff'ffff'ffff;
  uint64_t temp5 = (temp3 ^ temp4) & 0xff'ffff'ffff;
  uint64_t temp6 = temp5 % 0x11EDC6F41;
  dest = ~(temp6);
  return dest;
};```**

#

ie. it doesn't calculate the correct checksum

#

wait

#

I think I got a fix

weary hound Jan 7, 2024, 10:58 PM

#

flint idol Is it just modulo

MOD2: Remainder from Polynomial division modulus 2

flint idol Jan 7, 2024, 10:59 PM

#

oh

flint idol Jan 7, 2024, 11:15 PM

#

If a cpu from 2008 supports SSE4.2 I'm not making a CRC32 algorithm

rapid pulsar Jan 7, 2024, 11:27 PM

#

SSE4.2 was available in Nehalem, so just barely shipped in 2008.

vernal chasm Jan 8, 2024, 12:19 AM

#

flint idol If a cpu from 2008 supports SSE4.2 I'm not making a CRC32 algorithm

Be like me and only support cpus made in the past year lmao

#

Or just assume every extension exists xD

flint idol Jan 8, 2024, 12:22 AM

#

I make stable os not unstable os

#

I will have to make a CRC32 algorithm if I port to a different arch

#

But sounds like a future me problem

vernal chasm Jan 8, 2024, 12:26 AM

#

flint idol I will have to make a CRC32 algorithm if I port to a different arch

A CRC33, damn 33 bit CRC is kinda nice

#

kekw

flint idol Jan 8, 2024, 12:36 AM

#

CRC32 checksum for the GPT Header structure. This value is computed by setting this field to 0, and computing the 32-bit CRC for HeaderSize bytes.
I calculate the CRC32 exactly like it said with the CPU algorithm, and the checksum still doesn't match

#

That rather means the header is corrupted or qemu's crc32 algorithm is broken

vernal chasm Jan 8, 2024, 12:37 AM

#

You trying to generate a gpt disk image?

flint idol Jan 8, 2024, 12:37 AM

#

I'm trying to load one

#

in my kernel

#

(well actually I'm implementing it as a module but whatever)

vernal chasm Jan 8, 2024, 12:37 AM

#

kekw

weary hound Jan 8, 2024, 12:38 AM

#

flint idol That rather means the header is corrupted or qemu's crc32 algorithm is broken

or the crc32 in the gpt header uses a different polynomial

vernal chasm Jan 8, 2024, 12:38 AM

#

Should use the standard polynomial

#

I remember writing the code back in december 2022, but I've lost it sadcat

flint idol Jan 8, 2024, 12:40 AM

#

If the primary GPT is corrupt, software must check the last LBA of the device to see if it has a valid GPT Header and point to a valid GPT Partition Entry Array. If it points to a valid GPT Partition Entry Array, then software should restore the primary GPT if allowed by platform policy settings (e.g. a platform may require a user to provide confirmation before restoring the table, or may allow the table to be restored automatically). Software must report whenever it restores a GPT.

vernal chasm Jan 8, 2024, 12:40 AM

#

flint idol > If the primary GPT is corrupt, software must check the last LBA of the device ...

Yeah

#

Did you forget to make the backup?

flint idol Jan 8, 2024, 12:40 AM

#

it's the iso that's regenerated every build so that's fine

#

I'm trying to implement GPT

weary hound Jan 8, 2024, 12:42 AM

#

also note that when computing the checksum you should treat the checksum field as if it's 0

flint idol Jan 8, 2024, 12:42 AM

#

I do

vernal chasm Jan 8, 2024, 12:43 AM

#

Are you computing it in little or big endian O_O

flint idol Jan 8, 2024, 12:43 AM

#

idk

#

(maybe that's a bad sign)

flint idol Jan 8, 2024, 1:07 AM

#

I ran a crc32 checksum on the data on a website and it's the same I get

weary hound Jan 8, 2024, 1:08 AM

#

what's the expected checksum and the one you get?

flint idol Jan 8, 2024, 1:08 AM

#

Expected: 0xc9b9428a
Calculated: 0x7d77ef9d

vernal chasm Jan 8, 2024, 1:09 AM

#

Hmm, sounds a lot like the funny bugs I ended up getting with the AES encryption kekw

flint idol Jan 8, 2024, 1:09 AM

#

It would be if it weren't for the fact that two websites I've tried have given the same checksum that I calculated

weary hound Jan 8, 2024, 1:12 AM

#

isn't " crc32 rax, [rdi]\n" missing byte ptr before [rdi]?

flint idol Jan 8, 2024, 1:12 AM

#

I fixed that bug

#

I'm looking at the image I'm loading, and the checksum matches

#

With the one I calculated

#

it could be memory corruption of the buffer containing the sector

#

I just regenerated the iso and calculated the checksum, the checksum is different on the website, in the header, and the one I calculated.

vernal chasm Jan 8, 2024, 1:21 AM

#

lmao

flint idol Jan 8, 2024, 1:22 AM

#

and I looked at the image on disk, and the checksum is different there too

#

nooo

#

At this point, why even calculate checksums

#

and check them

#

it's not like the data is from the network

vernal chasm Jan 8, 2024, 1:25 AM

#

I mean we could've just use a SHA-1 hash as a checksum kekw

flint idol Jan 8, 2024, 1:37 AM

#

vernal chasm I mean we could've just use a SHA-1 hash as a checksum <a:kekw:11664533944336426...

it'd be funny gdisk actually did that for the checksum

#

or xorriso whatever

vernal chasm Jan 8, 2024, 1:38 AM

#

It computes the SHA-1 hash and just uses the first coefficient result kekw

#

Would be dumb

#

uint32_t crctab[256];

void crcInit()
{
    uint32_t crc = 0;
    for (uint16_t i = 0; i < 256; ++i)
    {
        crc = i;
        for (uint8_t j = 0; j < 8; ++j)
        {
            uint32_t mask = -(crc & 1);
            crc = (crc >> 1) ^ (0xEDB88320 & mask);
        }
        crctab[i] = crc;
    }
}

uint32_t crc(const char* data, size_t len)
{
    uint32_t result = ~0U;
    for (size_t i = 0; i < len; ++i)
        result = (result >> 8) ^ crctab[(result ^ data[i]) & 0xFF];
    return ~result;
}

This should work

#

Should stress it only works on little endian machines, most likely

#

Also the crctab thing could be pre computed and just placed within the binary

flint idol Jan 8, 2024, 1:43 AM

#

what if I said that algorithm didn't work

vernal chasm Jan 8, 2024, 1:43 AM

#

Then you've got some funky shit...

flint idol Jan 8, 2024, 1:43 AM

#

ie. returned 0

vernal chasm Jan 8, 2024, 1:44 AM

#

Also how?

#

What's the input?

flint idol Jan 8, 2024, 1:44 AM

#

nvm

#

I have stack corruption

vernal chasm Jan 8, 2024, 1:47 AM

#

🤦‍♂️

flint idol Jan 8, 2024, 1:47 AM

#

Pretty weird, the main cause of the stack being corrupted is a thread being run on two different cpus, that isn't the case

vernal chasm Jan 8, 2024, 1:48 AM

#

Good job xD

#

So if you run on a single threaded system it'll work?

flint idol Jan 8, 2024, 1:49 AM

#

I fixed those scheduler bugs a few weeks ago

#

With the same thread being ran at the same time on seperate CPUs

#

I have some sort of stack/memory corruption bug

#

I ran the algorithm again and the checksums matched

vernal chasm Jan 8, 2024, 1:52 AM

#

Nice, but also oof

flint idol Jan 8, 2024, 1:52 AM

#

Sounds like a problem for me tomorrow after school

#

Development of this project will slow down a lot because of school

flint idol Jan 8, 2024, 1:55 AM

#

flint idol I have some sort of stack/memory corruption bug

It'd be really funny if that was because of the AHCI driver doing a DMA access to the stack's memory

flint idol Jan 9, 2024, 12:36 AM

#

This is interesting...
I'm page faulting on line 32 of the ahci driver's command.cpp because pPort is = 0x10, and that is unmapped. Even though:

flint idol Jan 9, 2024, 12:55 AM

#

I got a idea for improving the stability of the kernel

#

There's an instruction, rdpid to read the proccesor id

#

Which the kernel sets

#

So instead of always using GS_BASE for the cpu local pointer, I just use the processor id and add that to g_cpuInfo

#

That way if I make a syscall that did wrmsr, and a program writes the GS_BASE msr, nothing should page fault or something because GS_BASE was some random address

#

There's this tool for windows called msr-tools, if you write to GS_BASE, you'll get a BSOD with UNEXPECTED KERNEL MODE TRAP

#

*msr-cmd

#

Might not do this though because I could just well, not make a syscall for writing to MSRs

flint idol Jan 9, 2024, 1:23 AM

#

I just hate it when I'm looking at the stack trace of panic and I see external code for driver code

#

It'd be funny if I fix that

flint idol Jan 9, 2024, 1:53 AM

#

Done, now just gotta test it

flint idol Jan 9, 2024, 10:44 PM

#

flint idol There's this tool for windows called msr-tools, if you write to GS_BASE, you'll ...

On linux it segfaults the program

#

I think the reason windows crashes is because the tool uses a kernel-mode driver to access MSRs

#

and on linux the tool might use some syscall that checks for bad msrs

#

(rather that or the tool doesn't work at all)

flint idol Jan 10, 2024, 12:12 AM

#

Don't you just love it when you add one variable to an allocator structure and the entire kernel falls apart

#

struct memBlock
{
    uint32_t magic = MEMBLOCK_MAGIC;
    
    size_t size = 0;
    void* allocAddr = 0;

    memBlock* next = nullptr;
    memBlock* prev = nullptr;

    void* pageBlock = nullptr;
#ifdef OBOS_DEBUG
    void* whoAllocated = nullptr;
    void* padding = nullptr;
#endif
};```

#

I added those last two variables for debugging purposes, and everything fell apart

flint idol Jan 10, 2024, 12:26 AM

#

flint idol Done, now just gotta test it

Doesn't work

#

Time to fix it...

#

Fixed

#

I think I've found a memory leak

#

in LoadModule

#

Nvm

flint idol Jan 10, 2024, 1:02 AM

#

I think I've found the source of stack corruption

#

Interrupt frames

flint idol Jan 10, 2024, 1:45 AM

#

I've now added a limit of three kernel mode exceptions before a manual reset is initiated

#

But anyway I can now parse GPT partitions

#

So I can start the FAT32 driver

vernal chasm Jan 10, 2024, 3:20 AM

#

flint idol So I can start the FAT32 driver

You should probably build a single driver for all fat types

lean glen Jan 10, 2024, 3:21 AM

#

Yeah that should include a driver for your mom

vernal chasm Jan 10, 2024, 3:21 AM

#

Since FAT12, 16 and 32 use the same base, exFAT is somewhat different but is very similar

vernal chasm Jan 10, 2024, 3:21 AM

#

lean glen Yeah that should include a driver for your mom

O_O

lean glen Jan 10, 2024, 3:21 AM

#

https://tenor.com/view/roasted-oh-shookt-gif-8269968

Tenor

lean glen Jan 10, 2024, 3:21 AM

#

vernal chasm O_O

Yk cuz she's fat

vernal chasm Jan 10, 2024, 3:21 AM

#

sad

flint idol Jan 10, 2024, 5:21 PM

#

vernal chasm You should probably build a single driver for all fat types

Yeah that's what I meant

#

I think it'd be interesting to see how the panic screen would look if it didn't clear the screen

#

Lemme do that

#

Yeah I prefer the other one

flint idol Jan 11, 2024, 12:53 AM

#

The FAT32 spec is surprisingly small...
https://academy.cba.mit.edu/classes/networking_communications/SD/FAT.pdf

#

Rather that or I got the wrong spec

vernal chasm Jan 11, 2024, 1:15 AM

#

That's the correct one yes

ornate ginkgo Jan 11, 2024, 1:15 AM

#

flint idol The FAT32 spec is surprisingly small... https://academy.cba.mit.edu/classes/netw...

Ironic

vernal chasm Jan 11, 2024, 1:15 AM

#

Tho microsoft also has their slightly updated version posted somewhere

#

Didn't save the link tho xD

flint idol Jan 11, 2024, 1:16 AM

#

I found exfat

vernal chasm Jan 11, 2024, 1:16 AM

#

That one big...

flint idol Jan 11, 2024, 1:16 AM

#

flint idol I found exfat

from microsoft

vernal chasm Jan 11, 2024, 1:16 AM

#

At least that upcast table...

#

https://en.wikipedia.org/wiki/Design_of_the_FAT_file_system
Also on wiki

vernal chasm Jan 11, 2024, 7:59 AM

#

http://download.microsoft.com/download/1/6/1/161ba512-40e2-4cc9-843a-923143f3456c/fatgen103.doc
It was on the osdev wiki

flint idol Jan 11, 2024, 11:42 AM

#

It wouldn't let me download it

vernal chasm Jan 11, 2024, 12:34 PM

#

Wait what

#

📎 fatgen103.doc

#

There you go

flint idol Jan 12, 2024, 12:43 AM

#

Bug with the AHCI driver, don't know how or why, but while reading sector 1, it reads sector 0

#

Is LBA zero-based or one-based

astral mason Jan 12, 2024, 12:45 AM

#

0 based iirc

flint idol Jan 12, 2024, 12:45 AM

#

Then that's weird

#

Reading for LBA offset 2 returns the 2nd sector, which wouldn't happen if it were zero-based

#

Logical block addressing (LBA) is a common scheme used for specifying the location of blocks of data stored on computer storage devices, generally secondary storage systems such as hard disk drives. LBA is a particularly simple linear addressing scheme; blocks are located by an integer index, with the first block being LBA 0, the second LBA 1, and so on.

#

From wikipedia

astral mason Jan 12, 2024, 12:47 AM

#

flint idol Reading for LBA offset 2 returns the 2nd sector, which wouldn't happen if it wer...

oh? idk then. That is odd.

flint idol Jan 12, 2024, 2:03 AM

#

I can successfully recognize a FAT partition

#

At least FAT12

#

Bug with the AHCI driver ugh

astral mason Jan 12, 2024, 2:13 AM

#

Well, that's better than being able recognize nothing.

vernal chasm Jan 12, 2024, 4:36 AM

#

flint idol At least FAT12

PogW

#

But where fat16 and 32?

flint idol Jan 12, 2024, 11:44 AM

#

Untested

#

My FAT32 partition isn't getting read properly by the AHCI driver

vernal chasm Jan 12, 2024, 1:00 PM

#

flint idol My FAT32 partition isn't getting read properly by the AHCI driver

kekw

flint idol Jan 12, 2024, 1:05 PM

#

Yes very funny

#

Idk why or how but it ain't getting read

#

I'm just getting zeroes

vernal chasm Jan 12, 2024, 1:15 PM

#

You sure you're not just reading at the wrong location?

flint idol Jan 12, 2024, 3:11 PM

#

I might be

flint idol Jan 12, 2024, 10:56 PM

#

Ok this is pretty weird. LBA for the disk with the kernel's iso is one-based, but on the disk image I created it's zero-based

astral mason Jan 12, 2024, 10:58 PM

#

Yeah... that sound very odd.

flint idol Jan 12, 2024, 10:59 PM

#

Maybe there's a field in the IDENTIFY packet for either of the disks that I missed

#

I set device in the FIS for the read command to zero when bit 6 had to be set

astral mason Jan 12, 2024, 11:11 PM

#

I would presume that's not what we want. (don't know what FIS is)

flint idol Jan 12, 2024, 11:12 PM

#

A "FIS" is a command packet to a SATA drive

astral mason Jan 12, 2024, 11:12 PM

#

flint idol A "FIS" is a command packet to a SATA drive

So... I'm guessing we need that if we want to use a SATA drive.

flint idol Jan 12, 2024, 11:13 PM

#

astral mason Jan 12, 2024, 11:13 PM

#

flint idol

ah, yeah that would explain it.

flint idol Jan 12, 2024, 11:21 PM

#

My AHCI driver is so slow it lags qemu

#

Problem with the disk image

#

It has a FAT32 partition with a cluster count of less than 65525

astral mason Jan 12, 2024, 11:29 PM

#

flint idol It has a FAT32 partition with a cluster count of less than 65525

Ah, mystery solved!

flint idol Jan 13, 2024, 12:58 AM

#

partition->handlesReferencing.size--;
This line somehow decrements both partition->handlesReferencing.size and drive->nPartitions

#

How?

#

Whoops

#

if (m_node != m_driveNode)
{
    PartitionEntry* partition = (PartitionEntry*)m_driveNode;
    HandleListNode* handleNode = (HandleListNode*)m_handleNode;
    if (handleNode->next)
        handleNode->next->prev = handleNode->prev;
    if (handleNode->prev)
        handleNode->prev->next = handleNode->next;
    if (partition->handlesReferencing.head == handleNode)
        partition->handlesReferencing.head =  handleNode->next;
    if (partition->handlesReferencing.tail == handleNode)
        partition->handlesReferencing.tail =  handleNode->prev;
    partition->handlesReferencing.size--;
    delete handleNode;
}```

#

See the problem? Hint: It's with the initialization of partition.

flint idol Jan 13, 2024, 1:26 AM

#

Bug with the allocator

#

ugh

#

The allocator is giving out a block of memory that a) is too small b) overlaps with another block of memory

#

Heap corruption

#

Whyyyyyyyyyyyyyyyyy

#

Not just heap corruption, the heap is absolutely fucked

flint idol Jan 13, 2024, 1:46 AM

#

Uhhh

#

That is not right.

#

Found the bug

#

if (!block)
{
    OBOS_ASSERTP(currentPageBlock->lastBlock->magic == MEMBLOCK_MAGIC, "Kernel heap corruption detected for block %p, allocAddr: %p, sizeBlock: %p!", "",
        currentPageBlock->lastBlock,
        currentPageBlock->lastBlock->allocAddr,
        currentPageBlock->lastBlock->size);
    uintptr_t addr = (uintptr_t)currentPageBlock->lastBlock->allocAddr;
    addr += currentPageBlock->lastBlock->size;
    OBOS_ASSERTP(addr > 0xfffffffff0000000, "Kernel heap corruption detected for block %p, allocAddr: %p, sizeBlock: %p!", "",
        currentPageBlock->lastBlock,
        currentPageBlock->lastBlock->allocAddr,
        currentPageBlock->lastBlock->size);
    block = (memBlock*)addr;
}```
Assumes that `lastBlock` is the highest block

#

But it isn't

#

Tried to fix that bug, didn't work so great

astral mason Jan 13, 2024, 2:00 AM

#

flint idol Tried to fix that bug, didn't work so great

Yikes.

flint idol Jan 13, 2024, 2:45 AM

#

Yeah I think I might change the kernel allocator to be free list instead of whatever I did

#

Less buggy and most likely faster

flint idol Jan 13, 2024, 11:46 PM

#

What genius' idea was it to allow LFN to be out of order

#

First entry I see: 0x42 (Second Entry, Last Entry)
Second entry I see: 0x01 (First Entry)

flint idol Jan 14, 2024, 12:34 AM

#

I think I can successfully list files on FAT32

#

I just want to stress test it a bit

flint idol Jan 14, 2024, 1:01 AM

#

Well it kinda worked:

dir
test.txt
dir/other_test.txt
dir/subdirectory
dir/subdirectory/.you_guessed_
dir/subdirectory/you_guessed_it_another_test.txt```

#

Except what is dir/subdirectory/.you_guessed_ doing there

vernal chasm Jan 14, 2024, 1:33 AM

#

flint idol What genius' idea was it to allow LFN to be out of order

They should be reverse order

flint idol Jan 14, 2024, 1:34 AM

#

But, why though?

#

Anyway I fixed that bug so it's fine.

vernal chasm Jan 14, 2024, 1:34 AM

#

Ask michaelsoft

flint idol Jan 14, 2024, 1:54 PM

#

Good idea

#

I should ask microsoft

#

@ kelowerirql

#

I can successfully mount a FAT32 partition!

#

Time to test reads

#

YEAH

#

I CAN FUCKING READ A FILE

#

FROM DISK

#

FINALLY

#

#

#

If only I could test it on read hardware

#

*real

#

I could

#

But I'd need to make a new partition

#

on my disk and that might corrupt stuff

#

The new code has been pushed

ashen goblet Jan 14, 2024, 2:25 PM

#

flint idol on my disk and that might corrupt stuff

Couldnt you plug in an external disk

#

Steal one from your mom’s laptop or smth

flint idol Jan 14, 2024, 2:26 PM

#

Probably not

#

Assuming you mean like with usb, then unless it's emulated by the AHCI controller, then no

flint idol Jan 14, 2024, 2:27 PM

#

ashen goblet Steal one from your mom’s laptop or smth

I do actually have like two laptops from <2010 that are broken so I could do that

#

Or I could just make another partition on my drive that's formatted as FAT32

#

and test that way

ashen goblet Jan 14, 2024, 2:28 PM

#

flint idol Or I could just make another partition on my drive that's formatted as FAT32

Thats riskier but go ahead if you want

flint idol Jan 14, 2024, 2:28 PM

#

I already made the partition sooooooo

#

I accidentally made the cluster size for the partition 512 bytes

#

whoops

#

I guess that's another way to test the driver

#

Anyway I'm going to reboot, hope I didn't corrupt anything

#

It seems to hang while mounting the initrd

#

It sometime does that on qemu so I'll reboot again

flint idol Jan 14, 2024, 2:49 PM

#

It hangs on loading the fat driver

#

it stopped hanging

#

On loading the fat driver

#

Now it hangs while listing the files

#

In the EFI system partition of course

vernal chasm Jan 14, 2024, 2:53 PM

#

flint idol

I like your y

flint idol Jan 14, 2024, 2:53 PM

#

Hehe

#

Anyway it now fails while in the allocator

vernal chasm Jan 14, 2024, 2:54 PM

#

Also your o, I... mean p

flint idol Jan 14, 2024, 2:56 PM

#

vernal chasm Jan 14, 2024, 2:56 PM

#

Good job

flint idol Jan 14, 2024, 2:57 PM

#

Yeah I know KEKW

#

I think I have a memory leak somewhere

vernal chasm Jan 14, 2024, 2:58 PM

#

oof

flint idol Jan 14, 2024, 2:58 PM

#

Specifically in pathStrcmp

vernal chasm Jan 14, 2024, 2:58 PM

#

Meanwhile I still haven't started the vmm after like 13 hours today

#

All I have so far lmao

flint idol Jan 14, 2024, 3:03 PM

#

If only I could connect a debugger to the kernel on real hardware sigh

#

print debugging is nice but not always enough

vernal chasm Jan 14, 2024, 3:04 PM

#

flint idol If only I could connect a debugger to the kernel on real hardware *sigh*

Just set up a serial connection over usb

flint idol Jan 14, 2024, 3:04 PM

#

With what gdb stub

vernal chasm Jan 14, 2024, 3:04 PM

#

and then debug the serial connection

vernal chasm Jan 14, 2024, 3:04 PM

#

flint idol With what gdb stub

¯_(ツ)_/¯

flint idol Jan 14, 2024, 3:04 PM

#

The one I was working on before but gave up on

vernal chasm Jan 14, 2024, 3:04 PM

#

Another thing would be to implement networking

flint idol Jan 14, 2024, 3:04 PM

#

troll2

#

Slow down there

vernal chasm Jan 14, 2024, 3:05 PM

#

flint idol The one I was working on before but gave up on

Sounds great lmao

flint idol Jan 14, 2024, 3:05 PM

#

It didn't work btw

vernal chasm Jan 14, 2024, 3:05 PM

#

vernal chasm Another thing would be to implement networking

Or you could make a sound driver that plays tones that are decoded by another machine kekw

flint idol Jan 14, 2024, 3:06 PM

#

https://github.com/oberrow/obos/tree/master/src/oboskrnl/arch/x86_64/gdbstub

vernal chasm Jan 14, 2024, 3:06 PM

#

Wireless communication at the very high speed of a 343 meters a second

flint idol Jan 14, 2024, 3:06 PM

#

lol

vernal chasm Jan 14, 2024, 3:06 PM

#

flint idol https://github.com/oberrow/obos/tree/master/src/oboskrnl/arch/x86_64/gdbstub

Yeah saw it

#

Was a bit like "But hey we have qemu for that" kekw

flint idol Jan 14, 2024, 3:07 PM

#

It was because it was annoying to debug because the scheduler would preemept a function I'm debugging

vernal chasm Jan 14, 2024, 3:08 PM

#

flint idol It was because it was annoying to debug because the scheduler would preemept a f...

lmao

flint idol Jan 14, 2024, 3:08 PM

#

I could make my own debugging protocol (not)

#

I'm sure that'd work

vernal chasm Jan 14, 2024, 3:08 PM

#

https://tenor.com/view/do-it-get-to-work-gif-21630516

Tenor

flint idol Jan 14, 2024, 3:08 PM

#

Ok bet

vernal chasm Jan 14, 2024, 3:08 PM

#

https://tenor.com/view/i-dare-ya-dare-dare-ya-ace-venture-ace-gif-23721821

Tenor

flint idol Jan 14, 2024, 3:09 PM

#

I'll do it

#

It'll be simple af though

#

If I end up not giving up

#

Now... should I implement it in the kernel or as a module thonk

vernal chasm Jan 14, 2024, 3:10 PM

#

Implement it in the firmware

lean glen Jan 14, 2024, 3:10 PM

#

flint idol https://github.com/oberrow/obos/tree/master/src/oboskrnl/arch/x86_64/gdbstub

just get a serial cable and use that

flint idol Jan 14, 2024, 3:10 PM

#

It doesn't work though

vernal chasm Jan 14, 2024, 3:10 PM

#

Then make it work

#

kekw

lean glen Jan 14, 2024, 3:10 PM

#

fix it then

flint idol Jan 14, 2024, 3:11 PM

#

lean glen just get a serial cable and use that

I could but I have no computer with a serial port, and I ain't buying a computer with a serial port just for osdev

vernal chasm Jan 14, 2024, 3:11 PM

#

flint idol I could but I have no computer with a serial port, and I ain't buying a computer...

usb ports

#

usb can do serial

#

like serial is literally in the name usb

#

O_O

#

It's just a few thousand loc for the usb side of things tho

flint idol Jan 14, 2024, 3:11 PM

#

If I need a USB driver to do that, then no

#

haha no

#

I'm fine with print debugging on real hardware

#

I'm not implementing USB yet

#

I want to start userspace ~~programs~~ services in the kernel

lean glen Jan 14, 2024, 3:13 PM

#

there must be some kind of serial to usb

flint idol Jan 14, 2024, 3:13 PM

#

iirc there are serial port to usb adapters

vernal chasm Jan 14, 2024, 3:13 PM

#

lean glen there must be some kind of serial to usb

That too, but you still need a usb driver on one end

lean glen Jan 14, 2024, 3:13 PM

#

yeah on linux

#

thats fine

vernal chasm Jan 14, 2024, 3:13 PM

#

Well if his test rig only has usb ports then it'd need a usb driver

#

kekw

flint idol Jan 14, 2024, 3:14 PM

#

I think I might have a serial port, rather that or it's VGA output

vernal chasm Jan 14, 2024, 3:14 PM

#

This?

flint idol Jan 14, 2024, 3:15 PM

#

lemme check

#

it's VGA

vernal chasm Jan 14, 2024, 3:16 PM

#

lmao

#

Just tell your machine to make the usb port a slave port, and then connect your main rig to that port :>

#

And communicate over usb

vernal chasm Jan 14, 2024, 3:17 PM

#

vernal chasm Just tell your machine to make the usb port a slave port, and then connect your ...

Tbf idk how many motherboards actually support being a slave device

flint idol Jan 14, 2024, 3:17 PM

#

Wait here's what I do, I get Arduino

#

Then I make arduino gdb

#

galaxybrain

lean glen Jan 14, 2024, 3:18 PM

#

not a bad idea

vernal chasm Jan 14, 2024, 3:18 PM

#

flint idol Wait here's what I do, I get Arduino

kekw

flint idol Jan 14, 2024, 3:18 PM

#

I do have an arduino though

vernal chasm Jan 14, 2024, 3:18 PM

#

That's like me when I needed to do wireless communication between an extremely old device that only supported a bluetooth wifi network and my machine...

flint idol Jan 14, 2024, 3:19 PM

#

I have an ATMEGA2560 on the arduino iirc

vernal chasm Jan 14, 2024, 3:19 PM

#

PogW

flint idol Jan 14, 2024, 3:19 PM

#

I'm about to make the lightest gdb ever lol

vernal chasm Jan 14, 2024, 3:19 PM

#

That guy is a beast for being avr

#

esp32 much better at similar or cheaper price

flint idol Jan 14, 2024, 3:20 PM

#

I could use a raspberry pi and avoid having to make my own gdb

#

Since they're ARM chips

#

that also use linux

#

So I could just make a x86_64-elf-gdb for arm

#

Then connect it to the test machine with usb somehow

lean glen Jan 14, 2024, 3:21 PM

#

you'd still need usb

vernal chasm Jan 14, 2024, 3:21 PM

#

Yeah kekw

vernal chasm Jan 14, 2024, 3:22 PM

#

flint idol Then connect it to the test machine with usb somehow

Guess this means it's time for you to write a fully functioning usb driver

flint idol Jan 14, 2024, 3:22 PM

#

or I could just write the bare minimum (probably a lot) for sending/receiving data on USB

vernal chasm Jan 14, 2024, 3:23 PM

#

Well you still have to read the entire device hierarchy lmao

flint idol Jan 14, 2024, 3:23 PM

#

Or I could just use print debugging

vernal chasm Jan 14, 2024, 3:23 PM

#

Tbf that's not too many commands

flint idol Jan 14, 2024, 3:25 PM

#

how about before thinking about making a usb driver for the sole purpose of using a gdb stub that doesn't even work yet, I finish the gdb stub

vernal chasm Jan 14, 2024, 3:26 PM

#

Yes

flint idol Jan 14, 2024, 3:26 PM

#

I don't like the gdb stub protocol though, so I'll make my own KEKW

#

Average OS developer who has nothing better to do

#

I think my motherboard supports serial just doesn't expose a serial port

#

I remember seing in bios settings something about serial port configuration

vernal chasm Jan 14, 2024, 3:31 PM

#

flint idol Average OS developer who has nothing better to do

Average developer who is like "We have 13 different ways of doing it, we need a standard!" next year "We have 14 different ways of doing it, fuck did we do"

flint idol Jan 14, 2024, 3:32 PM

#

uhh that definitely isn't me

#

I may not even make anything to do with debugging stubs for a while

#

Got more important stuff to deal with (userspace, vfs)

vernal chasm Jan 14, 2024, 3:34 PM

#

oof

flint idol Jan 14, 2024, 3:35 PM

#

I just tested the FAT driver on real hardware

#

I got file not found for each file

#

Pretty weird considering the mount point exists, maybe the file just doesn't exist at all

#

I'll loop over the directory entries to find out

vernal chasm Jan 14, 2024, 3:38 PM

#

yikes

#

Multiple aligned Pages: 0x0000000007C00000
That's at least 2 MiB aligned

#

PogW

flint idol Jan 14, 2024, 3:46 PM

#

Uhh

#

I think you got the wrong post

vernal chasm Jan 14, 2024, 3:46 PM

#

kekw

#

Starting to show my tiredness xD

flint idol Jan 14, 2024, 3:47 PM

#

The mount point was referring to the EFI system partition that's why the file wasn't found

vernal chasm Jan 14, 2024, 3:47 PM

#

lmao

#

Did you double mount or something?

flint idol Jan 14, 2024, 3:47 PM

#

There seems to be a bug with the FAT driver though

flint idol Jan 14, 2024, 3:48 PM

#

vernal chasm Did you double mount or something?

It was because I assumed that "1:/" was the partition with the test files

flint idol Jan 14, 2024, 3:48 PM

#

flint idol There seems to be a bug with the FAT driver though

Specifically with parsing 8.3 filenames

vernal chasm Jan 14, 2024, 3:49 PM

#

flint idol Specifically with parsing 8.3 filenames

How

#

O_O

#

It's just 11 characters

#

kekw

flint idol Jan 14, 2024, 3:49 PM

#

Space padding

vernal chasm Jan 14, 2024, 3:49 PM

#

But that's only on the ends tho

flint idol Jan 14, 2024, 3:49 PM

#

Yeah

#

Exactly

#

I copy 8 characters into the filename iirc

#

And then append a dot

#

And then copy the extension

vernal chasm Jan 14, 2024, 3:50 PM

#

Oof

flint idol Jan 14, 2024, 3:50 PM

#

I'd assume that 8.3 entries cannot have a space in the filename because of that

vernal chasm Jan 14, 2024, 3:53 PM

#

I mean since there's no length specifier or null terminator for it yeah

#

I mean they could've just had a null terminator up to 8 in length, but no it's got to be partially space terminated lmao

flint idol Jan 14, 2024, 4:18 PM

#

Idk how or why but my PCI identification code just stopped working

#

I'll just rebuild and see what happens

#

Doesn't fix the problem

#

It's a bug with the initrd driver (or a problem with the initrd)

#

Just discovered a bug with the mount code

#

If partitionId = 0, it assumes initrd

#

But partitionId=0 can also mean Drive Id=0 Partition Id=0

#

It's rather a bug with mount or the initrd

#

cache->dataEnd = cache->dataStart + ((filesize / 512 + 1) * 512);

#

that is the offending line

#

I add one even if the filesize is aligned to 512

#

This should fix it:
cache->dataEnd = cache->dataStart + ((filesize / 512 + (filesize % 512 != 0)) * 512);

#

Yeah that fixed that bug

#

... and caused another

#

cache->entryFilesize++; // Add one to the filesize.

thonk

#

Why does that exist

#

🤦‍♂️

#

It was to fix this when nToRead == cache->filesize:

if ((cache->dataStart + nToSkip + nToRead) >= cache->dataEnd)

flint idol Jan 14, 2024, 5:06 PM

#

Fixed those bugs

#

Time to test on real hardware

#

I get a GPF then triple fault

#

It also page faults

#

Do you know I have a fat date
||c++ struct fat_date { byte day : 5; byte month : 4; byte year1980 : 7; } __attribute__((packed));||

vernal chasm Jan 14, 2024, 5:55 PM

#

flint idol Do you know I have a fat date ||```c++ struct fat_date { byte day : 5; ...

kekw

flint idol Jan 14, 2024, 7:56 PM

#

After some more bug fixes, I gpf on one device, and I boot successfully boot on another

#

Except the other doesn't have a fat32 partition

flint idol Jan 14, 2024, 8:23 PM

#

After some more bug fixes, I get OBOS_ERROR_ALREADY_EXISTS while mounting the partition

flint idol Jan 15, 2024, 11:20 PM

#

#1141057599584878645 message
Hehe whoops

#

I went a "bit" off track

#

I wonder what will happen if I load the kernel as a process and run it

#

Might as well try

#

I think it will fail while mapping the kernel because, well it's already mapped

#

Yeah nvm I'm not going to try, that's basically guarenteed to happen

#

After fixing an allocator bug, I'm going to see if I can read from disk

#

Ok now it's just being annoying

#

Kind of makes me wish I could debug the kernel

#

on real hardware

#

I think it might be an access on block->lastBlock which can technically be nullptr

#

Ok now I'm being messed with

#

This is actually annoying

#

I have checks for null

#

And it's faulting on null

#

I fixed that bug

#

However I can't read from the FAT32 partition (rather that or it is the wrong partition)

#

I think I'm causing SeaBios to triple fault lol

#

I seem to have a bug with the fat driver

#

On real hardware

#

To put it simply, no directory entries are listed

#

If only I could use a gdb stub...

#

As even if I had a working one I don't have a computer with a serial port

#

and like I said, I'm not implementing EHCI for some time

flint idol Jan 16, 2024, 12:36 AM

#

flint idol To put it simply, no directory entries are listed

Nvm I wasn't printing them lol

#

Would a nic driver be too hard to work on if I chose a "simple" card and did nothing with a network stack

#

That could work for a gdb stub on real hardware

rapid pulsar Jan 16, 2024, 12:38 AM

#

e1000 series is pretty straightforward, and is the default in QEMU these days, and you have a chance of it working on real hardware

flint idol Jan 16, 2024, 12:40 AM

#

I'll see if I have that on the test computer

#

It has a "RTL810xE PCI Express Fast Ethernet controller"

#

Is that the https://wiki.osdev.org/RTL8169 that I found on the osdev wiki

rapid pulsar Jan 16, 2024, 12:46 AM

#

Maybe. Check the PCI IDs. Realtek hardware has a lot of similar-sounding names that are totally unrelated chipsets. It's kinda cursed.

flint idol Jan 16, 2024, 12:46 AM

#

Also if I connect my test laptop to my dev computer will I destroy anything

#

with an ethernet cable

rapid pulsar Jan 16, 2024, 12:47 AM

#

No, it is highly unlikely you would be able to do anything in software that would cause hardware damage over a modern NIC (or any NIC, for that matter).

flint idol Jan 16, 2024, 12:48 AM

#

I just did and neither sides recognize a connection

rapid pulsar Jan 16, 2024, 12:51 AM

#

You can't connect two NICs to each other directly with a normal Ethernet cable. You need a cable that swaps the transmit and receive wires around, called a crossover cable. You need to use a switch or hub if you have standard patch cables.

#

(You may be able to use a standard patch cable with support on the NIC hardware, but it's likely support you need to enable specifically in your driver.)

#

(Technically, modern gigabit-and-better Ethernet doesn't actually have dedicated "transmit" and "receive" wiring, but it relies on the slower standards as part of autonegotiation or the two NICs won't be able to figure out how to talk to each other.)

flint idol Jan 16, 2024, 1:02 AM

#

Well this is weird, the contents of the first FAT32 partition mounted (efi system partition) and the second FAT32 partition I mounted (the one I made) have the exact same contents

#

Rather a bug with mount or the fat driver

#

Seems like a bug with mount

#

Judging by the output of another computer that has a FAT32 partition and a fat32 partition (that was falsely recognized as FAT12 because of the cluster count) returned the same stuff for both mount points

#

and well my fat driver doesn't support FAT12 yet

#

Tbh partition IDs for mount points now make absolutely no sense

#

I'm going to remake that

weary hound Jan 16, 2024, 1:22 AM

#

rapid pulsar (You _may_ be able to use a standard patch cable with support on the NIC hardwar...

just put a switch in between :^)

#

most newer ones (even cheap ones) should support auto mdix

rapid pulsar Jan 16, 2024, 1:22 AM

#

You need to use a switch or hub if you have standard patch cables.

weary hound Jan 16, 2024, 1:23 AM

#

oh i lack the ability to read

#

apologies

flint idol Jan 16, 2024, 2:16 AM

#

The kernel heap is extremely fragmented

#

I'll fix it later

empty kernel Jan 16, 2024, 8:08 AM

#

flint idol The kernel heap is extremely fragmented

I've been having the same problem with my kernel heap. It gets very fragmented after the VFS is initialised.

devout niche Jan 16, 2024, 11:15 AM

#

flint idol The kernel heap is extremely fragmented

what allocation strategies are used?

#

the slab allocator is famous for its properties in this

vernal chasm Jan 16, 2024, 12:13 PM

#

Yeah perhaps you're using a somewhat unoptimal allocation strategy

flint idol Jan 16, 2024, 12:13 PM

#

Or unusual

vernal chasm Jan 16, 2024, 12:14 PM

#

I know as a fact that whatever I'm making is and probably will cause some insane amounts of fragmentation if you're like "p = Alloc(1); n = Alloc(1); Free(p); p = Alloc(1); Free(n)" kekw

flint idol Jan 16, 2024, 9:19 PM

#

Maybe the way to stop the fragmentation is by stopping reallocating like 8 pages at once with kmalloc and change to the VMM

flint idol Jan 16, 2024, 10:14 PM

#

I think I might've fixed that bug

#

I also made a change to krealloc to see if there is space after the block

#

I wish I'd known about gdb's monitor command earlier

#

Like when I was making my SMP trampoline

#

Because reasons I had to do x/i $pc in the qemu monitor

#

and not in gdb

#

Anyway now I know

#

Time to test on real hardware

#

It better work!

#

It doesn't work

#

At least it doesn't page fault

#

I think I'm going to implement backbuffering in my console

#

It takes forever for text to be printed if it needs to scroll

flint idol Jan 16, 2024, 11:42 PM

#

That failed terribly lol

#

Triple faulted before even printing anything

#

Most likely before even the gdt was initialized

#

It's concerning how almost everything implicitly includes console.h

#

and of course the only source file that actually needs it doesn't include it

#

(klog.cpp)

flint idol Jan 16, 2024, 11:48 PM

#

flint idol Triple faulted before even printing anything

I fixed that bug

flint idol Jan 17, 2024, 12:03 AM

#

That didn't work well...

#

What's weird is it's actually at the point where the fat driver gets initialized

flint idol Jan 17, 2024, 12:21 AM

#

Uhhh that doesn't look right

#

For some reason it doesn't like printing correctly on some lines

#

void Console::putChar(char ch, uint32_t x, uint32_t y, uint32_t fgcolor, uint32_t bgcolor)
{
    int cy;
    int mask[8] = { 128,64,32,16,8,4,2,1 };
    const uint8_t* glyph = m_font + (int)ch * 16;
    y = y * 16 + 16;
    x <<= 3;
    if (x > m_drawingBuffer->width)
        x = 0;
    if (y > m_drawingBuffer->height)
        y = 0;
    for (cy = 0; cy < 16; cy++) 
    {
        uint32_t realY = y + cy - 12;
        uint32_t* framebuffer = m_drawingBuffer->addr + realY * (m_drawingBuffer->pitch / 4);    
        framebuffer[x + 0] = (glyph[cy] & mask[0]) ? fgcolor : bgcolor;
        framebuffer[x + 1] = (glyph[cy] & mask[1]) ? fgcolor : bgcolor;
        framebuffer[x + 2] = (glyph[cy] & mask[2]) ? fgcolor : bgcolor;
        framebuffer[x + 3] = (glyph[cy] & mask[3]) ? fgcolor : bgcolor;
        framebuffer[x + 4] = (glyph[cy] & mask[4]) ? fgcolor : bgcolor;
        framebuffer[x + 5] = (glyph[cy] & mask[5]) ? fgcolor : bgcolor;
        framebuffer[x + 6] = (glyph[cy] & mask[6]) ? fgcolor : bgcolor;
        framebuffer[x + 7] = (glyph[cy] & mask[7]) ? fgcolor : bgcolor;
    }
}```

#

I don't see why that could fail

#

I did have optimizations on for that function only, so I'll disable them and see what happens

#

Same effect

flint idol Jan 17, 2024, 12:38 AM

#

flint idol For some reason it doesn't like printing correctly on some lines

Specifically every other line

#

It seems to not like the first line either

#

This seems like a bug with SwapBuffers

#

It is a bug with SwapBuffers

#

After I changed it to not use the modification bitfield

#

Ok I fixed it

#

Except it doesn't like the first character of the terminal when clearing the screen

#

and still page faults in putChar while printing something

#

Specifically when scrolling the terminal it seems

#

I think there was a silent memory corruption bug with my terminal scroll function

#

and I think it is y--;

#

It's so weird now

#

Because every three lines, it swaps the buffers, the text is slightly outdated on the screen

#

but I see the current text on the debug con console thing

flint idol Jan 17, 2024, 1:08 AM

#

idk how or why but a function is being skipped in the stack trace

#

Ok I fixed that

#

It was because an assembly function omitted setting up the stack frame (push rbp mov rbp, rsp)

flint idol Jan 17, 2024, 1:54 AM

#

Ok the scrolling code does not work correctly

astral mason Jan 17, 2024, 3:08 AM

#

And this my friends, is why we test our OS on hardware often, VERY often.

vernal chasm Jan 17, 2024, 6:45 AM

#

astral mason And this my friends, is why we test our OS on hardware often, *VERY* often.

Too afraid of the hardware getting damaged O_O

astral mason Jan 17, 2024, 7:24 AM

#

vernal chasm Too afraid of the hardware getting damaged O_O

That's fair enough.

#

Can't you just make a different partition though, or does that not help.

vernal chasm Jan 17, 2024, 7:28 AM

#

astral mason Can't you just make a different partition though, or does that not help.

It's more like "This command exists, but you know if you use it incorrectly the device will be destroyed"

#

Tbf I have ended up removing some usb device firmwares by mistake whilst figuring out how to control it...

astral mason Jan 17, 2024, 7:29 AM

#

Oh, uh, I just use the Disks utility on Linux Mint

vernal chasm Jan 17, 2024, 7:32 AM

#

Again not really any issues with disk stuff, cuz for me I could just disconnect my normal drives and leave my linux one (which I haven't touched in about 2 years now)
It's more the other devices my pc depends on that could be destroyed by giving incorrect commands to them... (Mostly just the corsair devices I have which have multiple commands for removing their firmware xD)

astral mason Jan 17, 2024, 7:33 AM

#

vernal chasm Again not really any issues with disk stuff, cuz for me I could just disconnect ...

Oh... that makes more sense, I don't even have a kernel yet |_|; so I guess I don't have to worry about that yet.

limber wave Jan 17, 2024, 1:14 PM

#

vernal chasm Too afraid of the hardware getting damaged O_O

very rare on modern compooters

#

even if you fuck something up most computers nowadays should be able to go through it

#

and not cause any permanent damage

vernal chasm Jan 17, 2024, 1:16 PM

#

limber wave very rare on modern compooters

Thankfully in my case all I had to do was just restart the corsair drivers and it would reinstall the firmware, but you never know lmao

limber wave Jan 17, 2024, 1:16 PM

#

you shouldnt be worried imo

flint idol Jan 17, 2024, 1:51 PM

#

Except when you corrupt more than like 30 kib of data after the frame buffer

limber wave Jan 17, 2024, 1:52 PM

#

it probably wont fuck up anything permanently though]

#

unless the firmware is shite

flint idol Jan 17, 2024, 1:56 PM

#

Yeah I don't think it did send commands to any random device

#

Luckily

#

I gtg now

limber wave Jan 17, 2024, 1:57 PM

#

bye

#

i wrote /dev/urandom to /dev/ports and my thinkpad died for one boot but got itself together the next time

limber wave Jan 17, 2024, 3:52 PM

#

limber wave i wrote /dev/urandom to /dev/ports and my thinkpad died for one boot but got its...

what is /dev/ports?

limber wave Jan 17, 2024, 5:12 PM

#

limber wave what is /dev/ports?

memory mapped io ports

limber wave Jan 17, 2024, 5:33 PM

#

limber wave memory mapped io ports

Oh ok, thanks

flint idol Jan 17, 2024, 9:27 PM

#

Yeah I can't figure out why this doesn't scroll the terminal

#

utils::dwMemcpy(
    m_drawingBuffer->addr,
    m_drawingBuffer->addr + m_drawingBuffer->pitch / 4 * 16,
    (static_cast<size_t>(m_drawingBuffer->height) - 16) *
    m_drawingBuffer->pitch / 4
    );
utils::dwMemset(
    m_drawingBuffer->addr + (m_drawingBuffer->pitch / 4 * 16),
    0,
    m_drawingBuffer->pitch / 4
);```

#

the dw prefix means dword if it weren't obvious

#

The font size is assumed to be 8x16

#

BPP is assumed to be 32

#

I should probably give the output

#

#OBOS (not vibecoded)