OBOS (not vibecoded) | Operating System Development | Page 13

flint idol Aug 4, 2024, 2:15 AM

#

this is quite annoying

#

I need to send an NMI

#

then call the debug exception handler

main girder Aug 4, 2024, 2:16 AM

#

do you mean freezing all the other processors

flint idol Aug 4, 2024, 2:16 AM

#

yes

flint idol Aug 4, 2024, 2:16 AM

#

flint idol then call the debug exception handler

which freezes the processor

main girder Aug 4, 2024, 2:16 AM

#

thats not that hard i did that in like an hour

flint idol Aug 4, 2024, 2:16 AM

#

yeah it's not hard

#

just annoying

main girder Aug 4, 2024, 2:16 AM

#

not really

flint idol Aug 4, 2024, 2:16 AM

#

I find it annoying

main girder Aug 4, 2024, 2:16 AM

#

just do it.

flint idol Aug 4, 2024, 2:16 AM

#

I'm basically done

main girder Aug 4, 2024, 2:16 AM

#

flint idol I find it annoying

then you are allowing it to wear you down.

flint idol Aug 4, 2024, 2:20 AM

#

only one cpu decides to freeze

#

the others don't wanna

#

oops

#

I really gotta stop writing code that's kind of critical to the kernel while I'm tired

#

now all the cpus are frozen

#

except for the interrupted one

#

and now my thread view is broken

#

how delightful

#

ok it's fixed

#

except the main thread and the bsp's idle thread have the same registers

#

wait I know why

#

I now get a correct snapshot of the kernel

#

when it's interrupted

flint idol Aug 4, 2024, 2:33 AM

#

flint idol except the main thread and the bsp's idle thread have the same registers

*in gdb

#

gdb_ctx dbg_ctx = current_g_thread->masterCPU->arch_specific.dbg_ctx;
if (current_g_thread->masterCPU == CoreS_GetCPULocalPtr() && current_g_thread != Core_GetCurrentThread())
    dbg_ctx.interrupt_ctx = current_g_thread->context;```

main girder Aug 4, 2024, 2:33 AM

#

this is one cool thing about an nt-like irql hierarchy is that for one IPIs are always serviced almost instantly which is cool and for two that means the debugger can freeze everyone successfully even if someone was stuck deadlocked on a spinlock or something

#

because they didnt disable interrupts before trying to grab the spinlock they just raised irql

#

a bit

#

and IPI can still come in

flint idol Aug 4, 2024, 2:34 AM

#

speaking of spinlock, I (think) I need to change the default irql it raises to from IRQL_MASKED to IRQL_DISPATCH

main girder Aug 4, 2024, 2:35 AM

#

whats the former

flint idol Aug 4, 2024, 2:35 AM

#

IRQL 15

#

all irqs are masked

main girder Aug 4, 2024, 2:35 AM

#

yeah that defeats the point cuz youre gonna be holding spinlocks a lot in like the scheduler

#

which is then no longer interruptible

#

if spinlocks mask all interrupts

flint idol Aug 4, 2024, 2:36 AM

#

iirc I set it to do IRQL_MASKED before the scheduler and never bothered to change it after

#

I did that without anything breaking

#

on the topic of locks, I need a mutex

#

I'll do that tomorrow

flint idol Aug 4, 2024, 3:01 AM

#

I can now continue execution in the kernel

#

all that's left for code execution is step

#

then there's also breakpoints

#

idk how that works

#

wait how does it work

#

I don't see any special packets here

#

it's sent on continue

flint idol Aug 4, 2024, 3:37 AM

#

the hashmap has decided it doesn't like the string 'vMustReplyEmpty'

#

as in it hangs if you try to touch the hashmap with that key

#

I'm too tired for this shit

#

wait it's not the key

#

it's the amount of elements in the hashmap

#

yeah taht was it

flint idol Aug 4, 2024, 3:09 PM

#

flint idol it's the amount of elements in the hashmap

I seem to have forgotten to add in a calculation when I was making the hashmap use fixed point

#

which made the growat variable be some very large number

hybrid kraken Aug 4, 2024, 3:16 PM

#

Threaded DPCs are passive level DPCs that run on a priority 31 (maximum) thread

#

They either work like this if globally enabled, or if disabled they work like normal DPCs

flint idol Aug 4, 2024, 6:09 PM

#

the gdbstub is basically complete

#

just needs some final packets

#

it can step

#

(albeit very slowly)

#

I also need to implement interrupting the kernel when gdb requests it

#

maybe I'll just throw a hook into the com driver

#

on irq, yell at the gdb stub to wake up

#

from these, I need:
M, G, and qRcmd

#

and then I also need to fix eflags

#

ok I fixed it

#

I also gotta fix some memory corruption bugs in the gdb stub

main girder Aug 4, 2024, 6:16 PM

#

hybrid kraken Threaded DPCs are passive level DPCs that run on a priority 31 (maximum) thread

Yeah so additionally they form an extra (lower) dpc priority level

#

But they can't block or anything because they still need to be able to run as normal dpcs

#

If they could block they'd just be a worse version of work queue items

hybrid kraken Aug 4, 2024, 6:41 PM

#

main girder But they can't block or anything because they still need to be able to run as no...

Well normal DPCs can preempt the threaded DPC function as well as APCs and other highest-priority real-time threads.

#

Tbh almost no one uses threaded DPCs and never has

#

They also require more irql checks around sync functions

main girder Aug 4, 2024, 6:48 PM

#

hybrid kraken Well normal DPCs can preempt the threaded DPC function as well as APCs and other...

I am pretty sure threaded dpcs can't be preempted by threads but I could be wrong

lunar narwhal Aug 4, 2024, 6:51 PM

#

at least now they can identify long dpcs iirc

#

and do something like use normal thread? i forgot what resolution the long dpcs have

flint idol Aug 4, 2024, 9:29 PM

#

my recv packet code is kinda borked

#

the packet size is -3

#

in one specific case

#

the checksum offset is -1

#

I suspect the bug is somewhere here:


while(offset < szPacket && raw_packet[0] != '$')
{
    raw_packet++;
    offset++;
}
checksumOffset -= offset;
if (offset == szPacket)
{
    ack = '-';
    goto acknowledge;
}
uint8_t checksum = KdbgH_hex2bin(&raw_packet[checksumOffset], 2);
size_t packetSz = checksumOffset-2;```

#

szPacket (the raw packet) was 9

#

offset was 8

#

raw_packet was "$"

#

and of course I can't get the bug to reproduce

#

the exact commands to get this were:

target remote :1534
info reg
c
[crashes here]```

#

yeah that reproduces the bug

#

it's because there is utter bs being received

#

vCont?#+$

#

if you couldn't tell that isn't a valid packet

flint idol Aug 4, 2024, 9:58 PM

#

it seems to have disappeared

#

after changing something

#

so I tried to implement interrupting the kernel if gdb asks it

#

and it half-works

#

as in the kernel debugger does something

#

it sends the response

#

that it was interrupted

#

but then ignores everything else

#

o wait it crashes

#

ok I fixed it

#

gdb is being weird

#

thread one has died

#

it recognizes that

#

then when I do info thread it still shows the thread

#

probably just the kernel sending it despite it being dead

#

but goddamn is this slow

#

it takes half a second for it to tell me that the kernel was interrupted

#

next is terribly slow

#

so is step

flint idol Aug 4, 2024, 10:07 PM

#

flint idol probably just the kernel sending it despite it being dead

that was it

#

KASAN is being stupid

flint idol Aug 4, 2024, 10:29 PM

#

flint idol `vCont?#+$`

I fixed this bug

#

by adding a check for szPacket-offset < minimumPacketSize

#

and sending NACK if it's true

#

now I want to implement more proper stepping

flint idol Aug 4, 2024, 10:54 PM

#

done

#

now if I step one thread, only that thread steps

#

I think the problem was the DPC init function thought that the dpc was already enqueued

#

since the cpu field was set

#

because it's never cleared in the dpc dispatcher

#

and

if (dpc->cpu && LIST_IS_NODE_UNLINKED(dpc_queue, &dpc->cpu->dpcs, dpc))
        return OBOS_STATUS_DPC_ALREADY_ENQUEUED;```

#

holy shit with the com driver using DPCs, the gdbstub is slow af

#

how about I flush the data to be sent (only done when the com port's output buffer is full) in the dpc

#

and also break into the gdb stub if it was requested

#

in the dpc handler

#

implementing breakpoints rn

#

I just made the world's simplest disassembler:

uint8_t offset = 0;
if (*(uint8_t*)(frame->rip - 1) == 0xcc /*int3*/)
    offset = 2; // ... int3 (0xcc xx)
if (*(uint8_t*)(frame->rip - 1) == 0x3)
    offset = 3; // ... int 3 (0xcd, 0x03, xx)

#

if anyone was wondering, this is to adjust rip to before the int3 instruction for gdb

#

bruh

#

gdb just crashed trying to read symbols

#

I love it when the n command randomly breaks /s

#

it only breaks on int3

#

it seems

#

yeah I fixed it

#

I was setting the wrong rip

#

I just fixed wrong uses of IRQL_MASKED around the kernel

#

most of them should've been IRQL_DISPATCH

#

oh great

#

now it's broken

#

I fixed it it seems

#

and also all bugs related to this specific bug

#

now I get a recursive lock

#

ok I fixed that

#

TODO: Is keeping the irq interface unlocked while the irq dispatcher is running a good idea

#

idk if I'm just hallucinating, but the kernel is visibly faster when I removed that lock

#

well it's not removed, it's just not acquired within the irq dispatcher at all anymore

flint idol Aug 5, 2024, 12:23 AM

#

I think I know how to implement watch points

#

without hw breakpoints

#

I'll just use KASAN

#

since it captures basically every single read and write

flint idol Aug 5, 2024, 12:39 AM

#

I've implemented breakpoints

#

now I gotta test them

flint idol Aug 5, 2024, 1:02 AM

#

amazing

#

they work*

#

*except it crashes when you try to continue

#

fixed

#

it was because the rip was being incorrectly set

#

nvm I can't read

#

I think I fixed it

#

kernel has 16k loc

#

over 124 files

#

74 of which are compiled

#

41 of those files are from the kernel

#

the rest are from uACPI

flint idol Aug 5, 2024, 1:25 AM

#

flint idol I think I fixed it

50/50 chance of it working

#

nvm 0% chance

#

completely forgot about instruction cache

#

I modified the instruction through the hhdm to avoid having to set the page mapping

#

which was probably causing the cpu to be confused

main girder Aug 5, 2024, 1:56 AM

#

flint idol which was probably causing the cpu to be confused

on x86?

flint idol Aug 5, 2024, 1:56 AM

#

yes

main girder Aug 5, 2024, 1:56 AM

#

no

#

x86's icache is coherent

#

fully

#

and it would need to be virtually tagged for accesses thru varying virtual addresses to confuse it

#

and its not

flint idol Aug 5, 2024, 1:57 AM

#

main girder and it would need to be virtually tagged for accesses thru varying virtual addre...

wdym virtually tagged

main girder Aug 5, 2024, 1:58 AM

#

a thing that x86 does not do

flint idol Aug 5, 2024, 1:58 AM

#

dam

main girder Aug 5, 2024, 1:58 AM

#

further information can be found on google

flint idol Aug 5, 2024, 2:01 AM

#

the kernel just imploded :)

rose mortar Aug 5, 2024, 2:05 AM

#

flint idol I suspect the bug is somewhere here: ```c while(offset < szPacket && raw_packet...

here's my battle tested thing if interested
https://github.com/FlorenceOS/BED/blob/master/src/lib/gdb.zig#L28-L66

flint idol Aug 5, 2024, 2:05 AM

#

that bug was fixed

#

but ty

rose mortar Aug 5, 2024, 2:07 AM

#

ah okay cool

#

grats and good luck

flint idol Aug 5, 2024, 2:07 AM

#

ty

flint idol Aug 5, 2024, 3:05 AM

#

ok I fixed it

#

I was doing something smooth-brained

#

I was setting the rip in the debug context in the bp instruction handler

#

instead of the rip in the frame

#

causing stoobid stuff to happen

#

this is so slow 😭

#

the gdb stub can be used

#

now

#

but I just need to:

make it faster (specifically stepping and stopping)
implement write to memory
implement write registers

#

maybe if I give the s (step) packet a shortcut into getting handled (as in, it doesn't need to go through the hashmap), it'll run faster

#

oh god

#

it steps so much with the step command

#

many many instructions

flint idol Aug 5, 2024, 3:30 AM

#

flint idol it steps so much with the `step` command

and on each step, it queries the registers as well

#

that's not very fast

#

maybe a 130 byte format string isn't good for sprintf

#

// rax,rbx,rcx,rdx,rsi,rdi,rbp,rsp
"%016lx%016lx%016lx%016lx%016lx%016lx%016lx%016lx"
// r8-r15
"%016lx%016lx%016lx%016lx%016lx%016lx%016lx%016lx"
// rip, eflags, cc, ss, ds, es, fs, gs
"%016lx%08x%08x%08x%08x%08x%08x%08x"```

#

I changed that

#

and now everything's acting stoobid

#

except for the actual register view

#

all goes to shit

#

as to why all goes to shit, I suspect that the timeout for receive is too small, so it just sends a nack after like 3 seconds

#

confusing the shit outta gdb

#

making the timeout longer seems to aid it

#

I need to remove the timeout, but doing so causes it to hang occasionally

#

when I wake up I'll do that

#

rn I'm too tired to deal with this shit

flint idol Aug 5, 2024, 9:37 AM

#

flint idol I need to remove the timeout, but doing so causes it to hang occasionally

I think I might just rewrite the recv packet

#

*function

#

Since it's so unreliable

flint idol Aug 5, 2024, 9:59 AM

#

flint idol I think I might just rewrite the recv packet

Basically how it'll work now is it it will discard all bytes until it finds a $ (start of packet), then it'll read all into a buffer, and change any escaped chars back to normal. While the chars are being read, the mod256 checksum is being calculated. It stops reading when it finds a # (start of checksum), then it reads the next two bytes into the a remote checksum variable. At that point, it verifies the checksum, if they match the function sends an ACK and returns, otherwise it will send a NACK and try again.

flint idol Aug 5, 2024, 11:52 AM

#

flint idol Basically how it'll work now is it it will discard all bytes until it finds a $ ...

I implemented that

#

the kernel doesn't go to shit

#

but instead the gdbstub gives up on responding to gdb after a bit

#

I made the tlb shootdown ipi async

#

which was of course the reason of the gdb stub hanging

#

now step is being funny

#

probably after I fixed the H packet

#

because gdb chose all threads when it sent the s command

#

Hc-1

#

which causes the gdb stub to do this when the step command is set:

for (size_t i = 0; i < Core_CpuCount; i++)
{
    Core_CpuInfo[i].arch_specific.dbg_ctx.interrupt_ctx.frame.rflags |= RFLAGS_TRAP;
    Core_CpuInfo[i].arch_specific.dbg_ctx.interrupted_thread->flags &= ~THREAD_FLAGS_DEBUGGER_BLOCKED;
}```

#

and for some reason I get random traps

#

and also the gdb stub gives up eventually:

w +$qsThreadInfo#c8
r <Timeout: 2 seconds>
w $qsThreadInfo#c8
r -
w $qsThreadInfo#c8
r <Timeout: 2 seconds>
w $qsThreadInfo#c8
r -<Timeout: 2 seconds>
w -
r <Timeout: 2 seconds>
w -
r <Timeout: 2 seconds>
w -+```

#

(r is received packet, w is sent packet)

#

I fixed it

#

I forgot to put in a

goto retry;

#

#

stupid gdb

#

well lldb hasn't crashed yet

#

(I wonder why)

#

except the back trace is broken (doesn't exist)

#

that was because there was no symbols loaded

flint idol Aug 5, 2024, 3:17 PM

#

I'm almost done implementing the M packet

#

it works

#

now to see if it works within gdb

#

*lldb

flint idol Aug 5, 2024, 3:47 PM

#

this stack trace doesn't quite make sense

#

ah yes, flush idt called initialize irq interface

#

actually that might make sense

#

except it doesn't

#

ah yes thread starvation prevention (todo: rename to priority booster) called the LAPIC default irq handler

#

thinkong

#

gdb doesn't have the same issue

#

I've implement the monitor command:

#

technically

flint idol Aug 5, 2024, 4:05 PM

#

flint idol now to see if it works within gdb

it works

#

now for the G (write registers) packet

#

nooo

#

that's done

flint idol Aug 5, 2024, 4:31 PM

#

ok VFS is next after I make the gdb stub faster

flint idol Aug 5, 2024, 5:05 PM

#

vfs is booooooring

#

time for m68k port trl

inland radish Aug 5, 2024, 5:05 PM

#

of course it is

devout niche Aug 5, 2024, 5:06 PM

#

flint idol time for m68k port <:trl:829316641191886888>

based plan

flint idol Aug 5, 2024, 5:07 PM

#

indeed

#

now where to find m68k bootloader

#

@ornate ginkgo you said you had a limine stub

#

I won't bother learning m68k asm

#

because hopefully I won't need too much of that

#

hopefully I didn't make the kernel too unportable

devout niche Aug 5, 2024, 5:17 PM

#

flint idol because hopefully I won't need too much of that

the entirety of the keyronex m68k codebase appears to use only move, movec, movem, add, sub, jsr, pflush, ptest, rts, and rte

flint idol Aug 5, 2024, 5:17 PM

#

that isn't much at all

#

so I think I'll be fine

#

also where can I find a m68k btldr

#

I don't want to make my own (but I will if I have to)

devout niche Aug 5, 2024, 5:18 PM

#

i would use r4's for now

flint idol Aug 5, 2024, 5:18 PM

#

and where is that

devout niche Aug 5, 2024, 5:19 PM

#

flint idol so I think I'll be fine

it's considered by most to be by far the most enjoyable assembly language ever devised so it will be a pleasant experience

flint idol Aug 5, 2024, 5:19 PM

#

which one of the m68k processors had an mmu

#

I think it was the '040

#

Right now I need:

A bootloader
A cross compiler

#

I'll compile the cross compiler while I look for r4's bootloader

flint idol Aug 5, 2024, 6:03 PM

#

@devout niche Will I need libgcc for an m68k-elf toolchain?

#

I'm trying to compile it and I am getting:

checking for suffix of object files... configure: error: in `/home/serveradmin/m68k-gcc/build-gcc/m68k-elf/libgcc'```

#

gcc and binutils compiled

devout niche Aug 5, 2024, 6:09 PM

#

flint idol <@391791686881247234> Will I need libgcc for an m68k-elf toolchain?

very useful to have

#

what's it erroring on?

flint idol Aug 5, 2024, 6:10 PM

#

flint idol I'm trying to compile it and I am getting: ``` checking for suffix of object fil...

this

#

make all-target-gcc is the command I ran

devout niche Aug 5, 2024, 6:10 PM

#

how's the error log look?

flint idol Aug 5, 2024, 6:10 PM

#

I'll check

#

configure: error: in `/home/serveradmin/m68k-gcc/build-gcc/m68k-elf/libgcc':
configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details
make: *** [Makefile:13847: configure-target-libgcc] Error 1```

#

I need to scp config.log

#

then I'll paste it

#

📎 config.log

#

libisl isn't installed it seems

#

yeah that was it

flint idol Aug 5, 2024, 6:50 PM

#

I got a m68k toolchain compiled

#

now time for qemu

#

I got qemu

#

that finished really quickly

flint idol Aug 5, 2024, 7:20 PM

#

@devout niche Do I need to emulate 16 IRQL levels on the m68k

#

or can I map the 8 it has to the 16 IRQL levels needed

#

actually maybe that was some random thing

devout niche Aug 5, 2024, 7:23 PM

#

flint idol <@391791686881247234> Do I need to emulate 16 IRQL levels on the m68k

if you want 16 you'd have to emulate

#

the -M virt of qemu unfortunately has quite an inflexible interrupt controller (the so-called GoldFish, which actually comes from the android emulator) and doesn't allow useful things like the ability to raise a softint

#

on the other hand the amiga's CIA interrupt controllers do

flint idol Aug 5, 2024, 7:26 PM

#

devout niche if you want 16 you'd have to emulate

ok

#

it's actually quite nice that my IRQ interface has an enterirqhandler

#

because I can use that to emulate the IRQLs

flint idol Aug 5, 2024, 7:51 PM

#

I'm able to get cmake to compile m68k code

#

except nothing compiles

#

because of some kasan option

#

-fasan-shadow-offset

#

I set that to zero (TODO: set it to something proper)

#

I need to add a 32-bit elf header

#

this'll be fun

#

currently 101 errors reported (so far)

#

I fixed a couple

#

these are mainly from macro/struct defines not existing

#

since some are arch-specific

flint idol Aug 5, 2024, 9:33 PM

#

flint idol <@337185227719966721> you said you had a limine stub

well I found it

#

I don't understand it one bit though

#

ok I get it now

#

northport is MIT license

#

so I'm snatching the loader

ornate ginkgo Aug 5, 2024, 9:55 PM

#

Go for it, that's the beauty of open source

flint idol Aug 5, 2024, 9:57 PM

#

Currently I've stolen two stuff from northport

ornate ginkgo Aug 5, 2024, 9:57 PM

#

Things to know about it:

I recall that getting a PIE kernel running was a bit flaky, depending on which version of binutils it was linked with. The generated relocations were wrong.
it doesn't do 8k pages, even if you request them.

flint idol Aug 5, 2024, 9:57 PM

#

Bootloader and panic ascii art

flint idol Aug 5, 2024, 9:58 PM

#

ornate ginkgo Things to know about it: - I recall that getting a PIE kernel running was a bit...

Well neither of those should be a problem for me

ornate ginkgo Aug 5, 2024, 9:58 PM

#

I'm slowly infiltrating your codebase

flint idol Aug 5, 2024, 9:58 PM

#

O_O

ornate ginkgo Aug 5, 2024, 10:01 PM

#

Ah and of course it uses some limine protocol defs I just made up for m68k, theyre available around here https://github.com/DeanoBurrito/northport/blob/master/kernel/include/boot/Limine.h#L256 and https://github.com/DeanoBurrito/northport/blob/master/kernel/include/boot/Limine.h#L362

flint idol Aug 5, 2024, 10:01 PM

#

Okie

flint idol Aug 5, 2024, 11:38 PM

#

I got the kernel to compile

#

but not link

#

because there's no libgcc

#

but after I get that I'm getting bombarded with linker errors

#

to "solve" those, I'm marking the target-specific symbols as weak

#

so I can gradually implement them

#

while I'm waiting for libgcc I'll mark them as weak

#

and theoretically, I'll get a working kernel once I implement all arch-specific stuff for the m68k

ornate ginkgo Aug 5, 2024, 11:49 PM

#

flint idol to "solve" those, I'm marking the target-specific symbols as weak

You can also implement them with a call to panic("not implemented")

#

That way you'll know how far you're getting

flint idol Aug 5, 2024, 11:49 PM

#

nullptr is basically the same thing

ornate ginkgo Aug 5, 2024, 11:49 PM

#

Lol

flint idol Aug 5, 2024, 11:49 PM

#

wait does your limine stub give a framebuffer

#

if it's avaliable

ornate ginkgo Aug 5, 2024, 11:49 PM

#

It does not

flint idol Aug 5, 2024, 11:49 PM

#

shi

#

how do I get debug locks

#

*logs

ornate ginkgo Aug 5, 2024, 11:50 PM

#

The only framebuffer on m68k right now is virtio, bewides board specific stuff

#

Ramfb is coming in a future patch though

flint idol Aug 5, 2024, 11:50 PM

#

b r u h
b r u h
b r u h
b r u h

flint idol Aug 5, 2024, 11:50 PM

#

ornate ginkgo Ramfb is coming in a future patch though

ok

ornate ginkgo Aug 5, 2024, 11:50 PM

#

Future patch for qemu I should say*

flint idol Aug 5, 2024, 11:51 PM

#

so how do I get logs again?

ornate ginkgo Aug 5, 2024, 11:51 PM

#

For logs there's the goldfish devices

#

Take a look at the tty

#

Also the documentation is completely wrong about some of the goldfish stuff (the pic and timer I think), so don't be afraid to reference northport/keyronex/qemu

ornate ginkgo Aug 5, 2024, 11:52 PM

#

ornate ginkgo Take a look at the tty

I have a very minimal implementation of using it in the NPL_LOG macro in the loader, iirc its in until.h

flint idol Aug 5, 2024, 11:53 PM

#

okie

#

I'll take a look at it as soon as I fix libgcc

#

rn my priority is getting it to link

#

checking for suffix of object files... configure: error: in `/home/serveradmin/m68k-gcc/build-gcc/m68k-elf/libgcc':
configure: error: cannot compute suffix of object files: cannot compile
See `config.log' for more details
make: *** [Makefile:13847: configure-target-libgcc] Error 1```

#

thinkong

ornate ginkgo Aug 5, 2024, 11:56 PM

#

Hmm

#

I don't have much experience with this part, it worked first time for me

flint idol Aug 5, 2024, 11:56 PM

#

some random stack overflow answer said to add this:

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/lib/x86_64-linux-gnu/```

#

nope

ornate ginkgo Aug 5, 2024, 11:57 PM

#

How are you configuring gcc?

flint idol Aug 5, 2024, 11:57 PM

#

../gcc/configure "--target=$TARGET" "--prefix=$PREFIX" --disable-nls --enable-languages=c,c++ --without-headers```

#

TARGET is m68k-elf

ornate ginkgo Aug 5, 2024, 11:57 PM

#

Non solution, but have you tried using clang in the meantime?

flint idol Aug 5, 2024, 11:58 PM

#

nope

#

(doesn't mean I will, too much work)

#

I think it worked after reconfiguring it

#

nope

#

I ran distclean

#

then reconfigured

#

and am now waiting for it to finish

#

this is so slooooooooooooooooooow

#

ahhhhhhhhhhhhhhhh

#

nvm smth is happening

#

nvm

#

god damn it

#

same error

flint idol Aug 6, 2024, 12:15 AM

#

flint idol this is so slooooooooooooooooooow

considering it decided to rebuild gcc, it wasn't actually too slow

#

I'll just recompile gcc

#

then compile libgcc

#

and see what happens

#

flip

flint idol Aug 6, 2024, 1:15 AM

#

since I was compiling gcc's master branch

#

I decided to checkout the gcc 14.2 tag

#

and see what happens

#

it's currently compiling

ornate ginkgo Aug 6, 2024, 1:16 AM

#

yeah I wouldnt use master for a project like gcc

flint idol Aug 6, 2024, 1:16 AM

#

I'll keep the thread updated

#

when the build finishes

#

bruh

#

I'm just going to delete all my build stuff but the source

#

WAIT

#

it could've been some files were owned by root

#

so it couldn't modify them

#

welp

#

it's too late now

flint idol Aug 6, 2024, 2:13 AM

#

FINALYL

#

libgcc compiled

#

flint idol Aug 6, 2024, 2:44 AM

#

it compiled+linked

#

which is cool I guess

#

except it can't really do shit

#

I need to compile the loader

#

which I will continue tomorrow

#

as I need to port it to be freestanding enough

flint idol Aug 6, 2024, 10:38 AM

#

I shall now start making r4's loader work without northport utility libraries

#

rather that or I'll just copy+paste them

ornate ginkgo Aug 6, 2024, 10:39 AM

#

ah yes, not something I planned for

#

its only headers it depends on

#

good luck

flint idol Aug 6, 2024, 10:40 AM

#

obvious solution is to make OBOS a fork of northport meme

real pecan Aug 6, 2024, 10:43 AM

#

flint idol wait does your limine stub give a framebuffer

hyper m68k stub when

flint idol Aug 6, 2024, 10:43 AM

#

~~bet~~

#

once I learn more about m68k, it could be fun to do that

#

but now I'm pretty clueless when it comes the m68k processor

real pecan Aug 6, 2024, 10:44 AM

#

same but i think its relatively close to x86

#

the i386 one

#

at least the abi from what ive seen

flint idol Aug 6, 2024, 10:44 AM

#

r4 what's LoaderEntryNext

real pecan Aug 6, 2024, 10:44 AM

#

barring big endianness meme

flint idol Aug 6, 2024, 10:44 AM

#

and why is it undefined

real pecan Aug 6, 2024, 10:44 AM

#

flint idol ~~bet~~

based

ornate ginkgo Aug 6, 2024, 10:44 AM

#

real pecan same but i think its relatively close to x86

there's a handful of oddities to it, but its a very easy arch to work with

ornate ginkgo Aug 6, 2024, 10:45 AM

#

flint idol r4 what's LoaderEntryNext

its in Main.cpp

#

definitely defined lol

#

its the high level entry function

real pecan Aug 6, 2024, 10:45 AM

#

ornate ginkgo there's a handful of oddities to it, but its a very easy arch to work with

is it soft refilled tlb?

flint idol Aug 6, 2024, 10:45 AM

#

so my build system is being stupid

ornate ginkgo Aug 6, 2024, 10:46 AM

#

real pecan is it soft refilled tlb?

nah

flint idol Aug 6, 2024, 10:46 AM

#

because I compile main.cpp

ornate ginkgo Aug 6, 2024, 10:46 AM

#

I found the interrupt handling to be the weirdest part

real pecan Aug 6, 2024, 10:46 AM

#

ornate ginkgo nah

ah

ornate ginkgo Aug 6, 2024, 10:46 AM

#

but not bad weird, just different

#

and the page table sizes

real pecan Aug 6, 2024, 10:46 AM

#

what about them?

flint idol Aug 6, 2024, 10:47 AM

#

4k and 8k aren't too weird?

#

is it two level paging

#

?

ornate ginkgo Aug 6, 2024, 10:47 AM

#

kind of

#

iirc the pml2 has 128 entries, while the pml1 is variable in size (depending on page size)

#

and there's indirect PTEs that let you refer to another set of page tables to complete a translation, rather than filling it in with a physical address directly.

real pecan Aug 6, 2024, 10:49 AM

#

uhhhh

real pecan Aug 6, 2024, 10:49 AM

#

ornate ginkgo iirc the pml2 has 128 entries, while the pml1 is variable in size (depending on ...

well on arm these are variable size as well depending on the VA bitness u set

ornate ginkgo Aug 6, 2024, 10:50 AM

#

I've heard about that

#

I know nothing about arm yet 😦

real pecan Aug 6, 2024, 10:50 AM

#

its insane

#

like 4 times more complex than x86, at least imo

flint idol Aug 6, 2024, 10:51 AM

#

TO(NOT)DO: Arm

ornate ginkgo Aug 6, 2024, 10:51 AM

#

damn ok - I was thinking about arm as my next port

real pecan Aug 6, 2024, 10:51 AM

#

its TLB invalidation instruction has like 99999 permutations

#

u have to have a pHD to understand what they do

#

same with atomics

#

99999 trillion barrier types

#

u have to put in correct places

flint idol Aug 6, 2024, 10:52 AM

#

TO(DEFINITELY NOT)DO: Arm

real pecan Aug 6, 2024, 10:52 AM

#

lmao

white mulch Aug 6, 2024, 10:52 AM

#

just do aarch64

flint idol Aug 6, 2024, 10:52 AM

#

obos is unstable enough on x86-64

real pecan Aug 6, 2024, 10:52 AM

#

aarch64 is arm lol

weary hound Aug 6, 2024, 10:53 AM

#

x86 programmers are just too coddled by strong memory ordering halfmemeleft

real pecan Aug 6, 2024, 10:53 AM

#

and MTRRs

#

and bios that works

#

and cache coherency

white mulch Aug 6, 2024, 10:53 AM

#

real pecan aarch64 _is_ arm lol

yeah but the page tables are sane at least (with 48bit va), idk about 32-bit arm

#

its basically the same as on x86_64

real pecan Aug 6, 2024, 10:54 AM

#

yeah page tables are pretty good

real pecan Aug 6, 2024, 10:54 AM

#

white mulch its basically the same as on x86_64

if u set bit width to 48 yeah

weary hound Aug 6, 2024, 10:54 AM

#

even with 5 level paging it's still mostly the same as on x86_64

real pecan Aug 6, 2024, 10:54 AM

#

yeah

#

page mapping code u can mostly reuse

weary hound Aug 6, 2024, 10:55 AM

#

except that the topmost table has less entries because the address size is limited to 52 bits iirc

real pecan Aug 6, 2024, 10:55 AM

#

oh and manual dirty bit managment meme

#

u will have infinite page faults until u figure that one out

weary hound Aug 6, 2024, 10:55 AM

#

just need to read the armarm carefully :^)

ornate ginkgo Aug 6, 2024, 10:55 AM

#

ah yeah I ran into that on riscv

real pecan Aug 6, 2024, 10:56 AM

#

weary hound just need to read the armarm carefully :^)

just buy an expensive arm device that has automatic management

weary hound Aug 6, 2024, 10:56 AM

#

omw to buy an ampere server to run managarm on

flint idol Aug 6, 2024, 10:56 AM

#

real pecan u will have infinite page faults until u figure that one out

that sounds retarted, why would it page fault if the dirty bit is set

real pecan Aug 6, 2024, 10:56 AM

#

apple M cpus have it iirc?

real pecan Aug 6, 2024, 10:56 AM

#

flint idol that sounds retarted, why would it page fault if the dirty bit is set

it page faults if its not set

weary hound Aug 6, 2024, 10:56 AM

#

the page fault thing is for thee access flag

#

not dirty

real pecan Aug 6, 2024, 10:56 AM

#

so u can set it yourself

white mulch Aug 6, 2024, 10:56 AM

#

one annoying thing about arm in qemu is that if you happen to screw up exception handling somehow you end up in an infinite log spam because there is no triple fault equivalent

weary hound Aug 6, 2024, 10:57 AM

#

dirty you emulate by mapping as read-only

real pecan Aug 6, 2024, 10:57 AM

#

yeah

#

u basically have two present bits by default lol

real pecan Aug 6, 2024, 10:57 AM

#

white mulch one annoying thing about arm in qemu is that if you happen to screw up exception...

yeah

weary hound Aug 6, 2024, 10:57 AM

#

the access flag thing is a bit silly but since it has a separate exception type it's faster than having to walk the pts to figure out whether the page is not present or just not yet accessed

real pecan Aug 6, 2024, 10:58 AM

#

tbh most hobby kernels dont care

#

just set to 1 to begin with

real pecan Aug 6, 2024, 10:58 AM

#

weary hound the access flag thing is a bit silly but since it has a separate exception type ...

oh yeah that reminds me

#

decoding the reason for arm exceptions is also insane

weary hound Aug 6, 2024, 10:58 AM

#

is it?

white mulch Aug 6, 2024, 10:59 AM

#

eh

flint idol Aug 6, 2024, 10:59 AM

#

it's just a never ending list isn't it

white mulch Aug 6, 2024, 10:59 AM

#

I don't think its that bad

real pecan Aug 6, 2024, 10:59 AM

#

theres at least 3 websites

#

that decode it for u

#

if u paste in the registers

white mulch Aug 6, 2024, 10:59 AM

#

its just some bit fields

real pecan Aug 6, 2024, 10:59 AM

#

white mulch I don't think its that bad

well okay not insane but kind of way way harder than x86

white mulch Aug 6, 2024, 10:59 AM

#

but yeah if you mean like understanding it just from the number itself

flint idol Aug 6, 2024, 11:00 AM

#

obvious solution is to never crash

weary hound Aug 6, 2024, 11:00 AM

#

real pecan theres at least 3 websites

i only know of esr.arm64.dev

#

what are the other 2

real pecan Aug 6, 2024, 11:00 AM

#

https://esr.arm64.dev/

real pecan Aug 6, 2024, 11:00 AM

#

weary hound what are the other 2

oh idk

#

its been a while

#

maybe it was just esr

weary hound Aug 6, 2024, 11:01 AM

#

esr.arm64.dev has two other pages but it's for different things

real pecan Aug 6, 2024, 11:01 AM

#

exception syndrome LULW

#

like they even called it that

#

oh also the whole VHE and EL2 vs EL1

#

how registers alias each other

#

its not trivial

flint idol Aug 6, 2024, 11:02 AM

#

there's just a never ending list of includes I need to borrow from northport

weary hound Aug 6, 2024, 11:02 AM

#

oh yeah EL2 vs EL1 kinda sucks

real pecan Aug 6, 2024, 11:02 AM

#

yeah

#

because they decided kernels want to run at el2

#

but they all use el1

#

so now we alias them

#

its like

#

and there are corner cases

weary hound Aug 6, 2024, 11:03 AM

#

well the problem is that bare el2 only has the lower half

real pecan Aug 6, 2024, 11:03 AM

#

well

weary hound Aug 6, 2024, 11:03 AM

#

and el0 uses the el1 ttbrs

real pecan Aug 6, 2024, 11:03 AM

#

it must support the upper ttbr

#

if it supports VHE

weary hound Aug 6, 2024, 11:03 AM

#

for that yeah

real pecan Aug 6, 2024, 11:03 AM

#

so its basically exactly like EL1

#

with extra control

#

otherwise the kernel is run at el1 with a stub at el2

#

kind of like a vm lol

#

like its interesting how the entire runtime is controlled by KVM basically

#

even for the bare kernel

#

anyway sorry for stealing the obos thread

flint idol Aug 6, 2024, 11:05 AM

#

np

#

arch/m68k/loader/Main.cpp:218:25: error: 'memcmp' is not a member of 'sl

ornate ginkgo Aug 6, 2024, 11:09 AM

#

flint idol `arch/m68k/loader/Main.cpp:218:25: error: 'memcmp' is not a member of 'sl`

should be in Syslib.cpp?

flint idol Aug 6, 2024, 11:10 AM

#

it is

#

it probably just isn't defined in a header

ornate ginkgo Aug 6, 2024, 11:10 AM

#

if you give me a sec I'm working on some ideas to simplify the header inclusion situation

#

ok I had a look, here's a few suggestions and the list I came up with. You will need formats/Elf.h (which requires formats/Elf[32|64].h), you only need the first two functions of that file (ValidateElfHeader and ComputeRelocation), the rest can be removed and so can the #include <containers/Vector.h> line.

containers/List.h is also needed, and brings in CppUtils.h, but it only needs the Swap() function from there (you can just copy-paste it at the top).

The following files you'll need as is: Maths.h, Memory.h, NativePtr.h, PlacementNew.h and NanoPrintf.h (although this is mostly the same as the upstream one, so you can just grab that).

#

I might be missing some things that get unintentionally propagated, but I think that should be everything.

flint idol Aug 6, 2024, 11:14 AM

#

ok

#

got it compile

ornate ginkgo Aug 6, 2024, 11:18 AM

#

coolmeme

flint idol Aug 6, 2024, 11:18 AM

#

now I assume I pass this as -kernel to qemu

ornate ginkgo Aug 6, 2024, 11:19 AM

#

uh I think so, I forgot what I do

flint idol Aug 6, 2024, 11:19 AM

#

and it loads the kernel in the asm global scope

#

asm("\
.global KERNEL_BLOB_BEGIN \n\
.global KERNEL_BLOB_END \n\
.section .rodata \n\
.balign 0x10 \n\
KERNEL_BLOB_BEGIN: \n\
    .incbin \"../../out/oboskrnl\" \n\
KERNEL_BLOB_END: \n\
.previous \n\
");```

ornate ginkgo Aug 6, 2024, 11:19 AM

#

i know -initrd and -append work like you expect

ornate ginkgo Aug 6, 2024, 11:19 AM

#

flint idol ```c asm("\ .global KERNEL_BLOB_BEGIN \n\ .global KERNEL_BLOB_END \n\ .section ....

ah yes, late night engineering

#

that looks right

flint idol Aug 6, 2024, 11:20 AM

#

why didn't you just make the kernel the first binary passed as a module or something

ornate ginkgo Aug 6, 2024, 11:20 AM

#

I think the initrd option only takes one file

#

I may be wrong

flint idol Aug 6, 2024, 11:20 AM

#

what does -append do

ornate ginkgo Aug 6, 2024, 11:20 AM

#

its for the kernel command line

flint idol Aug 6, 2024, 11:21 AM

#

ah

devout niche Aug 6, 2024, 11:21 AM

#

flint idol TO(NOT)DO: Arm

try arm when you dare, but i can attest that it's weird

#

either that or i missed something very important

flint idol Aug 6, 2024, 11:22 AM

#

lol

#

it just sounds like a very weird arch

real pecan Aug 6, 2024, 11:27 AM

#

it is but aarch32 (before it became aarch32) is even worse

#

aarch64 is at least somewhat sane and usable

white mulch Aug 6, 2024, 11:30 AM

#

the linux boot protocol is kinda annoying tho especially if you try to turn on the mmu and get the kernel to the higher half inside the kernel itself without a separate preloader

flint idol Aug 6, 2024, 11:30 AM

#

I think I might've done something wrong KEKW

ornate ginkgo Aug 6, 2024, 11:32 AM

#

the PC isnt much help either, neither the loader or kernel should be there

#

what do the logs say?

flint idol Aug 6, 2024, 11:32 AM

#

it hung after saying the kernel physical base

#

#

then after a bit crashed

#

the kernel is being stupid

ornate ginkgo Aug 6, 2024, 11:33 AM

#

your kernel wants to be loaded at vaddr=0

#

which I do not protect against lol

flint idol Aug 6, 2024, 11:34 AM

#

well, I added
. = 0xC0000000;

#

to my linker script

ornate ginkgo Aug 6, 2024, 11:35 AM

#

what does readelf -l say about your kernel?

flint idol Aug 6, 2024, 11:35 AM

#

there are some deranged phdrs

#

that want to be loaded at 0x00000000

#

they are set as LOAD

ornate ginkgo Aug 6, 2024, 11:36 AM

#

that'll do it

flint idol Aug 6, 2024, 11:36 AM

#

so the bootloader probably takes that as the base

ornate ginkgo Aug 6, 2024, 11:36 AM

#

yep it does

flint idol Aug 6, 2024, 11:37 AM

#

the offending phdrs in the linker script seem to be pageable_data and pageable_rodata

#

I fixed the deranged phdrs by adding something to those two sections

sharp pike Aug 6, 2024, 11:40 AM

#

that's normal

#

or rather

#

that's what ld.bfd does

#

because reasons

#

ld.lld just omits the PHDRs like a sane linker

#

in any case a sane bootloader will ignore those 0-PHDRs :^) (Limine)

ornate ginkgo Aug 6, 2024, 11:41 AM

#

lol

flint idol Aug 6, 2024, 11:41 AM

#

I fixed your stuff

ornate ginkgo Aug 6, 2024, 11:42 AM

#

sharp pike that's normal

good to know though

sharp pike Aug 6, 2024, 11:42 AM

#

flint idol I fixed your stuff

wdym my stuff

flint idol Aug 6, 2024, 11:43 AM

#

not yours

#

r4's

sharp pike Aug 6, 2024, 11:43 AM

#

oh

flint idol Aug 6, 2024, 11:43 AM

#

should've made that clear

#

mb

#

and by fix I mean add:

if (!phdrs[i].p_vaddr)
    continue;

ornate ginkgo Aug 6, 2024, 11:44 AM

#

its still a fix

#

feel to PR it if you want, or I will add it at some point when I remember

sharp pike Aug 6, 2024, 11:44 AM

#

wait

#

what bootloader are we talking about again?

flint idol Aug 6, 2024, 11:45 AM

#

r4's limine stub

#

for the m68k

sharp pike Aug 6, 2024, 11:45 AM

#

ah

ornate ginkgo Aug 6, 2024, 11:45 AM

#

its not really a bootloader

sharp pike Aug 6, 2024, 11:45 AM

#

i'm sorry for implicitly calling it not sane

ornate ginkgo Aug 6, 2024, 11:45 AM

#

lol

sharp pike Aug 6, 2024, 11:45 AM

#

i thought we were still talking about Hyper

ornate ginkgo Aug 6, 2024, 11:45 AM

#

nah its ok, its just a hack really

flint idol Aug 6, 2024, 11:46 AM

#

I made cmake recompile the limine stub after kernel compilation

#

ornate ginkgo Aug 6, 2024, 11:47 AM

#

cool, thats exciting

flint idol Aug 6, 2024, 11:47 AM

#

now I need to recompile my gdb to take xml register descriptions

ornate ginkgo Aug 6, 2024, 11:48 AM

#

hmm it should be printing each lbp request as it sees them iirc

flint idol Aug 6, 2024, 11:48 AM

#

I don't use any

#

so far

ornate ginkgo Aug 6, 2024, 11:48 AM

#

oh ok, that would be why them haha

flint idol Aug 6, 2024, 11:48 AM

#

my goal yesterday would be to get something to "run"

#

while(1);```

#

and that technically counts

ornate ginkgo Aug 6, 2024, 11:51 AM

#

yeah that counts

#

congrats

flint idol Aug 6, 2024, 11:51 AM

#

the first thing I'll port is

#

IRQL

#

since the entire kernel kind of needs it

ornate ginkgo Aug 6, 2024, 11:52 AM

#

you'll have no trouble with that

flint idol Aug 6, 2024, 11:52 AM

#

now, since I unconditionally need 16 levels, I'll need to emulate IRQLs

ornate ginkgo Aug 6, 2024, 11:53 AM

#

why do you need 16?

#

that seems like a lot

flint idol Aug 6, 2024, 11:54 AM

#

because that's how many the kernel has on x86-64

#

and IRQL is basically entirely based on how x86-64 treats it

#

enum
{
    IRQL_PASSIVE,
    IRQL_DISPATCH = 2,
    IRQL_TIMER = 3,
    IRQL_MASKED = 0xf,
    IRQL_INVALID = 0xff,
};```

ornate ginkgo Aug 6, 2024, 11:55 AM

#

but thats only 4 + invalid!

flint idol Aug 6, 2024, 11:56 AM

#

iirc the gold fish interrupt controller has 8

#

right?

ornate ginkgo Aug 6, 2024, 11:57 AM

#

the arch gives you 8 (although one is interrupts disabled)

#

not sure about the gf pic

flint idol Aug 6, 2024, 11:58 AM

#

and how many interrupt vectors are there?

#

I've seen 256 vectors and 64 vectors

#

idk which one

ornate ginkgo Aug 6, 2024, 11:58 AM

#

I dont have my notes handy, but from memory the way it works is the gf pics route to the auto vectors

#

which there's 7 of, one for each interrupt priority level

#

iirc pic0 (lowest priority) only has the uart attached to it, pics 1-4 have the virtio interrupts connected to them, and pic5 (highest priority) has the timer interrupt connected to it.

flint idol Aug 6, 2024, 12:01 PM

#

pic6 is unused?

ornate ginkgo Aug 6, 2024, 12:01 PM

#

there's a diagram in a qemu source file somewhere

#

yeah

#

https://elixir.bootlin.com/qemu/v8.0.1/source/hw/m68k/virt.c#L41 here we go

#

ah ok, they use 1-based numbering for the pics

#

so all the user-available vectors they list in the manuals are unused for the virt board

flint idol Aug 6, 2024, 12:03 PM

#

it sucks how the IPLs are fixed to a specific IRQ

ornate ginkgo Aug 6, 2024, 12:05 PM

#

it is annoying.

#

it worked out well for me because thats the order of my runlevels (irql)

#

clock above all other interrupts

flint idol Aug 6, 2024, 12:05 PM

#

my kernel does the literal opposite

#

the TIMER irql is one of the lowest priorities

#

so is dispatch (basically scheduler timer)

#

in fact, the scheduler IS the lowest priority

ornate ginkgo Aug 6, 2024, 12:06 PM

#

oh one thing I'll mention because I just saw it in my code: the pic registers that deal with enabling/disabling/pending stuff are bitsets.

flint idol Aug 6, 2024, 12:07 PM

#

oooooo

ornate ginkgo Aug 6, 2024, 12:07 PM

#

so you dont write the pic pin you want to mask to the disable register, you write 1s in the places of which pin to disable

#

hmm

flint idol Aug 6, 2024, 12:07 PM

#

I think I have an idea

#

basically

#

the arch defines how many IRQLs it can support

ornate ginkgo Aug 6, 2024, 12:08 PM

#

flint idol the TIMER irql is one of the lowest priorities

you can just mask and unmask the clock and fake it

flint idol Aug 6, 2024, 12:08 PM

#

any power of two from 1-16

#

then the kernel will prioritize the IRQLs across different arches the same

#

despite them having a different literal IRQL value

flint idol Aug 6, 2024, 12:09 PM

#

ornate ginkgo so you dont write the pic pin you want to mask to the disable register, you writ...

the m68k port will do this

#

but then I remember reading somewhere

#

about something

#

#x86 message

#

nvm that's x86-only

#

assuming this bitmask is more sane than the x86 PIC's bitmask

#

I'll be fine

devout niche Aug 6, 2024, 12:24 PM

#

i think the goldfish rtc is at least wired to the highest IPL

flint idol Aug 6, 2024, 12:24 PM

#

2nd highest

devout niche Aug 6, 2024, 12:24 PM

#

what's highest?

flint idol Aug 6, 2024, 12:25 PM

#

NMI

#

devout niche Aug 6, 2024, 12:25 PM

#

reasonable

flint idol Aug 6, 2024, 12:26 PM

#

should I emulate IRQL or should I use the goldfish pic's irq mask bitfield

#

@devout niche

devout niche Aug 6, 2024, 12:28 PM

#

flint idol should I emulate IRQL or should I use the goldfish pic's irq mask bitfield

it's up to you, you can certainly emulate multiple hardware levels that way

flint idol Aug 6, 2024, 12:29 PM

#

I think I'll emulate it

flint idol Aug 6, 2024, 12:53 PM

#

shit

#

I accidentally just pushed a commit from the m68k port to master

#

git is being weird

#

ok there

#

I fixed the commit history

#

https://github.com/OBOS-dev/obos/tree/m68k-port

GitHub

GitHub - OBOS-dev/obos at m68k-port

I was bored... Contribute to OBOS-dev/obos development by creating an account on GitHub.

flint idol Aug 6, 2024, 2:26 PM

#

new favorite archiecture: m68k

#

no PUNPCKLQDQ shit

#

VCVTNE2PS2BF16

#

those instructions looked like someone mashed their keyboard keys and called it an instruction

#

vdfsfdsgr

#

for all I know that could be an x86 instruction

#

VFMADDRND231PD

flint idol Aug 6, 2024, 3:17 PM

#

@devout niche
what's wrong with my instruction (I want to move d0 into sr)

move.l %d0, %sr```

#

Error: operands mismatch -- statement `move.l %d0,%sr' ignored

devout niche Aug 6, 2024, 3:17 PM

#

flint idol <@391791686881247234> what's wrong with my instruction (I want to move d0 into ...

the length suffix

#

SR is 16 bits

#

you want move.w

flint idol Aug 6, 2024, 3:17 PM

#

oh yeaaaaaaah

#

ok it's good now

#

ty

#

is there an equivalent to the x86 int instruction on the m68k

#

to raise an irq

devout niche Aug 6, 2024, 3:21 PM

#

flint idol is there an equivalent to the x86 `int` instruction on the m68k

you can trap, there are specific vectors for those i should add

flint idol Aug 6, 2024, 4:36 PM

#

@devout niche is there a register I can use to store a pointer to the cpu local struct (one similar to gs_base on x86).

#

oh wait does it even have smp

#

it doesn't

#

ok that's good

#

nvm

flint idol Aug 6, 2024, 4:59 PM

#

I'm done IRQL emulation

#

all I need is to implement callDeferredIrq

#

(TODO)

flint idol Aug 6, 2024, 5:22 PM

#

the m68k is pretty based

#

movem.l %d0-%d7/%a0-%a6, %sp@-

#

if I knew about this when I started this rewrite, it would've initially been for the m68k /j

ashen goblet Aug 6, 2024, 5:36 PM

#

I feel stupid asking but is m68k a 32 or 64 bit arch

#

32 right

flint idol Aug 6, 2024, 5:36 PM

#

32-bit

flint idol Aug 6, 2024, 5:56 PM

#

@devout niche I saw that the m68k port of keyronex has screenshots, so I checked the source, and saw that your framebuffer was just normal memory

#

what qemu/shell commands do you use

#

to get the screenshot

devout niche Aug 6, 2024, 5:57 PM

#

flint idol <@391791686881247234> I saw that the m68k port of keyronex has screenshots, so I...

it's by a (primitive) virtio-gpu driver

devout niche Aug 6, 2024, 5:57 PM

#

flint idol what qemu/shell commands do you use

alt-prtscr

flint idol Aug 6, 2024, 5:58 PM

#

devout niche it's by a (primitive) virtio-gpu driver

oof

#

I think I'll just stick with memsave for now

#

with the framebuffer address

#

first screenshot of OBOS on the m68k

lean glen Aug 6, 2024, 6:02 PM

#

flint idol I think I'll just stick with `memsave` for now

tf is memsave

flint idol Aug 6, 2024, 6:02 PM

#

a qemu command

ashen goblet Aug 6, 2024, 6:02 PM

#

flint idol first screenshot of OBOS on the m68k

epic

#

does your current code structure hold well on multiple archa

flint idol Aug 6, 2024, 6:03 PM

#

yes

#

#

I fixed the screenshot

#

to not look like shit

ashen goblet Aug 6, 2024, 6:05 PM

#

green was more fun

flint idol Aug 6, 2024, 6:05 PM

#

idk why the text is cyan

ashen goblet Aug 6, 2024, 6:06 PM

#

maybe similar issue as Fadanoid had

#

"""issue"""

flint idol Aug 6, 2024, 6:07 PM

#

nope

#

just an invalid fb format

#

ashen goblet Aug 6, 2024, 6:10 PM

#

I‘m just now noticing the weirdly coloured top row

flint idol Aug 6, 2024, 6:10 PM

#

probably me just reading the wrong offset

#

at&t syntax is hurting my brain

devout niche Aug 6, 2024, 6:42 PM

#

flint idol first screenshot of OBOS on the m68k

looks quite aesthetic there

flint idol Aug 6, 2024, 6:42 PM

#

KEKW

#

image magick was being stoobid

devout niche Aug 6, 2024, 6:46 PM

#

it looked right somehow

#

compare to keyronex on amiga

flint idol Aug 6, 2024, 6:47 PM

#

currently I'm trying to figure out where the hell the int number is pushed
I know where it is at IRQ entry

#

just I have no idea where it is after I push all my shit

#

oops

#

I misread the user manual

#

after gdb magic I got the offset

#

nvm I didn't

flint idol Aug 6, 2024, 7:31 PM

#

finally figured out the offset after a short break

#

it was bytes from my position 0x48 to the interrupt number

flint idol Aug 6, 2024, 7:47 PM

#

traps seem to work

flint idol Aug 6, 2024, 8:11 PM

#

#

hold up

#

this screenshot looks a bit wrong

#

#

the second log message is right

#

as trap 0 corresponds to the 32nd interrupt on m68k

#

and with that- the irq interface has been ported

#

TODO: SendEOI function

#

otherwise, assuming I didn't get any macro definitions wrong, the irq interface should work perfectly fine

#

its only dependencies are the arch-specific functions it defined

#

which there are 5 of

flint idol Aug 6, 2024, 8:59 PM

#

I think I'll port the scheduler next

#

the scheduler's only dependencies are ones to manipulate thread context

#

one to switch to a context

#

and one to save the current context and call the scheduler

#

that's all that's needed to run the scheduler

#

at the bare minimum

lean glen Aug 6, 2024, 9:01 PM

#

nah do memory stuff first

#

well you could do sched too actually

#

depends how intertwined they are

flint idol Aug 6, 2024, 9:02 PM

#

not interwined at all

lean glen Aug 6, 2024, 9:02 PM

#

do whatever feels fun, that's the most important

#

meme

flint idol Aug 6, 2024, 9:02 PM

#

also if the scheduler depended on the memory manager, it wouldn't be depending on arch specific functions of the mm

#

there is one thing in my vmm that I need to change

#

idk how to word it

#

so I won't

#

basically just an optimization

#

I mean ngl the m68040 mmu sounds goofy

#

and fun

#

@devout niche confirmation?

devout niche Aug 6, 2024, 9:04 PM

#

flint idol <@391791686881247234> confirmation?

it is quite fun

#

the 030 mmu is much funner but qemu doesn't implement it

flint idol Aug 6, 2024, 9:05 PM

#

so I'll do that after my soccer game

flint idol Aug 7, 2024, 12:57 AM

#

flint idol so I'll do that after my soccer game

*tomorrow

#

I'm far too tired rn to do osdev without doing some funky shit

flint idol Aug 7, 2024, 12:43 PM

#

I'll work on the scheduler first

flint idol Aug 7, 2024, 2:19 PM

#

I have implemented CoreS_SwitchToThreadContext

flint idol Aug 7, 2024, 2:49 PM

#

flint idol I have implemented `CoreS_SwitchToThreadContext`

it had a couple bugs (mainly because of my infamiliarity with m68k asm and at&t syntax)

#

but I fixed those and can switch to a thread context

#

time to test yield I guess

#

(which directly tests the entire scheduler)

flint idol Aug 7, 2024, 3:25 PM

#

well the scheduler works...

#

BUT

#

it corrupts a bunch of global variables :)

#

just a slight problem

#

I fixed that

inland radish Aug 7, 2024, 3:34 PM

#

flint idol I fixed that

what was the problem?

flint idol Aug 7, 2024, 3:35 PM

#

I had my label for the temporary stack in the wrong place

#

Arch_TempStack:
.skip 0x10000

#

was pointing to the top of the stack already

#

then I added 0x10000 bytes when I was loading it into sp

#

Yielding works

#

#

now to port the MM

#

but, first, I need a pmm

#

(aka: copy and paste the x86_64's pmm)

white mulch Aug 7, 2024, 4:03 PM

#

why does the pmm even have to be arch specific?

flint idol Aug 7, 2024, 4:03 PM

#

because I'm too lazy to make it not

white mulch Aug 7, 2024, 4:03 PM

#

lol

flint idol Aug 7, 2024, 4:05 PM

#

anyway the pmm has been ported

#

time for virtual memory management

#

because I don't want to forget anything, I'll say the functions I need to implement for the VMM here:

- OBOSS_UnmapPage
- OBOSS_GetPagePhysicalAddress
- OBOSS_AllocatePhysicalPages
- OBOSS_FreePhysicalPages
- MmS_QueryPageInfo
- MmS_GetCurrentPageTable
- MmS_SetPageMapping```

flint idol Aug 7, 2024, 5:30 PM

#

Each of the 128 pointer-level table descriptors corresponds to a 256-Kbyte block of memory.

#

This isn't making sense

#

basically:
root_pt[i] points to a 256 kib block of memory

#

that's what I understood

real pecan Aug 7, 2024, 5:35 PM

#

yeah

flint idol Aug 7, 2024, 5:39 PM

#

instead of trying to understand that

#

I looked in r4's bootloader code

devout niche Aug 7, 2024, 6:38 PM

#

flint idol basically: `root_pt[i]` points to a 256 kib block of memory

root_pt[128] entries point to a pointer_pt[128] entries point to a pt[64]

#

a pt[64] describes 256 kib

#

all assuming 4kib pages

flint idol Aug 7, 2024, 6:39 PM

#

I understood that eventually

devout niche Aug 7, 2024, 6:41 PM

#

or as i call them pml3, pml2, and pml1

inland radish Aug 7, 2024, 6:41 PM

#

same

real pecan Aug 7, 2024, 6:43 PM

#

devout niche `root_pt[128]` entries point to a `pointer_pt[128]` entries point to a `pt[64]`

why did they pick such a strange format

flint idol Aug 7, 2024, 6:43 PM

#

so this all makes sense

devout niche Aug 7, 2024, 6:43 PM

#

real pecan why did they pick such a strange format

no idea, the 030 you could configure the MMU very thoroughly

flint idol Aug 7, 2024, 6:43 PM

#

but for some reason r4's limine stub doesn't like my requests

devout niche Aug 7, 2024, 6:43 PM

#

anything from 2 to 6 level paging on the 030

#

the 040 i think just standardised on a compromise fixed scheme that should satisfy most 030 people

flint idol Aug 7, 2024, 6:43 PM

#

why in the world would one need 6 level paging for 32-bit addresses

real pecan Aug 7, 2024, 6:44 PM

#

wouldnt that level of indirection be super slow considering ram latencies on that stuff

devout niche Aug 7, 2024, 6:44 PM

#

it was for AI

#

i'm not joking

#

this is the application that the 030 manual suggests would be suited to many-level paging

#

well it only suggested 3 or 4 levels in that case

real pecan Aug 7, 2024, 6:45 PM

#

huh

#

what sort of ai were they doing back then lol

main girder Aug 7, 2024, 6:45 PM

#

This sounds more like it's for lisp whose most common usage was AI

devout niche Aug 7, 2024, 6:45 PM

#

oh, i should mention, you can configure the page size to many reasonable powers of 2

devout niche Aug 7, 2024, 6:46 PM

#

main girder This sounds more like it's for lisp whose most common usage was AI

it is for exactly that

#

the lists is a giveaway

lean glen Aug 7, 2024, 6:46 PM

#

Based lisp moment

devout niche Aug 7, 2024, 6:46 PM

#

but another subtler sign that the lisp adept would recognise is that many lisp implementations of that time partitioned virtual address space into regions dedicated to one kind of object

#

and then considered the high bits of a pointer to determine what kind of object it is

inland radish Aug 7, 2024, 6:47 PM

#

Sounds familiar

flint idol Aug 7, 2024, 6:48 PM

#

@ornate ginkgo your thing broke

real pecan Aug 7, 2024, 6:48 PM

#

time to make a hyper stub

devout niche Aug 7, 2024, 6:48 PM

#

cool thing in the 030 manual btw, they have pseudocode for a sample PMM and VMM in there

flint idol Aug 7, 2024, 6:49 PM

#

real pecan time to make a hyper stub

good idea

real pecan Aug 7, 2024, 6:49 PM

#

lol

flint idol Aug 7, 2024, 6:49 PM

#

flint idol <@337185227719966721> your thing broke

so basically I have 4 limine requests

devout niche Aug 7, 2024, 6:49 PM

#

it has page swapping of course

flint idol Aug 7, 2024, 6:49 PM

#

flint idol so basically I have 4 limine requests

two get filled

#

the other two are ignored

lean glen Aug 7, 2024, 6:50 PM

#

devout niche it has page swapping of course

That's actually cool

flint idol Aug 7, 2024, 6:51 PM

#

all of the requests do exist in the stub's code

#

but two are never populated

#

I suspect the bug is in LbpNextRequest

real pecan Aug 7, 2024, 6:52 PM

#

flint idol but two are never populated

well they did mention it was a limine stub and not a full impl meme

flint idol Aug 7, 2024, 6:52 PM

#

those two requests are supported

#

by the stub

devout niche Aug 7, 2024, 6:53 PM

#

seadh

#

what request?

flint idol Aug 7, 2024, 6:53 PM

#

hhdm and kernel file

#

I have confirmed that my kernel has set the ID correctly

flint idol Aug 7, 2024, 7:14 PM

#

I fixed by doing this:

uintptr_t* iter = (uintptr_t*)current;
if (!iter)
    iter = (uintptr_t*)(kernelBase + kernelSlide);
else
    iter++;
for (; (uintptr_t)iter < kernelTop; iter++)
    if (sl::memcmp(iter, CommonMagic, sizeof(uint64_t)*2) == 0)
        return (LbpRequest*)iter;```

#

in LbpNextRequest

#

instead of the code that was previously there

#

I can successfully map a page

#

now time to see if it'll still be successful when I set up the page tables

#

(probably not)

ornate ginkgo Aug 7, 2024, 7:50 PM

#

Are the requests 8 byte aligned?

flint idol Aug 7, 2024, 7:50 PM

#

idk

#

but I fixed it by removing that code, and replacing it with my own code to the same thing

#

they weren't

flint idol Aug 7, 2024, 8:09 PM

#

so my brain has decided to forget paging

#

while I'm trying to fix paging

real pecan Aug 7, 2024, 8:10 PM

#

who needs it

flint idol Aug 7, 2024, 8:10 PM

#

what? brain or paging?

real pecan Aug 7, 2024, 8:10 PM

#

both tbh

flint idol Aug 7, 2024, 8:11 PM

#

oh wait paging isn't broken

#

I forgot that the pt I was mapping to wasn't in use

#

(intended)

#

ok I think mapping the kernel working

#

it doesn't crash

#

but I don't trust that it won't crash in the future

#

why is every kernel address mapped to zero

#

okie I fixed it

flint idol Aug 7, 2024, 9:19 PM

#

@devout niche does the m68k have smp

#

I don't think so

#

but might as well ask in case

devout niche Aug 7, 2024, 9:20 PM

#

flint idol <@391791686881247234> does the m68k have smp

only some very rare systems

flint idol Aug 7, 2024, 9:20 PM

#

ok

flint idol Aug 7, 2024, 9:39 PM

#

flint idol because I don't want to forget anything, I'll say the functions I need to implem...

5/8 are done

#

*9

#

I also need to implement the pf handler

flint idol Aug 7, 2024, 9:52 PM

#

flint idol *9

*10

#

I also need to implement something to act as swap

#

temporarily

#

the kernel's supposed to eventually use an AHCI disk driver to do that

#

but it can't because it doesn't have one

#

I have a slight problem

#

a very small part of the kernel wasn't designed to split kernel page tables and user page tables

#

like how the m68040 does (urp and srp)

#

i.e., it was designed around the idea that there is one pointer to the page table

#

except it's only two lines of code

flint idol Aug 7, 2024, 10:03 PM

#

flint idol i.e., it was designed around the idea that there is one pointer to the page tabl...

this is because of a function with the signature:

page_table MmS_GetCurrentPageTable()

#

which didn't make a distinction between the current kernel pt and the current user one

#

but since it's only used during vmm init, I added a note that it always returns the current kernel pt.

#

@devout niche difference between access fault and address error?

#

nvm I found it

#

in the manual

#

but is there an equivalent to the cr2 register on the m68k

#

so that I can see the faulting address on access fault

#

so far I haven't been able to find such a thing

#

or really any info on where fault info is stored on access fault

ornate ginkgo Aug 7, 2024, 10:18 PM

#

flint idol a very small part of the kernel wasn't designed to split kernel page tables and ...

You can load the same table into both

#

They even cover that in the manual

flint idol Aug 7, 2024, 10:18 PM

#

~~who even reads that thing~~

flint idol Aug 7, 2024, 10:18 PM

#

flint idol or really any info on where fault info is stored on access fault

I figured that out

#

now I have a struct taken from keyronex

#

(md_intr_frame.format7)

#

hope you don't mind @ fadanoid

#

I really gotta stop pinging fadanoid lol

flint idol Aug 7, 2024, 10:59 PM

#

I can page fault successfully

flint idol Aug 7, 2024, 11:55 PM

#

I decided to test on x86_64 to make sure I didn't mess anything up

#

and I did!

#

I fixed a bug where the dirty+accessed bugs were never being cleared

#

and now the working set's exceeded its size

flint idol Aug 8, 2024, 12:13 AM

#

this math isn't mathing

#

if I only append pages to the working set if the current size of pages in the working set are less than its capacity

#

then why would-

#

race condition

#

but no other CPUs are running vmm code

#

and the vmm is locked

#

I fixed it

#

now a page in the working set was paged out

#

it doesn't seem to be exactly that

#

a page is in the working set's list

#

but it reports itself to not be in the working set (page->inWorkingSet=false)

main girder Aug 8, 2024, 12:24 AM

#

flint idol but it reports itself to not be in the working set (page->inWorkingSet=false)

this will break btw

#

cuz of shared pages

#

if you want to track something like that youll need a count of working sets rather than a boolean

flint idol Aug 8, 2024, 12:25 AM

#

main girder cuz of shared pages

forgot about those

#

I'll do that

#

so a count of the working sets the page is in

main girder Aug 8, 2024, 12:26 AM

#

yeah

#

not to be confused with the refcount

#

a page could be referenced while in 0 working sets

#

imagine theres outstanding IO to the page and its pinned into an MDL, but got trimmed from the process working set after that

#

refcount 1, working set count 0

flint idol Aug 8, 2024, 12:27 AM

#

yeah

#

surely a 16-bit count will be enough

#

but if a library like libc is shared between 65536 processes, then that'd break

flint idol Aug 8, 2024, 12:47 AM

#

flint idol but it reports itself to not be in the working set (page->inWorkingSet=false)

this seems to because the node doesn't want to be removed

#

idk how it got to choose that

#

it somehow has two lives

#

and by that I mean it's in the working set list twice

flint idol Aug 8, 2024, 1:10 AM

#

I fixed that

flint idol Aug 8, 2024, 1:11 AM

#

flint idol and now the working set's exceeded its size

now this is back

#

the working set has exceeded its size by ONE page

#OBOS (not vibecoded)