#pmOS - microkernel OS for RISC-V, x86, and LoongArch

1 messages ยท Page 9 of 1

carmine nacelle
#

Cool, Rust seems to be working?

#

Fuck it, I'm pushing it to main

#

I'm tired of it

#

Time to port to ARM trl

vital geode
#

lol

carmine nacelle
#

Why did no one tell me that cmake can fetch git repos?

toxic torrent
#

i'm more suprised you didn't know that

twilit talon
#

if it exists, cmake can do it

carmine nacelle
#

I think I've successfully made that thing compile libyaml

#

For my init server

fast bolt
#

I don't like requiring network to build

carmine nacelle
#

Yeah, but there's Rust anyway

carmine nacelle
#

my build system is a dumpster fire

carmine nacelle
#

(bidirectionally)

#

So the drivers could publish handles to devices, and the init server could match those based on the user provided rules, and start more stuff

carmine nacelle
#

Cool, now my PS/2 server is trolling me

#

Nevermind

#

It's the kernel nooo

#

I think I'll just pretend that it's working as intended trl

carmine nacelle
#

The timers are completely broken bruh

twilit talon
#

Dead sad

carmine nacelle
#

Unfortunately

#

I've been thinking about picking it back up but I have no energy

twilit talon
#

So many old huge projects dying these days

#

Keyronex, astral, pmos

carmine nacelle
#

Maybe I need to find some contributors or whatever trl

#

I have a lot of stuff to do though

#

I've gotten stuck on yaml parsing in my init server to have a proper ... init system that's integrated with pmbus and stuff

twilit talon
#

Yaml is way more complex than json

carmine nacelle
#

I've found a library for it

twilit talon
#

Ah

carmine nacelle
#

Too much work

toxic torrent
#

damn..

carmine nacelle
#

I want the system to publish stuff on pmbus (which is mbus close)

#

And that to be able to launch drivers

#

Through the init server

#

So e.g. devices server could publish PCI devices, which would launch XHCI driver, which would publish USB devices and so on and that could launch USB drivers

#

Or whatever

vale hinge
carmine nacelle
#

I mean it's not completely dead I hope

vale hinge
#

๐Ÿ˜”

carmine nacelle
toxic torrent
twilit talon
toxic torrent
#

btw

#

why don't you just return instead of the goto meme

carmine nacelle
#

it's probably missing free()

#

I need to get back to working on it

toxic torrent
#

doit

carmine nacelle
twilit talon
#

We back?

carmine nacelle
#

I hope so

#

Tbh it's not the first time that I was taking a 2 months break

carmine nacelle
#

I have decided that I will not merge bootstrap and process servers

#

But now I don't know what to do with filesystems/logs

carmine nacelle
#

They're 2 separate servers now

#

And I can just make them talk over IPC and it should work as well

carmine nacelle
fringe grove
#

i have it saved from idk when

toxic torrent
#

:( dead chat

carmine nacelle
#

Yeah

#

Maybe there will be a comeback some day

manic girder
#

Day 1 of waiting for pmos comeback halfmemeleft

carmine nacelle
#

Oh no, I forgot how to build it

#

I think I've basically done no programming (in general) since August

#

(except a fling with Zigbee switches)

carmine nacelle
#

Cool, it's working, and the timers are still casually broken

carmine nacelle
twilit talon
#

pmos is back letsgo

carmine nacelle
#

Idk if I'll stick to it
This looks too complicated now

carmine nacelle
#

Nice return type

#

(Anyway, I'm finally implementing a way to pass the data from process to process when it's started, so I can pass capability, args and environmental variables. And maybe I'll do posix_spawn eventually)

left dew
#

use a struct, perhaps?

#

probably easier to manage if you ever need to add another item to the tuple lol

twilit talon
#

^

#

tuple/pair is a lazy antipattern imo

hexed knot
#

my favorite compile time killer

#

also shared_ptr<page table> is kinda lame, tbh

carmine nacelle
#

I have RCU for some stuff

twilit talon
carmine nacelle
#

Shared pointers are obnoxious

hexed knot
#

and depending on what a paging::Arch_Page_Table is

carmine nacelle
#

It is a bad pointer, with external structure for weak_ptr

hexed knot
#

eugh

#

thats even worse

#

weak rc sucks so much

fast bolt
#

I don't hate pair for a return value where the parts are not really connected

hexed knot
#

yes the only issue is that pair and tuple are really not good for compile times

carmine nacelle
#

I'm writing a microkernel, it's not (supposed to be) that much code anyway for it to matter...

hexed knot
#

lol

twilit talon
#

write a macro kernel

carmine nacelle
#

Hybrid kernelโ„ข
(linux)

plain jungle
carmine nacelle
#

(which is annoying to debug since my screen driver is in userspace)

plain jungle
#

don't you have a small uart driver in the kernel?

carmine nacelle
#

I do, bit it's only for kernel messages

carmine nacelle
#

Actually, the issue is not where I was expecting it and I've no idea why it pagefaults

#

Bruh, it's a weird UB in C with pointer arithmetic which removed the null check?

carmine nacelle
#

(I've managed to get it to work...)

#

Now I need to decide how to hook into pmbus to match the devices for drivers

#

(namely, how do I make it match PCI and ACPI devices)

carmine nacelle
#

Why did I use libyaml, this is pain

carmine nacelle
#

Oh no, Rust is pagefaulting (trl)

#

But my PCI routing is also broken for some reason

carmine nacelle
#

Ok, the rust situation has been rectified

carmine nacelle
#

I feel like just copying mbus's conjunction/disjunction thing trl

#

(the only problem is that I'm doing it in 3 programming languages, and I'm afraid it's gonna be a bit obnixious in C...)

#

(and I forgot how to write Rust)

carmine nacelle
#

A small progress update: I'm currently trying to make the filter thing to work in init server, which should make it able to start i8042 driver (and later after this, I wanted it for disks...) by matching its PNP ID from ACPI
(but my yaml parsing is broken for some reason, and it's too late for me to be fixing it today)

carmine nacelle
#

The kernel has chosen to troll me

carmine nacelle
#

(and now I have to touch the complicated kernel code NOOO)

carmine nacelle
#

This looks promissing

carmine nacelle
# toxic torrent wtf ๐Ÿ˜ญ

I mean limine doesn't complain about it, and it seems to work trl
(I'll move it to the proper config files on the disk (or/and initramfs), once I have necessary drivers)

toxic torrent
#

obv but just have like an mbus filter shit or something

#

idk how I'm going to do it yet

carmine nacelle
toxic torrent
#

yeah but just put that in the drivers lol

carmine nacelle
#

The difference with managarm is that it launches drivers, and those then talk to mbus

#

I launch drivers once it finds a device in mbus

#

(Or plan to do it, I need to implement some more stuff...)

#

it seems like a nicer system

toxic torrent
#

does it?

#

what if the user doesn't want to have a driver launched

carmine nacelle
hexed acorn
carmine nacelle
hexed acorn
#

for late drivers that use udev the flow is: posix-subsystem scans mbus and fills out sysfs, udev scans sysfs and processes rules, the rules tell it to bind a device to a driver (potentially launching the driver if it wasn't running before)

carmine nacelle
#

Unfortunate (I need to replace my libc)

#

But ignoring broken puts, this is very promissing...

vital geode
carmine nacelle
#

the returned written number of bytes is messed up when the text is too long

#

(and I don't know where the bug is)

vital geode
carmine nacelle
#

It has to do with the fact that if it gets too long, I send several IPC messages

carmine nacelle
#

Now, this looks extremely promissing

#

(Now I just need to rewrite everything in that chain to use capabilities (a.k.a. rights), instead of juggling the ports directly)

#

(And then implement dependencies in my init thing)

#

(and then to decide how I'm gonna pass that to the drivers (it's probably gonna be cmdline args (so that needs implementing as well)))

carmine nacelle
toxic torrent
hexed acorn
#

(and conjunction is and)

carmine nacelle
#

Cool, it seems to be fully working, but using capabilities

#

(but I still need to implement notifications about closed rights...)

#

(which isn't really a priority though)

carmine nacelle
#

Ok, I need to do something with auxiliary vector, but I've no idea how I should do it

carmine nacelle
#

Maybe I should implement futex first instead? Thonk

carmine nacelle
#

I think I'll add rights to threads and page tables as well...

#

(though the page tables would be complicated)

carmine nacelle
#

Ok, fuck it, I'm changing everything to proper aux vector stuff instead of my thing that I have now, and moving elf loading back to userspace

carmine nacelle
#

I think I'm coming back full circle to using u64 for pointers (even on 32 bit platforms...)

toxic torrent
#

uintptr_t:

carmine nacelle
toxic torrent
#

I don't get it

#

you're using u64 on 32-bit platforms?

carmine nacelle
#

32 bit userspace with 64 bit kernel

toxic torrent
#

ew

twilit talon
#

wdym ew lol

carmine nacelle
twilit talon
#

its much better than a 32 bit kernel

toxic torrent
#

yeah but like

#

why use u64 then

#

what for

carmine nacelle
#

So that you can load 64 bit processes from 32 bit bootstrap server or whatever

#

Like yeah it's pointless, but kinda cool, and not that difficult to implement

toxic torrent
#

fair enough

#

cursed tho

carmine nacelle
toxic torrent
#

btw do you mean actual 32-bit or just x32

carmine nacelle
#

i686 userspace mixed with amd64 userspace and kernel

#

compat mode

#

I haven't bothered looking into x32

twilit talon
#

what is x32

carmine nacelle
#

Though that can be implemented as well, I guess?

carmine nacelle
toxic torrent
#

yeah it's useless

twilit talon
#

wtf

toxic torrent
#

saves memory trust me

carmine nacelle
#

linux supports it

#

or at least did, I think there were talks about removing it?

#

or something like that

toxic torrent
#

i refuse to mix different address sizes

carmine nacelle
#

why

carmine nacelle
toxic torrent
#

it's ugly

carmine nacelle
toxic torrent
#

everywhere

#

my kernel also doesn't support 32-bit period

carmine nacelle
#

I wanted to do m68k eventually

#

and ppc

#

So porting to i686 was a logical first step

#

and I started that with porting userspace by adding compat mode to kernel

toxic torrent
#

ppc64le KEKW

carmine nacelle
#

I don't have ppc64 hardware

#

Also be sounds fun

toxic torrent
#

noooooo

carmine nacelle
#

Small progress update: the elf loader in userspace is almost working. But I think I have a new fun issue, the scheduling seems to die after anything pagefaults in userspace?

#

The loader is working, but it seems that I have a deadlock somewhere...

carmine nacelle
#

or not?

carmine nacelle
#

with the magic of printf, I am concluding that it is

#

Yep, it was the set...

carmine nacelle
carmine nacelle
#

I've changed the TLS logic to use data from ELF pheaders directly...

#

And now I'm trying to build loongarch64 version, but Rust is refusing to cooperate

#

(Since riscv64 is also broken, but I couldn't event compile rust toolchain for it in the first place)

carmine nacelle
carmine nacelle
#

I think I desperetly need to fix my build system

plain jungle
#

you need to enable pthreads in your gcc config

carmine nacelle
#

It's clang

#

I think I forgot to rebuild the compiler, and I've been using an old version which didn't define _pmos_ for loongarch

#

For my future reference:
cmake -S ../llvm-pmos/runtimes -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind;compiler-rt" -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HOME/pmos/sysroot/usr/ -DLLVM_TARGET_TRIPLE=loongarch64-pmos -DCMAKE_CXX_COMPILER_TARGET="loongarch64-pmos" -DCMAKE_C_COMPILER="clang" -DCMAKE_CXX_COMPILER="clang++" -DCMAKE_C_COMPILER_TARGET="loongarch64-pmos" -DCMAKE_ASM_COMPILER_TARGET="loongarch64-pmos" -DCOMPILER_RT_BUILD_BUILTINS=ON -DCMAKE_C_FLAGS="--sysroot=$HOME/pmos/sysroot -nostartfiles" -DCMAKE_CXX_FLAGS="--sysroot=$HOME/pmos/sysroot -nostdlib" -DLIBCXXABI_USE_LLVM_UNWINDER=true -DLIBCXX_ENABLE_SHARED=OFF -DLIBUNWIND_ENABLE_SHARED=OFF -DLIBCXXABI_ENABLE_SHARED=OFF -DCOMPILER_RT_BUILD_BUILTINS=ON -DCOMPILER_RT_BUILD_ORC=OFF -DCOMPILER_RT_BUILD_SANITIZERS=OFF -DCOMPILER_RT_BUILD_MEMPROF=OFF -DCOMPILER_RT_BUILD_XRAY=OFF -DCOMPILER_RT_BUILD_LIBFUZZER=OFF -DCOMPILER_RT_BUILD_PROFILE=OFF -DCOMPILER_RT_INSTALL_LIBRARY_DIR="$HOME/pmos/sysroot/usr/lib" -DCAN_TARGET_loongarch64=1 -DCOMPILER_RT_BUILD_CTX_PROFILE=OFF

carmine nacelle
#

cool, rust is still broken on my other pc on loongarch64

#

So my rust version is just broken?

#

fuck me I don't have energy to rebase it

left dew
#

is your compiler 2 years out of date?

carmine nacelle
#

1 year out of date

carmine nacelle
#

Rebasing wasn't too bad... But I now need to figure out libc and cc-rs crates, which I remember being a huge pita

carmine nacelle
carmine nacelle
#

(I'm trying to get cc-rs generate new targets from my rustc, but it's refusing to do it)

#

ok, it cooperated

toxic torrent
#

cooperate daemon

carmine nacelle
#

I think I'm rebuilding rustc like 7th time today?

#

(the only saving thing is that I'm using my desktop, where unlike my laptop wsl2 doesn't crash every time I compile anything, because it (laptop) "only" has 32GB of RAM...)

carmine nacelle
#

Why this now

carmine nacelle
#

Oh no, I need to define my libc in libc crate

carmine nacelle
#

What am I doing wrong nooo

#

I think I now understand the hate of the compilers by the os developers

#

I want to kms (figuratively)

carmine nacelle
#

Cool, it's still broken with the latest Rust pain

#

But at least x86 still works

carmine nacelle
#

Nevermind, I think it compiled on my laptop?

carmine nacelle
#

My loongarch64 port is fully alive???

#

(I assume RISC-V would be as well then)

carmine nacelle
#

I think I'll finally do auxvec thing now

dense carbon
#

loongarch seems to be in fashion nowadays

carmine nacelle
#

I mean I haven't touched it for a year

carmine nacelle
#

I love C++

#

(Removed lto and recompiled the kernel)

#

Guess I'm just stuck with O3 and LTO pain

#

Also, something in here wants std::__1::__libcpp_verbose_abort(char const*, ...) for some reason?

carmine nacelle
#

I think at least the kernel's aux vector stuff is working...

carmine nacelle
#

I think userspace auxvec stuff is also working correctly?

#

Which means I can finally pass arguments when starting processes

carmine nacelle
#

So the kernel + libc is now using auxvec stuff

#

Which means it's time for futex?

carmine nacelle
#

Though finally having a disk sounds more fun (I already had an AHCI driver, I just need to redo VFS...)

carmine nacelle
#

x2apic has chosen violence

carmine nacelle
#

Cool, I've lost interrupts nooo

#

In both modes

carmine nacelle
#

smp breaks with x2apic now?

#

or maybe it's working but interrupts don't seem to work for some reason...

carmine nacelle
#

fuck it, I'm doing iommu

plain jungle
#

lol

#

Why fuck it?

carmine nacelle
#

I want to have bash already

#

But then I end up going down these rabbit holes

plain jungle
#

Didn't pmos have a lot more than bash already?

carmine nacelle
#

I don't have a shell

toxic torrent
carmine nacelle
#

I also wanted to figure out evdev or whatever before that

toxic torrent
#

why

carmine nacelle
#

Because I have no idea how io devices should be managed

#

Like I have a keyboard input and flanterm output. But what needs to happen so that you get a shell with pipes to those?

#

Hence 60k (or maybe 70k) LoC with no shell

twilit talon
#

Why do u need iommu lol

#

As a priority I mean

plain jungle
#

Ppl with kernels that consist of 90% log ring code are not allowed to argue about priorities halfmemeleft

left dew
#

the 30k loc log ring

#

enterprise grade logging infrastructure

carmine nacelle
kindred venture
carmine nacelle
#

Also, intel docs don't seem to be making too big of a deal of vt-d?

carmine nacelle
kindred venture
#

wdym

carmine nacelle
#

I've tried it and it doesn't work with x2apic

#

Idk maybe something else is busted

#

Like 1 << 24 for lapic 1

twilit talon
#

Do u treat icr as one register?

left dew
#

(u should)

carmine nacelle
#

my kernel doesn't crash

#

yeah

twilit talon
kindred venture
#

you use id << 56 for ioapic entries and msis, and id << 32 for ipis?

carmine nacelle
#

Ok, I'm dumb, I removed id << 32 and added 1d << 24 instead

#

it works

#

Does managarm support the amd vt-d clone thing?

#

My intel laptop died

kindred venture
carmine nacelle
#

Cool, less pages than intel

twilit talon
#

Vibe code it without a spec trl

carmine nacelle
#

no

#

even my copilot student thing had expired

twilit talon
#

Copilot is trash

#

Just use cursor

carmine nacelle
#

minecraft

twilit talon
carmine nacelle
#

Idk, I'm still reading

#

Doesn't seem too difficult?

plain jungle
#

Interrupt remapping is just a giant table

twilit talon
#

Idk how apic virt works

left dew
#

isn't it kinda like IOAPIC but on steroids

carmine nacelle
#

looks like it

#

ioapic for ioapic?

twilit talon
#

No I get that, im just curious what was the initial purpose of remapping interrupts

left dew
#

virtualization, servers

#

i cant think of many

plain jungle
#

It takes a (cpu, vector) pair and remaps it to an (actual cpu, actual vector)

left dew
#

but just because im dumb

twilit talon
carmine nacelle
#

it has a posted mode

plain jungle
#

There is one such table per BDF

twilit talon
#

Maybe im dumb I dont understand why that's needed

plain jungle
#

Because you don't want a PT device to interrupt storm an arbitrary host vector

twilit talon
#

So every time u move a vcpu thread u edit the table?

plain jungle
#

No, you just allocate some vectors for the VM and only let the VM use those vectors

#

In a way that's transparent to the VM

twilit talon
#

But that can be done even without a remapper no?

left dew
#

sure, but i assume it's not as straightforward as with the iommu approach

#

idk what im talking about tho

twilit talon
#

Same

#

I need to look into how kvm virtualizes the apic

left dew
#

vt-x has apic virtualization support

plain jungle
#

Remember that the device can write to arbitrary MSI addresses

left dew
#

but you can also make apic accesses trap(vmexit)

twilit talon
#

So where do u redirect to? Just arbitrary kvm handler that then injects the irq into the guest?

plain jungle
#

At least that's the simplest case

#

As mishakov said, there also a posted interrupt mode but that's relatively new

#

Iirc gen 12+ or something like that?

twilit talon
#

What's a posted interrupt?

plain jungle
#

Basically a table in RAM that vt-x can read to inject the interrupt into the guest

#

Without a kvmexit

twilit talon
#

Damn

carmine nacelle
#

thonk do I finally switch my kernel to uACPI (table only thing)?

#

Or maybe it's not that bad actually

plain jungle
#

Ah yeah don't forget to map the RMRRs or some hw will fail catastrophically :^)

twilit talon
#

Uacpi might have some of those tables but probably not all

carmine nacelle
#

Then I can look into adding them?

twilit talon
#

That would be cool since ill also need them

carmine nacelle
#

Also there's a lot of other stuff which uses ACPI

#

in my kernel

twilit talon
#

If u do pls take a look at the other tables in the file so u can match the naming style etc

carmine nacelle
#

ofc

#

but I'd need to switch to it uACPI first then

#

before adding this stuff

twilit talon
#

Barebones mode should be pretty lightweight

toxic torrent
twilit talon
#

Smp probably

carmine nacelle
#

timers, ioapic, a bunch of stuff in risc-v and loongarch

carmine nacelle
#

(or if I decide to add ultra protocol to amd64 port as well)

#

probably more stuff

#

idr, I haven't thouched it in a long time

#

Do they want me to move PCI enumeration to kernel??

twilit talon
#

I guess that's why managarm does it in the kernel

carmine nacelle
#

This implies moving full ACPI into kernel as well

kindred venture
kindred venture
carmine nacelle
kindred venture
#

yeah but not to discover ioapics

carmine nacelle
#

Some platforms may also expose I/OxAPICs as PCI-discoverable devices

kindred venture
#

that doesn't mean they're not in the madt

#

that just means they're in both

carmine nacelle
#

oh

kindred venture
carmine nacelle
kindred venture
#

if the device is behind a (tree of) bridges the device id is the one of the first bridge

#

afaict

carmine nacelle
carmine nacelle
toxic torrent
carmine nacelle
kindred venture
#

and when setting up the device-specific iommu configuration

carmine nacelle
#

I need to think about it

#

Because I wanted to do iommu management in kernel, together with ioapic

#

But things are connecting in my mind yet

#

how this whole thing works

twilit talon
#

If u want iommu management in kernel u need acpi there as well

#

Like the full thing I guess

kindred venture
#

no you don't?

twilit talon
#

Because it's tied to PCI

kindred venture
#

only the matching

carmine nacelle
#

Not really

twilit talon
carmine nacelle
#

But that's what I'm trying to understand

kindred venture
#

you have to blindly trust at least some of userspace if you want to put acpi there

carmine nacelle
#

You'll get a handle to device as a driver

kindred venture
#

since acpi needs to be able to access arbitrary physical addresses

carmine nacelle
#

and then the devices server will have to give a handle to iommu or something?

#

Anyway

#

320 pages document

kindred venture
#

the way i'd do it is devices server gives driver a handle to an iommu domain

carmine nacelle
#

but then I wanted to (eventually) tie it to memory allocation

#

or not?

#

it has to be tied to vmm

kindred venture
#

that'd be a good idea yes since the iommu is involved in allocating dma buffers

kindred venture
carmine nacelle
#

Like I have memory objects, so there will be a syscall to give some device access to that

kindred venture
#

and that 'dma space' can then finally be used to allocate and/or map dma buffers

carmine nacelle
#

Yeah

#

I think I'd need to read amd's thing as well before I implement all of this

#

(and I want to do interrupts first anyway)

kindred venture
#

i think amd vi is very similar to vt-d

carmine nacelle
#

and I don't have MSI

carmine nacelle
carmine nacelle
kindred venture
#

also i'd design the api in such a way that it can support a bounce-buffer-based 'software iommu', so that the drivers can be written in a way that assumes an iommu is present

carmine nacelle
#

how would it work?

kindred venture
#

allocate would just allocate in a certain physical range, as indicated by the dma space setup parameters

kindred venture
#

map would map in a preallocated bounce buffer instead of the given address, and then upon unmap copy the contents of that buffer

carmine nacelle
#

Ok, yeah

kindred venture
#

but then again i don't think iommu mapping (as opposed to allocation) is an operation that needs to be exposed to userspace

carmine nacelle
#

I would need an explicit unmap

kindred venture
#

it's only necessary for io into the page cache, which is initiated by the kernel vmm

carmine nacelle
kindred venture
#

you could have the kernel, upon pagein/out, set up the dma mappings, give userspace a dma address to perform io with, and upon transaction end tear down the dma mappings again

carmine nacelle
#

Yeah

#

What I need is a way to signal end of transaction to kernel

kindred venture
#

you need that for pagein/out anyway

#

since the kernel needs to unblock the thread(s) that caused the event

carmine nacelle
#

I had something like this already

#

Anyway

carmine nacelle
#

(But I had that working in the beginning)

carmine nacelle
#

What I don't understand is this

#

(Also, I have a thunderbolt 3 egpu, I wonder if I could support it eventually...)

kindred venture
#

it does this by having an 'interrupt source id' in the interrupt table entry

#

this section documents how to get the source id for a given ioapic

#

you don't have to worry about it though since this source verification is optional and can be disabled

#

look at section 9.9 fields SVT, SQ, and SID

plain jungle
#

For example, Bluerock's stack has iommu in kernel, acpi in userspace

plain jungle
#

for the IOMMU case

kindred venture
#

true but not every dma transaction requires the buffer to also be mapped in the virtual address space, so for those transactions you'd have a pretty large overhead due to the cpu side of mapping/unmapping the buffer

#

tlb shootdowns for example

plain jungle
#

re exposing mapping vs allocation: On Managarm you can "just" map a memory object into multiple spaces

#

so you can take your CoW or page cache object and map it into a VM, for example

twilit talon
#

@kindred venture how's your os going btw

kindred venture
#

still waiting on motivation

twilit talon
#

Lol

carmine nacelle
#

Wait, how does this happen?

plain jungle
#

you have both local and remote changes

#

it says so in the msg

carmine nacelle
#

I'm dumb, I confused pull and clone

#

Anyway, this is fun

inland temple
#

new sysdep dropped

carmine nacelle
#

But why unknown type in uacpi's headers?

kindred venture
#

do you override arch_helpers.h?

carmine nacelle
#

This is weird

kindred venture
#

because that's what defines uacpi_interrupt_state

carmine nacelle
#

Yeah, I have it

kindred venture
#

then it's just that your version doesn't define that

carmine nacelle
#

I don't think so

kindred venture
#

well yeah you have the file and the override

#

but your override doesn't define one of the things that it has to

carmine nacelle
#

but it's not the override, I think?

kindred venture
#

sure looks like it

carmine nacelle
#

Yeah

#

Sorry

#

I had this

kindred venture
#

yeah the latest major version update of uacpi added another thing to arch_helpers.h that you'll have to override

#

uacpi_interrupt_state

carmine nacelle
#

I need to figure out how I was doing S3 stuff

#

I think this will do trl

vital geode
hexed acorn
carmine nacelle
#

Wait

carmine nacelle
#

Can I use uacpi_setup_early_table_access with dynamic size?

carmine nacelle
#

I dreamt that I should just malloc a large buffer and that's it KEKW

twilit talon
carmine nacelle
#

where?

twilit talon
#

If it fails with some size, just 2x the size and call it again

twilit talon
#

#1217009725711847465 message

carmine nacelle
#

Cool

#

Didn't think of that

carmine nacelle
#

I must have accidentally erased it???

#

This is fun trl

#

This pointer looks off...

carmine nacelle
#

Did anyone test it?

#

(cc @twilit talon)

#

It works otherwise

twilit talon
#

wdym

#

did anyone test parsing garbage instead of rsdp?

carmine nacelle
#

yes

twilit talon
#

i mean, it will map it, then it's up to you what virtual address you return as a result

carmine nacelle
#

but why did it crash

twilit talon
#

if u catch a part where it's missing OOB checks that can be fixed

carmine nacelle
#

like trupple fault

carmine nacelle
#

Because my kernel doesn't use hhdm

twilit talon
#

i mean if it triple faults that means u didnt catch the GPF/page fault

carmine nacelle
#

So because everyone else probably does, there might be a bug somewhere?

carmine nacelle
twilit talon
#

do u have idt setup at this point?

#

u should really have exception vectors setup at this point

#

its like the first thing u should do

#

gdt+idt init

carmine nacelle
#

no

#

I set it up after this (but also because afaik it's using tls? though if I get interrupt in ring 0, it should just use kernel's stack

carmine nacelle
#

Cool, loongarch is just trolling me

#

Everything is broken, unless I enable a framebuffer in kernel

twilit talon
#

this is like astral

#

it needs a printf to work

carmine nacelle
#

but it was working before

carmine nacelle
#

I'm blaming uACPI trl

#

(or maybe it's the same bug as on that cursed laptop where there was no framebuffer output for no reason)

#

This patch is sus

#

(Which got pulled in after rebasing the kernel framebuffer branch)

#

I sense some serious trolling

#

queue in discovering something basic being extremely broken in a year

#

(like my set implementation, which was just randomly and casually locking up the kernel on deletion)

#

No, this is serious trolling

#

I think I'll just assume I haven't seen anything and carry on as if it was working perfectly trl

#

It's probably some UB, which I'll probably catch on risc-v

#

Wtf, why is there RCHT and RHCT?

#

Wait, it's just my headers

carmine nacelle
#

Huh, risc-v rustc had suddenly decided to compile???

carmine nacelle
#

Cool, RISC-V also just works

#

So loongarch64 is the only one broken?

#

(I haven't fixed i686 yet though...)

carmine nacelle
#

(though this is weird)

#

how did this stuff even work in the first place?

#

I haven't touched this in a long time

carmine nacelle
#

I give up, I think I'll just wrap it with mutex

carmine nacelle
#

This is so obnoxious, how do I just force it?

twilit talon
carmine nacelle
#

It's not working nooo

#

I think I need to patch clang

#

I'm going insane

twilit talon
#

And it must be at the very top

carmine nacelle
#

set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY") worked...

#

But I need to figure out how to disable libyaml

#

(tbh the best solution would be figuring out why clang doesn't append it automatically)

carmine nacelle
#

New fun issue with hyper: my limine config

twilit talon
#

What's a module command line

#

I mean if its just a string I could add that if you want

vital geode
#

Instead of having a metadata struct inside the module

twilit talon
#

Fair question

carmine nacelle
#

I was doing this in hyper. but the issue is with quotation marks

carmine nacelle
#

or package it into an archive

#

the modules can't just be elf files

vital geode
#

What

carmine nacelle
#

Those are not kernel modules

vital geode
#

Who parses them then?

carmine nacelle
#

userspace

twilit talon
vital geode
#

Well the elf interpreter that parses them can still look for a metadata struct?

twilit talon
#

U can use '"data"'

carmine nacelle
#

I don't know python galaxybrain

twilit talon
#

U can wrap strings in '...'

#

Then have " inside

carmine nacelle
#

cool

#

then it's not an issue

kindred venture
twilit talon
#

^

#

Thats kind of what it was designed for

vital geode
#

This too

carmine nacelle
vital geode
#

Although I have no idea how those work so I just use a struct

carmine nacelle
#

But I also wanted to pass initramfs that way

#

eventually

#

when I fix my vfs

#

then this would not be an issue, since I can just pass it separately

carmine nacelle
kindred venture
#

the linker will recognize the SHT_NOTE and put it into a PT_NOTE

carmine nacelle
#

So I just need to throw this in a random C/asm file?

kindred venture
#

yeah

carmine nacelle
#

cool

kindred venture
#

if you do it in C i'd use a global asm statement with .pushsection/.popsection

hexed acorn
#

we use elf notes in managarm for passing a bunch of info between eir and thor

twilit talon
#

Pushsection is overpowered

vital geode
hexed acorn
kindred venture
#

i really like elf notes because it's a standardized format for passing arbitrary non-runtime data in an elf file

carmine nacelle
#

what I don't like about it is that the file itself encodes the capabilities

#

but since everything kinda has access to everything anyway it's not an issue

twilit talon
#

Well because it knows its capabilities?

carmine nacelle
#

(for now)

twilit talon
#

I think it makes sense

kindred venture
#

if the capabilities are inherent to the file, for example if it describes what the code supports, that's a good thing

twilit talon
#

Like a service should be responsible for listing the capabilities it supports I think

kindred venture
carmine nacelle
#

yeah, that's an (eventual) idea

#

(I mean it's easy to implement, just not the priority)

#

Yesterday I decided I wanted to do iommu first galaxybrain

kindred venture
#

huh i thought it'd need SHT_NOTE

hexed acorn
#

and we don't do anything special aside from that, just mention them in the linker script so they end up in a segment

kindred venture
#

unless the assembler automatically uses that type for the .note prefix

hexed acorn
#

might be

#

for .note.GNU-stack you also only need ```
.section .note.GNU-stack,"",%progbits

carmine nacelle
#

I'll check later

hexed acorn
#

in an assembly file

kindred venture
#

it's not a "real" note

carmine nacelle
kindred venture
#

since it ends up in PT_GNU_STACK

hexed acorn
kindred venture
carmine nacelle
#

This is really cool

hexed acorn
carmine nacelle
#

Just drop a random elf file, you have a driver

kindred venture
#

well technically it's to say what stack permissions the assembly code needs

#

but in reality yeah it's just to make the linker shut up

hexed acorn
#

i mean yeah but in the kernel you allocate stacks at runtime anyway (although ig you could go and look at gnu-stack and respect it)

twilit talon
#

Jit in kernel with executable stacks

hexed acorn
#

hmm today i will define a function in a statement expression and return a pointer to it and call it

#

(it'll put exec code on the stack)

twilit talon
#

Yeah thats a funny gnu extension

kindred venture
#

also officially a note type id is only valid for a given namespace (which is specified by name and namesz) but in reality a bunch of tools assume some type ids have the same meaning regardless of namespace

#

just something to be aware of

carmine nacelle
#

(and it's not even my i8042 definition)

twilit talon
#

Im going to increase it I think

#

There's an open pr

#

From oberrow

carmine nacelle
#

512 chars is not enough trl

carmine nacelle
twilit talon
#

Technically its not limited by anything, its just the loader currently stores that on the stack iirc

carmine nacelle
#

Ok, I'll move it to elf notes

twilit talon
#

Basically I need to get rid of that limit, especially since the protocol specifies that its limited by u32 size

carmine nacelle
#

Also, I think the PR is for cmdline?

twilit talon
#

Ah right

carmine nacelle
#

Ok, I don't know what I want to do now

carmine nacelle
#

DMAR table btw

#

"x2apic is easy!"

carmine nacelle
carmine nacelle
#

thonk though if x2apic is supported, but iommu isn't, then it implies it shouldn't be used?

carmine nacelle
twilit talon
carmine nacelle
twilit talon
#

thats relevant only when you use iommu

#

fwiw linux doesnt check this field at all

carmine nacelle
#

queue in "linux doesn't boot on my random server" lkml thread

twilit talon
#

dmar.c is from 2006

#

kinda doubt it has any problems like that

carmine nacelle
#

x2apic is newer

#

anyway

#

it weird, like as an operating system you're expected to set up iommu before apic?

#

(maybe what's weird is doing smp in bootloader)

twilit talon
#

actually i was wrong

#

it just duplicates acpi bits for some reason so i didnt find it initially

twilit talon
carmine nacelle
#

None

twilit talon
#

just dont use x2apic if u use iommu and see opt-out

carmine nacelle
#

x2apic only systems

#

and limine

twilit talon
#

what about them

carmine nacelle
#

since it doesn't check this

twilit talon
#

u dont have to use the smp feature (and u shouldnt)

carmine nacelle
#

I mean I have init code from Ultra

#

(on i686 though)

#

Though it should be easily adaptable I guess

carmine nacelle
#

So you are forced to use it there

twilit talon
#

well risc-v doesnt have this problem

carmine nacelle
#

yeah

twilit talon
#

btw

#

if im reading linux code correctly

#

x2 opt out doesnt disable x2

#

it just limits apic ids to 255

carmine nacelle
#

huh

kindred venture
twilit talon
carmine nacelle
#

It wasn't there when I ported to risc-v

twilit talon
#

it does try to disable x2 apic, but that checks if x2 apic is locked, and if it is it just doesnt disable it

carmine nacelle
#

But I think the bios might lock you from disabling apic (which prevents you from groing x2 -> x), but still allows xapic?

twilit talon
#

how can it lock disabling apic?

carmine nacelle
#

Idr, there was some bit for it

#

Somewhere in MSRs

twilit talon
#

dont think so, at least i didnt see such a bit

carmine nacelle
#

I just read SDM on it

twilit talon
#

or checks for it in linux

carmine nacelle
#

This is how x2apic lock works afaic?

twilit talon
#

u sure ur not mixing it up with apic enable/disable in apic base?

carmine nacelle
#

I could be

#

It's almost 1 am

twilit talon
carmine nacelle
#

XAPIC_DISABLE_STATUS is separate from IA32_XAPIC_DISABLE_STATUS[0] ?

#

I mean idk how this works

carmine nacelle
twilit talon
#

Lol

carmine nacelle
#

tbh, it looks similar to pci hotplug?

kindred venture
#

yes? what does this have to do with that

carmine nacelle
#

Is it not optional then?

kindred venture
#

it is

carmine nacelle
#

Or am I confusing it with something else?

kindred venture
#

i think you are yes

#

ATC stands for address translation cache

#

it has nothing to do with interrupt source verification

carmine nacelle
#

oh

#

nevermind then

#

atc didn't look too difficult

#

I'll check it tomorrow

#

I forgot the comment anyway

#

THe tables are not too bad

#

(and the whole thing in general)

#

AMD is also just one table

twilit talon
#

At a quick glance looks good

#

Although for reserved I use rsvd

carmine nacelle
#

I think I forgot to mention, I've completely switched my kernel to uACPI (barebones mode) for table parsing

twilit talon
#

Cool cool

carmine nacelle
twilit talon
#

Damn

carmine nacelle
#

Yeah, there are several places where it's reserved

twilit talon
#

Did I actually write that

#

Or is that contributed by someone

carmine nacelle
#

time for git blame

#

(I don't know how to use it)

twilit talon
#

Just git blame <filename>

#

Btw is dmar entry HDR different from generic entry HDR?

carmine nacelle
#

yes

#

it's 2 uint16_ts

#

I think I saw some table in acpi.h where it was {u16; u8;} as well?

#

or something like that

#

(but I think it's because these entries might be larger than 256 bytes)

#

because most of them have paths and stuff

twilit talon
#

Thats really annoying

twilit talon
carmine nacelle
#

yeah, but not too insane

carmine nacelle
#

just grep over it?

twilit talon
#

Not at my PC but yeah

carmine nacelle
#

I mean I'm also in bed with my laptop

twilit talon
#

Lol

vale hinge
#

it wasn't really hard to find

plain jungle
#

Managarm runs on Limine on rv without SMP request just fine

carmine nacelle
#

I kinda read the entire AMD spec, and it doesn't look too complicated

carmine nacelle
carmine nacelle
carmine nacelle
carmine nacelle
#

type flags length

twilit talon
carmine nacelle
#

I think I'm too unmotivated to do iommu now

#

I'll probably do disk instead

toxic torrent
#

what disk

carmine nacelle
#

ahci

#

I had a working driver

#

But I wanted to do all of the stuff around it

#

Since it would just start, discover the ahci controllers, and fork() itself for each controller, and I don't like fork

#

So I wanted to finish the pmbus pci discovery thing, then the disk discovery, then finally do the ext4 driver, and then rewrite the VFS mess, after which I could do futex, and switch to mlibc, and hopefully have bash

#

etc., etc.

carmine nacelle
#

I think I'll just get rid of SMP request from limine first KEKW

#

Since I have code for i686 already, I just need to adapt the trampoline to AMD64??

#

Wtf is this comment

#

The misteries of programming

#

I remember writing it, but don't remember why

carmine nacelle
#

What if I randomly add multiboot2 back? thonk trl

#

(I want to unify amd64 and i686 initialization code anyway)

carmine nacelle
#

Lol, aparently I've been using limine base rivision 1

#

this whole time

carmine nacelle
#

This is concerning...

#

I think I have memory corruption??

#

I'm blaming the new limine revision trl

#

Like it didn't find madt when initializing lapics, but did find it for ioapic??

#

I think the issue is with the idle_cr3 variable...

#

What the fuck

#

I changed SMP initialization, and my whole mm just decided to die

#

I need a second opinion from a different bootloader nooo

#

No, my vmem is returning weird addresses

carmine nacelle
#

Another stupid vmem bug rectified KEKW

#

Idk how, but the code is not too difficult, and it's used all over the kernel, but every so often some bug comes up in there, for some stupidest reason

#

Now the kernel just happily tripple faults KEKW

#

(I'm going insane)

carmine nacelle
#

Epic commit though

#

Cool, limine is refusing to boot in bochs

#

Nevermind

carmine nacelle
plain jungle
#

How do you even get your kernel to triple fault

carmine nacelle
#

smp init code's busted

plain jungle
#

this basically means that you double fault handler isn't working

carmine nacelle
#

It was working like a year ago

twilit talon
carmine nacelle
#

(Because I've ditched limine's MP request, and doing SMP by muself)

#

For some reason the kernel isn't mapped

#

(But like I've merged x86_64 and i686 code together without looking, so there's probably a dumb error somewhere in there)

carmine nacelle
#

It gets to the -2GB code (with the correct address) and tripple faults

carmine nacelle
#

Now it just crashes KEKW

#

But at least it's after the full userspace init and stuff??

#

It's probably the race condition

#

I think it's working? (though CPU id 5 is fishy...)

carmine nacelle
#

But x2APIC is broken for no reason?

#

(Omg, -O3 lto is just one big function, this is insane)

twilit talon
#

mfw no abortable instructions trl

carmine nacelle
#

It doesn't like wrmsr

#

To APIC_ICR_HIGH?

#

Wtf

twilit talon
#

theres no apic icr high

carmine nacelle
#

Why is it writing to that in x2apic mode

twilit talon
#

theres one icr register

carmine nacelle
#

I know

twilit talon
carmine nacelle
#

But like the thing is

#

And looking at assembly it's just one giant initialization function

#

And my kernel doesn't compile without -O3 -flto

twilit talon
twilit talon
carmine nacelle
twilit talon
#

no i mean

#

addr2line should work

#

for gdb backtrace to work u need proper cfi annotations in your isr code

carmine nacelle
#

I have -g

twilit talon
#

at least do -g3

carmine nacelle
carmine nacelle
twilit talon
#

huh

#

well

carmine nacelle
#

But like this is the assembly

#

It's all over the place

twilit talon
#

if u want wrmsr to not kill your kernel u should make them abortable

twilit talon
#

i implement it same way you would implement copy_to_user

carmine nacelle
#

I mean it doesn't matter probably

twilit talon
#

catch the exception and move rip

carmine nacelle
#

oh

#

I was looking in the wrong file

twilit talon
#

btw

#

how did u manage to make a kernel that only builds with o3

carmine nacelle
#

Idk

#

Maybe it builds with less

twilit talon
#

whats the error with o0

carmine nacelle
#

But linker crashes without -flto

twilit talon
#

which error?

carmine nacelle
#

(because I don't wait for anything, etc.)

carmine nacelle
twilit talon
#

All excluding self wakeup is completely wrong as well

#

you wake them up in the order specified in madt

#

some cpus might be marked as disabled also

carmine nacelle
#

can you wake them up in paralel?

twilit talon
#

if you ipi them you might die etc

twilit talon
#

linux sends the entire init-sipi sequence in the order

#

then waits

carmine nacelle
#

This is with -O0

twilit talon
#

huh

carmine nacelle
#

(Not too bad tbh)

#

It's some std::optional::get or something

#

I should've just used *

#

It gets optimized away with -O3

#

(the checks)

carmine nacelle
twilit talon
#

nope ofc not

#

its more complicated then that

#

Check if lapic entries exist at all, if they do, you must use them for apic_id < 0xFF

#

lapic entries take precedense, otherwise use x2

#

they may be located in madt after x2, so u have to iterate it multiple times

carmine nacelle
#

I'm just using a set so I don't go over them twice

twilit talon
#

u must go over twice to know what to ignore

#

first iteration is just a check whether lapic entries exist at all

carmine nacelle
#

So this is also how I can detect that I'm not booted on BSP?

twilit talon
#

nah

#

whether ur on a bsp is very easy to check

#

msr_base & 1 << 9

carmine nacelle
#

the flag in the msr?

twilit talon
#

yes

carmine nacelle
#

does it get cleared if I disable lapic?

#

e.g. to go into xapic from x2apic

twilit talon
#

no ofc not

carmine nacelle
#

though I'm restoring it

carmine nacelle
#

thonk Do I also fix my lapic/tsc calibration?

twilit talon
#
static uacpi_iteration_decision detect_lapics(
    uacpi_handle user, struct acpi_entry_hdr *hdr
)
{
    struct madt_iter_ctx *ctx = user;
    u32 apic_id;
    u32 flags;

    switch (hdr->type) {
    case ACPI_MADT_ENTRY_TYPE_LAPIC: {
        struct acpi_madt_lapic *lapic = (struct acpi_madt_lapic*)hdr;

        if (lapic->id == LAPIC_ID_NONE)
            return UACPI_ITERATION_DECISION_CONTINUE;

        flags = lapic->flags;
        apic_id = lapic->id;
        break;
    }

    case ACPI_MADT_ENTRY_TYPE_LOCAL_X2APIC: {
        struct acpi_madt_x2apic *x2apic = (struct acpi_madt_x2apic*)hdr;

        if (ctx->has_valid_lapic_entries && x2apic->id < 0xFF)
            return UACPI_ITERATION_DECISION_CONTINUE;

        if (x2apic->id == X2APIC_ID_NONE)
            return UACPI_ITERATION_DECISION_CONTINUE;

        flags = x2apic->flags;
        apic_id = x2apic->id;
        break;
    }

    case ACPI_MADT_ENTRY_TYPE_LSAPIC:
        /*
         * Apparently these might exist even on x86, but there's no confirmation
         * of that ever happening, so guard against that explicitly.
         */
        panic("Local SAPIC entries are not supported\n");

    default:
        return UACPI_ITERATION_DECISION_CONTINUE;
    }

    if (cpu_usable(flags)) {
        error_t ret;

        ret = register_cpu(apic_id);
        if (is_error(ret))
            return UACPI_ITERATION_DECISION_BREAK;
    }

    return UACPI_ITERATION_DECISION_CONTINUE;
}
#

This is what I do

carmine nacelle
#

This is what I get without lto btw

twilit talon
#

your build flags are very fucked

carmine nacelle
#

It's PIE

carmine nacelle
kindred venture
carmine nacelle
#

It should be compiling as PIE

kindred venture
#

are you doing that before or after creating the kernel target

#

if you're doing it after it has no effect

#

you should replace both of those with target_compile_options and target_link_options

twilit talon
#

i have one extra iteration before that that sets has_valid_lapic_entries = true

#
static uacpi_iteration_decision check_if_madt_has_lapic(
    uacpi_handle user, struct acpi_entry_hdr *hdr
)
{
    struct madt_iter_ctx *ctx = user;

    if (hdr->type == ACPI_MADT_ENTRY_TYPE_LAPIC) {
        struct acpi_madt_lapic *lapic = (struct acpi_madt_lapic*)hdr;

        if (!cpu_usable(lapic->flags))
            return UACPI_ITERATION_DECISION_CONTINUE;

        if (lapic->id == LAPIC_ID_NONE)
            return UACPI_ITERATION_DECISION_CONTINUE;

        ctx->has_valid_lapic_entries = true;
        return UACPI_ITERATION_DECISION_BREAK;
    }

    return UACPI_ITERATION_DECISION_CONTINUE;
}
carmine nacelle
twilit talon
#

lapic cant have id > FF

#

so u take any x2apic entries u can

kindred venture
twilit talon
#

linux treats 0xFF as invalid id

#

or 0xFFFFFFFF for x2

kindred venture
#

yeah but this is x2

twilit talon
#

yeah good catch that has to be <=

#

wait no

#

it is correct

#

<= 0xFF means it would skip an apic id of 0xFF

plain jungle
#

It shouldn't

twilit talon
# plain jungle why does INIT order matter?
To ensure that the boot processor is supported post initialization, two guidelines should be followed. The first is that OSPM should initialize processors in the
order that they appear in the MADT. The second is that platform firmware should list the boot processor as the first
processor entry in the MADT.
#

this is acpi spec

#

as for why idk, probably doesnt matter on most hw

carmine nacelle
carmine nacelle
#

so if x2apic entry comes before xapic, you initialize it in that order?

twilit talon
#

only if it has id < 0xFF

carmine nacelle
#

I don't understand

#

ok

twilit talon
#

or rather

#

if madt HAD any lapic entries that are valid: discard x2apic entries with apic id less than 0xFF

carmine nacelle
#

cool

#

then it's not that complicated?

twilit talon
#

yeah, just make sure u iterate it before u record any lapics

#

so u check if lapics exist

carmine nacelle
#

I guess I should also check the mailbox wakeup thing

twilit talon
#

not sure if anything ever implemented that

carmine nacelle
#

I mean it would have been needed for x86s

twilit talon
#

if only it hadnt been discarded

carmine nacelle
#

Anyway, this works now trl

twilit talon
#

any% smp

carmine nacelle
#

I mean it worked on my physical hardware as well

#

I think it will be easy to fix later

kindred venture
#

also i seem to remember someone saying a broadcast like that is a bad idea?

twilit talon
#

it is a really bad idea yeah

kindred venture
#

something along the lines of firmware-disabled CPUs responding to it

twilit talon
#

yep