uACPI - a portable and easy-to-integrate ACPI implementation | Operating System Development | Page 49

rustic compass Apr 16, 2025, 4:01 PM

#

done

torpid root Apr 16, 2025, 4:02 PM

#

I heard that thin lto is the recommended kind for incremental builds but it costs me like 700k points

rustic compass Apr 16, 2025, 4:40 PM

#

who needs incremental builds? trl

frank canopy Apr 16, 2025, 4:41 PM

#

i dont use incremental lol

#

its still in early experimental phase for zig

rustic compass Apr 16, 2025, 4:47 PM

#

also for getter with unknown size like arrays try to use one interface without allocations (for example use a index approach), one with alloc and one with allocator_api

#

as an example look at the trait AcpiTable

rustic compass Apr 16, 2025, 5:08 PM

#

fn get_ptr_to_table_payload(&self) -> *mut u8

points to the first byte after the table header

slim panther Apr 16, 2025, 5:30 PM

#

rustic compass who needs incremental builds?<:trl:829316641191886888>

i do

trim frost Apr 16, 2025, 5:30 PM

#

rustic compass done

did you publish it to cargo or should i clone the repo?

torpid root Apr 16, 2025, 5:32 PM

#

rustic compass who needs incremental builds?<:trl:829316641191886888>

thin lto just supposedly makes faster than the full and fat ones

#

but takes more space and the performance is worse

rustic compass Apr 16, 2025, 5:33 PM

#

trim frost did you publish it to cargo or should i clone the repo?

clone the repo

#

its not ready to be published to cargo

trim frost Apr 16, 2025, 6:09 PM

#

#

@rustic compass

rustic compass Apr 16, 2025, 6:11 PM

#

branch "dev-1.0" btw

trim frost Apr 16, 2025, 6:12 PM

#

ffs i didnt even look at it

#

main branch

rustic compass Apr 16, 2025, 6:12 PM

#

also ignore the ton of errors

kind mantle Apr 16, 2025, 6:16 PM

#

trim frost

__popcountdi2 is a libgcc (and FWIW probably clang too) function that count how many bits are set in an integer

#

So if you don't compile that into your kernel you may need to define some of them yourself.

trim frost Apr 16, 2025, 6:16 PM

#

rustic compass also ignore the ton of errors

how can i ignore that many errors
what do i do about it not compiling because of said errors

trim frost Apr 16, 2025, 6:17 PM

#

kind mantle So if you don't compile that into your kernel you may need to define some of the...

i was on a wrong branch

rustic compass Apr 16, 2025, 6:18 PM

#

you can mask most of them by removing the aml_interpreter feature and probably replace the two errors in tables.rs with todo

#

"full_acpi_hardware"

trim frost Apr 16, 2025, 6:20 PM

#

still ton of errors

#

mostly types

#

error: could not compile uacpi (lib) due to 102 previous errors; 7 warnings emitted

rustic compass Apr 16, 2025, 6:23 PM

#

with default = [] i only get 3 errors

#

one of them is stupid rustc

#

comment out line 393 in types.rs and replace both implementations in tables.rs with todo!()

trim frost Apr 16, 2025, 6:25 PM

#

line 393 in types.rs is inactive since no default features (aml_interpreter)

#

why do i get 102 errors?

rustic compass Apr 16, 2025, 6:26 PM

#

rustc still reports an error there

#

dunno

#

where do you get them?

trim frost Apr 16, 2025, 6:27 PM

#

mostly u32s instead of i32s

#

these 2 in tables

#

yeah its those 2 in tables and rest are u32s instead of i32s errors

rustic compass Apr 16, 2025, 6:28 PM

#

trim frost these 2 in tables

replace those with todo!() temporarily

trim frost Apr 16, 2025, 6:29 PM

#

trim frost mostly u32s instead of i32s

what about these?

rustic compass Apr 16, 2025, 6:32 PM

#

on what system do you compile it?

trim frost Apr 16, 2025, 6:33 PM

#

linux

#

arch

rustic compass Apr 16, 2025, 6:34 PM

#

aparently on linux c_int is a u32 while on windows its a i32

trim frost Apr 16, 2025, 6:34 PM

#

bruuuh

kind mantle Apr 16, 2025, 6:36 PM

#

That doesn't make any sense

#

Why is c_int ever a u32?

#

It should be signed, like it implies with its name.

trim frost Apr 16, 2025, 6:37 PM

#

its because its not c_int

#

its c_uint

rustic compass Apr 16, 2025, 6:39 PM

#

looks like bindgen chooses a c_int for me

trim frost Apr 16, 2025, 6:39 PM

#

yeah

#

thats weird

kind mantle Apr 16, 2025, 6:40 PM

#

LLM answer: Enum sign is not specified so it can be either.

#

Maybe you can tell bindgen you want a specific sign for your enums?

trim frost Apr 16, 2025, 6:41 PM

#

thats what im trying to figure out rn

rustic compass Apr 16, 2025, 6:41 PM

#

i use clang

trim frost Apr 16, 2025, 6:42 PM

#

found it

#

.default_enum_style

#

i think

#

hold on

#

.default_enum_style(bindgen::EnumVariation::Rust { non_exhaustive: false })
.rustified_enum_variant_type(Some(bindgen::EnumVariantCustomType::Int32))

#

gpt halucinating as always

trim frost Apr 16, 2025, 7:15 PM

#

et voila

#

i fixed it

#

very horribly

#

build.rs

#

@rustic compass is this it?

#

intellisense doesnt show core functionality

#

okay i will wait for uacpi-rs to be finished and meanwhile i will use the bindgen output

#

aww hell naw

rustic compass Apr 16, 2025, 7:46 PM

#

kernel api isnt implemented yet

trim frost Apr 16, 2025, 7:48 PM

#

how can i edit includepaths for the c files?

rustic compass Apr 16, 2025, 7:49 PM

#

? just modify them in build.rs

trim frost Apr 16, 2025, 7:49 PM

#

trim frost aww hell naw

doesnt work clearly

#

going to the c files gives me errors like this

torpid root Apr 16, 2025, 8:10 PM

#

rustic compass aparently on linux c_int is a u32 while on windows its a i32

int is always signed

rustic compass Apr 17, 2025, 5:55 AM

#

trim frost going to the c files gives me errors like this

It's possible that git didn't clean up when setting the submodule to a newer commit

#

And I forgot to check that and update the build script

rustic compass Apr 17, 2025, 6:06 AM

#

torpid root int is always signed

its still a problem that bindgen chooses c_int for windows and c_uint for linux

trim frost Apr 17, 2025, 6:10 AM

#

trim frost build.rs

this must do for now

rustic compass Apr 17, 2025, 7:17 AM

#

what compiler do you use?

#

based on github bindgen issues linux isnt spec compliant here, enum should always be i32 on x86_64

#

https://github.com/rust-lang/rust-bindgen/issues/1907#issuecomment-799605221

GitHub

Add an option to override the representation of enums · Issue #190...

In https://github.com/nbigaouette/onnxruntime-rs I use bindgen in onnxruntime-sys/build.rs to generate bindings on different platforms (macos, linux, windows). The original C header file is https:/...

trim frost Apr 17, 2025, 7:34 AM

#

rustic compass what compiler do you use?

im pretty sure its rust-lld

#

oh i read as linker

#

latest nightly

#

rustic compass Apr 17, 2025, 7:51 AM

#

i mean the c compiler that cc/bindgen use

left orbit Apr 17, 2025, 7:54 AM

#

it'll be env!("CC")

#

so unless they explicitly provide that it should be gcc

thorny orbit Apr 19, 2025, 8:26 PM

#

K so, this might be unrelated to uACPI but is it normal that there's no 'FACS' table in memory

#

because i made a QEMU memory dump, read all the RSDP,R/XSDT, found all addresses by myself, and in the FADT i can see there's a value in the FIRMWARE_CTRL field

#

even tho it brings nowhere

#

}

kindred beacon Apr 19, 2025, 8:33 PM

#

when in doubt you can dump ACPI tables from the UEFI shell and inspect those

fiery turtle Apr 19, 2025, 11:20 PM

#

thorny orbit K so, this might be unrelated to uACPI but is it normal that there's no 'FACS' t...

nope

#

and im pretty sure qemu does have FACS

torpid root Apr 20, 2025, 12:37 AM

#

it does

frank canopy Apr 20, 2025, 4:05 AM

#

thorny orbit because i made a QEMU memory dump, read all the RSDP,R/XSDT, found all addresses...

isnt there an X_FIRMWARE_CTRL and if thats present you need to use that instead? idk though

#

and i dont know if the FACS region has a header since its not really a table

#

i could be wrong though

thorny orbit Apr 20, 2025, 6:36 AM

#

I mean, I'm pretty much leaving everything to uACPI, which does check for x_firmware_ctrl

#

But the problem is that it finds an address where there's no FACS apparently

#

Even tho I might try NVME shill's hint and see where this goes

fiery turtle Apr 20, 2025, 8:53 AM

#

can u explain the problem in more detail

#

what are you trying to do, what happens

#

if uacpi gets a bogus address that probably means you dont handle misaligned mappings correctly

#

read the comment above uacpi_kernel_map

thorny orbit Apr 20, 2025, 9:25 AM

#

So, to start, i was trying to initialize uACPI in my kernel, almost all tables get initlialized but when checking for FACS signature it simply doesn't find a signature (and there isn't any since i checked the memory dump)

#

so the problem is not anything related to address aligment since all the other tables worked fine 🤷‍♂️

fiery turtle Apr 20, 2025, 9:31 AM

#

Are you using early table access?

thorny orbit Apr 20, 2025, 9:31 AM

#

nope

#

or, from what i saw, early_table_access is still set to false at init

fiery turtle Apr 20, 2025, 9:32 AM

#

Set to false?

#

Wdym

thorny orbit Apr 20, 2025, 9:32 AM

#

one sec

fiery turtle Apr 20, 2025, 9:33 AM

#

Is this invalid table thing logged by you or uacpi?

thorny orbit Apr 20, 2025, 9:35 AM

#

I meant this value btw

thorny orbit Apr 20, 2025, 9:35 AM

#

fiery turtle Is this invalid table thing logged by you or uacpi?

uacpi

fiery turtle Apr 20, 2025, 9:35 AM

#

Oh ok

#

Can you show where it does it

thorny orbit Apr 20, 2025, 9:35 AM

#

OK

#

So, in the initialize_tables function, it tries to check for the table signature right here

fiery turtle Apr 20, 2025, 9:37 AM

#

Any way you're just not mapping it correctly?

#

Or corrupting uacpis context

thorny orbit Apr 20, 2025, 9:38 AM

#

i don't really think so

fiery turtle Apr 20, 2025, 9:38 AM

#

Yeah looks like you do

torpid root Apr 20, 2025, 9:38 AM

#

mapping it incorrectly could be the problem

fiery turtle Apr 20, 2025, 9:38 AM

#

There's an extra bogus 16 on top of its address

torpid root Apr 20, 2025, 9:38 AM

#

what's in your kernel api map function?

fiery turtle Apr 20, 2025, 9:38 AM

#

If u look at uacpis output

thorny orbit Apr 20, 2025, 9:38 AM

#

fiery turtle There's an extra bogus 16 on top of its address

oh wait

#

about that

#

it's because it's not actually written correctly in the log function

fiery turtle Apr 20, 2025, 9:39 AM

#

Hm?

#

Its written correctly

thorny orbit Apr 20, 2025, 9:40 AM

#

nope

gentle peak Apr 20, 2025, 9:40 AM

#

there's a literal 0x016 in front

fiery turtle Apr 20, 2025, 9:40 AM

#

Oh lol

#

Yeah

#

So that's not the issue then

thorny orbit Apr 20, 2025, 9:40 AM

#

yeah

fiery turtle Apr 20, 2025, 9:40 AM

#

Can u run something like Linux and see in dmesg what address it reports for facs

thorny orbit Apr 20, 2025, 9:41 AM

#

ok

fiery turtle Apr 20, 2025, 9:41 AM

#

And yeah show us your mapping function

#

Because that address looks somewhat correct

thorny orbit Apr 20, 2025, 9:42 AM

#

void *uacpi_kernel_map(uacpi_phys_addr addr, uacpi_size len) {
    uint64_t aligned = ROUND_DOWN(addr, PFRAME_SIZE);
    size_t offset    = (size_t)(addr - aligned);

    size_t actual_len = len + offset; // this is BYTES!!!
    size_t pages      = ROUND_UP(actual_len, PFRAME_SIZE) / PFRAME_SIZE;

    void *virt = vma_alloc(get_current_ctx(), pages, (void *)aligned);

    // re-align the pointer to original addr offset

    virt += offset;

    return virt;
}

gentle peak Apr 20, 2025, 9:43 AM

#

did you omit the actual mapping part or

thorny orbit Apr 20, 2025, 9:44 AM

#

or if you want the full code

https://github.com/purpleK2/purpleK2/blob/dev-unstable/src/kernel/acpi/uacpi_wrappers.c

GitHub

purpleK2/src/kernel/acpi/uacpi_wrappers.c at dev-unstable · purple...

Freestading 64-bit kernel (currently in very early development stage)) - purpleK2/purpleK2

fiery turtle Apr 20, 2025, 9:44 AM

#

Lol

thorny orbit Apr 20, 2025, 9:44 AM

#

gentle peak did you omit the actual mapping part or

that's part of vma_alloc

gentle peak Apr 20, 2025, 9:44 AM

#

you're not passing it the physical address though

#

how does it know what to map

#

oh wait you are

#

nevermind

thorny orbit Apr 20, 2025, 9:44 AM

#

yup

gentle peak Apr 20, 2025, 9:44 AM

#

the void * cast tripped me up

thorny orbit Apr 20, 2025, 9:45 AM

#

understandable

fiery turtle Apr 20, 2025, 9:45 AM

#

Btw how does your round down behave if the address is already aligned

#

Can you show it as well

thorny orbit Apr 20, 2025, 9:45 AM

#

oke

#

#define ROUND_DOWN(n, a) ((n) & ~((a) - 1))
#define ROUND_UP(n, a)   (((n) + (a) - 1) & ~((a) - 1))

they're just some macros

fiery turtle Apr 20, 2025, 9:47 AM

#

Would be easier if you had an hhdm we could quickly replace this logic with but oh well

thorny orbit Apr 20, 2025, 9:47 AM

#

you'd be right

#

🤔

#

now that you told me about it

fiery turtle Apr 20, 2025, 9:48 AM

#

thorny orbit ```c #define ROUND_DOWN(n, a) ((n) & ~((a) - 1)) #define ROUND_UP(n, a) (((n) ...

But this is fine I think

gentle peak Apr 20, 2025, 9:49 AM

#

is it possible you're corrupting the facs memory itself?

thorny orbit Apr 20, 2025, 9:49 AM

#

ermmm...

#

bruh

fiery turtle Apr 20, 2025, 9:49 AM

#

Ye

#

I guess your mapper is misbehaving

thorny orbit Apr 20, 2025, 9:50 AM

#

mh

#

well uhmm thanks and sorry nooo

fiery turtle Apr 20, 2025, 9:54 AM

#

Lol np

#

We found a print bug as well which is useful

thorny orbit Apr 20, 2025, 9:54 AM

#

👍

kind mantle Apr 20, 2025, 10:39 AM

#

}}}}}}}}}}

fiery turtle Apr 20, 2025, 4:32 PM

#

technically an api break so uacpi 3.0 time? trl

#

(jk i wont do that)

#

also as per shinribo's request, clarified the table api usage a bit

frank canopy Apr 20, 2025, 4:39 PM

#

nice

hollow elm Apr 20, 2025, 8:09 PM

#

fiery turtle technically an api break so uacpi 3.0 time? <:trl:829316641191886888>

add a new define and mark the old one as deprecated

#

then when enough deprecated defines accumulate make 3.0 that removes them?

fiery turtle Apr 20, 2025, 8:13 PM

#

i think i can go out on a limb and say that no one used this define

#

but a proper way would be that yeah

hollow crescent Apr 21, 2025, 7:17 AM

#

fiery turtle i think i can go out on a limb and say that no one used this define

https://github.com/search?q=uacpi_poisitive_decode&type=code several people actually

GitHub

Build software better, together

GitHub is where people build software. More than 150 million people use GitHub to discover, fork, and contribute to over 420 million projects.

fiery turtle Apr 21, 2025, 7:58 AM

#

most of these are just copied code or autogenerated bindings

#

but threre are two uses i think still

#

damn

#

okay then

#

ill do what qookie said

#

also why is there jai bindings damn lmfao

fiery turtle Apr 21, 2025, 8:24 AM

#

is there any way to make macros deprecated with a warning

hollow crescent Apr 21, 2025, 9:10 AM

#

fiery turtle is there any way to make macros deprecated with a warning

#ifdef X
#warning Y
#endif

i guess

torpid root Apr 21, 2025, 9:11 AM

#

hollow crescent #ifdef X #warning Y #endif i guess

me when -werror:

hollow crescent Apr 21, 2025, 9:12 AM

#

your fault for using -Werror

torpid root Apr 21, 2025, 9:12 AM

#

I don't but I can't stand having warnings

#

build log needs to be clean

fiery turtle Apr 21, 2025, 9:15 AM

#

hollow crescent #ifdef X #warning Y #endif i guess

i mean uacpi is what defines the warning

#

or rather, the macro itself

torpid root Apr 21, 2025, 9:19 AM

#

you could have a global function/object marked as deprecated and somehow reference it in the macro definition

#

#define UACPI_POISITIVE_DECODE (deprecated, 0)

fiery turtle Apr 21, 2025, 9:30 AM

#

interesting

#

could i make it a static inline function instead?

#

__attribute__((deprecated)) static inline int uacpi_poisitive_decode_value() { return X; }

#define UACPI_POISITIVE_DECODE uacpi_poisitive_decode_value()

#

maybe like this

calm latch Apr 21, 2025, 10:02 AM

#

the issue is that this is not a constant expression?

#

I think C allows to place deprecated onto variables? https://en.cppreference.com/w/c/language/attributes/deprecated

fiery turtle Apr 21, 2025, 10:06 AM

#

you can't make a global static inline tho

#

so i'd have to add it as an extern

#

which is fine i guess idk

calm latch Apr 21, 2025, 10:06 AM

#

yeah

fiery turtle Apr 21, 2025, 10:07 AM

#

calm latch I think C allows to place deprecated onto variables? https://en.cppreference.com...

this is c23, but the gcc attribute is pretty old yeah

calm latch Apr 21, 2025, 10:07 AM

#

but at that point maybe the function would be the best option

calm latch Apr 21, 2025, 10:07 AM

#

fiery turtle this is c23, but the gcc attribute is pretty old yeah

it was probably added after the compilers

#

like since everyone was using it so it became a language feature?

fiery turtle Apr 21, 2025, 10:08 AM

#

i'll just do #if __has_attribute(deprecated) etc

#

otherwise u get no warning and its your fault

calm latch Apr 21, 2025, 10:09 AM

#

hollow crescent https://github.com/search?q=uacpi_poisitive_decode&type=code several people actu...

This looked like it was just for bindings?

kind mantle Apr 21, 2025, 10:12 AM

#

I recommend what I did last time I had a similar problem: Keep the old one around but deprecate it. Since it's a macro you can't do much better than put a warning comment before it. Then next major version (3.x) you can remove the misspelled version.

fiery turtle Apr 21, 2025, 10:19 AM

#

yeah see the discussion above

fiery turtle Apr 21, 2025, 8:52 PM

#

Alright I think I'm back from my Minecraft/gtnh semi hiatus because I kind of hate how AE2 works, I might focus on my os ™️ project

jaunty fox Apr 21, 2025, 9:28 PM

#

fiery turtle Alright I think I'm back from my Minecraft/gtnh semi hiatus because I kind of ha...

based

#

i can't wait to see ultraos form

fiery turtle Apr 21, 2025, 9:28 PM

#

nahh its gonna be way worse than keyronex

kind mantle Apr 21, 2025, 9:35 PM

#

cmon we're not here to make perfect things now are we?

#

Make UltraOS

fiery turtle Apr 21, 2025, 9:35 PM

#

but i wanted it to be good nooo

kind mantle Apr 21, 2025, 9:36 PM

#

Good need not be perfect

fiery turtle Apr 21, 2025, 9:36 PM

#

fair

calm latch Apr 21, 2025, 9:40 PM

#

how could a bad OS even come from you?

fiery turtle Apr 21, 2025, 9:41 PM

#

watch me trl

calm latch Apr 21, 2025, 9:42 PM

#

calm latch how could a bad OS even come from you?

like after you writing the best ACPI library coolmeme

fiery turtle Apr 21, 2025, 9:43 PM

#

well thanks

left orbit Apr 21, 2025, 10:21 PM

#

yooooo

#

finally

#

after all these months it's finally true

frank canopy Apr 21, 2025, 10:50 PM

#

👀

fiery turtle Apr 21, 2025, 10:51 PM

#

lol

thorny orbit Apr 22, 2025, 2:56 PM

#

uOS ultrameme

#

}

fiery turtle Apr 22, 2025, 10:32 PM

#

i did a bit of ultra work, mostly remembering whatever i was doing there and changing some things around

#

but its a start ig

calm latch Apr 22, 2025, 10:37 PM

#

you just need to rename this thread risc

fiery turtle Apr 22, 2025, 10:37 PM

#

nah ill make a new one when im confident im actually gonna work on it

#

and it will generate less traffic since its not actually a useful project to anyone

calm latch Apr 22, 2025, 10:38 PM

#

calm latch you just need to rename this thread <:risc:815143445764243476>

to instantly become an operating system with a lot of stars

kind mantle Apr 22, 2025, 10:38 PM

#

fiery turtle nah ill make a new one when im confident im actually gonna work on it

Yeah but you've built up quite some fame, I'm sure people would be interested

fiery turtle Apr 22, 2025, 10:40 PM

#

speaking of that, ill need to integrate uacpi at least for early table access mode very soon which is nice

#

ill probably get rid of my printf and just steal uacpi's symbols

fiery turtle Apr 22, 2025, 10:40 PM

#

kind mantle Yeah but you've built up quite some fame, I'm sure people would be interested

watch me lose all of it because of how bad the os is trl

kind mantle Apr 22, 2025, 10:41 PM

#

KEKW

calm latch Apr 22, 2025, 10:41 PM

#

can I shill a microkernel design? /j

fiery turtle Apr 22, 2025, 10:42 PM

#

nah im not doing that lol

#

basically my idea is very simple and not innovative, i want to get it running very quickly so that i can focus on stuff that i enjoy, which is drivers

#

and i want to get a basic drm up and running

calm latch Apr 22, 2025, 10:43 PM

#

fiery turtle watch me lose all of it because of how bad the os is <:trl:829316641191886888>

anyway, you had some good drivers

fiery turtle Apr 22, 2025, 10:43 PM

#

if i were to make a microkernel id have to focus on the details which im not interested in

#

im considering dropping aarch64 and i686 atm as well

fiery turtle Apr 22, 2025, 10:44 PM

#

fiery turtle if i were to make a microkernel id have to focus on the details which im not int...

whereas a monolithic design just allows we to focus on getting stuff done

fiery turtle Apr 22, 2025, 10:45 PM

#

calm latch anyway, you had some good drivers

yeah thats the only half decent thing about that old os lol

#

basically i want it to be old proxima like which was very simple and very fast

#

although i do want to support numa as well from the start

#

we'll see

calm latch Apr 22, 2025, 10:47 PM

#

that's still a good goal

fiery turtle Apr 22, 2025, 10:48 PM

#

my long term goal is to be able to run a simple opengl 3 or 4 game with native accel on my 2010 igpu

#

with that game being a binary built for linux userspace and just cp'ed to my image

#

i might just get some game from my steam library

#

maybe factorio, idk which opengl it uses

#

or balatro would be cool lol

#

but thats obviously a giant amount of work so yeah

#

hopefully i get there eventually and dont abandon it lol

vast kestrel Apr 22, 2025, 11:50 PM

#

Run Minecraft Smuggy

#

Then while you are playing it instead of working you could at least do it as a stress test for your os

fiery turtle Apr 22, 2025, 11:50 PM

#

lol

#

yeah i guess actually

vast kestrel Apr 22, 2025, 11:51 PM

#

Today I did some vulkan thing and I got reminded about your (I think it was you) igpu triangle hello world and how much smaller it was compared to vulkan KurisuSip

fiery turtle Apr 22, 2025, 11:52 PM

#

lol yeah

#

it was me

fiery turtle Apr 23, 2025, 10:36 PM

#

Spent a few hours figuring out how linuxes log ring works, I think I have a pretty good idea of the algorithm

#

Its pretty neat

#

Since I'm doing Linux compat might as well have a syslog ring

left orbit Apr 24, 2025, 12:05 AM

#

are you going linux binary compat?

#

that's pretty sick

#

definitely takes away a lot of the porting groundwork, you can just throw an alpine rootfs at it and work on the kernel instead :^)

fiery turtle Apr 24, 2025, 7:00 AM

#

left orbit are you going linux binary compat?

Yeah basically

#

Ill probably have some of my own syscalls that live at some offset, sort of like xnu does

#

But Linux syscalls and vfs layout will be the native way to use it still

fiery turtle Apr 24, 2025, 7:05 AM

#

left orbit definitely takes away a lot of the porting groundwork, you can just throw an alp...

That's basically why I'm doing that lol

#

Also this will allow me to really understand how Linux is built in depth

flat badge Apr 24, 2025, 7:25 AM

#

fiery turtle Spent a few hours figuring out how linuxes log ring works, I think I have a pret...

the in-kernel implementation or /dev/kmsg?

fiery turtle Apr 24, 2025, 7:29 AM

#

flat badge the in-kernel implementation or /dev/kmsg?

sorta, the implementation of that device is just based on the internal prink_ringbuffer struct and all the helpers around that

#

this stuff is also reused for printing out to consoles etc

#

basically the in-kernel, yes

flat badge Apr 24, 2025, 9:10 AM

#

~~just copy managarm~~

#

Jokes aside you probably don't need a fancy strategy until you're using NMIs

fiery turtle Apr 24, 2025, 9:11 AM

#

its nothing fancy tbf

#

they just have a ring of log entries basically that have a pointer to text and its length + log level/facility etc

#

each log entry has a 64 bit sequence number

#

that u can use to extract its position in the ring with a simple modulo etc

#

the text buffer is also just a fixed length dynamic array

flat badge Apr 24, 2025, 9:13 AM

#

Well, at least in case of linux (and managarm) it needs to be NMI safe though

fiery turtle Apr 24, 2025, 9:13 AM

#

it does have all sorts of complicated locking stuff so that u can use it in every possible context yeah

flat badge Apr 24, 2025, 9:14 AM

#

which means it must be able to cope with the log function getting entered in an nmi while the context that was interrupted by the nmi is currently writing to the ringbuffer

fiery turtle Apr 24, 2025, 9:15 AM

#

yeah

#

for that it has multiple states for the log entries

#

like available, commited etc

calm latch Apr 24, 2025, 9:16 AM

#

What are you even supposed to do when you get an NMI and you're a microkernel?

#

Like your console driver would probably be running in userspace

flat badge Apr 24, 2025, 9:26 AM

#

buffer the message and send it to userspace after the NMI is done

calm latch Apr 24, 2025, 9:43 AM

#

But aren't you supposed to crash?

flat badge Apr 24, 2025, 9:49 AM

#

no

#

not in general

#

NMI can be used to deliver critical failures but the more modern mechanism for that is MCE

#

NMI is also used for watchdogs / profiling / etc

#

and you don't want to crash in these situations

calm latch Apr 24, 2025, 10:25 AM

#

But even in case of a critical failure, you should still somehow be able to get to userspace to show an error message, and I don't get what's the best way to do that

fiery turtle Apr 24, 2025, 10:26 AM

#

if your kernel is fucked to an extent where it gets a watchdog NMI it's not really an option

#

aka you cant rely on userspace working at all in that case probably

calm latch Apr 24, 2025, 10:31 AM

#

I was thinking about going to userspace and crossing fingers for it to work enough to print an error message to the screen

#

Since now when the kernel crashes, it only prints error to the debugcon

#

So like if you're running it on physical hardware without COM port and the kernel dies, it just freezes and you have no way of knowing what happened

mortal yoke Apr 24, 2025, 10:36 AM

#

why couldn't you also print to the fb in the kernel in case of a panic?

left orbit Apr 24, 2025, 10:38 AM

#

mortal yoke why couldn't you also print to the fb in the kernel in case of a panic?

drm

#

well, linux has drm_panic i guess

#

but idk how that works

#

but if your framebuffer has been taken over by a driver then i don't think you can simply just print to the framebuffer anymore

mortal yoke Apr 24, 2025, 10:42 AM

#

well yeah if you have a proper gpu driver then you'd need to do it with it

calm latch Apr 24, 2025, 10:42 AM

#

mortal yoke why couldn't you also print to the fb in the kernel in case of a panic?

I mean I do have debug branch which does that, but it doesn't have to be framebuffer

left orbit Apr 24, 2025, 10:42 AM

#

idk how feasible that is if you're in the "we're fucked" state :^)

calm latch Apr 24, 2025, 10:42 AM

#

Like it could be some cursed serial otput or something, and I want userspace drivers

#

Maybe I want too much though

left orbit Apr 24, 2025, 10:44 AM

#

imo if you're a microkernel your kernel shouldn't crash no matter what

#

you could still enter a user thread context from an nmi handler context, it's just that your system might be in less than ideal state at that point so you should probably do as little as possible

#

it's not like the cpu is dysfunctional and switching to another thread will blow things up

hollow crescent Apr 24, 2025, 10:46 AM

#

left orbit it's not like the cpu is dysfunctional and switching to another thread will blow...

if you receive an NMI it probably is

left orbit Apr 24, 2025, 10:46 AM

#

the cpu? why lol

hollow crescent Apr 24, 2025, 10:47 AM

#

i dont know why you would receive an NMI other than hardware failure

left orbit Apr 24, 2025, 10:47 AM

#

hardware does not necessarily mean cpu

hollow crescent Apr 24, 2025, 10:47 AM

#

it could also mean memory

#

which is similarly bad

mortal yoke Apr 24, 2025, 10:47 AM

#

something like ram would be more likely yeah

left orbit Apr 24, 2025, 10:47 AM

#

exactly, as i said - do as little as possible

#

hopefully it's not all gone to shit yet

#

maybe you can get your last words out before dying

hollow crescent Apr 24, 2025, 10:48 AM

#

meanwhile windows

#

(it used to just clear the screen blue and say NMI - Hardware failure. Contact your hardware vendor)

calm latch Apr 24, 2025, 10:49 AM

#

left orbit imo if you're a microkernel your kernel shouldn't crash no matter what

it usually doesn't, but what if a bit gets flipped by a cosmic ray

left orbit Apr 24, 2025, 10:49 AM

#

ECC is a thing

calm latch Apr 24, 2025, 10:50 AM

#

but some fuckery could still happen in hardware

fiery turtle Apr 24, 2025, 10:55 AM

#

hollow crescent i dont know why you would receive an NMI other than hardware failure

profiling

hollow crescent Apr 24, 2025, 10:55 AM

#

if you have some kind of special hardware that issues NMIs then maybe

left orbit Apr 24, 2025, 10:56 AM

#

i think infy meant actual architectural profiling

fiery turtle Apr 24, 2025, 10:56 AM

#

i think its the standard x86 way of doing profiling

#

yeah

flat badge Apr 24, 2025, 11:21 AM

#

NMI is not used for error reporting on modern hardware

#

NMI is used for watch dogs and profiling

#

And yes, that is architectural

#

If you're talking about MCE: if you encounter a synchronous memory failure in the kernel, you're probably fucked

urban galleon Apr 24, 2025, 11:24 AM

#

hollow crescent meanwhile windows

Real, it keeps bsoding on Meteor lake Arc iGPU

flat badge Apr 24, 2025, 11:24 AM

#

if it's in userspace you can poison the page, maybe kill the process, continue

urban galleon Apr 24, 2025, 11:24 AM

#

It might be a intel problem but I blame windows

fiery turtle Apr 24, 2025, 11:24 AM

#

flat badge if it's in userspace you can poison the page, maybe kill the process, continue

linux supports migrating pages so it wouldnt even have to kill the process

#

only if its nonmovable memory

flat badge Apr 24, 2025, 11:24 AM

#

can't migrate the page if you can't read it anymore

fiery turtle Apr 24, 2025, 11:25 AM

#

hm tru

flat badge Apr 24, 2025, 11:26 AM

#

if it's a kernel page, the most reasonable thing you can do is probably log a kernel panic to whatever devices the kernel can talk to

#

realistically ppl will run memtest upon random crashes / detect faulting memory during POST so that should be good enough anyway

#

another option is to write error info to some nvram

#

or even normal RAM

#

and then doing a soft reboot

#

linux can also do that

#

https://docs.kernel.org/admin-guide/kdump/kdump.html

mortal yoke Apr 24, 2025, 11:31 AM

#

there is also ramoops https://docs.kernel.org/admin-guide/ramoops.html

flat badge Apr 24, 2025, 11:31 AM

#

essentially this does a kexec on panic

north hull Apr 24, 2025, 11:02 PM

#

has anyone else ran into this

#

if I build with release meson doesnt like it

#

nothing else seems to really break

fiery turtle Apr 25, 2025, 9:25 PM

#

bro the linux printk_ringbuffer design is genius

#

you can reserve entries completely lockless

#

and they made it completely NMI safe

dense steppe Apr 25, 2025, 9:28 PM

#

A fully reentrant ringbuffer is a very good exercise in concurrency and reentrancy, as I believe @flat badge (sorry for ping) has pointed out.

fiery turtle Apr 25, 2025, 9:28 PM

#

the log entries have 4 states: reusable, reserved, commited and finalized

Reusable just means free, reserved means someone allocated an entry, but is not yet done writing to it, so readers will be aborted when trying to continue reading the ring at this point, but writers can continue concurrently allocating more entries. Commited means someone has commited the data, but may want to append more later, this is only used during early boot for multiline logs. Then finalized is a read-to-read entry.

#

and this ring is actually 3 separate rings: descriptors, info, and actual text prepended by descriptor id

fiery turtle Apr 25, 2025, 9:29 PM

#

dense steppe A fully reentrant ringbuffer is a very good exercise in concurrency and reentran...

yeah im definitely not gonna bother making it NMI safe as thats insane amount of work and I simply dont care about NMIs at this point

#

but it can still be concurrent with a few atomic ops

dense steppe Apr 25, 2025, 9:31 PM

#

fiery turtle the log entries have 4 states: reusable, reserved, commited and finalized Reusa...

btw re commited: AFAIK those multiline messages you're refering to are printk(KERN_CONT) messages

fiery turtle Apr 25, 2025, 9:32 PM

#

yeah the KERN_CONT flag makes printk code not finalize the record

#

but if a new record after is reserved the previous one is instantly finalized if its commited

#

so it cant really be used after early boot

dense steppe Apr 25, 2025, 9:35 PM

#

fiery turtle yeah the KERN_CONT flag makes printk code not finalize the record

I'm pretty sure I've seen code like

int arr[10];
// ...
printk(KERN_INFO "Some things:");
for(int i = 0; i < 10; i++)
        printk(KERN_CONT " %d", arr[i]);
printk(KERN_CONT "\n");

#

note the lack of \n

fiery turtle Apr 25, 2025, 9:35 PM

#

yeah

#

dense steppe Apr 25, 2025, 9:35 PM

#

ah.

fiery turtle Apr 25, 2025, 9:36 PM

#

what's cool is readers won't see the commited message unless a new one is added after or the commited message is finalized

#

so its very well thought out

fiery turtle Apr 26, 2025, 11:00 AM

#

hmm do i wanna waste a week making this log ring completely lockless like linux, I think I understand how to make that work entirely...

left orbit Apr 26, 2025, 11:02 AM

#

bro said he wants to focus on writing drivers and yet he's making one of the most sophisticated logging systems any hobby kernel has seen :^)

fiery turtle Apr 26, 2025, 11:02 AM

#

lmfao thats kinda true, but i already dumped way too much time into studying how it handles every edge case trl

left orbit Apr 26, 2025, 11:03 AM

#

lockless stuff is always fun as long as it works correctly

#

it can be extremely tricky to get right too

fiery turtle Apr 26, 2025, 11:03 AM

#

the amount of barrier pairs and comments there do make me a bit scared tho

#

that maybe i wont make it in a week

left orbit Apr 26, 2025, 11:04 AM

#

oh shit 💀

fiery turtle Apr 26, 2025, 11:05 AM

#

like that log ring is truly cool, u can have an infinite number of cpus concurrently writing to it and then readers reading at the same time, and all u need for a reader's iterator is one u64 seqnum

#

i would probably care less about it if it wasnt exposed to userspace as /dev/kmesg etc

#

or maybe i should just say fuck it an do spin_lock() trl

left orbit Apr 26, 2025, 11:07 AM

#

tbf that is probably good enough for any hobby os

#

and it will let you work on other important stuff

fiery turtle Apr 26, 2025, 11:08 AM

#

most hobby osses probably dont even have a log ring, they just pipe the input string to sinks directly

#

but yeah idk, i kinda want it to be better than your avg os

left orbit Apr 26, 2025, 11:09 AM

#

doesn't mean you can't have your kernel log ring be simple :^)

fiery turtle Apr 26, 2025, 11:09 AM

#

this design is also nice because u can do the actual output work in a separate thread

fiery turtle Apr 26, 2025, 11:09 AM

#

left orbit doesn't mean you can't have your kernel log ring be simple :^)

tru but it's tempting lmfao

left orbit Apr 26, 2025, 11:09 AM

#

so relatable

fiery turtle Apr 26, 2025, 11:09 AM

#

to start the ultra kernel with a 3k loc log ring

#

that doesnt even have smp

#

or malloc

left orbit Apr 26, 2025, 11:09 AM

#

if only i had the motivation to read about stuff like that

fiery turtle Apr 26, 2025, 11:10 AM

#

well i just browsed the sources for like 10 hours total probably

#

until i got a literal headache

#

trl

flat badge Apr 26, 2025, 11:10 AM

#

ulog library when

fiery turtle Apr 26, 2025, 11:11 AM

#

lol

#

i'd have to make tons of tests for it anyway

#

so maybe

#

if it's lockless that makes it nmi safe as well which is nice

flat badge Apr 26, 2025, 11:11 AM

#

tbh the linux implementation looks way make complicated than the Managarm one

#

how does linux achieve NMI safety?

fiery turtle Apr 26, 2025, 11:12 AM

#

everything is reentrant, there isnt a single critical section there

#

they have id consistency checks and are able to detect concurrent modifications of a descriptor

flat badge Apr 26, 2025, 11:12 AM

#

https://github.com/managarm/managarm/blob/master/kernel/thor/generic/thor-internal/ring-buffer.hpp#L236

GitHub

managarm/kernel/thor/generic/thor-internal/ring-buffer.hpp at maste...

Pragmatic microkernel-based OS with fully asynchronous I/O - managarm/managarm

fiery turtle Apr 26, 2025, 11:12 AM

#

they store ID and STATE as a combined ORed value

#

that is atomically modified

flat badge Apr 26, 2025, 11:13 AM

#

and how do they prevent the messages themselves from being overwritten?

#

writing descriptors in an NMI safe way is relatively easy

fiery turtle Apr 26, 2025, 11:13 AM

#

they have 3 separate rings

#

the text ring is an array of [descriptor id, text data]

flat badge Apr 26, 2025, 11:13 AM

#

flat badge writing descriptors in an NMI safe way is relatively easy

but writing the messages is harder

#

an array?

#

so "text data" is fixed size?

fiery turtle Apr 26, 2025, 11:14 AM

#

rings are all fixed size

#

text data is dynamic ofc

#

but it's reserved when allocating the descriptor

flat badge Apr 26, 2025, 11:15 AM

#

so they have a sophisticated allocation algorithm for that?

fiery turtle Apr 26, 2025, 11:15 AM

#

yeah

flat badge Apr 26, 2025, 11:15 AM

#

sophisticated = not just circular reuse

fiery turtle Apr 26, 2025, 11:15 AM

#

its very sophisticated

flat badge Apr 26, 2025, 11:15 AM

#

ah

fiery turtle Apr 26, 2025, 11:15 AM

#

well no

#

it is circular reuse

flat badge Apr 26, 2025, 11:16 AM

#

but it jumps over stuff that is still in use by another context? or how does it work

fiery turtle Apr 26, 2025, 11:16 AM

#

i can explain

#

so a new record is allocated via prb_reserve(), which accepts the needed text length.
The first thing it does is allocate a new descriptor by changing its state to reserved and advancing the descriptor ring tail pointer. It then allocates the text for this descriptor by advancing the tail text ring pointer, and then walking over it for the length of the needed text data and invalidating any descriptors currently referencing this text data (since text data has an inline header with descriptor ID that references this text, and text data is always aligned at 8 bytes to fit it)

#

so the older descriptors referencing text data in the new span are invalidated into reusable state

dense steppe Apr 26, 2025, 11:20 AM

#

what if we prb_reserve a lot without ever finalizing, such that we wrap around?

fiery turtle Apr 26, 2025, 11:21 AM

#

the reader does the following

desc = get_descriptor(seqnum)
memcpy(caller_buffer, desc->text, desc->size)
return desc_still_alive_and_has_the_same_id(desc);

#

so concurrent modifications of the descriptor are safe

fiery turtle Apr 26, 2025, 11:22 AM

#

dense steppe what if we prb_reserve a lot without ever finalizing, such that we wrap around?

yeah thats a good question

#

basically, it isnt allowed to recycle reserved descriptors

#

so if you reserve but never commit, the ring will stall

#

aka all new writers will be aborted

dense steppe Apr 26, 2025, 11:23 AM

#

ah, ok.

fiery turtle Apr 26, 2025, 11:23 AM

#

if it wraps around

dense steppe Apr 26, 2025, 11:23 AM

#

and that voids messages?

fiery turtle Apr 26, 2025, 11:23 AM

#

yup

#

but it's very hard to trigger, as you need a buggy writer or a writer that literally does a sleep between reserve and commit

#

and the default implementation of reserve() disables interrupts

#

so its expected that u do commit very quickly

dense steppe Apr 26, 2025, 11:24 AM

#

so if I have one hundred thousand concurrent printk's some of them may be voided meme

fiery turtle Apr 26, 2025, 11:24 AM

#

only if u have that many CPUs

#

because it disables irqs on reserve

dense steppe Apr 26, 2025, 11:24 AM

#

yeah

fiery turtle Apr 26, 2025, 11:24 AM

#

but the default implementation also scales the log ring according to max number of cpus

dense steppe Apr 26, 2025, 11:24 AM

#

and AFAIK it has some memory per CPU for this too

#

yeah

fiery turtle Apr 26, 2025, 11:25 AM

#

so you wont run into this

dense steppe Apr 26, 2025, 11:25 AM

#

ingenious design

fiery turtle Apr 26, 2025, 11:25 AM

#

yeah

left orbit Apr 26, 2025, 11:25 AM

#

what happens on commit that makes the log descriptors not lost when it wraps around?

fiery turtle Apr 26, 2025, 11:25 AM

#

wdym? thats the idea of the ring, if it wraps around the old entries are lost

#

there's klogd in userspace which journals kmsg entries

#

so they're always available

left orbit Apr 26, 2025, 11:26 AM

#

ah that's the part i was missing

#

that's why dmesg goes back all the way to the first message

dense steppe Apr 26, 2025, 11:26 AM

#

no

fiery turtle Apr 26, 2025, 11:26 AM

#

nah dmesg reads kmgs directly

left orbit Apr 26, 2025, 11:26 AM

#

oh?

fiery turtle Apr 26, 2025, 11:26 AM

#

the ring is just pretty large

#

by default

left orbit Apr 26, 2025, 11:26 AM

#

how large are we talking?

fiery turtle Apr 26, 2025, 11:27 AM

#

so it fits everything just fine usually

dense steppe Apr 26, 2025, 11:27 AM

#

if you can somehow perform lots of printk's (e.g. via a kernel module) and then do dmesg, you will see that it does not go back to the beginning

fiery turtle Apr 26, 2025, 11:27 AM

#

left orbit how large are we talking?

the default one is 1 << 17 i think

#

max_number_of_cpus

dense steppe Apr 26, 2025, 11:27 AM

#

left orbit how large are we talking?

a megabyte or something

fiery turtle Apr 26, 2025, 11:27 AM

#

well 1 << 17 * AVG_MSG_SIZE (32)

left orbit Apr 26, 2025, 11:28 AM

#

that would be 4M

#

it's not that bad

#

but yeah that makes sense

#

so it's just a set of really big ring buffers

fiery turtle Apr 26, 2025, 11:28 AM

#

well it's 3 arrays

#

descriptors, log info, and text

left orbit Apr 26, 2025, 11:29 AM

#

yup

#

they are still pretty big though

#

:^)

#

i wouldn't have thought to "waste" a few megabytes of memory for logging

fiery turtle Apr 26, 2025, 11:29 AM

#

actually I lied, the default is 1 << 14

left orbit Apr 26, 2025, 11:30 AM

#

so it can hold 16K "average sized messages"

dense steppe Apr 26, 2025, 11:30 AM

#

dense steppe if you can somehow perform lots of printk's (e.g. via a kernel module) and then ...

whenever my monitor blanks (e.g. from inactivity) and then turns on again, if HDR is used the kernel will print a bunch of stuff about HDR. and if this happens enough, I actually see in dmesg that the first few messages are lost

fiery turtle Apr 26, 2025, 11:30 AM

#

/*
 * Define the average message size. This only affects the number of
 * descriptors that will be available. Underestimating is better than
 * overestimating (too many available descriptors is better than not enough).
 */
#define PRB_AVGBITS 5    /* 32 character average length */

#if CONFIG_LOG_BUF_SHIFT <= PRB_AVGBITS
#error CONFIG_LOG_BUF_SHIFT value too small.
#endif
_DEFINE_PRINTKRB(printk_rb_static, CONFIG_LOG_BUF_SHIFT - PRB_AVGBITS,
         PRB_AVGBITS, &__log_buf[0]);

#

basically this is the default

fiery turtle Apr 26, 2025, 11:30 AM

#

dense steppe whenever my monitor blanks (e.g. from inactivity) and then turns on again, if HD...

yeah or if u sleep and resume many times

dense steppe Apr 26, 2025, 11:30 AM

#

yeah

left orbit Apr 26, 2025, 11:31 AM

#

interesting

fiery turtle Apr 26, 2025, 11:31 AM

#

config LOG_BUF_SHIFT
    int "Kernel log buffer size (16 => 64KB, 17 => 128KB)"
    range 12 25
    default 17
    depends on PRINTK
    help
      Select the minimal kernel log buffer size as a power of 2.
      The final size is affected by LOG_CPU_MAX_BUF_SHIFT config
      parameter, see below. Any higher size also might be forced
      by "log_buf_len" boot parameter.

      Examples:
             17 => 128 KB
             16 => 64 KB
             15 => 32 KB
             14 => 16 KB
             13 =>  8 KB
             12 =>  4 KB

#

so 128KB, but - 5 bits because of text

#

so its actually under a meg

#

or even under 512KiB for all

#

and then it's 12 bits per cpu

left orbit Apr 26, 2025, 11:32 AM

#

damn that's some crazy stuff lol

fiery turtle Apr 26, 2025, 11:33 AM

#

or well, extra 4K of log ring per cpu

dense steppe Apr 26, 2025, 11:33 AM

#

the kernel has a static char printk_ringbuffer[128 kilobytes]; or something, and then it reallocates using vmalloc at a later stage I think

fiery turtle Apr 26, 2025, 11:34 AM

#

it reallocates using memblock

#

so the very early memory map allocator

#

before smp startup

dense steppe Apr 26, 2025, 11:34 AM

#

ah

#

I had a similar thing

fiery turtle Apr 26, 2025, 11:35 AM

#

https://elixir.bootlin.com/linux/v6.14.3/source/arch/x86/kernel/setup.c#L1074

dense steppe Apr 26, 2025, 11:35 AM

#

initmem, which was an initmeme in my old kernel

fiery turtle Apr 26, 2025, 11:35 AM

#

i think x86 calls it here

dense steppe Apr 26, 2025, 11:35 AM

#

however I did not have any printk ringbuffer

fiery turtle Apr 26, 2025, 11:36 AM

#

it's definitely a huge overkill for my kernel lmfao

#

but i might give it a shot

#

i'll need all of the atomic helpers and barriers anyway

fiery turtle Apr 26, 2025, 11:37 AM

#

left orbit damn that's some crazy stuff lol

yeah like whoever designed this algo is probably a literal genius

dense steppe Apr 26, 2025, 11:38 AM

#

fiery turtle yeah like whoever designed this algo is probably a literal genius

it evolved with time. Linux did not always have a printk ringbuffer, and it did not always have this printk ringbuffer.

fiery turtle Apr 26, 2025, 11:38 AM

#

it was actually added only in 2020

#

its very new

#

and this commit description is already outdated

dense steppe Apr 26, 2025, 11:39 AM

#

evolution

fiery turtle Apr 26, 2025, 11:39 AM

#

stuff like this

#

looks like it was made by these guys https://www.linutronix.de/

Linutronix Linux for Industry

Linutronix is Linux for Industry

flat badge Apr 26, 2025, 11:40 AM

#

Managarm achives nmi safety by cancelling in progress memcpys to the log ring when an nmi happens

fiery turtle Apr 26, 2025, 11:41 AM

#

in this case this isnt needed since a descriptor is either reserved or it isnt

#

and its guaranteed to be safe to memcpy if prb_reserve succeeded

flat badge Apr 26, 2025, 11:42 AM

#

Yeah but the cost of that is that you need to handle the case that the ring wraps around while the nmi happens

fiery turtle Apr 26, 2025, 11:42 AM

#

yeah they have two separate ID systems because of this

#

the seqnum, and the internal descriptor id which is limited to unsigned long bits - state bits

#

a lot of workarounds for 32 bit unsigned long

flat badge Apr 26, 2025, 11:43 AM

#

Managarm's ring can't ever stall

#

since stuff is committed atomically with no pre-reservation

fiery turtle Apr 26, 2025, 11:44 AM

#

well this one kinda cant as well unless every single CPU gets an NMI at the same time while its writing to the ring

#

and then writes in the NMI handler as well

#

then some entries might get dropped

#

since it scales for the maximum number of cpus at boot time

#

each cpu gets a 4K wiggle room by default

flat badge Apr 26, 2025, 11:46 AM

#

Managarm's ring as a single ring buffer and it works like this on write:

it invalidates existing entries that'd overlap the new one
it copies the new entry to the ring
if the previous operation succeeded without being interrupted by reentrancy, it advances the head ptr

fiery turtle Apr 26, 2025, 11:46 AM

#

so it's not lockless right

flat badge Apr 26, 2025, 11:47 AM

#

It is

fiery turtle Apr 26, 2025, 11:47 AM

#

oh also managarms ring doesnt support many readers as far as i can see

#

since dequeue is global

flat badge Apr 26, 2025, 11:48 AM

#

Managarm is stateless at the reader side

fiery turtle Apr 26, 2025, 11:48 AM

#

linuxes ring has no such thing as dequeue

flat badge Apr 26, 2025, 11:48 AM

#

So it supports any number of readers

dense steppe Apr 26, 2025, 11:48 AM

#

flat badge Managarm's ring as a single ring buffer and it works like this on write: - it in...

to me this seems like it is lockless but you are "cheating" wrt reentrancy

fiery turtle Apr 26, 2025, 11:48 AM

#

flat badge So it supports any number of readers

wdym

#

any new reader will only read the new contents

#

that the previous reader hasnt seen

flat badge Apr 26, 2025, 11:49 AM

#

A reader simply knows its position in the ring, checks if the entry at that position is still valid, copies out of the ring and checks that no invalidation happened in the meantime

#

No, readers are independent

fiery turtle Apr 26, 2025, 11:49 AM

#

oh ok

#

yeah i guess its lockless but not concurrent

#

many writers will just keep spinning in retries

#

but its definitely a simpler design

flat badge Apr 26, 2025, 11:50 AM

#

That's right, it's per cpu

#

And a kernel thread dumps it to sinks

fiery turtle Apr 26, 2025, 11:51 AM

#

ohh wait

#

it has to collect records from every cpu's ring basically?

flat badge Apr 26, 2025, 11:52 AM

#

yes

fiery turtle Apr 26, 2025, 11:52 AM

#

i guess this is only bad because it it's hard to keep timestamp ordering

flat badge Apr 26, 2025, 11:52 AM

#

Or rather, all cpus push to a centralized sink

#

But go to a per-cpu ring first

fiery turtle Apr 26, 2025, 11:52 AM

#

cpus push themselves?

dense steppe Apr 26, 2025, 11:53 AM

#

the strategy is to defer the work of pushing to the centralized sink/logbuffer/whatever to a context that can e.g. block on mutexes?

flat badge Apr 26, 2025, 11:54 AM

#

The full story is this: logs go to per CPU ring buffer -> a per-CPU soft irq dumps them to a global sink -> a kernel thread serves the userspace visible /dev/kmsg

flat badge Apr 26, 2025, 11:54 AM

#

dense steppe the strategy is to defer the work of pushing to the centralized sink/logbuffer/w...

Yes

fiery turtle Apr 26, 2025, 11:54 AM

#

how does this ensure timestamp ordering?

dense steppe Apr 26, 2025, 11:54 AM

#

why would you need timestamp ordering?

#

Linux doesn't guarantee timestamp ordering.

flat badge Apr 26, 2025, 11:54 AM

#

Essentially this separates the reentrancy from concurrency

fiery turtle Apr 26, 2025, 11:54 AM

#

well because u want earlier logs to appear before later logs?

fiery turtle Apr 26, 2025, 11:54 AM

#

dense steppe Linux doesn't guarantee timestamp ordering.

wdym

flat badge Apr 26, 2025, 11:55 AM

#

rdtsc?

fiery turtle Apr 26, 2025, 11:55 AM

#

it does

#

u have no guarantees that a cpu pushes its earlier logs before some other cpus later logs

#

basically

flat badge Apr 26, 2025, 11:55 AM

#

Linux messages are ordered by seqnum not by timestamp, aren't they?

fiery turtle Apr 26, 2025, 11:55 AM

#

both

dense steppe Apr 26, 2025, 11:56 AM

#

fiery turtle wdym

alright.
thread level does sched_clock()
NMI
NMI handler does sched_clock(), prb_reserve() ..., returns
thread does prb_reserve

#

-> not ordered

flat badge Apr 26, 2025, 11:56 AM

#

I don't think that's true, the /dev/kmsg docs specifically state that there is no guaranteed ordering

fiery turtle Apr 26, 2025, 11:56 AM

#

dev kmsg docs generally kinda suck

fiery turtle Apr 26, 2025, 11:56 AM

#

dense steppe alright. thread level does sched_clock() NMI NMI handler does sched_clock(), prb...

lemme check the code if this is possible

flat badge Apr 26, 2025, 11:57 AM

#

flat badge I don't think that's true, the /dev/kmsg docs specifically state that there is n...

Ah actually no, they state that it's ordered by seqnum

fiery turtle Apr 26, 2025, 11:57 AM

#

ok so prink grabs the timestamp on entry

#

so i guess its possible

#

but it would be an exceptional situation

#

more or less

flat badge Apr 26, 2025, 11:58 AM

#

Well in general you can't guarantee both seqnum and timestamp order, right?

dense steppe Apr 26, 2025, 11:59 AM

#

fiery turtle but it would be an exceptional situation

yeah it is very unlikely to happen

flat badge Apr 26, 2025, 11:59 AM

#

Unless you do heavy synchronization around getting the timestamp

fiery turtle Apr 26, 2025, 11:59 AM

#

if you really wanted to u could

#

yeah

#

i dont think its that useful

dense steppe Apr 26, 2025, 11:59 AM

#

flat badge Well in general you can't guarantee both seqnum and timestamp order, right?

you can if you lie about the timestamps

fiery turtle Apr 26, 2025, 11:59 AM

#

but overall its nice if they come in order unless u get exceptional situation like NMIs

#

like managarm would just push in whichever order right

#

here its at least mostly in order i think

#

do u see any unordered ones in your /dev/kmsg?

dense steppe Apr 26, 2025, 12:02 PM

#

I'm on my phone right now meme

fiery turtle Apr 26, 2025, 12:02 PM

#

technically if two cpus log at the same time, but the other does it slightly later and then the first one gets preempted before prb_reserve this might happen

#

but strangely i dont see anything like that

dense steppe Apr 26, 2025, 12:02 PM

#

but I've never seen unordered messages in dmesg. although dmesg might be doing some sorting of its own.

fiery turtle Apr 26, 2025, 12:02 PM

#

dense steppe but I've never seen unordered messages in dmesg. although dmesg might be doing ...

maybe

#

actually i think the x86 kernel is usually non preemptible

#

so thats why

dense steppe Apr 26, 2025, 12:04 PM

#

fiery turtle actually i think the x86 kernel is usually non preemptible

wrong

#

fiery turtle Apr 26, 2025, 12:05 PM

#

well yours says preempt_dynamic

#

which means off by default

dense steppe Apr 26, 2025, 12:05 PM

#

oh, really?

fiery turtle Apr 26, 2025, 12:05 PM

#

yeah it can be overriden in the command line

#

but also it says CONFIG_PREEMPT=y, lemme check idk how these two cooperate

flat badge Apr 26, 2025, 12:10 PM

#

fiery turtle like managarm would just push in whichever order right

In Managarm it is almost always ordered except for NMI contexts or other situations where it can't push to the central sink

fiery turtle Apr 26, 2025, 12:14 PM

#

dense steppe oh, really?

basically yeah, in dynamic mode preemption is off by default, unless the command line has preempt=none/voluntary/full/etc

#

so check you /proc/cmdline

dense steppe Apr 26, 2025, 12:14 PM

#

ah

#

yeah it doesn't have preempt=

fiery turtle Apr 26, 2025, 12:15 PM

#

or actually im wrong https://elixir.bootlin.com/linux/v6.14.4/source/kernel/sched/core.c#L7617

#

if i understand correctly with your config it will enter the last else

dense steppe Apr 26, 2025, 12:16 PM

#

yeah it goes into preempt_dynamic_full

#

fiery turtle Apr 26, 2025, 12:17 PM

#

your /sys/kernel/debug/sched might contain the mode if u have that built

fiery turtle Apr 26, 2025, 12:18 PM

#

dense steppe

ah yeah

dense steppe Apr 26, 2025, 12:18 PM

#

or just dmesg shows it

fiery turtle Apr 26, 2025, 12:18 PM

#

this is x86 right?

left orbit Apr 26, 2025, 12:19 PM

#

[iretq@raptor linux-mlibc-jinx]$ sudo cat /sys/kernel/debug/sched/preempt
none voluntary (full) lazy```

#

arch linux does preempt=full apparently?

fiery turtle Apr 26, 2025, 12:19 PM

#

they're always on the edge i guess

#

because it hasnt been stable on x86 iirc

dense steppe Apr 26, 2025, 12:20 PM

#

fiery turtle this is x86 right?

no this is xr17032 meme

#

yes this is x86

fiery turtle Apr 26, 2025, 12:21 PM

#

lol

#

xr linux port when

left orbit Apr 26, 2025, 12:23 PM

#

btw are you planning to open source your kernel from the start or will your work on it privately first?

#

i am kinda curious to see how it develops meme

fiery turtle Apr 26, 2025, 12:24 PM

#

I mean its already open source, i'm just not upstreaming stuff yet because there isnt much to upstream

#

https://github.com/UltraOS/Ultra

#

last commit 2 years ago trl

left orbit Apr 26, 2025, 12:25 PM

#

well, yeah, i meant the new stuff you're working on :^)

#

(or will be working on, idk)

fiery turtle Apr 26, 2025, 12:25 PM

#

yeah i will be upstreaming that stuff ofc

#

i need to decide if im dropping 32 bit support

#

i probably should because im never getting to driver dev then

hollow crescent Apr 26, 2025, 12:34 PM

#

fiery turtle the amount of barrier pairs and comments there do make me a bit scared tho

so why do you need this

#

what purpose does it provide

left orbit Apr 26, 2025, 12:35 PM

#

very robust logging

fiery turtle Apr 26, 2025, 12:37 PM

#

hollow crescent what purpose does it provide

Lockless NMI safe logging

#

That scales to any number of readers and writers

#

Also works under panic or any other condition

gentle peak Apr 26, 2025, 12:39 PM

#

just spinlock and if an nmi happens during logging too bad

fiery turtle Apr 26, 2025, 12:47 PM

#

True

frank canopy Apr 26, 2025, 1:50 PM

#

dang now you're making me tempted to do the same

#

would take more stack space in my actual logging handler since I'll need to print out to a buffer though

#

rn my log handler just writes directly to the qemu debug port and I've been wanting to add other sink options for a hot minute now

fiery turtle Apr 26, 2025, 2:21 PM

#

frank canopy would take more stack space in my actual logging handler since I'll need to prin...

nah

#

it takes 0 stack space

#

since you print out to a pre-reserved buffer

#

the buffer comes directly from the ring

frank canopy Apr 26, 2025, 2:22 PM

#

id need to use a counting null-output writer and do the format in two passes then

jaunty fox Apr 26, 2025, 2:23 PM

#

they went to a lot of effort around kprintf in linux

frank canopy Apr 26, 2025, 2:23 PM

#

zig logging handler happens before the formatting so you can do a formatting write directly to output to save on format buffer if possible

jaunty fox Apr 26, 2025, 2:23 PM

#

it was for decades the blocker to RTLinux™️ integration

fiery turtle Apr 26, 2025, 2:24 PM

#

yeah they also did non-blocking consoles specifically for rt

#

a lot of work has been put into this

frank canopy Apr 26, 2025, 2:24 PM

#

also im def not doing a full kprintf type thing that soon, i may consider myself ok at lockless using interlockedxxx but zig atomics take an ordering like cpp ones and i dont understand those at all on a good day let alone while having been sick for a week lmao

#

also i dont need realtime anything if thats why its so thoroughly nonblocking i just want nmi correctness in theory

fiery turtle Apr 26, 2025, 2:25 PM

#

well linux atomics also take ordering

#

they just have simplified macros

jaunty fox Apr 26, 2025, 2:27 PM

#

fiery turtle well linux atomics also take ordering

do the operation macros themselves take them? i think they are having separate barrier macros

fiery turtle Apr 26, 2025, 2:27 PM

#

nah macros dont, they abstract it away

#

and yeah they use barriers

#

they have e.g. atomic_read_acquire

#

which is an acquire barrier

frank canopy Apr 26, 2025, 2:28 PM

#

the only atomics ive felt like i understood what i was doing using are the InterlockedXXX functions that microsoft has which iirc dont

fiery turtle Apr 26, 2025, 2:28 PM

#

microsoft's ones have suffixes i think

frank canopy Apr 26, 2025, 2:28 PM

#

well ive never used a suffixed version lol

fiery turtle Apr 26, 2025, 2:28 PM

#

on x86 its not really that needed, only to protect from compile reorderings

#

because cpu stuff is mostly already serialized

frank canopy Apr 26, 2025, 2:30 PM

#

yeah the docs for InterlockedCompareExchange say to use the aqcuire and release versions for itanium performance critical things

#

but thats it

vast kestrel Apr 26, 2025, 2:32 PM

#

interlocked is x86 semantics :^)

#

C# to this day doesn't have atomicswith proper memory ordering

frank canopy Apr 26, 2025, 2:35 PM

#

yeah and thats how i learned atomics and lockless

#

so i never learned proper ordering and now ive gotten too dumb to understand it lmao

fiery turtle Apr 26, 2025, 2:36 PM

#

https://www.kernel.org/doc/Documentation/memory-barriers.txt

#

this is the perfect doc to understand it

frank canopy Apr 26, 2025, 2:36 PM

#

ill give that a read

vast kestrel Apr 26, 2025, 2:36 PM

#

do note, linux has a different memory model to the c memory model

#

because linux defined a memory model for themselves before the standard defined one

frank canopy Apr 26, 2025, 2:37 PM

#

idk im using zig so i really dont know what would actually apply

fiery turtle Apr 26, 2025, 2:37 PM

#

frank canopy ill give that a read

its huge but its worth it

fiery turtle Apr 26, 2025, 2:37 PM

#

vast kestrel because linux defined a memory model for themselves before the standard defined ...

well clearly it works under gcc and clang lol

frank canopy Apr 26, 2025, 2:37 PM

#

people have sent me memory order stuff before but its always gone over my head lol

vast kestrel Apr 26, 2025, 2:37 PM

#

fiery turtle well clearly it works under gcc and clang lol

yeah iirc the kernel memory model is more relaxed than the c memory model, but I don't remember the exact details

frank canopy Apr 26, 2025, 2:38 PM

#

it was one of my three things that i cant wrap my head around no matter how much i try but maybe this changes that (the other two things are the rust borrow checker and being able to read lisps)

vast kestrel Apr 26, 2025, 2:38 PM

#

not like old gcc ports even listen to the standard memory model

fiery turtle Apr 26, 2025, 2:39 PM

#

yeah

#

its also a very language lawyer-y thing

frank canopy Apr 26, 2025, 2:39 PM

#

yeah my struggle isnt so much in understanding the basically-legalese spec terms its in translating that to practice

#

its like ok i can understand this combination of specification words but i have no clue what to plug in when actually writing code

fiery turtle Apr 26, 2025, 2:40 PM

#

its probably the hardest part of concurrent programming

vast kestrel Apr 26, 2025, 2:41 PM

#

I have a friend who did his thesis on memory orderings, and from him the only way to ensure that what you do is safe and to ensure all the barriers actually interact correctly with each other is to describe the memory in relations like happens-before

#

like, in practice things just "happen to work" because of how cpus handle things in the real world, but it quickly breaks down under the correct conditions of the cpu

frank canopy Apr 26, 2025, 2:42 PM

#

the part that sucks for me rn is i know and get x86 semantics and am targeting x86 only but because of compiler reordering i have to learn the rest of this anyway

fiery turtle Apr 26, 2025, 2:44 PM

#

The key idea is you make barrier pairs, so e.g. a release is paired with acquire somewhere

#

If you have multiple atomic accesses you don't want reordered you use a sequentially consistent ordering

vast kestrel Apr 26, 2025, 2:45 PM

#

the pairs ensure the relationships between the memory accesses

frank canopy Apr 26, 2025, 2:45 PM

#

ok cool

vast kestrel Apr 26, 2025, 2:45 PM

#

the main goal is to not think about atomics on a timeline, but as a graph of accesses

frank canopy Apr 26, 2025, 2:46 PM

#

oh its fuckin DAG ordering

vast kestrel Apr 26, 2025, 2:46 PM

#

there is no timeline when dealing with memory, and the only way to ensure a happens-before is with the atomic barriers

frank canopy Apr 26, 2025, 2:46 PM

#

ok that makes way more sense suddenly just saying it that way

slim panther Apr 26, 2025, 2:54 PM

#

@fiery turtle do you already have a framework 13 dump or do you want it

#

i kinda wanna see what all the blobs are for

fiery turtle Apr 26, 2025, 2:54 PM

#

i dont think i have seen any framework dumps

slim panther Apr 26, 2025, 2:55 PM

#

insyde efi nooo

#

📎 acpi.tar

fiery turtle Apr 26, 2025, 3:01 PM

#

thx

#

anything interesting there?

slim panther Apr 26, 2025, 3:02 PM

#

dunno

#

a lot of ssdts

fiery turtle Apr 26, 2025, 6:03 PM

#

Linux memory barriers and atomics have very stupid implied ordering and semantics, QEMUs seem a lot more sane so I'm stealing the idea

#

https://github.com/qemu/qemu/blob/master/docs/devel/atomics.rst

GitHub

qemu/docs/devel/atomics.rst at master · qemu/qemu

Official QEMU mirror. Please see https://www.qemu.org/contribute/ for how to submit changes to QEMU. Pull Requests are ignored. Please only use release tarballs from the QEMU website. - qemu/qemu

flat badge Apr 26, 2025, 7:20 PM

#

Why don't you just use the c11 memory model?

fiery turtle Apr 26, 2025, 7:25 PM

#

wdym

#

like _Atomic?

flat badge Apr 26, 2025, 7:25 PM

#

yes

fiery turtle Apr 26, 2025, 7:26 PM

#

qemu uses the c model, just with their own wrappers using gcc's builtins

flat badge Apr 26, 2025, 7:27 PM

#

You can also just use stdatomic.h without weird wrappers

fiery turtle Apr 26, 2025, 7:28 PM

#

i hate it because it requires _Atomic prefix which is a special type which doesnt work with anything and changes the default behavior for operators

#

is stdatomic even a freestanding header?

flat badge Apr 26, 2025, 7:28 PM

#

yes

jaunty fox Apr 26, 2025, 7:29 PM

#

fiery turtle is stdatomic even a freestanding header?

i think so

flat badge Apr 26, 2025, 7:29 PM

#

Just use the aliases like atomic_uint32_t etc

jaunty fox Apr 26, 2025, 7:29 PM

#

the __builtin_atomic intrinsics are basically once removed from it

fiery turtle Apr 26, 2025, 7:31 PM

#

ah

flat badge Apr 26, 2025, 7:32 PM

#

I'd use stdatomic.h just because it's standard and well established

#

Writing correct concurrent code is hard enough

fiery turtle Apr 26, 2025, 7:32 PM

#

i really dont understand how they help with anything

#

gcc builtins work well enough

flat badge Apr 26, 2025, 7:33 PM

#

So if you want to talk to others about it / get code reviews etc it's just an extra burden if you're using your own barriers

jaunty fox Apr 26, 2025, 7:34 PM

#

the main niggle people are having with the c11 atomics is that they want memory ordering parameters to the operations while most people think rather in terms of explicit barriers

fiery turtle Apr 26, 2025, 7:34 PM

#

is smp_Xmb() harder to read than atomic_thread_fence(...)?

slim panther Apr 26, 2025, 7:35 PM

#

fiery turtle i hate it because it requires _Atomic prefix which is a special type which doesn...

use c23 meme

fiery turtle Apr 26, 2025, 7:35 PM

#

how does it help?

flat badge Apr 26, 2025, 7:35 PM

#

fiery turtle is smp_Xmb() harder to read than atomic_thread_fence(...)?

Yes

slim panther Apr 26, 2025, 7:35 PM

#

there it's called atomic

#

meme

jaunty fox Apr 26, 2025, 7:35 PM

#

fiery turtle i hate it because it requires _Atomic prefix which is a special type which doesn...

i note it doesn't work properly in objective-c which is part of what led me to go with gcc builtins instead

flat badge Apr 26, 2025, 7:36 PM

#

Gcc built-ins are fine as well

#

But smp_mb() etc is stupid

jaunty fox Apr 26, 2025, 7:36 PM

#

and nice as the type safety may be i found some subtle differences between clang and gcc's take on them and that put me off that stuff altogether

fiery turtle Apr 26, 2025, 7:36 PM

#

^

fiery turtle Apr 26, 2025, 7:37 PM

#

flat badge But smp_mb() etc is stupid

Why?

flat badge Apr 26, 2025, 7:37 PM

#

What exactly are the guarantees of smp_mb()?

#

No offense, but if you think that this question is easy to answer, you probably haven't done enough research yet

fiery turtle Apr 26, 2025, 7:38 PM

#

flat badge What exactly are the guarantees of smp_mb()?

same as atomic_thread_fence(ACQ_REL)?

flat badge Apr 26, 2025, 7:38 PM

#

That fact alone is not obvious at all

fiery turtle Apr 26, 2025, 7:39 PM

#

why is it not obvious?

#

you know there's a read-modify barrier, you know there's a write-modify barrier

flat badge Apr 26, 2025, 7:39 PM

#

For example, why is it not a full memory barrier?

fiery turtle Apr 26, 2025, 7:40 PM

#

because linuxes model was invented before this stuff was added into the standard

flat badge Apr 26, 2025, 7:40 PM

#

Yeah but how many ppl know the intricacies of the linux model? Probably 20?

#

And how many ppl know the c11 model? Probably many thousands

fiery turtle Apr 26, 2025, 7:41 PM

#

to be fair they have the new atomic header which is closer to that

#

https://elixir.bootlin.com/linux/v6.14.4/source/include/linux/atomic/atomic-instrumented.h

flat badge Apr 26, 2025, 7:42 PM

#

Yeah but my point is that for a greenfield project you shouldn't go with something like the linux model at all

fiery turtle Apr 26, 2025, 7:42 PM

#

ofc

#

thats why im doing the gcc builtins

flat badge Apr 26, 2025, 7:43 PM

#

Then we're not disagreeing at all

#

The gcc built-ins follow the c11 model

fiery turtle Apr 26, 2025, 7:43 PM

#

you're probably gonna hate qemu tho trl

#

#define smp_mb()                     ({ barrier(); __atomic_thread_fence(__ATOMIC_SEQ_CST); })
#define smp_mb_release()             ({ barrier(); __atomic_thread_fence(__ATOMIC_RELEASE); })
#define smp_mb_acquire()             ({ barrier(); __atomic_thread_fence(__ATOMIC_ACQUIRE); })

dense steppe Apr 26, 2025, 7:44 PM

#

fiery turtle thats why im doing the gcc builtins

remember to use the __atomic_* builtins instead of the __sync_* ones

dense steppe Apr 26, 2025, 7:44 PM

#

fiery turtle ```c #define smp_mb() ({ barrier(); __atomic_thread_fence(__...

why the barrier()?

fiery turtle Apr 26, 2025, 7:44 PM

#

didnt know about the sync ones

#

whats the diff?

dense steppe Apr 26, 2025, 7:44 PM

#

the __sync primitives are old and outdated

fiery turtle Apr 26, 2025, 7:44 PM

#

dense steppe why the `barrier()`?

one for compiler reorders one for cpu reorders

dense steppe Apr 26, 2025, 7:44 PM

#

all the cool kids use __atomics

dense steppe Apr 26, 2025, 7:45 PM

#

fiery turtle one for compiler reorders one for cpu reorders

but __atomic_thread_fence() is a compiler intrinsic.

fiery turtle Apr 26, 2025, 7:45 PM

#

/* Manual memory barriers
 *
 *__atomic_thread_fence does not include a compiler barrier; instead,
 * the barrier is part of __atomic_load/__atomic_store's "volatile-like"
 * semantics. If smp_wmb() is a no-op, absence of the barrier means that
 * the compiler is free to reorder stores on each side of the barrier.
 * Add one here, and similarly in smp_rmb() and smp_read_barrier_depends().
 */

#

no compiler barrier there

dense steppe Apr 26, 2025, 7:45 PM

#

-_-

fiery turtle Apr 26, 2025, 7:45 PM

#

which is kind of crazy

dense steppe Apr 26, 2025, 7:45 PM

#

why

#

just why

#

wtf

#

GNU, wtf?

fiery turtle Apr 26, 2025, 7:46 PM

#

yeah idk

#

why the hell did qemu just slap a seq_cst everywhere

jaunty fox Apr 26, 2025, 7:47 PM

#

fiery turtle ```c /* Manual memory barriers * *__atomic_thread_fence does not include a com...

bloody hell

dense steppe Apr 26, 2025, 7:47 PM

#

fiery turtle why the hell did qemu just slap a seq_cst everywhere

because it's easy to use correctly

#

as opposed to dealing with the intricacies of acquire/release

#

not to speak of consume

fiery turtle Apr 26, 2025, 7:48 PM

#

consume is only relevant for dead arches i think

dense steppe Apr 26, 2025, 7:49 PM

#

dense steppe as opposed to dealing with the intricacies of acquire/release

apparently the lack of a total load/store order can be problematic for some lockless algorithms unless they are done using seq_cst

jaunty fox Apr 26, 2025, 7:49 PM

#

fiery turtle consume is only relevant for dead arches i think

alpha

fiery turtle Apr 26, 2025, 7:49 PM

#

yeah i still dont understand entirely when u need seq_cst

fiery turtle Apr 26, 2025, 7:50 PM

#

jaunty fox alpha

yeah

flat badge Apr 26, 2025, 8:36 PM

#

fiery turtle yeah i still dont understand entirely when u need seq_cst

when you need guarantees about the order in which two atomic vars are modified

#

For example this: #programming message

fiery turtle Apr 26, 2025, 8:37 PM

#

This one makes sense

vast kestrel Apr 26, 2025, 8:48 PM

#

fiery turtle is smp_Xmb() harder to read than atomic_thread_fence(...)?

Yeah the Linux macros are horrible, no one who does concurrency uses the Linux terms (which don't always align perfectly with what people usually use), but instead uses the C/C++ memory model terms

vast kestrel Apr 26, 2025, 8:50 PM

#

fiery turtle yeah i still dont understand entirely when u need seq_cst

My friend made an example of a bakery lock that requires a seq_cst barrier on x86 Smuggy

flat badge Apr 26, 2025, 8:55 PM

#

Consume is not only relevant for alpha btw

#

Alpha is the only arch where consume needs an explicit fence

#

But you can exploit that dependencies imply ordering on other archs as well

#

It's generally not worth it over acq/rel though except in niche cases

calm latch Apr 26, 2025, 8:58 PM

#

fiery turtle ```c /* Manual memory barriers * *__atomic_thread_fence does not include a com...

What's even the point when you can use __atomic_load/store stuff?

fiery turtle Apr 26, 2025, 9:03 PM

#

calm latch What's even the point when you can use __atomic_load/store stuff?

For example you do memcpy to a buffer and you don't want it to be reordered with some other store

calm latch Apr 26, 2025, 9:03 PM

#

thonk

fiery turtle Apr 26, 2025, 9:03 PM

#

Or you do multiple relaxed atomic stores followed by a stricter barrier

calm latch Apr 26, 2025, 9:04 PM

#

you can just make the last store release?

fiery turtle Apr 26, 2025, 9:04 PM

#

That works too

#

As far as I understand barriers and normal ordering flags for other operations are identical

calm latch Apr 26, 2025, 9:05 PM

#

fiery turtle For example you do memcpy to a buffer and you don't want it to be reordered with...

what situation do you use this in?

#

maybe only copies from userspace

#

(at which point you don't need hardware barriers?)

fiery turtle Apr 26, 2025, 9:06 PM

#

memcpy(data)
atomic_fence(release)
data_is_set.store(true, relaxed)

calm latch Apr 26, 2025, 9:06 PM

#

fiery turtle memcpy(data) atomic_fence(release) data_is_set.store(true, relaxed)

this can also be replaced by release I think? thonk thonk

fiery turtle Apr 26, 2025, 9:07 PM

#

Perhaps

#

https://en.cppreference.com/w/cpp/atomic/atomic_thread_fence

cpp/atomic/atomic thread fence

#

this has some examples

flat badge Apr 26, 2025, 9:08 PM

#

fiery turtle As far as I understand barriers and normal ordering flags for other operations a...

Hm?

fiery turtle Apr 26, 2025, 9:08 PM

#

flat badge Hm?

x.store(y, release) is the same as x.store(y, relaxed) fence(release)

flat badge Apr 26, 2025, 9:08 PM

#

A release barrier is not the same as a store-release

fiery turtle Apr 26, 2025, 9:09 PM

#

well the cppref says it is

flat badge Apr 26, 2025, 9:09 PM

#

No that's wrong

fiery turtle Apr 26, 2025, 9:09 PM

#

how?

flat badge Apr 26, 2025, 9:09 PM

#

The latter is stronger

jaunty fox Apr 26, 2025, 9:09 PM

#

calm latch What's even the point when you can use __atomic_load/store stuff?

unless i misremember, the abstract specification of the c11 atomics says something to the effect that only explicit barriers are specified to order anything other than explicit atomic ops

flat badge Apr 26, 2025, 9:09 PM

#

The barrier orders all proceeding stores vs all following stores

#

While the store release orders all proceeding stores vs. exactly the store done by the store-release

fiery turtle Apr 26, 2025, 9:11 PM

#

how can the barrier order stores that come after the barrier

#

unless im misunderstanding your point

flat badge Apr 26, 2025, 9:11 PM

#

That's the point of release semantics

flat badge Apr 26, 2025, 9:12 PM

#

fiery turtle x.store(y, release) is the same as x.store(y, relaxed) fence(release)

Btw this is also wrong, the barrier needs to be before the store

fiery turtle Apr 26, 2025, 9:12 PM

#

release semantics means all stores before the barrier are not reordered with the following stores

#

im not sure what u mean

flat badge Apr 26, 2025, 9:12 PM

#

With the barrier after the store, it does nothing

fiery turtle Apr 26, 2025, 9:13 PM

#

true

flat badge Apr 26, 2025, 9:13 PM

#

fiery turtle release semantics means all stores before the barrier are not reordered with the...

No that's not correct

#

Acquire/release semantics mean that if an acquire operation reads the value written by a store-release, then all future reads will read newer (or just as new) values than the value that was read by the read acquire

fiery turtle Apr 26, 2025, 9:15 PM

#

yeah but still what exactly is the difference between a release barrier and just a store release

flat badge Apr 26, 2025, 9:17 PM

#

If i do

a.store_relaxed(1)
b.store_release(2)
c.store_relaxed(3)

#

And another thread read acquires c = 3 and then reads a, there is no guarantee that it'll read a=1

fiery turtle Apr 26, 2025, 9:18 PM

#

https://stackoverflow.com/questions/42177234/in-c-is-there-any-effective-difference-between-a-acquire-release-atomic-acces

Stack Overflow

In C++, is there any effective difference between a acquire/release...

Specifically, is there any effective difference between:

i = a.load(memory_order_acquire);
or

a.store(5, memory_order_release);
and

atomic_thread_fence(memory_order_acquire);
i = a.load(

#

I think this has some answers

flat badge Apr 26, 2025, 9:18 PM

#

flat badge And another thread read acquires c = 3 and then reads a, there is no guarantee t...

If you replace the store release by a barrier + relaxed store, this guarantee holds true

fiery turtle Apr 26, 2025, 9:19 PM

#

So you're saying relaxed stores can still be reordered with respect to each other

flat badge Apr 26, 2025, 9:19 PM

#

No I'm saying a release barrier acts on all following stores while a store release acts on one store

#

A release barrier is strictly stronger than a store release

fiery turtle Apr 26, 2025, 9:20 PM

#

flat badge No I'm saying a release barrier acts on all following stores while a store relea...

I don't understand this part

#

How can a happen after b

flat badge Apr 26, 2025, 9:21 PM

#

Compare the snippet that i posted above to:

#

a.store_relaxed(1)
release_barrier()
b.store_relaxed(2)
c.store_relaxed(3)

#

Then the release barrier affects c as well

#

While the store release only ever affects b

#

If a read acquire now reads c=3, a following read is guaranteed to read a=1

#

Which is not true for the store-release

fiery turtle Apr 26, 2025, 9:23 PM

#

So tldr barrier affects all atomic stores and a store ordering only affects this store

flat badge Apr 26, 2025, 9:23 PM

#

yes

fiery turtle Apr 26, 2025, 9:23 PM

#

How is this even possible at CPU level

fiery turtle Apr 26, 2025, 9:24 PM

#

flat badge If i do ``` a.store_relaxed(1) b.store_release(2) c.store_relaxed(3) ```

But if it acquires b is a guaranteed to be 1?

flat badge Apr 26, 2025, 9:25 PM

#

Yes

fiery turtle Apr 26, 2025, 9:25 PM

#

Oh ok makes sense I think

#

Thanks

flat badge Apr 26, 2025, 9:28 PM

#

I have no idea how it works at the hw level but at least aarch64 and rv have explicit acq/rel load/store instructions, so it can apparently be optimized more aggressively in hw than barriers

fiery turtle Apr 26, 2025, 9:31 PM

#

interesting

#

the acq rel stuff on x86 is just a no-op other than the compiler barrier i think

#

a seq cst barrier might emit mfence

frank canopy Apr 26, 2025, 11:17 PM

#

realising i dont think zig even has an equivalent to barrier()

fiery turtle Apr 26, 2025, 11:25 PM

#

Yeah i wouldn't be surprised

lilac knoll Apr 26, 2025, 11:33 PM

#

fiery turtle Linux memory barriers and atomics have very stupid implied ordering and semantic...

these are just sequentialyl consistent

fiery turtle Apr 26, 2025, 11:34 PM

#

Yeah qemu seems to not trust its developers

lilac knoll Apr 26, 2025, 11:34 PM

#

seems not worth using a wrapper

lilac knoll Apr 26, 2025, 11:37 PM

#

fiery turtle How is this even possible at CPU level

'wait for all stores to finish' vs 'wait for this one store to finish'

fiery turtle Apr 26, 2025, 11:38 PM

#

Well independent stores before release are sequenced before the atomic store, that's kind of the point, so you cant just wait for this specific store I think

lilac knoll Apr 26, 2025, 11:52 PM

#

fiery turtle Well independent stores before release are sequenced before the atomic store, th...

hm true

#

is the question 'what's the difference between release store and release barrier in hw'?

fiery turtle Apr 26, 2025, 11:52 PM

#

Basically

kindred beacon Apr 26, 2025, 11:52 PM

#

lilac knoll these are just sequentialyl consistent

sequentianyl

#

good shit

fiery turtle Apr 26, 2025, 11:53 PM

#

kindred beacon sequentianyl

lilac knoll Apr 27, 2025, 12:42 AM

#

fiery turtle Basically

I don't think there's a difference on modern hw because stores pretty much have to retire in-order anyway: reordering stores A and B is risky because there may be a branch mispredict or exception that occurs between A and B, so this is only possible when you know that both instructions are non-speculative

lilac knoll Apr 27, 2025, 2:35 AM

#

(what I'm saying is that formally there is a difference in the ISA between store-release and fence-release, so you still need different instructions for those, but I don't think that a fence release would realistically be worse because store-store reordering is too difficult anyway)

#

nice, another couple hours of time wasted thinking about memory orderings

flat badge Apr 27, 2025, 6:56 AM

#

that's assuming that the architecture has release fences and release stores though

#

for example on risc-v you get acq/rel with no extra instructions (since it's a bit in the load-reserve/store-conditional isns)

#

but a fence needs an explicit instruction

#

and a fence may also degrade to a stronger fence than necessary

lilac knoll Apr 27, 2025, 11:29 AM

#

lilac knoll I don't think there's a difference on modern hw because stores pretty much have ...

I was completely wrong as per usual

fiery turtle Apr 27, 2025, 11:32 AM

#

Why?

lilac knoll Apr 27, 2025, 11:41 AM

#

I just misremembered how store buffering works entirely

#

once a store retires, it's known to be non-speculative. at that point it gets entered into the store buffer

#

so entries in the store buffer can drain in whatever order the CPU likes. on x86, due to TSO, this must (at least appear to) be FIFO

#

but on aarch64 it's pretty plausible that a store could jump ahead of a prior entry in the store buffer if there's a cache miss or something

#

https://stackoverflow.com/questions/79594627/what-store-store-reordering-do-modern-cpus-do-in-practice

Stack Overflow

What Store/Store reordering do modern CPUs do in practice?

Aarch64 and RISC-V WMO seem to allow Store/Store reordering according to their formal specifications.
However, Store/Store reordering seems very tricky to perform in practice: the CPU would need to

#

a store release effectively says 'this store cannot be moved ahead of an earlier store'

a fence-release says 'any later stores cannot be moved ahead of an earlier store'

so a cpu could in theory see the following sequence:

store 1
store-release
store 2

and reorder this to:
store 2
store 1
store-release

whereas a fence-release prohibits this

#

I do suspect this is a possible reordering that aarch64 stlr permits in practice but I'd need to cook up a litmus test to demonstrate that

fiery turtle Apr 27, 2025, 11:54 AM

#

interesting

lilac knoll Apr 27, 2025, 11:55 AM

#

I could be completely wrong again but who knows

dense steppe Apr 27, 2025, 12:09 PM

#

hmmm, that's a very long RSDP

#

thonk

fiery turtle Apr 27, 2025, 12:10 PM

#

the address is bogus as well

dense steppe Apr 27, 2025, 12:10 PM

#

no. the address is the copy in the multiboot data

fiery turtle Apr 27, 2025, 12:10 PM

#

are you giving it the v2 version of the tag?

#

otherwise it cuts off the extended part

#

and uacpi gets garbage

#

managarm ran into this as well

dense steppe Apr 27, 2025, 12:11 PM

#

dense steppe hmmm, that's a very long RSDP

this only happens with BIOS, not UEFI

fiery turtle Apr 27, 2025, 12:12 PM

#

do u handle misaligned mappings correctly

dense steppe Apr 27, 2025, 12:14 PM

#

yeah, I'm very sure I do

fiery turtle Apr 27, 2025, 12:15 PM

#

yeah idk but im pretty sure this has something to do with multiboot crap

dense steppe Apr 27, 2025, 12:15 PM

#

also here's the code that "finds" the RSDP:

    multiboot_tag *rsdp1 = nullptr, *rsdp2 = nullptr;
    for (multiboot_tag *tag : mb2_tags) {
        switch (tag->type) {
        case MB2_TAG_RSDPv1:
            rsdp1 = tag;
            break;
        case MB2_TAG_RSDPv2:
            rsdp2 = tag;
            break;
        default:
            break;
        }
    }

    if (rsdp2)
        rsdp1 = rsdp2;
    if (!rsdp1)
        panic ("No ACPI RSD PTR was provided by the bootloader.");

    acpi_set_rsdp (virt_to_phys ((uintptr_t) rsdp1) + 8);

dense steppe Apr 27, 2025, 12:16 PM

#

dense steppe hmmm, that's a very long RSDP

it also says v00, which is "interesting"

fiery turtle Apr 27, 2025, 12:16 PM

#

00 is what qemu sets it to

fiery turtle Apr 27, 2025, 12:18 PM

#

dense steppe also here's the code that "finds" the RSDP: ```c++ multiboot_tag *rsdp1 = nu...

can u just print out the rsdp fields yourself right here?

#

see if they make sense

#

also how does your virt_to_phys work

dense steppe Apr 27, 2025, 12:18 PM

#

static inline uintptr_t
virt_to_phys (uintptr_t x)
{
    return x - HHDM_OFFSET;
}

#

kernel direct map

fiery turtle Apr 27, 2025, 12:19 PM

#

are you sure you construct it correctly?

dense steppe Apr 27, 2025, 12:19 PM

#

100%

fiery turtle Apr 27, 2025, 12:19 PM

#

fiery turtle can u just print out the rsdp fields yourself right here?

anyway try this

dense steppe Apr 27, 2025, 12:19 PM

#

sure, I'll just have to grab the multiboot definitions for that

#

1sec

#

-_-

gentle peak Apr 27, 2025, 12:20 PM

#

i'm fairly sure this is a uacpi bug, it reads the length field from the rsdp unconditionally but that field is only present in revision >= 2 https://github.com/uACPI/uACPI/blob/22895b49e372303e1fb8338cead78534ed44b966/source/tables.c#L118

GitHub

uACPI/source/tables.c at 22895b49e372303e1fb8338cead78534ed44b966 ...

A portable and easy-to-integrate implementation of the Advanced Configuration and Power Interface (ACPI) - uACPI/uACPI

dense steppe Apr 27, 2025, 12:21 PM

#

ah. that's it.

#

because everything seems to work, it loads all the other tables correctly etc.

fiery turtle Apr 27, 2025, 12:21 PM

#

gentle peak i'm fairly sure this is a uacpi bug, it reads the length field from the rsdp unc...

thats not rsdp tho

dense steppe Apr 27, 2025, 12:21 PM

#

~~it is~~ ah the linked code

gentle peak Apr 27, 2025, 12:21 PM

#

oh sorry yeah that's rsxdt

fiery turtle Apr 27, 2025, 12:21 PM

#

thats {R,X}SDT

#

thats retrieved from RSDP

gentle peak Apr 27, 2025, 12:22 PM

#

https://github.com/uACPI/uACPI/blob/22895b49e372303e1fb8338cead78534ed44b966/source/tables.c#L84 what i said's still correct, just linked the wrong line

dense steppe Apr 27, 2025, 12:22 PM

#

but the RSDP handling in dump_table_header is incorrect

#

    if (!uacpi_memcmp(hdr, ACPI_RSDP_SIGNATURE, sizeof(ACPI_RSDP_SIGNATURE) - 1)) {
        struct acpi_rsdp *rsdp = hdr;

        uacpi_info(
            "RSDP 0x%016"UACPI_PRIX64" %08X v%02X (%6.6s)\n",
            UACPI_FMT64(phys_addr), rsdp->length, rsdp->revision,
            rsdp->oemid
        );
        return;
    }

gentle peak Apr 27, 2025, 12:22 PM

#

seems to be a purely cosmetic issue though

fiery turtle Apr 27, 2025, 12:22 PM

#

ah, the dump function is wrong

#

yeah fair

dense steppe Apr 27, 2025, 12:24 PM

#

just out of curiosity, apart from faking a RSDP yourself, is it possible to initialize uACPI from a RSDT or XSDT?

#

afaict no

fiery turtle Apr 27, 2025, 12:24 PM

#

nope

dense steppe Apr 27, 2025, 12:25 PM

#

anyway. daily atomic warcrime:

#

note the relaxed

fiery turtle Apr 27, 2025, 12:26 PM

#

probably need an RCU here

#

or just a console_lock

dense steppe Apr 27, 2025, 12:26 PM

#

fiery turtle probably need an RCU here

no, because there is no console_unregister

#

meme

fiery turtle Apr 27, 2025, 12:26 PM

#

then relaxed should be fine no?

dense steppe Apr 27, 2025, 12:26 PM

#

yup, it is fine.

gentle peak Apr 27, 2025, 12:27 PM

#

you probably don't even need an atomic there except for the initial load of the head, if there's no console_unregister the next field of a console in the list will never be written to again

#

unless you insert at tail instead of at head

dense steppe Apr 27, 2025, 12:28 PM

#

it is fine for two reasons:

seq_cst load in printk_emit synchronizes with seq_cst store in console_register, which happens after past seq_cst stores. thus the seq_cst load happens after all seq_cst stores it can see, and the relaxed load(s) cannot be moved before.
I wasn't planning on having more than one console for the time being

dense steppe Apr 27, 2025, 12:28 PM

#

gentle peak you probably don't even need an atomic there except for the initial load of the ...

good point, will remove

fiery turtle Apr 27, 2025, 12:28 PM

#

why not just acquire here and release in register instead?

#

you have one writer and one reader

dense steppe Apr 27, 2025, 12:29 PM

#

true

#

idk, because I felt like it

#

anyway I will replace this with a 3000 LoC message ringbuffer sooner or later anyway meme

fiery turtle Apr 27, 2025, 12:29 PM

#

or well, if you have multiple registers, you probably want an ACQ_REL cmpxchg loop in register

#

and just acquire here

dense steppe Apr 27, 2025, 12:30 PM

#

meme

fiery turtle Apr 27, 2025, 12:30 PM

#

L

dense steppe Apr 27, 2025, 12:31 PM

#

dense steppe

(printk does not take console_lock. printk takes another lock instead.)

fiery turtle Apr 27, 2025, 12:33 PM

#

gentle peak <https://github.com/uACPI/uACPI/blob/22895b49e372303e1fb8338cead78534ed44b966/so...

good catch btw, ill mention u in the release logs again

fiery turtle Apr 27, 2025, 1:30 PM

#

should i get rid of 32-bit support in the ultra kernel think3d

#uACPI - a portable and easy-to-integrate ACPI implementation