Ultra

[prime wraith]

[ 0.000000] Command line: console=ttyS0 root=/dev/sda retard.bar=3 earlyprintk=serial kvm.enable_vmware_backdoor= -- 1hello=world - nokaslr

it's after -- so it's not part of the kernel command line

oh well i guess its backward compat at this point

[ornate jasper]

ill talk about it more in my own thread when i do it, but theres legit just a variable in @import("builtin") (which is where all the target etc info that would be preprocessor symbols in c go) thats true if its compiled in test mode and false otherwise, and a variable there that has a comptime list of functions (iirc not function pointers, the comptime functions themselves) for each test case and you can just check in main if the test variable is true and iterate the list of tests and run them if it is

[red parcel]

Yeah, I actually helped someone to build a firewall with that

We dropped any non-whitelisted udp packed (by source IP), where whitelisting comes from a usermode tcp server

And it helped them handle huge ddos waves on the servers without a problem

Because the udp packet was dropped wayyyy before it reached the network stack and could hog resources

[willow fern]

there are explicit points where it is called, and then there are kProbes which are just replacing a particular instruction with INT3

[carmine token]

a lot of uses of ebpf are gated behind root though (and the ones that are not are severely limited in what they can do)

because while the verifier ensures memory safety, there is still a lot of nefarious stuff that you can do

seccomp uses bpf, not ebpf

[prime wraith]

Yeah

[carmine token]

nefarious things such as leaking kernel pointers / leaking secrets / slowing down execution / performing side channel attacks

[ornate torrent]

AKA things that are fun

all the fun stuff you can do to a live system is guarded behind root (or CAP_SYS_ADMIN or whatever)

[prime wraith]

If you have root u can just crash the system by writing to a sysrq procfs file, so like

[carmine token]

well, you can't load new modules though

if module signing is on

you can probably halt the system in one way or another but you don't get arbitrary kernel code exec as root in a locked down linux system

[prime wraith]

True

[ornate torrent]

yeah but there are also fun ways to crash the system, this is boring.

[wide ether]

you can also crash linux by zeroing out the hpet page with /dev/mem

[fossil nexus]

I would really hope that's disabled on most distros

[dusky kraken]

you need to boot with iomem=relaxed at least

[fossil nexus]

isnt there a kconfig option

or at least away to restrict it to non-kernel memory

[red parcel]

By default /dev/mem won't allow you to access ram

Only anything marked as reserved in the memory map (and anything not marked at all in it)

With kernel lockdown it won't let you do anything I think

(which most distros enable when booted with secure boot)

[prime wraith]

added vga-text as an earlycon for fun

[onyx turret]

yummy

[prime wraith]

it's only possible to enable if there isn't a fb

static error_t vga_text_console_init(void)
{
    if (g_boot_ctx.fb != NULL)
        return EBUSY;

    return register_console(&vga_text_console);
}

[onyx turret]

makes sense

[prime wraith]

until i add proper support for the kernel text console i guess ill just use this

[thorny kelp]

vga text mode is love, vga text mode is life

[onyx turret]

i personally hate it

but everyone likes their own shit

[prime wraith]

actually this should also check that it was booted via bios

[onyx turret]

framebuffers my beloved

[prime wraith]

well u say that because u have flanterm

handrolling text rendering is annoying af

[onyx turret]

🥀

[twilit geode]

I like the vibes

but its annoying to work with

cuz low screen space

[onyx turret]

js use flanterm ez fix

[thorny kelp]

80x50 text mode

[prime wraith]

25

[onyx turret]

25 years until the sun explodes?

im confused

[onyx turret]

Intel

[prime wraith]

how

[thorny kelp]

sec

[onyx turret]

the voices in my head?

thats crazy

[prime wraith]

static void vga_write(struct console *con, const char *str, size_t count)
{
    size_t i;
    struct vga_state *vs = con->priv;
    u8 *cursor = phys_to_virt(0xB8000);
    size_t offset = vs->y * (80 * 2) + vs->x * 2;

    for (i = 0; i < count; i++) {
        if (str[i] == '\n') {
            vs->y++;
            vs->x = 0;
            offset += 80 * 2;
            continue;
        }

        cursor[offset++] = str[i];
        cursor[offset++] = 15;
    }
}

this is all it takes, well without scrolling and stuff ofc

[onyx turret]

https://tenor.com/view/the-voices-meme-cat-gif-23917781

[thorny kelp]

mov ax, 0x1112
mov bl, 0
int 0x10

its not a separate mode but different font

[prime wraith]

ah okay, yeah i dont support that in my bootloader

i just do video-mode=unset atm

in the config

[thorny kelp]

you should :^)

jk

[wide ether]

not really?

[onyx turret]

prs are welcome

[prime wraith]

lol

[onyx turret]

lol

[wide ether]

the annoying part is ansi escape sequence parsing

[prime wraith]

yes

[thorny kelp]

eeeeh its not that hard

[prime wraith]

i have a pixel font i've handrolled for the old ultra

[twilit geode]

add a vga text mode backend to flanterm ezpz

[thorny kelp]

lol

[onyx turret]

txet

[wide ether]

i didnt say hard

annoying

[onyx turret]

mretnalf

anyways

[thorny kelp]

true

[wide ether]

because there's a lot of logic involved

[onyx turret]

id imagine ansi escape code parsing to be really complicated

or well

annoying

[wide ether]

just look at flanterm

[onyx turret]

fun

[sterile kayak]

wouldn't it be better/make more sense to add another framebuffer memory model

because that's pretty much exactly what it is

[prime wraith]

like a text mode fb?

[sterile kayak]

and then you don't have to think of all the corner cases to determine whether to use the VGA console, you just rely on the bootloader to tell you

yeah that's what the VGA text console is

it's also how multiboot represents it for what that's worth

[prime wraith]

yeah i should add that as a video mode value

it's not really part of the protocol atm, i just know the bios bootloader sets that mode if no video mode was requested

[carmine token]

Supporting vga text mode is weird

Is there any situation where you ever want to use that? I don't think so

[prime wraith]

it's a temporary thing until i have some basic drm in place where i can properly use a framebuffer for the kernel mode console

[twilit geode]

Y not just be serial only then

Oh wait real hw

[prime wraith]

vscode debugger silently swallows stdout somewhere

idk where

and yeah real hw

[sterile kayak]

how did you set it up

i used the c/c++ extension's debugServerPath thing

with that, qemu stdout is logged in the debugger tab along with the gdb output

[prime wraith]

I just have it run qemu via my script and then run gdb

Via launch.json or whatever

But ill look into that extension

[wide ether]

fake

codelldb works for me

the downside is that it's not gdb

[prime wraith]

Never heard

[wide ether]

it's a vscode extension using lldb

[prime wraith]

I'll check it out

[sterile kayak]

i've tried codelldb before and my main issues with it were the lack of a good way to autolaunch qemu and breakpoints sometimes just not triggering if set before starting the debugging session

have you figured out solutions to those?

[wide ether]

the latter happens with kvm only for me

[sterile kayak]

happened with tcg too for me

strange

[wide ether]

and i always launch qemu myself

[sterile kayak]

fair enough

[wide ether]

but you can have it as an extras step

i can look again later

[sterile kayak]

the closest thing i managed to get to proper autolaunch is a pre launch background task that started qemu

but that had a few issues as well

for one, terminating the debugging session didn't close qemu, which was annoying

but also if i accidentally left qemu open in the background it'd silently fail and i'd have to wait for the lldb connection timeout

[wide ether]

for me, my distro is completely seperate from the kernel so the launch json is actually part of the distro

[sterile kayak]

i do that too yeah

[wide ether]

i often want to change kvm/tcg, mem and smp, so I don't autolaunch anything

(it needs a kernel + image rebuild anyways)

[sterile kayak]

for me having to change the vm config is very uncommon compared to simple code changes so i value a quick edit-and-relaunch cycle

and i have multiple launch configs for the most common vm changes

[carmine token]

Why don't you just set up some temporary framebuffer

Imho vga text mode support is an anti feature

Supporting it probably makes the code quality worse

[prime wraith]

Hm?

[carmine token]

Like, instead of adding 20 lines for font drawing you now need several hundred lines of useless infrastructure

Like backbuffers that support text

Scrolling code for text

A way to communicate a fb format that is incompat with everything else

[prime wraith]

That's if you want to fully support it, then yeah. I just use it as stdout

(For the early console)

[ornate torrent]

really?

you have your struct console list or whatever, and printk sends stuff to those.

one console for framebuffers, one console for text mode VGA

[carmine token]

there is literally no hw that supports vga text mode but no proper framebuffer

[ornate torrent]

I agree with you that text mode VGA is a thing of the past

but supporting it as an early sink for output doesn't lower code quality (outside of the file that implements text mode VGA as a printk console)

[carmine token]

like, the bit banging required to put vga into a framebuffer mode using the raw vga registers is literally shorter than the text mode support code

It's not a thing of the past, it was already useless when it released

[carmine token]

well, you still need to add detection, hand off, boot protocol support, memory mapping

[ornate torrent]

yeah

but atleast it doesn't become a mess in the framebuffer console code

[sterile kayak]

the same is true for actual framebuffers

[sterile kayak]

these are also necessary for proper-framebuffer-based text consoles

including the format communication (memory model)

[carmine token]

yes, it's also true for actual framebuffers

but for actual framebuffers you need the code anyway

[sterile kayak]

all the infrastructure necessary for text mode consoles is also necessary for pixel framebuffer text consoles, so if you want the latter anyway you might as well add support for the former

is my point

[carmine token]

i don't think that's true

you don't need vga detection for proper framebuffers etc

and text mode to graphical hand off

or boot protocol support to communicate text mode

if there was any hardware that only supported vga text mode, i'd agree that it's worth supporting

but there is (and never was) any hardware where text mode is the best option on x86

if you want 640x400 text mode on a raw VGA card with no extra support, just set that mode using the vga regs and use proper drawing code afterwards

[sterile kayak]

• vga detection: not necessary, you just use it if the bootloader tells you that's the framebuffer format and otherwise you don't. on the bootloader side you intrinsically know it's supported because it has been since the original ibm pc (therefore if you're on bios you have text mode)
• text mode to graphical handoff: this is just runtime video mode switching, and the exact same code is necessary for text-to-graphical as graphical-to-graphical
• boot protocol support: the only thing that's necessary is a new value in the memory model enum, which you need for proper framebuffers as well

i agree it's not worth supporting in graphics card drivers but that's not what we're discussing

[carmine token]

if you're using a fully featured bootloader anyway, just tell it to set the mode

or let me rephrase the argument:

• clearly supporting VGA text mode has non-zero cost
• but where is the non-zero benefit?

[sterile kayak]

pre-VBE that's not particularly practical because you either need the user to tell you whether you have cga/ega/vga or you need to restrict yourself to the modes cga supported

or assume you have vga

[sterile kayak]

the non-zero benefit is that on pure VGA compatibles and prior it's the highest resolution and color depth combination for text that does not involve bank switching

it's 640x400 with 16 colors, so an equivalent pixel framebuffer is 125k of vram, and iirc VGA cannot expose that amount of vram simultaneously; you'd have to actually talk to the vga hardware when writing to the fb to make it switch which vram range is exposed to the main memory bus

and if you're going to talk to the vga hardware anyway, text mode allows you to have hardware scrolling with a much bigger buffer than the same resolution pixel-based framebuffer would (since you can change where in vram the top-left corner is), which has a pretty noticeable performance impact on systems so old they don't have vbe

[carmine token]

if you have a system that only supports cga with no vbe or extended vga modes, it won't support 32 bit protected mode either

[sterile kayak]

wrong

compaq deskpro 386

[carmine token]

i guess

well yeah, imho the benefits are outweighted by the costs anyway, even if you find one machine where it's slightly more convenient

[sterile kayak]

i don't really see any significant costs

the bootloader knows it exists simply because it's running on bios and bios machines always have it, the kernel knows it exists because that's the memory model the bootloader passed, etc

the only thing that needs to be adapted is the text drawing code, and that's extremely easy because, well, text mode

[carmine token]

bios machines do not always have vga textmode

in fact, there are probably 2+ orders of magnitude more machines still in use that have bios but no textmode

compared to VGA only but 32 bit pmode

[sterile kayak]

can you give examples? any machine that wants to maintain backwards compatibility has to have text mode because the original ibm pc had it

[carmine token]

any bios machine that you boot with a recent nvidia or discrete intel card

[sterile kayak]

emulated via smm

[carmine token]

no?

how would that even work

[sterile kayak]

that's what seabios does for example on q35 (as evidenced by the lack of output with q35,smm=off)

[carmine token]

the bios relies on the option ROM to set the mode

i know for a fact that if you disable VGA on intel (there is a register with a lock bit for it), there won't be any emulation

https://github.com/coreboot/seabios/blob/master/src/optionroms.c#L449

seabios also calls into the option ROM to set the mode

https://github.com/coreboot/seabios/blob/b686f4600792c504f01929f761be473e298de33d/src/bootsplash.c#L40

in particular, it calls into int 10

which is provided by the option rom

(or not, on current gen hw)

[prime wraith]

i'll have to double check but im pretty sure even my 50 series gpu has an emulated text mode

[carmine token]

i could be wrong but i thought that nvidia dropped bios option rom support

and now only has uefi option rom support

[prime wraith]

if i boot in CSM mode i still have graphics, but maybe that still uses the uefi option rom? idk

[carmine token]

yes, it does

[prime wraith]

so are you saying my csm mode won't have a text mode?

[carmine token]

no, i'm saying: if you put the same card into a native bios machine, you won't have text mode

[prime wraith]

oh ok, i mean thats not really an issue

who's using pre-2011 hardware with a 2020 gpu

[carmine token]

it's probably more widespread than pre-vbe hw :^)

[sterile kayak]

huh i could've sworn seabios did text mode emulation via smm for graphics devices without text mode support like virtio

but apparently not?

[prime wraith]

i mean it stops working without smm

so it has to be that

[carmine token]

virtio supports text mode

[prime wraith]

no like, the default bochs vga device

it doesnt work if u disable SMM

i mean text mode

[carmine token]

https://docs.oasis-open.org/virtio/virtio/v1.3/csd01/virtio-v1.3-csd01.html#x1-4150007

[carmine token]

might be coincidental

[prime wraith]

iirc igpus have a similar thing

u can disable the legacy vga registers via some bit

[carmine token]

i'm not sure what never intel gpus do (intel arc may just not have VGA support at all?)

but older intel cards have vga registers

[prime wraith]

the descrete ones might not idk

[carmine token]

with a lock bit to disable VGA until next reboot

[onyx turret]

infy are u doing trying to disable vga text mode????? im so confused reading the conversation here

also how are u lol

[prime wraith]

uhhhh

lmfao

im good hbu?

[onyx turret]

well im feeling better too

mostly due to the sleep tablet

otherwise it would be sameey

😭

[prime wraith]

basically in my bootloader config atm i do video-mode=unset, which leaves the 80x25 VGA text mode it sets for itself

[onyx turret]

right

[prime wraith]

i just added an option to use that text mode in the kernel

well im probably not gonna upstream it but

i have it now

[onyx turret]

the sleep tablet for me improves my mood so it makes me able to feel smt for a short amount of time lol

[prime wraith]

glad to hear

[onyx turret]

it only lasts an hour tho

[prime wraith]

perfect time to speedrun nyaux dev

[onyx turret]

ima do that

[prime wraith]

this is kinda me and energy drinks

[prime wraith]

ftp.gnu is a piece of crap

I wonder if there is something else i can use

[wide ether]

ftpmirror.gnu.org

[prime wraith]

thanks

[prime wraith]

currently slightly unfucking my vsnprintf in preparations for a lockless logger

i wonder how much time its gonna take me

i forgor most stuff from when i've looked at linux

so ill have to restudy it

also spent most time playing resident evil 1

kinda enjoy it but also hate it a lot

[ornate jasper]

when you finish lockless logging id be interested in seeing the code btw, or when youve got progress anyway. the linux source is hard to read lol

[prime wraith]

Yeah also their memory model is fucked

Ill let you know

[ornate jasper]

👍

also good luck with it

[prime wraith]

Lol thanks

[carmine token]

By lockless locking you mean nmi safe logging, i assume?

Logging is never completely lockless

[carmine token]

Managarm has nmi safe logging

[ornate torrent]

probably lockless message enqueueing

AFAIK that's pretty much what the Linux prb is

[carmine token]

yeah

But to actually flush out messages you still need locks

Or at least any sane implementation would use locks

[prime wraith]

the flushing is lockful ofc

i mean actually allocating and storing a message

nmi safe & reentrant yeah

[carmine token]

i wouldn't call that lockless logging though

[prime wraith]

imo flushing is unrelated to logging

[carmine token]

it's more than half of the logging engine lol

[prime wraith]

ehh not really

[carmine token]

making a lock free ringbuffer is relatively easy

but the interaction with the actual logging backends etc is also not trivial

[prime wraith]

for (console in consoles)
while (console->seqnum < logbuf->seqnum)
console->seqnum = logbuf->getmsg(buf, console->seqnum);
console->write(buf);

[carmine token]

yeah that's not going to fly

for example because that does not work from nmis

[prime wraith]

flushing is never done in an nmi context

[carmine token]

I'd be very surprised if Linux has that limitation

given that Managarm doesn't have it

it'd also mean that for synchronous MCEs you'd get no output at all

etc

[prime wraith]

it does flushing by waking up a klogd thread so yeah

the calling thread never does flushing

[carmine token]

i'm pretty sure you do get output on linux even if an NMI or MCE panics

it'd be insane not to support that

and this means that you need to be able to flush from NMI context

[prime wraith]

i mean the panic handler probably forces a flush

at that point it doesnt matter

https://elixir.bootlin.com/linux/v6.16/source/kernel/printk/printk.c it has some panic-related stuff here

[carmine token]

I have no idea what linux does but managarm's handling is more sophisticated than that

[prime wraith]

what is managarm handling

[carmine token]

For non-urgent messages we raise a self ipi which then unblocks the flushing task immediately after nmi (we can't unblock the flushing task from nmi because the outer frame might currently have task and scheduler locks)

For urgent messages we first check if we can enter the normal logging path despite being in nmi context

If that's not possible because locks are currently taken by the same cpu, we enter an expedited logging path which only considers backends that can cope with reentrancy

[carmine token]

which excludes everything that needs to wake up userspace etc

[prime wraith]

thats interesting

[fossil nexus]

why do you need to flush in NMI though

outside of panic

[carmine token]

debugging nmi stuff, for example

Or logging from nmi storm :^)

which is actually not that hard to trigger with PMCs

[onyx turret]

yo im so sorry for not checking on ultra, ive js been locked in on nyaux. i see your working on the lock free logging very cool infy!

[prime wraith]

Thank you

Not much work has been done yet so you didn't miss anything

[onyx turret]

understandable but its still good to see progress!

[prime wraith]

this is all my progress

[onyx turret]

😭

[prime wraith]

ikr

[ornate jasper]

mood lmao

my progress looks suspiciously like playing terraria rn myself

[onyx turret]

real

[fossil nexus]

i dont rly see why that requires flushing from nmi as opposed to flushing after

you can also use debugger for that

nmi storm yeah but that should just become a panic

[carmine token]

you cannot always easily attach a debugger

[carmine token]

well, it should probably lead to the source getting masked

some back-to-back nmis are not always avoidable

For example when using pmcs you may need some backoff logic when you encounter nmi storms

[fossil nexus]

if you are getting so many nmis that the kernel cannot make forward progress that should become a panic no?

[carmine token]

being able to log in these situations is useful

[fossil nexus]

sure but im not quite seeing the value of needing to flush inside the nmi there

you can still log and it can be flushed later no?

[carmine token]

You can just turn the PMCs off or increase their thresholds

[fossil nexus]

for the pmu case yeah

id be curious to know what linux does in any case

[carmine token]

It's not that you won't ever make forward progress again, it's that you may encounter situations where you have 10+ nmis back to back

[carmine token]

It does back off when pmcs fire too often

[fossil nexus]

i mean nmi logging

not disputing the pmc backoff stuff

[carmine token]

Ah, no idea

It's true that in production the log level probably won't be that high and you'll most likely never need to flush from nmi context

Mnaagarm also only does it for urgent messages (including panics)

But it's still quite useful for debugging and I'd be surprised if Linux can't do it

[carmine token]

At least, if it can't even do it in some limit form e.g. to uarts or other in kernel outputs

[prime wraith]

Currently afk for a bit spending time with the gf, but hopefully will resume shortly

[proper tulip]

teach her to code, point her to the linux osurce code and make her re-implement parts of it so we can get ultra mvp faster

[twilit geode]

real

[prime wraith]

Real

I actually started teaching her python

[twilit geode]

my bf learned some python but he wants to switch to cs and my uni throws you into the trenches in the first semester (c, racketlang and some risc made up processor assembly)

[prime wraith]

What the hell is racketlang

[twilit geode]

Its like similar to scheme

[dusky kraken]

isnt it a superset of scheme?

[prime wraith]

Looks like lisp

[dusky kraken]

scheme is a descendant of lisp yeah

[prime wraith]

Ah

Thank god I didn't have to learn that

[twilit geode]

Our exams were on paper

Fun times

[dusky kraken]

smh i learned scheme for fun

[prime wraith]

You are built different

[onyx turret]

you have a gf????????????????????

?!?!??!

when did this infy lore drop

[prime wraith]

For the last 2 years yes

[onyx turret]

how dude tell me the strat

2 years dude

how

like dude

i cannot keep a relationship for even a month anymore, nor can i get a gf anymore too

[prime wraith]

U have like 9999 girlfriends wdym bro

[onyx turret]

crazyyy 💀

[prime wraith]

Ikr

You should teach us

Vro is pretending he's weak

[twilit geode]

Gfs are so outdated the real deal are bfs

[wide ether]

⏰

[uneven abyss]

rolling release

[haughty notch]

But the difficulty of finding one is even higher

[onyx turret]

miskalov likes boys? new lore dropped?

[haughty notch]

Dude my discord profile colors is of a bi flag

[onyx turret]

i did not realize lol

[uneven abyss]

playing for both sides.... yet still bi-yourself...

[onyx turret]

roasted!

[haughty notch]

• I've told it like a million times

(once an #lounge-0)

(it can't be more obvious )

[uneven abyss]

are you good at git bisect

bisectual

[haughty notch]

(though I don't know at this point)

[ember reef]

but... he's a guy. how does that work

[uneven abyss]

check patch notes on Life version 5.3.2-r45.3

[twilit geode]

So you have a bicycle right
When you have a square wheel and a round wheel, it doesnt move
But when you have two round wheels it moves

[uneven abyss]

copycat

[inland wave]

what about two square wheels

round wheels are overrated

[onyx turret]

well well well

[uneven abyss]

we should build square wheels to appreciate the round ones better

[haughty notch]

those are lesbians

[uneven abyss]

✂️

[ember reef]

?

I don't get it

Guys like girls.

[onyx turret]

im a guy and i like boy- THATS A JOKE I DO NOT LIKE BOYS

[uneven abyss]

and other lies you can tell yourself

[onyx turret]

lies lies lies

[twilit geode]

Im a guy and I like a guy :3

[uneven abyss]

straighest member of the osdev discord server:

[inland wave]

call me dumb but I genuinely can't tell if you're joking or not

[uneven abyss]

local zygote uninstalled irony module from his kernel: what happens next will shock you

[ember reef]

I'm not.... I don't understand

[twilit geode]

Are you..... Homeopatic.......

[uneven abyss]

i'm... homozygous....

[inland wave]

I'm home-schooled leave me alone XD

[haughty notch]

are you homo sapiens?

[twilit geode]

@prime wraith I was talking in another server about 3d stuff and it made me remember about your plans for udrm

What happened to that

[prime wraith]

That wasnt my project lol

[twilit geode]

I remember you wanted to do something like it

[prime wraith]

Yeah I might try to make it a library once I reach that stage in the os where I can actually support one, but not sure it might just be my own non portable code

Since it may be a bit too complex to make kernel api for

Although Nvidia open exists so maybe not

[twilit geode]

Yeah it seems quite complex

[prime wraith]

Designing my "public" log ring api, this is what i have so far:

#pragma once

#include <common/types.h>
#include <common/error.h>
#include <common/string_container.h>

struct log_ring;

struct log_record_info {
    reg_t id;

    // Nanoseconds since boot when this log message was committed
    u64 timestamp_ns;

    // Length of the log message NOT including the NULL terminator
    u32 length;

    // Syslog information
    u8 facility : 5;
    u8 level : 3;
};

struct log_record {
    struct log_record_info *info;
    struct string text;
};

struct log_ring_reservation {
    struct log_ring *ring;
    // TODO: irq_state
    reg_t id;
};

/*
 * Initialize a log record to be written to the ring.
 *
 * 'length' is the size in bytes of the message to be written including the NULL
 * terminator.
 */
void log_record_setup_for_writing(
    struct log_record *rec, size_t length
);

/*
 * Initialize a log record to be read from the ring.
 *
 * The reader provides a buffer 'out_buf' that accepts a maximum of 'length'
 * bytes where a message from the ring will be copied to.
 */
void log_record_setup_for_reading(
    struct log_record *rec, struct log_record_info *out_info,
    const char *out_buf, size_t length
);

/*
 * Reserve space for a log record in the ring.
 *
 * On success, the specified log record's fields are set to point to the
 * reservation. A successful reservation also disables interrupts until a
 * following log_ring_commit.
 *
 * NOTE:
 * Interrupts are disabled to reduce the likelihood of the log ring clogging
 * since a live reservation prevents the tail from advancing when a wrap occurs.
 */
error_t log_ring_reserve(
    struct log_ring*, struct log_ring_reservation*, struct log_record*
);

/*
 * Reserve space for an extra log record by extending the last commited entry
 * instead of adding a new one.
 *
 * NOTE:
 * This is only viable for very early initialization stages before SMP since a
 * call to 'log_ring_reserve' will finalize any commited descriptors thus making
 * it impossible to extend them.
 */
error_t log_ring_reserve_extend(
    struct log_ring*, struct log_ring_reservation*, struct log_record*
);

/*
 * Commit a reservation but don't make it available to the readers yet as to
 * allow it to be possibly extended later.
 *
 * NOTE:
 * A reservation can be extended via 'log_ring_reserve_extend' as long as no
 * calls to 'log_ring_reserve' have been made in between.
 */
void log_ring_commit(struct log_ring_reservation*);

/*
 * Publish a log ring reservation by making it available to the readers.
 *
 * This reservation can no longer be modified.
 */
void log_ring_publish(struct log_ring_reservation*);

Only thing left is to implement it

[prime wraith]

will try to get this test case working tomorrow hopefully

i have all preliminary api and structs done

[carmine token]

mfw the lockless log ring is not lockless

but i guess in linux it's the same

this is a copy of linux's design, isn't it?

it's not truly lockless since a reservation is a lock

actually, what happens if:
reservation on CPU 1 -> CPU is stalled for extended period of time -> ring is filled completely by CPUs != 1 -> NMI on CPU 1 -> log from NMI?

[prime wraith]

Hm?

Reservation just prevents the ring from wrapping past itself

[carmine token]

yeah. a cpu holding a reservation can prevent other CPUs from making progress

which is the definition of a lock

[prime wraith]

Well in theory it can

[prime wraith]

If its stalled with interrupts disabled something has seriously gone wrong lol

But yeah everything would be dropped if that happens

There's no way around that really

[prime wraith]

Yep

[carmine token]

well, Managarm's design has no reservations so it doesn't run into this problem

[prime wraith]

It won't prevent progress, just stuff will be dropped

[prime wraith]

How do u prevent the ring wrapping while a cpu is in nmi then?

[carmine token]

we don't

[prime wraith]

So it might contain mangled messages?

[carmine token]

no, the commit fails

[prime wraith]

So same idea then

Or well

I see what u mean actually

[carmine token]

No, the commit in the outer context fails. the NMI can log just fine

[prime wraith]

But in this case the buffer is owned by the ring so commit is basically a cmpxchg or store

[carmine token]

that's also true for us

but the NMI detects whether this store has already happened or not

if yes, all is good. if no, it moves the outer frame IP to a failure path

[prime wraith]

Yeah but how do u make an interruped writer detect whether it can dirty the buffer

[prime wraith]

Oh hmm

So u abort live writers if needed?

[carmine token]

yes

[prime wraith]

Ah well ig that works

[prime wraith]

now that i think about it, this works for you because your rings are per-cpu right?

[carmine token]

it's definitely made easier by that, yes

[prime wraith]

so u must have some console keep the records in a separate heap buffer when they're pushed out?

[carmine token]

it would probably be possible to do it with a shared ring but the implementation complexity is not worth using a shared ring imho

[carmine token]

Hm?

[prime wraith]

idk how managarms /dev/kmesg works

how does it collect all records

from all cpus

[carmine token]

there is a separate unified ring for /dev/kmesg

that is not lockless

[prime wraith]

ah ok

[carmine token]

(it doesn't have to be / would not gain anything from being lockless)

[prime wraith]

and that collects all dumped stuff from all rings?

[carmine token]

yes

[prime wraith]

yeah thats what i was asking basically

for a shared one its doable ofc but sounds ridiculously complex

[prime wraith]

i wish C had a feature where u could forward declare a struct and give it a size

so the caller can stack allocate it and not have access to its fields

thats not possible in C++ also

well ig it has private: but u still leak everything via the header

like this log ring thing will have a billion internal structures i must leak to every translation unit that wants to use it

[sterile kayak]

kinda impossible to not put it in the header because the size depends on the fields

[prime wraith]

yeah

well u have modules these days ig

although nothing supported them last time i checked

[latent geode]

most tools do now

mostly

[prime wraith]

and it would ice if u look at it wrong

[prime wraith]

does cmake support it these days?

[latent geode]

yes

for some time now

[prime wraith]

thats cool ig

[latent geode]

can't you just export the size and just use void* or similar for public api

[prime wraith]

it needs caller-allocated structures for most things

so the structures must be exported to the caller

and they reference internal structures so basically the entire implementation is leaked

[latent geode]

hmm yeah modules solve that :P

[prime wraith]

yeah ig

[latent geode]

gcc was last to support modules

[prime wraith]

its crazy how long this is taking

[latent geode]

yeah

only major build system that I know which doesn't support modules is meson

sadly

[prime wraith]

lol

thats surprising

[ornate jasper]

separate decl with an array of bytes in it?

[prime wraith]

that will cause conflicts because of multiple types with the same name

[ornate jasper]

oh yeah rip

tbh if you can't trust a caller with the fields you shouldn't trust it to allocate anyway so it's probably fine to just expose stuff

[prime wraith]

its not really about trust, just pollution and implementation details that make it really difficult to read headers

[ornate jasper]

ig

[prime wraith]

but fair

[ornate jasper]

that part can be solved by organizing the code in a more readable way and/or by declaring the internals in a sep header included in the main one

[prime wraith]

yeah sorta

thanks clion

thats exactly what i wanted

[twilit geode]

irqflags ultra following astral codestyle real???

[prime wraith]

yup

its the best codestyle afterall

[ornate jasper]

lmao that shit is why I have the full line completion ai crap turned off

[prime wraith]

usually its really good for completing patterns

but for more complex stuff it produces bullshit like this

[sterile kayak]

this function is the bane of my existence (impossible to correctly add cfi annotations to)

[prime wraith]

why is that?

[sterile kayak]

on x86 the only way to implement it involves pushf

and you don't know whether the compiler picked rsp or rbp as cfi register

for rsp you need to adjust cfi offset but for rbp you don't

[prime wraith]

yeah if u wanna have cfi annotation you must use an assembly stub

[prime wraith]

you could write bytecode to do it conditionally probably

on one hand i wanna have cfi for it, but on the other this is a hot function and id rather have it inlined in debug

[sterile kayak]

in my latest rewrite (which is unpublished because it's pretty much just a hello world so far) i solved this by just designing it in a way that that function isn't necessary

[prime wraith]

so how do u solve it exactly?

also why a new rewrite lmfao

anyway i think it can definitely be solved in dwarf bytecode tbh

i might look into it since thats literally the backbone of my stack unwinding even without gdb

[sterile kayak]

interrupts are managed via irql (lazily; when an interrupt arrives while they should be disabled it sets the bit corresponding to the vector in a bit mask and sets an irql based software interrupt pending that runs the handlers, clears rflags.if in the IRQ frame, and returns immediately before doing anything else. this also makes it possible to mask nmis)

and I'm doing a new rewrite because I want to be able to run on 32 bit platforms (so no hhdm)

[prime wraith]

wdym by mask nmis?

[sterile kayak]

if you raise to IRQL_NMI, and an nmi comes in, all it'll do is set a software interrupt pending for that IRQL, which won't trigger until lowered

so you can effectively mask nmis

[prime wraith]

oh u mean just defer it in software

[sterile kayak]

kinda defeats the point of nmis but still

[prime wraith]

you still have to handle edge cases with it ig

[sterile kayak]

not really because for stuff that needs to be nmi safe you can just raise to irql_nmi instead of irql_hwirq

[prime wraith]

btw why?

[sterile kayak]

I want to be able to port it to i486 and xr17032

no real underlying reason beyond that

[prime wraith]

ohh

thats definitely interesting

[carmine token]

that only works if you don't use iret to exit the NMI though

and then you need extra precaution on your other code paths that can iret if you have any

[sterile kayak]

nmis are edge triggered

removing the cpu level mask is no problem

[carmine token]

not on all archs

even on x86 you can easily get an NMI storm if you don't mask the source

it's true that other archs don't have the iret problem though

[thorny kelp]

copilot is always on meth

[prime wraith]

static enum descriptor_state log_descriptor_state(reg_t control, reg_t id)
{
    /*
     * Race lost, the state is not relevant because the id is no longer what we
     * expect it to be.
     */
    if (DESC_ID(control) != id)
        return DESCRIPTOR_STATE_LOST;

    return DESC_STATE(control);
}

static enum descriptor_state log_descriptor_acquire(
    struct log_descriptor_ring *desc_ring, reg_t id,
    struct log_descriptor *out_desc, reg_t *out_seq
)
{
    struct log_descriptor *desc;
    struct log_info_record *info;
    enum descriptor_state state;
    reg_t control;

    desc = log_descriptor_from_id(desc_ring, id);
    info = log_info_record_from_id(desc_ring, id);

    control = atomic_load_acquire(&desc->control);

    state = log_descriptor_state(control, id);
    if (state == DESCRIPTOR_STATE_LOST || state == DESCRIPTOR_STATE_RESERVED)
        goto out;

    if (out_desc)
        memcpy(out_desc->position, desc->position, sizeof(desc->position));

    if (out_seq)
        *out_seq = info->seq_num;

    control = atomic_load_relaxed(&desc->control);
    if (out_desc) {
        /*
         * Do a sequentially consistent load here to prevent the memcpy above
         * from being reordered against this load.
         */
        atomic_thread_fence(MO_SEQ_CST);
    } else {
        atomic_thread_fence(MO_ACQUIRE);
    }

    state = log_descriptor_state(control, id);

out:
    if (out_desc)
        out_desc->control = control;
    return state;
}

Is this good in terms of atomic stuff? @carmine token @loud radish

[carmine token]

I can take a look later today

[prime wraith]

thanks

[loud radish]

dont use fences

[prime wraith]

why not?

[loud radish]

they are slow

very

use an acquire load

[prime wraith]

then memcpy can be reordered to after the acquire

[loud radish]

or a seqcst load

[prime wraith]

so it can copy stale stuff

[prime wraith]

so ig just change fences to direct loads?

[loud radish]

yes

id just always do seqcst

[prime wraith]

maybe

its just not needed for some code paths

[loud radish]

it doesnt come with a crazy penalty iirc

[prime wraith]

maybe on x86

but perhaps yeah

[loud radish]

i would want to see the rest of the code

[prime wraith]

me too

[loud radish]

lol

[prime wraith]

ill see how codegen changes with if seq_cst else acquire vs seq_cst

once im done

[loud radish]

it gets worse

[prime wraith]

u mean in the former case?

[loud radish]

yes

ofc

[prime wraith]

did u actually look?

[carmine token]

seqcst is more expensive than acq/rel on pretty much all archs

[loud radish]

more than a likely mispredicted branch?

also, not for loads on arm in at least some cases

[carmine token]

hard to tell, depends

[loud radish]

yep

i think the cost of a mispredict is higher but idk its just vibes

[carmine token]

but it translates to a lock xchg on x86 and a dmb ish on aarch64

[loud radish]

but it also makes codegen and scheduling worse

[loud radish]

it translates to ldar on arm

or ldapr for acquire if you have it

?asm cclang_trunk -O3 ```c
int x(_Atomic(int)* p) { return *p; }

hmm

;asm cclang_trunk -O3 ```c
int x(_Atomic(int)* p) { return *p; }

[crimson wagon]

Assembly Output

x:
  mov eax, dword ptr [rdi]
  ret

pitust | 84ms | cclang_trunk | x86-64 clang (trunk) | godbolt.org

[prime wraith]

lol

[loud radish]

uh

[prime wraith]

;asm cclang_trunk -O3

int x(int* p) { return __atomic_load_n(p, __ATOMIC_SEQ_CST); }

[crimson wagon]

Assembly Output

x:
  mov eax, dword ptr [rdi]
  ret

8infy | 78ms | cclang_trunk | x86-64 clang (trunk) | godbolt.org

[prime wraith]

yeah idk

[carmine token]

i mean sure, that's the load side but the seqcst store side needs a strong barrier

aka lock xchg

[vestal zealot]

AFAIK x86 loads don't need barriers but the stores do in these cases

;asm cclang_trunk -O3

void a(int *p, int v) { __atomic_store_n(p, v, __ATOMIC_SEQ_CST); }

[prime wraith]

;asm cclang_trunk -O3

void x(int* p) { __atomic_store_n(p, 1, __ATOMIC_SEQ_CST); }

[crimson wagon]

Critical error:

You are sending requests too fast!

Requested by: 8infy

Assembly Output

a:
  xchg dword ptr [rdi], esi
  ret

robotman2412 | 65ms | cclang_trunk | x86-64 clang (trunk) | godbolt.org

[carmine token]

in TSO the only possible reordering is that loads that succeed stores in program order can be reordered

[prime wraith]

im doing a load so i guess ill just do seq_cst unconditionally

[vestal zealot]

What are you building, anyway?

[carmine token]

If you do a seqcst load on the reader side, you also need a seqcst store at the writer side

[prime wraith]

#1385970208631427173 message

a log ring

[prime wraith]

why?

[carmine token]

Otherwise it's no stronger than a load acquire

Because all seqcst operations are globally ordered but not all operations between the seqcst ops

[prime wraith]

anyway take a look at the code i posted when u can and let me know what the best way to express what i want is

because idk

[vestal zealot]

BTW reordered against what load specifically? I only see a store after the atomic_thread_fence

[prime wraith]

memcpy above

[prime wraith]

oh load above this if lol

[vestal zealot]

Also IDK what type desc->position actually is, but did you mean to take the pointer of it in the memcpy? This would only be a valid copy if it's an array type (seems unlikely for something called position).

[prime wraith]

yeah idk why clion isnt complaning, it should be a pointer ofc

[vestal zealot]

You will need to move the barrier between the two for it to be guaranteed to work

[prime wraith]

🤷‍♂️

i think i also found a bug in the linux log ring which will cause it to retry indefinitely because of a bogus if check

[vestal zealot]

Send patches to them

[prime wraith]

yeah i might

since im stealing their algorithm basically verbatim

just translating it to the C memory model

https://elixir.bootlin.com/linux/v6.16.3/source/kernel/printk/printk_ringbuffer.c#L787 basically here the check is bogus

since tail_id is already the previous wrap id value

it basically unwinds it by two wraps here

so this check will never succeed

[vestal zealot]

lol

[prime wraith]

ill send a patch i think, at least if im wrong ill know why

but it definitely shouldnt be possible for the ring to ever have ids two generations behind

[carmine token]

^ this is correct. if you want to order the load_relaxed vs. the memcpy(), you need a fence between the memcpy() and the load_relaxed()

that said, we'd need to see the other side of this code to check if the barrires are correct

if you want to ensure that the atomic_load_relaxed(&desc->control); is at least as new as the data read by the memcpy, you should put an acquire fence between the memcpy and the atomic_load_relaxed(&desc->control);

note that this also requires you to use a release store (or barrier) when writing desc->control

[prime wraith]

But how does an acquire barrier help to prevent memcpy from being reordered after the acquire load

Doesn't it only ensure that whatever is after happens after

[carmine token]

Acquire doesn't allow preceding loads to be moved across the barrier

It's roughly equivalent to an (r, rw) barrier

at least if paired with a suitable release barrier on the writer side

[prime wraith]

Can you expand on this

I'm confused again

I thought acquire just made sure that stuff that follows cant be reordered to before the barrier

[dusky kraken]

we should go back to single-core in-order cores ts got too complicated

[twilit geode]

one core no multithreading anything else is mental illness

wanted to do more things in parallel? we had a tool for that

it was called BUY MORE COMPUTERS

[carmine token]

Barriers always order some access before the barrier vs. some accesses after the barrier

Intuitively, you can either think of acquire as: everything after the barrier is at least as new as everything read before the barrier

or as: no read before the barrier can be recorded against any read or write after the barrier

[prime wraith]

what about writes before the barrier

also, does this apply to a normal acquire-load without a thread fence?

btw does this mean you were wrong here #1217009725711847465 message?

[carmine token]

yes

[carmine token]

writes before the barrier can be moved past the acquire

[carmine token]

uhm depends on what you meant by the post back then

acquire is strong enough to keep the loads done by the memcpy() before the barrier

but not strong enough to keep the stores done by the memcpy() before the barrier

[prime wraith]

i mean i dont care when the stores complete

i just want the content to be up to date

[carmine token]

then acquire is enough

[prime wraith]

hmm thanks

[carmine token]

but if you have an acquire fence on the load side you need a release fence at the store side

[prime wraith]

the store side does an acq_rel cmpxchg i think

[prime wraith]

how would you word this for a release barrier?

[carmine token]

(rw, w)

[prime wraith]

so loads after a release store can be reordered to before?

or well, loads before a release store ig

nvm i get it i think

so if i have

X = Y
acquire(Z)

Y is guaranteed to be loaded but the store to X is not guaranteed to be completed yet, right @carmine token ?

[carmine token]

correct

that's btw also the only kind of reordering that can happen on x86

[prime wraith]

ah

and for release

A = B
release(C)
D = E

A = B is guaranteed to be completed before a C release and a write to D is guaranteed to finish after but E might've already been loaded?

[carmine token]

yes

[prime wraith]

makes sense

[clever pewter]

I’ve already done this 😭😭

@prime wraith nice progress

[prime wraith]

Thanks but there's very little so far

[clever pewter]

You’ve got more than me and I’ve been at ts for almost a year

[prime wraith]

what do u have atm

[clever pewter]

Scheduler(smp), synchronization primitives, full memory shit (vmm, pmm, page cache, heap, the works), all the cpushit in the world, userspace, a few syscalls, vfs, and a tmpfs with refcounting

But underwhelming

Still

[prime wraith]

you might be confusing me with someone because i have literally none of that

[clever pewter]

UltraLegacy?

Has more than that

[prime wraith]

ah that one

well who cares about it

[clever pewter]

I did 😢

[prime wraith]

[clever pewter]

I mean it has 50 stars and I look through it sometimes

[prime wraith]

i had no idea what i was doing, but i guess it more or less worked at least

[latent geode]

what is this emoji 💀

[latent geode]

r/osdev has taught me that stars mean nothing

[clever pewter]

It’s fucking hilarious

[prime wraith]

from zig server iirc

[clever pewter]

True actually

[latent geode]

and my first kernel actually
it has 150 stars and is worse than gdt init ok level kernels

ring 0 shell and everything

[prime wraith]

lol

[latent geode]

the only reason it got so many stars is that repo is literally called "kernel"

[clever pewter]

I mean I’m looking at it rn and it seems to be kinda an average hobby os

But yeah no ring3 is a bit diabolical

[latent geode]

yeah I should just private that repo

[clever pewter]

I mean it’s not that bad man I promise

This code isn’t atrocious

[latent geode]

hmmm

I didn't even remember that it had a scheduler

[clever pewter]

It has elf loading in it too

[latent geode]

probably copied from somewhere

[clever pewter]

https://tenor.com/view/truth-nuke-truth-true-soyjak-wojak-gif-14828627546768133163

[latent geode]

wow ipv4 and some net drivers too

my memory truly is shite

[prime wraith]

lol

inb4 better than linux

[twilit geode]

astralcodestyle

[uneven abyss]

🗣️🗣️🗣️

[thorny kelp]

not that bad

look at https://github.com/9xbt/bentobox/blob/main/kernel/misc/elf64.c#L394

[prime wraith]

Do u also do Linux compressed ksyms?

[thorny kelp]

nope

[prime wraith]

Ah

Thought u did from the names

[thorny kelp]

ah

[prime wraith]

im basically done with log ring core, ended up as 700 loc, im gonna do tests right now but i doubt its gonna work first try, theres a lot of state checks where i have no idea if its possible or not

and its kinda messy so tons of clean up remaining

also support for descriptor re-open for pr_cont

[prime wraith]

i have a headache from this shit

[twilit geode]

Take a break

[prime wraith]

yeah i should lol

the biggest annoying thing is you want the reader to have a view of contiguous increasing sequence number for every log message, but internally some bits are consumed by state so u dont have the full 64-bit number

so u must have the logic of mapping internal descriptor ids to this sequence number

which is annoying af

especially to bootstrap in an empty ring

[carmine token]

why do sequence numbers have to be contiguous?

[prime wraith]

so consoles or /dev/kmsg readers can catch up at any point in time

they know their sequence number and the log ring knows the tail sequence number

this can also be used to calculate the number of dropped records etc

[carmine token]

are you trying to match linux ABI?

[prime wraith]

this just seems like the sane thing to do

abi with what exaclty? like the linux c files? no lol

[carmine token]

for /dev/kmsg or whatever

[prime wraith]

dev kmsg will have the same format yeah

how else do u implement dev ksmg if u dont have any sort of sequence numbers?

you need some sort of an iterator one way or another

for aba issues (ig less likely on 64-bit)

like a reader can open the file, then read first two lines, then sleep for an hour, then try reading it again

and you dont know whether line 2 maps to message 2 in the ring or if it has wrapped multiple times since then

even if you dont do kmsg directly, but any other way to expose kernel logs to userspace

[ember reef]

how long have you been working on this log

[prime wraith]

maybe 12 hours combined

but a lot of prior reading

[ember reef]

do you actually need this insane log

[prime wraith]

well define need ig lol

It allows me to basically

• Log as early as the first line of kernel main, but dump messages as soon as a console is attached with nothing lost
• Log from NMI/irq handlers and not worry about deadlocking
• Not worry about allocating temporary printf buffers on the stack
• Deferred console output via a separate thread
• Easily implement /dev/kmsg/syslog later

[carmine token]

well for /dev/kmsg we match linux ABI because Managarm's posix has source level compat with linux

but for our internal rings we don't have contiguous sequence numbers

we have a lot of ring buffers that the kernel exposes

for stuff like kernel allocation tracing, generic os level tracing, kernel profiling etc.

[prime wraith]

how do u solve this problem?

[carmine token]

for these rings we use the running byte offset of the message instead of sequence numbers

where running byte offset = byte offset of the message if the ring was infinite in size

[prime wraith]

yeah nvm i keep forgetting your final ring is lockful so u can afford to do anything

[carmine token]

this also works for lockless rings

we also do that for lockless rings

[prime wraith]

yeah u can use the byte offset as id sure

[carmine token]

also, our final ring is lockless for readers, only uses locks for writers

readers have no kernel-side state

[prime wraith]

they keep the byte offset no?

[carmine token]

yes, but that's in user space

[prime wraith]

ah

[carmine token]

they remember the running byte offset in userspace

[prime wraith]

yeah i see

makes sense

[carmine token]

our dequeue API is:

    frg::tuple<bool, uint64_t, uint64_t, size_t>
    dequeueAt(uint64_t deqPtr, void *data, size_t maxSize) {

[prime wraith]

what about metadata and stuff?

or do u not have any

[carmine token]

where the return value is a 4 tuple (success, offset of dequeued message, offset of next message, size of dequeued message)

[prime wraith]

thats decent yeah

[carmine token]

if the ring has wrapped around compared to the input deqPtr, the user space can tell that this happens because it can compare deqPtr to the return value

the meta data is part of the bytes in the buffer

[prime wraith]

so it really dequeues some sort of a variable length struct?

[carmine token]

well yeah but the caller has to know a max size

or it has to retry if it failed to deque everything

[prime wraith]

yeah this is the same as the linux api

well except the metadata is a separate ring

[carmine token]

why does linux do that?

[prime wraith]

previously it was stored as part of the descriptor (the thing that stores the id + offset of text in the text ring) but it was moved to a separate ring to keep the descriptor structure light since its often allocated on the stack to memcpy locally

as for why not inline as part of text, dunno probably just convenience

it could be stored inline as text as well

like there isnt an architectural reason

[ornate jasper]

if/when you get this done, is it ok if I port this log ring to zig for imaginarium? still not sure it's what I'll actually go with for my logging but even if I don't id like to try porting it anyway tbh

[prime wraith]

Of course

[prime wraith]

after 9999 bugfixes, it shows signs of life

multithreaded tests are gonna be fun thats for sure

[prime wraith]

And I can actually see my logs from before the earlycon was even registered!

[twilit geode]

infy fellow wsl enjoyer

[prime wraith]

yep

[wide ether]

damn

[prime wraith]

i actually hate the api atm, gonna have to change it

its too easy to misuse

[carmine token]

it's from linux, of course it's pain

[prime wraith]

yeah never attempt to recreate linux api

[wide ether]

uLinux

[prime wraith]

Noticed another semi-bogus thing in the linux ringbuffer

made this patch

will see if they accept it

[twilit geode]

infy becoming a real kernel dev

[prime wraith]

nah lol i already have a bunch of patches upstreamed lol

even from my acpi days

[clever pewter]

Wtffff

https://tenor.com/view/giga-chad-gigachad-big-gif-21053844

[prime wraith]

linux:

and qemu:

[twilit geode]

crazy

the only notable project I ever contributed to was mlibc

[prime wraith]

its easy once u start closely looking at something

u usually notice a bunch of things that have not been tested very well

[ornate jasper]

multimc is the only semi-big thing I've ever contributed to. did enough that I'm credited in the program though so made up for the relative small project with amount of contributions

[prime wraith]

thats cool

[ornate jasper]

and then I retired from the project so now I just get pinged constantly in discord by randos who refuse to do their own troubleshooting and for whenever someone needs moderating and it's made me jaded af lmao

[prime wraith]

lol

[prime wraith]

anyway this patch seems to work, unless im absolutely fucking retarded, this is an off-by-one bug they've had for 5 years

like it doesnt matter that much, but it makes it so if u have a record of 32 bytes and 32 bytes left at the end of the data ring, it would discard whatever is at the tail and put data there

[prime wraith]

with this patch:

infy@DESKTOP-IGT40G0:~/linux$ ./run.sh
** 118 printk messages dropped **
[    0.000000] RCU Tasks: Setting shift to 1 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=2.

without:

infy@DESKTOP-IGT40G0:~/linux$ ./run.sh
** 119 printk messages dropped **
[    0.186687] RCU Tasks: Setting shift to 1 and lim to 1 rcu_task_cb_adjust=1 rcu_task_cpu_ids=2.

even linux boot with a small ring repros the issue

[proper tulip]

based

great work as always

[prime wraith]

lol thanks

inb4 i get told by maintainers that im retarded

[wet quail]

could you pin link to github?

[prime wraith]

To what?

[latent geode]

linux doesn't accept patches with github prs

[stuck agate]

maybe you'll get your own linus rant lol

[ember reef]

linus tends to rant at big experienced maintainer of big thing because he expects a lot from them

doubt hed rant at little guy infy

no offense infy

[stuck agate]

it was a joke

[ember reef]

which i used as an opportunity to exposit random information

see how that work

love expositing

[wet quail]

????

[wet quail]

Or whatever remote you host Ultra on

[stuck agate]

https://github.com/UltraOS/Ultra

[latent geode]

what?

[wet quail]

thanks

[latent geode]

oh I thought you meant a pr for linux lol

didn't quite read it right

[wet quail]

how would I possibly mean a pr for linux

i did not mention a pull request nor linux

[latent geode]

the conversation before was about infy submitting a patch to linux

and you mentioned a github link

[wet quail]

oh

i did not notice that

[prime wraith]

Lol

[prime wraith]

He mostly rants at maintainers submitting pull requests, aka patches that have already been accepted/made by them and that they send to Linus for application to the master branch but yeah

[prime wraith]

multithreaded tests seem to work!

(gdb) p g_reader_contexts
$1 = {{ring = 0x5555555e0da0 <local_ring>, messages_read = 89679, num_dropped = 193499404}, {
    ring = 0x5555555e0da0 <local_ring>, messages_read = 79985, num_dropped = 181743381}, {
    ring = 0x5555555e0da0 <local_ring>, messages_read = 77839, num_dropped = 185027650}, {
    ring = 0x5555555e0da0 <local_ring>, messages_read = 74956, num_dropped = 192309535}}
(gdb) p g_writer_contexts
$2 = {{ring = 0x5555555e0da0 <local_ring>, id = 0, num_posted = 315197, num_failed = 1766}, {
    ring = 0x5555555e0da0 <local_ring>, id = 1, num_posted = 397550, num_failed = 1816}, {
    ring = 0x5555555e0da0 <local_ring>, id = 2, num_posted = 351502, num_failed = 1870}, {
    ring = 0x5555555e0da0 <local_ring>, id = 3, num_posted = 370492, num_failed = 1600}}

these are the stats after 5 seconds

im a bit confused as to why writers sometimes fail, but maybe it makes sense

ill have to think about it

num_dropped is bogus here because i forgot to update seqnum in the reader

[carmine token]

under what conditions can writers fail in your / Linux's design?

[prime wraith]

if the tail is still reserved on wrap

[carmine token]

ah

[prime wraith]

Reader 0: read: 2879148, dropped: 5460263
Reader 1: read: 2929813, dropped: 5460332
Reader 2: read: 2659324, dropped: 5460332
Reader 3: read: 2766961, dropped: 5460197
Writer 0: posted: 1334402, failed: 0
Writer 1: posted: 1392401, failed: 0
Writer 2: posted: 1361430, failed: 0
Writer 3: posted: 1378942, failed: 0

Hmm okay if i increase the number of descriptors it stops failing

maybe there really is contention there

1 << 7 descriptors works, 1 << 6 doesnt

actually, let me check which place the enospc comes from

there's only a few

[prime wraith]

yeah no something is off

[tacit scroll]

are you unit testing "irq/nmi" safety?

[prime wraith]

well sort of, just that the ring can handle concurrent writers and readers

[tacit scroll]

like using signals to test it?

[prime wraith]

just pthread_create

[prime wraith]

or maybe not actually, i think this happens because a writer gets preempted so it blocks all other writers, which is obviously not going to happen on bare metal since it disables interrupts for that window

[carmine token]

that's probably a good idea

since reentrancy is different from multi threaded execution

[tacit scroll]

you can use something like setitimer

[prime wraith]

yeah i should add signals as well

right now i just want it to work how i expect

[tacit scroll]

then you can have a threadlocal simulated irq flag to just return from the signal handler if irqs are disabled or continue if you decide to test nmi safety

[prime wraith]

well thats the thing irqs being disabled doesnt affect NMIs

so thats probably not super useful

[tacit scroll]

no i mean do something like

threadlocal irq_state;

void signal_handler() {
    // NMI
    if (rand() % ...) {
        write something to log from nmi
    } else {
        if (irq_state == IRQ_OFF)
            return
        write something to log
    }
}

[prime wraith]

yeah why not

So i think the failures stop when the ring runs out of descriptor faster than it runs out of text

when i lie about the average message size and make it small, it ends up dropping a ton since there's a lot of descriptors that quickly cause the text ring to fill up without anything being recycled

also my fans go insane if i cause text ring contention lmfao

the log message i use to test is about 83 bytes long, so 92 bytes aligned, so 1 << 7

this causes 0 writer drops

reducing it to 1<<6 causes about 7% to be dropped, sometimes more

• cpu fans blowing up

[prime wraith]

yeah so basically what i've learned about this is text ring contention is basically deadly on this ring

so you better get descriptor to text ratio right

otherwise you risk dropping stuff

seems like adding a few retires in the text allocator kinda solves it but

im not sure its worth it

or maybe it is

[loud radish]

have you tried using rr for race finding?

[prime wraith]

the race that happens is basically

1. Thread 0 reads the text data tail
2. Thread 1 reads the text data tail
3. Thread 1 reads the owner of text data and sees that its a published log record
4. Thread 1 advances the data tail and kills of the old log record
5. Thread 1 moves the descriptor into reserved state
6. Thread 0 reads the owner of text data and sees the freshly allocated descriptor in reserved state and assumes its actually an old descriptor thats still not published so its impossible to make progress, log record gets dropped

[prime wraith]

what is that?

the race is intentional here btw

[loud radish]

ahh okay

[prime wraith]

like this is basically the linux log ring behavior, so for some reason they didnt attempt to solve it

1 retry solves basically most cases

but ig as long as you dont overuse the promised average message bits you wont ever run into this

or maybe even an infinite amount of retries is fine as long as the tail changes

ill probably make an RFC and see what they say

if my old patch gets accepted

like almost this exact code path but for descriptor reuse is infinite retires as long as there's progress made

[prime wraith]

damn, so nice to see:

Reader 0: read: 2955259, dropped: 26629
Reader 1: read: 2936695, dropped: 45193
Reader 2: read: 2961579, dropped: 20309
Reader 3: read: 2936582, dropped: 45306
Reader 4: read: 2946065, dropped: 35823
Reader 5: read: 2948574, dropped: 33314
Reader 6: read: 2968530, dropped: 13358
Reader 7: read: 2959457, dropped: 22431
Reader 8: read: 2933810, dropped: 48078
Reader 9: read: 2935829, dropped: 46059
Reader 10: read: 2955768, dropped: 26120
Reader 11: read: 2953869, dropped: 28019
Reader 12: read: 2968500, dropped: 13388
Reader 13: read: 2969171, dropped: 12717
Reader 14: read: 2951648, dropped: 30240
Reader 15: read: 2948739, dropped: 33149
Writer 0: posted: 243741, failed: 0
Writer 1: posted: 175258, failed: 0
Writer 2: posted: 157398, failed: 0
Writer 3: posted: 208847, failed: 0
Writer 4: posted: 186275, failed: 0
Writer 5: posted: 159655, failed: 0
Writer 6: posted: 200971, failed: 0
Writer 7: posted: 148215, failed: 0
Writer 8: posted: 197405, failed: 0
Writer 9: posted: 154293, failed: 0
Writer 10: posted: 235560, failed: 0
Writer 11: posted: 162303, failed: 0
Writer 12: posted: 153949, failed: 0
Writer 13: posted: 251347, failed: 0
Writer 14: posted: 148544, failed: 0
Writer 15: posted: 221211, failed: 0

in a balanced ring even 16 concurrent writers never drop anything

i also added a ton of checks on the reader side

it sscanfs the message and cross checks it against the sequence number and thread id

all threads just post #define MSG_TEMPLATE "message from thread %u, my sequence number is %llu"

[prime wraith]

nvm that was a skill issue

or rather, the way i handled it was a skill issue

ive looked at linux and they actually do reload the tail to see if it changed

well at least i understand the algorithm much deeper now ig

[twilit geode]

Local man spends 3 months on hello world

[prime wraith]

Yup

I'm not even close to done yet btw

I need to redo the api, unfuck messy code with debug prints etc

[loud radish]

even shkwve is better than that

the fifth reimplementation that is

which is again at the point of a nonfunctional nvme driver

[prime wraith]

Thats a bootloader right

Do u have a thread btw

[ember reef]

hows that pronounced again

shakweevee

[loud radish]

shockwave

[loud radish]

yes i think so

[ember reef]

doesnt help

hows that pronounced

shuhkwavey

[loud radish]

the same as the english word "shockwave"

[ember reef]

still not getting it but whatever

[loud radish]

https://translate.google.com/?sl=en&tl=en&text=shockwave&op=translate

click this button

[ember reef]

so i was right the first time

[loud radish]

yea i guess so

yeah

[clever pewter]

💀

[prime wraith]

are you trolling lol

[ember reef]

i wouldnt trol

[prime wraith]

Yea

[ember reef]

but yeah i was

[livid talon]

"If i could i would, but i can't so i shan't".

[prime wraith]

why the hell does this SIGFPE with O3+lto

0x55555555876b <add_test_case(test_case*, char const*)+379>: divq 0x714a76(%rip) # 0x555555c6d1e8 <_Z13g_test_groupsB5cxx11+8>

nvm its the fucking global constructor order

std::unordered_map<std::string, std::vector<test_case*>>& test_groups()
{
    static std::unordered_map<std::string, std::vector<test_case*>> groups;
    return groups;
}

this solved it

interesting that LTO reversed every existing global constructor

anyway the ring works at o3 and lto

[prime wraith]

The guy who authored the log ring is reviewing my patch now

apparently they're from here https://www.linutronix.de/

[carmine token]

maybe it'd be easier to not introduce subtle bugs if they didn't overcomplicate the whole thing

[prime wraith]

lol

its funny that even the guy who made it doesnt quite understand the subtleties

[prime wraith]

so my patch got acked with a few minor comments basically

Btw the metadata ring is a huge source of wasted space and they want to collapse it to be a part of the text ring: https://lore.kernel.org/lkml/[email protected]
@carmine token just like u said lol

[prime wraith]

another maintainer lol

but also acked

[wide ether]

hey i know this guy

[prime wraith]

who is he?

anyway, v2 of the patch sent

[wide ether]

look at his email

[prime wraith]

or rather, its a series of 2 patches now

[prime wraith]

lol

[prime wraith]

yay got a reviewed-by

[prime wraith]

-fsanitize=thread seems happy about my impl now

also an interesting thing is at O0 for some reason the number of read messages is always the same

Reader 0: read: 65536, dropped: 0
Reader 1: read: 65536, dropped: 0
Reader 2: read: 65536, dropped: 0
Reader 3: read: 65536, dropped: 0
Writer 0: posted: 17581, failed: 0
Writer 1: posted: 17737, failed: 0
Writer 2: posted: 16950, failed: 0
Writer 3: posted: 16950, failed: 0

with tsan its always 65536

at O3 or without tsan its always random