uACPI - a portable and easy-to-integrate ACPI implementation | Operating System Development | Page 12

hollow elm Dec 11, 2024, 2:20 PM

#

seems to be about the same

jaunty fox Dec 11, 2024, 2:21 PM

#

almost all allocations are under 2048 bytes (the number at which i don't use the predefined slabs and instead escalate to the kernel wired memory allocator.) these should do well usually

#

oh hold on, i see one ever-escalating allocation that turns up repeatedly, ramping up to well over 2048 bytes

#

i get very inconsistent results generally between runs

#

this revision adds a 4kib slab cache

📎 barebones.iso

#

and doesn't include some sanity checks designed to check for double-free

#

and this one doesn't free anything at all, which makes a useful comparison

📎 barebones-nofree.iso

#

i find a modest improvement with the 4k slab cache added and little difference with no freeing

#

wait, these are built with -Og

rustic compass Dec 11, 2024, 2:33 PM

#

Uacpi speedrunning has been born

jaunty fox Dec 11, 2024, 2:36 PM

#

i have found a major limiting factor

#

namely the IPL management

#

@hollow elm this one should perform significantly faster

📎 barebones-lazyipl.iso

#

the build is not generally usable (it uses the lazy IPL mechanism, a mechanism i started work on which allows the issuing of writes to %cr8 to be deferred until such a time as they are actually necessary, i.e. until such a time as an interrupt exceeding the current soft IPL arrives - then it will write to %cr8 and set a flag indicating that %cr8 was raised and must fall later. this is helpful because the %cr8 write can induce a vmexit in some cases. it's broken however for edge-triggered interrupts)

fiery turtle Dec 11, 2024, 2:46 PM

#

jaunty fox <@183510341894930432> this one should perform significantly faster

interesting

#

how does this one perform on your machine

#

and the other ones?

jaunty fox Dec 11, 2024, 2:48 PM

#

my final submission, this one includes a faster memcpy and memset too

📎 barebones-lazyipl-fastmemcpy.iso

#

this one is consistently around 1.4m/s

fiery turtle Dec 11, 2024, 2:49 PM

#

jaunty fox this one is consistently around 1.4m/s

Daamn

#

Getting close

jaunty fox Dec 11, 2024, 2:49 PM

#

before any changes, let me see what i was getting

#

before changes, [uacpi]: successfully loaded 1 AML blob, 1726 ops in 4ms (avg 370306/s)

#

so the improvement is very large indeed

#

unfortunately a huge chunk of the improvement is the lazy IPL

fiery turtle Dec 11, 2024, 2:51 PM

#

3.5 times

jaunty fox Dec 11, 2024, 2:52 PM

#

which is sadly broken and i fear would require significant work to fix

#

and to become nontrivial in implementation

fiery turtle Dec 11, 2024, 2:52 PM

#

Is it difficult to implement properly?

jaunty fox Dec 11, 2024, 2:52 PM

#

fiery turtle Is it difficult to implement properly?

the problem is edge triggered interrupts

#

the basic idea is this: instead of strictly maintaining %cr8 (alias of the upper 4 bits of the lapic's TPR), a value in memory is maintained instead, and if an interrupt arrives which based on the value in memory should have been masked, then %cr8 is written to with the value in memory

#

and a bool is set to indicate that this happened

#

and splx() (which lowers IPL and is paired with spl raises) will test that bool, and if it's true, it will write the new lower value to %cr8

#

and then your level-triggered interrupts which were previously masked, they'll come through now

#

but the edge triggered ones won't

hollow elm Dec 11, 2024, 2:55 PM

#

jaunty fox my final submission, this one includes a faster memcpy and memset too

[uacpi]: successfully loaded 1 AML blob, 1726 ops in 1ms (avg 1542020/s)

jaunty fox Dec 11, 2024, 2:56 PM

#

hollow elm ``` [uacpi]: successfully loaded 1 AML blob, 1726 ops in 1ms (avg 1542020/s) ```

fantastic, this is what i hoped for, cheers

hollow elm Dec 11, 2024, 2:56 PM

#

jaunty fox <@183510341894930432> this one should perform significantly faster

out of curiosity: ```
[uacpi]: successfully loaded 1 AML blob, 1726 ops in 2ms (avg 735246/s)

jaunty fox Dec 11, 2024, 2:57 PM

#

it really is down in large part to having a better memcpy and memset than the trivial implementation, then

#

and writing to %cr8 really is slow under VMs, slow enough that i should try to get away from it

hollow elm Dec 11, 2024, 2:57 PM

#

i guess it's always a vmexit

jaunty fox Dec 11, 2024, 2:58 PM

#

i saw reference to intel APICv which should help, but i don't know how new it is

#

presumably not old/widespread enough to get the benefit

hollow elm Dec 11, 2024, 2:58 PM

#

does qemu use apic virtualization?

jaunty fox Dec 11, 2024, 2:58 PM

#

and it looks like you need to opt in to TPR shadow

jaunty fox Dec 11, 2024, 3:00 PM

#

hollow elm does qemu use apic virtualization?

i would think so, it looks to be an old feature, i'll have a look

jaunty fox Dec 11, 2024, 3:15 PM

#

for the most common case (IPL manipulations between 0 and dispatcher level) i can avoid touching %cr8 at all

#

now i can consistently reach around 1.4m/s which is fine

fiery turtle Dec 11, 2024, 4:19 PM

#

jaunty fox the basic idea is this: instead of strictly maintaining %cr8 (alias of the upper...

interesting

fiery turtle Dec 11, 2024, 4:19 PM

#

jaunty fox now i can consistently reach around 1.4m/s which is fine

big!

north holly Dec 11, 2024, 4:27 PM

#

jaunty fox now i can consistently reach around 1.4m/s which is fine

Well I'll be damned

torpid root Dec 11, 2024, 5:20 PM

#

infy you have made uacpi a benchmark

#

as well as allocator debugger

fiery turtle Dec 11, 2024, 5:44 PM

#

lmao

loud ice Dec 11, 2024, 5:53 PM

#

@fiery turtle you should probably make use of a microsecond-precise timer for measuring uacpi perf

#

to reduce variance

fiery turtle Dec 11, 2024, 6:00 PM

#

hm?

#

i dont use any specific timers, whatever the kernel returns

#

under the hood it could be using us converted to ns for the purposes of that api

#

the main use has been the Timer opcode in AML which expects 100ns ticks

#

oh did u think it was using ms internally? its nanoseconds converted to ms

#

(for printf specifically)

loud ice Dec 11, 2024, 6:17 PM

#

ah i see

#

interesting ig

fiery turtle Dec 11, 2024, 6:19 PM

#

yeah i just didnt want the log to be way too long

loud ice Dec 11, 2024, 6:22 PM

#

fair enough

vale isle Dec 11, 2024, 6:25 PM

#

heyy

#

it's been some time since i checked this thread out

#

what major things have happened in the last 2 months?

#

there was some work on reactos i remember?

fiery turtle Dec 11, 2024, 6:25 PM

#

thread safety, lots of new api and helpers, opaque objects

vale isle Dec 11, 2024, 6:26 PM

#

is 1.0 here

fiery turtle Dec 11, 2024, 6:26 PM

#

vale isle there was some work on reactos i remember?

havent heard from that person yet

fiery turtle Dec 11, 2024, 6:26 PM

#

vale isle is 1.0 here

nope

#

but getting closer

vale isle Dec 11, 2024, 6:26 PM

#

ah ok cool

#

life is hard

fiery turtle Dec 11, 2024, 6:26 PM

#

yes

vale isle Dec 11, 2024, 6:27 PM

#

i hate the fact that i need to study this hard to get into a university to slack off for 4 years and then struggle to find a job anyway

fiery turtle Dec 11, 2024, 6:27 PM

#

they had an almost ready pr but then something happened and they disappeared idk

vale isle Dec 11, 2024, 6:27 PM

#

vale isle i hate the fact that i need to study this hard to get into a university to slack...

we have central university entrance exam

#

which means

#

many people who otherwise aren't interested in CS just pick CS because they can and "they don't want to waste their ranking"

#

medicine and CS are overly inflated

#

to study in a university that is not any random "apartment university" and actually has recognition abroad

#

i need to be like

#

top 1000

#

out of 3 million people entering the exam

vale isle Dec 11, 2024, 6:31 PM

#

vale isle to study in a university that is not any random "apartment university" and actua...

universities that have recognition abroad: METU, BOUN, maybe ITU, maybe YTU and a couple private universities like Bilkent, Sabancı, Koç

#

number of universities in turkey: 209

#

if you heard anything it was probably METU (middle east technical university)

frank canopy Dec 11, 2024, 6:36 PM

#

oh wait, uacpi perf is largely allocator based? im using zig's std GeneralPurposeAllocator backed by a free-list based page allocator

#

or at least at this scale of benchmarking that is

fiery turtle Dec 11, 2024, 6:36 PM

#

the interpreter is very allocator bound yeah

frank canopy Dec 11, 2024, 6:37 PM

#

my page allocator is a free list slab thing that idk if it has its own name but i think of it as the florence pmm because i took the concept and basic impl from florence https://github.com/FlorenceOS/Florence/blob/aaa5a9e568197ad24780ec9adb421217530d4466/subprojects/flork/src/memory/pmm.zig

#

afaik its basically half of a buddy allocator but im not 100% sure what the terminology would be

fiery turtle Dec 11, 2024, 6:38 PM

#

whats your current ops/s?

frank canopy Dec 11, 2024, 6:38 PM

#

#1217009725711847465 message

#

949526 in ReleaseFast is my current best

fiery turtle Dec 11, 2024, 6:39 PM

#

ah

#

nice

frank canopy Dec 11, 2024, 6:39 PM

#

240107 in ReleaseSafe which is where im usually operating because safety checks are nice for debugging and catching issues

loud ice Dec 11, 2024, 6:40 PM

#

fiery turtle the interpreter is very allocator bound yeah

maybe you should optimize that

fiery turtle Dec 11, 2024, 6:41 PM

#

its possible with an object cache

#

since a lot of those allocations are objects

loud ice Dec 11, 2024, 6:41 PM

#

you could do escape analysis meme

fiery turtle Dec 11, 2024, 6:41 PM

#

lol

loud ice Dec 11, 2024, 6:42 PM

#

what actually allocates in uacpi?

#

like theres a bunch of stuff that kinda has to

#

but most things shouldn't?

frank canopy Dec 11, 2024, 6:44 PM

#

pooling might not be a bad thing to have in uacpi tbh since a lot of stuff is objects that presumably mostly have a known size

loud ice Dec 11, 2024, 6:45 PM

#

yeah but thats a general allocator thing

#

what do the profiles look like i wonder

#

tbh the bytecode construction doesn't feel particularly fast in general

#

if i was going for maximum performance, i'd definitely try doing a compile step so that you don't jit parse aml

frank canopy Dec 11, 2024, 6:53 PM

#

i mean rn we're comparing the initial load where the aml gets parsed, or at least thats what people are reporting throughput for

loud ice Dec 11, 2024, 6:54 PM

#

frank canopy i mean rn we're comparing the initial load where the aml gets parsed, or at leas...

okay fair

#

hm

fiery turtle Dec 11, 2024, 7:12 PM

#

loud ice what actually allocates in uacpi?

Call frames, locals, intermediate expression evaluation, temporary objects

#

Some of those could be elided probably

fiery turtle Dec 11, 2024, 7:13 PM

#

loud ice tbh the bytecode construction doesn't feel particularly fast in general

Yeah it's designed to suck

#

But its still very tiny amount of time compared to the rest of kernel init

loud ice Dec 11, 2024, 7:13 PM

#

fiery turtle Call frames, locals, intermediate expression evaluation, temporary objects

ah so basically everything got it

fiery turtle Dec 11, 2024, 7:13 PM

#

Lol

#

Basically

jaunty fox Dec 11, 2024, 7:14 PM

#

vale isle universities that have recognition abroad: METU, BOUN, maybe ITU, maybe YTU and ...

what a funny synchronicity

#

i was just reading an article by a guy at bilkent

#

http://istvanaranyosi.net/resources/Should we fear qt final.pdf this one specifically

vale isle Dec 11, 2024, 7:16 PM

#

lol

#

i love it when this happens

calm latch Dec 12, 2024, 6:52 PM

#

What am I supposed to use instead of uacpi_namespace_node_get_object?

fiery turtle Dec 12, 2024, 6:53 PM

#

purpose?

calm latch Dec 12, 2024, 6:54 PM

#

I'm trying to update uACPI

#

And I have this function

uacpi_iteration_decision find_pci_device(void *param, uacpi_namespace_node *node, uint32_t depth)
{
    struct DeviceSearchCtx *ctx = param;
    uint64_t addr = 0;
    (void)depth;

    uacpi_object *obj = uacpi_namespace_node_get_object(node);
    if (obj == NULL || uacpi_object_get_type(obj) != UACPI_OBJECT_DEVICE)
        return UACPI_ITERATION_DECISION_CONTINUE;

    uacpi_status ret = uacpi_eval_integer(node, "_ADR", UACPI_NULL, &addr);
    if (ret != UACPI_STATUS_OK && ret != UACPI_STATUS_NOT_FOUND)
        return UACPI_ITERATION_DECISION_CONTINUE;

    if (addr == ctx->addr) {
        ctx->out_node = node;
        return UACPI_ITERATION_DECISION_BREAK;
    }

    return UACPI_ITERATION_DECISION_CONTINUE;
}

fiery turtle Dec 12, 2024, 6:55 PM

#

for_each_device can now filter by type

#

or rather

#

for_each_child

#

with UACPI_OBJECT_DEVICE_BIT

#

will automatically do what u want

calm latch Dec 12, 2024, 6:57 PM

#

uacpi_for_each_device_resource?

fiery turtle Dec 12, 2024, 6:57 PM

#

uacpi_namespace_node_for_each_child

calm latch Dec 12, 2024, 6:58 PM

#

Are there examples?

fiery turtle Dec 12, 2024, 6:59 PM

#

managarm

#

osdev wiki

calm latch Dec 12, 2024, 6:59 PM

#

I wrote this code half a year ago and don't remember what it does 😅

fiery turtle Dec 12, 2024, 7:00 PM

#

looks like it finds the address of a pci device

frank canopy Dec 12, 2024, 7:00 PM

#

theres even a dedicated function for adr iirc too

#

plus a function to get the type of a node

fiery turtle Dec 12, 2024, 7:01 PM

#

for_each_child can filter by node type already but yeah

calm latch Dec 12, 2024, 7:01 PM

#

    if (class == 0x06 && subclass == 0x04) { // Host bridge
        struct DeviceSearchCtx ctx = {
            .addr = d->device << 16 | d->function,
            .out_node = NULL,
        };
        uacpi_namespace_for_each_child_simple(node, find_pci_device, &ctx);

        uint8_t secondary_bus = pci_secondary_bus(c);
        printf("PCI secondary bus %x\n", secondary_bus);

        uacpi_pci_routing_table *pci_routes;
        uacpi_status ret = uacpi_get_pci_routing_table(ctx.out_node, &pci_routes);
        if (ret != UACPI_STATUS_OK) {
            fprintf(stderr, "Warning: Could not get PCI routing for root bridge bus %0x: %i\n", bus, ret);
        } else {
            parse_interrupt_table(g, secondary_bus, pci_routes);
            uacpi_free_pci_routing_table(pci_routes);
        }

        check_bus(g, secondary_bus, d, ctx.out_node);
    }

frank canopy Dec 12, 2024, 7:01 PM

#

though you probably want to just use the filter

calm latch Dec 12, 2024, 7:01 PM

#

Wiki has this

    uacpi_namespace_for_each_child(
        uacpi_namespace_root(), acpi_init_one_device, UACPI_NULL,
        UACPI_OBJECT_DEVICE_BIT, UACPI_MAX_DEPTH_ANY, UACPI_NULL
    );

fiery turtle Dec 12, 2024, 7:02 PM

#

thats it

calm latch Dec 12, 2024, 7:02 PM

#

Thonk

#

Oh and I remove this ```c
uacpi_object *obj = uacpi_namespace_node_get_object(node);
if (obj == NULL || uacpi_object_get_type(obj) != UACPI_OBJECT_DEVICE)
return UACPI_ITERATION_DECISION_CONTINUE;

fiery turtle Dec 12, 2024, 7:05 PM

#

yup

calm latch Dec 12, 2024, 7:06 PM

#

Ok it seems to compile

#

Thanks

fiery turtle Dec 12, 2024, 7:06 PM

#

lets see your ops/s

calm latch Dec 12, 2024, 7:07 PM

#

(for now)

#

(I couldn't get it to work so I'm updating it)

#

New uacpi_kernel_log?

fiery turtle Dec 12, 2024, 7:08 PM

#

hm?

calm latch Dec 12, 2024, 7:09 PM

#

fiery turtle Dec 12, 2024, 7:09 PM

#

jesus how old is yor uacpi lol

#

u need #define UACPI_FORMATTED_LOG or whatever

calm latch Dec 12, 2024, 7:11 PM

#

Will it format itself if I don't pass that?

#

My printf is 'ok'

fiery turtle Dec 12, 2024, 7:11 PM

#

yeah

#

u dont have a good printf in userspace?

calm latch Dec 12, 2024, 7:11 PM

#

I wrote it myself

fiery turtle Dec 12, 2024, 7:11 PM

#

why not reuse existing libc

calm latch Dec 12, 2024, 7:12 PM

#

So it can't be fully trusted

#

Idk

#

I'm writing my own

fiery turtle Dec 12, 2024, 7:12 PM

#

anyway then just keep this logger

frank canopy Dec 12, 2024, 7:49 PM

#

the fact that uacpi can do its own printf format is so nice for my case since zig's format doesnt use the same format string format as c does and this way i dont need to implement it in a shim somewhere

fiery turtle Dec 12, 2024, 7:50 PM

#

yeah language interop is a big reason why it was added

calm latch Dec 12, 2024, 8:06 PM

#

Do I need UACPI_KERNEL_INITIALIZATION?

#

Atm I just call ```c
uacpi_initialize(&init_params);
uacpi_namespace_load();
uacpi_set_interrupt_model(UACPI_INTERRUPT_MODEL_IOAPIC);
// init pci
uacpi_namespace_initialize();
uacpi_finalize_gpe_initialization();

#

And uacpi_initialize isn't working...

frank canopy Dec 12, 2024, 8:08 PM

#

uacpi_initialize takes a flags bitmask now

#

if youre passing rsdp theres a kernel_api function you implement for that

frank canopy Dec 12, 2024, 8:08 PM

#

calm latch Do I need `UACPI_KERNEL_INITIALIZATION`?

pretty sure this one is for getting callbacks to the kernel but im not 100% sure

calm latch Dec 12, 2024, 8:12 PM

#

Oh uacpi_kernel_get_rsdp

fiery turtle Dec 12, 2024, 8:18 PM

#

calm latch Do I need `UACPI_KERNEL_INITIALIZATION`?

thats a question to you

calm latch Dec 12, 2024, 8:18 PM

#

What do I need to give it in uacpi_kernel_get_thread_id()?

fiery turtle Dec 12, 2024, 8:18 PM

#

pthread_self?

calm latch Dec 12, 2024, 8:19 PM

#

Ok

#

I'm getting internal error

fiery turtle Dec 12, 2024, 8:20 PM

#

mutex acquire failed?

#

perhaps

calm latch Dec 12, 2024, 8:21 PM

#

Thonk

fiery turtle Dec 12, 2024, 8:22 PM

#

thats the only path that may fail there

calm latch Dec 12, 2024, 8:22 PM

#

fiery turtle Dec 12, 2024, 8:22 PM

#

fix your nanoseconds_since_boot btw

calm latch Dec 12, 2024, 8:23 PM

#

My lapic is too slow

#

Event initialization has to do with interrupts, right?

fiery turtle Dec 12, 2024, 8:24 PM

#

well it says right there

#

unable to install interrupt handler

calm latch Dec 12, 2024, 8:25 PM

#

fiery turtle fix your nanoseconds_since_boot btw

What is the best way to implement it btw

fiery turtle Dec 12, 2024, 8:26 PM

#

kvm clock -> invariant tsc -> hpet -> ACPI PM timer -> manual tick counting

calm latch Dec 12, 2024, 8:28 PM

#

Thonk time to write another server

fiery turtle Dec 12, 2024, 8:29 PM

#

wtf

calm latch Dec 12, 2024, 8:29 PM

#

I had HPET in userspace

fiery turtle Dec 12, 2024, 8:29 PM

#

bruh imagine the latency of that

calm latch Dec 12, 2024, 8:30 PM

#

Maybe I'll add invariant tsc

#

Since hpet requires ioapic

#

Which I also manage from userspace...

fiery turtle Dec 12, 2024, 8:30 PM

#

reading hpet counter requires quite literally nothing

calm latch Dec 12, 2024, 8:30 PM

#

But it needs interrupts.....................

fiery turtle Dec 12, 2024, 8:30 PM

#

it needs them if you need them

#

which u dont

#

for interrupts u use lapic

calm latch Dec 12, 2024, 8:31 PM

#

And it has 64 bit counters

#

Ok

#

I'll add a driver for that

slim panther Dec 12, 2024, 8:31 PM

#

calm latch Since hpet requires ioapic

it does?

#

that's news

calm latch Dec 12, 2024, 8:31 PM

#

No I've been using it before for event timer

fiery turtle Dec 12, 2024, 8:32 PM

#

calm latch And it has 64 bit counters

yeah

calm latch Dec 12, 2024, 8:32 PM

#

Like for sleep()

#

And then I've switched/unified everything to use lapic after porting to risc-v which only had local timers

fiery turtle Dec 12, 2024, 8:33 PM

#

you're mixing up rescheduling timers vs current time readback

#

those are unrelated concepts

calm latch Dec 12, 2024, 8:33 PM

#

I know

#

I'm just updating a global counter at the moment

#

From lapic interrupts

#

On bsp core

#

Which are 10-100* ms

#

(*which could be anything since the kernel is nonpreemptive)

#

(and it's running in single shot mode)

#

anyways

#

I think it initialized

#

But it doesn't shut down 😦

#

fiery turtle Dec 12, 2024, 8:36 PM

#

calm latch I think it initialized

did u fix nanoseconds_since_boot btw?

calm latch Dec 12, 2024, 8:36 PM

#

No

#

I'll fix it in 10 minutes - 1 month

fiery turtle Dec 12, 2024, 8:36 PM

#

not bad

calm latch Dec 12, 2024, 8:36 PM

#

Since I have homework due today

fiery turtle Dec 12, 2024, 8:37 PM

#

as accurate as your timer readback LULW

calm latch Dec 12, 2024, 8:37 PM

#

calm latch Since I have homework due today

Which I hadn't started

calm latch Dec 12, 2024, 8:37 PM

#

fiery turtle as accurate as your timer readback <:LULW:543420553021882371>

Who needs a precise time anyway

#

+- 10 seconds accuracy is fine for most use cases

fiery turtle Dec 12, 2024, 8:38 PM

#

indeed

calm latch Dec 12, 2024, 8:39 PM

#

calm latch But it doesn't shut down 😦

How can I debug this?

#

It was working with the old version

fiery turtle Dec 12, 2024, 8:40 PM

#

well all it does to shutdown in qemu is port io

#

so debug your port io

calm latch Dec 12, 2024, 8:40 PM

#

calm latch It was working with the old version

(and 64 bit userspace and before I made a bunch of changes to everything)

#

But it did get interrupt from the power button

calm latch Dec 12, 2024, 8:41 PM

#

fiery turtle so debug your port io

So that should be working

fiery turtle Dec 12, 2024, 8:41 PM

#

add printfs

#

or at least addr2line where its stuck

#

i remember someone having a similar bug

#

i dont recall how they fixed it tho

calm latch Dec 12, 2024, 8:44 PM

#

#

It's not stuck

#

I think my sleep is broken

#

galaxybrain

fiery turtle Dec 12, 2024, 8:44 PM

#

bruh

calm latch Dec 12, 2024, 8:46 PM

#

calm latch I think my sleep is broken

Commenting sleep(3) out makes it shut down KEKW

fiery turtle Dec 12, 2024, 8:46 PM

#

maybe its trying to sleep for a month meme

calm latch Dec 12, 2024, 8:47 PM

#

Ok I think I'll finish the uni stuff and try to fix timers (in about 1 hour)

#

Or I could keep procrastinating Thonk

#

I think it was indeed sleeping for a month KEKW KEKW KEKW

#

I have been passing size_t to a function which takes uint64_t

#

And it probably got garbage in the upper 32 bits

fiery turtle Dec 12, 2024, 8:50 PM

#

lol

#

fixed upstream

fiery turtle Dec 12, 2024, 8:52 PM

#

calm latch

unrelated but why is your AHCI segfaulting lmao

calm latch Dec 12, 2024, 8:53 PM

#

My fork is broken

#

It corrupts heap

#

(I think)

#

I haven't investigates

#

And AHCI driver forks for every controller it finds

fiery turtle Dec 12, 2024, 8:54 PM

#

this kernel might be more unstable than obos

#

ultrameme

calm latch Dec 12, 2024, 8:55 PM

#

The kernel is quite stable

#

(to my standards)

#

The userspace though...

fiery turtle Dec 12, 2024, 8:55 PM

#

is fork done via userspace?

calm latch Dec 12, 2024, 8:55 PM

#

No/yes

#

My fork clones 2 threads

#

Because each process has a library threads

#

So that library thread saves the state of the forking thread, clones itself, and recreates the thread that forked

#

And that saves all registers to .data

#

Which are defined as 16 longs

fiery turtle Dec 12, 2024, 8:57 PM

#

bruh why is this so complicated

calm latch Dec 12, 2024, 8:57 PM

#

Which are 32 bit on i686

#

But I haven't added a special case for that in kernel

#

(I think this is what's happening)

calm latch Dec 12, 2024, 8:58 PM

#

fiery turtle bruh why is this so complicated

It usually works fine

calm latch Dec 12, 2024, 8:58 PM

#

calm latch But I haven't added a special case for that in kernel

Since it's 64 bit

calm latch Dec 12, 2024, 9:00 PM

#

fiery turtle bruh why is this so complicated

Because I didn't think of a good way to make kernel interrupt userspace on signals

#

So I came up with this

#

(though I didn't implement signals yet)

#

Lol even more things are failing now...

#

But it doesn't wait for a month anymore tabien

calm latch Dec 12, 2024, 9:05 PM

#

calm latch Ok I think I'll finish the uni stuff and try to fix timers (in about 1 hour)

calm latch Dec 12, 2024, 10:04 PM

#

Ok, time to fix nanoseconds_since_boot

calm latch Dec 12, 2024, 10:29 PM

#

Are there 32 bit hpet systems in existence?

fiery turtle Dec 12, 2024, 10:29 PM

#

yes

gentle peak Dec 12, 2024, 10:30 PM

#

yeah, my dev system (b550) has a 32 bit hpet

calm latch Dec 12, 2024, 10:30 PM

#

oof

#

How many seconds is that? Thonk

gentle peak Dec 12, 2024, 10:31 PM

#

I think it was 42 seconds?

#

idk I don't use hpet for anything other than tsc calibration

#

actually no I think that was a different system, this one was 5 minutes

calm latch Dec 12, 2024, 10:34 PM

#

I think I can just detect rollovers periodically

#

Wiki says TSC is unreliable

gentle peak Dec 12, 2024, 10:35 PM

#

as long as the maximum time between reads is less than the rollover time you're fine

gentle peak Dec 12, 2024, 10:35 PM

#

calm latch Wiki says TSC is unreliable

wiki must've been written in 2001

#

there's a cpuid bit that tells you whether the tsc is usable for general timekeeping and it's set even on the oldest systems I own

fiery turtle Dec 12, 2024, 10:35 PM

#

calm latch Wiki says TSC is unreliable

invariant tsc bit

calm latch Dec 12, 2024, 10:36 PM

#

And it's a couple of MSRs...

#

Ok I'll support that instead

#

It seems trivial

gentle peak Dec 12, 2024, 10:36 PM

#

tsc used to be a simple cycle counter which could change freq which is why it was unreliable, but for decades now it's been a fixed frequency

fiery turtle Dec 12, 2024, 10:37 PM

#

calm latch Ok I'll support that instead

for vms u want kvm clock

jaunty fox Dec 12, 2024, 10:38 PM

#

calm latch Wiki says TSC is unreliable

it was but nowadays you could probably get away with denying systems that don't have an invariant, constant, nonstop tsc

gentle peak Dec 12, 2024, 10:38 PM

#

calm latch It seems trivial

yeah just rdtsc, also has the advantage that you don't need actual syscalls for time stuff. to get the frequency just read both it and a known-frequency clock like hpet before and after sleeping a few milliseconds

#

on newer Intel systems and in hypervisors cpuid can tell you the freq but that's not ubiquitous yet

calm latch Dec 12, 2024, 10:40 PM

#

I'm callibrating LAPIC with PIT anyway

#

So I can measure TSC during that as well

calm latch Dec 12, 2024, 10:41 PM

#

calm latch I'm callibrating LAPIC with PIT anyway

(which needs to be changed to HPET/time base CPUID stuff but it doesn't matter)

#

Is it synchronized between cores?

#

Wiki says it isn't

#

And I haven't reached to the explanation in SDM...

#

Oh no

per-CPU differences in TSC values in a NUMA system

gentle peak Dec 12, 2024, 10:44 PM

#

important distinction there: "in a NUMA system"

calm latch Dec 12, 2024, 10:44 PM

#

So multi CPU

gentle peak Dec 12, 2024, 10:44 PM

#

I'm not sure if it's explicitly guaranteed anywhere but I've never seen a system where the CPUs in a single package have different TSC values or frequencies

gentle peak Dec 12, 2024, 10:44 PM

#

calm latch So multi CPU

NUMA != multi CPU

calm latch Dec 12, 2024, 10:45 PM

#

I have Ryzen 9, it has 2 CCDs

gentle peak Dec 12, 2024, 10:45 PM

#

Still not NUMA

calm latch Dec 12, 2024, 10:45 PM

#

But I think it's not NUMA

calm latch Dec 12, 2024, 10:46 PM

#

calm latch So multi CPU

As two sockets

gentle peak Dec 12, 2024, 10:46 PM

#

oh yeah multi socket systems are more likely to be numa

calm latch Dec 12, 2024, 10:46 PM

#

But I don't have those

gentle peak Dec 12, 2024, 10:46 PM

#

but then again you'll most likely never encounter those; the approach I take is to simply not use any CPUs that aren't in the same package as the boot CPU

calm latch Dec 12, 2024, 10:47 PM

#

gentle peak but then again you'll most likely never encounter those; the approach I take is ...

I don't think my OS is useful enough for that to happen anyways

#

So I just rdtsc on boot cpu and use it as the reference?

gentle peak Dec 12, 2024, 10:49 PM

#

that'll work on basically any consumer system yeah

calm latch Dec 12, 2024, 10:50 PM

#

The synchronization of TSC adjustment can be simplified by using the 64-bit IA32_TSC_ADJUST MSR

#

Ok that doesn't apply to me I think

#

Also I can add a mechanism to syncronize the timers

gentle peak Dec 12, 2024, 10:51 PM

#

that just helps with increasing/decreasing the tsc value by the same decrement on every cpu

#

for stuff like NTP I think?

#

or just adjusting system time in general

calm latch Dec 12, 2024, 10:54 PM

#

Hmm I think I can just replace LAPIC with TSC deadline...

gentle peak Dec 12, 2024, 10:54 PM

#

calm latch Also I can add a mechanism to syncronize the timers

I experimented a bit with tsc synchronization but from what I can tell the hardware does it for you and it's incredibly difficult to do it more accurately than the hardware does

calm latch Dec 12, 2024, 10:54 PM

#

calm latch Hmm I think I can just replace LAPIC with TSC deadline...

Although it doesn't matter

gentle peak Dec 12, 2024, 10:54 PM

#

calm latch Hmm I think I can just replace LAPIC with TSC deadline...

if it's available yeah but there's plenty of systems where it isn't

#

for example, my Ryzen 5xxx doesn't support it

calm latch Dec 12, 2024, 10:54 PM

#

gentle peak I experimented a bit with tsc synchronization but from what I can tell the hardw...

So the counters are just syncrhonized and I don't have to do anything?

gentle peak Dec 12, 2024, 10:55 PM

#

yep

calm latch Dec 12, 2024, 10:55 PM

#

gentle peak for example, my Ryzen 5xxx doesn't support it

That's what I have lol

#

And I've recently bought b550 mobo

#

Ok

#

It makes it easy (?)

#

(I need to stop saying that and keep reeding SDM)

vast kestrel Dec 12, 2024, 10:56 PM

#

technically I don't think there is any guarantee in the spec about tsc being synchronized between cores

calm latch Dec 12, 2024, 10:56 PM

#

Do I need to enable it or something?

vast kestrel Dec 12, 2024, 10:57 PM

#

Practically because it's a single timer for all cores in a single socket it is synchronized

#

Invariant/constant tsc just exists, it just means that's how the timer behaves

calm latch Dec 12, 2024, 10:58 PM

#

The invariant TSC is based on the invariant timekeeping hardware (called Always Running Timer or ART), that runs
at the core crystal clock frequency. The ratio defined by CPUID leaf 15H expresses the frequency relationship
between the ART hardware and TSC.

vast kestrel Dec 12, 2024, 10:58 PM

#

On some newer cpus you can use the cpuid to calculate it directly, on others you can't

gentle peak Dec 12, 2024, 10:58 PM

#

vast kestrel technically I don't think there is any guarantee in the spec about tsc being syn...

yeah I haven't seen a synchronization guarantee in the sdm either but every system I've tested on is synchronized

gentle peak Dec 12, 2024, 10:59 PM

#

calm latch > The invariant TSC is based on the invariant timekeeping hardware (called Alway...

that's only supported on new-ish Intel CPUs, AMD doesn't have it

vast kestrel Dec 12, 2024, 10:59 PM

#

Tbh I would dig in the lkml instead, alot of guarantees only appear there and no where else

#

And if Linux does synchronize it at boot then there is most likely a reason (which might just be numa, but that's why you wanna check why they added it)

frank canopy Dec 12, 2024, 11:01 PM

#

gentle peak that's only supported on new-ish Intel CPUs, AMD doesn't have it

wait does amd seriously not have invariant tsc? wow

gentle peak Dec 12, 2024, 11:01 PM

#

it does, just not the cpuid leaf that tells you the freq

vast kestrel Dec 12, 2024, 11:01 PM

#

Yeah they don't

#

I don't think it fully holds to invariant tsc

gentle peak Dec 12, 2024, 11:01 PM

#

the invariant TSC itself is still supported

vast kestrel Dec 12, 2024, 11:01 PM

#

It might be constant tho

#

Invariant also works across P/C states

gentle peak Dec 12, 2024, 11:01 PM

#

you just need to calibrate it to figure out the freq

gentle peak Dec 12, 2024, 11:01 PM

#

vast kestrel I don't think it fully holds to invariant tsc

it does

vast kestrel Dec 12, 2024, 11:02 PM

#

Then why didn't they enable it lol

gentle peak Dec 12, 2024, 11:02 PM

#

they do

vast kestrel Dec 12, 2024, 11:02 PM

#

I mean the cpuid leaf

frank canopy Dec 12, 2024, 11:02 PM

#

I wish there was a way to detect constant tsc without requiring the power p/c state stuff for invtsv tbh. unless there is and I don't know it anyway

gentle peak Dec 12, 2024, 11:02 PM

#

they set the bit that indicates it's supported, they just don't have the frequency leaf

vast kestrel Dec 12, 2024, 11:02 PM

#

Ah

#

I think early ryzens also didn't have invariant tsc

#

And I thought you were talking about that

frank canopy Dec 12, 2024, 11:03 PM

#

the lack of frequency leaf i knew because its making things more annoying to me because I only have amd cpus lol

north holly Dec 12, 2024, 11:03 PM

#

My cpu is Haswell so it doesn't have invtsc

#

Intel i5 4590T

vast kestrel Dec 12, 2024, 11:03 PM

#

Yeah it was before the bit existed I think

#

It does have constant tsc

gentle peak Dec 12, 2024, 11:04 PM

#

vast kestrel I think early ryzens also didn't have invariant tsc

I doubt that considering the supported bit is in the AMD CPUID block and my pre-Ryzen Intel system supports it (Haswell)

vast kestrel Dec 12, 2024, 11:04 PM

#

I am also certain it doesnt have it, but I don't have a Ryzen to test rn

#

I think since Prescott the tsc is constant

#

But for a while it wasn't invariant and would change across C states

#

Or maybe it was during sleep idr

hollow elm Dec 12, 2024, 11:06 PM

#

north holly My cpu is Haswell so it doesn't have invtsc

are you sure? intel introduced it in like 2008 or so

calm latch Dec 12, 2024, 11:06 PM

#

Can I assume rdtsc always exists?

hollow elm Dec 12, 2024, 11:06 PM

#

er, no, that's constant tsc

north holly Dec 12, 2024, 11:06 PM

#

hollow elm are you sure? intel introduced it in like 2008 or so

Pretty sure, I can check when I get to my pc

hollow elm Dec 12, 2024, 11:07 PM

#

vast kestrel Or maybe it was during sleep idr

iirc constant vs invariant is across deep sleep states

north holly Dec 12, 2024, 11:07 PM

#

Constant TSC is between C states n stuff right

vast kestrel Dec 12, 2024, 11:07 PM

#

Could be that

hollow elm Dec 12, 2024, 11:07 PM

#

so acpi sleep etc

frank canopy Dec 12, 2024, 11:07 PM

#

yep

vast kestrel Dec 12, 2024, 11:07 PM

#

Ok so it seems nehalem and up should have invariant tsc across p/c states

#

But not across s states

frank canopy Dec 12, 2024, 11:07 PM

#

invariant is across s states and constant is p/c states afaik

#

the problem is iirc theres no way to check for constant unless you also have invariant

hollow elm Dec 12, 2024, 11:08 PM

#

solution: ```
if(invariant)
return;
if(!constant)
panic();
turn_off_acpi_sleep();

fiery turtle Dec 12, 2024, 11:08 PM

#

hollow elm solution: ``` if(invariant) return; if(!constant) panic(); turn_off_acpi...

id assert

vast kestrel Dec 12, 2024, 11:08 PM

#

before nehalem it seems that the tsc was constant across everything but c states excluding hlt state

#

Idk if there were other sleep states tbh

north holly Dec 12, 2024, 11:08 PM

#

frank canopy the problem is iirc theres no way to check for constant unless you also have inv...

Well if Linux can figure it out on my cpu without invtsc, then surely there is some way

gentle peak Dec 12, 2024, 11:09 PM

#

north holly Intel i5 4590T

My Intel i5 4200M has the bit set

vast kestrel Dec 12, 2024, 11:09 PM

#

But I am almost sure that before Prescott it wasn't constant at all

north holly Dec 12, 2024, 11:09 PM

#

gentle peak My Intel i5 4200M has the bit set

Maybe I just misremembered

#

Although I doubt it

gentle peak Dec 12, 2024, 11:10 PM

#

north holly Constant TSC is between C states n stuff right

from Intel sdm:

The invariant TSC will run at a constant rate in all ACPI P-, C-. and T-states

north holly Dec 12, 2024, 11:10 PM

#

Then constant TSC is what

gentle peak Dec 12, 2024, 11:11 PM

#

I think it's the other way around, constant TSC is consistent as long as you don't change C-states and invariant TSC is always consistent

vast kestrel Dec 12, 2024, 11:11 PM

#

The hell is T state

vast kestrel Dec 12, 2024, 11:12 PM

#

gentle peak I think it's the other way around, constant TSC is consistent as long as you don...

And it seems that anything P5+ on the Intel side will basically be invariant as long as you don't enter any sleep state other than hlt

gentle peak Dec 12, 2024, 11:12 PM

#

yeah that's constant I think

#

and invariant is the same but you are allowed to enter sleep states

vast kestrel Dec 12, 2024, 11:13 PM

#

Technically hlt enters a c state, it's just that in the cpu impl case it also increments the tsc normally

north holly Dec 12, 2024, 11:13 PM

#

Invariant also means it's synchronized with all other CPUs

calm latch Dec 12, 2024, 11:13 PM

#

What was the trick for this, but without overflowing?
return rdtsc() - boot_tsc * 1000000 / tsc_ticks_per_1_ms;

vast kestrel Dec 12, 2024, 11:13 PM

#

north holly Invariant also means it's synchronized with all other CPUs

Where you saw that?

north holly Dec 12, 2024, 11:13 PM

#

Right

#

?

calm latch Dec 12, 2024, 11:13 PM

#

(except missing parenthesis)

north holly Dec 12, 2024, 11:13 PM

#

vast kestrel Where you saw that?

I was asking

gentle peak Dec 12, 2024, 11:14 PM

#

north holly Invariant also means it's synchronized with all other CPUs

i think so on all hardware and I assume so in my kernel but I don't think it's explicitly guaranteed in the sdm

#

if it is I can't find it

vast kestrel Dec 12, 2024, 11:14 PM

#

I think the spec says nothing

north holly Dec 12, 2024, 11:14 PM

#

If Linux and windows assume it then surely you can too

vast kestrel Dec 12, 2024, 11:14 PM

#

But practically every single socket cpu has a single tsc

north holly Dec 12, 2024, 11:14 PM

#

But idk if they assume it

vast kestrel Dec 12, 2024, 11:14 PM

#

I am not sure if they do

#

Linux almost certainly does not because of numa, and they might as well always sync it just in case

gentle peak Dec 12, 2024, 11:14 PM

#

calm latch What was the trick for this, but without overflowing? `return rdtsc() - boot_tsc...

you change the multiplier in such a way that the divisor becomes a power of two, then use 128-bit multiplication and shift right instead of division

#

I forgot the formulas for how to do that but they're in this server somewhere

vast kestrel Dec 12, 2024, 11:15 PM

#

Imagine using anything but the acpi timer smh

hollow elm Dec 12, 2024, 11:15 PM

#

https://github.com/managarm/managarm/blob/master/kernel/thor/generic/thor-internal/util.hpp#L14

calm latch Dec 12, 2024, 11:17 PM

#

But will I have 128 bit multiplication in i686? Thonk

vast kestrel Dec 12, 2024, 11:17 PM

#

You can also always use the hypervisor provided timer :^)

#

32bit is for the weak

calm latch Dec 12, 2024, 11:18 PM

#

I have a specific hardware which I want to target

gentle peak Dec 12, 2024, 11:18 PM

#

calm latch But will I have 128 bit multiplication in i686? <:Thonk:1271231807945441402>

not in a single instruction but I believe it won't need to use libgcc for it

calm latch Dec 12, 2024, 11:18 PM

#

vast kestrel 32bit is for the weak

It's the other way around

vast kestrel Dec 12, 2024, 11:19 PM

#

Might as well use i286

calm latch Dec 12, 2024, 11:20 PM

#

It's Celeron M

#

Or Celeron D?

hollow elm Dec 12, 2024, 11:20 PM

#

i think on 32-bit it'll be 4 muls and some additions for a 64x64=128 bit multiplication

calm latch Dec 12, 2024, 11:20 PM

#

I don't remember

calm latch Dec 12, 2024, 11:21 PM

#

hollow elm <https://github.com/managarm/managarm/blob/master/kernel/thor/generic/thor-inter...

I don't understand any of it

gentle peak Dec 12, 2024, 11:21 PM

#

actually iirc GCC doesn't support 128 bit arithmetic at all on <64 bit arches

#

as in the int128 type doesn't exist

calm latch Dec 12, 2024, 11:21 PM

#

(I have to take another look tomorrow)

#

I'm using clang

gentle peak Dec 12, 2024, 11:21 PM

#

then idk

calm latch Dec 12, 2024, 11:21 PM

#

fuck gcc

#

it's buggy and obnoxious

gentle peak Dec 12, 2024, 11:22 PM

#

agree but I would like my kernel to at least be compilable with it

vast kestrel Dec 12, 2024, 11:22 PM

#

GCC is based since it has the new struct changes in c23 already

calm latch Dec 12, 2024, 11:23 PM

#

But I would need to rebase my patches on the latest version

hollow elm Dec 12, 2024, 11:23 PM

#

<source>:4:22: error: __int128 is not supported on this target
    4 |     return (unsigned __int128)a * (unsigned __int128)b;
      |                      ^
``` clang also doesn't support it on 32-bit

gentle peak Dec 12, 2024, 11:23 PM

#

gentle peak then idk

the only times I've messed around with 32 bit is while writing for the absolute lowest common denominator (Compaq Deskpro 386) so I didn't have to deal with tsc

calm latch Dec 12, 2024, 11:23 PM

#

Otherwise atm I have gcc 12

vast kestrel Dec 12, 2024, 11:23 PM

#

The last time I messed around with not 64bit is either using armv7-m or an 8bit cpu

fiery turtle Dec 12, 2024, 11:24 PM

#

vast kestrel GCC is based since it has the new struct changes in c23 already

which

calm latch Dec 12, 2024, 11:24 PM

#

gentle peak the only times I've messed around with 32 bit is while writing for the absolute ...

It doesn't matter, I just use uint64_t's for timers in my kernel

vast kestrel Dec 12, 2024, 11:24 PM

#

fiery turtle which

Where multiple definitions of the same struct are allowed (as long as they have the exact same fields)

fiery turtle Dec 12, 2024, 11:24 PM

#

why is that based

vast kestrel Dec 12, 2024, 11:24 PM

#

It allows to have generic macros without also needing a define if the type

gentle peak Dec 12, 2024, 11:24 PM

#

oh yeah same even in that one I used 64 bit but for timers as fast as the tsc you really want to be able to have 128 bit intermediate values

vast kestrel Dec 12, 2024, 11:24 PM

#

So things like a result type are easy to impmement

fiery turtle Dec 12, 2024, 11:24 PM

#

vast kestrel It allows to have generic macros without also needing a define if the type

hm?

vast kestrel Dec 12, 2024, 11:25 PM

#

Like a imagine a macro defining ```c
struct result_int { err_t err; int value; }

#

Where the int is pasted from a macro parameter

#

So RESULT(int)

#

You can not use that macro in both the function signature and when returning

#

And it would be the exact same type

fiery turtle Dec 12, 2024, 11:26 PM

#

vast kestrel You can not use that macro in both the function signature and when returning

how would that work

hollow elm Dec 12, 2024, 11:27 PM

#

hollow elm i think on 32-bit it'll be 4 muls and some additions for a 64x64=128 bit multipl...

with 32x32=64 multiplication, treating 64-bit numbers as two digit numbers in base 2**32:

AB * CD
= uint128_t(B * D)
+ uint128_t(D * A) << 32
+ uint128_t(C * B) << 32
+ uint128_t(C * A) << 64

#

amazing, thanks discord for the code block

vast kestrel Dec 12, 2024, 11:28 PM

#

RESULT(int) my_func() {
return OK(int, 123);
}

And OK can be (RESULT(int)){.value = 124};

fiery turtle Dec 12, 2024, 11:28 PM

#

ah

#

cool

vast kestrel Dec 12, 2024, 11:29 PM

#

Like it's almost entirely for macros defining structs to have a easier time

gentle peak Dec 12, 2024, 11:29 PM

#

every time I hear something about c23 I didn't know before the urge to target it in my next project gets more intense

vast kestrel Dec 12, 2024, 11:30 PM

#

I mean ignoring the cool libc additions, I would take it just for the #embed lol

gentle peak Dec 12, 2024, 11:30 PM

#

the only thing stopping me right now is that afaik the implementations aren't fully complete yet

vast kestrel Dec 12, 2024, 11:30 PM

#

So many projects where I need to setup an objcopy just for thatv

#

Yeah both clang and GCC are missing stuff

#

Also I am not sure if anything fully added the libc additions

gentle peak Dec 12, 2024, 11:31 PM

#

vast kestrel I mean ignoring the cool libc additions, I would take it just for the #embed lol

eh it'd be nice but I just run the files in question through xxd manually

vast kestrel Dec 12, 2024, 11:31 PM

#

Ouch

#

Or well

#

I have at works projects where we need to include mbs of files in the binary

gentle peak Dec 12, 2024, 11:31 PM

#

works fine enough for small files and embedding large files would give me the ick anyway

vast kestrel Dec 12, 2024, 11:31 PM

#

Which can take literal minutes to compile sometimes

gentle peak Dec 12, 2024, 11:32 PM

#

oh yeah fair enough, the largest files I've ever needed to embed are like 4k-16k

vast kestrel Dec 12, 2024, 11:32 PM

#

In one project we have an entire rootfs embedded in the binary (we really like standalone static binaries)

#

The bit precise integers are also cool

#

Especially cause no integer promotion

#

Oh and yeah, typed enum

#

Wait that's the uacpi channel wtf

calm latch Dec 12, 2024, 11:34 PM

#

@fiery turtle

fiery turtle Dec 12, 2024, 11:34 PM

#

the record on this server is 1.5M/s

#

so ur a bit behind

calm latch Dec 12, 2024, 11:35 PM

#

What does that even mean?

north holly Dec 12, 2024, 11:35 PM

#

I am 2nd worst

#

nooo

calm latch Dec 12, 2024, 11:35 PM

#

Am I the 1st worst?

north holly Dec 12, 2024, 11:35 PM

#

Feels bad to be nyaux though

#

I thought mine was bad

#

Which it is

#

But nyaux' was terrible

calm latch Dec 12, 2024, 11:35 PM

#

I'm gonna recompile to 64 bits...

fiery turtle Dec 12, 2024, 11:36 PM

#

calm latch Am I the 1st worst?

nah, nyaux is the worst at 150/s

gentle peak Dec 12, 2024, 11:36 PM

#

It's mostly an allocator and mutex benchmark right?

fiery turtle Dec 12, 2024, 11:36 PM

#

nah

#

memset+memcpy/allocator

gentle peak Dec 12, 2024, 11:36 PM

#

ah

calm latch Dec 12, 2024, 11:37 PM

#

I'm using dlmalloc

vast kestrel Dec 12, 2024, 11:37 PM

#

fiery turtle nah, nyaux is the worst at 150/s

Is it running on a big endian subleq cpu by any chance

fiery turtle Dec 12, 2024, 11:37 PM

#

lmao

north holly Dec 12, 2024, 11:37 PM

#

I think uacpi's memset and memcpy are a big culprit for me

#

My normal ones are accelerated

#

But idk if uacpi uses those ones

gentle peak Dec 12, 2024, 11:39 PM

#

I kinda wanna try integrating it into my kernel just for the benchmark but so far I've managed to get away with no real kernel vmm (all kernel allocations are <=PAGE_SIZE and in hhdm) and I'd like to keep it that way

frank canopy Dec 12, 2024, 11:39 PM

#

calm latch <@353593948523986945>

that looks about the same as what i get when building in releasesafe (no idea what the equiv is)

fiery turtle Dec 12, 2024, 11:39 PM

#

north holly But idk if uacpi uses those ones

not if u made it use those

north holly Dec 12, 2024, 11:39 PM

#

Yeah ik

fiery turtle Dec 12, 2024, 11:40 PM

#

look at the libc header

frank canopy Dec 12, 2024, 11:40 PM

#

north holly But idk if uacpi uses those ones

you can override them if it isnt, thats what im doing (though just forwarding to zig's builtins for me)

north holly Dec 12, 2024, 11:40 PM

#

I override livc

#

Libc

#

I think

#

I indeed do

#

void *uacpi_memcpy(void *dest, const void* src, size_t sz)
{
    return memcpy(dest,src,sz);
}
void *uacpi_memset(void *dest, int src, size_t cnt)
{
    return memset(dest, src, cnt);
}```

frank canopy Dec 12, 2024, 11:40 PM

#

zig's builtins might be accelerated but idk, i also have all simd/vector things turned off in my kernel anyway so idk how much it can accel apart from rep instruction stuff

north holly Dec 12, 2024, 11:41 PM

#

that's probably not the culprit

frank canopy Dec 12, 2024, 11:41 PM

#

well in that case it is using yours

north holly Dec 12, 2024, 11:41 PM

#

I use rep movsb and friends

north holly Dec 12, 2024, 11:41 PM

#

frank canopy well in that case it is using yours

I know

#

but memcmp

int uacpi_memcmp(const void *src1, const void *src2, size_t cnt)
{
    const uint8_t* b1 = (const uint8_t*)src1;
    const uint8_t* b2 = (const uint8_t*)src2;
    for (size_t i = 0; i < cnt; i++)
        if (b1[i] < b2[i])
            return -1;
        else if (b1[i] > b2[i])
            return 1;
        else
            continue;
    return 0;
}```

#

does this

#

which isn't very efficient

frank canopy Dec 12, 2024, 11:41 PM

#

id assume zig uses rep movsb but im legit not sure

north holly Dec 12, 2024, 11:42 PM

#

void *uacpi_memmove(void *dest, const void* src, size_t len)
{
    if (src == dest)
        return dest;
    // Refactored from https://stackoverflow.com/a/65822606
    uint8_t *dp = (uint8_t *)dest;
    const uint8_t *sp = (uint8_t *)src;
    if(sp < dp && sp + len > dp)
    {
        sp += len;
        dp += len;
        while(len-- > 0)
            *--dp = *--sp;
    }
    else
        while(len-- > 0)
            *dp++ = *sp++;
    return dest;
}```

#

memmove can probably be improved

fiery turtle Dec 12, 2024, 11:42 PM

#

north holly ```c void *uacpi_memcpy(void *dest, const void* src, size_t sz) { return mem...

yeah thats not uh

#

how any of this works

north holly Dec 12, 2024, 11:42 PM

#

what

frank canopy Dec 12, 2024, 11:42 PM

#

oh right its #defines i forgot about that

north holly Dec 12, 2024, 11:43 PM

#

fiery turtle how any of this works

oh yeah I know

fiery turtle Dec 12, 2024, 11:43 PM

#

north holly what

libc.h

north holly Dec 12, 2024, 11:43 PM

#

#define uacpi_memcpy uacpi_memcpy
#define uacpi_memset uacpi_memset
#define uacpi_memmove uacpi_memmove
#define uacpi_memcmp uacpi_memcmp
#define uacpi_strcmp uacpi_strcmp
#define uacpi_strncmp uacpi_strncmp
#define uacpi_strnlen uacpi_strnlen
#define uacpi_strlen uacpi_strlen
#define uacpi_snprintf uacpi_snprintf```

#

I do fnuy macro stuffs

fiery turtle Dec 12, 2024, 11:43 PM

#

why extra indirection lol

#

just do #define uacpi_memcpy __builtin_memcpy etc

calm latch Dec 12, 2024, 11:43 PM

#

calm latch <@353593948523986945>

This is in userspace btw

fiery turtle Dec 12, 2024, 11:43 PM

#

yeah only 3 orders of magnitude behind managarm meme

north holly Dec 12, 2024, 11:44 PM

#

fiery turtle just do `#define uacpi_memcpy __builtin_memcpy` etc

well my strcmp and memcmp functions are non conformant

#

I am missing memmove

#

and strnlen

fiery turtle Dec 12, 2024, 11:44 PM

#

north holly well my strcmp and memcmp functions are non conformant

💀

calm latch Dec 12, 2024, 11:44 PM

#

fiery turtle yeah only 3 orders of magnitude behind managarm <:meme:575445094589661215>

What do they do

north holly Dec 12, 2024, 11:44 PM

#

they return a bool instead of int

fiery turtle Dec 12, 2024, 11:44 PM

#

memcpy and memset should still be builtin

calm latch Dec 12, 2024, 11:44 PM

#

Stack allocator?

fiery turtle Dec 12, 2024, 11:44 PM

#

calm latch Stack allocator?

just a normal frigg allocator

gentle peak Dec 12, 2024, 11:44 PM

#

north holly well my strcmp and memcmp functions are non conformant

if memcmp is nonconformant you're violating one of gcc's assumptions

north holly Dec 12, 2024, 11:45 PM

#

you're right

#

although,

#

I'll fix it later

fiery turtle Dec 12, 2024, 11:47 PM

#

managarm and keyronex are both at 1.5M, I think for keyronex reducing vmexits for cr8 writes and improving memcpy was the fix

#

initially keyronex was at 250K i think

#

oh and a 4k slab

north holly Dec 12, 2024, 11:47 PM

#

I forgor how much mine was

fiery turtle Dec 12, 2024, 11:49 PM

#

#1217009725711847465 message

#

the big flex

fiery turtle Dec 12, 2024, 11:50 PM

#

fiery turtle initially keyronex was at 250K i think

astral is at 250k also

calm latch Dec 12, 2024, 11:51 PM

#

That reading was wrong

#

Oof

north holly Dec 12, 2024, 11:51 PM

#

my cpu does indeed have invtsc

fiery turtle Dec 12, 2024, 11:51 PM

#

+invtsc

calm latch Dec 12, 2024, 11:51 PM

#

(and it didn't use tsc)

fiery turtle Dec 12, 2024, 11:51 PM

#

in qemu

north holly Dec 12, 2024, 11:52 PM

#

but idk why linux doesn't report it

gentle peak Dec 12, 2024, 11:52 PM

#

calm latch That reading was wrong

QEMU explicitly clears it because it messes up VM migration

north holly Dec 12, 2024, 11:52 PM

#

in /proc/cpuinfo

fiery turtle Dec 12, 2024, 11:52 PM

#

calm latch Oof

this is why with qemu+kvm u use the kvm clock

#

it just gives u the values to convert tsc to elapsed ns

gentle peak Dec 12, 2024, 11:52 PM

#

Add support for kvmclock or use -M migratable=off (both only have an effect if kvm is used)

calm latch Dec 12, 2024, 11:53 PM

#

I wasn't using KVM

fiery turtle Dec 12, 2024, 11:53 PM

#

wait why

gentle peak Dec 12, 2024, 11:53 PM

#

I haven't found a way to get QEMU to report invariant TSC while using TCG

fiery turtle Dec 12, 2024, 11:53 PM

#

for no kvm 20k is a decent result

north holly Dec 12, 2024, 11:53 PM

#

oberrow@AcerAIO:~$ cpuid -l0x80000007
CPU 0:
   RAS Capability (0x80000007/ebx):
      MCA overflow recovery support = false
      SUCCOR support                = false
      HWA: hardware assert support  = false
      scalable MCA support          = false
   Advanced Power Management Features (0x80000007/ecx):
      CmpUnitPwrSampleTimeRatio = 0x0 (0)
   Advanced Power Management Features (0x80000007/edx):
      TS: temperature sensing diode           = false
      FID: frequency ID control               = false
      VID: voltage ID control                 = false
      TTP: thermal trip                       = false
      TM: thermal monitor                     = false
      STC: software thermal control           = false
      100 MHz multiplier control              = false
      hardware P-State control                = false
      TscInvariant                            = true
      CPB: core performance boost             = false
      read-only effective frequency interface = false
      processor feedback interface            = false
      APM power reporting                     = false
      connected standby                       = false
      RAPL: running average power limit       = false
      fast CPPC                               = false```

hollow elm Dec 12, 2024, 11:53 PM

#

also fun fact, tcg doesn't explicitly emulate invariant tsc, but the tsc is just the host's tsc value on x86_64 hosts so if the host has an invariant tsc the tcg guest does too :^)

gentle peak Dec 12, 2024, 11:54 PM

#

yeah I just decided to assume the TSC was invariant if kvmclock isn't supported, no matter what CPUID says

fiery turtle Dec 12, 2024, 11:54 PM

#

u support kvmclock also?

gentle peak Dec 12, 2024, 11:54 PM

#

yeah

fiery turtle Dec 12, 2024, 11:54 PM

#

damn

#

second kernel here

#

do u have a link to your kernel btw

gentle peak Dec 12, 2024, 11:54 PM

#

if it's available I use it, if not TSC regardless of what CPUID says

#

nope, it's a private repo and will likely stay as such for a very long time

fiery turtle Dec 12, 2024, 11:55 PM

#

why is that

hollow elm Dec 12, 2024, 11:55 PM

#

hollow elm also fun fact, tcg doesn't explicitly emulate invariant tsc, but the tsc is just...

worst case tcg's tsc is just the host's CLOCK_MONOTONIC time in nanoseconds

gentle peak Dec 12, 2024, 11:55 PM

#

just personal preference, I like to keep things private until they're usable

fiery turtle Dec 12, 2024, 11:55 PM

#

hollow elm worst case tcg's tsc is just the host's CLOCK_MONOTONIC time in nanoseconds

syscall per rdtsc 💀

hollow elm Dec 12, 2024, 11:55 PM

#

a little thing called vdso

fiery turtle Dec 12, 2024, 11:55 PM

#

tru

gentle peak Dec 12, 2024, 11:56 PM

#

like my kernel can do quite a lot but it's not at the point where I'd like to make it public

fiery turtle Dec 12, 2024, 11:56 PM

#

works until the clock source in linux is assumed unstable

#

at which point it switches to hpet

#

and vsdo no workie

calm latch Dec 12, 2024, 11:57 PM

#

It didn't work 😦

fiery turtle Dec 12, 2024, 11:58 PM

#

it?

calm latch Dec 12, 2024, 11:58 PM

#

Am I checking it wrong?

fiery turtle Dec 12, 2024, 11:58 PM

#

what did u change

calm latch Dec 12, 2024, 11:58 PM

#

    u64 c = cpuid(0x80000007);
    if (c & (1 << 8)) {
        global_logger.printf("[Kernel] Info: TSC Invariant bit is set\n");

fiery turtle Dec 12, 2024, 11:58 PM

#

your qemu command line..

vast kestrel Dec 12, 2024, 11:59 PM

#

Qemu doesn't pass invariant tsc by default

calm latch Dec 12, 2024, 11:59 PM

#

fiery turtle what did u change

sudo qemu-system-x86_64 -device pcie-root-port,id=root_port1,port=0x10,chassis=1,multifunction=on,slot=1,bus=pcie.0,addr=0x10 -device x3130-upstream,id=switch_upstream,bus=root_port1,addr=0x0 -device xio3130-downstream,id=switch_downstream1,bus=switch_upstream,chassis=3,addr=0x1 -device xio3130-downstream,id=switch_downstream2,bus=switch_upstream,chassis=4,addr=0x2 -device pcie-pci-bridge,id=pci_bridge1,bus=switch_downstream1,addr=0x0 -device pcie-pci-bridge,id=pci_bridge2,bus=switch_downstream2,addr=0x0 -device e1000e,id=nic1,bus=pci_bridge2,addr=0x0 -drive file=limine/pmOS.iso,if=none,id=hdd0 -device ide-hd,drive=hdd0 -M q35 -smbios type=0,uefi=on -bios ovmf-x86/OVMF.fd -m 2G -smp 4 -serial stdio -cpu host,+invtsc -enable-kvm

fiery turtle Dec 12, 2024, 11:59 PM

#

-M migration=off

gentle peak Dec 12, 2024, 11:59 PM

#

I keep forgetting whether it's migration or migratable

hollow elm Dec 12, 2024, 11:59 PM

#

iirc migratable?

fiery turtle Dec 12, 2024, 11:59 PM

#

could be the latter

calm latch Dec 12, 2024, 11:59 PM

#

qemu-system-x86_64: Property 'pc-q35-6.2-machine.migration' not found

fiery turtle Dec 12, 2024, 11:59 PM

#

migratable

hollow elm Dec 13, 2024, 12:00 AM

#

also isn't it a property of cpu and not machine?

fiery turtle Dec 13, 2024, 12:00 AM

#

machine i think

calm latch Dec 13, 2024, 12:00 AM

#

bruh qemu-system-x86_64: Property 'pc-q35-6.2-machine.migratable' not found

fiery turtle Dec 13, 2024, 12:00 AM

#

hm

hollow elm Dec 13, 2024, 12:00 AM

#

-cpu migratable=off

fiery turtle Dec 13, 2024, 12:00 AM

#

u have an old ass qemu btw

gentle peak Dec 13, 2024, 12:00 AM

#

is it migrateable?

#

if that doesn't work either it's probably just because your QEMU is too old

hollow elm Dec 13, 2024, 12:01 AM

#

from managarm's qemu script cpu_model = "host,migratable=no"

calm latch Dec 13, 2024, 12:02 AM

#

📎 pmOS.iso

#

I'll try it baremetal

#

on my laptop

hollow elm Dec 13, 2024, 12:02 AM

#

calm latch `sudo qemu-system-x86_64 -device pcie-root-port,id=root_port1,port=...

btw why do you have such a complex pcie topology

calm latch Dec 13, 2024, 12:03 AM

#

I've been implementing PCI interrupts routing

vast kestrel Dec 13, 2024, 12:04 AM

#

That's enough to have the bit on my setup

#

Aka just the host and invtsc

fiery turtle Dec 13, 2024, 12:04 AM

#

@vast kestrel btw what happened to AML OS?

calm latch Dec 13, 2024, 12:05 AM

#

calm latch ```c++ u64 c = cpuid(0x80000007); if (c & (1 << 8)) { global_log...

Is this correct?

hollow elm Dec 13, 2024, 12:05 AM

#

calm latch I've been implementing PCI interrupts routing

ah

vast kestrel Dec 13, 2024, 12:05 AM

#

Never had the chance to continue sadly

fiery turtle Dec 13, 2024, 12:05 AM

#

F

vast kestrel Dec 13, 2024, 12:05 AM

#

Like I had a super busy time at work

fiery turtle Dec 13, 2024, 12:05 AM

#

happens

vast kestrel Dec 13, 2024, 12:05 AM

#

And the last week I am on vacation but too burned out to really work

fiery turtle Dec 13, 2024, 12:05 AM

#

my vacation starts this saturday so im happy

vast kestrel Dec 13, 2024, 12:07 AM

#

Noice

fiery turtle Dec 13, 2024, 12:07 AM

#

Btw @winter orbit any updates regarding ReactOS? I think you mentioned it was almost ready or w/e but then something happened?

winter orbit Dec 13, 2024, 12:07 AM

#

I’m basically rewriting the entirety of hal ACPI and pci at the same time

vast kestrel Dec 13, 2024, 12:08 AM

#

Lmao

fiery turtle Dec 13, 2024, 12:08 AM

#

damn

calm latch Dec 13, 2024, 12:08 AM

#

How fun, it (uACPI) died

fiery turtle Dec 13, 2024, 12:08 AM

#

make sure to fetch the latest uacpi, a lot has changed since then!

winter orbit Dec 13, 2024, 12:08 AM

#

It workwd though for what I needed ROS to do

fiery turtle Dec 13, 2024, 12:08 AM

#

calm latch How fun, it (uACPI) died

skill issue

#

try managarm on this pc meme

#

(laptop)

vast kestrel Dec 13, 2024, 12:09 AM

#

assertos

calm latch Dec 13, 2024, 12:09 AM

#

And it didn't detect TSC either

fiery turtle Dec 13, 2024, 12:10 AM

#

didnt it use to work lol

calm latch Dec 13, 2024, 12:10 AM

#

Bare metal

winter orbit Dec 13, 2024, 12:10 AM

#

Here’s some screen shots using uACPI though

fiery turtle Dec 13, 2024, 12:11 AM

#

daaamn

winter orbit Dec 13, 2024, 12:11 AM

#

One on the left finally detects the touch screen

fiery turtle Dec 13, 2024, 12:11 AM

#

80 fps is quite decent

winter orbit Dec 13, 2024, 12:11 AM

#

One on the right is using the gpu properly

fiery turtle Dec 13, 2024, 12:11 AM

#

winter orbit One on the left finally detects the touch screen

what had to be changed?

winter orbit Dec 13, 2024, 12:12 AM

#

Our current ACPI driver doesn’t even detect the touch screen

#

so.. yes

fiery turtle Dec 13, 2024, 12:12 AM

#

big

#

amazing work

calm latch Dec 13, 2024, 12:12 AM

#

fiery turtle skill issue

uACPI bug until proven otherwise ultrameme

winter orbit Dec 13, 2024, 12:12 AM

#

Should really be saying that to you lol uACPI works great

north holly Dec 13, 2024, 12:12 AM

#

calm latch uACPI bug until proven otherwise <:ultrameme:729026898666979428>

Hey that's my line

winter orbit Dec 13, 2024, 12:12 AM

#

I’m almost done porting it to windows 10

north holly Dec 13, 2024, 12:13 AM

#

calm latch How fun, it (uACPI) died

Face reveal

fiery turtle Dec 13, 2024, 12:13 AM

#

calm latch uACPI bug until proven otherwise <:ultrameme:729026898666979428>

almost impossible to segfault uacpi, 99.9% confidence its your skill issue ultrameme

calm latch Dec 13, 2024, 12:13 AM

#

winter orbit Here’s some screen shots using uACPI though

Minecraft is impressive

fiery turtle Dec 13, 2024, 12:13 AM

#

winter orbit Should really be saying that to you lol uACPI works great

thank you! happy to hear that

winter orbit Dec 13, 2024, 12:13 AM

#

Wanna see something funnier

#

fiery turtle Dec 13, 2024, 12:13 AM

#

yes!

winter orbit Dec 13, 2024, 12:13 AM

#

StarCraft 2

fiery turtle Dec 13, 2024, 12:13 AM

#

lol

#

damn

#

can it run crisis tho

winter orbit Dec 13, 2024, 12:13 AM

#

yeah

vagrant hull Dec 13, 2024, 12:14 AM

#

only gamers get that joke 😏

fiery turtle Dec 13, 2024, 12:14 AM

#

its a reference to one of their vids actually running crisis on reactos

calm latch Dec 13, 2024, 12:14 AM

#

calm latch uACPI bug until proven otherwise <:ultrameme:729026898666979428>

It crashed in uacpi_execute_table

vagrant hull Dec 13, 2024, 12:14 AM

#

and i am referencing Jensen Huang being cringe

calm latch Dec 13, 2024, 12:14 AM

#

How do I compile with dwarf?

fiery turtle Dec 13, 2024, 12:15 AM

#

calm latch It crashed in uacpi_execute_table

if u want me to prove that it's your bug give me the table dump and ill run those in userspace with asan and ubsan meme

winter orbit Dec 13, 2024, 12:15 AM

#

fiery turtle can it run crisis tho

https://youtube.com/shorts/8UwZ3FLgq6E?si=1BjutWZxwaR7y7bK

YouTube

The_DarkFire_

ReactOS runs Crysis at 4K medium settings with GPU work arounds! (R...

▶ Play video

#

This isn’t uACPI though

#

Everything I’ve sent today otherwise is :)

fiery turtle Dec 13, 2024, 12:15 AM

#

still crazy

winter orbit Dec 13, 2024, 12:16 AM

#

yeah turns out GPUs were getting broken on pci, Hal, ACPI etc

#

So I’m rewriting stufff to be more like windows 10

#

That’s why it’s taking so long, it’s kinda a pain oof and will probably still take many months

fiery turtle Dec 13, 2024, 12:16 AM

#

daamn

#

I wanna hear progress reports tho meme

calm latch Dec 13, 2024, 12:17 AM

#

fiery turtle if u want me to prove that it's your bug give me the table dump and ill run thos...

I've sent it to you before

winter orbit Dec 13, 2024, 12:17 AM

#

I’ll make a thread next interesting thing happens and start posting on it

fiery turtle Dec 13, 2024, 12:18 AM

#

oh cool

winter orbit Dec 13, 2024, 12:18 AM

#

My bad I get tunnel vision

fiery turtle Dec 13, 2024, 12:18 AM

#

pls do

fiery turtle Dec 13, 2024, 12:18 AM

#

calm latch I've sent it to you before

was it here or

calm latch Dec 13, 2024, 12:18 AM

#

I don't remember

fiery turtle Dec 13, 2024, 12:19 AM

#

winter orbit My bad I get tunnel vision

yeah thats quite understandable

#

i wonder what sort of ops/s is reactos getting

calm latch Dec 13, 2024, 12:19 AM

#

I think it crashed in prepare_table_load

#

Ok but seriously how do I make objdump show code with clang

fiery turtle Dec 13, 2024, 12:20 AM

#

calm latch I think it crashed in prepare_table_load

cant u just addr2line the rip?

calm latch Dec 13, 2024, 12:21 AM

#

??:?

fiery turtle Dec 13, 2024, 12:21 AM

#

do u compile with -g?

calm latch Dec 13, 2024, 12:21 AM

#

Yes

fiery turtle Dec 13, 2024, 12:21 AM

#

try -ggdb3

calm latch Dec 13, 2024, 12:22 AM

#

Still nothing

fiery turtle Dec 13, 2024, 12:22 AM

#

is it even a valid rip

calm latch Dec 13, 2024, 12:22 AM

#

Ok, llvm-objdump worked for some reason

calm latch Dec 13, 2024, 12:23 AM

#

fiery turtle is it even a valid rip

Yes, I looked at assembly

#

It crashed in ```x86asm
; return cause != UACPI_TABLE_LOAD_CAUSE_INIT;
224683: 41 83 fe 02 cmpl $0x2, %r14d
224687: 41 0f 94 c6 sete %r14b
; in_method->code = dsdt->definition_block;
22468b: 48 8d 43 24 leaq 0x24(%rbx), %rax
22468f: 48 89 45 c8 movq %rax, -0x38(%rbp)
; in_method->size = dsdt->hdr.length - sizeof(dsdt->hdr);
224693: 8b 43 04 movl 0x4(%rbx), %eax
224696: 83 c0 dc addl $-0x24, %eax
224699: 89 45 d8 movl %eax, -0x28(%rbp)
; in_method->named_objects_persist = UACPI_TRUE;

#

uACPI/source/interpreter.c:1260

fiery turtle Dec 13, 2024, 12:24 AM

#

whats the cr2 btw

calm latch Dec 13, 2024, 12:24 AM

#

4

fiery turtle Dec 13, 2024, 12:25 AM

#

so the dsdt pointer was null?

calm latch Dec 13, 2024, 12:25 AM

#

Idk

#

It printed half of the tables and then crashed

fiery turtle Dec 13, 2024, 12:25 AM

#

this code is executed way past that

#

i mean the table printing

calm latch Dec 13, 2024, 12:26 AM

#

calm latch How fun, it (uACPI) died

Like this is what it printed

fiery turtle Dec 13, 2024, 12:26 AM

#

can u recheck your mapping callback?

calm latch Dec 13, 2024, 12:26 AM

#

What do I need to check?

fiery turtle Dec 13, 2024, 12:26 AM

#

calm latch Like this is what it printed

this looks like all of the tables to me

#

whats missing

calm latch Dec 13, 2024, 12:27 AM

#

The last one looks cut off

fiery turtle Dec 13, 2024, 12:27 AM

#

facs doesnt have those fields

#

so they arent printed

#

its special cased in the dump_table_header fn

#

so it dies when trying to load the DSDT, when preparing it for load

#

u can add a printf to uacpi_execute_table to see the value of tbl

#

btw can u not add a trailing newline to uacpi logs because it looks ugly

calm latch Dec 13, 2024, 12:37 AM

#

fiery turtle Dec 13, 2024, 12:37 AM

#

aa000?

calm latch Dec 13, 2024, 12:37 AM

#

It's normal

#

If it's from the mapping callback

fiery turtle Dec 13, 2024, 12:39 AM

#

well

#

looks like a stack corruption or something similar, because mapping is checked for nulls

#

i can see it propagating the error

#

        tbl->ptr = uacpi_kernel_map(tbl->phys_addr, tbl->hdr.length);
        if (uacpi_unlikely(tbl->ptr == UACPI_NULL))
            return UACPI_STATUS_MAPPING_FAILED;

#

maybe log all the mappings happening?

calm latch Dec 13, 2024, 12:47 AM

#

#

Does it call unmap right after mapping?

#

Lol how do I dump the tables

#

From Windows

fiery turtle Dec 13, 2024, 12:52 AM

#

calm latch Does it call unmap right after mapping?

after loading, yeah

fiery turtle Dec 13, 2024, 12:52 AM

#

calm latch From Windows

acpidump > dump.txt

#

acpixtract -a dump.txt

fiery turtle Dec 13, 2024, 12:52 AM

#

calm latch

huh, mapping is not even called?

#

that might be a heap corruption that happens to overwrite the table's reference count value

#

try increasing UACPI_STATIC_TABLE_ARRAY_LEN

#

to like 128 or something

#

[uACPI][INFO] RSDP 0x00007FFE47B42BE0 00000024 v02 (uOEMID)
[uACPI][INFO] XSDT 0x0000611000000CC0 000000DC v02 (LENOVO uTESTTBL)
[uACPI][INFO] DSDT 0x0000630000000400 0000E586 v02 (LENOVO AMD_EDK2)
[uACPI][INFO] FACP 0x00006120000004C0 00000114 v06 (uOEMID uTESTTBL)
[uACPI][INFO] SSDT 0x000061E000000080 000009AE v01 (LENOVO UsbCTabl)
[uACPI][INFO] SSDT 0x000062D000000400 00008416 v02 (LENOVO AmdTable)
[uACPI][INFO] SSDT 0x000062A000000200 0000547E v02 (LENOVO AmdTable)
[uACPI][INFO] SSDT 0x0000619000000A80 000003D5 v02 (LENOVO Tpm2Tabl)
[uACPI][INFO] SSDT 0x000061B000000080 00000655 v02 (LENOVO OEMPMF  )
[uACPI][INFO] SSDT 0x000061F000000080 00000BEB v02 (LENOVO CPMDFIG4)
[uACPI][INFO] SSDT 0x0000626000000100 00002C36 v02 (LENOVO CDFAAIG2)
[uACPI][INFO] SSDT 0x000062E000000400 00009F19 v02 (LENOVO CPMCMN  )
[uACPI][INFO] SSDT 0x000061F000000E80 00000CC7 v02 (LENOVO NVME    )
[uACPI][INFO] SSDT 0x000061E000000C80 0000098D v02 (LENOVO GpMsSsdt)
[uACPI][INFO] SSDT 0x0000608000000220 00000058 v02 (LENOVO GPP1    )
[uACPI][INFO] SSDT 0x00006080000002A0 00000058 v02 (LENOVO GPP0    )
[uACPI][INFO] SSDT 0x0000622000000100 00001678 v02 (LENOVO UPEP    )
[uACPI][INFO] SSDT 0x000061C000000080 0000074E v02 (LENOVO THERMAL0)
[uACPI][INFO] SSDT 0x0000621000000100 0000108F v02 (LENOVO GPP_PME_)
[uACPI][INFO] SSDT 0x000062D00000A400 00009B61 v02 (LENOVO INTGPP03)
[uACPI][INFO] SSDT 0x000062A000006200 000052B4 v02 (LENOVO INTGPP01)
[uACPI][INFO] SSDT 0x000061E000001880 00000B69 v02 (LENOVO CPMGPIO0)
[uACPI][INFO] SSDT 0x0000611000000E00 000000E7 v02 (LENOVO MHSP    )
[uACPI][INFO] SSDT 0x000061D000000080 0000097B v02 (LENOVO CPMMSOSC)
[uACPI][INFO] SSDT 0x000060D0000002B0 00000090 v02 (LENOVO CPMMSLPI)
[uACPI][INFO] SSDT 0x0000621000001500 00000FDF v02 (LENOVO CPMACPV5)
[uACPI][INFO] FACS 0x0000606000000320 00000040
[uACPI][INFO] successfully loaded 23 AML blobs, 10241 ops in 259ms (avg 39409/s)
[uACPI][INFO] namespace initialization done in 22ms: 106 devices, 1 thermal zones

#

@calm latch

calm latch Dec 13, 2024, 12:57 AM

#

Hmm

#

Ok it's getting too late

#

I'm gonna try it tomorrow

fiery turtle Dec 13, 2024, 12:58 AM

#

lets try more stuff tomorrow yeah

fiery turtle Dec 13, 2024, 12:58 AM

#

fiery turtle try increasing `UACPI_STATIC_TABLE_ARRAY_LEN`

this is what i recommend

#

if behavior changes its a heap corruption

calm latch Dec 13, 2024, 12:59 AM

#

What could cause it though

fiery turtle Dec 13, 2024, 1:00 AM

#

who knows, could be a bug in your kernel_api impl, could be a bug in the kernel's vmm handing out the same phys page for allocator storage

#

anything

#

u use an existing malloc impl right

calm latch Dec 13, 2024, 1:02 AM

#

dlmalloc

fiery turtle Dec 13, 2024, 1:03 AM

#

Ye

calm latch Dec 13, 2024, 1:04 AM

#

I've recently changed how tlb shootdowns are handled, and that might have changed something in my vmm and I think it's been a bit fishy

fiery turtle Dec 13, 2024, 1:04 AM

#

The only path through which this doesnt call map is when a table already has alive references

#

Which is impossible in this case

calm latch Dec 13, 2024, 1:04 AM

#

Can I print references?

fiery turtle Dec 13, 2024, 1:04 AM

#

Yeah ofc

#

table_ref_unlocked

calm latch Dec 13, 2024, 1:04 AM

#

But tomorrow

fiery turtle Dec 13, 2024, 1:04 AM

#

U can add a printf

fiery turtle Dec 13, 2024, 1:05 AM

#

calm latch I've recently changed how tlb shootdowns are handled, and that might have change...

That sounds quite plausible

fiery turtle Dec 13, 2024, 1:05 AM

#

calm latch But tomorrow

Yeye

calm latch Dec 13, 2024, 1:06 AM

#

I can try 32 bit userspace trl

fiery turtle Dec 13, 2024, 1:06 AM

#

Try it

calm latch Dec 13, 2024, 1:10 AM

#

The strange thing is that it's consistent

#

Or maybe not strange

#

No, it is strange

#

Printfs should have changed the order of stuff

fiery turtle Dec 13, 2024, 1:12 AM

#

Well I mean its not multithreaded so why would it change

calm latch Dec 13, 2024, 1:12 AM

#

The kernel is multithreaded

#

It's running 33 threads

fiery turtle Dec 13, 2024, 1:12 AM

#

Which printfs do you mean

calm latch Dec 13, 2024, 1:12 AM

#

On 16 threads CPU

#

Of allocs

#

They do IPC and stuff, which is slow

fiery turtle Dec 13, 2024, 1:13 AM

#

Ah

#

Anyways im off to bed but happy to assist with debugging tomorrow

calm latch Dec 13, 2024, 1:13 AM

#

Like if the pages get stolen and double mapped or something

#

Yeah it's 2 am where I live

fiery turtle Dec 13, 2024, 1:13 AM

#

Memory corruption is evil

#

Gn

calm latch Dec 13, 2024, 1:14 AM

#

Gn

calm latch Dec 13, 2024, 10:55 AM

#

Ok I think I've fixed invariant tsc

#

My CPUID function was broken...

calm latch Dec 13, 2024, 5:27 PM

#

hollow elm <https://github.com/managarm/managarm/blob/master/kernel/thor/generic/thor-inter...

This is somewhat unrelated, but I can't get this to work

#

    u32 ticks   = apic_read_reg(APIC_REG_TMRCURRCNT);
    u64 divisor = 1'000'000;

    apic_freq = computeFreqFraction(ticks, divisor)
    auto l = apic_freq*1'000'000'000;

    printf("%li %li\n", l, ticks*1000);

prints 4294339902999 3667574296

#

(and 3667574296 should be correct I think)

gentle peak Dec 13, 2024, 5:56 PM

#

the apic timer counts down so you have to subtract the read value from your starting value

calm latch Dec 13, 2024, 7:04 PM

#

I reset It in the beginning

gentle peak Dec 13, 2024, 7:26 PM

#

yes but it still counts down

#

when you read it, you get the ticks remaining

#

that calculation needs the ticks elapsed so you need to subtract from the starting value

calm latch Dec 13, 2024, 7:31 PM

#

So -1

calm latch Dec 13, 2024, 7:31 PM

#

calm latch So -1

Ok I'm dumb that has worked

#

Thx

#

I think I have TSC

#

¿

#

Can it take 0ms?

#

@fiery turtle

gentle peak Dec 13, 2024, 7:42 PM

#

calm latch So -1

what do you write to ICR when you start the timer? the value you want is icr - ccr

calm latch Dec 13, 2024, 7:42 PM

#

I write -1, start the timer, and read its value

gentle peak Dec 13, 2024, 7:42 PM

#

ok yeah then it's UINT32_MAX - value_read_from_timer

calm latch Dec 13, 2024, 7:43 PM

#

So - what I've read should be number of ticks

#

Yes

#

Maybe I'm missing +1

#

Or -1

#

Anyway it's 662379999 662380000

#

And I'm using PIT for calibration...

#

So it probably doesn't matter

calm latch Dec 13, 2024, 7:44 PM

#

fiery turtle try increasing `UACPI_STATIC_TABLE_ARRAY_LEN`

Time to test this...

#

#

Does this mean I have SEVERE heap corruption?

#

(and power button didn't work sadmeme )

calm latch Dec 13, 2024, 8:10 PM

#

Could ACPI stuff be in bootloader reclaimable memory?

#

(LIMINE_MEMMAP_BOOTLOADER_RECLAIMABLE)

loud ice Dec 13, 2024, 8:15 PM

#

calm latch Could ACPI stuff be in bootloader reclaimable memory?

maybe

#

probably not tho

#

not with limine anyway

calm latch Dec 13, 2024, 8:18 PM

#

Because I'm reclaiming that

#

Started to recently

fiery turtle Dec 13, 2024, 8:52 PM

#

calm latch Does this mean I have **SEVERE** heap corruption?

yeah

fiery turtle Dec 13, 2024, 8:52 PM

#

calm latch Could ACPI stuff be in bootloader reclaimable memory?

thats unrelated

#

you're corrupting internal uacpi structures

#

that are on the heap

calm latch Dec 13, 2024, 8:53 PM

#

But that's weird

fiery turtle Dec 13, 2024, 8:53 PM

#

calm latch (and power button didn't work <:sadmeme:659848285510565907>)

probably needs an EC driver, just like all modern laptops

#

managarm has an example

calm latch Dec 13, 2024, 8:53 PM

#

My printf has been leaking memory and maybe that was masking it

#

(don't ask how)

fiery turtle Dec 13, 2024, 8:54 PM

#

bruh

calm latch Dec 13, 2024, 8:54 PM

#

Some quality code

#

I need to try static analyzers

#

Or dynamic ones

#

Although the only thing that should be allocating in between uACPI initialization is my PCI routing stuff...

calm latch Dec 13, 2024, 8:57 PM

#

calm latch My printf has been leaking memory and maybe that was masking it

Lol it still does it

#

I don't know how that came to be

#

I need to stop using copilot

fiery turtle Dec 13, 2024, 8:59 PM

#

indeed

fiery turtle Dec 13, 2024, 8:59 PM

#

calm latch

btw that AndOp error is a bug in your AML blob

#

they didnt account for osses without an EC driver

#

so if EC is not available it uses uninitialized variables

#

not critical but yeah

#

firmware dev brain rot

calm latch Dec 13, 2024, 9:00 PM

#

Does it calloc entries?

#

Because .data gets cleared by elf

#

By heap doesn't

fiery turtle Dec 13, 2024, 9:01 PM

#

If (ECAV)
{
    If ((Acquire (LFCM, 0xA000) == 0x00))
    {
        Local0 = BA1P /* \_SB_.PCI0.LPC0.EC0_.BA1P */
        Release (LFCM)
    }
}

If ((Local0 & 0x01))
{
    Return (0x1F)
}

It initializes Local0 only if ECAV, which is EC availablility

fiery turtle Dec 13, 2024, 9:01 PM

#

calm latch Does it calloc entries?

it mallocs them

calm latch Dec 13, 2024, 9:01 PM

#

And clears them?

fiery turtle Dec 13, 2024, 9:02 PM

#

lets see

#

waiiiiiiiiiiiiiiiiit

#

waiiiiiiiit

#

might be a uacpi skill issue

#

can u revert table array len

#

and test one thing

calm latch Dec 13, 2024, 9:03 PM

#

Yes

#

One second

#

How was it called?

#

Ok UACPI_STATIC_TABLE_ARRAY_LEN

fiery turtle Dec 13, 2024, 9:03 PM

#

ye

calm latch Dec 13, 2024, 9:03 PM

#

I reverted it

#

And I need to change malloc to calloc?

fiery turtle Dec 13, 2024, 9:04 PM

#

no

#

one sec

calm latch Dec 13, 2024, 9:04 PM

#

I've switched to git submodule btw

fiery turtle Dec 13, 2024, 9:04 PM

#

diff --git a/source/tables.c b/source/tables.c
index 7e8de51..a30b070 100644
--- a/source/tables.c
+++ b/source/tables.c
@@ -563,6 +563,7 @@ static uacpi_status verify_and_install_table(
     dump_table_header(phys_addr, hdr);

     uacpi_memcpy(&table->hdr, hdr, sizeof(*hdr));
+    table->reference_count = 0;
     table->phys_addr = phys_addr;
     table->ptr = virt_addr;
     table->flags = flags;
@@ -571,7 +572,7 @@ static uacpi_status verify_and_install_table(
     if (out_table == UACPI_NULL)
         return UACPI_STATUS_OK;

-    table->reference_count = 1;
+    table->reference_count++;

#

idk why i even bother running ubsan and asan

#

they dont catch shit

calm latch Dec 13, 2024, 9:05 PM

#

One sec I need to google how to apply patch

fiery turtle Dec 13, 2024, 9:05 PM

#

lol

#

just do it by hand

#

its 2 lines of code

#

green is new stuff red is removed stuff

calm latch Dec 13, 2024, 9:05 PM

#

Ok

hollow elm Dec 13, 2024, 9:05 PM

#

fiery turtle ```diff diff --git a/source/tables.c b/source/tables.c index 7e8de51..a30b070 10...

maybe also set MALLOC_PERTURB_=170 or something?

hollow elm Dec 13, 2024, 9:06 PM

#

fiery turtle idk why i even bother running ubsan and asan

er wanted to respond to this

fiery turtle Dec 13, 2024, 9:06 PM

#

hollow elm maybe also set `MALLOC_PERTURB_=170` or something?

is that like a thing you have to export at runtime?

hollow elm Dec 13, 2024, 9:06 PM

#

yeah

#

it's glibc-specific but it causes malloc to fill all fresh bytes with the given value

#

in this case 0xaa

fiery turtle Dec 13, 2024, 9:07 PM

#

hmm are u sure thats the name?

calm latch Dec 13, 2024, 9:07 PM

#

calm latch ¿

Btw is this plausible?

fiery turtle Dec 13, 2024, 9:07 PM

#

0ms is possible yes, but you're getting 5 million a second

#

thats highly questionable

hollow elm Dec 13, 2024, 9:08 PM

#

fiery turtle hmm are u sure thats the name?

       M_PERTURB (since glibc 2.4)
              If  this  parameter  is  set to a nonzero value, then bytes of allocated memory (other than allocations via calloc(3)) are initialized to the complement of the value in the least significant byte of value, and when allocated
              memory is released using free(3), the freed bytes are set to the least significant byte of value.  This can be useful for detecting errors where programs incorrectly rely on allocated memory being  initialized  to  zero,  or
              reuse values in memory that has already been freed.

   Environment variables
...
       MALLOC_PERTURB_
              Controls the same parameter as mallopt() M_PERTURB.

calm latch Dec 13, 2024, 9:08 PM

#

I have SSE in userspace 👉 👈

hollow elm Dec 13, 2024, 9:08 PM

#

man 3 mallopt

#

i did get the values wrong

#

it's ~0xaa when mallocing, then 0xaa after freeing

fiery turtle Dec 13, 2024, 9:09 PM

#

doesnt repro for some reason

#

with that variable

calm latch Dec 13, 2024, 9:09 PM

#

Ok I'm trying it

fiery turtle Dec 13, 2024, 9:09 PM

#

❯ MALLOC_PERTURB_=170 /mnt/d/uacpi/tests/runner/build-linux-64bits/test-runner dsdt.dat -x ssdt1.dat ssdt2.dat ssdt3.dat ssdt4.dat ssdt5.dat ssdt6.dat ssdt7.dat ssdt8.dat ssdt9.dat ssdt10.dat ssdt11.dat ssdt12.dat ssdt13.dat ssdt14.dat ssdt15.dat ssdt16.dat ssdt17.dat ssdt18.dat ssdt19.dat ssdt20.dat ssdt21.dat ssdt22.dat ssdt23.dat ssdt24.dat

calm latch Dec 13, 2024, 9:09 PM

#

(I have to restart laptop (which is obnoxious with windows 11 for some reason), disable secure boot, reboot it again so it sees my usb drive, and then do everything in reverse)

fiery turtle Dec 13, 2024, 9:10 PM

#

thats painful

#

anyways 99% confidence its a uacpi bug so i feel dumb

fiery turtle Dec 13, 2024, 9:11 PM

#

calm latch I have SSE in userspace 👉 👈

well

#

sure but u get like a 3.5x speedup vs fastest kernel here

#

so idk

calm latch Dec 13, 2024, 9:11 PM

#

Though the new time source is like 60% probability broken

#

Why does it always boot windows

fiery turtle Dec 13, 2024, 9:13 PM

#

these types of bugs are why most linux distros ship with unconditional zeroing of absolutely all allocated memory

#

(in the kernel i mean)

vast kestrel Dec 13, 2024, 9:13 PM

#

Yeah kernel devs just forget

#

In our embedded code we just define malloc to zero to not have a chance to forget to zero

calm latch Dec 13, 2024, 9:14 PM

#

I like to live dangerously, my kernel doesn't

#

(and userspace as well)

fiery turtle Dec 13, 2024, 9:14 PM

#

best bet for production

calm latch Dec 13, 2024, 9:15 PM

#

It worked

#

vast kestrel Dec 13, 2024, 9:16 PM

#

Speed

fiery turtle Dec 13, 2024, 9:16 PM

#

thanks for testing

#

sorry i blamed u

#

still cant repro this bug locally somehow

calm latch Dec 13, 2024, 9:16 PM

#

😎

fiery turtle Dec 13, 2024, 9:17 PM

#

that avg is insane if true, but i doubt it tbh

calm latch Dec 13, 2024, 9:17 PM

#

I'm gonna check get_nanoseconds_since_boot() against sleep

#

Which is definitely working and uses lapic

fiery turtle Dec 13, 2024, 9:18 PM

#

like even the log spam above it

#

takes way longer than 1ms

calm latch Dec 13, 2024, 9:18 PM

#

Why would it

#

It's asynchronous

fiery turtle Dec 13, 2024, 9:18 PM

#

oh then maybe not

#

--- a/tests/runner/interface_impl.cpp
+++ b/tests/runner/interface_impl.cpp
@@ -228,6 +228,8 @@ void* uacpi_kernel_alloc(uacpi_size size)
         return ret;

     allocations[ret] = size;
+    memset(ret, 0xFF, size);
+
     ret

#

i did this

#

but still not repo

#

repro

#

idk

calm latch Dec 13, 2024, 9:19 PM

#

With my laptop's tables?

#

Idk

fiery turtle Dec 13, 2024, 9:19 PM

#

it should work with any tables as long as there's more than static array len

calm latch Dec 13, 2024, 9:20 PM

#

Could it be a particular case when the pointer is null but the rest isn't?

fiery turtle Dec 13, 2024, 9:20 PM

#

ohhh wait yes

#

im retarded

calm latch Dec 13, 2024, 9:20 PM

#

Ok it booted back...

fiery turtle Dec 13, 2024, 9:20 PM

#

or actually no

#

bogus pointer value should segfault

calm latch Dec 13, 2024, 9:21 PM

#

Where does out table come from?

#

Ok I'm gonna remove the prints just to be extra sure

#

That I didn't boot an old iso or something

#

And also check nanoseconds stuff

fiery turtle Dec 13, 2024, 9:23 PM

#

ohhhh

#

its because i was using early_table_access

#

so my structures are copied from there on init

#

    static uint8_t early_table_buf[4096];
    auto st = uacpi_setup_early_table_access(
        early_table_buf, sizeof(early_table_buf)
    );
    ensure_ok_status(st);

#

this stuff hid the bug

#

/mnt/d/uACPI/source/interpreter.c:1259:23: runtime error: member access within null pointer of type 'struct acpi_dsdt'
/mnt/d/uACPI/source/interpreter.c:1260:32: runtime error: member access within null pointer of type 'struct acpi_dsdt'
AddressSanitizer:DEADLYSIGNAL
=================================================================
==1189==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000004 (pc 0x55bd8f039a97 bp 0x7ffc4f2720f0 sp 0x7ffc4f2720a0 T0)
==1189==The signal is caused by a READ memory access.
==1189==Hint: address points to the zero page.
    #0 0x55bd8f039a97 in prepare_table_load /mnt/d/uACPI/source/interpreter.c:1260
    #1 0x55bd8f039d04 in do_load_table /mnt/d/uACPI/source/interpreter.c:1272
    #2 0x55bd8f03cb19 in uacpi_execute_table /mnt/d/uACPI/source/interpreter.c:1539
    #3 0x55bd8f00c2a1 in uacpi_table_load_with_cause /mnt/d/uACPI/source/tables.c:1088
    #4 0x55bd8f01b4e5 in uacpi_namespace_load /mnt/d/uACPI/source/uacpi.c:395
    #5 0x55bd8ef6577a in run_test /mnt/d/uACPI/tests/runner/test_runner.cpp:486
    #6 0x55bd8ef67b81 in main /mnt/d/uACPI/tests/runner/test_runner.cpp:622
    #7 0x7f7517df3d8f in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58
    #8 0x7f7517df3e3f in __libc_start_main_impl ../csu/libc-start.c:392
    #9 0x55bd8ef5f4e4 in _start (/mnt/d/uacpi/tests/runner/build-linux-64bits/test-runner+0x19d4e4)

AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /mnt/d/uACPI/source/interpreter.c:1260 in prepare_table_load
==1189==ABORTING

#

there u go

#

this is the crash u were seeing

#

yup verified that fix works

#

will push shortly

calm latch Dec 13, 2024, 9:30 PM

#

The timer is broken

calm latch Dec 13, 2024, 9:30 PM

#

fiery turtle will push shortly

Yay

#

Another bug found

#

which is not my fault

#uACPI - a portable and easy-to-integrate ACPI implementation