#uACPI - a portable and easy-to-integrate ACPI implementation
1 messages · Page 19 of 1
But I think the expensive cases are past the benchmark
Because all aml buffers have a corresponding size field etc
It’s mostly to do with e.g. generate_absolute_path(), or HID strings or similar
forgot to respond to this, it probably is because no code is ever truly optimal but i can't see how. maybe there's a better non-simd memcpy and memset than rep movsb and rep stosb? most other things are pretty much as fast as i can get them while also still being thread safe and having all the features i want
the one thing that comes to mind is PCI access (currently i scan all ECAM ranges for every UACPI access) but i don't think that gets called by uacpi during the benchmark
Yeah don't think so
managarm phd memcpy 
Yeah I think its actually faster than rep
Although its in c++ lol
It does heavy unrolling which I guess really helps for small memcopies
this is what i mean by this btw, the intended usage is to call get_pci_config once during init and then use that handle (it's actually just a pointer into ecam) using the read/write methods which are all just one memory access. uacpi doesn't have the api to use that properly, so i need to do get_pci_config on every access
This is why I thought about introducing a uacpi_kernel_pci_device_open
Which would give back some sort of a handle
that'd be nice yeah
is aml structured in a way that'd make that worthwhile? or would it amount to open+access+close for every pci op?
Its only used for opregions so I can optimize this to open once actually
oh yeah then i think that'd be worth adding
Yeah definitely
I wanted that for uapi anyway
Will add tomorrow I guess, I have a draft PR open for opregion rework
Anyways very based work today, thx iretq and monkuous 
Im gonna go sleep now
Gn
gn
gn
i tested it just in case and wow that made a much larger difference than i thought it would.
- unsized frees, kfree gives empty pages back to pmm, page size is 4k: 4 582 951/s
- unsized frees, kfree gives empty pages back to pmm, page size is 16k (this is what's currently on github): 4 781 911/s
- unsized frees, kfree keeps empty pages, page size is 4k: 4 830 247/s
- unsized frees, kfree keeps empty pages, page size is 16k: 5 090 935/s
- sized frees, kfree gives empty pages back to pmm, page size is 4k: 4 996 014/s
- sized frees, kfree gives empty pages back to pmm, page size is 16k (this is what's going to be on github): 5 163 677/s
- sized frees, kfree keeps empty pages, page size is 4k: 5 112 014/s
- sized frees, kfree keeps empty pages, page size is 16k: 5 379 635/s
alright i pushed the change and updated the benchmark message
I just remembered that my pmm memsets memory too
could've gotten a few more points

yes sir
Also take qemu and underlying hardware characteristics into account to perfectly optimize for your stack
Damn
hmmm could anyone test this? I don't think much has improved though
qemu-system-x86_64 -serial stdio -m 512M -cpu max,migratable=off,+hypervisor,+invtsc,+tsc-deadline -M q35 -accel kvm -cdrom image-x86_64.iso
1359972/s
i love how this has turned into a competition and it totally dragged everyone away from actually working on their kernels
technically we are still working on kernels and trying to make them faster
i don't count hyperoptimizing your mem* and uacpi kernel api as actual work tho 
i mean u use mem stuff everywhere around the kernel right
so its a useful optimization
who knows
but yeah any work is useful work of course!
no one did a generalized benchmark before 
maybe we could do actual profiling when we replace acpica in linux
i know linux can do pgo
so there must be a good way to profile it
better way than sampling rip outside of qemu lol
true
yeah as far as implementing kernel api and initializing it goes
i did a submodule + link from drivers/acpi/uacpi/source to ../../../uacpi/source
1 sec
oh wtf
yeah it's a lazy copy i guess
its a gigantic repo
probably yeah makes sense
hacker@raptor:~/sources/linux$ git fetch uacpi master
error: cannot run ssh: No such file or directory
fatal: unable to fork```
what the fuck
HOW DO I NOT HAVE SSH
lmao
ah
and the lower one is the actual uacpi submodule
its a bit uhh
i didnt know how to properly organize it
but this keeps most of the actual uacpi stuff under drivers/acpi/uacpi
except for the submodule
is this really the intended way for drivers to add include paths?
highly doubt
actually hold on ill see if there isnt a better place for this
but idk the right way
looking at /include, it seems it might be, because headers for all drivers are there
so symlink uacpi/include/uacpi -> include/uacpi?
i guess
and put the two uacpi headers there maybe 
ye
do i just put uacpi_{types,libc}.h in include/?
It made me realize how shit my mutexes are and that I need to use a real lock
I guess
Once I swtich over I can probably break 4 mil on qwincis computer
i wonder if linux semaphores are just crap
i used linux's normal mutex type and i believe my result was similar to iretq's
so i doubt it's due to the semaphores
ok building again
the only uacpi-based optimization i've done that i can think of is the sized frees thing, other than that all improvement in proxima so far has just been due to allocator improvements i would've made anyway
@fiery turtle pushed, should be better now
big
next step is probaly putting it behind a config option
also, do you want to replace acpica completely or make it an option to choose between these two?
aight merged
lfg
nah, it would be a maintenance hell to support both, since linux doesnt abstract it away in any way unlike nt for example
2025 will be the year of the uacpi desktop
doesn it?
so only full replace is realistic
i thought drivers/acpi was a wrapper around acpica
its acpi-specific drivers that all work on top of direct calls to acpica api
oh ew
yeah i guess replacing it then is the only sensible option
think so
huh, when i compile proxima with -march=native -mtune=native i get a KVM emulation failure
that's unexpected
classic kvm error
do u do -cpu host?
yeah
hmm
and if i didn't i would've expected #UD instead of an emulation failure
depends
there are probably edge cases that cause a kvm error
like setting some msr bits that dont exist
idk
rip points to bextr %eax,0x10(%r12,%rbx,1),%edx
this can be caused by e.g. instructions that KVM cannot emulate doing MMIO
O3 doing funny stuff?
We need uDRM urgently
do u have special compile flags that cause that?
im not the author of udrm sooo
don't think so, my flags are just -march=native -mtune=native -O3 -DNDEBUG -std=gnu11 -flto=thin -fno-common -Wall -Wextra -ffreestanding -fno-pie -fno-stack-check -fno-stack-protector -mcmodel=kernel -mgeneral-regs-only -mno-red-zone
So that every hobby kernel suddenly gets good PCI infrastructure
yeah i'll probably do that
in your kernel?
linux eventually gets into a function that does a switch on a width and volatile loads or stores
there isnt assembly there
or well on x86 it does actually
my bad
#define build_mmio_read(name, size, type, reg, barrier) \
static inline type name(const volatile void __iomem *addr) \
{ type ret; asm volatile("mov" size " %1,%0":reg (ret) \
:"m" (*(volatile type __force *)addr) barrier); return ret; }
#define build_mmio_write(name, size, type, reg, barrier) \
static inline void name(type val, volatile void __iomem *addr) \
{ asm volatile("mov" size " %0,%1": :reg (val), \
"m" (*(volatile type __force *)addr) barrier); }
build_mmio_read(readb, "b", unsigned char, "=q", :"memory")
build_mmio_read(readw, "w", unsigned short, "=r", :"memory")
build_mmio_read(readl, "l", unsigned int, "=r", :"memory")
build_mmio_read(__readb, "b", unsigned char, "=q", )
build_mmio_read(__readw, "w", unsigned short, "=r", )
build_mmio_read(__readl, "l", unsigned int, "=r", )
build_mmio_write(writeb, "b", unsigned char, "q", :"memory")
build_mmio_write(writew, "w", unsigned short, "r", :"memory")
build_mmio_write(writel, "l", unsigned int, "r", :"memory")
build_mmio_write(__writeb, "b", unsigned char, "q", )
build_mmio_write(__writew, "w", unsigned short, "r", )
build_mmio_write(__writel, "l", unsigned int, "r", )
idk what the switch was for i cant find it
this should probably be made into a kernel api then
it used to be
or it should use inline asm within uacpi
uacpi would have to provide inline assembly for all platforms
and i think thats what infy doesn't want to do
true and real
why not? how many supported platforms are there?
Only 2 real platforms exist anyways anything else is mental illness
*3 
Which platforms I mean is left as an exercise to the reader
seems like it works fine rn, but apparently compilers just use non-mmio instructions for volatile ptrs whenever they feel like it, so it seems a bit fragile
ACPICA does it like this, I think i should be fine
fwiw i don't think volatile guarantees that the compiler doesn't use insane access modes that the memory controller doesn't support
x86, arm, itanium
no riscv?
uacpi itself is completely platform independent, no?
riscv yeah
it is
based
so im definitely not adding arch specific stuff tehre
its not aware of what arch its executing on
then i'd suggest to turn this into a kernel API
maybe you can make it a weakly linked kernel api and provide a default impl that does volatile?
uacpi_kernel_system_memory_read
then again it's not a particularly hard kernel api to implement
most osses will just do this volatile switch
so probably fine to have a default
idk
@fiery turtle whats next for uacpi-linux?
anyway proxima now only uses volatile pointers for framebuffer and uacpi so that's nice
code drivers/acpi/*.c then manually edit acpica calls and replace with uacpi
first probably bus.c and more important drivers
i guess im doing the thing that korona suggested now
every read and write, be it io, pci, or mem is done on a previously mapped handle instead of directly
in case of virtual memory ofc the handle is expected to be the actual virtual address tho
for non-aml io
like tables etc
makes sense
oh wow the -march=native -mtune=native actually had a noticeable impact on speed, i didn't expect that (5.1M/s -> 5.3M/s)
because it works everywhere atm
interesting
ok i guess i'll take a look at drivers/acpi/*.c then
converting driver code sounds like pain
i mean i'm not sure what i'll do but i'll take a look at the very least
its mostly convert acpi_x to uacpi_x
some are literally like that, some are harder probably
converting the types everywhere is gonna be fun
but yeah i see what you mean
its mostly just tedious work
yup
but careful so its annoying
also they have patterns where they create an object on the stack then shove it into a method as an argument
thats just not something u can do safely
Can't you make a acpica compat lib on top of uacpi?
you can of course, technically
it would be very slow and insane
like implementing dx12 in terms of vulkan
every struct would have to be converted, a lot of hidden allocations, etc
i dont think an abomination like that would be upstreamable ever
thats why e.g. NT draws the line of abstraction higher
so it expects acpi.sys to export more higher level helpers, that don't have shit like eval_aml()
is this due to my lack of an EC driver orr
oh wow that quality is terrible wtf
whatever
can I have your blob
lmao
what's the command for that again
acpidump > dump.txt
do u install a fixed event handler?
try pmos 
so far this looks bogus af lol
like a handler was uninstalled midway
switch (event->handler_type) {
case GPE_HANDLER_TYPE_AML_HANDLER: {
uacpi_object *method_obj;
method_obj = uacpi_namespace_node_get_object_typed(
event->aml_handler, UACPI_OBJECT_METHOD_BIT
);
if (uacpi_unlikely(method_obj == UACPI_NULL)) {
uacpi_error("GPE(%02X) AML handler gone\n", event->idx);
break;
like i cant even imagine how to trigger this error
no, right now i don't do anything with uacpi except the init sequence
what's the repo for that so i can compile it
no i mean the iso lol
#1236769772805554206 message
the fact that u even get a fixed event here is bogus, because its explicitly disabled by uacpi init code
i wonder whose bug it is 
PMOS hangs on a blank screen at boot
inb4 some uacpi skill issue again
nooo lol
bruh
what the fuck
this is why u dont make a recursive aml interpreter
this is all encoded as part of 1 opcode
i wonder if the issue is fixed if i add a power button handler
probably, I still dont understand the AML handler gone thing
any way u can enable tracing as well?
yeah 2s
thats just uacpi_context_set_log_level before doing anything with uacpi right
anywhere yeah
here should be fine
sure
does it say VALZ, 0x8E there? the quality is really bad lol
right I'm gonna have to delay every print because this just scrolls past all useful info before it can be recorded
it does
I think my framebuffer driver is seriously broken
and yeah idk what happened to the quality, it's fine locally on my phone so something happened during upload to discord
hmm so the GPE handler for _L02 does fire once, but then it just somehow gets corrupted
wait
fuck
--- a/source/event.c
+++ b/source/event.c
@@ -445,7 +445,6 @@ static void async_run_gpe_handler(uacpi_handle opaque)
uacpi_status_to_string(ret)
);
}
- uacpi_object_unref(method_obj);
Can u do this real quick
btw this is like 100 memory reads just to evaluate this while
least insane AML
the cool news is your blob seems to be old school in that it does notifications via GPE events and not the embedded controller
u can try plugging/unplugging the charger and might see something
same goes for lid stuff
alright uploading the trace log on youtube now
did u apply the patch?
this laptop doesn't have a battery (it was Expanding a few years ago so it is now Gone from my home)
ah well
when i recorded the video no, i'm applying and testing the patch now
the lid u can still open and close
i refactored something in a retarded way so it was a UAF at best
oh yeah u see those GRSV reads? thats the while being evaluated lmao
the patch seems to have fixed it
each of those later ignoring firmware ... prints is one press of the power button
what about the lid
uacpi: warn: ignoring firmware Notify(\_SB_.LID_, 0x80) request, no listeners
uacpi: warn: ignoring firmware Notify(\_SB_.VALZ, 0x8F) request, no listeners
uacpi: warn: ignoring firmware Notify(\_SB_.LID_, 0x80) request, no listeners
oh damn
even lid works
0x80 is status change so its correct
if u hook up notifications to the lid device u can now detect when the lid is closed or opened
awesome
whats strange is fixed event 2 doesnt fire after its disabled again by uacpi
but literally all events are disabled on init
i wonder if your firmware toggles it
lmao
with that 100 conditions while i wouldnt be surprised
but its a cosmetic problem so whatever
in a real os you would ofc have a power button handler installed
(which is fixed event 2)
maybe they just havent tested with that event disabled lmao
so the firmware ignores the enable bit
maybe its not, usually there's a timeout between how often power button presses can be generated
as to not confuse the host os
if u wait a bit it might get fired again
but im glad you tested on your laptop, very few actually use this notification mechanism so it was barely tested
its an old one right?
tried waiting 20 seconds between presses and still didn't fire the event again so
guess its my former guess then
pretty old yeah, portege r30-a, came with windows 8 (not 8.1) so i believe its from 2012
or maybe need to wait longer :^)
cool cool
Device (VALZ)
{
Name (_HID, EisaId ("TOS6208")) // _HID: Hardware ID
Name (_DDN, "VALZeneral") // _DDN: DOS Device Name
I wonder what this VALZ thing is that keeps getting notified
ACPI-COMPLIANT VALUE ADDED LOGICAL AND GENERAL PURPOSE DEVICE
proprietary chipset specific crap
ACPI compliant == horribly proprietary and probably not compliant
huh, i just noticed, linux complains about the aml during init
ACPI Error: AE_NOT_FOUND, While resolving a named reference package element - \_SB_.PCI0.EHC1.HUB0.RMH0.PRT6 (20240827/dspkginit-438)
is this an acpica skill issue?
anyway this firmware's pretty fucked regardless. if the device loses power or it's force powered off (aka power button held down while in acpi mode) it seems to triple fault during POST
yeah
nice
ill cherry pick the bug fixes & windows clang support i guess
since that opregion extension stuff is gonna take a while
waiting for large tests to finish then ill merge
next on the list:
- uacpi_kernel_pci_device_open
- make pci r/w use the handle
- optional
uacpi_kernel_memory_{read/write} - new address space support
oh right i don't think i ever notified you, the acpi dumps of that laptop and my main dev machine are in https://github.com/UltraOS/ACPIDumps/pull/3
Oh damn, really cool, thanks for improving the collection
how did you come up with these ids?
i think they were somewhere in the hw-probe output?
yeah they're the value of HWid
ah
alright merged the bugfixes for GPE handlers and clang for windows builds (cc @rustic compass )
merged this as well, thanks a lot
removes raw memory helpers
adds raw memory helpers back, rebranded
there's a huge difference
their purpose was performing io on a physical address
the new ones will be for MMIO abstraction
and they will have a default implementation
(and obviously work on virtual addresses instead)
time to bump my uacpi commit and hope nothing goes to shit
barely anything changed lol
reminder, this is obos
ok it boots on qmeu, I couldn't be bothered to test anything else atm
@north holly thoughts on this aml
While ((GP50 || (GP51 || (GP52 || (GP53 || (GP54 || (
GP55 || (GP56 || (GP57 || (GP60 || (GP61 || (GP62 || (GP63 ||
(GP66 || (GP67 || (GP70 || (GP71 || (GP93 || (GP94 || (
GPAP || (GPBP || (GPCP || (GPDP || (GPEP || (GPFP || (GPGP ||
(GPHP || (FCDP || (FTVP || (FQSP || (GP96 || (GP97 || (
GPD0 || (GPD1 || (GPD2 || (GPD3 || (GPD4 || (BPFE || (TPBP ||
(ECDS || (B1ST || B2ST)))))))))))))))))))))))))))))))))))))))))
fent
if you make a recursive aml interpreter the call stack would be
parse_logical_or
parse_logical_or
parse_logical_or
..
parse_logical_or
parse_if
pretty confident this would exhaust any kernel stack
@pine yew FYI, here's a real hw example
exception thrown: ur aml is dogshit (so is your interpreter) and it caused your interpreter to stack overflow
was that some random guy who wanted to make an AML interpreter
This is the person behind the rust ACPI crate
ah
whats interesting is monkuous' hw also tells you the physical location of every xHCI port
yeah because that's obviously needed for power mangement
Nice lmao
what machine did that aml come from
monkuous' laptop
someone should lose their job for that
seeing that in the wild does make me lol at the recursive approach yeah
if NT's interpreter was less resilient the firmware devs wouldn't be able to get away with that dogshit though
true
rewriting the rust crate is on my todo list after feb-ish
big
fein
i wanna die from this aml
thanks
❯ grep -rni '= Timer' -l
Convertible/ASUSTek Computer/Q325/Q325UAR/7E4A9E65FDE9
Convertible/ASUSTek Computer/ZenBook/ZenBook UX563FD_UX563FD/4965BD15F4B6
Convertible/Dell/Latitude/Latitude 7400 2-in-1/5DA0C196CB26
Convertible/Hewlett-Packard/ENVY/ENVY x360 2-in-1 Laptop 15-ew0xxx/211A1085E85B
Convertible/Hewlett-Packard/ENVY/ENVY x360 Convertible 15-dr1xxx/6FE4CE9270F1
Convertible/Lenovo/Yoga/Yoga C640-13IML 81UE/91A793680B40
Notebook/ASUSTek Computer/ASUS/ASUS TUF Gaming F15 FX506HC_FX566HC/2418D4F13406
Notebook/ASUSTek Computer/VivoBook_ASUSLaptop/VivoBook_ASUSLaptop K6500ZE_K6500ZE/674A4C791E8B
Notebook/ASUSTek Computer/ZenBook/Zenbook UX5401ZAS_UX5401ZAS/297345551BD5
Notebook/Avell High Performance/A70/A70 MOB/16859F37F144
Notebook/Dell/Latitude/Latitude 5511/A37FB9368F2A
Notebook/Dell/Precision/Precision 3571/AD37470CEC0D
Notebook/Dell/Precision/Precision 7550/FE48AAC0D405
Notebook/Gigabyte Technology/AORUS/AORUS 15G KB/A7DC3F8318A8
Notebook/Hewlett-Packard/EliteBook/EliteBook 840 G7 Notebook PC/05F5CA53EEBB
Notebook/Hewlett-Packard/EliteBook/EliteBook 850 G8 Notebook PC/5BEFC8C16D82
Notebook/Hewlett-Packard/ENVY/ENVY Laptop 13-aq0xxx/A34009921DC1
Notebook/Hewlett-Packard/ZHAN/ZHAN 99 Mobile Workstation G1/CB1EEAC36F91
Notebook/HUAWEI/CREF-XX/CREF-XX/651E7E8E95D2
Notebook/Lenovo/IdeaPad/IdeaPad L340-15IRH Gaming 81LK/5B5E032BF08D
Notebook/Lenovo/Legion/Legion 7 16IAX7 82TD/23401686E604
Notebook/Lenovo/Legion/Legion Y540-15IRH-PG0 81SY/6658FB40979F
Notebook/Lenovo/Legion/Legion Y7000 81FW/814C5DAC1BEE
Notebook/Lenovo/ThinkPad/ThinkPad E15 Gen 2 20TD0005MH/188EB681251A
Notebook/Lenovo/ThinkPad/ThinkPad L15 Gen 2 20X4S27200/CCA34DA7A4E9
Notebook/Lenovo/ThinkPad/ThinkPad P1 Gen 4i 20Y4S0KS00/4012A98BFAA3
Notebook/Lenovo/ThinkPad/ThinkPad P15s Gen 2i 20W6001VUS/C49AF6F23385
Notebook/Lenovo/ThinkPad/ThinkPad T14s Gen 1 20T1S2WF00/35B0F227248A
Notebook/Lenovo/ThinkPad/ThinkPad T490s 20NX000DRT/14F305E2ED3B
Notebook/Lenovo/ThinkPad/ThinkPad X1 Carbon Gen 8 20UAS89500/282408B7FDEC
Notebook/Lenovo/ThinkPad/ThinkPad X1 Carbon Gen 9 20XW007YPB/C06A0E31B5D6
Notebook/Lenovo/Yoga/Yoga Slim 7 ProX 14IAH7 82TK/BD603C52BF3B
Notebook/Notebook/NLx0/NLx0MU/17E69D6CB4D0
Notebook/PC Specialist/NS50/NS50MU/30E58AFD506A
Notebook/Teclast/F15Plus/F15Plus 2/5D2F5E56BCAB
Notebook/Timi/RedmiBook/RedmiBook Pro 15S/3509D1F696E7
Tablet/Microsoft/Surface/Surface Pro 3/074C5C2E091B
Stats about AML u didnt care about
about 10% of blobs use TimerOp
Notebook/Dell/Precision/Precision 3571/AD37470CEC0D:103573: Debug = Timer
Notebook/Dell/Precision/Precision 3571/AD37470CEC0D:103623: Debug = Timer
Notebook/Dell/Precision/Precision 3571/AD37470CEC0D:139046: Debug = Timer
Notebook/Dell/Precision/Precision 3571/AD37470CEC0D:139749: Debug = Timer
Some literally spam it to stdout
@fiery turtle uacpi-rs builds now on uacpi master branch, you just gotta ignore the billion warnings 
lemme guess, the stupid uninitialized struct thing?
wow linux w/uacpi is actually in the works now???
it boots but there's still thousands of acpica api calls that must be changed
i will have roughly 5 days of free time in the semester break
i would love to try uacpi linux on my hardware
and maybe fix a few bugs
what else would it be using
idk something in house
uacpi ofc
like, we just implemented kernel api and called uacpi_initialize lol
its not that big of a deal
all drivers are still calling into acpica
they work alongside
I wonder if uacpi linux will fix the bug where my friend's laptop starts smoking 
oh but that simply means the work was done way before this with uacpi's kernel api being similar to acpica i think
i wonder if the acpi errors on boot on my laptop is a linux/acpica issue or broken fw
well if u give me the dump and errors i can tell u
i do not remember the errors at all
ah sad
wow that's interesting
what affects the performance btw?
other than hardware
locking and stuff?
and allocation
yeah, locks, mem* functions, allocator performance
i think best we got out of it is 3.4M ops/s, whereas e.g. proxima gets 5.2M at peak
turns out a complex project is not as fast as a really simple one
i have always wanted to host a build server
remind me in a year
to set up something
and benchmark homebrew oses
on every commit maybe
or every release if they do releases
I got proxima to 5.398M by making my heap leak memory (ish, since new allocations can still use the "leaked" memory, it's just not returned by the pmm)
unfortunately I could not get it to 5.4M
what I mean by "leaking memory" is pretty much the same thing most heaps do
most heaps never shrink and return memory to the os (pmm in this case), only ever expanding
that's what I mean here
ah
but returning memory to the pmm is a property I'd really like to have, which is why I'm not removing the shrinking and why I'm not counting that result or submitting it for the leaderboard
fair
Mostly
frigg outperforms linux, confirmed 
yeah frigg + phd memcpy do a solid job
Year of managarm desktop soon
yes
damn @fiery turtle i get a weird crash in uacpi and i'm not sure why, you think you could help me figure it out? i'm pretty out of date but i don't think that matters, i crash on interpreter.c:66 which is c DYNAMIC_ARRAY_WITH_INLINE_STORAGE_IMPL( op_context_array, struct op_context, static )
and the cr2 is 0x905cf which is even more confusing lol
allocator corruption probably
alignment bug?
size = align_up(size, alignof(max_align_t));
nyaux had a weird crash in dynamic array release was related to that
logging is probably not related to that
i hope my prints will reveal something
yeah i know
i'm just saying that i apply it project-wide
ye
also if u have the entire backtrace that would be useful
and inspect the array state with gdb as well
i don't do backtraces unfortunately, sorry
i'' hack that together if prints don't reveal anything
just connect gdb lol
btw that doesnt guarantee anything
well prints don't really tell me much so i'll attach gdb
is your bump allocator storage just a char array?
ah
ill try a static array
one crash we had was because zig was adding extra alignment checks to c code lol
via some clang flags or whatever
nah its not that, that was dying on a ud2
ah i see
yeah zig likes to do that
also i think it might have been the allocator lol
using a static char array it gets much further it seems
slow as balls because i print in every kernel api
how do u fuck up a bump allocator 💀
lmao
maybe my memory manager is fucked up?
oberrow had this problem as well in rewrite 22
wouldn't be surprised
i never tested it properly
also uacpi is freeing lots of 0x0's
that normal?
yeah im on an old commit
damn, this is like really old
what was the bug?
¯_(ツ)_/¯
just fixed itself?
im running with the shitty bump™️
and that works
so i'll investigate
but first uacpi update
no no i replaced my "proper" bump with one that uses a static array
it's probably something funky going on with my memory mnaager
or maybe my bump just has some stupid bug lol
nah it's not
pmm bug but i will investigate in a little bit
currently fixing up kernel api
i dont even know what changed since that commit
btw u even have realloc, u actually use that api in the kernel?
also ticks->ns ☠️
💀
i dont think so
linux has like 50 i think
in the entire kernel
compare to idk how many normal kmalloc calls
or no, 135 files vs 2684
FACS 0x0000000007FE0000 00000040
successfully loaded 1 AML blob, 1705 ops in 6ms (avg 277825/s)
unable to install SCI interrupt handler: unimplemented```
wow that's shit
nope kvm lmaoo
💀
literally how ☠️
😃
is your hhdm mapped cached?
so uncached?
but getting rid of assert(bump_offset + size <= PAGE_SIZE_L2); in alloc gets it up to 580k lol
is that built in debug mode or release mode
im pretty sure debug
inb4 6 mil
lmao
hacker@raptor:~/sources/osdev$ objdump -d build/kernel | grep -B 10 ffffffff8001c6cc
ffffffff8001c6c1 <_write_cr0>:
ffffffff8001c6c1: 0f 22 c7 mov %rdi,%cr0
ffffffff8001c6c4: c3 ret
ffffffff8001c6c5 <_write_cr3>:
ffffffff8001c6c5: 0f 22 df mov %rdi,%cr3
ffffffff8001c6c8: c3 ret
ffffffff8001c6c9 <_write_cr4>:
ffffffff8001c6c9: 0f 22 e7 mov %rdi,%cr4
ffffffff8001c6cc: c3 ret```
the crash rip points at the ret in _write_cr4
it's a pf with e=0000
how does writing to cr4 generate a page fault
what did u do 💀
then the cr4 write probably changed the expected page table format
inb4 NX kernel .text
whats the cr2 value tho
oh u either changed the PT format or NXed yourself
do u have uninitialized variables?
and do u read it before updating
this _write_cr4 is faulting
and it doesn't set any of these bits above, i checked
OH WOW IT'S TRYING TO WRITE 0
hmm
ok, so i print cr4 after reading it initially, after writing it the first time and before writing it the 2nd time
i get 0x20, 0x220 and 0x0??
that is some mad stack corruption i am smelling
cpuid corrupting the stack?
yueah
it actualy is
printf("before cpuid cr4: %llx\n", cr4);
_cpuid(0x7, 0, &cpuid);
printf("after cpuid cr4: %llx\n", cr4);
before cpuid cr4: 220
after cpuid cr4: 0```
that is crazy
wait you arent preserving any regs there, are they all scratch?
lol
i have a feeling debugoptimized with lto is not gonna be fun to debug
2. Clobbering of %ebx: The cpuid instruction modifies the %eax, %ebx, %ecx, and %edx registers. If this code is expected to adhere to System V AMD64 ABI calling conventions, keep in mind that %rbx is a callee-saved register. If %rbx is used in the caller, it should be preserved. You might need to save and restore %rbx around the cpuid instruction.
Idk chat gpt said this
lmao
is this true?
Functions preserve the registers rbx, rsp, rbp, r12, r13, r14, and r15; while rax, rdi, rsi, rdx, rcx, r8, r9, r10, r11 are scratch registers.
hmmm
i might be dumb
looks like u should preserve it
np lol
nice
somehow 3 times slower than proxima with everything implemented
probably due to "shitty" mem* functions
oh yeah
yeah they have very optimized mem
just tested, proxima with uacpi's builtin mem* funcs is 3.4M/s
HOOOOOOOOOW
WHAT
successfully loaded 1 AML blob, 1705 ops in 0ms (avg 2884452/s)
with inline asm memcpy/memset
just need a fast cpu :^)
successfully loaded 1 AML blob, 1705 ops in 0ms (avg 3045507/s)
random improvement because rerun
this largely assumes my time keeping is good™️
Who said built-in stuff was crap 
-# you
(I did)
(it is)
Lmao
if i replace #define memset __builtin_memset with size_t memset(void *dest, int value, size_t n); it goes down to 2.8M/s
I dont understand what it is you do that it's so fast when iretq literally has everything stubbed AND fast mem AND a bump allocator
ok when i initialize uacpi after i switch to and from a test thread that sleeps for 1s i get consistent 3.02M
mostly because calloc doesn't use uacpi builtins
Maybe bump allocator is worse because cold cache
and if i use uacpi_memset for calloc it goes back up to 3.4M/s
wtf?
Uacpi has PhD memset
calling a function that uses rep stosb is apparently slower than calling uacpi builtin memset
probably due to LTO and clang turning it into PhD memset
Lmao
because calling rep stosb can't be LTO'd since that's implemented in asm
Makes sense
Whats needed to support LTO in a kernel
nothing
well, patience when compiling
and not having UB
but other than that nothing
Hmm
I should try astral with it
it's great
LTO is basically just that the compiler outputs IR instead of machine code
uncovered the first bug in my kernel 👍
how could it be? LTO is done by compiling c/c++ into ir and writing the ir into object files
and that IR is turned into machine code by the linker which has global knowledge
yeah i didn't know
assembly can't be turned into IR because that's already effectively machine code
i thought it was some linker magic™️
Wait so if I have standalone asm files I cant lto anything?
well, it is, but not as magic as i thought 
can but not the symbols in them
lto only affects the c/c++ files built with lto
you can still LTO but if you call an asm function it won't be inlined
that kinda thing
just like you can lto a program that calls a shared library function

the github source seems outdated but from what i'm seeing there the main slowdown might be that you implement things like pio and mmio in standalone asm files instead of with inline asm
That memory needs some empathy
void*
alloc(size_t size)
{
size = align_up(size, alignof(max_align_t));
void* ptr = (void*)(bump_buffer + bump_offset);
bump_offset += size;
memset(ptr, 0, size);
return ptr;
}```
Inline asm?
how can that be at fault :(
I am pretty sure inline asm can be inlined
if it were inline asm it could be lto'd yeah but mine are in a standalone .S file
Then don't :^)
Also something something small and unaligned buffers being slow with the string instructions on older machines
not quite the same thing but if i make kfree a no-op proxima goes from 5.1M/s to 3.5M/s
and if i make the allocator unconditionally zero memory it goes from 5.1M/s to 4.5M/s
if i do both it goes from 5.1M/s to 3.1M/s
so stuff like that is pretty significant
Bruh enabling lto in astral gives a missing popcount refence
Ill look into that in a bit
Makes sense
I guess uacpi has a pattern of freeing and reallocating quickly
Which makes memory reuse fast
Bruh
.global memcpy
.type memcpy,"function"
memcpy:
cld
push %rdi
mov %rdx, %rcx
rep movsb
pop %rax
ret
.size memcpy, .-memcpy
.global memset
.type memset,"function"
memset:
cld
push %rdi
mov %rdx, %rcx
mov %sil, %al
rep stosb
pop %rax
ret
.size memset, .-memset
anything wrong with these?
cld is unnecessary since sysv abi requires it upon function call
fair
and the pushing can be replaced with copying to a scratch register that movsb/stosb don't touch
which removes an extra memory access
you can just mov rdi to rax at the start
these are mine for reference ```
memcpy:
mov %rdi, %rax
mov %rdx, %rcx
rep movsb
ret
memset:
mov %rdi, %r8
mov %rsi, %rax
mov %rdx, %rcx
rep stosb
mov %r8, %rax
ret```
i am getting weird uacpi errors
You seem to know your optimizations 
semantically those should be fine yeah
FACS 0x0000000007FE0000 00000040
unable to lookup named object 'W1�N' within (or above) scope '\_SB_.PCI0': invalid AML name string
aborting table load due to previous error: invalid AML name string
#0 in \()
failed to load 'DSDT' (OEM ID 'BOCHS ' OEM Table ID 'BXPC '): invalid AML name string
loaded 1 AML blob in 7ms (1 error)```
like wha
W1�N yum
bringing the old C versions of those functions makes it work again
ill try one at a time
Any reason why u do a separate file and not inline
honestly idk
probably doesn't matter since the usages are via __builtin_X and the actual functions only get called if it wouldn't have been inlined anyway
i could try and see if it'd be better with inline asm
lol it works now? wtf
I wonder how the perf would change if every memcpy call is inlined to that asm
right now i also have memcmp implemented in ASM (though not using rep cmpsb ofc) and i wonder whether a C implementation of it would perform better
What does that do
Oh
so i had a bunch of random code "commented out" with semicolons
Lol
lto does not like my pmm
wait I think it might be due to flanterm defining a memset too
idfk
the performance is Exactly The Same!
Exactly The Same™️
your memset prototype is incorrect. i don't think it matters in this case since the call convention doesn't change but value is supposed to be int not ulong
the size is still 64 bit
since it's size_t
value is supposed to be truncated to the lowest CHAR_BIT bits by the actual memset anyway
The second param is what to memset to 💀
Wait I had a brainfart nvm
Lmao
My guess from it dying that early is something relating to the memset and the page struct not being zero
wut
I don't believe I have ever fucked up a bump allocator
why is your memset taking an unsigned long?
memset is supposed to truncate the given value to a byte and fill the range with that so an int is more than adequate
I didnt look at the type when I implemented it but its literally uust changinng to an int
Lmao
No I remembered u fucking it up lol
you are doing WHAT to nyaux?
WHAT
what?

what
+#ifdef UACPI_NATIVE_CALLOC
/*
* Allocate a block of memory of 'count' * 'size' bytes.
* The returned memory block is expected to be zero-filled.
*/
void *uacpi_kernel_calloc(uacpi_size count, uacpi_size size);
+#endif
Fuck it
Less potential host bugs to worry about
i'll just have a default implementation in terms of alloc+memset
how about uacpi_kernel_calloc_zeroed? lets be honest, there will be at least one person who forgets to memset to zero otherwise (probably still at least 1 person even if its in the function name)
also is const uacpi_char *uacpi_status_to_string(uacpi_status); backed by a predefined list of names for all status variants?
found the big switch statement
not needing to cast c strings into rust strings makes it so much easier
calloc by definition zeroes the memory
calloc is always zeroed
also the comment says "The returned memory block is expected to be zero-filled."
if you don't read the comments it's your fault
:)
like calloc is the classic C/posix api for zeroed allocation
so calloc_zeroed is kinda redundant lol
in hindsight i should've just called it alloc_zeroed of course
but i thought calloc was apparent enough
i only knew the zeroed part from the comment but im also coming from zig without much of a direct C background
but the comment was there so it was fine
yeah my fault for assuming everyone knows C or has that background
imo that - in osdev especially - if people cant read the comments/docs thats their problem. maybe im just tired of fielding questions in other places from people who wont bother to read the intel x86 manual though
fair i guess
i think my calloc shim actually does something that im not 100% sure is required that a direct alloc(count * size) + memset wouldnt which is alignment
calloc in theory requires fundamental alignment rather than arbitrary alignment
and im not sure if normal alloc does
i think malloc is just guaranteed to return whatever the maximum alignment needed for fundamental types
which is enough for most cases
yeah i think same for calloc but i didnt see it in the malloc bit lol
how did you make it optional?
macro
fair
yeah just ifdef
#ifndef UACPI_NATIVE_ALLOC_ZEROED
void *uacpi_kernel_alloc_zeroed(uacpi_size size)
{
void *ptr;
ptr = uacpi_kernel_alloc(size);
if (uacpi_unlikely(ptr == UACPI_NULL))
return ptr;
uacpi_memzero(ptr, size);
return ptr;
}
#endif
could you maybe also add a uacpi_realloc kernel api that's optional? some allocators could make use of the fact some allocations can be resized without actually resizing but idk if it's worth the hassle
yep that might be nice if its ever used
the only place it could really be used is dynamic_array but yeah
Yeah it could be utilized by the dynamic array for sure
Wonder how many kernels have a realloc
i do
(Here)
Nice
so my zerod alocator will be faster

i will have one
Didnt u provide a calloc before anyway lol
yeah but it just returns alloc(size * n)
Whats the diff?
well technically zig does - zig's std provides a general-purpose allocator that just needs a backing page allocator and which has a resize function, and then the zig std provides a general realloc based on resize (resize returns true if it could resize in place)
wait is that def not what I'm thinking
Like calloc already allocates zeroed memory
Its just funny api because count and size
Oh interesting
Is it well suited for kernel use?
oh
well add realloc because it would play very well with my allocator 
I hear you 
I guess scores might change if I add a realloc
I mean slab free and alloc is o(1)
Dynamic array resize is what might be tanking some
at least on the small slabs in astral
promixma about to snap 7M
its not the absolute most high-perf thing out there but its still very good. even lets you swap out the mutex type it uses for thread safety which i do for my kernel one. im not sure all the details of what exact scheme it does other than being bucket-based. also in releasesafe and debug it can automatically detect double frees and some leaks
Lmao
Inb4 monkuous does some 4 line of code optimization for realloc that pumps out 10M/s
Nice
switching my mutex to a spinlock acquire like doubles performance so once I get a real lock I will be cooking
Zig is cool ill try it once its more stable

better try now before they fuck it up even more
It is genuinely a fairly competent language thus far, and I have fairly high hopes
Lmao
im totally not salty
What did they do
Recently they disabled emitting dynamic binaries in freestanding targets, which was a little frustrating but i hope this is merely a hiccup
the zig generalpurposeallocator im using is used instead of malloc on basically every zig program that doesnt link libc or have more specific needs (theres also an arena allocator and object pools in the std that have the same backing allocator setup. even a fixed buffer allocator and a fixed buffer with fallback allocator for overflowing the buffer) - zig just links ntdll or does manual syscall stuff on linux to get pages allocated as needed
var variables that aren't mutated are a compile time error, and unused variables are also errors which is definitely annoying
Niice
and you cannot turn that off lol
Sounds really tempting
technically zig also has heapallocator for the windows heap api since thats in ntdll but most people just use generalpurposeallocator on top of the page allocator
I also love zigs multiline strings, I haven't seen a language that does it better
tbh i like the unmutated var and unused const warnings
Honestly i dont really mind this
also comptime rules
what is cool is you can use std.fmt.comptimePrint + asm :)
and you can literally write code that generates inline assembly
comptime being what if constexpr was taken to the extreme
Yeah it's crazy
if you ever do decide try zig out for a kernel project i am more than sure i'd contribute
Also im biased because zig used spaces
thats how i capture interrupt stack frames actually. each field in the saved registers struct is named after a register and ive got comptime generating the pushes and pops for my interrupt handlers
damn that is nice :^)
once we get @unrollArgumentTuple ill be really happy, rn ive got to implement it with a bit of a comptime mess
Damn
i only used it for grabbing the cs offset from the iret frame to check if i need to swapgs
Really cool
never crossed my mind to generate the push/pops
and it uses sizeof and comptimePrint to figure out the rsp offset of the bottom of the struct for making the pointer before calling the actual handler
just mov %rsp, %rdi tho
I might give it a try for a cli app to get a feel for it
damn zig actually seems kinda cool
Ikr
https://github.com/Khitiara/imaginarium/blob/main/src/krnl/hal/arch/idt.zig#L231-L321 this is the stuff i mentioned, im rather proud of it
for a while I've thought of it as: oh some other dumb language, ignore
i was on the verge of learning c to do osdev when i found zig and decided to give it a go instead lmao
you did not make a mistake
nooby made me learn osdev and zig at the same time
and it was so fun :^)
i miss old zig
Smart
do you do 32-bit x86?
my memory was bad, im using it with offsetof to get the vector for indexing the handler lol
only long mode
%gs/%fs will always be zero
Zig freestanding: literal allocator
C++ freestanding: here's your stdint and fuck off, oh btw u have to implement initializer_list as well
yeah im planning to rework this and remove those lol
sounds good
not only do you get an allocator, you get like all of the standard library
yeah
given you provide the necessary os things
C freestanding: here stdint, have fun
but if you provide a page allocator it's enough for like 60% of the std
that's C in general
the ability to override the os bindings is a bit in flux rn sadly but not entirely
zig is C on steroids
they removed the ability to override all the io stuff temporarily while trying to figure out a better abstraction than just letting you override the posix functions
I also like that its not forcing borrow checking or other stuff down your throat
yeah thats why i cant handle rust lol
rust is brainrot
borrow checker sounds cool but i cannot wrap my head around what it wants
Apparently once you figure it out completely its good
But you need a PhD in language theory
one thing that i do thats kinda fucked up tbh but im still doing it anyway is that zig's std (and compiler_rt aka builtins) are, like everything else in zig, included as source instead of linking, so i literally have a fork of the zig std for my os that overrides stuff lmaoo
ultra zig rewrite when 🙏
I think it could be pretty good for some userspace stuff but ive kinda given up on the idea of rust in the kernel
I've heard it doesn't work well for long lived apps, but good for data processing like compilers etc
i feel like kernel stuff is inherently incompatible with the idea of rust
Yeah perhaps
you dont restart yours every 10 seconds?
sure you can write safe abstractions over kernel concepts
nyaux is a perfect example of a kernel that doesn't live long
but it all needs to be unsafe lol
rn just to fix a bug that the pr hasnt been merged yet where the cryptographic rng doesnt use overridden getrandom if you tell it to always use getrandom instead of just for seed and then a prng, which bug means i need language threadlocals to work which isnt happening
Yeah have u seen Linux rust drivers
Its insane
nyauxmaster: ```diff
+1
-1
(10 hours debugging)
anyway back to trying to figure out unit tests for an avl tree because i decided to put vmm in my what to do next poll and suddenly i need a self-balancing tree
OMG my brain couldn't handle rbtrees, like it did work in the end but I still barely understand it
Especially removing nodes
U have to recursively walk up and reconfigure it if there's a double black node
i found a c avl tree lib to copy but i also have to port it to zig in the process lol
zig has a treap in the std but its not intrusive so not really suited to my kernel work
side note, @fieldParentPtr is goated in zig
How hard is it to make a linked list in zig
container_of?
yeah
Yeah container of is a based idea
takes a comptime string field name
Does zig have reflection btw
Big
and @Type to create a type from type infoi