#Ultra

1 messages · Page 16 of 1

pearl meteor
#

yes

sinful river
#

2 levels with 2 bits:

root bitmap: 01
[0] sublevel bitmap: 10 (represents two CPUs, one has quiesced the other hasn't)
[1] sublevel bitmap: 11

When the other cpu in [0] notices it was the last one to set its bit it'll set the bit in the root bitmap

prime wraith
#

I see

sinful river
#

yeah they configure fanout based on NR_CPUS

pearl meteor
#

yea

prime wraith
#

Because nr_cpus is like 8k

sinful river
#

no it is NR_CPUS

#

it's compile time

pearl meteor
#

yes

prime wraith
#

Oh

sinful river
#

last i checked at least

pearl meteor
#

but obviously not all structs are allocated

sinful river
#

well even if they were it wouldnt be that bad

#

even with 8k cpus

prime wraith
#

I just dont see the point since u know at boot time how many max cpus u can have

sinful river
#

tho in practice I think the bits are all 1s and the cpus clear the bits

pearl meteor
#

because its easier to test for 0 than for a specific amount of 1s

sinful river
#

well yeah but you can also do ~0

#

if you fix the bitmaps at u64s

#

which I think they do?

pearl meteor
#

what if you have 4 cpus

sinful river
#

ah right

pearl meteor
#

and the bitmap is 64 bits

sinful river
#

idk if I'll implement rcu in my current project, im thinking the alternatives look nicer

prime wraith
#

Which

sinful river
#

freebsd GUS comes to mind

#

and xnu's SMR which is basically the same thing

prime wraith
#

Ah

pearl meteor
#

and the way grace periods are started it's also a little bit insane
so, in each rcu_node and rcu_data, there are two fields, gp_seq and gp_seq_needed
gp_seq is the current grace period, and it's modified by the rcu gp kthread. the lower 2 bits, if non-zero indicate that a grace period is in process (aka. the rcu gp kthread is executing, and waiting for all cpus to report quiescence). a cpu that wants a start a grace period, samples without taking any locks and using read_once the gp_seq of the root, does a simple addition in order to get the number for the next grace period, and updates the gp_seq_needed from its rcu_data all the way to the root. this is done taking locks. the advantage of this is that if there are 2 or more cpus on different nodes starting grace periods, they will eventually meet, and only one will proceed. moreover, if a cpu lags behind, it can stop from there. when the gp_seq_needed is updated all the way to the root, if a grace period was not already requested, the rcu_gp kthread is woken up

https://elixir.bootlin.com/linux/v6.19.11/source/kernel/rcu/tree.c#L3442
https://elixir.bootlin.com/linux/v6.19.11/source/kernel/rcu/tree.c#L999

#

as i said earlier, the rcu gp kthread works as a state machine, so that if there was no gp already, it is in an init state. from there, it takes notice of cpus that have went online/offline in order to update the rcu_node structures they belong to, and after that, it modifies all the rcu_node structures in order to reset their qs needed bitmaps, and also to set the new value of gp_seq, which, is not taken from gp_seq_needed, but instead, samples a new one from the root

https://elixir.bootlin.com/linux/v6.19.11/source/kernel/rcu/tree.c#L2259
https://elixir.bootlin.com/linux/v6.19.11/source/kernel/rcu/tree.c#L1803

#

the cpus notice that a new grace period has started on the sched clock tick, in the rcu sched clock tick function, where it compares, without taking locks, its gp_seq number of its rcu_data to the gp_seq number of its parent rcu_node. if they dont match, a grace period is started or ended, which is when the rcu core (the per-cpu kthread or softirq) is invoked, which handles beginnings and ends of grace periods, is responsible for the callbacks, and reports quiescence.
if a new grace period is started, rcu_node->core_needs_qs is set. if the rcu tick sees that the cpu is in a quiescent, and core_needs_qs is set, it invokes the rcu core to report this upwards

https://elixir.bootlin.com/linux/v7.0.8/source/kernel/rcu/tree.c#L2696
https://elixir.bootlin.com/linux/v7.0.8/source/kernel/rcu/tree.c#L3710
https://elixir.bootlin.com/linux/v7.0.8/source/kernel/rcu/tree.c#L2835
https://elixir.bootlin.com/linux/v7.0.8/source/kernel/rcu/tree.c#L2498
https://elixir.bootlin.com/linux/v7.0.8/source/kernel/rcu/tree.c#L1273

#

theres also special provisions for handling expedited rcu, handling callbacks which have been pending for too long, cpus that are unfortunate enough to only have small windows of quiescence state which the sched clock tick can't account for

#

and finally, it's in the rcu gp cleanup where the gp_seq is advanced on all nodes, gp_seq_needed is advanced, and possibly a new grace period is started

#

in conclusion: run

prime wraith
#

ill pin it to read later

carmine token
#

tree RCU is the implementation strategy not a separate RCU flavor

#

Also worth noticing that the fanout of each node is very high so the "tree" is just a single node on all non server CPUs

#

Apparently it's 1024 by default

prime wraith
#

Interesting

prime wraith
#

Cleanest uacpi printf integration of all time chad

void uacpi_kernel_log(uacpi_log_level lvl, const uacpi_char *fmt, ...)
{
    va_list vlist;
    struct nested_printf npf;

    va_start(vlist, fmt);
    npf.fmt = fmt;
    npf.vlist = &vlist;

    print(LOG_LEVEL_PREFIX"%cacpi: %pV", uacpi_log_level_to_syslog(lvl), &npf);

    va_end(vlist);
}
haughty notch
#

cacpi?

prime wraith
#

%c

#

in the output it just results in this:

acpi: starting uACPI, version 4.0.0
acpi: RSDP 0x00000000000F52C0 00000014 v00 (BOCHS )
acpi: RSDT 0x000000003FFE230E 00000038 v01 (BOCHS  BXPC    )
acpi: DSDT 0x000000003FFE0040 000020C6 v01 (BOCHS  BXPC    )
acpi: FACP 0x000000003FFE2106 000000F4 v03 (BOCHS  BXPC    )
acpi: APIC 0x000000003FFE21FA 00000078 v03 (BOCHS  BXPC    )
acpi: HPET 0x000000003FFE2272 00000038 v01 (BOCHS  BXPC    )
acpi: MCFG 0x000000003FFE22AA 0000003C v01 (BOCHS  BXPC    )
acpi: WAET 0x000000003FFE22E6 00000028 v01 (BOCHS  BXPC    )
#

but the log level is stored aside in metadata

haughty notch
#

that's cool

thorny kelp
prime wraith
#

no problem lol, i stole it myself

prime wraith
#

btw @haughty notch this is what clanker said about IOMMU + x2apic:

Standard PCI MSIs and IO-APIC Redirection Table Entries (RTEs) are fundamentally just MMIO writes to the APIC address space (typically starting at 0xFEE00000). In legacy xAPIC mode, the Local APIC actively claims this MMIO region. When a device DMAs a write to that address, the LAPIC intercepts it and processes the interrupt.

When you switch the processor into x2APIC mode, the LAPIC's MMIO interface is completely disabled in favor of the MSR interface. This creates a routing problem: if a PCI device sends a standard MSI, it issues a memory write to 0xFEExxxxx, but the LAPIC is no longer listening on that physical memory window.

Intel's hardware solution relies entirely on the VT-d IOMMU. The Interrupt Remapping engine intercepts those legacy MMIO writes from devices (or the IO-APIC) and translates them into the proper x2APIC interrupt messages on the system interconnect (QPI/UPI). Without the IR engine enabled, the chipset provides no architecturally guaranteed hardware path to route a legacy memory-mapped MSI to an MSR-only x2APIC.
#

idk if its completely true

ornate jasper
#

what were you asking it? because i didnt think that was true

prime wraith
ornate jasper
#

does it? i was looking at the x2apic paths and didnt see any iommu checks

#

(looked when i was trying to figure out the edge cases around ap startup with mixed x/x2apic cpu states because of kernel turning on x2apic before ap startup)

ornate jasper
#

oh thats a new change

#

in 7.0

#

because i was looking in 6.x and that function didnt exist

prime wraith
#

im looking at 6.x and its there also

ornate jasper
#

maybe im just stupid then lmao

prime wraith
#

idk how old it is honestly

prime wraith
#

tldr x2apic cant intercept msis without an iommu?

ornate jasper
#

oh i see how i missed this

#

i was looking at the enable x2apic path thats used for APs

pearl meteor
#

?

#

ah so yes, you indeed need the iommu?

ornate jasper
#

you need an iommu but you can leave it in compatibility mode by default

#

so the iommu just needs to exist not be tinkered with

#

(but you can only target cpus with x2apic id < 256 using the compat mode)

prime wraith
#

So as long as it exists and you didn't touch it, it will identity map them to the right cpu?

ornate jasper
#

for the cpus that are also reachable in xapic mode yes

prime wraith
#

I mean its not like u can encode other cpus in the MSI dst

ornate jasper
#

yeah exactly

#

if you turn on the iommu interrupt remapping stuff it uses a different message format that lets you do more

prime wraith
#

That makes sense ig

ornate jasper
#

(the remappable message format just encodes an index into the address and data and then uses that handle to look up in the remapping table iirc but i havent looked into that too far yet)

pearl meteor
#

the amd iommu does not have anything like that

#

so, i suppose the bios just sets up the individual device table entries interrupt remapping

#

ill check later

#

well, my iommu does not support x2apic KEKW

ornate jasper
#

my amd cpu doesnt even support x2apic afaik

prime wraith
prime wraith
pearl meteor
#
May 20 16:01:29 archlinux kernel: APIC: Static calls initialized
May 20 16:01:29 archlinux kernel: ACPI: APIC 0x0000000077223000 0000E8 (v05 _ASUS_ Notebook 01072009 AMI  00010013)
May 20 16:01:29 archlinux kernel: ACPI: Reserving APIC table memory at [mem 0x77223000-0x772230e7]
May 20 16:01:29 archlinux kernel: ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1])
May 20 16:01:29 archlinux kernel: IOAPIC[0]: apic_id 33, version 33, address 0xfec00000, GSI 0-23
May 20 16:01:29 archlinux kernel: IOAPIC[1]: apic_id 34, version 33, address 0xfec01000, GSI 24-55
May 20 16:01:29 archlinux kernel: APIC: Switch to symmetric I/O mode setup
May 20 16:01:29 archlinux kernel: x2apic: IRQ remapping doesn't support X2APIC mode
May 20 16:01:29 archlinux kernel: ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
May 20 16:01:29 archlinux kernel: ACPI: Using IOAPIC for interrupt routing
May 20 16:01:29 archlinux kernel: AMD-Vi: Extended features (0x246577efa2254afa, 0x0): PPR NX GT [5] IA GA PC GA_vAPIC
May 20 16:01:29 archlinux kernel: AMD-Vi: Virtual APIC enabled
prime wraith
#

So like, clanker was right then

pearl meteor
#

suck to be me tbh

#

i had already written the code for the x2apic nooo

#

now i have to deal with the apic bullshit

prime wraith
pearl meteor
#
May 20 16:01:29 archlinux kernel: ACPI: Using IOAPIC for interrupt routing
May 20 16:01:29 archlinux kernel: PCI: Using ACPI for IRQ routing
#

?

prime wraith
#

Nah, the apic routing, it should be around the same place

#

Like physical or logical

#

Also why the hell did amd even add an x2apic to that cpu if it cant receive MSIs

#

That makes it like literally useless

pearl meteor
#

idk

prime wraith
#

Unless there's still a hidden compat mode

pearl meteor
#

also idk what message you're talking about

prime wraith
#

Can you paste dmesg?

pearl meteor
#

if i grep for physical or logical there's nothing

#

and those are the only messages with apic

prime wraith
#

What if you grep for phys

pearl meteor
#

nope

prime wraith
#

Or flat

pearl meteor
#

neither

prime wraith
#

Bro

#

Oh well

#

Can you test MSIs on it with some hobby os (or your own)?

#

If they actually never work

pearl meteor
#

yeah, i just started writing the code for them

prime wraith
#

Cool

pearl meteor
#

this conversation rn has striked me a little bit

#

who knows if thanks to it i have saved hours of debugging

prime wraith
#

I mean they should work im 90% sure, otherwise they simply wouldn't add x2apic

#

Unless who knows

pearl meteor
#

ioapic interrupts i get with the x2apic

#

for msis ill have to see

prime wraith
#

Those aren't msis so yeah

#

Lets see tho

pearl meteor
#

i could write a pcie aer driver

#

unironically

#

on every boot i get a correctable error

#

and i think that those are signaled with msis

prime wraith
#

Lol

#

Is your Mobo broken?

pearl meteor
#

unless they're logged and linux just reads them

#

idk

#

to be fair bluetooth does not work properly

#

it disconnects randomly, and the xhci driver complains about electromagnetic interference

#

kernel: usb usb3-port3: disabled by hub (EMI?), re-enabling...

prime wraith
#

Lol

pearl meteor
#

and no fucking idea why

#

i have never tampered anything from my laptop

#

well, today i seem to be lucky, theres no error

#

it comes from the nvme btw

#

although i have never had any issues with it

#

kernel: pcieport 0000:00:02.4: AER: Multiple Correctable error message received from 0000:05:00.0 there, from last boot

haughty notch
haughty notch
pearl meteor
#

neither

#

when that happens the bluetooth icon disappears

#

it's so incredibly annoying

prime wraith
#

lol

pearl meteor
#

in linux the device just disappears from bluetoothctl and i get fucked anyways because i use wireless headsets

prime wraith
#

but yeah sounds like you have a fucked up laptop

#

which model is it

pearl meteor
#

asus tuf gaming a15

#

tbh cant complain

#

i bought it for 650 euros

prime wraith
#

thats decent ig

#

@pearl meteor APIC: Switched APIC routing to: physical flat I have this for example

pearl meteor
#

kernel: APIC: Switch to symmetric I/O mode setup

prime wraith
#

nah

pearl meteor
#

oh wait

#

no

prime wraith
#

i have that one as well

pearl meteor
#

nope, i dont have anything like that

prime wraith
#

😭

ornate jasper
#

i dont think i can get that info from windows or id give you it lol

prime wraith
#

its fine, doesnt really matter

#

why the hell does that need a fucking state machine KEKW

pearl meteor
#

lol

ornate jasper
#

lmao

prime wraith
#
static int __init check_ivrs_checksum(struct acpi_table_header *table)
{
    int i;
    u8 checksum = 0, *p = (u8 *)table;

    for (i = 0; i < table->length; ++i)
        checksum += p[i];
    if (checksum != 0) {
        /* ACPI table corrupt */
        pr_err(FW_BUG "IVRS invalid checksum\n");
        return -ENODEV;
    }

    return 0;
}

?????

#

bro u have acpica that does that for u

#

yeah that file contains some very interesting stuff

#

crazy code

ornate jasper
#

wild

haughty notch
prime wraith
#

Nice to see this still working after a huge amount of work on early hello world stuff

#

basically now i build my own direct map and don't use it for MMIO or other memory

#

so acpi/smbios/ioapic here is using early mmio helpers

#
  • a ton of cpu init added
#

pretty large pr

prime wraith
pale cairn
ornate jasper
#

its really good

prime wraith
ornate jasper
#

yeah im all set up i just havent had time to actually do any os testing with it yet

#

life has been super busy lately

prime wraith
#

ah unlucky

pearl meteor
#
May 21 18:21:01 AtiePC kernel: pcieport 0000:00:01.1: AER: Correctable error message received from 0000:01:00.0
May 21 18:21:01 AtiePC kernel: pcieport 0000:00:01.1: AER: Correctable error message received from 0000:01:00.0
May 21 18:21:01 AtiePC kernel: pcieport 0000:00:01.1: AER: Correctable error message received from 0000:01:00.0
May 21 18:21:01 AtiePC kernel: pcieport 0000:00:01.1: AER: Correctable error message received from 0000:01:00.0
#

from the nvidia gpu gnuslashtroll

prime wraith
#

did u have the chance to test out the MSIs with x2apic on it?

pearl meteor
#

im still on it

#

i expect today or tomorrow to have it finished

#

i think i am going to unironically write an aer driver

prime wraith
#

what the hell is up with github actions

haughty notch
#

thonk or maybe not

prime wraith
#

how is it possible to fuck up so often

haughty notch
#

They can't keep up with all of the AI slop

vestal zealot
#

Damn, do I actually need to start backing up shit I had on GH now?

#

Under 90% uptime is fucking insane

haughty notch
#

But like it's bad

#

It's not on their website

haughty notch
vestal zealot
#

Well either way people have been getting more and more antsy about keeping stuff there

haughty notch
vestal zealot
#

And I don't want to be one of the tiny unimportant people that can't recover data if/when shit does go sour

haughty notch
haughty notch
vestal zealot
#

Not of all my projects

#

I sometimes delete stuff I have on GH anyway

haughty notch
#

You can also mirror GitHub and gitlab easily

#

And others

sinful river
#

how is github even profitable

haughty notch
#

It's owned by Microsoft, they don't care

wicked juniper
sinful river
#

no way a few enterprise plans allow for unlimited storage basically for everyone

wicked juniper
#

it has a few limits i think

haughty notch
#

There's copilot

wicked juniper
#

copilot is fucked

#

and def not profitable

sinful river
#

I know they have github pro and stuff

haughty notch
#

Thought AI is a money furnace

sinful river
#

but idk who really pays that

#

(I have github pro)

wicked juniper
#

i have pro from the student thing XD

sinful river
#

yeah so do I

haughty notch
#

Me too

obtuse sparrow
prime wraith
#
$ cloc --vcs=git
     203 text files.
     186 unique files.
      18 files ignored.

github.com/AlDanial/cloc v 1.98  T=5.59 s (33.3 files/s, 3651.3 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                               49           1954            783           7892
C/C++ Header                   100           1055            643           4177
Python                           4            420             86           1355
CMake                           21            148             22            945
Assembly                         7             86             49            322
C++                              3             82              4            307
YAML                             1             12              3             50
TOML                             1              3              1             13
-------------------------------------------------------------------------------
SUM:                           186           3760           1591          15061
-------------------------------------------------------------------------------

Biggest gdt init you've seen

#

maybe behind the nadia kernel

pearl meteor
#

how are 1000 lines just cmake?

prime wraith
#

kconfig support mostly

#
  • symtable embedding with LTO etc, many small things like that add up
latent geode
#

mine is like ~250 loc

prime wraith
#

cmake right?

latent geode
#

yes

prime wraith
#

yeah fair

pearl meteor
#

i am still using the limine barebones template tbh lmao

prime wraith
#

lol

latent geode
#

wait that's not right

#

it's not counting lines in all of the files

prime wraith
#

i mean some features i have: linker script preprocessing, kconfig parsing/lazy reconfiguration, any amount of preliminary kernel relinks to stabilize the symbol table, free-after-init references checking, toolchain selection based on kconfig etc

pearl meteor
#

well that's definitely

latent geode
#

cloc says 745
250 was with tokei

pearl meteor
#

a lot of things

prime wraith
#

im also planning on doing the thing where u parse all used CONFIG_* in a translation unit and replace the config.h dependency on those specific key values

#

so you dont have to recompile the entire kernel if one config value changes

latent geode
#

I have modules, kernel dependencies, kallsyms and the kernel of course

prime wraith
#

yeah so similar size cmake

#

how do u ship the symbol table

latent geode
#

similar to how linux does it

prime wraith
#

do u also support debouncing it with LTO?

latent geode
#

what's that?

prime wraith
#

if u link the kernel without a symbol table, then generate the symbol table, then relink with the filled out symbol table all other symbol offsets will change if u have LTO enabled

#

so u have to redo this step multiple times

latent geode
#

yes I do that

prime wraith
#

ah cool

latent geode
#

link 3 times in total iirc

#

very slow with lto

prime wraith
#

same i think

#

actually no

set(ULTRA_NUM_RELINKS 1)
if (CONFIG_LTO)
    set(ULTRA_NUM_RELINKS 2)
endif ()
#

1 + 2 is 3 yes

#

i can count

latent geode
#

:D

prime wraith
#

u should add kconfig support btw

#

its based af

latent geode
#

I currently just pass a few options with -D

#

config is in my todo list :P

prime wraith
#

yeah i used to do the D thing

#

but its so much nicer to just open up menuconfig/guiconfig and select what u need

latent geode
#

what do they do?

prime wraith
#

u can disable them so the respective paravirt interface is not used and the thing is treated as bare metal

latent geode
#

hmm

prime wraith
pearl meteor
prime wraith
#

timekeeping, ipis, smp startup etc

latent geode
#

ah

pearl meteor
#

rn i have a config.h and that's it

prime wraith
pearl meteor
#

makefile

prime wraith
#

hmm lol

#

well

#

you will have to parse the resulting .config file and turn it into key=value pairs

#

other than that i guess not

#

u could see how linux does it i guess

#

since they're using makefiles also obviously

latent geode
#

hand-rolled makefiles my beloved ❤️

latent geode
prime wraith
#

test harness stuff

#

i test C code using C++ trl

#

and an assert throws an exception in C code

#

or well, in c++ code that also unwinds C code

latent geode
#

it's kinda cool that you can throw and catch exceptions across different languages

#

iirc for example you can throw in c# and catch in c++ or something like that

prime wraith
#

there are some rules regarding that for C iirc

#

it can be finicky

#

but yeah

#

I have stuff like void malloc_phys_range(uint64_t start, uint64_t size); which then makes that range work via phys_to_virt

#

that makes testing kernel stuff super nice

sinful river
#

It outputs key=value which you can import directly

prime wraith
#

Waait thats true actually

sinful river
#

I used to do that

#

Just include it directly

obtuse sparrow
ornate jasper
#

because name=value is valid make syntax iirc

obtuse sparrow
#

Yeah

ornate jasper
#

if you arent using make then rip tho lol

obtuse sparrow
#

Also valid bash syntax but make is just overcomplicated shell scripts meme

ornate jasper
#

well yeah

sinful river
ornate jasper
#

meson was less than one year old the last time i wrote c or c++ lmao

#

but yeah good to know

pearl meteor
#

@prime wraith

#

i am pleased to announce that msis work perfectly fine on my machine with x2apic and no iommu x2apic support

#

so linux is just bollocks lol

prime wraith
#

lol nice

#

I wonder how the hell they achieve that then

pearl meteor
#

is the linux change recent ?

prime wraith
#

Very old I think

ornate jasper
#

might be a quirk almost then

#

some hardware at the time didn't handle it right or smth

prime wraith
pearl meteor
#

there's a bit more on the msi story in amd

#

i don't know how it works on modern chipsets (>2006), but on those where the hypertransport specification was relevant, the parent bridges would catch memory accesses to 0xfee0...., and generate an interrupt message over hypertransport

#

in fact, those bridges have a hypertransport capability in the pci configuration space which still exists on modern chipsets, but is completely read only

#

what is remarkable about that capability is that it allows you to enable and disable generation of hypertransport interrupt messages, and on some chipsets it may allowed you to reprogram the msi address range

#

so it definitely isn't just some 0xfee...... bubbling upwards and making its space to the apic, but it's some pcie/infinity fabric thing working together making it work

#

the iommu specification, in the typical amd style of being vague, doesn't explain what are the other implications of not supporting x2apic, apart from not supporting 32 bit ids and iirc changing the interrupt redirection entry

#

so that's why you might have seen me in the server saying before i tested all of this that writing an iommu driver is not necessary for x2apic

carmine token
#

whether something is "necessary" or not is often arguable anyway

#

people sometimes make it sounds as if x86 was a single uniform architecture when in reality each individual machine often needs specific support at the OS level

#

larger commercial OS vendors have dedicated teams just to enable specific machines

ornate jasper
#

yeah if its any more uniform than other arches thats because of decades of convergent evolution

carmine token
#

yeah

ornate jasper
#

not because its actually uniform

carmine token
#

yeah

#

it doesn't even claim to support arbitrary x86 machines

ornate jasper
#

like theres a whole fuckin bit in the PTEs thats reserved on amd and free-for-software on intel, not even core architectural things like that are truly consistent

#

(its bit 8 on non-final entries, which is the global bit on page/block entries)

#

my desktop has nonstandard serial io port addresses

#

there is no uniformity

pearl meteor
#

any reasonable os should uh, have an acpi interpreter and not probe the standard ones, right meme

eternal sparrow
ornate jasper
#

yeah

prime wraith
#

I wonder if I even need "refined" tsc recalibration later in boot, it seems a 50 ms early calibration yields pretty much almost dead on result, like I get 1859.990.. on a 1860 mhz tsc

#

Maybe ill add that later

haughty notch
#

Just recalibrate against an RTC interrupt every second, then you can detect the unstable timer as well trl

prime wraith
#

💀

prime wraith
#

linux larp is in full effect (I have the [0.0000] thing now)

#

cant wait till i get to smp bringup tbh, that should be fun

wide ether
#

what did you do before

prime wraith
#

well literally nothing because i didnt have time

wide ether
#

ah

prime wraith
#

lmao

stable orchid
#

(compat)

prime wraith
#

depending on what you mean by larp

#

to an extent it is

carmine token
#

zinna output is still a bit nicer imo because it also has proper log levels

prime wraith
#

show

prime wraith
#

like color or

stable orchid
#

when osdev competition went from uacpi score to logging look

carmine token
#

uacpi score is boring show your mlibc compilation time

stable orchid
#

real

carmine token
prime wraith
#

yeah I have that planned also

#

right now the console subsystem is mostly a hack

pearl meteor
#

if you're not using panic() on the very important messages or events you're not doing it right

eternal sparrow
#

I color code by subsystem & log level tags postfixed them

haughty notch
#

panic panic level trl

sinful river
#

Imagine color coding 😎

pearl meteor
#

lol

prime wraith
#

buddy init... OK

[    0.061726] memory: populating memory map [0xFFFF900000000000 - 0xFFFF90000001C000] for phys [0x0000000000000000 - 0x0000000000800000]
[    0.064399] memory: populating memory map [0xFFFF900000000000 - 0xFFFF900001C00000] for phys [0x0000000000000000 - 0x0000000080000000]
[    0.070628] memory: populating memory map [0xFFFF900003800000 - 0xFFFF900008C00000] for phys [0x0000000100000000 - 0x0000000280000000]
[    0.100684] buddy: initial allocator state below
[    0.101700] buddy: order: [4k  ] [8k  ] [16k ] [32k ] [64k ] [128k] [256k] [512k] [1M  ] [2M  ] [4M  ] 
[    0.103653] buddy: free:   3      4      3      1      3      2      0      2      2      3      2013 
wide ether
#

ok buddy

prime wraith
#

triple faults

pearl meteor
#

lol

#

yeah having page sizes above 2mb is definitely larptrl

prime wraith
#

i mean its one constant lmao, i just did whatever

pearl meteor
#

yeah

#

i also have the same ngl

pearl meteor
#

this is the ultimate larp

#

to use a (tm) after the name of the project

#

Ultra(TM)

eternal sparrow
#

Ultra(TM)(TM) would go more hard

#

You just have to trademark the trademarking of ultra

sinful river
#

Linux IS a trademark tho

pearl meteor
#

yeah but it's funny because it's the only place where you see that

#

also it is registered by linus so it should be ®️

prime wraith
#
[    0.001533] memory: populating memory map [0xFF80000000000000 - 0xFF8000000000E000] for phys [0x0000000000000000 - 0x0000000000400000]
[    0.002184] memory: populating memory map [0xFF80000000000000 - 0xFF80000001C00000] for phys [0x0000000000000000 - 0x0000000080000000]
[    0.014967] memory: populating memory map [0xFF80000003800000 - 0xFF80000008C00000] for phys [0x0000000100000000 - 0x0000000280000000]
[    0.079866] buddy: initial allocator state below
[    0.079986] buddy: order: [4k  ] [8k  ] [16k ] [32k ] [64k ] [128k] [256k] [512k] [1M  ] [2M  ] [4M  ] 
[    0.080312] buddy: free:   3      5      4      1      1      2      1      2      2      3      2013  
[    0.080871] alloc: now online, builtin caches listed below
[    0.080995] alloc:   Name         Object Size  Order    Max Empty   
[    0.081161] alloc:   builtin_8    8            0        16          
[    0.081296] alloc:   builtin_16   16           0        16          
[    0.081404] alloc:   builtin_32   32           0        16          
[    0.081533] alloc:   builtin_64   64           0        16          
[    0.081664] alloc:   builtin_128  128          0        16          
[    0.081787] alloc:   builtin_256  256          0        16          
[    0.081895] alloc:   builtin_512  512          0        16          
[    0.082031] alloc:   builtin_1k   1024         0        16          
[    0.082169] alloc:   builtin_2k   2048         0        16          
[    0.082308] alloc:   builtin_4k   4096         0        16          
[    0.082450] alloc:   builtin_8k   8192         1        8 

I made the thing use huge pages for vmemmap where possible + added clanker-generated phd memset/memcpy

#

80ms to initialize 8G of memory with TCG on a crappy laptop is pretty good imo

#

128mib takes 10ms

cosmic ibex
prime wraith
#

i think 10ms for pretty late kernel init is decent

#

(esp with tcg)

#

phd memset gives a 9x speedup on tcg

#

with O0 at least

eternal sparrow
prime wraith
#

i had a byte-by-byte = 0

eternal sparrow
#

rep movsb also heavily sped me up on tcg. i havent tried phd memset

#

mind testing if you got a moment?

prime wraith
#

well mine is not really phd, its just this

void *memset(void *dest, int ch, size_t count)
{
    void *ret = dest;
    u64 pattern;
    size_t qwords, bytes;

    pattern = 0x0101010101010101ULL * (u8)ch;
    qwords = count >> 3;
    bytes = count & 7;

    asm volatile (
        "rep stosq" : "+D"(dest), "+c"(qwords) : "a"(pattern) : "memory"
    );

    if (bytes) {
        asm volatile (
            "rep stosb" : "+D"(dest), "+c"(bytes) : "a"(pattern) : "memory"
        );
    }

    return ret;
}
prime wraith
eternal sparrow
#

nah just using this for memset

#
void* memset(void* dst, int c, size_t n) {
    void* tmp = dst;
    asm volatile("rep stosb" : "+D"(dst), "+c"(n) : "a"(c) : "memory");
    return tmp;
}
prime wraith
#

lets see

eternal sparrow
#

i think it can be faster than stosq in some cases? esp on EMRSB or whatever the cpuid bit is

prime wraith
eternal sparrow
cosmic ibex
#

wait...its slower?

prime wraith
#

the qword + byte version gives 200ms, this gives 600ms

eternal sparrow
#

Less code != faster

prime wraith
#

but its probably because tcg can catch this and direct to phd glibc memset

stable orchid
#

this blud didnt saw glibc impl

eternal sparrow
#

my kernel boot is alr so fast i cant even tell id i used that version PSP1G_pspTrollar

prime wraith
#

does it not slow down if u add more memory?

eternal sparrow
#

oh right lmao

#

i do lazy freelist init so its only paging that would slow it down

#

and it uses 1gb pages

prime wraith
#

but tcg likes qword memset a lot more

cosmic ibex
#

wait

#

this is pretty old

eternal sparrow
#

i can notice like ~2ms diffrence between the two when it stalles on paging init and other?

#

but yeah the qword is better on tcg it seems

prime wraith
#

ill probably keep the qword version since it doesnt degrade bare metal but gives a 3x speedup on tcg

eternal sparrow
prime wraith
#

yeah

eternal sparrow
#

me and mint did some tests with rep stosb and on zen3 and zen1 it was faster than some C impls

#

and slower on zen5

#

it was really funky

prime wraith
#

yeah in general this mem* stuff seems very flakey and random between cpu generations

eternal sparrow
#

I wonder how worth it would be to have many implementations for uarches and then hot patch the function at runtime when you detect it

sinful river
#

Do the Linux static trampoline thing too

#

That is a real PhD memcpy

eternal sparrow
sinful river
eternal sparrow
prime wraith
eternal sparrow
#

And not an indirect function call

eternal sparrow
prime wraith
#

Its guaranteed by the abi

wicked juniper
#

k, it's gone now

wicked juniper
#

same for this one?

eternal sparrow
eternal sparrow
wicked juniper
wicked juniper
eternal sparrow
prime wraith
sinful river
#

There's a blog post that explains it

#

It's pretty complex

eternal sparrow
#

Maybe another day

#

I got better stuff to worry about regardless

pearl meteor
#

dead project xd

prime wraith
#

Nah I have a shit ton of downstream changes, just been busy with uacpi and IRL work

prime wraith
pearl meteor
#

very cool tbh

thorny kelp
prime wraith
#

lol

#

yeah thats fake, i just forced it to actually write to 0x3080

#

which is the SOL COM on that laptop

thorny kelp
#

ah

eternal sparrow
prime wraith
#

yeah, the proper way is either DBG2 or just find the SOL (serial over LAN) PCI device and get the BAR address

#

this laptop doesnt have DBG2 so only method 2 would would

#

but i just hardcode the bar address since i dont have anything like pci yet

#

Some laptops are nicer in that they just capture the normal COM1 output

#

but this one is not

eternal sparrow
#

I still need to finish DBG2 parsing but I have no devices that have it lmao

#

I only needed SPCR for QEMU

prime wraith
eternal sparrow
#

All mine is zen3 or older so maybe im just unlucky?

prime wraith
#

Or maybe disabled by bios

eternal sparrow
#

When I get a j*b or goto collage j want to get a zen6 machine

prime wraith
#

Same trl

#

Or pantherlake

eternal sparrow
#

Honestly nova lake may be good

#

Depending on benchmarks

#

That would have FRED and APX

#

Which I wana support my kernel running with APX

sinful river
#

the core ultra plus are really good for the price

eternal sparrow
#

They are using a new socket again so

#

That's one of the few reasons to use AMD even then

#

I hope they do commit to the new socket though please

#

😭

prime wraith
#

SteamDeck over PXE boot with new hyper

sinful river
#

is that linux or ultra

prime wraith
#

Ultra lol

sinful river
#

can't tell, too much larp

#

what is unattended up to <>

prime wraith
#

The amount of time the kernel can basically go idle until it needs to "tick" the timekeeper

sinful river
#

huh

#

for me PM timer is like 3 mins

prime wraith
#

for me its usually 2 seconds

#

this one is 32-bits i guess

sinful river
#

oh yeah that oops

#

sorry

prime wraith
#

havent seen 32-bit pm timer anywhere before

#

but i have no other amd hardware so

#

well other than my main pc which im not going to bother booting this on

#

..and the ps5

prime wraith
#

and didnt bother adding cpuid for tsc

eternal sparrow
prime wraith
#

What do u want to change?

thorny kelp
#

copy linux entirely

prime wraith
#

linux log is a fucking mess if u enable anything > pr_info

#

so i wouldnt go there

#

also it has no color support for any of the backends

eternal sparrow
#

And not super consistent though shit

#

Probably will be easier to do once everything is finished and I do my log ring shit tbf

prime wraith
#

ah

#

yeah the log ring has been super useful even at this stage

#

because this framebuffer console is registered way later after all the allocators are up so i can do vmalloc + wc remap

eternal sparrow
#

I just rebuild the limine HHDM

#

So I do always have it at boot

#

But some other logs would be nice to delay

prime wraith
#

yeah fair, hyper doesnt map any mmio, even the framebuffer so i dont have that

eternal sparrow
#

And I can probably cut some init stages

prime wraith
#

for later real hardware CI i will use usb ACM

#

which obviously can only be registered way way later

eternal sparrow
#

Because I have one that's for very early initialization that I had to add due to logging shit not being right at boot on some archs

prime wraith
#

yeah fair

eternal sparrow
#

I was planning to just get an old machine

#

With a serial port

#

Which would be good enough as a basic test running

prime wraith
#

i want to have a lot of different machines set up for auto CI so just old stuff wont cut it for me

#

and none of my shit has serial ports 😭

eternal sparrow
#

True true, I will probably have USB stuff but I think il only support kernel GDB over serial

#

Since it simplifies it so much

#

As every byte can trigger an NMI

#

And be able to hook into everything nicely

prime wraith
#

yeah thats nice ig

#

i hope just logs will be enough for me

#

like real pci hw u can just passthrough, and cpu behavior u can basically validate via kvm

#

so hopefully gdb on real hw wont be needed

eternal sparrow
#

Fwiw the serial debugging wouldn't be for CI even

#

It would be for needing to debug really stupid bullshit things

prime wraith
#

yeah im just saying its probably only really niche things that need debugging that way

#

like if its a driver u can just do a passthrough and do it in on linux

eternal sparrow
#

I think it was useful for me once iirc

prime wraith
#

and if it dies due to cpu stuff it will most likely die on kvm on that hardware

eternal sparrow
#

I forget what I used it for

#

But I do know it did find an issue

prime wraith
#

id be interested to know what it was u debugged with it

versed obsidian
eternal sparrow
#

Only happened on my machine

#

That or schedular corruption

prime wraith
#

who the hell posted ultra on hacker news

sinful river
#

Maybe you would've liked to post the repo later? Nope

prime wraith
#

wtf

#

i hope its like a comment or something

sinful river
#

you cant post duplicate links

twilit geode
#

Damn

twilit geode
#

I was able to send the astral minecraft link twice last year

sinful river
sinful river
vestal zealot
sinful river
#

It makes sense

vestal zealot
#

No it doesn't, what if someone just posts slander and now you can't post your own project anymore

sinful river
#

There's a threshold of votes you need to get tho

#

but I was pissed off cuz I wanted to post my thing on HN but someone else posted it at the wrong time without asking me first and now I can't repost it cuz the first one got like 20 votes

vestal zealot
#

This policy makes no sense

sinful river
#

You can email admins apparently but that's too much trouble

sinful river
#

Which makes sense

#

But yea it's annoying

vestal zealot
#

yeah ok sure but this is too draconian

#

also you can trivially get around it with an URL shortener

sinful river
#

That's probably forbidden

#

Osdev hackernews alternative when

#

(that's lobsters)

twilit geode
sinful river
#

Mathew do you have a lobsters account

#

I can hook you up if you want

twilit geode
#

I dont

#

Sure

haughty notch
thorny kelp
prime wraith
#

bruh who asked for this

versed obsidian
#

@covert raptor did 😭

pearl meteor
#

just why

#

new fear unlocked

sinful river
#

it would be nice if people did not do that

pearl meteor
#

this is a dick move imo

sinful river
#

yeah

pearl meteor
#

specially considering the person has 100 messages in this server

#

and the project is in a very very early stage

sinful river
#

but u might be lucky and be able to repost it since it had few votes

pearl meteor
#

(despite being a 20k loc gdt init)

sinful river
#

or something like that

covert raptor
covert raptor
covert raptor
#

@prime wraith I have contacted HN mods and they have deleted the submission now

#

I apologize again for posting it without your consent

prime wraith
#

thanks

prime wraith
#
edk2$ git show cf9ff46b088a1df9c8ab81f31ef9606826ca7977
commit cf9ff46b088a1df9c8ab81f31ef9606826ca7977
Author: Fu Siyuan <[email protected]>
Date:   Tue Jan 9 15:08:57 2018 +0800

    NetworkPkg: Fix incorrect parameter check in PXE.Mtftp() function.
    
    According to UEFI spec, the PXE.Mtftp() should return invalid parameter if the
    BufferPtr parameter was NULL and the DontUseBuffer parameter was FALSE.
    The DontUseBuffer is only used when perform MTFTP/TFTP read operation.
    
    Contributed-under: TianoCore Contribution Agreement 1.0
    Signed-off-by: Fu Siyuan <[email protected]>
    Reviewed-by: Jiaxin Wu <[email protected]>

diff --git a/NetworkPkg/UefiPxeBcDxe/PxeBcImpl.c b/NetworkPkg/UefiPxeBcDxe/PxeBcImpl.c
index 93f3bfa5ba..9068e0686c 100644
--- a/NetworkPkg/UefiPxeBcDxe/PxeBcImpl.c
+++ b/NetworkPkg/UefiPxeBcDxe/PxeBcImpl.c
@@ -855,11 +855,19 @@ EfiPxeBcMtftp (
       (Filename == NULL) ||
       (BufferSize == NULL) ||
       (ServerIp == NULL) ||
-      ((BufferPtr == NULL) && DontUseBuffer) ||
       ((BlockSize != NULL) && (*BlockSize < PXE_MTFTP_DEFAULT_BLOCK_SIZE))) {
     return EFI_INVALID_PARAMETER;
   }
 
+  if (Operation == EFI_PXE_BASE_CODE_TFTP_READ_FILE ||^M
+      Operation == EFI_PXE_BASE_CODE_TFTP_READ_DIRECTORY ||^M
+      Operation == EFI_PXE_BASE_CODE_MTFTP_READ_FILE ||^M
+      Operation == EFI_PXE_BASE_CODE_MTFTP_READ_DIRECTORY) {^M
+    if (BufferPtr == NULL && !DontUseBuffer) {^M
+      return EFI_INVALID_PARAMETER;^M
+    }^M
+  }^M
+^M

So this is why GET_FILE_SIZE wasnt working on real hardware UEFI

#

Another RetardK2 quirk

#

at least the fix is easy i guess

#

if buffer is null and DONT USE BUFFER, return INVALID_PARAMETER

#

truly genius programming

twilit geode
#

asking firmware devs to produce good software

carmine token
#

Programming against the uefi spec is useless

#

there are many areas where the spec is incomplete and/or doesn't match actual implementations

#

It's better to just look at why edk2 does in practice

dusky kraken
latent geode
#

lol

prime wraith
errant temple
#

I think it's edk2 policy to use windoze line breaks

dusky kraken
#

the old lines in the diff don't have them though

pearl meteor
#

wait

#

so, it was implemented, but locked ?

#

lol

prime wraith
#

but yeah it was implemented all along

#

to work around it u have to pass any non-null ptr as buffer

#

ill just pass 0xDEADDECAF

pearl meteor
#

lol

prime wraith
#

Finally all 3 vPRO laptops I have hooked up to auto-ci

This was a crazy amount of work tbh, from dealing with firmware bugs to dealing with dogshit BIOS UIs
I had to switch to PXE boot because IDE-R is unreliable af on some laptops, that meant I had to add PXE support to hyper UEFI
That wasnt working as it turns out due to an old EDK2 bug where GET_FILE_SIZE requires a non-null useless buffer ptr
Then I realized UEFI wasnt enough because an older vpro laptop I have doesnt support PXE over UEFI so I hadd to add a hyper BIOS PXE implementation as well
Then SOL wasnt working on one laptop so I had to debug and turns out I have to do config space initialization for the sol redirection device myself
Then some laptops work with amtterm, and some need a meshcmd amtterm, basically legacy vs new
Its hell but it works now!

#

I've also ordered an rpi zero w to hook up non vpro laptops i have to EHCI debug port and use that on those

errant temple
#

oh wait how does vPRO work

prime wraith
#

It offers Intel AMT stuff, which is basically remote management, including power management, serial-over-lan, KVM etc etc

ornate jasper
#

(with an api)

prime wraith
#

the api is completely fucked and uncodumented btw

ornate jasper
#

yup

prime wraith
#

and its quirky af because its all firmware

ornate jasper
#

the big feature not listed in the list infy gave is storage redirection. aka you can take your computer's normal storage and have access to it read a disk image over the network instead

#

and its only intel computers and only some

#

its cool af tho

prime wraith
#

yeah its called IDE-R, basically one AHCI port is fake and acts as networked storage

#

but it only works on some laptops

#

like 1/3 of the ones i have

ornate jasper
prime wraith
#

they have USB-R now, but i wasnt able to get it to work

#

at least boot via it

#

because otherwise i dont need it

#

USB-R should use the same API as IDE-R i think

ornate jasper
#

im more interested in how it looks to the guest if your normal storage is nvme

prime wraith
#

i dont think NVMe-R is a thing

#

because the way IDE-R works is emulated SCSI requests over network

ornate jasper
#

which tbh i could check if i wasnt lazy af since the laptop im typing this on has only nvme drives (and the controller keeps randomly dying and needing a full power cycle before anything can detect it again but i cant get actual info on why because windows explodes if you pull its C drive storage out from under it and without a disk drive nothin be logging but besides the point) and has vPRO

prime wraith
#

u can try hooking it up to meshcommander and see what it offers

#

but meshcommander is also abandonware now btw

ornate jasper
#

it has the storage redirect

prime wraith
#

they only offer proper software to corporations now

ornate jasper
#

i checked all the features at some point

prime wraith
#

if i had to guess its routed as an emulated usb drive over xhci

ornate jasper
#

i just dont know how itll look to the guest because i cant be assed to actually run a guest on it yet

prime wraith
#

wdym by guest

errant temple
#

got any pointers for setting that up

prime wraith
#

go into bios and click ME Setup

#

then enable Intel AMT

ornate jasper
#

you may need to actually set a password and if so it'll fail without telling you why if you don't meet the password requirements which I could only find on a forum somewhere

prime wraith
#

yep

#

and it has insane password requirements so i just always use Qwerty1@

errant temple
#

I don't see anything about AMT in the BIOS

#

even tho the laptop claims vPro and AMT support

prime wraith
#

its under the ME settings

#

its a serparate menu thats not in the general bios settings

errant temple
#

I don't see ME settings lol

#

even in the boot selection menu

#

(F12 on the dell laptop)

prime wraith
#

when u enter general setup it has like Enter BIOS Setup, Enter Boot Menu, and then Enter ME Settings

#

not in like the big bios ui

errant temple
#

I understand that part

ornate jasper
#

well sorta

prime wraith
#

oh ok, well leo has a dell also

ornate jasper
#

in the bios settings there was a nested menu somewhere near the bottom of the list to turn the whole subsystem on

#

actually im leaving home soon and going to shut down the laptop so ill fuckin reboot into bios and take a potato cam shot of it lmao

errant temple
#

I'm currently flashing a newer BIOS, sec

#

that laptop is 12 releases behind meme

ornate jasper
#

here's where the setting was for me

errant temple
#

I don't think I see the Managebility thing at all

#

note to self: only buy laptops with vPro in the future

ornate jasper
#

maybe updating bios will help lol

prime wraith
#

i mean if it says it has one it probably has it hidden somewhere

ornate jasper
#

and then this menu option to get into it

errant temple
#

oh could it be that it doesn't work in docked mode?

ornate jasper
#

that'd be weird but maybe ig

#

mine works plugged in but I don't have a laptop dock so idk about docked mode

errant temple
#

sad leo noises

ornate jasper
#

unless you turn on the thing in the bios setup it won't be there

#

so id basically just go through the whole bios setup menu lol

errant temple
ornate jasper
#

and you'll need to set an admin password in the bios (not system password that's different) to turn it on I think

errant temple
#

oh

#

I don't have that, sec

prime wraith
#

what do u have under management?

ornate jasper
#

idk if not having that hides it tho

#

I just went through the whole tree menu until I found it tbh

errant temple
#

still nothing

#

the product page lied then I assume

ornate jasper
#

F

prime wraith
ornate jasper
#

my laptop is also a used enterprise one idk if that matters though (like idk if it needs special firmware for the laptop that mine just already had because enterprise. id normally assume no but idk anymore)

prime wraith
#

yeah same

#

iirc normal consumer ones never have vpro

#

@errant temple two easy ways to check: vPRO on the cpu sticker or redirection/SOL device in lspci or dmesg | grep serial

errant temple
#

ah ok

#

no dice

#

unfortunate

ornate jasper
#

F

prime wraith
#

F

#

@ornate jasper if u want i can give u my script and the dnsmasq config

#

honestly thats why enterprise laptops are the best, they usually have the most features like that packed in and no gaming gimmick

#

Also interesting thing I noticed: all AMD hardware has 32-bit pmtimer and 32-bit HPET, while all intel hardware has 24-bit pmtimer and 64-bit HPET

prime wraith
#

and this pattern is the same across all hw, like an AMD C50 laptop thats old dogcrap and steamdeck which is relatively recent amd apu

red parcel
#

Maybe on desktop it's different?

prime wraith
#

maybe yeah, i cant check from windows i think

thorny kelp
prime wraith
#

nah apparently its just amd vs intel

thorny kelp
#

(i don't handle hpet rollback currently in my kernel)

thorny kelp
prime wraith
#

not that it matters, who uses hpet if u have tsc

red parcel
#

Yeah

thorny kelp
#

me KEKW

red parcel
#

Smh

thorny kelp
#

because i haven't implemented tsc yet lmao

prime wraith
#

"implemented"

thorny kelp
#

maybe i should work on that

prime wraith
#

rdtscp

thorny kelp
#

lmao

#

implemented as in support it

#

whatever

prime wraith
#

-cpu qemu64,-tsc trl

#

that just clears the cpuid bit tho, tsc still works I checked

thorny kelp
#

maybe i could fix this too lmfao

#

(don't really need to tho)

ornate jasper
prime wraith
prime wraith
sterile kayak
thorny kelp
#

same- oh wait i have the same exact cpu

prime wraith
#

vPRO and serial-over-lan is a life savior for laptops honestly, otherwise your other options are EHCI-debug-port which requires a special cable with a microcontroller which company that made them went out of business or some board (baglebone black, rpi w zero, etc) which has a device mode usb port which can emulate the said microcontroller in software via a linux module. For xHCI its even worse, it also has a debug port capability but that requires a special cable which has vpwr stripped and only one single company makes such cables

#

not to mention xhci/ehci debug port capability requires a bit more code than just a standard x86 uart

twilit geode
prime wraith
#

lol

pearl meteor
#

or the hda timer

prime wraith
#

Not doing any ultra dev atm since i've been focused on the loader too much

#

Every since I added PXE support it got me a bit sidetracked

#

theres just too many things i hated about the boot device detection

#

Also reworked the memory mapper to only map usable ram

#

i have a gigantic amount of code just sitting there that i need to go through commit by commit and review etc

#

Also apparently EDK2 for aarch64 dies on startup with -cpu max so i had to do cortex-a72

#

as a sidenote having pxe boot is so convenient its crazy

#

the only downside is i wish laptops supported pxe over wifi

#

Also hyper will have to be excluded from the no clanker list, since i've been using it quite a lot for this. Lately i just started writing out all apis + functions in a .c file with empty bodies and have clanker fill them out and then just use that as a basis

eternal sparrow
prime wraith
#

Claude is pretty decent at this, although it usually generates complete dogshit with code duplication so you have to either prompt it a billion times or rewrite everything yourself

prime wraith
eternal sparrow
#

one time i had it call __builtin_memset instead of memset from string.h that i procide

#

🥀

prime wraith
#

yeah lol

#

it loves doing ad-hoc lazy bullshit

eternal sparrow
#

i feel like thats somehow more effort

#

when iirc string.h was alr in that file

#

with memcpy used

#

😭

prime wraith
#

I usually tell it which files to reference and what apis are already available, e.g. i had to explain to it my string_view etc

#

but yeah lol

eternal sparrow
#

Since I only use it for stupid shit it kinda just works if I tell it to just do it 💀

#

I try and code as much as I can myself

#

But so times I CBA

prime wraith
#

yeah lol

#

i like coding parts that are interesting to me

#

but unfortunately the design and api layout you still have to do yourself basically always or it will produce really bad results

#

which is not that bad ig

eternal sparrow
#

That stuff I do like tbh

prime wraith
#

yeah sketching out apis is fun ig

eternal sparrow
#

It's the really boring shit when I'm half asleep and don't want to use my little brain lower for

prime wraith
#

yee

prime wraith
#

Wait Downloading https://ftpmirror.gnu.org/gnu/binutils/binutils-2.45.tar.gz... even this fails now

#

Is there another mirror I can use lol

prime wraith
#

oh yeah maybe

thorny kelp
#

thats what i use

#

oh bruh

#

binutils

prime wraith
#

i really wanna drop gcc support im so tired of it not supporting cross compiling by default

#

i only use it to verify my shit is not ub'ed to work only under clang

thorny kelp
prime wraith
#

yeah yeah same idea

thorny kelp
#

but i misread it as coreutils lmao

eternal sparrow
#

I also have a mirror on my git server

prime wraith
thorny kelp
#

just try again

#

it might've chosen a dead server

thorny kelp
eternal sparrow
#

ah

#

this is the one i used

#

https://mirrors.ocf.berkeley.edu/gnu/gcc/gcc-${version}/gcc-${version}.tar.xz

prime wraith
#

Oh thanks

#

I should make a tier system with various gcc mirrors at this point

eternal sparrow
sterile kayak
#

And also sometimes ftpmirror itself is down

prime wraith
#

Bruh

thorny kelp
#

like 70kbps and lower

twilit geode
#

ftpmirror on its way to give me a random african mirror instead of a brazilian or even south american one

obtuse sparrow
#

ftpmirror giving you a mirror on butan

eternal sparrow
#

It would be really nice if they fixed ftp mirror

#

And monitored things for being down

#

To not direct to them

#

And to have countrycode.ftpmirror

#

So you can get one that's actually local

prime wraith
#

yea... im stuck with the loader shit for a while longer

#

i have a lot more things id like to fix before i continue with the kernel

#

the loader is 4k loc bigger than the kernel KEKW

#

20k vs 16k atm

#

about to change further tho trl

vestal zealot
prime wraith
#

like i genuinely dont even think so, that thing doesnt even have a gui

#

thats just how much it takes

vestal zealot
#

i'm just joshing ya

prime wraith
#

lol

eternal sparrow
prime wraith
#

What do you support atm?

#

If its limine, the way that protocol is designed is fully compatible with it and you can support both in one binary like pmos

#

You can dynamically detect how you were booted by checking the magic number

eternal sparrow
#

Just limine, but I want a pre kernel anyways

#

It will also give me an excuse to learn elf relocations and other things

#
  • MB2 in the future later
prime wraith
#

ah

vestal zealot
#

Is the Ultra loader compatible enough I can use it in place of Limine for shits and giggles?

prime wraith
#

Well all the structures are different

vestal zealot
#

Didn't Ultra also implement Limine protocol though?

prime wraith
#

Nope

vestal zealot
#

oh

#

which bootloader was it, then? I recall Limine not being the only impl of its own protocol... ?

prime wraith
#

No idea

vestal zealot
#

Maybe @little aurora knows?

latent geode
#

@loud radish might have been dabbling in that

#

iirc

prime wraith
#

it's a pretty gigantic protocol so i cant imagine anything would implement it for fun

#

maybe a small subset or smth

loud radish
#

i had a semi broken impl yeah

sinful river
#

What I would do is have a prekernel loaded by limine

#

or ultra whatever

prime wraith
#

terminal feature trl (ik its removed now)

prime wraith
# loud radish its really not that bad

requirements have grown in the last few revisions, e.g. premapping acpi tables which now requires an acpi walker in the bootloader, and iirc there were a few more things like that

#

its not that bad but yes its bad

#

as in a lot of work

sinful river
#

If you wanna do NUMA

#

Well you could do without but it'd be annoying

prime wraith
#

what sort of numa would you do in a bootloader

sinful river
#

Setting things up for the kernel

prime wraith
#

what things?

sinful river
#

You tell it its numa nodes

prime wraith
#

cant the kernel read SRAT itself

sinful river
#

Yeah

#

But it's annoying cuz you have a chicken and egg thing so you use your boot allocator

#

idk

prime wraith
#

does limine do numa stuff?

#

i havent really thought about that much yet

sinful river
latent geode
#

literally unusable

sinful river
#

tf you mean limine won't help my hobby OS run properly on a dual socket server ship

latent geode
#

ikr

loud radish
#

numa is fake news

ornate jasper
sinful river
#

Yeah it's like the pfndb, it's doable but annoying

#

Maybe I'll do bootloader stuff soon

ornate jasper
#

oh nvm i am having the loader tell me numa stuff

#

i already had acpi walking to make sure tables get mapped tho

#

(or rather i use it to put the FACS register area thing in non-temporary vmem range)

#

having a u32 field in the memory map entries for numa node is all you really need to clean up the chicken and egg thing tho

sinful river
#

Yeah that and also info for each cpu

ornate jasper
#

i dont have any per-cpu info from my loader

#

its easy enough to discover when you are building percpu structs later

#

the SRAT gives you both but my bootloader only cares about the memory bits

#

you basically just use the node for each memmap entry when building up your memblock allocator and then use that to build up everything else

sterile kayak
#

Yeah (fully) implementing the limine protocol is a pretty big task

sinful river
#

Might as well do it all in the bootloader idk

ornate jasper
#

no need to do in bootloader

sinful river
#

I don't like it

ornate jasper
#

i think memory map numa nodes is the only thing i do in bootloader and nothing else is even remotely annoying enough to chicken-egg to do in bootloader

sinful river
#

I'd rather do it all in the loader

#

idk

ornate jasper
#

id rather do as little as possible in the loader personally

sinful river
#

Nah, the nicer the environment for the kernel the better

ornate jasper
#

different philosophies ig

#

im already doing a lot more in the loader than even windows does lmao

#

most of this is due to having literally no mechanism for stack unwinding in uefi (due to what might be llvm bug might be zig bug i dont think anyone understands it enough to know which yet) though

#

id have to at least double the LOC of my bootloader to get it to trace stacks on issues

#

so its so hard to debug

carmine token
#

why not just compile with frame pointers? or is that what you're doing?

sinful river
#

I think

#

idk

ornate jasper
sinful river
#

ah

ornate jasper
sinful river
#

What I'm gonna do is have a prekernel loaded by limine that just puts the kernel in a nice state with all APs started

ornate jasper
#

i turn on frame pointers for the efi bootloader and nothing changes or happens and i still dont have frame pointers

#

and loading the pdb from the efi filesystem stuff would probably double the lines of code in my loader

#

(plus idk how to even get the current directory of the current image in efi to load the pdb from)

sinful river
#

I don't do unwinding

#

well, dwarf stuff at all

#

I just use frame pointers

#

And have my own symbol table

ornate jasper
#

as i mentioned, on efi frame pointers dont work at all for me

#

the option just gets ignored

#

so i still cant unwind with them

latent geode
#

I should do cfi

sinful river
ornate jasper
#

ive got frame pointers and cfi for my kernel

ornate jasper
#

thats why i keep the loader simple

#

and only do the bare minimum

#

because that way it doesnt crash and i dont have to debug it

sinful river
#

Don't write bugs 😎

ornate jasper
#

lmao

#

id rather do a prekernel than add more to the loader

sinful river
#

I'm not sure how the prekernel should load the kernel and where it should sit though, should it sit at the place where the kernel would otherwise exist? And then the prekernel loads the kernel after that?

ornate jasper
#

that question is why i never tried to make a prekernel lol

sinful river
ornate jasper
#

winload.efi sets up the memory map, calls SetVirtualAddressMap on efi, loads a few extra acpi tables if theyre missing, and then loads the kernel and calls main
my bootloader sets up the memory map with bonus numa info, calls SetVirtualAddressMap, and then loads the kernel and calls main

#

oh and if the MP protocol for efi is available, winload uses it to invoke the winbvd instruction on all APs after which the APs are all back in standby mode as if nothing happened

#

why it does that i dont fuckin know but it sure does do that

#

i dont even know what the winbvd instruction does lmao

#

oh and winload builds up a gigantic-ass struct it passes to the kernel

carmine token
#

not sure if that answers the question

ornate jasper
#

i think was more about where in the address space they go

#

thats the question i had about prekernels anyway

sinful river
carmine token
#

in virtual memory?

#

or physical

#

the prekernel maps the kernel to its proper place in virtual memory

ornate jasper
#

where in virtual memory is the prekernel then/how are conflicts avoided?

sinful river
#

Yea

carmine token
#

ah

#

the prekernel is either identity mapped (for all boot protocols except limine) or very high (for limine)

obtuse sparrow
#

In my prekernel at least, i start on 32-bit protected mode and the bootloader itself loads both the prekernel as the main executable and the kernel as a module

prime wraith
carmine token
obtuse sparrow
sinful river
#

I think my heap sits there actually