OBOS (not vibecoded) | Operating System Development | Page 36

devout niche Jan 28, 2025, 9:27 AM

#

you'll be wanting to do userspace

#

dhcp clients are usually using PF_LINK sockets (which are for directly dealing with ethernet frames) due to their particular requirements

flint idol Jan 29, 2025, 12:08 AM

#

was readin the TCP spec

#

https://datatracker.ietf.org/doc/html/rfc9293

IETF Datatracker

RFC 9293: Transmission Control Protocol (TCP)

This document specifies the Transmission Control Protocol (TCP). TCP is an important transport-layer protocol in the Internet protocol stack, and it has continuously evolved over decades of use and growth of the Internet. Over this time, a number of changes have been made to TCP as it was specified in RFC 793, though these have only been documen...

#

why tf is it so long

#

nooo

short mortar Jan 29, 2025, 12:19 AM

#

nah it's easy

flint idol Jan 29, 2025, 1:00 AM

#

mhm...

#

I'm going to start TCP soon

#

later today maybe

#

just need to finish something

haughty abyss Jan 29, 2025, 9:06 AM

#

TCP is long because of all the stuff that was added i think

thick jolt Jan 29, 2025, 12:15 PM

#

short mortar nah it's easy

SOOOOO easyyy

short mortar Jan 29, 2025, 12:16 PM

#

what, it's just a sliding window and sending acks and naks when you feel like it

flint idol Jan 29, 2025, 9:54 PM

#

yes definitely

flint idol Jan 29, 2025, 11:56 PM

#

short mortar what, it's just a sliding window and sending acks and naks when you feel like it

if (rand() % 2)
    send_ack();
else
    send_nak();```

flint idol Jan 30, 2025, 12:42 AM

#

It seems like obos' r8169 driver just works™️ on the r8139

sterile sparrow Jan 30, 2025, 6:28 AM

#

Well the 8169 is just a 1000mbps version of the 8139, right?

flint idol Jan 30, 2025, 7:06 AM

#

Idk something like that

thick jolt Jan 30, 2025, 9:54 AM

#

boo

timid elk Jan 30, 2025, 11:02 AM

#

sterile sparrow Well the 8169 is just a 1000mbps version of the 8139, right?

linux has different drivers for them iirc

#

8139 being more generic and 8169 is a fork of it or something

#

i don't think OBOS does anything where that would matter tho

sterile sparrow Jan 30, 2025, 1:01 PM

#

Ah lmao

#

I've seen drivers that work with both so idk

flint idol Jan 30, 2025, 1:04 PM

#

I haven't been able to figure out how to communicate with obos when it runs in qemu

#

So until then

proper trellis Jan 30, 2025, 1:06 PM

#

what networking mode did you specify to qemu?

flint idol Jan 30, 2025, 1:07 PM

#

-nic user,model=r8139

proper trellis Jan 30, 2025, 1:07 PM

#

iirc you have to port forward

#

or use a tap device

#

smth like ,id=net0,hostfwd=tcp::2222-:22

thick jolt Jan 30, 2025, 3:55 PM

#

flint idol -nic user,model=r8139

check dms

#

nooo

vale nymph Jan 30, 2025, 5:22 PM

#

proper trellis smth like `,id=net0,hostfwd=tcp::2222-:22`

Ywah this is what I used to ssh into astral

#

You can even port forward on your router and have someone connect to obos

timid elk Jan 30, 2025, 7:37 PM

#

or just support ipv6 lol

#

you can probably do bridge networking to give qemu an entire ipv6 address

#

if you can't then add that to qemu

flint idol Jan 30, 2025, 7:40 PM

#

bah qemu networking stoobid

timid elk Jan 30, 2025, 7:42 PM

#

qemu is stupid roll your own emulator

#

obemu

short mortar Jan 30, 2025, 9:04 PM

#

At that point just roll your own isa

flint idol Jan 30, 2025, 11:29 PM

#

vale nymph You can even port forward on your router and have someone connect to obos

ngl anything that would be considered normal traffic would crash obos

#

fun fact: virtualbox does not have _S3

#

            pg->pagedCount--;
            MmH_DerefPage(pg);```

#

I just found

#

the stupidest lines of code

#

in obos

uncut narwhal Jan 31, 2025, 12:04 AM

#

congrats

flint idol Jan 31, 2025, 2:37 AM

#

I love it when bash segfaults:

[  LOG  ] User thread 7 SIGSEGV (rip 0x0, cr2 0x0, error code 0x00000014)```

#

trl

thick jolt Jan 31, 2025, 4:41 PM

#

trl

flint idol Feb 1, 2025, 2:01 AM

#

bash is driving me nuts

#

it keeps segfaulting

leaden carbon Feb 1, 2025, 2:02 AM

#

bashes fault

#

4am debugging goes crazy

flint idol Feb 1, 2025, 2:04 AM

#

yes definitely all bash's fault

#

I implemented a mount syscall

#

#

(the mount command is some makeshift one I made myself in 5 minutes)

flint idol Feb 1, 2025, 2:26 AM

#

#

added a umount command

#

flint idol Feb 1, 2025, 4:42 AM

#

damn obos rewrite 5 old days

#

still have those same logs

#

thick jolt Feb 1, 2025, 6:00 AM

#

flint idol

monkey

thick jolt Feb 1, 2025, 6:02 AM

#

flint idol

gdt OK

#

idt OK

#

cpu OK

#

printf OK

timid elk Feb 1, 2025, 9:17 AM

#

flint idol (the `mount` command is some makeshift one I made myself in 5 minutes)

gnu coreutils when

#

actually some bsd's coreutils would be interesting

flint idol Feb 1, 2025, 11:58 AM

#

timid elk gnu coreutils when

I have coreutils

flint idol Feb 1, 2025, 11:58 AM

#

thick jolt monkey

Orangutan

flint idol Feb 1, 2025, 3:06 PM

#

yeah ikr-

[  LOG  ] User thread 7 SIGSEGV (rip 0x0, cr2 0x0, error code 0x00000014)```

flint idol Feb 1, 2025, 7:06 PM

#

I've been working on adding better cross compilation support to obos-strap

#

as well as host and target packages

proper trellis Feb 1, 2025, 7:40 PM

#

what happend to tcp?

timid elk Feb 1, 2025, 7:43 PM

#

flint idol I've been working on adding better cross compilation support to `obos-strap`

you use autotools-inspired build/host/target platforms right?

#

e.g. build is the machine that builds, host is the mschine that runs and target is the machine that host compiles for

flint idol Feb 1, 2025, 9:04 PM

#

yes

flint idol Feb 1, 2025, 9:04 PM

#

proper trellis what happend to tcp?

soon™️

devout niche Feb 1, 2025, 10:20 PM

#

obostrap pictured

flint idol Feb 1, 2025, 10:26 PM

#

KEKW

flint idol Feb 1, 2025, 11:31 PM

#

these new changes to obos-strap have caused me to rebuild gcc thrice today

#

anyway so now host packages (like the bootstrap gcc binaries) are all built under build_root/host_pkgs whilst target packages are built under build_root/pkgs

flint idol Feb 2, 2025, 1:30 AM

#

In file included from /home/oberrow/Code/obos-pkgs/host_pkgs/lib/gcc/x86_64-obos/14.2.0/include/syslimits.h:7,
                 from /home/oberrow/Code/obos-pkgs/host_pkgs/lib/gcc/x86_64-obos/14.2.0/include/limits.h:34,
                 from /home/oberrow/Code/obos-master/src/oboskrnl/external/nanoprintf.h:65,
                 from /home/oberrow/Code/obos-master/src/oboskrnl/klog.c:38:
/home/oberrow/Code/obos-pkgs/host_pkgs/lib/gcc/x86_64-obos/14.2.0/include/limits.h:210:15: fatal error: limits.h: No such file or directory
  210 | #include_next <limits.h>                /* recurse down to the real one */

#

bruh what the hell

#

what why would it want mlibc's limits header

flint idol Feb 2, 2025, 2:19 AM

#

rebuilt gcc

#

works now

torpid wigeon Feb 2, 2025, 2:29 AM

#

flint idol these new changes to `obos-strap` have caused me to rebuild gcc thrice today

that's nothing

#

i'm currently on my 12th compile

timid elk Feb 2, 2025, 9:31 AM

#

flint idol these new changes to `obos-strap` have caused me to rebuild gcc thrice today

only thrice? I rebuild gcc every day

#

it's part of my routine now

#

i wake up

#

change stuff in mlibc

#

rebuild gcc (twice, bootstrapping it from the compiler that compiles the compiler)

#

then breakfast as it compiles

#

then repeat

flint idol Feb 2, 2025, 2:29 PM

#

I've been debugging the sleep command in coreutils

#

because it just like doesn't work in obos

#

(it hangs indefinitely)

#

    memzero(Core_GetCurrentThread()->context.extended_ctx_ptr, Arch_GetXSaveRegionSize());
    if (obos_expect(Arch_HasXSAVE, true))
        __builtin_ia32_xrstor(Core_GetCurrentThread()->context.extended_ctx_ptr, __builtin_ia32_xgetbv(0));
    else
        __builtin_ia32_fxrstor(Core_GetCurrentThread()->context.extended_ctx_ptr);```

#

idk why but I feel like doing this to reset extended context state is wrong

#

well I tried looking at astral

#

it only uses fxsave

#

The XSAVE header of a newly allocated XSAVE area should be initialized to all zeroes before saving context. An
operating system may choose to establish beginning state-component values for a task by executing XRSTOR from
an XSAVE area that the OS has configured. If it is desired to begin state component i in its initial configuration, the
OS should clear bit i in the XSTATE_BV field in the XSAVE header; otherwise, it should set that bit and place the
desired beginning value in the appropriate location in the XSAVE area

#

from intel sdm

#

volume 3 chapter 15

#

section 5.4

white mulch Feb 2, 2025, 2:58 PM

#

flint idol idk why but I feel like doing this to reset extended context state is wrong

do you set up the fcw and mxcsr in the area?

flint idol Feb 2, 2025, 2:58 PM

#

don't think so

#

this is before an execve call passes control to userspace btw

#

I also don't even touch fcw

white mulch Feb 2, 2025, 2:59 PM

#

https://github.com/Qwinci/crescent/blob/4b9ccc5ce59e786937191627d06a849652496571/src/arch/x86/arch_sched.cpp#L190

#

you are supposed to set the initial state to match what the sysv abi specifies

flint idol Feb 2, 2025, 3:01 PM

#

I see

#

static __attribute__((target("xsave"))) __attribute__((target("avx"))) void reset_extended_state()
{
    asm volatile("fninit");
    memzero(Core_GetCurrentThread()->context.extended_ctx_ptr, Arch_GetXSaveRegionSize());
    static const uint32_t mxcsr = 0x1f80;
    *((uint32_t*)(Core_GetCurrentThread()->context.extended_ctx_ptr + 24)) = mxcsr;
    *((uint16_t*)(Core_GetCurrentThread()->context.extended_ctx_ptr)) = 0x33f;
    if (obos_expect(Arch_HasXSAVE, true))
        __builtin_ia32_xrstor(Core_GetCurrentThread()->context.extended_ctx_ptr, __builtin_ia32_xgetbv(0));
    else
        __builtin_ia32_fxrstor(Core_GetCurrentThread()->context.extended_ctx_ptr);
}```

#

this is in an effort to get bash to stop segfaulting

#

and I think it worked

#

jinxed it

#

[  LOG  ] User thread 7 SIGSEGV (rip 0x0, cr2 0x0, error code 0x00000014)

#

yeah bash likes to segfault

#

I think it's a bug with fork

thick jolt Feb 2, 2025, 5:32 PM

#

hey ive come to check on progress with obos

#

it seems your encountering a bug with

#

fork()?

#

yep

flint idol Feb 2, 2025, 5:32 PM

#

It's a bug somewhere

#

I think it's a bug with fork

thick jolt Feb 2, 2025, 5:33 PM

#

understandable

#

well maybe i could help like look through the code myself

#

i wont write anything cause yk cast

#

but

#

ill try to look for it

#

can i see ur fork code

#

if thats okay omar

#

@flint idol

flint idol Feb 2, 2025, 5:34 PM

#

Ye sure

#

Hope you heal well btw

thick jolt Feb 2, 2025, 5:34 PM

#

thanks omar

flint idol Feb 2, 2025, 5:35 PM

#

https://github.com/OBOS-dev/obos/blob/master/src%2Foboskrnl%2Fmm%2Ffork.c

GitHub

obos/src/oboskrnl/mm/fork.c at master · OBOS-dev/obos

I was bored... Contribute to OBOS-dev/obos development by creating an account on GitHub.

thick jolt Feb 2, 2025, 5:37 PM

#

from what im looking at

#

the only thing im suspecting is this line

#

hold on

#

wait

#

no

#

it looks valid

flint idol Feb 2, 2025, 5:38 PM

#

it could also be a bug elsewhere

#

adding printfs seem to delay the bug happening

white mulch Feb 2, 2025, 5:39 PM

#

💀 those are the worst kinds of bugs tbh

flint idol Feb 2, 2025, 5:40 PM

#

if it is a memory manager bug, it's probably a use after free of a physical page

thick jolt Feb 2, 2025, 5:40 PM

#

flint idol if it is a memory manager bug, it's probably a use after free of a physical page

maybe it could be that cause the code all looks fine

flint idol Feb 2, 2025, 5:41 PM

#

I guess an easy way to figure that out is by examining each use of struct page

#

and making sure each one references it and dereferences it at the appropriate times

#

which I am doing now

#

static bool sym_cow_cpy(context* ctx, page_range* rng, uintptr_t addr, uint32_t ec, page* pg, page_info* info)
{
    info->prot.present = true;
    if (pg->refcount == 1 /* we're the only one left */)
    {
        // Steal the page.
        info->prot.rw = true;
        info->prot.ro = false;
        MmS_SetPageMapping(ctx->pt, info, pg->phys, false);
        pg->cow_type = COW_DISABLED;
        return true;
    }
    page* new = MmH_PgAllocatePhysical(rng->phys32, info->prot.huge_page);
    new->pagedCount++;
    memcpy(MmS_MapVirtFromPhys(new->phys), MmS_MapVirtFromPhys(pg->phys), info->prot.huge_page ? OBOS_HUGE_PAGE_SIZE : OBOS_PAGE_SIZE);
    info->prot.rw = true;
    info->prot.ro = false;
    info->phys = new->phys;
    MmS_SetPageMapping(ctx->pt, info, new->phys, false);
    pg->pagedCount--;
    MmH_DerefPage(pg);
    return true;
}

#

this is the actual CoW handler

main girder Feb 2, 2025, 5:45 PM

#

it handle cow?

flint idol Feb 2, 2025, 5:45 PM

#

which at a glance doesn't seem to do anything stupid

flint idol Feb 2, 2025, 5:45 PM

#

main girder it handle cow?

yes it moos

main girder Feb 2, 2025, 5:46 PM

#

what guards the decrement on pg->pagedCount--

#

that is not atomic

flint idol Feb 2, 2025, 5:46 PM

#

    _Atomic(size_t) pagedCount;

#

it's implcitily atomic

main girder Feb 2, 2025, 5:46 PM

#

that doesnt uh

#

i dont think that works

flint idol Feb 2, 2025, 5:46 PM

#

I think

main girder Feb 2, 2025, 5:46 PM

#

no

#

i do not believe that will necessarily cause a lock prefix on x86 or for an ll/sc sequence to be generated on riscs or whatever

#

you should check and see what its emitting

flint idol Feb 2, 2025, 5:47 PM

#

good idea

#

the compiler generates a lock sub instruction

main girder Feb 2, 2025, 5:57 PM

#

inchresting

#

what if you compile it for like risc-v

flint idol Feb 2, 2025, 5:58 PM

#

I don't support risc-v

#

but I could try m68k

#

void MmH_RefPage(page* buf)
{
    buf->refcount++;
    printf("refed page %p (now at %d)\n", buf->phys, buf->refcount);
}
void MmH_DerefPage(page* buf)
{
    printf("derefed page %p (now at %d)\n", buf->phys, buf->refcount - 1);
    if (!(--buf->refcount))
    {
        if (~buf->flags & PHYS_PAGE_MMIO)
            Mm_FreePhysicalPages(buf->phys, ((buf->flags & PHYS_PAGE_HUGE_PAGE) ? OBOS_HUGE_PAGE_SIZE : OBOS_PAGE_SIZE) / OBOS_PAGE_SIZE);
        // printf("removed page from tree (refcount %d)\n", buf->refcount);
        RB_REMOVE(phys_page_tree, &Mm_PhysicalPages, buf);
        Mm_PhysicalMemoryUsage -= ((buf->flags & PHYS_PAGE_HUGE_PAGE) ? OBOS_HUGE_PAGE_SIZE : OBOS_PAGE_SIZE);
        Mm_Allocator->Free(Mm_Allocator, buf, sizeof(*buf));
    }
}```

#

this is probably correct...

#

    mov eax, Sys_ProcessStart
    mov rsi, rbx
    mov rdx, 1
    syscall
    mov r14, rax

    // Get the PID
    // TODO: Can this be a race condition if the process ends fast enough?
    mov rdi, r14
    mov eax, Sys_ProcessGetPid
    syscall

#

this is my fork code in userspace

#

maybe through some miracle that race condition happens

#

and somehow breaks all

white mulch Feb 2, 2025, 6:03 PM

#

main girder no

Built-in increment and decrement operators and compound assignment are read-modify-write atomic operations with total sequentially consistent ordering (as if using memory_order_seq_cst).

main girder Feb 2, 2025, 6:04 PM

#

white mulch > Built-in increment and decrement operators and compound assignment are read-mo...

epic

main girder Feb 2, 2025, 6:04 PM

#

flint idol ```c void MmH_RefPage(page* buf) { buf->refcount++; printf("refed page %...

this is broken though

#

you have been fooled by atomic into thinking it will save you from all of your sins

#

even if those incs and decs are atomic im p sure this is broken

flint idol Feb 2, 2025, 6:05 PM

#

thonk

#

elaborate?

main girder Feb 2, 2025, 6:06 PM

#

do you ever call DerefPage at a time where the refcount may be 1 and some other guy may still have a pointer to the page?

#

but has not yet called RefPage himself

flint idol Feb 2, 2025, 6:06 PM

#

I'd hope not

#

it is possible

main girder Feb 2, 2025, 6:07 PM

#

how do you deal with putting pages on like standby/modified/whatever list

flint idol Feb 2, 2025, 6:08 PM

#

I looked into that

#

I forgot to reference the page 💀

weak kestrel Feb 2, 2025, 6:08 PM

#

main girder you have been fooled by atomic into thinking it will save you from all of your s...

this is a pinnable message

flint idol Feb 2, 2025, 6:14 PM

#

[  LOG  ] User thread 7 SIGSEGV (rip 0x0, cr2 0x0, error code 0x00000014)```

#

hmm

#

what if

#

it's calling a weak function

#

in mlibc

#

a weak sysdep specifically

#

I would expect mlibc to check for that as it usually does

#

and return ENOSYS

#

the error code passed by the cpu

#

so 0x14 means executing a non present page in user mode

#

it's executing nullptr trl

#

no

#

since the execute bit is set in the error code

#

and cr2=0

#

that means the program tried to execute 0x0

white mulch Feb 2, 2025, 6:27 PM

#

address

#

its always address

flint idol Feb 2, 2025, 6:31 PM

#

no

#

wrong

#

yeah no stack corruption

#

is the cause of the bug

#

but if that were the case

#

that would also mean rbp is broke

#

  4b8381:       5d                      pop    %rbp
  4b8382:       c3                      ret

that's what I get looking into the function bash calls before dying

white mulch Feb 2, 2025, 6:46 PM

#

https://github.com/managarm/mlibc/blob/71ddf2d365b56de6d7dff40dd35c2de572ff38be/options/posix/generic/posix_stdlib.cpp#L243 this is the only place in mlibc that I can find that lacks proper sysdep checking

flint idol Feb 2, 2025, 6:47 PM

#

yeah this PF is happening likely right after a ret statement

#

wait if the stack is corrupted then so is the popped rbp

#

(likely)

white mulch Feb 2, 2025, 6:47 PM

#

well I guess it depends

#

it could be that only stuff starting at the ret addr would be corrupted

flint idol Feb 2, 2025, 6:49 PM

#

possibly

#

just in case I will stub out sys_mkdir

#

yeah no it persists

#

but the thing is it doesn't always happen

white mulch Feb 2, 2025, 6:58 PM

#

how much stack space do you give to the thread?

flint idol Feb 2, 2025, 7:06 PM

#

white mulch how much stack space do you give to the thread?

4M

weak kestrel Feb 2, 2025, 7:07 PM

#

isn't that a bit much

flint idol Feb 2, 2025, 7:07 PM

#

Idk

#

#

Btw I saw what you deleted

#

https://tenor.com/view/i-saw-what-you-deleted-cat-gif-25407007

Tenor

white mulch Feb 2, 2025, 7:08 PM

#

yeah I deleted it because I realized that this probably doesn't happen when just running bash and not doing anything in there?

flint idol Feb 2, 2025, 7:09 PM

#

No it only happens in bash afaik

white mulch Feb 2, 2025, 7:09 PM

#

does it happen if you just run plain bash

#

like when you don't type anything to there

flint idol Feb 2, 2025, 7:10 PM

#

It only happens after typing a command

#

And it only happens when fork has been called at least once

#

Occasionally a SIGSEGV happens before that

#

And that segfault is because of a GPF

#

     293 text files.
     293 unique files.                                          
       0 files ignored.

github.com/AlDanial/cloc v 2.02  T=0.24 s (1200.9 files/s, 205999.5 lines/s)
-------------------------------------------------------------------------------
Language                     files          blank        comment           code
-------------------------------------------------------------------------------
C                              136           2591           3004          28549
C/C++ Header                   143           1799           2360           9889
Assembly                         9            226            124           1241
CMake                            5             72             35            372
-------------------------------------------------------------------------------
SUM:                           293           4688           5523          40051
-------------------------------------------------------------------------------

#

OBOS LoC update

#

I need to get rid of 51 un needed LoC

#

to get it to exactly 40k meme

flint idol Feb 3, 2025, 12:44 AM

#

I want to port obos to i686 /srs

#

I will start in a bit

#

anywhere from 1-2 buisness days

#

it will be my 3rd port

#

it will use the same drivers as x86_64

#

and interrupt controller stuff

#

stuff like that

#

not much will be different except for paging, gdt, and idt code

#

and syscall abi

#

and probably some other stuff that I can't think of off the top of my head

#

thread context info stuff

#

I will also probably shift away from using the HHDM

flint idol Feb 3, 2025, 12:51 AM

#

flint idol thread context info stuff

execve handoff code might need to be changed according to the sysv abi for i386

#

and signal code

#

I would also be adding PIC and PIT support

#

which actually might be tricky considering how my abstract IRQ interface works, but I think I'll be able to find it out

#

it should be fun, but until then...

#

actually I should poll what arch I should port to next

#

#polls message
pls vote

mortal dock Feb 3, 2025, 2:01 AM

#

1 vote risc-v 64 😭

timid elk Feb 3, 2025, 2:53 PM

#

flint idol It only happens after typing a command

strace it

flint idol Feb 3, 2025, 9:09 PM

#

Good idea

#

It's almost as if I never tried to do that

flint idol Feb 3, 2025, 9:51 PM

#

flint idol Feb 3, 2025, 10:08 PM

#

time to start itanium port

#

uh so this will be fun

#

I guess I'll start with compiling an ia64-gcc toolchain

#

on gcc 15

#

or gcc 13.3

#

since I want a tarball release I will go with 13.3

real pecan Feb 3, 2025, 10:12 PM

#

how do u boot on m68k atm?

#

did u roll something for that

flint idol Feb 3, 2025, 10:12 PM

#

I stole @ornate ginkgo's limine stub

#

hmm could I just use debian's packaged ia64 binutils and gcc

#

https://packages.debian.org/sid/binutils-ia64-linux-gnu

#

https://packages.debian.org/sid/gcc-ia64-linux-gnu

real pecan Feb 3, 2025, 10:14 PM

#

add support for ia64-be

flint idol Feb 3, 2025, 10:15 PM

#

I'll do that one day

#

itanium surely has ACPI, right?

#

I know it has EFI, iirc it was made for itanium (or with itanium in mind)

ornate ginkgo Feb 3, 2025, 10:16 PM

#

It has acpi

flint idol Feb 3, 2025, 10:16 PM

#

based

#

I will be the first to test uacpi on itanium

#

probably

ornate ginkgo Feb 3, 2025, 10:17 PM

#

GL with the port, itanium looks interesting

flint idol Feb 3, 2025, 10:17 PM

#

thanks

#

I need debports it seems

#

for that package

#

#

oh nvm

real pecan Feb 3, 2025, 10:32 PM

#

flint idol itanium surely has ACPI, right?

itanium has full acpi

#

not even hw reduced

flint idol Feb 3, 2025, 10:32 PM

#

waow

#

this should be fun

real pecan Feb 3, 2025, 10:32 PM

#

yeah

flint idol Feb 3, 2025, 10:32 PM

#

(famous last words)

#

currently compiling ia64-elf binutils

#

and ia64-elf-gcc

#

compiled ia64-elf binutils

#

it was able to assemble and link some random ass program I found online

#

wait I haven't linked it yet

#

was able to link it

#

I think gcc just finished compiling

#

this took way too little time

#

it's compiling libgcc now

#

checking for suffix of object files... configure: error: in `/home/oberrow/Code/obos-pkgs-ia64/bootstrap/bootstrap-gcc/ia64-elf/libgcc':
configure: error: cannot compute suffix of object files: cannot compile

#

flint idol Feb 3, 2025, 11:06 PM

#

libgcc isn't compiling

#

it's compiling about some libc headers

#

couldn't care less about libc headers

#

so I'm ignoring it

flint idol Feb 3, 2025, 11:41 PM

#

flint idol Feb 4, 2025, 12:34 AM

#

@dim bolt know of any itanium emulators?

dim bolt Feb 4, 2025, 12:34 AM

#

flint idol <@1223455220948930650> know of any itanium emulators?

ski

#

if you hate yourself

flint idol Feb 4, 2025, 12:35 AM

#

I'm making an itanium port, so of course I hate myself /j

#

https://github.com/trofi/ski

GitHub

GitHub - trofi/ski: ia64 (Itanium) instruction set simulator. Fork ...

ia64 (Itanium) instruction set simulator. Fork of http://ski.sourceforge.net - trofi/ski

#

this?

dim bolt Feb 4, 2025, 12:35 AM

#

ya

flint idol Feb 4, 2025, 12:35 AM

#

kthx

#

../configure: line 5959: syntax error near unexpected token `-Werror=implicit-int,'
../configure: line 5959: `AX_CHECK_COMPILE_FLAG(-Werror=implicit-int, CFLAGS="$CFLAGS -Werror=implicit-int", , -Werror)'

#

bruh

#

ok whatever

#

I got it to configure

#

yay it compiles

flint idol Feb 4, 2025, 1:52 AM

#

Ok I was able to get a basic itanium while loop running

#

Literally no firmware

#

No bootloader

#

Just while loop

honest citrus Feb 4, 2025, 2:01 AM

#

oberrow on his way to write the entire boot stack, including the firmware, up to OBOS just to flex ultrameme

dim bolt Feb 4, 2025, 2:05 AM

#

flint idol No bootloader

check the ski-bootloader
or just the ski boot functions

flint idol Feb 4, 2025, 2:07 AM

#

I'm using simics now

main girder Feb 4, 2025, 2:09 AM

#

honest citrus oberrow on his way to write the entire boot stack, including the firmware, up to...

not possible.

#

never been done.

dim bolt Feb 4, 2025, 2:10 AM

#

flint idol I'm using simics now

my condolences

flint idol Feb 4, 2025, 2:20 AM

#

yeah this makes no sense

#

itanium would be a lot more fun

#

if there were good emulators

#

so I might not do it

main girder Feb 4, 2025, 2:25 AM

#

port OBOS to R3000 Jazz

flint idol Feb 4, 2025, 2:25 AM

#

No.

main girder Feb 4, 2025, 2:26 AM

#

why

flint idol Feb 4, 2025, 2:26 AM

#

don't know what it is

main girder Feb 4, 2025, 2:26 AM

#

32 bit mips dev board that hasnt existed since 1990

dim bolt Feb 4, 2025, 2:26 AM

#

flint idol if there were good emulators

@upper tulip poke!

main girder Feb 4, 2025, 2:27 AM

#

main girder 32 bit mips dev board that hasnt existed since 1990

it now re-exists but as basically a fantasy computer that only boots an incredibly ancient windows nt build

dim bolt Feb 4, 2025, 2:27 AM

#

main girder 32 bit mips dev board that hasnt existed since 1990

wonderful.

main girder Feb 4, 2025, 2:27 AM

#

it could also boot OBOS.

upper tulip Feb 4, 2025, 2:29 AM

#

dim bolt <@333472970318151680> poke!

I need the science to do this to myself meme

upper tulip Feb 4, 2025, 2:29 AM

#

flint idol No.

port it to the WRT54G meme

main girder Feb 4, 2025, 2:29 AM

#

main girder it could also boot OBOS.

it would unironically be easier to port OBOS to r3k jazz than to port it to itanium

#

by like 2x

#

but if u did mips youd probably want to do it on something that exists physically outside of a museum

dim bolt Feb 4, 2025, 2:30 AM

#

upper tulip I need the science to do this to myself <:meme:575445094589661215>

trueee

flint idol Feb 4, 2025, 2:38 AM

#

since Itanium is proving to be difficult

#

I will port to something else

#

stay tuned™️

dim bolt Feb 4, 2025, 2:44 AM

#

flint idol I will port to something else

SPARC

upper tulip Feb 4, 2025, 3:02 AM

#

flint idol I will port to something else

SH3

honest citrus Feb 4, 2025, 5:52 AM

#

fall back to the 2nd closest option in the polls, PPC :)

real pecan Feb 4, 2025, 7:43 AM

#

flint idol since Itanium is proving to be difficult

why

devout niche Feb 4, 2025, 9:31 AM

#

real pecan why

Not a popular target

#

No one really knows about it besides a few devotees

short mortar Feb 4, 2025, 9:34 AM

#

main girder but if u did mips youd probably want to do it on something that exists physicall...

Routers bruh

upper tulip Feb 4, 2025, 9:35 AM

#

real pecan why

emulation isn't great

upper tulip Feb 4, 2025, 9:35 AM

#

short mortar Routers <:bruh:761542241381908491>

I already suggested WRT54G meme

short mortar Feb 4, 2025, 9:35 AM

#

You can run PPC port on game consoles and old Macs ultrameme

short mortar Feb 4, 2025, 9:36 AM

#

upper tulip I already suggested WRT54G <:meme:575445094589661215>

But that thing has 2MB of ram

#

There are newer ones

#

(I have many)

upper tulip Feb 4, 2025, 9:36 AM

#

short mortar But that thing has 2MB of ram

*16MB

short mortar Feb 4, 2025, 9:36 AM

#

Yeah, 2MB flash and 16MB RAM

#

upper tulip Feb 4, 2025, 9:36 AM

#

Plenty of room meme

short mortar Feb 4, 2025, 9:37 AM

#

But even openwrt requires 64 now

#

(I wanted to do powerpc power meme )

upper tulip Feb 4, 2025, 9:40 AM

#

short mortar (I wanted to do powerpc power <:meme:575445094589661215>)

I believe there are some powerpc routers out there

short mortar Feb 4, 2025, 9:40 AM

#

There are a lot of weird arches in routers

upper tulip Feb 4, 2025, 9:40 AM

#

Or...

ornate ginkgo Feb 4, 2025, 9:41 AM

#

now that would be an interesting port

short mortar Feb 4, 2025, 9:41 AM

#

Or

#

(I have both)

short mortar Feb 4, 2025, 9:42 AM

#

upper tulip Or...

But PS3 is not multiprocessor unfortunately

#

But Wii U is ultrameme

#

And very cursed

#

(also, there is a normal Wii)

white mulch Feb 4, 2025, 9:43 AM

#

or ps2 with mips64 KEKW

short mortar Feb 4, 2025, 9:43 AM

#

@upper tulip how is your bootloader doing?

upper tulip Feb 4, 2025, 9:50 AM

#

short mortar <@333472970318151680> how is your bootloader doing?

Progressing meme

honest citrus Feb 4, 2025, 1:34 PM

#

short mortar (also, there is a normal Wii)

indeed, and you happen to have one of the best resources for it available: me (literally the developer of the port of Linux for it KEKW )

#

I'm also in close contact with the dev of linux-wiiu

#

porting to either one would be hilarious, and you have the ultimate advantage in doing so

short mortar Feb 4, 2025, 2:01 PM

#

pmOS should be portable, but I don't want to figure out how to boot on it

#

(sorry for wrong thread)

flint idol Feb 4, 2025, 2:32 PM

#

N

#

p

barren merlin Feb 4, 2025, 2:32 PM

#

d

flint idol Feb 4, 2025, 3:14 PM

#

I'm going to keep the platform I'm porting to a secret

#

Only posting screenshots occasionally

#

Won't push anything to GitHub either

#

I will have a local git server just in case™️

white mulch Feb 4, 2025, 3:29 PM

#

flint idol I'm going to keep the platform I'm porting to a secret

meme

#

I should also port my kernel to something, though I am still in the middle of deciding what I want to do about things like gpu drivers (I could possibly manage to nih intel fully including accel render stuff but some other gpus like nvidia are very much likely even bigger pain because there are no proper specs and only nouveau)

#

its why I have been mostly working on the separate nt thing for now

honest citrus Feb 4, 2025, 4:42 PM

#

imagine oberrow ports to the Xbox 360 (the 5 people with modded consoles can run it)

real pecan Feb 4, 2025, 4:49 PM

#

ps4 port

short mortar Feb 4, 2025, 4:50 PM

#

X86 that's not a PC

flint idol Feb 4, 2025, 5:02 PM

#

honest citrus imagine oberrow ports to the Xbox 360 (the 5 people with modded consoles can run...

Good idea

timid elk Feb 4, 2025, 5:18 PM

#

port it to the lego ev3

short mortar Feb 4, 2025, 5:23 PM

#

so ppc, just as I though

flint idol Feb 4, 2025, 5:25 PM

#

You'll see what it is.

#

Maybe I'll do two ports

#

(this will be a hell to maintain NGL)

short mortar Feb 4, 2025, 5:46 PM

#

why

#

I have 3 ports and it's ok

flint idol Feb 4, 2025, 5:48 PM

#

Idk

#

I would then have 4 ports

#

I want to be the one with the most ports on the server, that would be cool

short mortar Feb 4, 2025, 5:50 PM

#

thonk then I need aarch64 urgently

#

and armv6

#

(since I have physical hardware where I can test it)

#

pmOS already supports device trees so it shouldn't be too difficult... (famous last words)

torpid wigeon Feb 4, 2025, 5:52 PM

#

https://tenor.com/view/clueless-gif-24395495

Tenor

white mulch Feb 4, 2025, 5:53 PM

#

I have a 32-bit arm tablet

flint idol Feb 4, 2025, 5:53 PM

#

I will say one thing, I am not porting to aarch64

#

Rn

white mulch Feb 4, 2025, 5:54 PM

#

but the thing is that I think 32-bit arm is pretty much annoying, especially considering that its 32-bit so using hhdm is not a good idea

flint idol Feb 4, 2025, 6:08 PM

#

After these ports, I want to get mlibc running on m68k

#

For obos

#

And just a userspace in general for m68k

#

Should be fun

real pecan Feb 4, 2025, 6:11 PM

#

Does obos support i686

timid elk Feb 4, 2025, 6:14 PM

#

short mortar and armv6

do armv5 instead!!

#

so you can port to lego ev3

short mortar Feb 4, 2025, 6:15 PM

#

white mulch but the thing is that I think 32-bit arm is pretty much annoying, especially con...

I already (almost) have i686 without HHDM, so it's not gonna be an issue for me

#

I mean not almost, I just want to add PAE before I merge it to main branch

white mulch Feb 4, 2025, 6:18 PM

#

well actually maybe its not that annoying if you abstract hhdm away into some kind of phys mem range accessor that just uses the hhdm if its available and else temporary mapping

short mortar Feb 4, 2025, 6:19 PM

#

that's exactly what I have

white mulch Feb 4, 2025, 6:20 PM

#

yeah I were thinking that if you'd have to just get rid of every use of the hhdm completely and unconditionally replace them it would be annoying and would cause unnecessary mapping on arches where you can have a hhdm

#

maybe I should try to do a 32-bit port too

short mortar Feb 4, 2025, 6:21 PM

#

Wouldn't compilers optimize it to the same code if you have HHDM anyway?

white mulch Feb 4, 2025, 6:22 PM

#

with the phys mem range accessor using hhdm if its available yeah, that was an unrelated though that I had before I realized that

#

like if you'd get rid of the hhdm on all arches

short mortar Feb 4, 2025, 6:24 PM

#

I didn't have it in the first place

flint idol Feb 4, 2025, 6:49 PM

#

real pecan Does obos support i686

No

timid elk Feb 4, 2025, 6:50 PM

#

does obos support lego ev3

real pecan Feb 4, 2025, 6:50 PM

#

unbased

timid elk Feb 4, 2025, 6:51 PM

#

i am in need of a hybrid kernel with advanced driver loading for my lego ev3

uncut narwhal Feb 4, 2025, 7:45 PM

#

while at it can we get lego nxt support? I need a new kernel for mine

#

lmao

flint idol Feb 4, 2025, 7:45 PM

#

timid elk i am in need of a hybrid kernel with advanced driver loading for my lego ev3

Lol I should change that description

short mortar Feb 4, 2025, 7:53 PM

#

Does it have MMU?

devout niche Feb 4, 2025, 7:54 PM

#

concerning ports

#

i have no objection to a powerpc port of OBOS

timid elk Feb 4, 2025, 7:58 PM

#

short mortar Does it have MMU?

lego ev3? it's a armv5tej CPU, and it runs linux

#

and there's a debian based distro ev3dev that runs a closer-to-mainline kernel

#

NXT is sadly a microcontroller

#

spike is also a microcontroller

#

lego ev3 is so funn

#

like the dpad lights are controllable with sysfs

#

and you can play doom on it. with sound

#

it has audio, monochrome LCD, a dpad, a host USB port (with an official wifi dongle but in ev3dev you can use any wifi dongle or ethernet adapter or flash drive or keyboard or mouse), bluetooth, a micro usb port that you can use for usb tethering to get internet through a laptop

#

and most importantly it has lego mindstorms connectors for sensors and motors

#

so you can make a robot that passes you butter

flint idol Feb 4, 2025, 8:31 PM

#

Found OBOS while watching football clips

flint idol Feb 5, 2025, 12:23 AM

#

#

I got logging to work

#

on the mysterious target

short mortar Feb 5, 2025, 12:25 AM

#

i686, I see

flint idol Feb 5, 2025, 12:25 AM

#

no

#

it is 32-bit though

short mortar Feb 5, 2025, 12:26 AM

#

thonk powerpc instructions are 4 bytes long...

flint idol Feb 5, 2025, 12:27 AM

#

it's a mystery platform

short mortar Feb 5, 2025, 12:27 AM

#

and mips

#

I need to see more stack traces

flint idol Feb 5, 2025, 12:27 AM

#

anyway I need to get an IRQ controller working on this

short mortar Feb 5, 2025, 12:27 AM

#

flint idol it's a mystery platform

mystery powerpc platform

flint idol Feb 5, 2025, 12:28 AM

#

IRQL might be annoying to get working

#

PPC is after this port.

#

probably

timid elk Feb 5, 2025, 1:02 PM

#

flint idol it is 32-bit though

is it armv5tej

flint idol Feb 5, 2025, 1:13 PM

#

timid elk is it armv5tej

Is that an x86 SSE mnemonic?

timid elk Feb 5, 2025, 1:14 PM

#

no it's arm version 5 with thumb and jazelle

weak kestrel Feb 5, 2025, 1:18 PM

#

flint idol Is that an x86 SSE mnemonic?

GF2P8AFFINEINVQB

honest citrus Feb 5, 2025, 5:18 PM

#

flint idol PPC is after this port.

based

#

which version(s) of PPC do you plan to target

#

ppc64le? ppc64be? ppc32be?

short mortar Feb 5, 2025, 5:53 PM

#

why not all?

real pecan Feb 5, 2025, 5:54 PM

#

ppc128

#

in middle endian mode

timid elk Feb 5, 2025, 6:28 PM

#

MIDDLE ENDIAN

#

middle eastern mode

#

ppc64me, middle eastern mode

flint idol Feb 5, 2025, 7:18 PM

#

honest citrus which version(s) of PPC do you plan to target

All ultrameme

honest citrus Feb 5, 2025, 7:18 PM

#

eyesshaking

#

hype overload

flint idol Feb 5, 2025, 10:57 PM

#

I got the IRQ controller working probably

#

time to get scheduling working

honest citrus Feb 6, 2025, 1:31 AM

#

I really wonder what platform this could be

flint idol Feb 6, 2025, 1:32 AM

#

I can't tell if you're being sarcastic or not

honest citrus Feb 6, 2025, 1:32 AM

#

so many options

#

there's tons of 32-bit architectures lmfao

#

you could literally be doing anything

#

ARM, MIPS, SPARC, RISC-V, etc

#

since it's not i686, or 32-bit ppc

#

but there's still tons

flint idol Feb 6, 2025, 1:33 AM

#

I'll tell you guys soon

timid elk Feb 6, 2025, 10:09 AM

#

it's r3000 jazz

#

or lego ev3

flint idol Feb 6, 2025, 3:11 PM

#

I got thread context switching done

flint idol Feb 6, 2025, 3:32 PM

#

turns out I overflow the goddamned stack

#

I'm going to get MMU code in now

#

I fucking hate 32-bit address spaces

#

I will identity map 0x1000-0xc0000000

#

then any physical pages > 0xc0000000 will be mapped at a fixed address

#

this fixed address will be protected by a spinlock, and should only be accessed at IRQL_MASKED

#

you might be like "but but identity map bad angr "

#

I don't care.

short mortar Feb 6, 2025, 3:51 PM

#

are you not planning on having userspace?

flint idol Feb 6, 2025, 3:51 PM

#

KPTI

short mortar Feb 6, 2025, 3:51 PM

#

thonk

flint idol Feb 6, 2025, 3:52 PM

#

only minimal kernel pages will be mapped in user contexts

#

like I do on x86-64

short mortar Feb 6, 2025, 3:52 PM

#

how do you copy things from user? Onto stack?

flint idol Feb 6, 2025, 3:53 PM

#

https://github.com/OBOS-dev/obos/blob/master/src/oboskrnl/memmanip.c#L20-L73

GitHub

obos/src/oboskrnl/memmanip.c at master · OBOS-dev/obos

I was bored... Contribute to OBOS-dev/obos development by creating an account on GitHub.

#

trl

short mortar Feb 6, 2025, 3:55 PM

#

Isn't it kinda very slow?

flint idol Feb 6, 2025, 3:55 PM

#

for large IO, I use a separate function

#

// Maps user pages into the kernel address space. This can be used in syscalls to avoid copying large amounts of memory.
// returns OBOS_STATUS_PAGE_FAULT when a portion of the user memory requested is not mapped.
// NOTE: The returned address is not aligned down to the page size.
OBOS_EXPORT void* Mm_MapViewOfUserMemory(context* const user_context, void* ubase, void* kbase, size_t nBytes, prot_flags protection, bool respectUserProtection, obos_status* status);

sweet compass Feb 6, 2025, 5:32 PM

#

flint idol https://github.com/OBOS-dev/obos/blob/master/src/oboskrnl/memmanip.c#L20-L73

oh god what is this?

flint idol Feb 6, 2025, 5:32 PM

#

I refuse to elaborate.

sweet compass Feb 6, 2025, 5:35 PM

#

this is how I do it:

        .globl __memcpy_from_userspace
        .type __memcpy_from_userspace, @function
__memcpy_from_userspace:
        .cfi_startproc
        jmp usercopy_common
        .cfi_endproc
        .size __memcpy_from_userspace, . - __memcpy_from_userspace

        .globl __memcpy_to_userspace
        .type __memcpy_to_userspace, @function
__memcpy_to_userspace:
        .cfi_startproc
        jmp usercopy_common
        .cfi_endproc
        .size __memcpy_to_userspace, . - __memcpy_to_userspace

        .globl __usercopy_pagefault_address

        .type usercopy_common, @function
usercopy_common:
        .cfi_startproc
        xorq %rax, %rax
        movq %rdx, %rcx
__usercopy_pagefault_address:
        rep movsb
        ret
        .cfi_endproc
        .size usercopy_common, . - usercopy_common

        .globl __usercopy_fixup_address
        .type __usercopy_fixup_address, @function
__usercopy_fixup_address:
        .cfi_startproc
        movl $EFAULT, %eax
        ret
        .cfi_endproc
        .size __usercopy_fixup_address, . - __usercopy_fixup_address

        .globl __userstrcpy_pagefault_address

        .globl __strncpy_from_userspace
        .type __strncpy_from_userspace, @function
__strncpy_from_userspace:
        .cfi_startproc

1:
__userstrcpy_pagefault_address:
        lodsb (%rsi), %al
        stosb %al, (%rdi)

        testb %al, %al
        jz 1f

        decq %rdx
        jnz 1b

1:      xorq %rax, %rax
        ret

        .cfi_endproc
        .size __strncpy_from_userspace, . - __strncpy_from_userspace

haughty abyss Feb 6, 2025, 5:35 PM

#

holy shit what

flint idol Feb 6, 2025, 5:35 PM

#

that's cool and all

sweet compass Feb 6, 2025, 5:35 PM

#

basically it's just memcpy but with special handling for page faults

flint idol Feb 6, 2025, 5:35 PM

#

but I'm playing minecraft

haughty abyss Feb 6, 2025, 5:35 PM

#

this is probably like 3 lines of asm total

sweet compass Feb 6, 2025, 5:35 PM

#

haughty abyss this is probably like 3 lines of asm total

yes. chad

#

and 50 lines of GCC black magic

haughty abyss Feb 6, 2025, 5:36 PM

#

also

torpid wigeon Feb 6, 2025, 5:36 PM

#

sweet compass this is how I do it: ```x86asm .globl __memcpy_from_userspace .t...

wtf

haughty abyss Feb 6, 2025, 5:36 PM

#

imagine having strcpy from userspace

#

so cringe

torpid wigeon Feb 6, 2025, 5:36 PM

#

free cve spawner

haughty abyss Feb 6, 2025, 5:36 PM

#

theres literally no reason to have it

#

just get a size

#

and copy that amount of bytes

sweet compass Feb 6, 2025, 5:36 PM

#

haughty abyss theres literally no reason to have it

counter argument: filesystem syscalls

#

but yeah

#

I agree, somewhat

haughty abyss Feb 6, 2025, 5:36 PM

#

sweet compass counter argument: filesystem syscalls

you have userspace strlen the file path

#

easy

sweet compass Feb 6, 2025, 5:36 PM

#

explicitly sized buffers are better

haughty abyss Feb 6, 2025, 5:37 PM

#

all you need is copyin and copyout

#

and the only thing they differ in is the direction

#

and they can share an extable entry

#

or whatever you use as an equivalent

#

they can share a page fault recovery location thingy

sweet compass Feb 6, 2025, 5:38 PM

#

torpid wigeon wtf

oh... you haven't seen my pagefault handler

static pagefault_status_t
handle_pagefault_in_vm (unsigned long addr, unsigned int error_code)
{
        struct pagefault_info fault;
        fault.addr = addr;
        fault.access_write = (error_code & PF_WR) ? 1 : 0;
        fault.access_exec = (error_code & PF_ID) ? 1 : 0;

        irq_enable ();
        pagefault_status_t status = vm_handle_pagefault (&fault);
        irq_disable ();
        return status;
}

extern char __usercopy_pagefault_address[];
extern char __userstrcpy_pagefault_address[];
extern char __usercopy_fixup_address[];

static inline bool
is_usercopy_addr (void *rip)
{
        return rip == __usercopy_pagefault_address
                        || rip == __userstrcpy_pagefault_address;
}

static bool
handle_usercopy_fault (unsigned long addr, struct entry_regs *regs)
{
        if (usercopy_ok ((void *) addr, 1) != ESUCCESS)
                return false;
        if (regs->error_code & PF_ID)
                return false;

        if (handle_pagefault_in_vm (addr, regs->error_code) != FAULT_HANDLED)
                regs->rip = (unsigned long) __usercopy_fixup_address;
        return true;
}

void
handle_PF_exception_k (struct entry_regs *regs)
{
        unsigned long addr = read_cr2 ();
        note_pagefault (addr, regs->error_code);
        if (is_usercopy_addr ((void *) regs->rip)) {
                if (handle_usercopy_fault (addr, regs))
                        return;
        }
        panic ("x86: Page Fault in kernelspace!  (rip=0x%lx  ss:rsp=%lu:0x%lx)",
                regs->rip, regs->ss, regs->rsp);
}

flint idol Feb 6, 2025, 5:38 PM

#

that's the entire thing?

sweet compass Feb 6, 2025, 5:38 PM

#

no. but that's the part that dispatches faults to vm

#

and changes the return rip to the fixup addr

haughty abyss Feb 6, 2025, 5:39 PM

#

lol linux is like 10x more

sweet compass Feb 6, 2025, 5:39 PM

#

haughty abyss lol linux is like 10x more

I'll get there eventually.

flint idol Feb 6, 2025, 5:39 PM

#

main girder Feb 6, 2025, 5:40 PM

#

flint idol https://github.com/OBOS-dev/obos/blob/master/src/oboskrnl/memmanip.c#L20-L73

I've stopped commenting on obos bc I'm afraid of being responsible for causing you to quit

sweet compass Feb 6, 2025, 5:40 PM

#

flint idol Feb 6, 2025, 5:40 PM

#

lol (uh oh)

#

I'll fix it eventually

main girder Feb 6, 2025, 5:41 PM

#

main girder I've stopped commenting on obos bc I'm afraid of being responsible for causing y...

Even just in that function there are huge issues beyond just the fact its slow

#

It is broken in numerous ways

flint idol Feb 6, 2025, 5:42 PM

#

ye

#

also I

haughty abyss Feb 6, 2025, 5:42 PM

#

btw

flint idol Feb 6, 2025, 5:42 PM

#

am playing minecraft

#

so talk to you later ig

haughty abyss Feb 6, 2025, 5:42 PM

#

you have UB here https://github.com/OBOS-dev/obos/blob/master/src/drivers/generic/ahci/main.c#L237

#

in two different ways!

flint idol Feb 6, 2025, 5:43 PM

#

is that not guaranteed to be read-modify-write?

haughty abyss Feb 6, 2025, 5:43 PM

#

first, no

#

and gcc WILL hunt you down

#

also it will emit a single or instruction

#

which may or may not work

sweet compass Feb 6, 2025, 5:44 PM

#

OBOS kernel pwn as a CTF challenge when?

flint idol Feb 6, 2025, 5:44 PM

#

sweet compass OBOS kernel pwn as a CTF challenge when?

Not a challenge, even bash crashes it relatively quickly

haughty abyss Feb 6, 2025, 5:44 PM

#

also a valid way to lower that loop is c if (!(HBA->ghc & BIT(31))) { while (1) { HBA->ghc; OBOSS_SpinlockHint(); } }

sweet compass Feb 6, 2025, 5:44 PM

#

flint idol Not a challenge, even bash crashes it relatively quickly

that's why it's a challenge. we need to keep it alive for long enough to extract the flag

flint idol Feb 6, 2025, 5:45 PM

#

I had thought about rewriting this kernel a few times, but never said anything about it

weary hound Feb 6, 2025, 5:46 PM

#

sweet compass this is how I do it: ```x86asm .globl __memcpy_from_userspace .t...

no length limit on the strcpy from user?

sweet compass Feb 6, 2025, 5:46 PM

#

weary hound no length limit on the strcpy from user?

obviously lol.

weary hound Feb 6, 2025, 5:46 PM

#

iirc linux limits it to PATH_MAX for fs stuff

flint idol Feb 6, 2025, 5:46 PM

#

flint idol I had thought about rewriting this kernel a few times, but never said anything a...

Mainly because I did stupid stuff and don't know how to fix it

sweet compass Feb 6, 2025, 5:46 PM

#

weary hound iirc linux limits it to PATH_MAX for fs stuff

this is what I do:

static errno_t
get_path_from_userspace (char *kbuf, const char *user)
{
        kbuf[PATH_MAX - 1] = 0;
        errno_t e = strncpy_from_userspace (kbuf, user, PATH_MAX + 1);
        if (e == ESUCCESS && kbuf[PATH_MAX - 1])
                e = ENAMETOOLONG;
        return e;
}

#

and callers of this function:

SYSCALL6 (void, stat, struct stat *, buf, size_t, buf_size, const char *, path,
                int, dirfd, struct pathwalk_info *, info, size_t, info_size)
{
        char kpath[PATH_MAX];
        struct stat kstat;
        struct pathwalk_info kpathwalk_info;

        if (info_size > sizeof (kpathwalk_info))
                syscall_return_error (E2BIG);

        memset (&kpathwalk_info, 0, sizeof (kpathwalk_info));
        errno_t e = ESUCCESS;
        if (info_size)
                e = memcpy_from_userspace (&kpathwalk_info, info, info_size);
        if (e != ESUCCESS)
                syscall_return_error (e);

// ...

#

werks

haughty abyss Feb 6, 2025, 5:47 PM

#

haughty abyss also a valid way to lower that loop is ```c if (!(HBA->ghc & BIT(31))) { while (...

and also your code contains a data race

short mortar Feb 6, 2025, 5:47 PM

#

sweet compass basically it's just `memcpy` but with special handling for page faults

I do the same

weary hound Feb 6, 2025, 5:47 PM

#

sweet compass this is what I do: ```c static errno_t get_path_from_userspace (char *kbuf, cons...

ah yeah thats fine, i missed the dec %rdx : jnz 1b part

haughty abyss Feb 6, 2025, 5:47 PM

#

because of the same "volatile is stupid and bad" rpoblem

sweet compass Feb 6, 2025, 5:48 PM

#

haughty abyss you have UB here <https://github.com/OBOS-dev/obos/blob/master/src/drivers/gener...

no centralised bus?

#

(aka static struct pci_driver driver = { ... }; register_pci_driver (&driver);)

flint idol Feb 6, 2025, 5:49 PM

#

no(t yet)

short mortar Feb 6, 2025, 5:49 PM

#

short mortar I do the same

https://gitlab.com/mishakov/pmos/-/blob/dev/kernel/arch/x86_64/user_access.S?ref_type=heads

white mulch Feb 6, 2025, 5:50 PM

#

sweet compass obviously lol.

I don't have any limits for userspace buffers in my kernel troll you can literally cause an oom just by using some syscall like syslog that takes in a pointer + length and if you give a bogus large length and a nullptr the kernel is going to die (or more like the specific cpu core is going to die, the others might still function depending on how the locking works, idr if I keep the alloc lock in the fail case)

short mortar Feb 6, 2025, 5:50 PM

#

(except that it gets back to the same context, and handles pagefaults from there ultrameme )

flint idol Feb 6, 2025, 5:50 PM

#

haughty abyss and also your code contains a data race

where

haughty abyss Feb 6, 2025, 5:51 PM

#

flint idol where

that same volatile load is a data race

#

the loop one

flint idol Feb 6, 2025, 5:51 PM

#

bruh

#

now if only I remembered what bit 31 did

#

ah yes that makes sense now

sweet compass Feb 6, 2025, 5:52 PM

#

I can recommend watching (and rewatching) the many cppcon talks about atomics recorded and posted on youtube.

haughty abyss Feb 6, 2025, 5:52 PM

#

volatile is stupid and bad

#

or the idea is okay

short mortar Feb 6, 2025, 5:52 PM

#

short mortar (except that it gets back to the same context, and handles pagefaults from there...

https://gitlab.com/mishakov/pmos/-/blob/dev/kernel/arch/x86/user_access.cc?ref_type=heads

haughty abyss Feb 6, 2025, 5:52 PM

#

but the semantics are stupid and bad

flint idol Feb 6, 2025, 5:53 PM

#

short mortar https://gitlab.com/mishakov/pmos/-/blob/dev/kernel/arch/x86/user_access.cc?ref_t...

thanks for the text block

white mulch Feb 6, 2025, 5:53 PM

#

it should really be a property of the read/write and not the pointer

haughty abyss Feb 6, 2025, 5:53 PM

#

white mulch it should really be a property of the read/write and not the pointer

thats not even the beginning of the problems lol

#

the real fun problem is that a volatile store doesnt place any ordering constraints

#

on anything

#

so like. the compiler can just remove stores you make

sweet compass Feb 6, 2025, 5:54 PM

#

tbh _Atomic is also stupid and bad (albeit in less ways than volatile)

white mulch Feb 6, 2025, 5:54 PM

#

haughty abyss so like. the compiler can just remove stores you make

I'd assume it to not do that tho

#

because like that would very likely break linux

haughty abyss Feb 6, 2025, 5:54 PM

#

white mulch I'd assume it to not do that tho

it cant remove volatile ones

#

but yeah it can reorder nonvolatile stores past volatile ones

#

volatile stores dont impose ANY ordering constraints

sweet compass Feb 6, 2025, 5:55 PM

#

haughty abyss volatile stores dont impose ANY ordering constraints

(except against other volatile loads and stores afaik)

#

(but that doesn't help much)

haughty abyss Feb 6, 2025, 5:55 PM

#

sweet compass (except against other volatile loads and stores afaik)

hmm thats an interesting question

#

do they

#

yeah probably

white mulch Feb 6, 2025, 5:55 PM

#

hopefully

haughty abyss Feb 6, 2025, 5:55 PM

#

but the value a load loads is not an input to the AM

#

only that a load occured

#

btw the rust people have come up with a """"""""fix""""""""" for this problem

#

get ready

#

volatile atomics!

#

please dont think about it

white mulch Feb 6, 2025, 5:56 PM

#

atomics on hw regs 💀 (that's not even really allowed)

haughty abyss Feb 6, 2025, 5:57 PM

#

white mulch atomics on hw regs 💀 (that's not even really allowed)

yeah but consider this

#

sanity is optional in rust land

flint idol Feb 6, 2025, 5:58 PM

#

haughty abyss also a valid way to lower that loop is ```c if (!(HBA->ghc & BIT(31))) { while (...

what does this do exactly?

haughty abyss Feb 6, 2025, 5:58 PM

#

flint idol what does this do exactly?

wdym?

#

idk i think its for like some handoff stuff

white mulch Feb 6, 2025, 5:58 PM

#

it doesn't really matter, the pointer is volatile its going to be just fine

haughty abyss Feb 6, 2025, 5:59 PM

#

white mulch it doesn't really matter, the pointer is volatile its going to be just fine

it is technically speaking UB

white mulch Feb 6, 2025, 5:59 PM

#

you have to accept the ub

haughty abyss Feb 6, 2025, 5:59 PM

#

no you dont

#

you can read and write in asm

flint idol Feb 6, 2025, 5:59 PM

#

#pragma GCC optimize("-O0")```

haughty abyss Feb 6, 2025, 5:59 PM

#

lol

flint idol Feb 6, 2025, 5:59 PM

#

trl

sweet compass Feb 6, 2025, 5:59 PM

#

it'd be very bad if the compiler could transform something like:

*(volatile uint32_t *) (base + IOREGSEL) = 0x22;
*(volatile uint32_t *) (base + IOWIN) = 0x00000040;

into

*(volatile uint32_t *) (base + IOWIN) = 0x00000040;
*(volatile uint32_t *) (base + IOREGSEL) = 0x22;

suddenly you're writing bogus into a bogus register in the IOAPIC

haughty abyss Feb 6, 2025, 5:59 PM

#

which solves this issue

white mulch Feb 6, 2025, 5:59 PM

#

I mean yeah but like still, at some point you just accept the ub

haughty abyss Feb 6, 2025, 5:59 PM

#

nahhh you dont you can write UB-free os code

white mulch Feb 6, 2025, 6:00 PM

#

its not really necessary tho

haughty abyss Feb 6, 2025, 6:00 PM

#

lets maybe start with not including data races :^)

sweet compass Feb 6, 2025, 6:00 PM

#

there's no UB in asm galaxybrain

haughty abyss Feb 6, 2025, 6:00 PM

#

sweet compass there's no UB in asm <:galaxybrain:701707619651420251>

exactly

haughty abyss Feb 6, 2025, 6:00 PM

#

sweet compass it'd be *very* bad if the compiler could transform something like: ```c *(volati...

on one hand yes, but on the other hand consider this:

#

compiler go brrr

timid elk Feb 6, 2025, 6:00 PM

#

sweet compass there's no UB in asm <:galaxybrain:701707619651420251>

i will make an assembler that outputs LLVM IR and optimizes the assembly

haughty abyss Feb 6, 2025, 6:01 PM

#

please dont think about how this sort of reordering is insanely cringe and stupid

#

and also these semantics make no sense

flint idol Feb 6, 2025, 6:01 PM

#

so having macros like:

#define MMIO_WRITE1(addr, x)
#define MMIO_READ1(addr)```

timid elk Feb 6, 2025, 6:01 PM

#

timid elk i will make an assembler that outputs LLVM IR and optimizes the assembly

time for UB in your asm

haughty abyss Feb 6, 2025, 6:01 PM

#

flint idol so having macros like: ```c #define MMIO_WRITE1(addr, x) #define MMIO_READ1(addr...

yeah pretty much

sweet compass Feb 6, 2025, 6:01 PM

#

timid elk i will make an assembler that outputs LLVM IR and optimizes the assembly

it would be interesting to see this.

flint idol Feb 6, 2025, 6:01 PM

#

iirc linux does something like that

timid elk Feb 6, 2025, 6:01 PM

#

sweet compass it would be interesting to see this.

langdev project!!

haughty abyss Feb 6, 2025, 6:01 PM

#

i have ```c
#if defined (aarch64)

#define arm64
#define arch_uses_fdt
#define arch_little_endian
#define arch_asm_read8 "ldrb %w[value], [%[address]]"
#define arch_asm_read16 "ldrh %w[value], [%[address]]"
#define arch_asm_read32 "ldr %w[value], [%[address]]"
#define arch_asm_read64 "ldr %[value], [%[address]]"

#define arch_asm_write8 "strb %w[value], [%[address]]"
#define arch_asm_write16 "strh %w[value], [%[address]]"
#define arch_asm_write32 "str %w[value], [%[address]]"
#define arch_asm_write64 "str %[value], [%[address]]"

#elif defined (riscv64)

#define rv64
#define arch_uses_fdt
#define arch_little_endian
#error TODO: riscv arch_asm_*

#else
#error unknown platform!
#endif

#define rmw_func(size, typ)
static inline typ read##size(usize address) {
typ value;
asm volatile(arch_asm_read##size : [value] "=r" (value) : [address] "r"(address) : "memory");
return value;
}
static inline void write##size(usize address, typ value) {
asm volatile(arch_asm_write##size :: [value] "r" (value), [address] "r"(address) : "memory");
}
static inline void set##size(usize address, typ value) {
write##size(address, read##size(address) | value);
}
static inline void clear##size(usize address, typ value) {
write##size(address, read##size(address) & ~value);
}
static inline void mask##size(usize address, typ clear, typ set) {
write##size(address, (read##size(address) & ~clear) | set);
}
static inline typ writeread##size(usize address, typ value) {
write##size(address, value);
return read##size(address);
} \

rmw_func(8, u8)
rmw_func(16, u16)
rmw_func(32, u32)
rmw_func(64, u64)

real pecan Feb 6, 2025, 6:01 PM

#

main girder It is broken in numerous ways

how

haughty abyss Feb 6, 2025, 6:01 PM

#

but thats the same thing basically

timid elk Feb 6, 2025, 6:01 PM

#

also allows you to translate between archs

#

m68k asm compiled to riscv

real pecan Feb 6, 2025, 6:02 PM

#

haughty abyss i have ```c #if defined (__aarch64__) #define arm64 #define arch_uses_fdt #defi...

the type is redundant

white mulch Feb 6, 2025, 6:02 PM

#

haughty abyss i have ```c #if defined (__aarch64__) #define arm64 #define arch_uses_fdt #defi...

I have this meme

📎 register.hpp

haughty abyss Feb 6, 2025, 6:02 PM

#

real pecan the type is redundant

which type?

real pecan Feb 6, 2025, 6:02 PM

#

can be derived from just u##size

haughty abyss Feb 6, 2025, 6:02 PM

#

oh yes

#

that is true

#

doesnt matter tho

sweet compass Feb 6, 2025, 6:03 PM

#

imo the only sane way to deal with volatile/atomics is to have

a library of things used commonly, eg. nmi-safe atomic list, spinlock, mutex, rwmutex, semaphore, atomic refcount
some helper functions for hardware register bitbashing
and then only use those.

main girder Feb 6, 2025, 6:03 PM

#

real pecan how

Toctou on the page access checks, and the accesses to the cpu local data are racy

timid elk Feb 6, 2025, 6:03 PM

#

haughty abyss i have ```c #if defined (__aarch64__) #define arm64 #define arch_uses_fdt #defi...

I don't like arch_uses_fdt

haughty abyss Feb 6, 2025, 6:03 PM

#

sweet compass imo the only sane way to deal with volatile/atomics is to have - a library of th...

the only sane way to deal with volatile is to not

sweet compass Feb 6, 2025, 6:03 PM

#

true

haughty abyss Feb 6, 2025, 6:03 PM

#

timid elk I don't like arch_uses_fdt

i use it to compile out FDT logic when not needed, also its a shit ass prekernel its allowed to be bad

timid elk Feb 6, 2025, 6:03 PM

#

timid elk I don't like arch_uses_fdt

what if I boot it with ACPI on riscv or aarch64

haughty abyss Feb 6, 2025, 6:03 PM

#

timid elk what if I boot it with ACPI on riscv or aarch64

doesnt matter, its not a "requires FDT", its a "supports FDT"

timid elk Feb 6, 2025, 6:04 PM

#

ah

#

it's ok then

#

tbh that shouldn't be an arch thing, I could boot x86 with FDT if I really wanted to

#

it's more like, default to compiling FDT support for this arch right?

haughty abyss Feb 6, 2025, 6:05 PM

#

timid elk it's more like, default to compiling FDT support for this arch right?

yeah its more like that

#

no

#

its more like

#

"compile in fdt shit"

#

right now i have one platform (arm64 with linux boot) and that has FDT always anyway

#

ill fix it later tho

#

maybe

#

or maybe not

timid elk Feb 6, 2025, 6:06 PM

#

opinion: keep build configuration out of headers, even a Kconfig like linux or uboot would be fine

timid elk Feb 6, 2025, 6:06 PM

#

haughty abyss right now i have one platform (arm64 with linux boot) and that has FDT always an...

I think arm64 with ACPI is a thing in linux boot

haughty abyss Feb 6, 2025, 6:07 PM

#

timid elk I think arm64 with ACPI is a thing in linux boot

still includes an FDT though

timid elk Feb 6, 2025, 6:07 PM

#

at least the raspberry pi 5 edk2 port mentions it

flint idol Feb 6, 2025, 6:07 PM

#

I think I will pause porting for a bit

haughty abyss Feb 6, 2025, 6:07 PM

#

timid elk opinion: keep build configuration out of headers, even a Kconfig like linux or u...

its a shitty prekernel

timid elk Feb 6, 2025, 6:07 PM

#

haughty abyss still includes an FDT though

oh?

haughty abyss Feb 6, 2025, 6:07 PM

#

yeah

#

you cant not

timid elk Feb 6, 2025, 6:07 PM

#

cool

#

is it like, almost empty

haughty abyss Feb 6, 2025, 6:07 PM

#

yeah

flint idol Feb 6, 2025, 6:07 PM

#

to fix this MMIO read/write "problem" thing

timid elk Feb 6, 2025, 6:07 PM

#

or is it anything useful

white mulch Feb 6, 2025, 6:07 PM

#

white mulch I have this <:meme:575445094589661215>

I actually like this (credit for idea goes to managarm), you have register/bitfield types ```cpp
namespace regs {
constexpr BitRegister<u64> GCIDR {0};
constexpr BitRegister<u64> GCR {0x10};
constexpr BasicRegister<u64> MCVR {0xF0};
}

namespace gcidr {
// bitfield of u32 type in bits 32..64 on an u64
constexpr BitField<u64, u32> COUNTER_CLK_PERIOD {32, 32};
}

void hpet_init() {
auto fs_in_tick = SPACE.load(regs::GCIDR) & gcidr::COUNTER_CLK_PERIOD;
auto ns_in_tick = fs_in_tick / 1000000;
auto ticks_in_s = NS_IN_US * US_IN_S / ns_in_tick;
}

timid elk Feb 6, 2025, 6:07 PM

#

does it like

haughty abyss Feb 6, 2025, 6:07 PM

#

flint idol to fix this MMIO read/write "problem" thing

you can copy my meme

timid elk Feb 6, 2025, 6:07 PM

#

timid elk does it like

"here's the RSDP"

#

that's how I would have done it

haughty abyss Feb 6, 2025, 6:08 PM

#

pretty much

#

yeah

timid elk Feb 6, 2025, 6:08 PM

#

pass RSDP in FDT

#

ah

haughty abyss Feb 6, 2025, 6:08 PM

#

white mulch I actually like this (credit for idea goes to managarm), you have register/bitfi...

the bitfield thing is kinda silly tbh

white mulch Feb 6, 2025, 6:08 PM

#

no its not

#

its nice

haughty abyss Feb 6, 2025, 6:08 PM

#

did you copy it from managarm?

white mulch Feb 6, 2025, 6:08 PM

#

then you don't have to think about shifts/masks

#

yes

haughty abyss Feb 6, 2025, 6:09 PM

#

i see i see

white mulch Feb 6, 2025, 6:09 PM

#

or well not copy but the idea is from there

haughty abyss Feb 6, 2025, 6:09 PM

#

clearly copied :^)

white mulch Feb 6, 2025, 6:09 PM

#

trl

#

I also included it in uhda

haughty abyss Feb 6, 2025, 6:10 PM

#

uhda is C++?

white mulch Feb 6, 2025, 6:10 PM

#

yes

haughty abyss Feb 6, 2025, 6:10 PM

#

hmm actually this is an interesting question

#

can i complain to infy about UB

#

ah no

#

i cant

#

its implementer ub

white mulch Feb 6, 2025, 6:11 PM

#

actually the uhda impl does volatile loads/stores with an added launder

#

not the arch specific asm that I have in that other copy of it

haughty abyss Feb 6, 2025, 6:12 PM

#

white mulch actually the uhda impl does volatile loads/stores with an added launder

oh cool so its UB!

white mulch Feb 6, 2025, 6:15 PM

#

as long as it works 🤷

#

though ig I might consider adding the asm impl but that would be the only arch-specific part in there

flint idol Feb 6, 2025, 6:18 PM

#

#if defined(__x86_64__) || defined(__i686__)
#   define MMIO_WRITE1(addr, x) do { asm volatile("movb %1, (%0);" : :"r"(addr), "r"(x) :"memory"); } while(0)
#   define MMIO_WRITE2(addr, x) do { asm volatile("movw %1, (%0);" : :"r"(addr), "r"(x) :"memory"); } while(0)
#   define MMIO_WRITE4(addr, x) do { asm volatile("movd %1, (%0);" : :"r"(addr), "r"(x) :"memory"); } while(0)
#   define MMIO_READ1(addr) ({ uint8_t x = 0; asm volatile("movb (%0), %1;" :"=r"(x) :"r"(addr) :"memory"); (x); })
#   define MMIO_READ2(addr) ({ uint16_t x = 0; asm volatile("movw (%0), %1;" :"=r"(x) :"r"(addr) :"memory"); (x); })
#   define MMIO_READ4(addr) ({ uint32_t x = 0; asm volatile("movd (%0), %1;" :"=r"(x) :"r"(addr) :"memory"); (x); })
#elif defined(__m68k__)
#   define MMIO_WRITE1(addr, x) do { asm volatile("move.b %1, (%0);" : :"r"(addr), "r"(x) :"memory"); } while(0)
#   define MMIO_WRITE2(addr, x) do { asm volatile("move.w %1, (%0);" : :"r"(addr), "r"(x) :"memory"); } while(0)
#   define MMIO_WRITE4(addr, x) do { asm volatile("move.d %1, (%0);" : :"r"(addr), "r"(x) :"memory"); } while(0)
#   define MMIO_READ1(addr) ({ uint8_t x = 0; asm volatile("move.b (%0), %1;" :"=r"(x) :"r"(addr) :"memory"); (x); })
#   define MMIO_READ2(addr) ({ uint16_t x = 0; asm volatile("move.w (%0), %1;" :"=r"(x) :"r"(addr) :"memory"); (x); })
#   define MMIO_READ4(addr) ({ uint32_t x = 0; asm volatile("move.d (%0), %1;" :"=r"(x) :"r"(addr) :"memory"); (x); })
#else
#   error Define for your architecture!
#endif```

#

ok just did this

#

time to use them

#

for MMIO

#

ig

#

before that, I will add:

// thanks a lot for this, pitust.```

haughty abyss Feb 6, 2025, 6:21 PM

#

lmao

#

you're welcome

real pecan Feb 6, 2025, 6:42 PM

#

haughty abyss can i complain to infy about UB

wait what

haughty abyss Feb 6, 2025, 6:43 PM

#

nah i was wondering if uacpi has UB

#

but no

real pecan Feb 6, 2025, 6:43 PM

#

somethign specific?

haughty abyss Feb 6, 2025, 6:43 PM

#

yes

#

volatile is basically always UB

real pecan Feb 6, 2025, 6:43 PM

#

i use volatile in like one place, just to prevent reorders iirc

haughty abyss Feb 6, 2025, 6:44 PM

#

not sure if thats enough

#

you should use atomics

real pecan Feb 6, 2025, 6:44 PM

#

probably

haughty abyss Feb 6, 2025, 6:44 PM

#

definitely

real pecan Feb 6, 2025, 6:44 PM

#

its followed by an atomic cmpxchg on the same memory location

haughty abyss Feb 6, 2025, 6:44 PM

#

is it to prevent cross-thread reorders?

real pecan Feb 6, 2025, 6:44 PM

#

so its enough

real pecan Feb 6, 2025, 6:44 PM

#

haughty abyss is it to prevent cross-thread reorders?

nah

haughty abyss Feb 6, 2025, 6:44 PM

#

so what is it for then?

#

for peripherals?

real pecan Feb 6, 2025, 6:44 PM

#

its the firmware global lock

haughty abyss Feb 6, 2025, 6:45 PM

#

ah

#

yeah no thats UB

real pecan Feb 6, 2025, 6:45 PM

#

i load it and then cmpxchg

haughty abyss Feb 6, 2025, 6:45 PM

#

its a data race

real pecan Feb 6, 2025, 6:45 PM

#

cmpxchg is atomic tho

haughty abyss Feb 6, 2025, 6:45 PM

#

but the volatile load is not

#

data race, straight to UB land

real pecan Feb 6, 2025, 6:45 PM

#

bruh

haughty abyss Feb 6, 2025, 6:45 PM

#

uacpi_status uacpi_system_memory_read(void *ptr, uacpi_u8 width, uacpi_u64 *out)
{
    switch (width) {
    case 1:
        *out = *(volatile uacpi_u8*)ptr;
        break;
    case 2:
        *out = *(volatile uacpi_u16*)ptr;
        break;
    case 4:
        *out = *(volatile uacpi_u32*)ptr;
        break;
    case 8:
        *out = *(volatile uacpi_u64*)ptr;
        break;
    default:
        return UACPI_STATUS_INVALID_ARGUMENT;
    }

    return UACPI_STATUS_OK;
}
``` this is UB too

#

or at least

real pecan Feb 6, 2025, 6:45 PM

#

on paper yes probably

flint idol Feb 6, 2025, 6:45 PM

#

real pecan bruh

yeah that's what I said too

trim bay Feb 6, 2025, 6:45 PM

#

using a relaxed atomic load would compile to the same thing and not be ub

haughty abyss Feb 6, 2025, 6:45 PM

#

actually no

#

infy's usage is real UB

real pecan Feb 6, 2025, 6:46 PM

#

what

haughty abyss Feb 6, 2025, 6:46 PM

#

the obos one is stretching semantics

#

this is straight UB that can blow your arm off

uncut narwhal Feb 6, 2025, 6:46 PM

#

haughty abyss ```c uacpi_status uacpi_system_memory_read(void *ptr, uacpi_u8 width, uacpi_u64 ...

doesnt that function not even exist anymore

haughty abyss Feb 6, 2025, 6:46 PM

#

idk

real pecan Feb 6, 2025, 6:46 PM

#

the compiler doesnt know anything

haughty abyss Feb 6, 2025, 6:46 PM

#

probably not

uncut narwhal Feb 6, 2025, 6:46 PM

#

oh wait thats the internal one my bad

#

im so used to seeing that pattern in the old kernel raw one lol

real pecan Feb 6, 2025, 6:46 PM

#

the write is behind a mutex from

#

a different place

flint idol Feb 6, 2025, 6:47 PM

#

wait infy removed uacpi_kernel_raw_memory_*

#

???

sweet compass Feb 6, 2025, 6:47 PM

#

haughty abyss ```c uacpi_status uacpi_system_memory_read(void *ptr, uacpi_u8 width, uacpi_u64 ...

wouldn't that depend on what ptr points to?

real pecan Feb 6, 2025, 6:47 PM

#

like a month ago

haughty abyss Feb 6, 2025, 6:47 PM

#

sweet compass wouldn't that depend on what `ptr` points to?

yeah true tbh

uncut narwhal Feb 6, 2025, 6:47 PM

#

yeah that was a while ago lmao

haughty abyss Feb 6, 2025, 6:47 PM

#

but its probably exposed to aml

#

and so ub

flint idol Feb 6, 2025, 6:47 PM

#

bruh

haughty abyss Feb 6, 2025, 6:47 PM

#

:^)

#

though admittedly

#

this wont break

real pecan Feb 6, 2025, 6:47 PM

#

it points to stuff returned via kernel_memory_map

sweet compass Feb 6, 2025, 6:47 PM

#

it's only as UB as the firmware devs are braindead

uncut narwhal Feb 6, 2025, 6:47 PM

#

it just always does map -> normal vmem access -> unmap now

haughty abyss Feb 6, 2025, 6:47 PM

#

sweet compass it's only as UB as the firmware devs are braindead

which is to say very

sweet compass Feb 6, 2025, 6:47 PM

#

:p

real pecan Feb 6, 2025, 6:48 PM

#

use SystemIO exclusively to be safe meme

uncut narwhal Feb 6, 2025, 6:48 PM

#

lmao

haughty abyss Feb 6, 2025, 6:48 PM

#

real pecan the compiler doesnt know anything

(a) doesnt matter, it can and i suspect sometimes will produce bad code (b) you need atomics anyway lol

#

a relaxed atomic load is fine here

real pecan Feb 6, 2025, 6:48 PM

#

id need a relaxed atomic load

haughty abyss Feb 6, 2025, 6:48 PM

#

yes

#

which is free

real pecan Feb 6, 2025, 6:48 PM

#

which on x86 is the same

haughty abyss Feb 6, 2025, 6:49 PM

#

and on arm

#

and riscv

#

lol

real pecan Feb 6, 2025, 6:49 PM

#

arm and risc-v are hw reduced acpi anyway

#

they dont have a global lock

haughty abyss Feb 6, 2025, 6:49 PM

#

ah yeah

#

please dont use volatile

#

especially here

real pecan Feb 6, 2025, 6:49 PM

#

here being?

haughty abyss Feb 6, 2025, 6:50 PM

#

the global lock one

real pecan Feb 6, 2025, 6:50 PM

#

ah

#

yeah

haughty abyss Feb 6, 2025, 6:51 PM

#

those loads and stores are UB too though

trim bay Feb 6, 2025, 6:51 PM

#

haughty abyss also a valid way to lower that loop is ```c if (!(HBA->ghc & BIT(31))) { while (...

I don't think this is true?

An object that has volatile-qualified type may be modified in ways unknown to the implementation or have other unknown side effects. Therefore, any expression referring to such an object shall be evaluated strictly according to the rules of the abstract machine, as described in 5.1.2.4
I interpret this as saying that the compiler cannot assume that the value hasn't changed between the two condition checks

real pecan Feb 6, 2025, 6:52 PM

#

haughty abyss those loads and stores are UB too though

not really

haughty abyss Feb 6, 2025, 6:52 PM

#

trim bay I don't think this is true? > An object that has volatile-qualified type may be ...

oh hmm

#

hmmmmmmmmmm

#

interestingggg

real pecan Feb 6, 2025, 6:52 PM

#

they refer to stuff mapped via kernel_map, which, as long as AML is correct stores this exact type it puns into

trim bay Feb 6, 2025, 6:54 PM

#

it's more so that the compiler is allowed to use any instruction it wants to effect the reads/writes, as well as being allowed to split said reads and writes

#

that's problematic because some instructions are not allowed to be used for MMIO

#

for example, a while ago I got bitten by clang trying to use some bit manipulation extension for setting the ioapic registers

real pecan Feb 6, 2025, 7:03 PM

#

lol

#

what im gonna do for mmio is have optional read_memory/write_memory kernel hooks that operate on virtual addresses

#

to combat that

trim bay Feb 6, 2025, 7:04 PM

#

sounds like a good idea

haughty abyss Feb 6, 2025, 7:32 PM

#

trim bay for example, a while ago I got bitten by clang trying to use some bit manipulati...

wait really?

#

can you reproduce?

#

or was it a normal read/write and not a volatile one?

trim bay Feb 6, 2025, 7:33 PM

#

this was like >1 month ago, i don't remember the commit it was on

haughty abyss Feb 6, 2025, 7:33 PM

#

ah

trim bay Feb 6, 2025, 7:33 PM

#

it was def volatile though

haughty abyss Feb 6, 2025, 7:33 PM

#

damn i would love to see how to do that lol

#

i thought only GCC engaged in this flavor of brain damage

trim bay Feb 6, 2025, 7:33 PM

#

i was using -march=native -mtune=native

#

didn't happen when targeting x86-64-v1

haughty abyss Feb 6, 2025, 7:33 PM

#

yeah doesnt matter tho

#

whats your -march?

trim bay Feb 6, 2025, 7:34 PM

#

znver3

haughty abyss Feb 6, 2025, 7:34 PM

#

ah

#

oh yeah!

#

damn

#

ah

#

clang enjoys this flavor of brain damage too TIL

#

ah maybe its GCC that didnt? idk

#

either way good to know

trim bay Feb 6, 2025, 7:36 PM

#

managed to repro it on the last commit before i switched to inline asm mmio

#

doesn't happen with gcc

#

the offending instruction is bextrl %eax, 0x10(%r12, %rbx), %edx

haughty abyss Feb 6, 2025, 7:38 PM

#

ah yeah

#

i can get it to produce a normal or lmao

#

basically the TLDR is

#

you cannot, ever, use volatile for MMIO

#

because it straight up produces broken code

trim bay Feb 6, 2025, 7:38 PM

#

yep

trim bay Feb 6, 2025, 7:39 PM

#

trim bay the offending instruction is `bextrl %eax, 0x10(%r12, %rbx), %edx`

r12 here maps to 0xfec00000

#

and rbx is 0

haughty abyss Feb 6, 2025, 7:39 PM

#

thats only broken on qemu right?

trim bay Feb 6, 2025, 7:40 PM

#

trim bay yep

unless the "mmio" is really just a different kind of ram, e.g. when writing to a framebuffer

trim bay Feb 6, 2025, 7:40 PM

#

haughty abyss thats only broken on qemu right?

probably, i'm not gonna bother testing on real hw

#

but i would assume that on actual cpus the instruction doing the mmio doesn't matter

#

then again on amd cpus you're only allowed to interface with ecam if you use mov to/from eax so what do i know

haughty abyss Feb 6, 2025, 7:41 PM

#

WTF

#

cursed

trim bay Feb 6, 2025, 7:41 PM

#

it's not actually enforced, doing it with some other register works fine on modern cpus

haughty abyss Feb 6, 2025, 7:41 PM

#

ahhh

#

okay

trim bay Feb 6, 2025, 7:42 PM

#

but it's in the bios and kernel developer's guide

haughty abyss Feb 6, 2025, 7:42 PM

#

lmfao

trim bay Feb 6, 2025, 7:42 PM

#

so it presumably was necessary on some previous generation

short mortar Feb 6, 2025, 8:18 PM

#

haughty abyss you cannot, ever, use volatile for MMIO

what's even the point of volatile then?

flint idol Feb 6, 2025, 8:18 PM

#

To confuse

#

people who are new to system dev

haughty abyss Feb 6, 2025, 8:19 PM

#

pretty much tbh

real pecan Feb 6, 2025, 8:19 PM

#

preventing compiler from assuming the value, preventing reordering

haughty abyss Feb 6, 2025, 8:19 PM

#

real pecan preventing compiler from assuming the value, preventing reordering

the former, yeah

#

the latter, no

#

you arent preventing reordering anyway

real pecan Feb 6, 2025, 8:20 PM

#

reordering with other code, yes

#

at least at code gen level

haughty abyss Feb 6, 2025, 8:20 PM

#

real pecan reordering with other code, yes

with other volatile writes

#

which you cant observe anyway

real pecan Feb 6, 2025, 8:20 PM

#

yes

haughty abyss Feb 6, 2025, 8:20 PM

#

and also it doesnt prevent it i think

real pecan Feb 6, 2025, 8:21 PM

#

haughty abyss which you cant observe anyway

hm?

haughty abyss Feb 6, 2025, 8:21 PM

#

idk maybe it does?

real pecan Feb 6, 2025, 8:21 PM

#

it does

#

linux abuses it for READ_ONCE/WRITE_ONCE

haughty abyss Feb 6, 2025, 8:21 PM

#

real pecan linux abuses it for READ_ONCE/WRITE_ONCE

yeah but linux relies on intense ub

real pecan Feb 6, 2025, 8:22 PM

#

/*
 * Prevent the compiler from merging or refetching reads or writes. The
 * compiler is also forbidden from reordering successive instances of
 * READ_ONCE and WRITE_ONCE, but only when the compiler is aware of some
 * particular ordering. One way to make the compiler aware of ordering is to
 * put the two invocations of READ_ONCE or WRITE_ONCE in different C
 * statements.
 *
 * These two macros will also work on aggregate data types like structs or
 * unions.
 *
 * Their two major use cases are: (1) Mediating communication between
 * process-level code and irq/NMI handlers, all running on the same CPU,
 * and (2) Ensuring that the compiler does not fold, spindle, or otherwise
 * mutilate accesses that either do not require ordering or that interact
 * with an explicit memory barrier or atomic instruction that provides the
 * required ordering.
 */

haughty abyss Feb 6, 2025, 8:22 PM

#

real pecan ``` /* * Prevent the compiler from merging or refetching reads or writes. The ...

Yes, this permits 64-bit accesses on 32-bit architectures. These will

actually be atomic in some cases (namely Armv7 + LPAE), but for others we

rely on the access being split into 2x32-bit accesses for a 32-bit quantity

(e.g. a virtual address) and a strong prevailing wind.

#

the "strong prevailing wind" comment does NOT inspire confidence

#

i also would call it blatant UB

real pecan Feb 6, 2025, 8:22 PM

#

how is it ub lol

haughty abyss Feb 6, 2025, 8:23 PM

#

it isnt preventing refetching

#

afaik

#

it can also produce torn writes without causing UB

#

actually it can produce a lot of things

real pecan Feb 6, 2025, 8:24 PM

#

yeah idk

haughty abyss Feb 6, 2025, 8:24 PM

#

its blatant UB reliance

real pecan Feb 6, 2025, 8:24 PM

#

language lawyering stuff can be counter productive

haughty abyss Feb 6, 2025, 8:24 PM

#

true

#

in this case the correct thing is an atomic

#

or an assembly load

#

but anyway the point is you cant use it to stop reordering cross-cpu

#

because that requires actual barriers

real pecan Feb 6, 2025, 8:25 PM

#

i mean yeah

haughty abyss Feb 6, 2025, 8:25 PM

#

and it doesnt stop reordering vs literally any other memory op you make

real pecan Feb 6, 2025, 8:25 PM

#

thats not its purpose tho

haughty abyss Feb 6, 2025, 8:25 PM

#

to do THAT you need a c11 signal fence

#

like yeah i know its stupid and rather lawyery but unfortunately that is the spec

#

and in practice the compiler can do many very bad things with volatile that preclude it being used for mmio anyway

flint idol Feb 6, 2025, 11:36 PM

#

was investigating a crash in obos-strap
only for it to be in cJSON

#

at first I thought I passed something invalid to the function

#

like nullptr or smth

#

No.

devout niche Feb 6, 2025, 11:36 PM

#

i like those managarm bit field things

#

it shows off C++'s merit as a systems language

flint idol Feb 7, 2025, 5:08 PM

#

Ngl I feel like making a new obos rewrite along this one

#

Like I work on both

#

Like some sort of OBOS2

zinc root Feb 7, 2025, 5:09 PM

#

why not just make this current version OBOS 1.x.x and make the rewrite OBOS 2.x.x

main girder Feb 7, 2025, 5:09 PM

#

i copyrighted the number 2

#

unfortunately

flint idol Feb 7, 2025, 5:09 PM

#

main girder i copyrighted the number 2

Damn it

#

What will I ever do now

zinc root Feb 7, 2025, 5:10 PM

#

zinc root why not just make this current version OBOS 1.x.x and make the rewrite OBOS 2.x....

then you can also update OBOS 1 with new version numbers without it getting confused with 2

flint idol Feb 7, 2025, 5:10 PM

#

Because I would want to make a separate repo for the rewrite

#

This rewrite got a bit far in userspace, but the kernel is kind of not amazing

#

The memory manager is my biggest concern really

#

Also when I designed the driver interface, for example, I designed it not amazingly

#

Not to mention all IO is done synchronously

devout niche Feb 7, 2025, 5:13 PM

#

flint idol Ngl I feel like making a new obos rewrite along this one

there can be a time for rewrites and a time not for

flint idol Feb 7, 2025, 5:13 PM

#

flint idol Also when I designed the driver interface, for example, I designed it not amazin...

Which seeped into the VFS

devout niche Feb 7, 2025, 5:14 PM

#

i started a sort of keyronex rewrite but now i'm folding in as much of what i worked on there back into keyronex

real pecan Feb 7, 2025, 5:14 PM

#

flint idol Ngl I feel like making a new obos rewrite along this one

whats wrong with the current obos

flint idol Feb 7, 2025, 5:14 PM

#

Also I kinda hate the naming convention lol

devout niche Feb 7, 2025, 5:14 PM

#

as while current keyronex is messy in many areas and there are new ways of doing things i want to introduce, there is no call for a blanket throwing out of everything

flint idol Feb 7, 2025, 5:14 PM

#

real pecan whats wrong with the current obos

idk I'm just thinking about it

#

It might not happen

real pecan Feb 7, 2025, 5:15 PM

#

what would u change

flint idol Feb 7, 2025, 5:15 PM

#

Driver interface is a big one

#

And VFS

#

The VMM isn't amazing rn

lean glen Feb 7, 2025, 5:17 PM

#

do like gaia2

#

start a rewrite

#

and don't work on it

#

meme

flint idol Feb 7, 2025, 5:17 PM

#

lol

lean glen Feb 7, 2025, 5:17 PM

#

doesnt help that i started the project just when classes started

flint idol Feb 7, 2025, 5:18 PM

#

And when did you start it lol

lean glen Feb 7, 2025, 5:18 PM

#

like a month ago at this point lol

#

and it's still basically a limine hello world + kernel debugger

flint idol Feb 7, 2025, 5:18 PM

#

lean glen like a month ago at this point lol

GDT init.. unimplemented

lean glen Feb 7, 2025, 5:18 PM

#

at least I'm doing somewhat productive stuff in my free time so

#OBOS (not vibecoded)