Jeff BezOS

[nova fiber]

my scheduler operates on the former struct because i dont want my scheduler to depend on my system objects

[hollow gorge]

so what are your tasks/threads?

oh wait nvm lol

hmmmmm

why not just copy the stuff in the scheduler entry into the task

rather than an intrusive block just store it in Thread

and have the scheduler only worry about Threads

no more scheduler entry

[nova fiber]

My old scheduler did that and it was really painful over the long run

anytime i wanted to change how the thread system object worked i had to also touch the scheduler alot

and testing the scheduler was basically impossible because it was so tied up with the entire system

[hollow gorge]

how come things like the TaskState are not things you care about from the Thread perspective

isn't that stuff like IDLE, BLOCKED, RUNNING, etc.

[nova fiber]

TaskState is only the machine cpu state

[hollow gorge]

ohh

[nova fiber]

so the registers and fpu state, nothing else

[hollow gorge]

why is this not stored in the thread

struct thread {
    uint64_t id;
    void (*entry)(void);
    void *stack;

    struct context regs;
    struct thread *next;
    struct thread *prev;

    _Atomic enum thread_state state;
    enum thread_priority perceived_prio; /* priority level right now */
    enum thread_priority base_prio;      /* priority level at creation time */
    enum thread_flags flags;
    volatile enum wake_reason wake_reason;

    int64_t curr_core;      /* -1 if not being ran */
    uint64_t time_in_level; /* ticks at this level */

    struct worker_thread *worker; /* NULL if this is not a worker */
};```

```c
struct context {
    uint64_t rbx;
    uint64_t rbp;
    uint64_t r12;
    uint64_t r13;
    uint64_t r14;
    uint64_t r15;
    uint64_t rsp;
    uint64_t rip;
};```

like that

[nova fiber]

from an architectural perspective my scheduler shouldn't need knowledge of the rest of the operating systems context

and practically its because im using the scheduler in multiple places and each place has different requirements for its extra data

I think im probably going to cave and end up either templating the scheduler entry or adding a void*

[hollow gorge]

seems like an interesting approach then

have fun

[nova fiber]

[prisma stirrup]

Yeah better add a little bit of dependency then to butcher the codebase with extra complexity imo

[nova fiber]

I forgot I could just use inheritance and extend the scheduler entry struct for each place

Duh I suppose

[hollow gorge]

typical c++ win

mfw oop is no longer useless

[trim owl]

this looks so weird without any context

[zealous mirage]

precedent for exactly what @nova fiber is doing

[trim owl]

i mean that image specifically

it's formatted in such a weird way

[zealous mirage]

they have a few fun diagrams in the mk++ documents

[nova fiber]

what a diagram

i didnt know the open group did anything beyond the posix and DCE spec tbh

[nova fiber]

inheritance seems to be working

still a fair bit of work to do but looks promising so far

new scheduler is definetly much more stable

[hollow gorge]

good ending 😃

awesome to hear that

have an absolute blast

[zealous mirage]

they used to do all kinds of cool things until they turned into document wrangers

this is from MK++

they applied this rigorous system of layering and inheritance to everything

e.g. this (dotted lines separate layers)

[nova fiber]

scheduler seems to be integrated somewhat

but i have memory leaks somewhere

so thats next i guess

and more testing

[nova fiber]

actually nvm not a memory leak

forgot to move the scheduler object into the correct place so it had an empty scheduler

[nova fiber]

fun c++ thing of the day

cout << "a"; // prints "a"
cout.operator<<("a"); // prints 0x6028f3280017

discovered while concocting this https://godbolt.org/z/4v4MYGEq6

tried to do stuff with mem_fn and bind and got very strange results

[nova fiber]

finally actually integrated the scheduler properly

seems to be actually stable

shocking

[nova fiber]

oh forgot to do address space swapping in the context switch

oops

[nova fiber]

mmm im very close

i nearly have a working userspace again

[nova fiber]

aaaaaa so close

page table loading is still a touch scuffed

[nova fiber]

hum fun issue of the day

somehow jumping halfway into an instruction in userspace

not sure how thats happening

[hollow gorge]

does it give up afterwards

"I will do the first half of this load, but nah, I won't writeback the instruction or retire it. just decode it and give up"

[obsidian grail]

did your OS host a yacht party?

[nova fiber]

oh i see what the issue is, more page table stuff

the offset i jump to is the instruction after syscall in my init program, but its somewhere else in my tty program

[hollow gorge]

orca crashed it

[nova fiber]

womp womp

[thick gulch]

how are you doing external unit testing of a kernel, is this done via qemu and port 0xe9?

[nova fiber]

I have some scripts that launch qemu/vbox/vmware/hyperv with a second serial channel enabled. I redirect the serial output for the second line to a file and output binary data into that about what's happening. Then parse it on the host to see if anything went wrong

[thick gulch]

ah nice that's what I was thinking of doing but not with Google test, because C

[nova fiber]

You can always use C++ just for your tests, I've seen alot of projects do it

[thick gulch]

I'm already using one of the com ports to output a .out profile file for kcachegrind when profiling is enabled

[thick gulch]

sure but if I go to that effort might as well start porting the whole thing for the advantage of stl containers etc

[nova fiber]

You can redirect serial through a unix domain socket and use grpc to multiplex all your stuff through it

But then you need something to actively listen on the other end to record stuff

[thick gulch]

a domain socket on the qemu side?

[nova fiber]

Socket on the host, have the serial line route into it

[thick gulch]

ah I see, nice, I never really bought into the whole Google ecosystem, not a fan of their code style

it's all very javalike, everything an object, camelCaseEverywhere, I would be tempted to wrap it and hide it

my non osdev big c++ project uses a custom unit test system

[nova fiber]

Googles C++ libraries are really good for the most part

Their style guide is basically just javas lol

[thick gulch]

my personal style is the same as the standard library, mostly snake case and shouting defines

it just made sense that the unofficial official style is that of the standard lib previously stl

[olive musk]

Google style is alright, and their libraries are really nice

abseil, gtest, benchmark

[thick gulch]

we disagree on the style being nice but this is ok, style is a personal preference

[hollow gorge]

gnu style

[nova fiber]

grpc is also pretty nice

protobufs are kinda meh in terms of performance but they're nice enough to use

[nova fiber]

incredible https://logging.apache.org/log4cxx/1.5.0/configuration-samples.html#default-initialization

[thick gulch]

does Jeff BezOS come with a yacht?

[hollow gorge]

not a yacht

twelve

but only twelve sadly

jeff is a poor billionaire who can only afford twelve yachts, and we expect him to take out 1% of their income to pay for our schools and public infrastructure? I think we should instead be robbing those wealthy, lavish individuals living under the poverty line, because clearly they have the most money

💔

[hollow gorge]

doxygen 👍

[nova fiber]

wsl2 does not like being abruptly shut down

which is weird because i thought ext4 was meant to not randomly explode on power loss

[hollow gorge]

the journal had bad juju today

[urban schooner]

It didn't explode tho, not all data was written out which is how page cache works if you dont flush it

[nova fiber]

my .git folder regularly has files go missing as well

[urban schooner]

Btw I recommend that u use git from windows

Seems to work more reliably with wsl

[prisma stirrup]

Idk how you guys break wsl, it works quite reliably to me

[nova fiber]

finally fixed that fuckass threading bug

2 months for like 5 lines of missing code lol

turns out using setjmp to implement yield was in fact not the best idea

not that im going to stop doing that

i amaze myself sometimes i mean just look at this

oh yeah i implement sleeps using yield thats probably why i was hitting this

[nova fiber]

shell nearly™ loads

[nova fiber]

I hate elf tls sections

[nova fiber]

My memory management also still doesnt inspire joy

really meh experience using it

[nova fiber]

Still slowly battering code into shape

I forgot just how much code I wrote goddamn

Thank god it has so many tests I'd be fucked without them

[nova fiber]

damnit i actually need to implement on demand paging at least somewhat for rpmalloc to work

Oh i'll also need to make my page tables backing size flexible

[nova fiber]

making the backing pages flexible turns out to be a bit of a chore

[nova fiber]

ough time for more weird specialized containers

this time its an ordered map with a fixed size implemented using a sorted array because i need some way of mapping a virtual address to a physical one without direct access to the current page table

[nova fiber]

halfway done doing the flexible backing pages

you can now give page tables more memory to store ptes in, it just cant give them back yet

i would like to thank my unit tests for letting me test paging in userspace

[olive musk]

I've ported my kernel to userspace based on your advice and it is such a lifesaver honestly

[nova fiber]

another happy customer

[olive musk]

currently doing thread stuff

using ucontext

[nova fiber]

mmm fun

the ucontext stuff gets a little tricky around fpu state, linux is really cagey about the real size of the fpu state for some reason

[olive musk]

I dont mess with that

I just use getcontext,makecontext and swapcontext

[nova fiber]

ah i see, yeah thats probably much easier

[olive musk]

what did you have in mind

btw ucontext is deprecated kinda so maybe I shouldnt use it

[nova fiber]

i turn ucontext_t into interrupt frames and jump to my interrupt handlers

[olive musk]

damn

[nova fiber]

well i try to at least, its still a bit finnicky

[olive musk]

I'm not really doing low-level emulation

interrupts i havent figured out yet

I was thinking signals

[nova fiber]

you'll probably want sigqueue specifically

so you can send ipi equivalents between threads

[olive musk]

well actually

do I really need interrupts

apart from IPI and timer (already handled via sigalrm)

[nova fiber]

if you want to do something like periodic timers for your scheduler i can't think of another way

[olive musk]

yeah I already do that

well actually i do one-shot timers kinda

[nova fiber]

im not sure if it counts but for paging or trapping instructions you'll need sigsegv

[olive musk]

yeah that makes sense

[nova fiber]

doing the debug register song and dance also requires fork and ptrace

[olive musk]

debug register?

[nova fiber]

for emulating mmio

[olive musk]

but like I said im not really aiming for low-level emulation (like pagetables and stuff)

[nova fiber]

you handle sigsegv to map in the memory, set a breakpoint on the instruction after, then handle the breakpoint and unmap the area so you can keep trapping on accesses to it

this is how i test some of my ioapic and lapic code

its really fiddly though so probably not worth the hassle

the execution model is also kinda insane since you ptrace in a child process

[olive musk]

yeah no I wont do mmio stuff

what you're doing is neat but I dont think it's really what I'm aiming for

I dont want to test drivers (yet, at least) but more like kernel logic

pitust and I recently discussed a way to do drivers which would be fun tho

https://discordapp.com/channels/440442961147199490/1151529419589025892/1422902789314445454

basically since my drivers will be in userspace I can write my mmio write/read handlers as remote calls to another kernel running in a VM which proxies the mmio stuff to real devices

[nova fiber]

that is a pretty cool idea

if i was doing a microkernel i might steal that

[olive musk]

Actually I think you can do that in the kernel as well

you have mmio wrappers that when running on userspace Linux does the IPC to the real kernel running in a vm

[olive musk]

do you know whether or not this is required? for some reason my signal handler doesnt get called when I'm running a thread

ah yeah seems like it is

[olive musk]

static void my_thing(int)
{
    ki_handle_timer_expiry(NULL);
    ke_ipl_lower(IPL_ZERO);
}

static void timer_handler(int, siginfo_t *si, void *ctx)
{
    (void)si;
    if (curcpu()->current_thread &&
        curcpu()->current_thread->state == KE_THREAD_RUNNING) {
        getcontext(&signal_context);
        signal_context.uc_stack.ss_sp = signal_stack;
        signal_context.uc_stack.ss_size = 8192;
        signal_context.uc_flags = 0;
        sigemptyset(&signal_context.uc_sigmask);
        makecontext(&signal_context, (void (*)(void))my_thing, 0);

        swapcontext(&curcpu()->current_thread->ctx.ucontext, &signal_context);
    } else {
        ki_handle_timer_expiry(NULL);
    }
}

I ended up doing this, which is only slightly cursed

and for switching threads instead of using swapcontext I use setcontext directly

idk if thats how you do it

[nova fiber]

yeah thats probably fine

[olive musk]

I'm not sure whether or not that ties switching threads to an interrupt context

Since I don't do swapcontext (maybe I can add a check in my switch function whether or not I'm in a signal)

[nova fiber]

finally i got around to kernel unit tests now that i know isa-debug-exit is a thing

it just works™

hardest part was figuring out the linker script to register tests at link time

turns out you can't use .rodata for that, but .data works just fine

anyway now i can do some decent testing for userland stuff

[olive musk]

Damn that's really cool

[nova fiber]

-device isa-debug-exit,iobase=0x501,iosize=0x02 my beloved

[urban schooner]

Waaait what is that device

[nova fiber]

you write to iobase port your exit code and then qemu exits with (code << 1) | 1

such a lifesaver

[urban schooner]

Thats insane

How did u find it

[nova fiber]

digging through qemu source code

theres like no docs on this device but its there

[urban schooner]

Ill be stealing that

Super useful

[nova fiber]

I also have a really bodgy C++ script to generate a kernel image and execute qemu with the right args

c++ is so great as a scripting language when you dont have someone in your ear telling you its unsafe

[urban schooner]

Lol I just use python

But a c++ script is funny

[nova fiber]

c++ as a scripting language is kinda slept on ngl

[urban schooner]

But why this over python?

Seems like you're mimicking it a bit too

[nova fiber]

strong types and library support for the things i want to do

[urban schooner]

Library support in python is insanely good

And strong typing is solved by hints and mypy

[nova fiber]

also python doesnt have defer

[urban schooner]

If you use pytest it does

You just yield from a fixture and then cleanup

[nova fiber]

fair enough

[urban schooner]

Doesn't matter that much ig

[nova fiber]

actually digging around i do have a few bits of python

wow what was i cooking here

[urban schooner]

Damn

[nova fiber]

oh yeah i use canbus over serial to write out statistics from the kernel

i should really just get grpc working in kernelspace and not do this lmao

[trim owl]

make the c++ file executable by doing ```
#!/bin/bash
x=$(mktemp); tail -n+3 $0 | g++ -xc++ - -o $x -std=c++23 && exec $x || exit 1
#include <print>
int main() {
std::println("hello!");
}

for the true scripting experience

[nova fiber]

very tempting

i use java as a scripting language at work and its shockingly not bad

surely c++ will be even better

[trim owl]

very sad that #! is not just a preprocessor comment or whatever

[urban schooner]

Tinyc

[tropic berry]

imagine using bash and not sh for that

[trim owl]

instead of thinking what's a bashism and what isn't i just slap /bin/bash on it and consider it done :^)

[tropic berry]

checkbashisms

:p

[nova fiber]

got flexible page tables working (i think)

[nova fiber]

time for on demand paging i suppose

[sour geyser]

What do you mean by flexible page tables?

[nova fiber]

i can add or remove backing page tables from a processes page table manager

previously every process just got 64 page tables

and then mimalloc tried to map 512mb of ram at startup, which requires more than 64 page tables to do

[nova fiber]

ough now i need to thread the toplevel memory object through all of my system code

will probably be easier to just rewrite it than do that

[nova fiber]

since im rewriting my process management stuff im now faced with a fun architectural choice

do i shove everything into the process like permissions, rlimits, pmm, vmm, etc. Or do i create a bunch of components that manage these things globally and only have the process contain handles that it can use to ask other components to do stuff on its behalf

I feel like the former will probably be a bit easier to start with but will get out of hand when i start adding more features

I guess the latter would also make testing and doing permissioning a bit easier

although for stuff like getrlimit it would be a bit stupid to need to call out to a bunch of different things to get the limits

then again getrlimit is probably far less commonly used than operations that effect resource usage so maybe i shouldn't care so much about that

meh the first time i tried this i went with the first approach and it didnt work, the worst that happens is this approach doesnt work either

[nova fiber]

I realize I have turned my system code into services now

The brainrot is getting to me

Although maybe I could autogenerate api descriptors and do permissions in a common place if everything is nice and uniform like this

[nova fiber]

nightmare codebase encountered

1000 line configure.h output

[thick gulch]

autoconf detected

[nova fiber]

still slowly chipping away at the os, i've been rewriting my pkgtool thing to work a bit differently

right now all the config ends up centralized in a single folder but i'd prefer if each packages config was in its own folder, so im making that work

also very unrelated but i managed to use boost context to implement fibers with io_uring

at the read(), write(), and close() function calls it actually suspends the method and calls into a different continuation

very shocked i got that right first try

[nova fiber]

liburing really has it all

a real usage for add_seals

[thick gulch]

you should call its daemon beachmaster

[nova fiber]

very fun

apparently in al, dx on intel triggers microcode verification

[sour geyser]

huh interesting

how did you come across that?

[tropic berry]

yes, welcome to cancer (Intel)

invest in AMD! (not financial advice)

[nova fiber]

https://youtu.be/Zda7yMbbW7s

[sour geyser]

thanks

[nova fiber]

Very not osdev at this point but i thought it was funny

gaze upon my works ye mighty and despair

full stack c++

frontend is C++ compiled to emscripten and backend is a C++ webserver

with a nice bonus of being able to call aws using the aws c++ sdk

its amazing that the only thing stopping the aws c++ sdk from usually supporting emscripten is cmake being dogshit and impossible to use

compiling it with meson made it basically free

[nova fiber]

Honestly after a day of fiddling with this im sold

[nova fiber]

I'm still working on bezos, just not very much progress

Trying to make it a bit more stable and figure out on demand paging

[nova fiber]

fixed another bug in the tlsf heap

no matter how much testing one does there always seems to be another edge case

this was a dumb one, it couldnt reserve a space in memory if the front of the reserved range was equal to the back of allocated range

now back to crashing while loading the shell

but at least the init program loads again

[nova fiber]

At some point mesons wrapdb gained a loongarch ci runner

yipee another architecture to expose even more strange ass bugs in the dependencies i write wraps for

[nova fiber]

also perl is very arcane

very strange language

unless (my $return = do $file) is not what i would call immediately obvious

oh do is for evaluating a file

thats certainly a choice

[nova fiber]

getting somewhere with the new pkgtool

actually have a working overlayfs daemon

its annoying i need one at all but oh well

[nova fiber]

built perl with meson because openssl needs it

elliothb@ELLIOT-SERVER:~/github/wrapdb-perl$ ./install/bin/perl --help

Usage: ./install/bin/perl [switches] [--] [programfile] [arguments]
  -0[octal/hexadecimal] specify record separator (\0, if no argument)
  -a                    autosplit mode with -n or -p (splits $_ into @F)
  -C[number/list]       enables the listed Unicode features
  -c                    check syntax only (runs BEGIN and CHECK blocks)
  -d[t][:MOD]           run program under debugger or module Devel::MOD
  -D[number/letters]    set debugging flags (argument is a bit mask or alphabets)
  -e commandline        one line of program (several -e's allowed, omit programfile)
  -E commandline        like -e, but enables all optional features
  -f                    don't do $sitelib/sitecustomize.pl at startup
  -F/pattern/           split() pattern for -a switch (//'s are optional)
  -g                    read all input in one go (slurp), rather than line-by-line (alias for -0777)
  -i[extension]         edit <> files in place (makes backup if extension supplied)
  -Idirectory           specify @INC/#include directory (several -I's allowed)
  -l[octnum]            enable line ending processing, specifies line terminator
  -[mM][-]module        execute "use/no module..." before executing program
  -n                    assume "while (<>) { ... }" loop around program
  -p                    assume loop like -n but print line also, like sed
  -s                    enable rudimentary parsing for switches after programfile
  -S                    look for programfile using PATH environment variable
  -t                    enable tainting warnings
  -T                    enable tainting checks
  -u                    dump core after parsing program
  -U                    allow unsafe operations
  -v                    print version, patchlevel and license
  -V[:configvar]        print configuration summary (or a single Config.pm variable)
  -w                    enable many useful warnings
  -W                    enable all warnings
  -x[directory]         ignore text before #!perl line (optionally cd to directory)
  -X                    disable all warnings
  
Run 'perldoc perl' for more help with Perl.
``` what an incredibly scary autotools script it has

this is naturally only the bootstrap miniperl, i now need to use this perl to build the full perl because it autogenerates parts of itself

[nova fiber]

pain

thats a later me problem

[nova fiber]

still hacking away at the new repo managment stuff

finally found a good off the shelf logging library for c++ as well so i don't have to keep copy pasting my own logging stuff into every project

(https://quillcpp.readthedocs.io/en/latest/overview.html)

[nova fiber]

like 50% of the way there to the new repo management stuff working

it can execute shell scripts and track dependencies so i could just do everything with shell scripts

but that seems kinda stupid

[nova fiber]

something that i didnt think about beforehand is that this really shits up my /proc/mounts file

real fun seeing mounts with like 30 lowerdirs in there

i also didnt know you could mount multiple mounts to the same path

ok at least my tests pass now

i still hate that linux makes you do everything via text files rather than having an api

i just love parsing procfs

[thick gulch]

this is the POSIX way; everything is a file.

except the things that arent

[nova fiber]

real

[nova fiber]

discovered that my lock free fixed size queue that i've been depending on for everything is in fact buggy

goddamnit

i'd like to thank gtest-parallel for letting me run the tests 10k times to find out that its buggy

occasionally the queue will drop messages

this does make me think that the freebsd queue i referenced also has the same bug

i at least managed to stop it from reading garbage

[nova fiber]

managed to fix it

unfortunately theres now more atomic operations than before but performance isnt too big of a concern since this is really only for getting data out of interrupts

[nova fiber]

hmm no nevermind still losing entries

making this thing reentrant is such a pain

[nova fiber]

i've just about had it with this damn queue

[nova fiber]

ok tenatively i think i've done it

such a palava though

its not possible to implement a reentrant enqueue method if the stored value cannot be assigned atomically

but i happen to have an atomic bitset so i can allocate indices atomically and use the indices in the queue which can be atomically assigned

so my queue is really a bitmap allocator and a queue

performance is what i'd call tolerable

good enough methinks

on the bright side the actual queue implementation part is alot simpler now

and i can write a specialization of the queue on std::atomic<T>::is_always_lock_free to get rid of the overhead for most stuff

[nova fiber]

god that was painful

done now, tested and comitted

[nova fiber]

that was a very stupid distraction

i just wanted to work on the repo tool

https://youtu.be/_UZFI-8D5uA

me

[calm shell]

been there done that

[nova fiber]

got far enough to build cmake and llvm for the host with my needed patches

tomorrow i'll try and get everything else done for the host and maybe build the kernel

[nova fiber]

nix seems to do what i want but how did they make a dsl this confusing

oh cmon

how is "give me gcc-12" not a simple thing to do

man i wish that the amazon internal build system was open source so i could just use it everywhere, its so much easier to deal with than anything else

[nova fiber]

mmm now that im doing real sysroots i'm discovering so many dependencies i was missing before lol

didn't know i needed libtool or help2man but apparently i did and had been coasting off the system versions

[nova fiber]

man its easy to forget how much of a molehill software in general is

building llvm needs cmake which needs gcc which needs autotools which needs texinfo which needs perl which needs make which needs etc etc etc

doesnt help that gnu software is as scuffed as it is

[nova fiber]

guh i spent my whole 2 week holiday just dealing with this fuckass problem

what a waste

maybe i should just use guix or nix rather than deal with this

[nova fiber]

jlibc builds properly again

i ended up just needing to build the host clang and host libc++ seperately to fix all the strange build issues

so close to a working image again

[nova fiber]

neat makefile trick i found in the wild ```makefile
help: ## Display this help
@awk 'BEGIN {FS = ":.##"; printf "\nUsage:\n make \033[36m<target>\033[0m\n\nTargets:\n"} /^[a-zA-Z_-]+:.?##/ { printf " \033[36m%-13s\033[0m %s\n", $$1, $$2 }' $(MAKEFILE_LIST)

elliothb@ELLIOT-SERVER:~/github/bezos$ make help
Usage:
  make <target>

Targets:
  help           Display this help
  clean          Clean build artifacts
  check          Run all kernel tests
  check-virt     Run all libvirt based tests
  check-all      Run all tests
  stress         Run kernel soak tests
  bench          Run kernel benchmarks
  coverage       Generate code coverage report
  qemu           Run the kernel in QEMU
  qemu-ovmf      Run the kernel in QEMU with OVMF UEFI firmware
  vbox           Run the kernel in VirtualBox
  vmware         Run the kernel in VMware Workstation
  hyperv         Run the kernel in Hyper-V
  pxe            Copy PXE boot image to TFTP server folder
  integration    Run all VM tests

[sour geyser]

That's pretty neat

[nova fiber]

got my hands on a disposable vape with a screen and mcu in it

time to port bezos to it

the mcu is some undocumented chip from china that might be Padauk based

13 bit addresses

no registers either, only memory addressing

[small stag]

@pure crane i think the thread should be locked

[hardy sand]

Why

Oh

I see

[small stag]

sparc got naenae'd

[hardy sand]

Yeah

[olive musk]

rip

[hardy sand]

Agreee