BadgerOS

[fading elm]

forget thread

[obtuse creek]

anyway this is in no way an actual immediate problem

just something to keep in mind

so no need to worry about it too much

[fading elm]

So now I mustm't;dshall figure this out maybe eventually idk

[obtuse creek]

yes it is an interesting problem

for now you should consider not making your ramfs use massive contiguous buffers since that probably will make the allocator unhappy

[fading elm]

I don't see a way to do this without a stack?

[obtuse creek]

you "simulate" a stack

just like how you simulate one for other defunctionalization problems to turn a recursive problem into an iterative one

[fading elm]

Yeah but a softstack is still a stack

[obtuse creek]

uh it can be a list

so not stack

no stack usage

[fading elm]

ig it's better than using 64 bytes or some shit per layer on the actual kernel stack

Actually it's probably more

[obtuse creek]

rust stack usage gameplay

[fading elm]

No C does this too

Rust does it in debug builds so for now I just make kstack larg

[obtuse creek]

how larg

fourty megan bytes

[fading elm]

I have 8K stacc right now.

At some point I will impl stacc guards so I can detect overflow

Anytime it does I make it larger and try again

I guess it's the same thing as using the softstack in the walk function

but in the I/O functor insteads.

[obtuse creek]

too small unused ram is wasted ram, make your entire stack swappable to disk and use all the unused address space as stack

[fading elm]

For real though why the fuck does anyone bother trying to swap literally any part of their kernels?

[obtuse creek]

some people use systems with a solid 2 Megan Bytes of RAM so they don't have the luxury of keeping everything mapped all the time

[fading elm]

well I'm dysphoria programming too hard to bother with that bullshit

[obtuse creek]

one cool thing you can trivially do is put init functions in a custom linker section and after all of them run, you unmap them and mark them as free in the physical memory allocator

lots of real kernels do this like linux with __init

it not very hard to do either

[fading elm]

And save what? 4 pages of memory?

Pfffff

[obtuse creek]

yep super important

it might increase as your system grows

[obtuse creek]

what if you want to boot your system on a dec alpha

very realistic concern

[fading elm]

realim

[obtuse creek]

actually those had up to 8 Giggler Bytes of RAM nvm I forgor

[compact ginkgo]

I don't know if there's any good reason to page kernel code anymore

But kernel heap there def is

Because there's hundreds of megabytes of that

[fading elm]

That mean I would need to tag krnel heap with swappable or not

[compact ginkgo]

Yes

Actually you'd have two kernel heaps, pageable and nonpageable

[fading elm]

imma go ahead and don't

[compact ginkgo]

More for me

[fading elm]

you can have my heap i wasnt gonna eat it

[obtuse creek]

the javascript developers will be very unhappy that the kernel is using 12 whole precious kilobytes of RAM that they can't use to keep their node modules cached in ram

[obtuse creek]

PAGE_FAULT_IN_NONPAGED_AREA 😔 ✊

[fading elm]

Windwos skill issue

pffffffff

why do i stae up so late,,,

[compact ginkgo]

Why are you talking like that

[fading elm]

because its late

im go to bed now cya later

[fading elm]

Now working on realpath and mount

[fading elm]

Those work, now make_file, link, unlink and rename...

[fading elm]

With that part of VFS mostly done, I'm going to add the character and TTY device classes now.

/dev/zero and /dev/null will become character devices

[fading elm]

My VFS now has the infrastructure needed for something like mount /dev/sdb /mnt to work

I will now attempt to make the switch over to the new VFS for the whole kernel

[fading elm]

This may take a while

I'm in a slightly akward spot where the C side is now going to be lobbing around fat pointers (read: a struct with two pointers in it). And my ELF loader (which is already a bad fit because it's designed to linked dynamic files into the current executable, not be a kernel static ELF loader) doesn't accept anything but normal pointers. Kinda tempted to rewrite that too but for now I'll just box it.

[pure seal]

good luck!

[abstract cypress]

Is it like a typical struct Path { DEntry *dentry; Mount *mount; }; kind of thing that you're moving everything over to?

Many things that only deal with individual files and never care how they are connected to the rest of the filesystem hierarchy are probably fine with dealing with DEntries directly.

[fading elm]

No the problem is that my file handles are Arc<dyn File> (a trait object) so they're fat pointers.

[abstract cypress]

Ah, ok.

[fading elm]

So it'd be a struct {void *data; void *metadata;} situation

[abstract cypress]

ah yeah so this is a C <-> Rust interop thing?

[fading elm]

Yeah

Existing C code will need to interact with the VFS. This mainly is the program loader.

[fading elm]

The VFS works so far.

Init crashes because it's a test for pipe and I haven't reimplemented that yet.

I also need to reconnect the devtmpfs to the code that puts stuff in it.

[fading elm]

Currently working on reconnecting the syscalls to the VFS. With a bit of luck, the VFS rewrite will be finished today, after which I'm going to make the logic for mounting the root filesystem.

[fading elm]

EVERYTHING the old one did is IN and WORKS

Even the pipe test!

Well except for the FAT fs driver. I still need to do that one.

I guess I'm going to make that now

[hazy crystal]

fucking balling

[fading elm]

Did some good progress on FAT the last two days, I'd say I'm 70%? To reading a known-good FAT filesystem.

[hazy crystal]

epicc

[obtuse creek]

which fat

32/16/12?

or exfat + vfat extensions and whatnot

lol if this is a unix system you'll probably have to fake inode numbers for FAT

🤪

pretty cool though

[obtuse creek]

check iunique from linux to see how they do it 👍

[fading elm]

VFAT32/16/12

[fading elm]

indeed

[obtuse creek]

awesome sauce

[fading elm]

We're testing FAT read tonight kings

(and queens (and other royalty))

[fading elm]

Oh I've tested the FAT filesystem alright

Doesn't work

[fading elm]

Reading dirents works now... Unfortunately, reading the FAT doesn't :P

[fading elm]

OK so I'm tired of VFS for the moment and since I also need to re-work my PMM (and a little bit the VMM too), I'm going to do that now.

Long live git branches.

[hazy crystal]

yooo at least dirents work !

[hazy crystal]

understandable

[fading elm]

I worked some on the PMM today, main goal is to make things like demand paging and CoW mappings possible. I'm making space for mmap'ing files and swapping userspace out, but that's more VMM than it is PMM.

[hazy crystal]

page cache is gonna be absolute pain

[fading elm]

To achieve this, I'm making a new page allocator (buddy alloc again) that uses a refcount instead of just an allocated/free state.

[fading elm]

Fortunately most of that is in already

[hazy crystal]

[fading elm]

Also my mom says hi

[hazy crystal]

hi mom

[fading elm]

Making new MMU abstractions in Rust right now, as the code that will be dealing with vmm is, of course, getting written in Rust now.

[hazy crystal]

makes sense

[fading elm]

I'll be doing just replacement of page table manipulation first, which I'll test with the existing C memprotect infra.

After, then knowing the page table code works, I will then replace the remaining memprotect with the new Rust VMM.

All this will be based on the old page allocator because it works fine and I don't need to replace it yet.

[fading elm]

Bored of Lily-CC again, now we ball wiff more VFS code.

[fading elm]

Now I shall implement the long filename extension for reading.

Yes I know I probably have ADHD lol

[fading elm]

You'll never guess what happened when I tested long file names

[fading elm]

Ok I fixed it, reading LFN no longer causes an I/O error

[fading elm]

Tested multi-cluster file reads, those work too now, very good

Writes to existing regular files work

[fading elm]

We doing more FAT today

I've mostly finished unlink/rmdir and am now working on mknod

[ornate wyvern]

That always happens to me when implementing filesystems

I don’t even know how to prevent that

WOW I DID NOT REALIZE THAT MESSAGE WAS OVER A MINTH OLD

[fading elm]

[fading elm]

While I was at it, I added support for the extension that lets you have all-lowercase filenames without LFN entries

[fading elm]

Almost done with the logic that writes new dirents

[fading elm]

Removing regular files works without leaking nor corrupting the filesystem!

Same for directories

[fading elm]

Have untested stat now, but first time for eep

[fading elm]

stat works, mknod time

[fading elm]

Ok so now I have working mknod

Only rename is left before FAT is completely finished 🎉

[thorn thorn]

Nice

[fading elm]

After some discussion #filesystems message, I've re-worked my dirent caches a little bit and made my VFS no longer implicitly rely on inode numbers. Instead, for filesystems without real inode numbers, they are faked, and the dirent cache is used to prevent duplicate file opens.

Damn, I really thought I'd be done by today.

But at least I'll end up with a proper proper VFS and FAT impl.

[fading elm]

Ok so mknod for the directories had a bug but that's fixed now.

And: I've got rename in too!

I've got FULL support for vfat!

No, I don't abuse my poor git repo, what are you talking about?

[fading elm]

Currently squashing FS and VFS bugs, and working on finishing umount.

If all goes well, I'll have FS automount done by this evening.

[fading elm]

I am beyond stunned that reclaiming memory directly from the limine_memmap response without caching it (thus overwriting the response as it's being read) hasn't caused a crash before.

93 files changed, 8620 insertions(+), 6172 deletions(-)
VFS rewrite is finally merged! And automounting root works!

[hearty brook]

epics

[fading elm]

I'm actually kinda proud of what I did here for the Ext2 on-disk structure definitions

@solid bronze Do you approve of my fancy Rust macros here? :D

[solid bronze]

cursed

[fading elm]

lol

[fading elm]

Almost ready to do a read test already!

[fading elm]

Simple mk2efs in build script, let's see how hard BadgerOS explodes when automounting this lmao

[fading elm]

Calculated addresses of shit wrong so now the block group descriptor table reads 0... classic.

[fading elm]

It's progress

[fading elm]

Ok ok ok

So

IT AUTO-MOUNTS AND READS THE ROOT FILESYSTEM

[untold elk]

@fading elm if you want to install your OS from a USB stick, you'll need to be able to write the fs, right?

[fading elm]

Yeah, I'm actively working on Ext2 write right now.

I'm going to use existing userspace tools for partitioning and filesystem formatting, then mount the filesystems and copy it all over.

Since I boot with Limine, a bonus would be porting the Limine installer binary so it can boot on BIOS (if somehow a 64-bit system has BIOS but not UEFI)

[untold elk]

that's what I did too

on Maestro I use GRUB. I have grub-install working so I am able to install it from a USB stick too

[fading elm]

AFAIK that's how most installers work

But I will need some stuff like USB drivers before any of that will happen

Modesetting would also be cool at some point, no idea how much effort that'd be

Oh and don't forget VMM refractor, fork+exec, shebang, TTY, actual shutdown and reboot, etc

[untold elk]

I don't have a USB driver. I just have GRUB loading the whole fs (with packages to install) in an initramfs (that's bad, don't do that)

[fading elm]

LMAO

Also fun fact: I don't support initramfs

[untold elk]

I implemented it just for this purpose

that's less efforts than USB

[fading elm]

I'm too much of a perfectionist to let myself do that

So USB suffrage it is

[solid bronze]

why lol

[fading elm]

Because I didn't write anything that unpacks one

[solid bronze]

cpio is easy asf

<100 lines

[fading elm]

You know what it also is?

Something I don't need right now.

[solid bronze]

[fading elm]

Also just remembered I'm gonna need to make a new ELF loader at some point because KBELF, though made by me, is not made for kernels.

(It's designed to load stuff into the running program)

((And even then only sort of))

[solid bronze]

whats kbelf

[fading elm]

My ELF loader library

Not intended to be a kernel ELF loader, though.

[fading elm]

https://github.com/robotman2412/KiloElfLoader Not the cleanest codebase ever either lmao

[fading elm]

I think the reason I'm getting Ext2 FS corruption is because my pagecache is broken

[fading elm]

It was

Look at this shit

[fading elm]

First Ext2 write without corrupting it!

Free block count is still wrong, but it no longer makes the FS unreadable

And now those counts are also correct 🎉

[tidal sage]

struggling with the same thing 😭

[fading elm]

Working on link and unlink now

[fading elm]

unlink without corrupting the FS 👌

link too, let's test making dirs and symlinks

Yeah mkdir still corrupts a bunch of crap lol

[fading elm]

Now there's just rename and some (other) codepaths for inode cleanup left

[fading elm]

MM rewrite time

New page table creation and walking code is already there, now it's time to build VMM on top of that.

[fading elm]

I feel like after this, a process subsystem re-write is in order.

[fading elm]

Ok the new page table building code works now

[fading elm]

Well then

Bad news: crippling dysphoria

Good news: I finished the last two little shits about Ext2 at 4AM

[fading elm]

Got the kernel to boot under new VMM, but the init process explodes and generally takes really long to make the page table mappings

[fading elm]

Rust outnumbers C now

That screenshot was shit have this one instead

[thorn thorn]

Damn, command lines people invent to avoid —vcs=git

[fading elm]

wait what?

XD

This is so much easier lmfao

Don't know where it's getting fortran from lol

[fading elm]

Thank you for bringing this to my attention XD

[thorn thorn]

Np lol

[fading elm]

Incorrect page table mappings

How am I supposed to debug my page table when this is what info mem prints???

There shouldn't even be anything mapped with accessed and dirty bits cleared

[fading elm]

Now I'm at the part of the rewrite where I need to actually change my PMM.

VMM is up to where it was, but now I want to implement CoW mappings and memory accounting, so I need to update my PMM to make that possible.

[fading elm]

After some discussing with ilobilo, korona and marvin, my buddy alloc is almost finished; I estimate by this weekend I'll have the new PMM done.

However, I don't know exactly how I want to go about mmap'ing files from the VFS perspective just yet.

[fading elm]

@flint shell My buddy alloc is almost done but I'm concerned that trying to allocate blocks from multiple threads concurrently may cause the operation to spuriously fail, even if there's enough available pages of that order after e.g. the other thread splits a larger block. Do you have any suggestions as to how I should go about this?

[flint shell]

I can only really give suggestions if you explain your proposed design

[fading elm]

Current design uses these:

/// Physical memory page metadata.
#[repr(C)]
pub struct Page {
    /// Page refcount, typically 1 for kernel pages.
    pub refcount: AtomicU32,
    /// Page flags and usage kind.
    pub flags: AtomicU32,
    // TODO: Pointer to structure that exposes where it's mapped in user virtual memory.
    // Kernel virtual mappings need not be tracked because they are not swappable.
}

/// Physical memory freelist link.
#[derive(Clone, Copy)]
struct FreeListLink {
    prev: PPN,
    next: PPN,
}

/// Physical memory freelist head.
struct FreeListHead {
    first: Spinlock<PPN>,
    amount: AtomicPPN,
}

/// Free lists per buddy order.
static FREE_LIST: [FreeListHead; MAX_ORDER as usize] = unsafe { core::mem::zeroed() };

Spinlock guards each order in the free lists, and the amount is used to atomically reserve one without needing to lock immediately

But I'm considering just using a global spinlock as the current design arguably still doesn't really let multiple threads truly concurrently allocate pages.

I could always help the concurrency bit by making a sort of wrapper around this that reserves a pool of pages per CPU

[flint shell]

you could use sharding for the free lists

= not have a global one but have around #cpus ones

[fading elm]

Okay

[flint shell]

but that ofc adds complexity because you need to look up the right freelist for the page that you're freeing etc

[fading elm]

Yeah

And if you're very memory-contented, you will need to "steal" from other CPUs' sections

I think I will keep it simple for now and just use the one spinlock for the freelist. If I need better perf in the future, then I can improve it.

[fading elm]

It manages to do VMM init now before crashing just after.

[fading elm]

Can confirm that my buddy's freelist is horrendously broken.

But at least I know in which way now, so I can fix it.

[fading elm]

Turns out I was accounting the pages incorrectly when they got split, causing the blocks that were created to think they were an order bigger

Now there's a nullptr crash in DTB, which is a bit interesting

I suspect that some part of the MM refactor caused this

[fading elm]

My DTB parsing code appers written in an incredibly bugprone and fundamentally broken way; I'm going to replace it and see if the issue persists.

[flint shell]

DTB parsing should be pretty straightforward

[fading elm]

Yeah, well, I wrote it on crack it appears

It's one of the oldest pieces of C in my kernel atp because most everything else has been updated since then or even rewritten in Rust.

[thorn thorn]

seems like we do need a uDTB afterall

[fading elm]

I mean

I could make the DTB rewrite into a more platform-independent way

But I won't

[thorn thorn]

how can you make dtb parser platform dependable 😭

[fading elm]

It's called my stupid ass thought it was a good idea to make my own version of string.h

But I'm not going to fix that right now because eventually C is going out of my kernel anyway.

[flint shell]

libfdt is basically uDTB

[thorn thorn]

true

[fading elm]

Fortunately DTB is a simple enough datastructure, even if you want to have a map from phandles to nodes.

[thorn thorn]

yeah its hard to come up with a simpler structure

[flint shell]

it's almost a bit too simple

you basically need to build an auxiliary data structure to work with it efficiently

since you can't just skip over a node without reading it entirely (and all its children etc)

i do realize that this is by design and that it allows firmware to modify it in a straightforward way by just memcpying data around (w/o changing any pointers or offsets etc)

but it's sometimes a bit annoying

[fading elm]

Yeah, it does result in unpacking it in its entirety.

[fading elm]

I do still kinda hate how DTB is big-endian for some stupid reason

[solid bronze]

we already have that

i think @scarlet bison made one?

nvm

it was @mental pagoda

https://github.com/DeanoBurrito/smoldtb

[fading elm]

honestly

I'm just gonna bind that for Rust and call it a day

Parsing DTB isn't high on my "want to do" list

Just goes to show how much more you can do with the help of others I suppose

[flint shell]

Isn't it easier to write DTB parsing code than it is to write rust bindings for that?

Just make a trait Visitor with two functions on_node and on_property, walk through the DT once and call these functions?

[fading elm]

I've just finished my new DTB parser. I will create tests for it later, then see if this fixes the problem.

If it doesn't I may consider binding smoldtb instead.

[fading elm]

No, writing bindings is still easier, but I was already most of the way though my own DTB parser so I just kinda finished it.

[thorn thorn]

Out of curiosity, which part are you having issues with?

[fading elm]

Like I said, the old one was just made badly

Don't know why

But it probably has to do with that it was written fairly early on

I've become a much better programmer over the course of this

[fading elm]

I might have to take another break from BadgerOS

I just want to get cool stuff running but all that's happening is fixing tech debt

[thorn thorn]

thats fair take your time

[fading elm]

I just realized I'd been arbitrarily slowing down BadgerOS' logging

I have no idea why the delay is in here

[frail perch]

I have nothing more to add

[fading elm]

Oh lol

Well I do. for one, no SUID binaries at all.

That means getting root permissions is talking to init

[verbal sphinx]

That’s not stopping you from leaning on work already done, also systemd run0 comes to mind

Hell Managarm didn’t support SUID for a while (we forgor whoops)

So that should work with some patching

[fading elm]

Another one is I'm adding a custom API for both input and window management because I have some funny ideas about making the same binary run on a microcontroller and on this cursed mess

(Yes, I still haven't given up on BadgerOS microcontroller edition)

[verbal sphinx]

Valid, doesn’t concern libc at all, requires the obvious patching to relevant stuff getting your backend in

[fading elm]

This bit right here is also why I was considering newlib, because the environment that said microcontroller runs has newlib

Don't worry, I'm not going to make this entire userspace run on a microcontroller

It's more like running what is on the microcontroller on the bigger computers too

[fading elm]

NS16550A time because I don't want to work on PMM right now but I do want to work on BadgerOS.

That break didn't take very long, did it?

I'm going to focus on some more fun stuff before I return to the chores like PMM and such

[untold elk]

https://youtube.com/watch?v=pzagBTcYsYQ

[fading elm]

haha yep

[fading elm]

I'm continuing work again on the device subsystem re-write

This time with much less unsafe shenanigans to cause shitfuckery

[fading elm]

The new Rusty DTB parser that isn't made out of bugprone-ness is finished

[fading elm]

Ok so that's actually going to be a ton of effort I don't want to do now. I will still rewrite it all to Rust eventually, but I'm going to try to debug the existing thing first.

[fading elm]

I actually at this when I saw it

Because this means my serial driver, interrupt subsystem and device subsystem all work properly!

[fading elm]

Ok so it turns out that had some PLIC bugs causing interrupts to only half work, but now that's fixed too.

[fading elm]

The new page table code I wrote for the MM refactor a while back is panicking due to an apparent corrupt page table.

I've also fixed some other bugs that were blocking adoption of this refactor, though there's probably loads more. Also, my way of finding an empty range of virtual memory is highly inefficient.

[fading elm]

Yooo the new page table code actually works!

Time to add the new features (CoW pages and mmaps)

[fading elm]

I'm changing how this is structured slightly, making a struct Memmap for all user memory maps (and perhaps the kernel too?); this will allow me to build an API where everything within is purely based on the intent of the mapping, rather than the old system of directly manipulating page tables.

[fading elm]

Turns out my new PMM is both horrifically slow and broken for unknown reasons

Oh wait the slowness is just a debug assert that walks the entire freelist.

Still broken though.

Turns out I was marking part of the page structs as usable memory

[fading elm]

Ok now realloc is broken

[fading elm]

I wasn't updating the block size correctly when allocating physical memory

[fading elm]

I am 1. somehow running out of virtual address space (should be impossible with only 1G RAM) and 2. still wondering why the PMM is slow

[fading elm]

Yeah so QEMU TCG (because RISC-V on an x86 host) + Rust being -O0 by default really does a number on performance

[short grail]

you can override optimization level for the debug and release profiles in Cargo.toml

i think you can do toml [profile.dev] opt-level = 2or something like that

[solid bronze]

i do opt-level 1 by default

O0 is unusable

[fading elm]

yep

[fading elm]

Look like I'm getting an interrupt storm now

Actually it's a deadlock?

Long live heisenbug land

Well it's not a heisenbug. It's just. Something that will be hard to track down.

I also fixed implicit hugepages so setting up VMM isn't super slow anymore.

In theory the new MM is good to use, but I don't want to merge before I've excluded this from being an MM bug.

[fading elm]

Turns out the new page table code has some more mutexes and stuff in it, which is fine until I realized that the thread reaping unmaps the stack. Trying to acquire the threads lock again because of the waitlist_notify when you release a mutex.

I'm probably going to have to rewrite 2.0 the scheduler at some point looking back at all this deranged garbage. But it works just barely fine enough.

[fading elm]

Actually that was only 1/2 of the problem

The interrupt storm is still there

I could see it breaking in this way if perhaps the I/O pages for the PLIC aren't mapped properly, causing interrupt sources to be misconfigured?

Ah. The interrupts are being enabled before the driver for the CPU-local interrupt controller is installed. Then a normal preemption timer interrupt fires.

Now why exactly this timer interrupt took nearly 30ms to deliver I do not know.

[fading elm]

Ok, all the new MM code works! Now, it's time to make mapping pages fast again.

[fading elm]

Tangent to make x64 run again with new MM

I hope to be able to run the QEMU profiler on that so I can optimize the MM a little bit.

[fading elm]

New MM merged, time for some drivers

[fading elm]

Ok so I know y'all voted ACPI and I did work on it but I put that on a branch for the time being. Now begins the process subsystem re-write because I wanted to.

[fading elm]

Funny how easy ELF loading is if you don't for some reason try to make your kernel do dynamic ELF loader duty.

[solid bronze]

trolled

[fading elm]

Yeah I clearly had no idea how to ELF load first time I made any of this crap

[solid bronze]

better than loading an elf using its sections

[fading elm]

Even though I didn't call it an actual rewrite, in effect I am rewriting most of BadgerOS since I decided to add Rust.

Funny how much faster it goes once you know what you're doing.

[fading elm]

did someone actually? lmao

[solid bronze]

[fading elm]

[fading elm]

I'm approaching a point where I need to go out of my way to rewrite things in Rust because the C code just gets in the way otherwise

Over 50% of my kernel is Rust now

I think in particular the scheduler is one of these things

[fading elm]

load_executable works, now it's time to reintegrate syscalls

(and also decide whether I want to keep doing it the same way)

[fading elm]

I've decided I will keep syscall working as they do now and put signals on hold. I will refactor how context switching happens after most of the new process subsystem is in, after which I can reimplement signals and possibly do some more polishing on the scheduler.

[fading elm]

After some discussion with @solid bronze, I've implemented safe user-copy abstractions and with that started reimplementing all the syscalls

[fading elm]

currently impl'ing signals

[fading elm]

Synchronous signals work™; I'm not dealing with the async ones until after the scheduler rewrite so now it's time for fork and exec!

Actually, I need to do some more VMM stuff before I can support that because fork() makes CoW mappings but the VMM doesn't actually implement those yet.

[fading elm]

CoW is in but untested

[fading elm]

CoW mappings work

Time to go into fork, then!

[solid bronze]

how bad is it

[fading elm]

2 years ago I wrote sched but didn't separate it from context switching at all

Makes it unreadable and inconsistent af

It's probably the same kinda effort it was for me to e.g. rewrite VFS

[fading elm]

Unfortunately I think fork is dereferencing a stale pointer and/or causing memory corruption

Rust didn't prevent this because there's a lot of switching between C and Rust layers here, making preservation of safety harder.

[fading elm]

Now we have a nice clean fork() except for a dirty scheduler side because I'm rewriting sched soon anyway. Next up will be exec() and then porting some fun stuff like busybox perhaps.

[fading elm]

Cool, exec() works too now

[storm marsh]

Wow, almost 2 years done now! Congrats.

[fading elm]

Thanks!

And after those nearly two years, it is now finally time for me to build a distro. First step: Port mlibc and make a cross compiler for BadgerOS userspace :)

[storm marsh]

looking forwards to it.

my toolchain is just f'ed.

looking forwards to taking inspiration from your code :D

[fading elm]

I'm probably going to look into using jinx

[fading elm]

I try to keep it sane and readable... Sometimes I succeed

[storm marsh]

for me the problem is my patches

and my build system which wont reqonise the target auroraos when building libgcc

[fading elm]

Yeah that's why jinx uses a Debian container

[storm marsh]

Well, it aint the container thats the problem. I straight up stole the patches from gloire.

And now its something that its missing.

[fading elm]

It is imperative that jinx be run under Linux, as the container environment relies on non-POSIX Linux features. You will run into issues on other systems.
Oh. I really want to be able to bootstrap at some point (compile BadgerOS from source on BadgerOS).
@sullen cipher Is there a chance I'll be able to implement the Linux-specific features jinx relies upon?

Eh I'll figure that out when I get there. Let's get jinx set up for now.

[short grail]

jinx wants user namespaces

[fading elm]

ah interesting

That does sound like a feature I'd want to implement some form of myself at some point too

But not exactly soon :P

[fading elm]

I cannot use Jinx; it is practically undocumented and I am not willing to read its code to find out how to use it.

[solid bronze]

xbstrap

[short grail]

it's partially documented

there's DOCUMENTATION.md

[fading elm]

Yeah but that's outdated by seemingly quite a bit

And also doesn't explain e.g. why this:

name=binutils
revision=1

imagedeps=build-essential
tarball_url=https://sourceware.org/pub/binutils/releases/binutils-2.45.tar.gz
tarball_sha256=5c71ec80884c0b4c0f7a4f8600946f1f6feebe584261b63185b0942cba4062d5b6cf2337539132d0ca03505c0a74f3c2760bc62ed08843c3b7db03df7f5798ad

configure() {
    ${source_dir}/configure \
        --prefix=${prefix} \
        --target=${TARGET_TRIPLET} \
        --with-pic
}

build() {
    make -j${parallelism}
}

package() {
    DESTDIR=${dest_dir} make install
}

Gets me the error "could not find source recipe 'binutils'"

I am clearly not using a source recipe here

[short grail]

host recipes can't have inline source recipes like that

you either need a recipe/binutils or source-recipe/binutils

it is kinda roughly mentioned in the .md file if you read hard enough lol

[flint shell]

to be fair xbstrap is also not documented :^)

[short grail]

me when schema.json is the documentation

[fading elm]

Actually tempted to go get some bullshit like how arch's packaging system works

Because it'll be documented

[short grail]

they are not very user friendly

they are made specifically for their distros

jinx and xbstrap are probably the most "portable" solutions out there

[fading elm]

oh great

I'll take SOME over NONE then

Let's get binutils going

Ok... I need to patch binutils configure it appears because it ofc hasn't seen the combination of BadgerOS and mlibc before

[fading elm]

Fingers crossed I hit configure over the head with my baseball bat correctly

[fading elm]

I'm actually at a point now where I can be making my mlibc sysdeps

[storm marsh]

ooo sounds good, you got both gcc and binutils done?

[fading elm]

Well GCC only to the point it complains about no headers since I hadn't ported mlibc yet

[storm marsh]

Nice!

[fading elm]

Currently writing and/or defining the mlibc ABI headers

[storm marsh]

I copied astral lol

and he copied linux

basically everyone copies linux's-

[fading elm]

I don't

Making my own types for this

Once I have them all defined I'm copying them into the kernel repo

[flint shell]

Making your own ABI is quite involved and also error prone

[fading elm]

For example, all of my ID types (PID, UID, GID, inode, etc.) are 64-bit. This is not true of Linux.

[flint shell]

and provides almost no benefit over an existing one

[fading elm]

So then what? Use Linux' but replace just stuff like the ID types myself?

[flint shell]

yeah you could do that

[fading elm]

Ok so I mostly took Linux but have my own fdset (because my limit is only 256) and integer types.

I also included my syscall headers in it, which are of course different than those of Linux.

[fading elm]

I've successfully build binutils, gcc, mlibc and libgcc. Initial bindings for BadgerOS are in mlibc. Now it's time for me to collect the ABI bits from mlibc into a neat little module for BadgerOS.

[storm marsh]

Wow Congrats!

[storm marsh]

Do you have a gh or codeberg repo?

[fading elm]

Yeah, though the one for the distro is currently not pushed

https://github.com/badgeteam/BadgerOS-Kernel

[storm marsh]

lmk when you pushed it, I would love to see you're patches cuz mine are broken asf.

[fading elm]

lmao you don't want to copy those from me

[storm marsh]

no but i want to check ur logic to build it cause mine is hella scuffed

[fading elm]

Yo WTF is wrong with the Linux ABI definitions

[flint shell]

hm?

[fading elm]

struct stat has different definitions per CPU architecture!

How is that better than making one struct stat definition for myself instead?

[flint shell]

well, that's indeed braindead

[fading elm]

Adding stat.h to my overrides

[lilac surge]

yeah it's a bit of a nightmare

linux initially was forked for every new architecture added

[solid bronze]

@fading elm why don't you join the managarm server?

https://discord.gg/7WB6Ur3

makes it easier to communicate

[fading elm]

My WIP mlibc port and distro are up now:
https://github.com/badgeteam/BadgerOS
https://github.com/badgeteam/badgeros-mlibc

Kernel repo was renamed:
https://github.com/badgeteam/BadgerOS-Kernel

[spare nymph]

managram

[solid bronze]

no bsd/glibc extensions?

[fading elm]

Not yet

Basic stuff first, extensions later

[solid bronze]

i just defined the kernel ABI, and delegated everything to the kernel

that way i don't need to make adjustments in mlibc every time i implement something

[fading elm]

Well the kernel ABI isn't done yet

[fading elm]

welp

time for the LARG sched rewrite :/

[flint shell]

LARG?

[fading elm]

very big

TL;DR: The scheduler is so bad it's better for me to rewrite it than fix the current one.

Has piss poor separations of concerns too.

But this is a pretty damn big re-write because it needs to interact with so many other systems.

[compact ginkgo]

you think thats bad. they had to rewrite the windows scheduler recently

[fading elm]

oh no

[jade needle]

What’s that down to, then?

Funny thing is, Microsoft were saying only recently that they'd be sprucing up the scheduler's performance for gaming - https://www.techpowerup.com/343895/microsoft-promises-more-performant-windows-11-optimized-for-gaming - most likely off the back of a fair few grumbles cropping up on Windows 11.

[spare nymph]

the old scheduler was fine

the elephant in the room is refused to be addressed 😭

[compact ginkgo]

i was pretty sure it was because of big.LITTLE

[jade needle]

If this has truly forced them to rewrite the scheduler (dispatcher?), then it's rather exciting, in a way.

[fading elm]

Damnit I just realized this scheduler rewrite is likely to also be an SMP rewrite

Yeah this one is gonna be a PITA

[solid bronze]

how

is your stuff THAT tangled?

[fading elm]

By being the stubborn kinda idiot that wants to do everything herself

I didn't read any literature about any of this, just some occasional wikis

That's the easy part finished.

[fading elm]

Also the SMP thing isn't because it meddles with the scheduler but more like it's not feasible to make proper interop between a C SMP layer and a Rust scheduler because most of the Rust types there can't be represented in C.

[fading elm]

Just wrote the new assembly file for the exception and system call trampoline for RISC-V 64-bits.

I will make some sort of more formal CPU spinup thing tomorrow and install it from there.

This 200-line clean assembly file (prolly woulda been shorter without the extensive comments) replaces about 500 lines of spaghetti asm

[mental pagoda]

lol, that was me a few years ago with this stuff. You live and learn.

[fading elm]

I think it's also just that, with the scheduler becoming Rust, I've reached the threshold where I need to rewrite C into Rust for the codebase to remain clean. I estimate that by the end of this rewrite, Rust will probably make up over 70% of the codebase.

[solid bronze]

what's wrong with dropping C altogether

[fading elm]

Too much to rewrite at once

I don't feel like having a kernel that doesn't even compile for multiple months because I decided to try rewriting everything at once.

[solid bronze]

no like, gradually

[fading elm]

That's basically what I've been doing the past few months

By the end of this scheduler 3.0 rewrite, some of the SMP stuff might be left as well as the device subsystem and misc utilities and libraries that C depends on (e.g. collections).

[short grail]

you really should take a look at the code bro

[solid bronze]

?

[fading elm]

If you look at my code, the last couple months have set a trend of replacing C code with improved versions written in Rust

[solid bronze]

i got that

[short grail]

i don't think anyone said dropping C is bad

in fact, the entire point of rewriting small part of the kernel in rust IS to drop C eventually

(i think, i'm not robot lol)

it just seems that writing a new kernel in rust is counterproductive

when you have all of that C code that works but can be improved upon

[fading elm]

Well atp most of the code is Rust

[solid bronze]

i wonder if it might make sense to write a microkernel in C and the drivers in Rust

(not that I'm planning to do that)

just hypothetically

[short grail]

sure

i was gonna do rust drivers for managarm lol

though the kernel is C++, but it's close enough

[flint shell]

make a proper tty stack in rust for Managarm :^)

[lilac surge]

RUSTREAMS

[short grail]

i don't even know how to make a proper one in C

[fading elm]

Integrating Rust with C++ properly isn't exactly trivial

[short grail]

it's as trivial as integrating rust with C

[fading elm]

Is it? I assumed it wouldn't since C++ has a bunch of shenanigans with classes.

(Which, of course, don't fit in Rust's type system)

[short grail]

"classes" are just data, but nobody forces you to use them

just like you don't use rust data types in your C code

i'm assuming you have some structs which are repr(C) that you share with the C code

or some which are bindgen'd

you can do the same with rust and C++, you can have POD structs that you use at language boundaries

that's inefficient i guess, there's also the cxx crate which is like bindgen for C++

[fading elm]

I pushed the scheduler 3.0 stuff to a branch just now since its proof of concept works.

Also fun fact: If you were to remove the sched3_test from main it'd function as normal with the old sched because the new one doesn't collide with it, what being written in Rust and all :P

This actually happened back when I did the VFS rewrite too

[solid bronze]

well

there's an entire abstraction layer between the kernel

bragi (similar to protobuf) can output code for all languages in theory

[fading elm]

I only need to implement RCU again and then I should be ready to switch over to the new scheduler.

[fading elm]

I'm going to do a plain translation of the C RCU into Rust and then it's time for the part that's a lot of work (switching everything outside to use new sched)

[fading elm]

Back at this again, currently doing actually switching over to sched 3.0. There's a bunch of stuff that couldn't really be prepared but just have to be done all at once.

Right now that means exception handling and the process of bringing up APs.

[fading elm]

Time for fallible instructions 2: electric boogaloo.

(My usercopy works by detecting page faults)

[abstract cypress]

yeah.. this is the sane way.

but what does usercopy have to do with the scheduler?

[fading elm]

Old usercopy depended on old interrupts and old interrupt depended on old scheduler for ugly reasons you can see above.

BTW @solid bronze I've reached a "milestone": my main.c needs to be nearly completely rewritten and most of the things it calls into are Rust anyway, so I'm going to be making that a main.rs

[solid bronze]

no lib.rs?

[fading elm]

(main.c was some misc init order stuff, not very long)

[fading elm]

Oh yeah it is

But same idea

[fading elm]

Also why the mallard duck lol

[solid bronze]

i like ducks

[fading elm]

lol

[solid bronze]

[fading elm]

Yep yep I'm actually at the point where it's no longer possible to have C interop in certain places. After this re-write, there'll be a new "kernel" subfolder that contains the scheduler, SMP, synchronization primitives and CPU-local data (all written in Rust ofc).

Re-writing my SMP almost verbatim into Rust because it needs to read and write the CPU-local data structs but those can't be represented in C.

[fading elm]

For some reason this isn't working...

I checked, and it does appear between the request delimiters

So I'm not sure why exactly Limine doesn't see these.

[solid bronze]

which limine are you using

or rather, which base revision

[fading elm]

3

Setting it to revision 3 in there fixes it.

Which makes me wonder why my old C code for this had rev 2 but didn't explode

[solid bronze]

btw the request is just called mp_request now

not smp

[fading elm]

My headers are old

Haven't bothered updating them

[lusty vale]

LLVM opts?

[fading elm]

It was revision number

[short grail]

why even use the request delimiters?

[fading elm]

You either have to use those, or put all requests in a specifically named section.

I chose the latter.

[short grail]

you have to do neither

[fading elm]

Pretty sure you do in the latest revision

[short grail]

nope

limine scans the entire binary for requests, the delimiters are used as an optimization

[fading elm]

Well then that's just nice, isn't it?

[short grail]

it will only consider requests between the delimiters, and stop scanning upon reaching the end delimiter

i mean, maybe for some people but i simply find them unnecessary :p

it still has to go through the entire binary to find the start delimiter, so the performance benefit isn't that big to me

and actually, the requests section support was deprecated and removed in base revision 1

but i was just curious why people do it, i guess because it's how limine templates are written

[scarlet bison]

if you put the section at the start of the binary then it doesn't have to scan the entire binary tho

and the deprecated part is the usage of a specifically named section, the request delimiters feature isn't deprecated

[short grail]

yeah i said the requests section support was deprecated

and that's a good point about putting the requests section at the start of the binary, but personally i still don't care about the little bit of performance you can gain from doing that

[fading elm]

DPC time

[fading elm]

So turns out it was not DPC time because those aren't actually required to do preemption.

I will support DPCs in the future probably but I just don't need them immediately.

I've re-written the RISC-V timekeeping code into Rust and soon I'll have preemption up again.

At that point all that's left is load balancing, which I will probably do with work stealing once again.

(I.e. if a core goes idle then it'll try to steal a runnable thread from whichever CPU is under the heaviest load)

[fading elm]

Preemption works sometimes and other times it'll just blow whatever thread was running to smithereens.

It looks like the register file is corrupting.

The last thing that happens is the user process calling syscall_proc_fork, then a preemption happens somewhere within the fork implementation, even though it switches right back to the same thread, and the result is corrupt register.

The stack is intact (ish?) because I can still backtrace and the backtrace makes sense.

That said the backtrace does break at some point so maybe some interrupt accidentally reuses the top of the stack there instead of continuing at the bottom?

Yeah that was it, forgot to add this to the exception handler because of course preemption happens from interrupts so we should probably tell it to use the correct stack for interrupts.

That leaves only AP bringup before I can finally merge scheduler 3.0 and get back to building userspace!

[fading elm]

Shit

Bringing up APs has brought to my attention that my current RCU is broken

As in the CPU that's starting up won't properly sync with the already running ones, locking it up, and thereby the boot process as the BSP waits for it to come online.

[fading elm]

Current activity: BadgerOS with this playlist in the background: https://www.youtube.com/playlist?list=PLrjpwekK1wIAhdAOTmRyAryvpm45MutRu

[fading elm]

RCU is fixed but now syscall_proc_exec sometimes hangs

Due to some kind of problem in copy_to_user in create_entry_stack

[fading elm]

Yeah I wasn't updating the PC properly so it would just retry the fallible_store_u8 over and over

[fading elm]

That was another small bug: Was decrementing the SP after writing to it instead of before in create_entry_stack

So it would just fail depending on stack alignment

Now it's time for 1. reaping dead threads and 2. work stealing as a simple form of load balancing.

BTW: Thanks to @flint shell for helping me fix RCU, they made the suggestion that ended up fixing my problem

[fading elm]

Almost have work stealing finished but now it randomly jumps to PC=0

I'm guessing because some of the thread state didn't migrate CPUs properly? idk

[solid bronze]

@fading elm

are you doing modules at any point?

[fading elm]

I currently have statically compiled modules but I will do dynamic ones in the future.

[solid bronze]

nice

eh what gives I'll stick to rust

i do have a C project tho

libudev-portable

[fading elm]

Not 100% sure on how I'll do dynamic modules but right now I'm thinking of making them depend on my kernel, both as dynamic objects, and the kernel will dynamically link them in at runtime.

[solid bronze]

that's what i'm doing

[fading elm]

This would allow at least dynamic modules to be built. As to changing the crate type to static and linking it into the kernel statically, I would need an external build system.

Around cargo

[solid bronze]

comes with the neat benefit of getting free dep tracking

[fading elm]

I don't think there's a way I can with the exact same sources make it a compile time option to choose between module (dynamic) and module (static)

[solid bronze]

that's one thing where i got stuck too

and this is where i'd use meson

[fading elm]

Can a Rust staticlib depend on a dylib?

If it can then I think I know how I'll link them to the kernel?

[solid bronze]

wdym

i cant follow

[fading elm]

The kernel needs to be a dylib for the purpose of dynamic modules, period.

[solid bronze]

with cargo there is no way to do that, yes

[fading elm]

But if a staticlib can depend on a dylib, then if you choose static for some module, it will build as a staticlib and be added to the linked command line.

You would, of course, need more than just cargo to achieve this.

[solid bronze]

i've been exactly where you are rn

because you only need to link it, the kernel doesn't actually depend on anything besides the "module" info

[fading elm]

But I was going to do that because I'm not satisfied with Rust's #[cfg(feature = "...")] for configuring the kernel anyway.

[solid bronze]

same

just rust problems ™

[fading elm]

At which point the only remaining question is which wrapper would give me the nicest config system

CMake has one in theory but it's very bad

[solid bronze]

meson has a options thing

and you can combine it with Kconfig guis

[coral estuary]

if only cargo supported -rdynamic

[solid bronze]

😔

i'm trying to figure out how to build-std without cargo

[fading elm]

Linux apparently passes std'd lib.rs directly to rustc

[solid bronze]

wdym

do they have their own?

[fading elm]

No, official rust std

[solid bronze]

or do they clone the rust tree?

can you drop the link?

[fading elm]

no because I heard this from someone else who didn't send a link

[solid bronze]

mfw

[coral estuary]

https://elixir.bootlin.com/linux/v6.18.6/source/rust/Makefile#L514
https://elixir.bootlin.com/linux/v6.18.6/source/rust/Makefile#L55

[solid bronze]

monkuous to the rescue again

oh damn

that seems simple enough lol

[fading elm]

Excuse me Rust, I need this please

[solid bronze]

nope

you need to preserve it

just steal my code smh

[fading elm]

This is the annoyance

[solid bronze]

/// Wrapper for the `cpuid` instruction.
#[inline]
pub fn cpuid(leaf: u32, sub_leaf: u32) -> CpuIdResult {
    let eax;
    let ebx;
    let ecx;
    let edx;

    // LLVM sometimes reserves `ebx` for its internal use, we so we need to use
    // a scratch register for it instead.
    unsafe {
        asm!(
            "mov {0:r}, rbx",
            "cpuid",
            "xchg {0:r}, rbx",
            out(reg) ebx,
            inout("eax") leaf => eax,
            inout("ecx") sub_leaf => ecx,
            out("edx") edx,
            options(nostack, preserves_flags),
        );
    }

    CpuIdResult { eax, ebx, ecx, edx }
}

[fading elm]

This is a burh moment

[abstract cypress]

huh, why does it reserve rbx?

[fading elm]

Ask LLVM

[solid bronze]

i think the diagnostic tells you

[fading elm]

Well

It just says "LLVM claimed it" basically

[fading elm]

Ah shit

My PIT-based TSC calibration code is fucked

I set QEMU to make it 1.0GHz but it thinks it's about 789'946'458'571'974 Hz (790THz)

    unsafe {
        ioport::outb(0x43, 0x32);
        ioport::outb(0x40, 0xff);
        ioport::outb(0x40, 0xff);
    }
    let before_tick = time_ticks();
    let mut after_tick = before_tick;

    let mut pit_tick = 0u64;
    while pit_tick < 100000 {
        after_tick = time_ticks();
        let mut tmp: u16;

        unsafe {
            ioport::outb(0x43, 0);
            tmp = ioport::inb(0x40) as u16;
            tmp |= (ioport::inb(0x40) as u16) << 8;
            tmp = !tmp;
        }

        if tmp < pit_tick as u16 {
            pit_tick += 0x10000;
        }
        pit_tick = (pit_tick & !0xffff) | tmp as u64;
    }
    let elapsed_nanos = pit_tick * 838095344 / 2000000000;
    let elapsed_ticks = after_tick - before_tick;

    logkf!(LogLevel::Debug, "PIT ticks elapsed:   {}", pit_tick);
    logkf!(LogLevel::Debug, "Nanoseconds elapsed: {}", elapsed_nanos);
    logkf!(LogLevel::Debug, "TSC ticks elapsed:   {}", elapsed_ticks);

    let ticks_per_sec = (elapsed_ticks as u128 * 1_0000_0000_000 / elapsed_nanos as u128) as u64;
    logkf!(LogLevel::Debug, "TSC frequency:       {} Hz", ticks_per_sec)

Here's what I do right now ^

[00000.000] DEBUG PIT ticks elapsed:   100000
[00000.000] DEBUG Nanoseconds elapsed: 41904
[00000.000] DEBUG TSC ticks elapsed:   331019164
[00000.000] DEBUG TSC frequency:       789946458571974 Hz

And the result ^

@solid bronze How do you establish the frequency of the TSC?

Because that's all I'm really interested in

[solid bronze]

depends on the cpu

[fading elm]

I should add that 32-bit CPUs and CPUs old enough not to have invariant TSC are unsupported on BadgerOS

[solid bronze]

if there is info in cpuid i read that

if there isn't i calibrate using hpet

[fading elm]

Yeah and it turns out the CPUID info depends on the core clock frequency, which I have yet to successfully find the source of

[solid bronze]

it contains that too

not on all cpus*

[fading elm]

Kinda wondering why my previous impl of this "worked" as well as it did...

[solid bronze]

i made some notes for this

[fading elm]

hmm

I will look into the rest of this tomorrow

Because having a timer is kinda useful

<- Getting x86_64 to run after sched3 re-write

I also tested on HW recently and neither RISC-V nor x86_64 booted so... Gotta fix that too.

[fading elm]

It still thinks my 1GHz TSC is 400GHz...

Calibration is certainly closer but not as it should be yet.

[fading elm]

I timed an exaggerated calibration of 10 million PIT ticks using a stopwatch and the difference between the elapsed nanos according to PIT and my stopwatch was well within the margin of error.

In other words: the PIT isn't the problem.

My host CPU is 3.949GHz and the result from calibration appears to be 100 times that for some reason

So apparently QEMU did not get the TSC down to 1.0GHz for one

    let ticks_per_sec = (elapsed_ticks as u128 * 1_0000_0000_000 / elapsed_nanos as u128) as u64;

What's wrong with this line, I wonder

@solid bronze look at what I was debugging all day ^

[solid bronze]

wth

1_000_000_000?

[fading elm]

some extra zeroes by accident

[fading elm]

shitshitshitshitshit

I don't think the random page fault was because of work stealing because now it's happening even without work stealing

Aaaaand threads are going missing again

FFS

I'm not good at schedulers holy shit man

[fading elm]

Meanwhile, test on real HW:

LinuxBot badgeros-kernel$ picocom -b 115200 /dev/ttyUSB1
picocom v3.1

port is        : /dev/ttyUSB1
flowcontrol    : none
baudrate is    : 115200
parity is      : none
databits are   : 8
stopbits are   : 1
escape is      : C-a
local echo is  : no
WARNING: Could not meaningfully match the boot device handle with a volume.
         Using the first volume containing a Limine configuration!
PANIC: No volume contained a Limine configuration file
Stacktrace:
  [0xfe693daa] <panic+0x98>
  [0xfe68cf98] <uefi_entry+0x298>
End of trace. System halted.
Terminating...
Thanks for using picocom
LinuxBot badgeros-kernel$

[fading elm]

Something's broken with the backtracing still but at least updating Limine has resolved this particular issue.

[fading elm]

Let's fix the backtrace so debugging on HW is slightly easier

[fading elm]

Ok cool, so it "boots" on hardware again (but I don't have drivers for SD/MMC so it can't mount the filesystem)

[fading elm]

I think I'm gonna merge the uapi branch. Or try to, at least.

Ah shit I have an old branch where I was working on the ACPI stuff to get x86 to actually boot

It's probably gonna end up easier to cherry-pick diffs to apply from scratch than try to merge it.

[fading elm]

Merging uapi was surprisingly simple. Now, let's make an image that contains the bare mimumum to boot.

[fading elm]

Not me needing to recompile GCC again

[short grail]

it takes 5 minutes it's not that bad

[fading elm]

It gets boring pretty quickly if you have to recompile often

[fading elm]

Thanks to some help from @echo valley, I've got a little distro built. However, the kernel still crashes, so there is some work to do.

This is almost certainly not the distro's fault, though.

It was a stack overflow. I'm going to need to check the recursion up there.

bruh

I'm reading an invalid PTE to copy into the user process

cringe

[fading elm]

How fucked up this page table is, is quite surprizing

This must be some kind free happening too early and something else allocating the same block as the page table here for something else?

I can check, actually, I'll look in the PFBDN using the debugger what the PageUsage is

The memory is marked as an in-use page-table.

So something else is corrupting it.

since the entire thing is affected, I'll try setting a hardware watchpoint on the first address within

[fading elm]

LMAO my elf loader breaks after mapping a grand total of two pages

After that every page repeats the second ever mapped

This is Ext2's fault

Or rather my driver for it

Looks like the indirect blocks are not working as they should

well

This is why you don't run your OS on something with a drive that has important data

Imagine having an FS bug like this one but you're writing instead of just reading

Instant data loss

[fading elm]

I'm just reimplementing my E2VNode::get_block_unlocked (the function which finds and/or allocates an inode's data block on disk)

[fading elm]

Much closer but init still page faults

[fading elm]

I think I have fixed program loading but on the other hand it page faults still

Looks like an mlibc variable that says whether TLS is available early is set to true while it should have been statically initialized to false

[fading elm]

Time for another try at the program loading

This time I have completely rewritten the function that loads a specific program header

If that works, I'll have mlibc running on BadgerOS.

[fading elm]

Ok cool, let me quickly create device nodes (e.g. ⁨/dev/null⁩, ⁨/dev/tty0⁩, ⁨/dev/sata0⁩)

[fading elm]

That's done! Very cool. Now, let me make TTY devices a real thing and make the kernel switch over to those for logging after early init. The init process will be started with a handle to the first found TTY device for FDs 0, 1 and 2.

[fading elm]

For some reason when the kernel tries to open ⁨⁨/dev/tty0⁩⁩ to give it to init, the file I get is actually ⁨⁨/dev/null⁩⁩?

Damnit, I didn't want to be dealing with complicated FS bugs right now...

[fading elm]

I am going to have an anneurism

Ok so

turns out that

I did not recompile init

And the old version explicitly opened /dev/null

[fading elm]

Okay so my page tables are broken again

The "Hello, mlibc!" demo no longer works.

[fading elm]

Yeah so turns out I forgot to actually put argv and envp on the stack but reported an argc of 1

I'm gonna need dynamic linking ASAP to get the image size under control

Shouldn't be too bad for the kernel

[solid bronze]

all you have to do is load an interpreter if one is requested

and load its base onto the stack

[fading elm]

i know

[fading elm]

Loading ⁨/usr/bin/echo⁩ fails on reading in the first program header

[solid bronze]

:(

[fading elm]

It had to do with my pages tabling again

I changed how I override the page table temporarily during ELF loading, lettuce try anew.

Let's go

This is ⁨/usr/bin/echo⁩

I should also test running for an extended period of time after a ⁨fork⁩ happened.

[fading elm]

Yeah okay. I am going to be porting the PMM, VMM and heap I'm making in Positron to BadgerOS eventually. I really like them.

They share a lot of primitives, most of the things that need adjusting are just where stuff is imported from.

[fading elm]

Do I write a new scheduler again in Positron and port it to BadgerOS later, or do I fix BadgerOS' scheduler when multiple cores are up and port it to Positron... 🤔

I think I'll try fixing BadgerOS sched first. Maybe I can get work stealing in there too.

??? BadgerOS' scheduler doesn't crash with SMP now?

I'll try adding back work stealing too then.

Seems like just the work stealing is borked.

[fading elm]

Ah shit

It seems as though the stack is being written to whilst the context switch is in process.

Which is a catastrophic bug either in the spinlock impl or in the thread queue impl.

Namely, it is overwritten from a perfectly logical return address to 0.

In the middle of the context switching into it.

Perhaps an interrupt is using that stack when it should really have been using a different one?

I guess I'll just see what happens with preemption disabled then.

It happens with preemption disabled too...

I need to rule out a thread being in the queue for multiple CPUs at once.

[fading elm]

OH FUCK

If a thread yields or gets preempted, there is a window where it is in the queue but the scheduler is still using its stack!

So it's not actually the sched it came from corrupting the one it went to, but the one it went to corrupting the one it came from!

My queue is currently a linked list and "choosing" a thread just pops the first one off the list, I'm not sure how to fix this...

I guess I'll just keep the lock for the queue during the entire ⁨Scheduler::sched_yield⁩ function

That answers the Positron sched question too: I will polish up that of BadgerOS and port it to Positron.

[fading elm]

Actually no, I can't do that either

Because now the owner of the queue spinlock can spontaneously migrate to a CPU it's not for

[fading elm]

OK that's fixed. Now I'm gonna have to fix whatever it is that causes dynamically linked stuff to just segfault

[fading elm]

UGH

Program loader must be borked for interpreted ELFs

Not a single syscall before segfault

Userspace wants to access address -2 for some reason

[fading elm]

ffs

Looks like I lot a lot of changes I made in the kernel before I was working on fixing the scheduler

This will almost certainly affect the program loading

And I do not want to deal with this BS right now so I'm just gonna not

[fading elm]

Yeah so since Zinnia as a microkernel is dead I'm just going to continue fixing BadgerOS now.

Just fixed a UaF (using a Limine response data after having reclaimed bootloader memory)

Process 1 is segfaulting, which is almost certainly the kernel's fault.

I'm going to see if I can band-aid fix it but either way I'm going to be converting into UVM-style virtual memory very soon.

[solid bronze]

👍

[fading elm]

Bug seems NOT to be my VMM but the entry stack layout instead... argc == 1 but argv[0] == NULL

argv[0] should be pointing to a string /sbin/init

[fading elm]

I just nuked create_entry_stack and made a new one. That part works now.

/usr/bin/echo appears to start, but it doesn't make any FS syscalls, so something is going wrong there.

LMAO didn't install coreutils so this is the fork succeeding but the exec failing

Anyway let's fix up dynamic ELFs

oof yeah the code for dynamic ELF interpreters got lost too

Lets find out what the auxvs I need are again...

[fading elm]

Ok now the dependency is on sys_vm_protect

I will do a dirty and map everything RWX and stub protect, because I will be rewriting VMM later

[fading elm]

Now I'm working on integrating the kernel into the build system of the distro

I've already removed all the old garbage from when I thought it was a good idea to have the kernel and distro be in the same repo

[fading elm]

Time to first rebuild coreutils and then impl more random syscalls

[spare nymph]

which coreutils are you using

[fading elm]

GNU

[spare nymph]

fair

[fading elm]

shit

I built userland as RV64GC

And now I'm being held to that promise (float op was executed by printf)

So uh time to do that I guess

Bit of a tangent

[solid bronze]

problem?

[fading elm]

I'm just on a tangent implementing float support in the kernel

In fact I should now be able to test said float support.

[fading elm]

FUCK my pid_t has to be int. Now I can't just rely on it never overflowing.

[spare nymph]

why cant you do long??

[fading elm]

Because of pgkill

The definition of it uses int, forcing pid_t = int

[spare nymph]

is it in some C library that you have to use

idk why cant you redefine it

[fading elm]

That's a POSIX function

[spare nymph]

oh

[fading elm]

Yeah

[spare nymph]

https://tenor.com/view/skeleton-gif-26826812

[fading elm]

Shit's fucked yo

[spare nymph]

this is outright evil

[solid bronze]

what is that

i don't see the function

do you mean killpg?

[solid bronze]

because then this is false

[fading elm]

Oh yeah it's killpg

[solid bronze]

where does it say int?

[fading elm]

int killpg(int pgrp, int sig); <- definition in both mlibc and glibc

[solid bronze]

that's incorrect

posix says

int killpg(pid_t pgrp, int sig);

[fading elm]

well

Both define it as such

[solid bronze]

doesn't mean it's correct

[fading elm]

Go make a PR

idk

[solid bronze]

yeah

[fading elm]

I'm going to impl more shit bash wants from me now

[solid bronze]

musl declares killpg like it's supposed to be

lol mlibc uses it correctly

but has incorrect arguments in the header

[fading elm]

lololol

[solid bronze]

mlibc#1707

[solid bronze]

@fading elm merged

[fading elm]

that was fast

I'm debating whether I should use the Linux ucontext_t structs or make up my own for signals

[fading elm]

I'll make my own because I don't feel like bothering to learn the Linux one's semantics

[fading elm]

NVM I'm using the Linux structs

[mental pagoda]

Lol

[fading elm]

No need to reinvent this wheel

[mental pagoda]

Yeah fair

[fading elm]

I'm going to make some excuse for a "shell" that needs minimal features and go around checking coreutils using it

Then try bash again

[mental pagoda]

Its just funny to me the pattern on this server of saying "I won't do x, I'll do my own thing" and then we get "oh no, I'll just do x".

[mental pagoda]

What trouble did you have with bash?

[fading elm]

Jumps to a nullptr and generally needs a lot more sysdeps working than I have RN

e.g. concept of working dir is missing

signals are my current WIP

All IDs are a stub

[mental pagoda]

Oh ok, yeah those sound useful for bash

[fading elm]

Bash 100% working is the next big milestone I'm slowly working towards

[mental pagoda]

I also need to implement signals (really the whole process management subsystem 😬)

[fading elm]

I'm also going to rip out my VMM entirely for a third time and replace it with UVM-style

[mental pagoda]

Only the third time? 😅

[fading elm]

lolololololol

[mental pagoda]

That sounds good though, uvm is really nice.

[fading elm]

I never bothered researching VMM for some reason

Then I look at the UVM paper once and I'm like "this"

[mental pagoda]

Its nice, I got interested in it after seeing fadanoid monologuing about it a while ago. There's some patterns that I think I'll find other uses for.

[fading elm]

Also

I'm going to have to port to x86_64 because I can no longer put up with how slow TCG makes shit

[fading elm]

Currently implementing waitpid

[fading elm]

Also doing SIGSTOP and SIGCONT while I'm at it

[fading elm]

Async signals and waitpid work now

[fading elm]

Oh BTW bash no longer segfaults

[spare nymph]

five hundred segfaults

what was the cause for it

best i can think of is you not mapping enough ram

[fading elm]

Probably bash assuming some random lib call would never fail except secretly it actually did fail

[spare nymph]

https://cdn.discordapp.com/attachments/1423054245430694053/1423055104952504450/cat_bomb.gif

howd you fix it

[fading elm]

Implementing random sysdeps

[fading elm]

[00000.000] INFO  ==============================
[00000.000] INFO  BadgerOS 0.1.0
[00000.000] INFO  bad3f7c-dirty 2026-03-18 04:00:52 CET
[00000.000] INFO  ==============================

New feature

Also added uname

[fading elm]

Bash prompt achieved!

[fading elm]

Really wishing my mutexes were recursive right now...

[fading elm]

I've added some more sysdeps that fcntl will use behind the scenes but it looks like I have more syscall debugging to do

Because bash will not show the prompt for some reason if F_GETFD is implemetned

It still runs just no prompt showing

[fading elm]

I'm looking for some input on how to do my device abstractions because I'm not satisfied with the current situation

Basically in the current situation my kernel represents devices in a tree analogous to the DTB (and indeed DTB information is copied into it on RISC-V). The problem comes when I try adding drivers to said devices. The devices are a union of all possible "device classes" recognised by the kernel. When a driver is added to a device, its class gets set accordingly, and the device is then usable through an interface specific to that class.

1. If a device has its driver removed, then another one added, users may not notice if they weren't using it in the mean time (it will retain its current class and handle)
2. I can't fit the current structure into Rust's memory model nor type system cleanly

I would like to see if it's possible to do use dynamic dispatch, e.g. Arc<dyn BlockDevice>. (A C++ analogy would be std::shared_ptr<block_device> where block_device is an interface)

I thought what I'd do is have the device tree contain only the abstract information about devices, and which drivers they currently have, then when you go to search for devices you get given a handle to its driver (which in turn refers to the info in the tree internally)

However, in this proposed model I would be unable to put a higher level interface above raw access to the driver (e.g. line discipline around a TTY, page caches above a block device)

[fading elm]

I've decided to work on UVM before the new device rewrite so that it doesn't need to be updated after the fact and can immediately use the new features I want to give it.

First off, I have implemented a radix tree that will be used to build the page caches on memory objects.

I can now continue writing out the UVM structs.

[fading elm]

Looks like I unfortunately can't use the class-like structure from UVM's memory objects directly in Rust

I think I will instead make the interface of one a trait, and create one or more helper structs for implementers to use.

[fading elm]

After some compromises on not being able to do inheritance with pure virtual function in Rust properly, the page cache is coming along nicely.

[solid bronze]

how did you end up designing it

[fading elm]

Passing the pager ops as a parameter to every PageCache function

[fading elm]

@fickle sail I post about my own OS here from time to time

[fading elm]

I've come to the conclusion that to check whether we had already mapped a certain page, we just ask the pmap.

From that information the page fault handler determines whether to get pages from the memory object, CoW them into an anon, or simply flush the local TLB.

[flint shell]

Having a "is page present" function sounds like a bad idea

That easily runs into TOCTOU

[fading elm]

It should be safe because the page tables are managed exclusively through the UVM abstractions, and only pages that virtually lie within an entry (of uvm_map, which is guarded by spinlock) may be modified by that entry.

Basically I don't want to run into having to keep an entire second radix tree just to keep a copy of what is already in the page cache for most memory objects anyway.

Unless you can propose some other solution that can avoid unnecessary overhead like this while still allowing the memory object to manage its own page cache.

You could theoretically run into my method being a problem if you had e.g. PPC paging where you'd need to evict some pmap entries to map others, but RISC-V, x86 and ARM don't do this so I don't really care.

[fading elm]

I'm almost done with the PoC version, all I need to do now is make the actual implementations of map, unmap, protect and fault in VmSpace

[flint shell]

you can design it in such a way that the PTE is transient

and can be thrown away / regenerated from the memory object

[fading elm]

And then what? Just add a another datastructure to the whole ordeal solely to keep track of the physical addresses of pages borrowed from the memory objects?

[flint shell]

It's a bit hard to tell w/o knowing how your VMM works but most of the time you need a separate data structure anyway

to support shared memory and page caches for files

[fading elm]

Well I have structs Anon, AnonMap and MapEntry for a reason

They correspond to the UVM structs anon, amap and whatever the entry for map was named

The only thing "missing" here is the information of whether a certain page at an offset in some memory object is or is not mapped at the moment

It's also on GH here so you can see what I mean: https://github.com/badgeteam/BadgerOS-Kernel/blob/vmm/src/mem/vmm

Particularly map.rs there is relevant to this discussion, it is meant to be the abstract representation of mappings that are then translated into actual page tables (which are stored in PhysMap from pmap.rs)

[flint shell]

Okay but then you have all the data structures already, right? On read-only PF you get the anonymous page if it exists. On write PF you get or allocate (and copy) the anonymous page.

[fading elm]

yeah but

the MemObject will still need to have its own internal refcounts

And the map should correctly tell it when it uses and frees the pages from it

There isn't something right now that stores this information, and I was planning to imply it from the page tables

[flint shell]

Obviously you need to increment refcount numbers on map and decrement on unmap, yeah

but this should probably be handled by your pmap code

[fading elm]

For all the pages in the mapping?

[flint shell]

No, for each individual page

[fading elm]

Not th physical pages I mean, but the pages within the MemObject

Basically a page dealt out by a MemObject needs to remain in its page cache at least for as long as it is physically mapped

Ideally the pmap doesn't deal with ownership semantics at all and could be treated as a cache written by software and read from by hardware

[flint shell]

Well, that's not possible, at least not combined with "I want no auxiliary data structure to remember the pages that are currently present"

since as you said, you somehow need to remember which pages have their refcounts bumped and which haven't

[fading elm]

Yeah

So the compromise was going to be checking the pmap; if it's in the pmap but not in a matching amap then it belongs to the memory object

But if I understand you, this is a bad idea?

[flint shell]

Why isn't the refcount for page cache pages attached to the physical page?

If it was, the pmap could just decrement it when unmapping

[fading elm]

I thought those were meant to track when a page was to be freed in terms of PMM

I could change these semantics to make the refcount used the way you suggest

I suppose this way of using that refcount makes more sense

[flint shell]

I think of it like this: while the page is owned by the memory object, the memory object decides for what purpose the refcount in the page struct is used

[fading elm]

And then the memory object checks that refcount to determine whether it must stay cached

okay

yeah

the refcount is for arbitrary use by the owner, and for memory objects that refcount means how many times it is mapped

This does mean some refactoring in my PMM now. I must change how it sees a page as being free or not.

[fading elm]

Accidentally left my VMM tree at home so I'm currently working on my new syscall marshalling system. I'll have a little python script generate me everything needed, from C wrappers and header files to Rust marshalling code that calls a safe implementation.

There's only two syscalls that'll need exceptions on the Rust side: syscall::proc::fork and syscall::proc::sigret

[fading elm]

Ok back to my PC, let's do the actually productive VMM work now

First, I'll try to adapt the entire kernel to the new (slightly different but not much) VMM API, then I'll implement the last couple functions needed for an initial test.

[mental pagoda]

Nice, gl with the initial test.

[fading elm]

Discovered a couple things for which I still need to think up the new API