#The Flopperating System (tfos) - x86 32 bit Hobby OS
1 messages · Page 3 of 1
I feel like I’m losing my mind
I’ll send you a paste bin in a bit
I have that assembly function
I’m just
Idk
Bro :(
Now it’s just stuck crashing
Like in a loop
Not even —no-shutdown does anything
and it keeps pointing to garbage in my kernel just random points
lmk when u can vc lol
Yeah shouldn’t be too long
okay well i at least got that crashing issue fixed
@crimson plover look at this
thread_t* thread_create(int (*fn)(void*), void* arg, uint32_t* stack) {
if (!stack) {
stack = kmalloc(STACK_SIZE);
stack = stack + (STACK_SIZE / sizeof(uint32_t));
}
thread_t* t = thread_alloc();
*--stack = 0x200;
*--stack = 0;
*--stack = 0;
*--stack = 0;
*--stack = 0;
*--stack = (uint32_t)arg;
*--stack = (uint32_t)fn;
*--stack = (uint32_t)_thread_entry;
t->esp = (uint32_t)stack;
thread_enqueue(t);
return t;
}
idk what to make of it
you should really check those allocations for null
The allocations work but I’ll add some asserts
ngl I just panic inside of my pmm if I run out of mem
lol

future me will hate me
btw @bold grove i had some similar crash, it's better if you do that in asm
the problem was setting the stack value in c++
this is how i ended up doing it
Yeah my issue was setting the stack value in C. I did it in assembly and it works now. This was my issue and you just saved my life
i just fixed my scheduler too XD
i was setting the next entry of a linked list to itself
and it took me 2 days to find why
lol
btw, how does a thread know when it should be stopped?
like, do you have to manually set it to terminated state?
ah f
syscall?
Nah not for now but I’ll make it one
this is how i do it
i could easily not even set it to TERMINATED
bc i just call schedule again
Ahhh I see
Well
I can get into the initialized state of the scheduler but
Idk
I’ll send a paste bin later
Maybe you could
Help me out
k
Damn boys
I haven’t
Updated this in a while
I’ve been working on the sch*^#ler
Schedulers are cancer. Not really but , you know
My actual algorithm is fine
I use a doubly linked list
For thread queues
And I can queue threads
But
Context switching is just fucky
hmm
Common Scheduler:
https://git.voxelgames.eu/diamantino/horizonos/src/kernel/kernel-common/src/threading/Scheduler.cpp
https://git.voxelgames.eu/diamantino/horizonos/src/kernel/kernel-common/src/threading/Scheduler.hpp
X86_64 Arch Scheduler:
https://git.voxelgames.eu/diamantino/horizonos/src/kernel/kernel-x86_64/src/threading/SchedulerX86.cpp
https://git.voxelgames.eu/diamantino/horizonos/src/kernel/kernel-x86_64/src/threading/SchedulerX86.hpp
X86_64 Ctx Switch and stack init: https://git.voxelgames.eu/diamantino/horizonos/src/kernel/kernel-x86_64/src/threading/SchedulerX86.s
if you wanna look at mine
Yeah let’s talk later but
I’m like pretty close to get Rid of the issue
gg
From assembly
lol
It like
Fucked up somefhing
Giving me a bullshit ESP
and eventually my instruction pointer was literally nowehere
The instruction pointer was 00FF0077 then a bunch of weird characters
Qemu info registers was
Just
00000000
Yeah
Now
I can call _thread_entry
And yield back to sched
But
Once I switch tasks like
Eventually the esp gets fucked yo again
Like the offset is fucked up like
I’m pushing the correct offset then I check it in C and it’s wrong
are you saving it correctly tho?
Yes
I followed literally
The intel manual
And I’m saving it correctly
I think it’s coming down to an issue with my allocator which is funny
Okay not my allocator but
The slab is just
Being weird
why slab
I lowkey might just
I fucked off offset somewhere and it’s causing issues
slab might be one of the worst ones for threads
are you also sharing kernel pages?
No
you should
Okay I’ll just map to kernel pages instead
so every process gets it's own context
Yeah
Yeah no I’m not doing that
Multiple threads can make up a process and they share the same address space
I gotta update this damn repo fuck
Sorry guys I’ve just been working on memorizing this stupid page of tchaikovsky
wtf is this mess
Classical music.
pain
It’s what I’m doing for my senior violin recital
lol, there is italian in there XD
that is so complicated tho
i only ever seen the simple ones XD
Yeah it’s a common “hard” violin piece that’s played with an orchestra
I would send a recording but I’m a butt ugly Bosnian so no thank you
lol, dw
I feel your pain
memorising music sucks
Yeah I have 2 weeks to memorize 11 pages
Not even 3 in
So the scheduler is done
well
now that this scheduler is done
i think i can restrcuture the source code
i might just name it
something else
idk
ayyy good job
okay im fixing some stuff
in the scheduler
some issies
arose
the function pointer inside of my thread_t was being pushed with the wrong offset
and i couldnt figure out where
so
but im fixing it
Aight
We are good
Time to restructure the source code
It’s gonna be
arch/
drivers/
fs/
mm/
sched/
userspace/
kernel/
flanterm/
multiboot/
limine/
Something actually reasonably
So imma have ia32
Which is what I’m doing now
But I’ll do arch separation
So I’ll get that limine c template
and make it use that for booting
So I can get long mode
And limine supports multiboot protocol so
Yeah imma do that tonight okay ty for reading :)
is said scheduler multicore
how will you do multicore scheduling
ill just create a queue for each cpu
how will you balance and sync it globally
round robin
that doesnt mean anything
you only have one global queue of tasks how does each core know what to schedule lol
no im gonna implement smp
wat
ok i think u might not be getting what I mean lol
so you have some tasks you wanna run
and you need to figure out how to assign each core to a given set of tasks
how will you do that
i'm gonna make core-local queues with periodic global resyncs and rebalancing
so i have a global circular doubly linked list and i take portions of that list for each task
oh wait cant i do priority scheduling
what does this mean
in like a round robin
why would this change anything tho
you cant run the same task on multiple cores at the same time
i know
💥
so why does it matter who is running the high priority tasks
just give them longer timeslices
💥
if (task_is_high_priority && timeslice_isnt_up) dont_schedule_next();
💥
i see what you mean
cool
hi guysss
im redoing the thread structsss
#include <stdint.h>
#include <stddef.h>
#include <stdbool.h>
#include "sched.h"
#include "thread.h"
#include "../mem/alloc.h"
#include "../apps/echo.h"
#include "../mem/pmm.h"
#include "../mem/vmm.h"
#include "../lib/logging.h"
#include "../lib/str.h"
#include "../lib/assert.h"
#include "../mem/utils.h"
typedef void (*thread_fn) (void *arg);
typedef enum {
THREAD_READY,
THREAD_RUNNING,
THREAD_EXITED,
THREAD_DEAD
} thread_state_t;
typedef struct thread_list_node {
struct thread_list_node* next;
struct list_node* prev;
} thread_list_node_t;
typedef struct {
thread_list_node_t* head;
thread_list_node_t* tail;
size_t count;
thread_list_node_t node;
} thread_list_t;
typedef enum {
USER,
KERNEL
} thread_mode_t;
typedef struct thread {
thread_fn fn;
uint32_t thread_id;
thread_state_t state;
thread_list_node_t node;
thread_mode_t mode;
uintptr_t esp;
uintptr_t base_esp;
} thread_t;
what do you think
and im just having threads point to C functions
which im assuming is fine
because i can enter whatever from a c function
once i get elf working
There’s like no difference between threads in ia32 and x64
when are we getting x64 support and testing on real hw ⁉️
Boots and every single feature works
is this the usb keyboard 😲
Yeah but most computers have ps2 emulation
i wonder how lenient qemu actually is with device drivers
Extremely
all 3 of my disk drivers might suck on real hw if disks love to fail
i have no fail state checks
The only one that would transfer well is the NVME
what about ahci and ide
Because it’s specification is solid and it’s not unreliable
do u know
IDE is a fucking mess
all the specifications are solid wdym
On my old kernel
is this your fault or ide's fault
Well yes, but Ide drives are more susceptible to failure
Idk but it worked completely fine in every emulator
so does that mean i need to re-attempt the command like a floppy if it fails
are you sure the disk was in ide emulation mode or whatever
Yes . I was able to get the signature but not read or write without corruption
was the disk in question highly problematic
512 Samsung evo ssd
Very new
And I know it works
I think the issue
Is that most hobbyist drivers
Don’t really account for
Cancer
That happens on real hardware
are you sure that these fancy new sata drives work fine with ide emulation
i dont think these new guys play well with IDE controller support
Oh well
they would probably like ahci better (perhaps)
Yeah never got around to that
Idk tho
Because I was able to
Like
Read from sectors
ahci easy peasy you should get to it
And detect them fine
Yeah I will after I do arch separation very soon
And we will finally have
64 bit
finally not just ia32
what was the logic behind just doing ia32 at first
are you rolling ur own bootloader
Nah multiboot 1
My dad made a hobby kernel in the 90s rhat was ia32 and I wanted to do what he did at first
is that not your own bootloader
huh what
so we dont exactly live in the 90s i think
A gnu boot specification
oh the magic number thing
I know but I wanted to make him proud and then further do 64 bit
interesting lore
Yeah it’s also why I did only vga text mode for a long time and vga graphics mode
Because my dad made games for his own OS
And he made this super sick spreadsheet application in vga text mode that is lowkey still really good
most sane 90's c programer
C programmers were on one 20 years ago
i wish my dad did shit like this he just made hard drives spin to sound like the doom soundtrack
Hiii welcome but yeah
My dad now does embedded systems at General Electric
does he make their engines too
No he makes the software that allows them to work
does flopper operating system have a thingy mabob like a roadmap
No. I’ve honestly been slacking and the source code is really messy
Honestly like
I’m very close to a new commit with a better structure
huge commits with 1 line messages
Absolutely
So
This next commit will come tonight but
The main deal right now is that I want to make syscalls
After I tidy up my filesystem
do you think my kernel is okayish in organization (shameless plug)
Not bad at all. Pretty good
menix is well organized (the legacy branch)
it is strangely organized
me when i kernel/thor/generic/thor-internal/arch-generic/cursor.hpp
u get used to it
We wait

Don’t worry guys im just depressed
Work on the os hasn’t started again but
It will soon I hope
implement the XHCI USB controller to be even more depressed
I’m still working on scheduling. I’ve gotten threads but now I need to actually get them in userspace
Although it wasn’t my goal
I’ll probably follow the posix api
For that
I plan to make all the software for it though so
Maybe I can do my own thing
i thought this was not a unix-like os
what 😭
void thread_enqueue(thread_t* thr)
Works and now I can create processes. Many threads can be attached to a process and share the same address space. I’m using a round robin and im going to implement SMP eventually
neat
My first process is going to be the reaper
Then im gonna implement syscalls
And then make an actual shell
The worst part right now is
fork() FOR FUCKS SAKE
why not take a break and work on something else
on-disk filesystems
block device drivers
usb
networking
audio cards
You’re right
I already got a lot done
I’ll push later
And then I’ll make a proper VFS
and then I’ll implement NVME
why not do those now not later
I like working on core stuff just not worrying about getting to userspace or anything
hell I am like 200 todo items away from starting to port anything
Bc I’m depressed and not next to my computer lol
When I get the urge to leave this bed after 45 hours I’ll do it
relatable
eat ur happy pills
Already did
I’ve ran out of like weed and coffee so
I’m kinda just on pause rn
I’ll never port anything ever because I’m not conforming to Unix stuff and im not gonna implement winapi lol
I parted ways with my todo list because it’s fucking massive
Todo: not make dogshit OS
okay
So
Working on VFS rn
difficulty: impossible
I’ve gotten basic operations
And proper vfs nodes and stuff
Now I can mount my temporary file system to /tmp
Now
I’m thinking
Either fat or ext2
I’ve implemented EXT2 in userspace before
So
Maybe I should do that but FAT isn’t that hard from what I’ve heard
My old ext2 thing for userspace was like 2000loc from 2018
So
Time to die looking through that
doesn't this mean you need tmpfs
Which I have
that's surprisingly small did you have everything
neat
All file operations and nodes and shit yeah
I made a JVM like environment
truncate, allocate, free, mkdir, rmdir, some basic integrity checks, all of it?
Yeqh and ref counting
do you honor ext2 flags like secure delete where you must zero all child blocks of a file before deleting it
Yeah it calloc() all of the blocks of the parent
wait so how did you keep it that small, did you only handle the first 11 direct blocks?
I have 2 different walk functions for blocks and dirents
what lol huh
that's not what secure delete does tho? calloc all blocks of a parent?
All of its child blocks
you probably shouldn't do that unless secure delete is specified tho
since like
slow
Yeah the interface was a command line and it had a flag
but cool that it works
—secure-del
oh wait I get it so you can just treat the /dev/device as your disk
Yes
but like a file or large stream
Exactly
oh that makes life easier
very cool that you got it all to work
It was painful
did you also fsck to see if fsck is happy about your impl
that was a little annoying because there were linux specific fields in the superblock for me
I just ignored em
fsck can yap all it wants
no but where is yours
Not posted because it’s ass and I hate the way it looks
it's not that bad i won't hate lol
I wrote some diabolical XHCI stuff earlier
would be hypocritical
It will be posted in about a week or so. The only issue is that I leak some memory when removing directories
Idk where
dang wow we got a shipping delay on an internet repository
Yes sir
Because I just don’t like it right now and it’s scattered into a bullshit api
And written in c++ which is its own issue
And it won’t even compile now after gcc updates
what did bro do
compiler behavior hacks is crazy
idc if it compiles lol I'm just curious to see how these are done purely in userspace
Well I’ll upload it soon but for now look at mishakovs
I’m also just not at my old computer so
And im too fucked up rn to go up the stairs
Also my girlfriend is clingy to me rn lol
She isn’t letting me get up bc she’s cuddling me
Sleepy girlfriend = terrifying
@small hound also for ext2 it’s extremely unnecessary to implement the Linux specific fields
It doesn’t affect posix compatibility
However , if you want to use Linux software eventually you might need to but idk
fsck was yelling at me
Like in an os dev context those are useless
i never implemented them
fsck is horrible outdated software
did you have sanity checks in your filesystem implementation
what lol
I had asserts everywhere
how would i verify filesystem integrity after modification
what does this mean?
Sanity checks
do you validate an inode's fields as being "sane enough" for the operation to continue
give an example
show like a code snippet
or something
Bet I don’t have it but I’ll just type it up real quick
no you can just say it
tell me like what kind of checks you had in place before operating on a possibly corrupted inode
cos i will probably add this soon
if (inode->size > (inode->blocks * BLOCK_SIZE)) {
return false;
}
Throw some sort of error
Instead of false
And then
This is useful
if (inode->nlinks == 0 || inode->nlinks > MAX_LINKS) {
return false;
}
huh what
oh intersting ok
Also
this... no?
if (inode->blocks > 0) {
bool all_zero = true;
for (int i = 0; i < 15; i++) {
if (inode->block[i] != 0) {
all_zero = false;
break;
}
}
if (all_zero) {
// give error maybe idk
}
}
what huh
what??? these are valid scenarios??
huhhuhuh
Yes
no no i'm saying
this is not corruption
you can have an inode with no blocks allocated to it
why would you produce an error here
💥
No no, you would check this BEFORE writing to an inode’s block for the first time
you can have an inode that has no links if it's still open somewhere but deleted, you'd access it by FD but it would be nonexistent if u access it by path
what ?!
an file that you just created with no data would have no blocks assigned to it
this is throwing an error for a perfectly valid scenario
crazy lol
There was some point where I checked something like that but yet again
perfectly valid filesystem throwing an FS_CORRUPT would be wild
so then your implementation is just incorrect here then
💥
So sorry
I explained this wrong
before assigning blocks to an inode for the first time, do a sanity check to make sure the inode does not already reference any blocks
If the inode has NOT been referenced
This doesn’t apply to like device nodes or empty files
why would you throw an error ??
just zero the blocks
if you have a lazily deleted inode that just removes the entry, of course it would reference blocks
Well because if that was happening im the first place something somewhere else had to have been fucked up
no?
you dont have to fully delete/zero all the contents of an inode when you remove its link
in ext2 you only have to remove it from the bitmap and remove all the child links from the bitmap (deallocate them)
so on-disk there will still be blocks on that inode when you re-allocate it
but it's fine, normal, and expected behavior
💥 did bro actually write this driver
But at least back then I thought it would be best to avoid this scenario
Yeah but remember this was 6 years ago lmao
i mean you were 12 so i'll give u credit for doing this at all or even getting remotely started
yea i'll give u some slack bro wasn't even a teenager
🔥
Apparently I wasn’t even 12
Because it’s dated to feb12 2019 but my birthday isn’t until March 29
ok now i'm starting to doubt you wrote this at all
I did, at that point I had already made a bootloader and snake game on gameboy in assembly
I’d been into low level stuff for a while
And stuff like that
age 11 ext2 driver?? wowzas
when will we drop the 11yo amar ext2 code 🙏
Fork() works
My funeral
can i see ur bootloader
i might add support for my OS to boot on it for no reason
It’s not necessarily a boot loader
why did u call it a bootloader
The game boy basically boots into a point of memory
Well
Because technically it is one
But the gameboy doesn’t have the facilities for me to be able to call it a true bootloader
shocker 💥
do u have these gameboy repos
awesome sauces
what's with ur internet harassment bot you made 😭
is this an april fools joke
Okay here’s the bootloader
This syntax is so cancer btw
Have fun reading any of that to be honest
neat
The game boy register set is
A B C D E H L
Also combined
Pairs
16 bit ones
AF
BC
DE
HL
isnt this literally a disassembly of the gameboy boot ROM with a few symbol names added
It’s based off of it but yeah
I didn’t actually use anything online like
I actually got the rom itself
And objdump the fuck out of it
and you.... changed symbol names and called it ur own ?
like these are bar-for-bar identical lol
Because it’s the only way you can boot anything on a gameboy it has to be that way
I’ll show you the guide I followed
cant you add your own custom little easter eggs
wow the instructions are all identical lol
Literally no
It has to be
256 bytes
In that exact order
No exceptions
so this wasnt much of a bootloader then
just objdump -> change symbols -> recompile
darn
did u ever do any other niche hardware
Wait you’re right
Let me see
What I can do for that
Fuck a VFS
💥
Now I’m purposely gonna add Easter eggs
Okay I figured out how to actually flow the code differently
'flow the code'
Alright guys
Enough bullshit
Enough stupid source structure
Rewriting
Using limine
I already got Flanterm and pmm working :)
And gdt idt
I’m gonna rename it Zarugan (Za - roo - gun) which is the name of my fictional language I made
cool
Wow its happening, good luck
Yessir
Wanna look at my new build system? Just started making it
Here’s the syntax, I already made the ast and parser and stuff so
decl cc = gcc;
decl src csource= main.c; // src is a built in
decl bin dest= build/main; // bin is a built in
decl args cargs = -foo;
method all (main.c) {
@cc @cargs -o @dest;
}
"We have make at home"
Idk it compiles my kernel quicker
And also compiles qemu decently quick with some problem spots lol
are you making another 50,000 line commit called "fixes."
No it’s a complete rewrite
In 64 bit
Using limine
And I have some stuff
"fixes."
+50000
-0
lol
“Fuck this shit. Fixes”
+10000000
-0
🥀💔
I rewrote paging.c 🙏
is this os still dead
unc fell off 💔
I’m actually working on it a lot rn
Unc is cooking a little
is unc making an asynchronous io stack ⁉️
oooooooooooooo
Nah nah nah I just rewrote my vmm and paging shit
And now cleaning up scheduler
I made all threads kernel threads
It’s just user mode threads are just kernel threads that jump to my user mode entry routine in C
Nice, good to see you're still going with this
I’m trying lol
void ctx_switch(cpu_ctx_t* _old, cpu_ctx_t* _new) {
asm volatile (
"pushf\n"
"pusha\n"
"movl %%esp, %[old]\n"
"movl %[new], %%esp\n"
"popa\n"
"popf\n"
"ret\n"
: [old] "=m"(*_old)
: [new] "m"(*_new)
: "memory"
);
}

32 bit 💥
why did you not call this the flopperating system
Phenomenal idea
I’m gonna do that
use an asm function ❌ use a C function that only has inline asm ✅
is your scheduler working
did u fix it
😃
was there a rewrite 😃
nah but i just finished cr3 switching so now the scheduler is basically done and i'm happy to do a rewrite
look!!
@small hound i wrote a goated vmm
that's it?
oh ok
cool beans
I thought there would be a bit more when you said it was goated
im making it a bit better
addibng comments and also todos
and some other fun methods
will we be getting demand paging
generic architecture agnostic paging system when
use a for loop to map pages and have each architecture implement an operation to do one layer of paging
wait where is your tlb shootdown
also doing a full tlb flush after any paging action is kinda wild
the invlpg in question:
why does your region unmapping also deallocate the pages in your physical memory allocator 🤨
int unmap_range(uintptr_t vaddr, size_t size) {
size_t pages = _align_up(size) / PAGE_SIZE;
vm_region_t *cur = vm_regions, *prev = NULL;
// iterate through the region list to find pages to unmap
while (cur) {
// check if the current region overlaps with the range to be unmapped
if (cur->start == vaddr && cur->end == vaddr + size) {
// if the current region is the first in the list, update the head of the list
if (prev) {
prev->next = cur->next;
}
// if the current region is the last in the list, update the tail of the list
else {
vm_regions = cur->next;
}
pmm_free_page(cur);
break;
}
prev = cur;
}
// iterate through the pages to be unmapped
for (size_t i = 0; i < pages; i++) {
if (unmap_page(vaddr + i * PAGE_SIZE) < 0)
return -1;
}
return 0;
}
``` sus
dunno why you'd wanna deallocate pages in page unmapping logic
im pretty sure i pressed enter too many times in copilot so im ficing that now
@small hound this should be better
int unmap_range(uintptr_t vaddr, size_t size) {
size_t pages = _align_up(size) / PAGE_SIZE;
vm_region_t *cur = vm_regions, *prev = NULL;
while (cur) {
// check if the range is within the current region
if (cur->start == vaddr && cur->end == vaddr + size) {
if (prev) {
prev->next = cur->next;
} else {
vm_regions = cur->next;
}
pmm_free_page(cur);
break;
}
// move to next region
prev = cur;
cur = cur->next;
}
for (size_t i = 0; i < pages; i++) {
uintptr_t va = vaddr + i * PAGE_SIZE;
if (unmap_page(va) < 0) {
// todo: handle error
return -1;
}
}
return 0;
}
its in my paging.c
here:
void _flush_tlb(void) {
uintptr_t cr3;
__asm__ volatile("mov %%cr3, %0" : "=r"(cr3));
__asm__ volatile("mov %0, %%cr3" :: "r"(cr3));
}
this is NOT tlb shootdown
what would it look like then
it is when you tell other processors to invalidate their TLB entries
on x86 this would be via an IPI
yeah
do you have that
no but
i had in my old src file
gonna go get it
wait
isnt it literally
just
asm volatile ("invlpg (%0)" : : "a" (vaddr));
in my case
yes and you need a way to send that address to other processors and wait for them to all invalidate that entry in their TLBs
flushing the entire TLB after changing one page mapping is very inefficient
how would i be able to sen that addr to other processors
static void do_tlb_shootdown(uint64_t addr) {
uint64_t this_core = get_sch_core_id();
uint64_t cores = scheduler_get_core_count();
for (uint64_t i = 0; i < cores; i++) {
if (i == this_core)
continue;
struct core *target = global_cores[i];
atomic_store_explicit(&target->tlb_shootdown_page, addr,
memory_order_release);
lapic_send_ipi(i, TLB_SHOOTDOWN_ID);
while (atomic_load_explicit(&target->tlb_shootdown_page,
memory_order_acquire) != 0)
cpu_relax();
}
}``` I cheesed this a bit with an atomic ptr
so if i dont have many cores rn
can i just
asm volatile ("invlpg (%0)" : : "a" (va));
for now
do you plan to support SMP
ok then just remember to do the tlb shootdown
okay
but is asm volatile ("invlpg (%0)" : : "a" (va)); good enough for now after unmapping?
yep 👍
do you support 2MB pages
not sure if 32 bit supports that
pretty sure it does
yes
its not that off from 64'
just remember 4gb limited mem
besides that its the same shit
also where is your actual page table walking logic
int map_page(uintptr_t vaddr, uintptr_t paddr, PageAttributes attrs) {
vaddr = _align_down(vaddr);
paddr = _align_down(paddr);
// get/create page table
PTE* pt = _get_or_make_pt(vaddr);
if (!pt) return -1;
// map physical frame to virtual address
uint32_t pt_idx = _pt_index(vaddr);
attrs.frame_addr = paddr >> 12;
SET_PF(&pt[pt_idx], attrs);
_flush_tlb();
return 0;
}```
where are you walking the page tables
void vmm_map_page(uintptr_t virt, uintptr_t phys, uint64_t flags) {
if (virt == 0) {
k_panic("CANNOT MAP PAGE 0x0!!!\n");
}
bool interrupts = spin_lock(&vmm_lock);
struct page_table *current_table = kernel_pml4;
for (uint64_t i = 0; i < 3; i++) {
uint64_t level = virt >> (39 - (i * 9)) & 0x1FF;
pte_t *entry = ¤t_table->entries[level];
if (!ENTRY_PRESENT(*entry))
pte_init(entry, 0);
current_table =
(struct page_table *) ((*entry & PAGING_PHYS_MASK) + hhdm_offset);
}
uint64_t L1 = (virt >> 12) & 0x1FF;
pte_t *entry = ¤t_table->entries[L1];
*entry = (phys & PAGING_PHYS_MASK) | flags | PAGING_PRESENT;
invlpg(virt);
do_tlb_shootdown(virt);
spin_unlock(&vmm_lock, interrupts);
}``` like I have this here
the loop
but where do you walk your page tables
ohhh i see
i hadthat implmented '
in paging.c
but forgot to add that subroutine
in map page
so let me
fix that
💀 this is like the most important part of your virtual memory management
is this blob of 300 lines of code you sent tested at all
does this even run
i literally wrote it all today
and yes it runs just having issues but that was the prob lem
and copilot too
how does it run if you aren't walking the page tables
how is it initializing if your mapping logic doesn't walk the page tables...
💥
if vmm_init(void) calls _map_kernel(pstart, ksize), and
_map_kernel(uintptr_t pstart, size_t size)
calls map_page(vaddr, paddr, k_pg_attrs), and
map_page(uintptr_t vaddr, uintptr_t paddr, PageAttributes attrs) does this
int map_page(uintptr_t vaddr, uintptr_t paddr, PageAttributes attrs) {
vaddr = _align_down(vaddr);
paddr = _align_down(paddr);
// get/create page table
PTE* pt = _get_or_make_pt(vaddr);
if (!pt) return -1;
// map physical frame to virtual address
uint32_t pt_idx = _pt_index(vaddr);
attrs.frame_addr = paddr >> 12;
SET_PF(&pt[pt_idx], attrs);
_flush_tlb();
return 0;
}```
then how is it initializing
👻
page table poltergeist
@small hound
👍
the goto is cursed
separate those out into functions
okay bet
Look fine besides that though?
you can also check if the higher level pages are also empty rather than just looking at the lowest one
for freeing
looks ok
👍
@small hound i modularized this nicer
int map_page(uintptr_t vaddr, uintptr_t paddr, PageAttributes attrs) {
vaddr = _align_down(vaddr);
paddr = _align_down(paddr);
if (vaddr == 0) return -1;
uint32_t indices[2] = {_pd_index(vaddr), _pt_index(vaddr)};
PTE* tables[2] = {RECURSIVE_PD, NULL};
for (int i = 0; i < 2; i++) { // walk page table levels
switch (i) {
case 0:
_pd_lvl(tables, indices, attrs);
case 1:
_pt_lvl(tables, indices, paddr, attrs);
}
}
asm volatile ("invlpg (%0)" : : "a" (vaddr));
return 0;
}
lol why is there a loop at all
^
I forgot you're on 32 bit 💀
so ive been losing my shit
✅ great job
heres how im doing ranges
int unmap_range(uintptr_t vaddr, size_t size) {
size_t pages = _align_up(size) / PAGE_SIZE;
vm_region_t *cur = vm_regions, *prev = NULL;
while (cur) {
// check if the range is within the current region
if (cur->start == vaddr && cur->end == vaddr + size) {
if (prev) {
prev->next = cur->next;
} else {
vm_regions = cur->next;
}
break;
}
// move to next region
prev = cur;
cur = cur->next;
}
for (size_t i = 0; i < pages; i++) {
uintptr_t va = vaddr + i * PAGE_SIZE;
if (unmap_page(va) < 0) {
// todo: handle error
return -1;
}
}
return 0;
}
int map_range(uintptr_t vaddr, uintptr_t paddr, size_t size, PageAttributes attrs) {
size_t pages = _align_up(size) / PAGE_SIZE;
vm_region_t* r = _make_region(vaddr, vaddr + size, attrs);
if (!r) {
return -1;
}
r->next = vm_regions;
vm_regions = r;
for (size_t i = 0; i < pages; i++) {
uintptr_t va = vaddr + i * PAGE_SIZE;
uintptr_t pa = paddr + i * PAGE_SIZE;
if (map_page(va, pa, attrs) < 0) {
// rollback if failed
// prevent mem leaks
// todo: actual error handling
for (size_t j = 0; j < i; j++) {
uintptr_t undo_va = vaddr + j * PAGE_SIZE;
unmap_page(undo_va);
}
vm_regions = r->next;
return -1;
}
}
return 0;
}
sounds great 😃
by the way mister flopmaster, my TLB eviction is kind of not the best way to do it. it's not exactly wrong but it also isnt the most efficient way
the better way would be to have a list (global or per core) of pages to invalidate, so that when you map/unmap a region, rather than waiting after each page, you add all pages to the list and send the IPI to start doing TLB shootdowns
yah and you can either send an IPI where the interrupt handler does the shootdown or you can have a thread that does that and just wake up the thread on the other cores
depends on how you wanna do it
or if you have it, you can use a deferred procedure call
and somehow get the other cores to run that DPC
I’m prob just gonna make a kernel thread to do that for me
I made a kernel thread to do it for me
does it work
you should make a unit and integration test suite to test these and such
Yeah right now my resting functions are just little static functions in my source files
So in vmm.c I have some static functions
resting functions?
oh ok
Like
Vmm_range_map_test
That bullshit
But yes I do plan to make a testing suite at some point
I haven’t pushed this yet only bc git is stupid on my laptop
It just gets stuck on “pushing”
internet issue?
Nah
did you sign in with gh-cli
for linux, be root, for windows, reboot
Holy fucking bars
still no flopperating system 😢
but there's plenty of floppy operating systems
i.e. operating systems for floppy disks!
I wish i could run mine on one
Unfortunately multiboot is more bloated than a polish drunk dad
Why the fck do I need an EGA video mode
Flopperating System
Changed thread name 🗣️🗣️
Just saying
I fucking love the buddy system
Doesn’t get easier than this
void* pmm_alloc_pages(uint32_t order, uint32_t count) {
if (order > MAX_ORDER || count == 0) return NULL;
void* first_page = NULL;
void* current_page = NULL;
for (uint32_t i = 0; i < count; i++) {
void* page = NULL;
for (uint32_t j = order; j <= MAX_ORDER; j++) {
if (pmm_buddy.free_list[j]) {
struct Page* p = pmm_buddy.free_list[j];
pmm_buddy.free_list[j] = p->next;
p->is_free = 0;
p->order = order;
while (j > order) {
j--;
buddy_split(p->address, j);
}
page = (void*)p->address;
break;
}
}
if (!page) {
if (first_page) {
pmm_free_pages(first_page, order, i);
}
log("pmm: Out of memory!\n", RED);
return NULL;
}
if (!first_page) {
first_page = page;
}
}
return first_page;
}
now make it NUMA aware 
In this economy? Only bitmap

wut
you can make a NUMA aware bitmap allocator
it will just be bad
That would be funny
Doesn’t that have nothing to do with bitmap / buddy
I thought it was where memory access time depends on the memory location relative to the processor
It’s an smp thing iirc
yes
non uniform memory access
Yes
Kinda a complicated concept but I get it after reading some more
Bro my parents are crazy fucks
They destroyed my fucking laptop which I just fucking bought for uni
For no fucking reason too besides being fucking assholes
It’s because I didn’t give them 200$ out my paycheck
When I literally
Don’t even make 500 after taxes a now each week
So consider the project on hold
All of my source code is gone
Computer is fucked
dang, I hope u can recover the disk off of the laptop and maybe get some stuff back
🙏
It’s just so fucking infuriating living with people like this
I got bread but god damn can I just have my laptop 😢
Update: I bought a new computer
Setting it up later
Flopperating system may be back
Hopefully I can fucking push git commits too
Okay we got it
dude
it took me so damn long
to write this shit
PDE* vmm_clone_pd(PDE* _src_pd) {
// allocate new page directory (zero ts)
PDE* new_pd = (PDE*)pmm_alloc_page();
if (!new_pd) return NULL;
for (uint32_t i = 0; i < PAGE_DIRECTORY_SIZE; i++) {
if ((i << 22) >= KERNEL_VADDR_BASE) {
new_pd[i] = kernel_pd[i];
continue;
}
if (!_src_pd[i].present)
continue;
// define the src pt and create a new one
PTE* src_pt = (PTE*)(_src_pd[i].table_addr << 12);
PTE* new_pt = (PTE*)pmm_alloc_page();
if (!new_pt) return NULL;
_zero_pt(new_pt);
for (uint32_t j = 0; j < PAGE_TABLE_SIZE; j++) {
if (!src_pt[j].present)
continue;
// allocate new frame
void* new_frame = pmm_alloc_page();
if (!new_frame)
return NULL;
// memcpy old frame to new frame
void* old_frame = (void*)(src_pt[j].frame_addr << 12);
flop_memcpy(new_frame, old_frame, PAGE_SIZE);
PageAttributes attrs = {
.present = src_pt[j].present,
.rw = src_pt[j].rw,
.user = src_pt[j].user,
.frame_addr = ((uintptr_t)new_frame) >> 12
};
SET_PF(&new_pt[j], attrs);
}
PageAttributes pt_attrs = {
.present = 1,
.rw = 1,
.user = 1,
.frame_addr = ((uintptr_t)new_pt) >> 12
};
SET_PF((PTE*)&new_pd[i], pt_attrs);
}
return new_pd;
}
its alright, these threads are for sharing your progress
I reckon 50% of that is either kernel panics or code snippets lol
or screenshots of code 😠
Never fast enough 😔
Courtesy of @small hound
Don’t worry my progress will be visible to you guys soon
I just fucking suck at version control
And it kinda makes me mad
10 exaflops/second
Lol take your time, no one here is rushing you
thanks man
kinda cooking atm so
we wil see
oh yea a thing I found helpful was to encapsulate all globals that are used in a bunch of places in one struct like this
struct charmos_globals {
char *root_partition;
struct vfs_mount *mount_list_head;
struct vfs_node *root_node;
volatile enum bootstage current_bootstage;
uint64_t core_count;
struct scheduler **schedulers;
struct core **cores;
volatile bool panic_in_progress;
};
extern struct charmos_globals global;```
idk u might find it helpful
I found it very neat and useful
Oh shit yeah
That’s not a bad idea
I should probably reserve a bit of memory in the kernel to hold a struct like that at boot
"reserve a bit of memory" wdym
just
struct charmos_globals global = {0};
yeah you right
anyways
i should invlpg only after unmapping, right?
do you ever change PTEs in any situation besides unmapping
no
are you sure
wait
if 0xfff000 is mapped to 0x4000, and you go and map 0xfff000 to 0x1000, you are indeed changing page table entries
actualy yeah
yeah so for mapping and unmapping
i need to invlpg
got it
✅
anywhere i change pt
i need to invlpg
well great, no paging fault now
okay yessir
got it
lemme show
how im mapping my memory
this is good i should add this as well
int map_kernel_space(uintptr_t vaddr, size_t size) {
if (vaddr < KERNEL_VADDR_BASE) {
return -1;
}
page_attrs_t attrs = {
.present = 1,
.rw = 1,
.user = 0
};
uintptr_t phys_addr = (uintptr_t)pmm_alloc_pages(0, size / PAGE_SIZE);
return map_range(vaddr, phys_addr, size, attrs);
}
int map_user_space(uintptr_t vaddr, size_t size) {
if (vaddr >= USER_SPACE_END) {
return -1;
}
page_attrs_t attrs = {
.present = 1,
.rw = 1,
.user = 1
};
uintptr_t phys_addr = (uintptr_t)pmm_alloc_pages(0, size / PAGE_SIZE);
return map_range(vaddr, phys_addr, size, attrs);
}
and the user gets 3gb