#HelixOS (ARM64 OS, not dead project!)

1 messages · Page 1 of 1 (latest)

static shore
#

it is made for freestanding evs

spring sonnet
static shore
#

oh, and gl with the arm architecture

spring sonnet
#

Thank youuu!!!

spring sonnet
forest timber
spring sonnet
#

Build instructions have now beed added to the README. If you run into issues duiring the build let me know and I'll assist in getting it resolved

spring sonnet
#

Got some 3rd part libs suck as flanterm, printf, and the freestanding headers going. Now that I've gotten flanterm up and going I can focus on exception handling. It's my favorite part haha

spring sonnet
#

Got the beginnings if physical memory managment underway. I still would like to figure out why when taking an exception, and using printf() I end up accessing physical regions that aren't mapped.

Screenshot_at_2025-03-14_14-39-01.png
unreal creek
unreal creek
spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
#

Beautiful!!

Screenshot_at_2025-03-14_16-01-12.png
#

Need to "pretty print" the sizes a bit better though so it looks clean. But, this is great. Had to work out some pointer arithmatic issues I was having, but got that all fixed.

spring sonnet
#

First simple PMM allocation test! It's far from done, but after lots of studying, understanding, and finally testing + debugging I'm able to allocate memory!! 🎉

Screenshot_at_2025-03-14_22-21-12.png
unreal creek
spring sonnet
unreal creek
#

the PMM

spring sonnet
#

Ohhh to my repo?

#

I did make a commit for the initialization part, but I'm working on the allocation now

unreal creek
unreal creek
unreal creek
spring sonnet
#

I don't seem to have the ability to pin 😅

spring sonnet
unreal creek
spring sonnet
#

Yeah, but I can always just share the link again if needed

unreal creek
#

what is kaslr option in your limine.conf ?

spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
unreal creek
lime sapphire
#

Here ya go

spring sonnet
spring sonnet
unreal creek
spring sonnet
unreal creek
#
  • 1 : you have a stack that contain the address of each free page you pop to get an address and push address to free
#
  • 2 : instead of just having a stack of address you have a stack of block each block as a start address and a size
  • to get an address you pop an block , decrement the size of the block and if it still superior to 0 you increment the base address of the block and repush it
  • to free : well idk
spring sonnet
#

That sounds interesting

#

I ended up opting for the freelist mothod... so far it's been rather simple to get initialized, but allocation for larger regions has been tricky to implement 😅

unreal creek
#

not next to each other

#

you for exmple allocate 0x1234000 0x4321000 and map it to 0x1000 and 0x2000

#

the only reason to allocate more than one page is for some devices

#

like eterneth card

spring sonnet
unreal creek
spring sonnet
unreal creek
#

then you made a kernel sbrk

#

to grow the heap

#

(using your PMM)

#

and then the VMM allocate space in the heap

#

if the heap is too small use sbrk to make it bigger

#

in this case the PMM don't need to be able to allocate small size

spring sonnet
#

Maybe that's just me being picky, but...

unreal creek
spring sonnet
#

I managed to get large allocations working!!! Hell yea!!!

Screenshot_at_2025-03-15_02-23-18.png
unreal creek
spring sonnet
#

Right there there's no way to free these allocations (I'll do that tomorrow it's getting late), but I'll push the commits

spring sonnet
# unreal creek good

Keep in mind the current allocation implementation will change around a bit... Right now it's just to get allocations going haha

#

I also think I need to rework it so that it starts from the "beginning" of the list

unreal creek
spring sonnet
# unreal creek is it now ?

I was doing some testing and noticed only the last added node was able to be accessed, and anything else I wasn't able to. Meaning I only could allocate that 4K region... it's due to the way I had setup my freelist though

#

I'm reworking some things though

unreal creek
spring sonnet
unreal creek
#

then you create a new pii.ter to the last

#

and you only set that one

spring sonnet
#

That would explain why I can't iterate the rest of them...

unreal creek
unreal creek
spring sonnet
unreal creek
#
  • 1 ft is change every time
  • 2 the special case for the first usable seg should finish with continue;so the case for other seg don't happen
  • 3 two nodes are created in the case for the first usable segment
spring sonnet
#

😅

unreal creek
spring sonnet
unreal creek
#

and also

#

don't allocate space in bootloader recalamable

#

as it's where is the memory map, the response of all limines request and the stack

spring sonnet
unreal creek
spring sonnet
#

I don't know why I was thinking to make use of those regions too lol

unreal creek
spring sonnet
unreal creek
#

in the wrost case you detroy your own stack

#

or the memap

#

you can technicly use it

#

but only when you stop using any response

#

and the stack

#

so just don't use it

spring sonnet
#

I've changed my pmm initialization to only make use of the usable regions. I don't really need to be destroing anything in reclaim while setting up the system for kernel use... that would be painful to debug 😂

unreal creek
spring sonnet
#

Not right yet... I'm working on debugging an issue in my exception handler where the stack ends up in a lower-half memory region rather than the higher-half it's supposed to be at. I was using the stack Limine provided upon startup of my kernel so I'm working on changing it to my own

#

Once I resolve that I'll be going back to the pmm stuff

unreal creek
#

oh yes i didn't see it

spring sonnet
unreal creek
#

but i think i'm gonna port my OS to it

#

cause i'm gonna get a rasberry pie and getting my OS to run of this would be awsome

spring sonnet
# 
Breakpoint 1, __aarch64_synchronous_handler () at /media/austin/Scratch/HelixOS/kernel/arch/arm64/vectors.s:107
107        bl exc_handler
(gdb) info r sp
sp             0x4760f730          0x4760f730
(gdb)

I still don't see how even in my own stack the stack address upon entry of the synchronous exception handler the stack is switched to a lower region. I need to see how things got configured and read the protocol to see if Limine made changes

#

Otherwise this would end up being an address translation issue most likely

spring sonnet
unreal creek
#

does limine work on rasberry pie ?

spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
#

I see a few solutions to fixing my exception issue

spring sonnet
unreal creek
unreal creek
spring sonnet
#

Now my handler works as expceted. I'll still need to implement things for it of course. But at least now when an exception occurs I'll know

spring sonnet
spring sonnet
#

I've now reworked the freelist initialization.

#

I understand now how I'm supposed to iterate the nodes, and keep the "head node" intact without overwriting it.

#

Also ensuring to make use of only the usable regions.

Screenshot_at_2025-03-15_22-22-23.png
#

Something I don't think I did in my previous attempt was to also account for the freelist pointer of the next node so if a write happened there it would end up getting overwritten

#

I can now properly iterate nodes all the way to the last one. Something that previously was broken.

Screenshot_at_2025-03-15_22-39-24.png
#

Commit has been pushed to the repo. Now I can work out allocation methods

unreal creek
spring sonnet
# unreal creek great

Feel free to check out the updated code. I'm working on the new allocation stuff now

unreal creek
#

okay so vector.s is defning a table with all the handler if i understand

unreal creek
spring sonnet
#

I may consider doing colored text though just to "pretty print" it

unreal creek
#

the osdev.org say that it's impossible to get a memory map on arm

spring sonnet
#

What do you mean by "impossible"...?

#

Every system gets a memroy map...

spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
unreal creek
#

but if it's an ARM device wihout BIOS/UEFI ?

spring sonnet
unreal creek
#

thanks

spring sonnet
#

Of course!

unreal creek
spring sonnet
unreal creek
spring sonnet
#

And to try to combat fragmentation I'll coalesce them

unreal creek
spring sonnet
#

very cool, also freelist allocator mentioned, based 💯

#

}

spring sonnet
spring sonnet
#

Finally got page allocation implemented. Keep in mind though it's a really simple implementation just to get going. I'll now be working on deallocation...

Screenshot_at_2025-03-17_23-52-10.png
unreal creek
spring sonnet
# unreal creek good

Fixed some allocator bugs I had ran into.... NOOOOW it properly allocated page regions

Screenshot_at_2025-03-19_00-21-42.png
unreal creek
#

just to know are you resting on multiples configuration?

spring sonnet
#

I need to fix the printing of incorrect addresses though lol

spring sonnet
unreal creek
#

to make sure it work

spring sonnet
#

I plan to, but for now I want to just keep my VM at 4GB

#

I realize my address prints are working as expected lol

#

Hell yes at last!! Time to commit the changes!!

Screenshot_at_2025-03-19_00-27-19.png
spring sonnet
#

I'll be working on deallocation, etc, but this is good so far

unreal creek
spring sonnet
#

After lots of issues and rewrites I finally got a properly working page frame allocator going.

Screenshot_at_2025-03-20_21-13-29.png
#

No way to deallocate right now, but had to fix several issues I ran into before, but now it's working fine

#

But why are the returned addresses not page-aligned tho?

#

You should do that

#

Because for stuff like paging everything in the first three hex digits are considered as flags and could screw up stuff like the root page table

spring sonnet
# spring sonnet You should do that

That's what I was going to do next, but because the page_frame struct contains the beginning and end of each allocated region I decided to offset the base by the size of that struct. Otherwise I just end up overwriting the pointer of that data

#

I can page align it, but then I won't really have a way to get that information at that point for deallocation. As you can see this design is already a bit flawed

#

Tho the concept of linked lists allocation is just that they point to available regions to use

#

When allocating you just move the linked list node to another position (aka current position + 4KiB since pages)

#

When deallocating you just add that pointer back to the list of regions

#

And obv coalesce if possible

spring sonnet
# spring sonnet And obv coalesce if possible

See that's where I start to have a hard time. If I allocate several contiguous pages for example:

    void *ptr = pmm_alloc();
    printf("ptr = 0x%lx\n", (uint64_t)ptr);

    void *ptr2 = pmm_alloc(); 
    printf("ptr2 = 0x%lx\n", (uint64_t)ptr2);

    void *ptr3 = pmm_alloc(); 
    printf("ptr3 = 0x%lx\n", (uint64_t)ptr3);

    void *ptr4 = pmm_alloc(); 
    printf("ptr4 = 0x%lx\n", (uint64_t)ptr4);

    void *ptr5 = pmm_alloc();
    printf("ptr5 = 0x%lx\n", (uint64_t)ptr5);

    void *ptr6 = pmm_alloc();
    printf("ptr6 = 0x%lx\n", (uint64_t)ptr6);

Then let's say I want to free ptr4 and keep the rest.... I kind of rely on the struct information of that start and end address to be able to remove that node from the list, and stich up the other nodes to the last one...

#

I can't really even think of a better way to do any of this.

#

If I don't offset I end up overwriting the pointer of the struct and that's not exactly what I want either. I don't really know how I can do it any differently

#

Idk you can try to take a look at this ig

https://github.com/RepubblicaTech/rtos/blob/main/src%2Fkernel%2Fmemory%2Fpmm.c

https://github.com/RepubblicaTech/rtos/blob/main/src%2Fkernel%2Fmemory%2Ffreelists%2Ffreelist.h

GitHub
rtos/src/kernel/memory/pmm.c at main · RepubblicaTech/rtos

Freestading 64-bit kernel (currently in very early development stage)) - RepubblicaTech/rtos

GitHub
rtos/src/kernel/memory/freelists/freelist.h at main · RepubblicaTec...

Freestading 64-bit kernel (currently in very early development stage)) - RepubblicaTech/rtos

spring sonnet
#

I see the keep an array of allocated nodes it looks like?

#

Hmm.. it seems kind of similar to what I'm doing

spring sonnet
#

But I rely on having metadata about the allocations so that I can later free them

#

The array is just used on initialization of the available regions

#

After that everything relies on the actual nodes

spring sonnet
#

I'll end up figuring it out

#

I just have a bit of troubles understanding how allocations and freeing them work without keeping metadata about them

spring sonnet
#

So I'll probably want to drop the metadata for the page frames

#

You just offset them

#

Freelist is simple because you just have nodes that point to usable memory ranges

#

Just as before, allocation just removes part of that node to offset it by X pages

#

And freeing just adds it to the linked list

spring sonnet
# spring sonnet You just offset them

That's what I was doing with my page frame freelists since the header is 18 bytes I offset the raw data pointer by that much so that I don't end up overwritting the pointer to the metadata

#

The thing I don't get is how is the pointer not overwritten in any of this? Because that's another reason why I've implemented it this way

unreal creek
unreal creek
#

well idk

spring sonnet
#

So something like this?

typedef struct page_frame {
   struct page_frame *next;
} page_frame_t;

I thought each pointer had to poinit to the page itself which means that any writes to the beginning of that page would result in the pointer getting overwritten?

spring sonnet
# unreal creek then when you need more node 
void *pmm_alloc(void) {
    page_frame_t *page = NULL;
    page_frame_t *head = NULL;
    uint64_t hhdm = mm_get_hhdm_offset();




    if (used_pages >= free_pages) {
        printf("PMM: No available pages for allocation!\n");
        return NULL;
    }

    if (!pages) {
        page = (page_frame_t *)(hhdm + m->base);
        page->base = m->base + sizeof(page_frame_t);
        page->end = page->base + PAGE_SIZE;
        page->next = NULL;

        pages = page;
        used_pages++;

        memset((void *)(hhdm + page->base), 0, PAGE_SIZE);

        printf("PMM: Allocated page: 0x%lx - 0x%lx\n", page->base, page->end);
        return (void *)(hhdm + page->base);

    }

    while(pages->next != NULL) {
        pages = pages->next;
    }

    // Place at the end of each allocated page
    page = (page_frame_t *)(hhdm + pages->end);
    page->base = pages->end + sizeof(page_frame_t);
    page->end = page->base + PAGE_SIZE;
    page->next = NULL;

    pages->next = page;
    used_pages++;

    memset((void *)(hhdm + page->base), 0, PAGE_SIZE);

    printf("PMM: Allocated page: 0x%lx - 0x%lx\n", page->base, page->end);
    return (void *)(hhdm + page->base);
}

void pmm_free(void *ptr) {
    page_frame_t *page = (ptr - sizeof(page_frame_t));

    printf("PMM: Freeing page: 0x%lx - 0x%lx\n", page->base, page->end);
}

This is how I currently handle allocations, and when I free I just do what malloc() would do, and offset the size of the struct from the pointer to get that metadata

unreal creek
spring sonnet
#

Since the struct itself is like 18 bytes I'm willing to sacrafice the space for it, but for page mapping this might be tricky

spring sonnet
unreal creek
# spring sonnet Oh... welp haha

start with something like this

typedef struct page_frame {
    stuct page_frame *next;
    void *data;
    size_t lenght;
} page_frame_t```
#

so the struct can be anywhere

#

not just before the page

#

let me a few second to check your implemenation

spring sonnet
unreal creek
unreal creek
spring sonnet
# unreal creek just a quick question you use the struct to represent free page or allocated p... 
typedef struct memory_frame {
    uintptr_t           base;
    uintptr_t           end;
    size_t              length;
    struct memory_frame *next;
} memory_frame_t;

typedef struct page_frame_node {
    uintptr_t               base;
    uintptr_t               end;
    struct page_frame_node  *next;
} page_frame_t;

One is used to keep a pointer to each available memory region, and the page frame struct is what I was using to keep a pointer to the allocated page starting at the beginning of the free memory, and increment as allocations are needed

#

Trying to push and pop off the list kind of confuse me without keeping some kind of metadata about the allocation

unreal creek
#

let me think a bit

spring sonnet
# 

static memory_frame_t *m = NULL;
static page_frame_t *pages = NULL;

static size_t free_pages = 0;
static size_t used_pages = 0;
static size_t total_mem = 0;










void pmm_init(void) {
    memory_frame_t *head = NULL;
    uint64_t hhdm = mm_get_hhdm_offset();
    uint32_t num_map_entries = mm_get_num_entries();


    for (uint32_t i = 0; i < num_map_entries; i++) {
        struct limine_memmap_entry *e = mm_entry_for_each(i);

        if (e->type != LIMINE_MEMMAP_USABLE) {
            continue;
        }

        memory_frame_t *frame = (memory_frame_t *)(hhdm + e->base);
        frame->base = e->base;
        frame->length = e->length;
        frame->end = e->base + e->length;
        frame->next = NULL;

        printf("PMM: Usable region: 0x%lx - 0x%lx region size: %uKB\n", frame->base, frame->end, frame->length / 1024);

        if (!m) {
            m = frame;
            head = frame;

        } else {
            while(m->next != NULL) {
                m = m->next;
            }

            m->next = frame;
        }

        total_mem += e->length;
    }

    // Set first region for use
    m = head;

    // Get total pages
    free_pages = SIZE_TO_PAGES(total_mem, PAGE_SIZE);
}

This is how I initialize as well

spring sonnet
unreal creek
#

you waste one page per region but at least evreything is aligned

spring sonnet
#

I prob have some notes I made long time ago about this

#

Lemme see if I have them

spring sonnet
spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
unreal creek
#

you won't be wasting pages

unreal creek
#

you don't need to

#

and when you need more node

#

you just allocate a page for it

#

then when you delete a node

#

you free the page it use

#

so technicly you don't waste anything

#

so

  • don't keem track of allocated pages you don't need to do that
  • first use the first page of each region as a node
  • when a new node must be created use your alloc to alloc space for it
  • when a node is deleteted free the page it use
#

@spring sonnet

spring sonnet
#

If you don't keep track of them then how are you supposed to free them when they're no longer needed...? That doesn't really make sense to me, but it's alright

#

I'll just figure it out. The way data pointer itself is just offset from the start of the page because I have the pointer to the metadata itself. I don't really have any good ways to implement a cache area to store this information either and still maintain a pointer to that data.

#

So this is pretty much the allocation part

IMG_20250321_081442.jpg
#

Oh also I forgot that when freeing I add a pages parameter to tell how many pages to give back to the linked list

(Ik this is hella insecure, but since this is the lowest level of MM that only stuff like paging and vmm would use it I don't really care about it)

#

(VMM and everything above it knows how much memory to free when a pointer is given)

unreal creek
#

you just free it

#

you don't need to keep track of it

spring sonnet
#

And this is the free

IMG_20250321_083933.jpg
#

Honestly..... this sounds to me that I just take a struct and that keeps a next pointer for a page allocation and go from there. I keep a set of nodes that each point to a free region of memory, but from there I was trying to allocate a page frame uising the beginning of that region

#

That's basically it

#

Tay had told me know when I asked that, and got confused

spring sonnet
# spring sonnet That's basically it

What I don't get though is if I'm keeping nodes of free memory ranges how do I keep track of the ranges allocated or to increment? That's why I implemented everything the weay I did because I need to keep track of this

#

Ehh. I'll figure it out

spring sonnet
lime sapphire
#

if you really want to track that data, you can do it externally (look up something called struct page), but imo you dont need that right now - a simple freelist will do just fine.

spring sonnet
spring sonnet
#

I'll take a look at the notes as well

spring sonnet
#

This freelist stuff is really not going well.

Screenshot_at_2025-03-21_09-08-57.png
#

I'm slowly being pushed to just use a bitmap instead.

#

I'll try again in a bit.

spring sonnet
#

@lime sapphire I remember you said something about dynamically allocating a page frame region during the allcoation rather than trying to create all the pages at once? If I tried to create the frames all at once it's waaaaaay too slow and takes forever on a 4GB VM

# 
static page_frame_t *pages = NULL;

static uintptr_t mem_start = 0;
static uintptr_t mem_end = 0;
static size_t mem_size = 0;

static uintptr_t page_addr = 0;

static size_t free_pages = 0;
static size_t used_pages = 0;


void pmm_init(void) {
    uint64_t hhdm = mm_get_hhdm_offset();
    uint32_t map_entries = mm_get_num_entries();


    for (uint32_t i = 0; i < map_entries; i++) {
        struct limine_memmap_entry *e = mm_entry_for_each(i);

        if (e->type != LIMINE_MEMMAP_USABLE) {
            continue;
        }

        if (mem_start == 0) {
            mem_start = e->base;
            mem_size = e->length;
            mem_end = mem_start + mem_size;

            free_pages = SIZE_TO_PAGES(mem_size, PAGE_SIZE);
        }
    }

    pages = (page_frame_t *)(hhdm + mem_start);

    printf("PMM: Page frame information:\n\n");
    printf("Memory region: [0x%lx - 0x%lx] length %uMB\n", mem_start, mem_end, mem_size / (1024 * 1024));
    printf("Free pages: %u, used pages: %u\n", free_pages, used_pages);
}

Decided to do something like this and use the largest memory region. Then I can just dynamcally allocate the page frame as needed, and when freeing I can just pop it from the list and decrement the address...

#

Okay I'm just confusing myself with this.

#

I can't create all the frames at once because trying to carve a 4GB region of memory in 4K chunks in a loop is too slow.

unreal creek
spring sonnet
# unreal creek or a stack

Not really sure how the stack method works. I also don't really see how anyone is getting the freelist thing working if they're creating all the frames at once. When you iterate a large 4GB 4K at a time it's pretty slow

spring sonnet
unreal creek
spring sonnet
#

Also, do you have any good documentation I can read on for bitmaps? I've been trying to look, and can't really find anything good haha

unreal creek
#

also storing last free page

spring sonnet
unreal creek
unreal creek
spring sonnet
unreal creek
#

each bit is a page

#

you search an empty bit

#

mark it as 1

#

then return it

spring sonnet
#

That's rather simple; Do I just select the largest region to use for the bitmap allocation?

unreal creek
#

so once you find a page you return it

#

you don't care about the rest

spring sonnet
lime sapphire
spring sonnet
#

The part that has be confused is if I'm suppoosed to hold a freelist of available pages it's hard to try to come up with a fast algorithm to create the list. Because looping a 3.5GB range 4K at a time is really slow

lime sapphire
#

Yeah so with this idea you only create 1 entry per region of memory up front.

#

The time you would spend setting up the list is distributed

lime sapphire
spring sonnet
#

I'll have to try and think of a different way to implement this or just keep a list of used pages

lime sapphire
#

Why do you need a list of used pages?

lime sapphire
spring sonnet
spring sonnet
#

Since trying to create a node for every 4K of a 3.5 region of memory takes a while....

lime sapphire
#

Yeah so start with 1 node per region instead of per page

spring sonnet
#

That's what I originally had did was keep a nide pointing to the start of each region; The next part comes carving a page out of this region which I've yet to implement without bugs...

spring sonnet
# lime sapphire Yeah so start with 1 node per region instead of per page 

static freelist_t *pages;
static size_t mem_size = 0;











void pmm_init(void) {
    uint64_t hhdm = mm_get_hhdm_offset();
    uint32_t map_entries = mm_get_num_entries();


    for (uint32_t i = 0; i < map_entries; i++) {
        struct limine_memmap_entry *e = mm_entry_for_each(i);

        if (e->type != LIMINE_MEMMAP_USABLE) {
            continue;
        }

        freelist_t *node = (freelist_t *)(hhdm + e->base);
        node->base = e->base;
        node->length = e->length;

        if (!pages) {
            pages = node;
        } else {
            while(pages->next != NULL) {
                pages = pages->next;
            }

            pages->next = node;
        }

        mem_size += e->length;
    }
}

This is what I originally had did was keep a node pointer at the start of each region

spring sonnet
# lime sapphire Yeah so start with 1 node per region instead of per page 


void pmm_init(void) {
    freelist_t *head = NULL;
    uint64_t hhdm = mm_get_hhdm_offset();
    uint32_t map_entries = mm_get_num_entries();


    for (uint32_t i = 0; i < map_entries; i++) {
        struct limine_memmap_entry *e = mm_entry_for_each(i);

        if (e->type != LIMINE_MEMMAP_USABLE) {
            continue;
        }

        uintptr_t base_aligned = ALIGN_UP(e->base + sizeof(freelist_t), PAGE_SIZE);
        size_t length_aligned = e->length - (base_aligned - e->base);

        if (length_aligned < PAGE_SIZE) {
            continue;
        }

        freelist_t *node = (freelist_t *)(hhdm + e->base);
        node->base = base_aligned;
        node->length = length_aligned;
        node->next = NULL;

        if (!mem) {
            mem = node;
            head = node;

            printf("PMM: Available region: [0x%lx - 0x%lx] region size: %uKB\n", node->base, node->base + node->length, node->length / 1024);
        } else {
            while(mem->next != NULL) {
                mem = mem->next;
            }

            printf("PMM: Available region: [0x%lx - 0x%lx] region size: %uKB\n", node->base, node->base + node->length, node->length / 1024);
            mem->next = node;
        }

        mem_size += length_aligned;
        free_pages += length_aligned / PAGE_SIZE;
    }

    mem = head;
    printf("PMM: Memory size: %uMB, free pages: %u, used pages: %u\n", mem_size / (1024 * 1024), free_pages, used_pages);
}

This is what I was doing before was just carve a 4K chunk out of the free region and go from there. Let me know your thoughts on this.

#

I'm really not trying to make it complicated lol

spring sonnet
#

I ended up just going the bitmap route for now....

spring sonnet
#

Finally getting starting with the virtual memory allocator...

spring sonnet
#

I'm excited to finally get things going....

#

The bitmap allocator route isn't ideal, but for now it'll work fine for my needs. I can always change things later down the road

lime sapphire
#

I think your lines of code are too long meme

spring sonnet
lime sapphire
#

also the lbp guarantees that usable entries have their bases and lengths page-aligned

#

so you can skip a lot of that - good that you're thinking about it though

spring sonnet
#

Oh absolutely!

lime sapphire
#

you could get away with:

struct freelist_node
{
    struct freelist_node* next;
    size_t count;
};

struct freelist_node* pm_freelist;

void pmm_init() 
{
    //...
    pm_freelist = NULL;
    for (size_t i = 0; i < entries_count; i++)
    {
        struct limine_memmap_entry* entry = mm_entry_for_each(i);
        struct freelist_node* node = (struct freelist_node*)(hhdm + entry->base);
        node->count = entry->length >> 12;

        node->next = pm_freelist;
        pm_freelist = node;
    }
}
#

this reverses the order of memory regions from the memory map, if you care about that

#

and allocation is easy, check if pm_freelist->count > 1, if it is decrement the count and the last page in the region (or the first and then adjust the head, there's many ways)

spring sonnet
#

This is actually much simpler than what I was orignally doing

#

And a hell of a lot more clean

unreal creek
spring sonnet
spring sonnet
#

Got initial paging going!!!!

spring sonnet
#

Very cool

spring sonnet
# spring sonnet Very cool

Thank you good sir!! I was finally able to escape Limines page tables; I pushed the code to my repo so feel free to have a look! 😁

unreal creek
spring sonnet
# unreal creek great

Now I can start workling on some other stuff like the slab allcoator, and kmalloc()...

unreal creek
#

cause the paging function are arch specific and not a part of the VMM

#

while the VMM isen't arch specific

spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
# unreal creek yes it is

Perfect!! I'm working on a way to unmap pages... then I'll be starting on a slab allocator, and then kmalloc()

#

So that I can allocate memory for object size stuff

unreal creek
spring sonnet
unreal creek
spring sonnet
unreal creek
#

i forgot to so that in my OS

#

what happend when you don't do that ?

spring sonnet
spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
spring sonnet
spring sonnet
#

It's just to get the kernel going on it's own page tables for now, but I plan to improve it

#

Oooo I can even reserve memory as well!

unreal creek
#

btw i put an start on the repo

#

i think you should map hddm as well

#

just itarate trought the memmap an mam any region (exluding BAD_MEMORY)

#

you also don't need to map the bitmap and the framebuffer if you do that

#

because they are inside the hddm

spring sonnet
spring sonnet
unreal creek
unreal creek
spring sonnet
#

That actually won't be difficult to implement to be honest

spring sonnet
#

I may also map the reserved regions where the GIC reside as well

unreal creek
spring sonnet
spring sonnet
unreal creek
#

just map the hddm

spring sonnet
# unreal creek just map the hddm

Because I'm on ARM64 Limine reserves some of the lower-half regions. The GIC, etc resides there, and Limine's page tabels marks these as reserved

#

But I'll map the GIC, etc, once I'm ready to instead of mapping the entire reserved range...

#

I had mapped the kernel sections like I did though so that I could write-protect my kernel's rodata, and text sections

#

I can always remap them for this though

unreal creek
#

okay

spring sonnet
# unreal creek okay

I also had some other ideas about remapping the modules, etc, into a different virtual address, etc...

#

But that's for when I start getting into the userspace stuff and start constructing the initrd, etc

spring sonnet
spring sonnet
spring sonnet
spring sonnet
#

Dude I'm beyond ecstatic that I've gotten virtual memory going

spring sonnet
#

@unreal creek I'm refactoring the hell out of my VMM to make mapping things more clean lol

unreal creek
spring sonnet
unreal creek
#

you forgot to free page

#

when tey are empty

#

like free the l3 when all entry in it are 0

#

@spring sonnet you have an UART driver ?does your os run ln raspberry PI?

spring sonnet
sage atlas
#

ooh limine based

unreal creek
spring sonnet
sage atlas
#

god I wish I could code

#

I wanna osdev so bad

#

😭

spring sonnet
#

It's so much fun haha

unreal creek
sage atlas
#

and definitely not to an osdev level

unreal creek
sage atlas
#

yea, just can't get mysefl to stay motivated

spring sonnet
spring sonnet
sage atlas
spring sonnet
sage atlas
#

so, is this like unix-like?

spring sonnet
#

Well... Sort of....

unreal creek
spring sonnet
#

Had to refactor my VMM a bit, and now have the ability to unmap pages

unreal creek
#

btw can you remove space in the place where you have like 10 empty line

#

that stress me out

spring sonnet
spring sonnet
spring sonnet
#

Got UART working finally. Time to get interrupts going.

Screenshot_at_2025-03-23_18-03-20.png
spring sonnet
#

Studying the Generic Interrupt Controller v3 docs, and getting stuff started now.

#

I want to make sure I've got interrupts in place, etc, then I plan to work on other allocators, etc. The sooner I get this shit implemented the better

unreal creek
#

okay

#

i most of the time do vfs tmpfd and inited after memory allocation

#

but do whanever you want

unreal creek
spring sonnet
spring sonnet
unreal creek
#

input ?

#

like ps2 keybaord driver ?

spring sonnet
#

Because I'd like to eventaully parse the disk

unreal creek
#

in a unix like OS

#

hard drive are expoxed under /dev/XXX

spring sonnet
unreal creek
spring sonnet
# unreal creek okay

I need to eventually start on the slab allocator.... and then implement kmalloc()... I beilieve from my studies kmalloc() should use virtual memory...

#

yup

spring sonnet
#

I'm going to bookmark that...

quiet valve
spring sonnet
quiet valve
spring sonnet
spring sonnet
#

Well... at least now I know interrupts can be triggered!

Screenshot_at_2025-03-24_21-54-52.png
#

It's a major work in progress. All day yesterday reading the GICv3 docs, and getting my header prepared. Today was getting things initialized, etc.

spring sonnet
#

Okay, so interrupts can't actually be triggered yet. But this is farther than I've gotten all day

unreal creek
spring sonnet
#

I've been having a hell of a time properly configuring the GIC, and if it's configured I can't seem to get IRQ's to forward to the CPU...

unreal creek
#

timer ?

#

keyboard ?

spring sonnet
# unreal creek IRQ of what

For right now UART (serial) interrupts, then I'd like to get a simple timer going. And after I planned to work on my slab allocator, and then implement kmalloc()

unreal creek
spring sonnet
# unreal creek okay

This GIC has not been easy to try and get going haha; Today's task will be to ensure I'm correctly initializing thing, and then test with a software generated interrupt to ensure the GIC is able to forward interrupts to the CPU interface

unreal creek
spring sonnet
spring sonnet
#

I was able to trigger an SGI after setting up the GIC, but I still ned to work out why the IRQ exception vector is never called

spring sonnet
#

I finally have a GICv3 driver in place!!!

spring sonnet
#

Got the UART IRQ's going!!! It turns out the UART IRQ was 33, and not 30!

#

Now I can start implementing the rest of my handler lol

#

@unreal creek Dude this is great hahah

spring sonnet
#

Currently implementing simple timer support up next... now that I've gotten the IRQ stuff going. I do need to get a slab allocator and kmalloc() implemented, but I'll get started after the timer stuff

#

Oooo maybe I can even get started on scheduling support too!!!

unreal creek
spring sonnet
unreal creek
spring sonnet
spring sonnet
#

I'll get it eventually. Just lots of debugging tonight haha

spring sonnet
#

Got priority handling in place, and making sure I set them properly for SGI's, PPI's, etc.

#

I had to set the UART IRQ to the lowst priority for now till I find the correct setting for it

#

I've set the timer IRQ to medium priority (0xA0) and am currently learnig more about how to confirure things.

#

So far so good though! The main thing I wanted to get done in my kernel was the ability to take IRQ's which that goal was achieved so I'm happy. Now I can get other things going

spring sonnet
#

Well this is progress on the timer! ISTATUS gets set, but the IRQ never fires still so I need to see what I'm not doing for this to happen.

Screenshot_at_2025-03-26_19-23-20.png
spring sonnet
#

It's been a pain in the ass debugging this timer not firing an IRQ...

#

It's the virtual timer too

spring sonnet
#

I dumped the device tree blob to examine for this....

#

Well I'm damn glad I did I can see the IRQ INTID's are different from what I thought they were.

spring sonnet
#

I'm able to set the timer, and ISTATUS get's set, but still no IRQ. Only the UART IRQ ends up firing.

Screenshot_at_2025-03-26_23-26-39.png
#

I'm not really sure why this is. I've been at it for two days now

unreal creek
spring sonnet
unreal creek
unreal creek
spring sonnet
unreal creek
spring sonnet
#

Limine seems to patch the libfdt for this stuff so I'll see what it patches, and work on that accordingly

spring sonnet
unreal creek
spring sonnet
#

But limine gives you a device tree BLOB

unreal creek
spring sonnet
# spring sonnet You have to decompile it

Indeed. And because in the kernel I still have yet to get my slab allocator, kmalloc() and vmm_allloc() in place using a library or parsing it from scratch an already be complex. So I just dumped it from QEMU, and decompiled it for examination

spring sonnet
#

But after making needed changes to my GIC I still can't get IRQ's to fire, and I still have an issue with the timer setting it's level back and forth from 1 to 0 at a rapid rate.

#

It's been really frustrating trying to get anything working to be honest.

#

Because before I made any of these changes the UART IRQ would fire, and now they don't.

spring sonnet
#

I'll continue to debug and work out my GIC setup.

spring sonnet
#

Yep I'm still not configuring things properly. So I had to go back to the docs to make sure the correct settings are used. Because my ass ended up setting things to reserved values lol

spring sonnet
#

So I've make sure to configure things for non-secure access, and make sure I've configures and set other things properly that previously were not set correctly. I still don't get interrupts from the timer, and I notice that despite disabling all IRQ's so that I can enable only what's needed IRQ's 0-33 all still seem to be "pending".

#

This is starting to piss me off a bit.

spring sonnet
#

My GIC semi-works. The distributor IRQ's (IRQ ID's >32) work, but I can't seem to get redistributor IRQ's to fire. The GIC redistributor appears to be awake after waking it and doing various checks to ensure of it.

#

I'm still debugging what could be going on. I'm wondering if there's somthing I didn't configure properly for this to occur. IRQ's <32 (in this case the timer IRQ 27) show as pending, but IRQ's never get fired.

spring sonnet
#

I GOT IT!!!

#

After some reasearch and digging... I see that when I set PMR to 0xFF to allow priorities reading this back shows 0xF8. RPR shows 0x00 (for highest level priorities). After setting my timer and UART IRQ's to be 0xFF or 0xF8 the timer IRQ fired!

#

I offically have a working GICv3 driver!!!

#

Took me a few days of debugging, but I finally got it.

#

Now I can start on a slab allocator, etc....

#

Holy fuck I feel good after getting this working!

#

And intresting thing I need to figure out though is how to not make the UART priority and timer priority interfear with each other. Because if I set both to have 0xFF (or 0xF8) since that's what the PMR was set to the timer IRQ fires. If I don't set a priority for UART then only the UART IRQ fires

#

Either way... at least interrupts work

lime sapphire
#

noice

spring sonnet
spring sonnet
#

Well it seems that was my only thing I need to ensure I handled when an interrupt was fired. I needed to check the higher priority register, and if there was an IRQ waiting to be handled I would do so. Otherwise I would fallback to the IAR, and handle the lower priority IRQ that way. With this all IRQ's now fire as expected!

frozen silo
spring sonnet
#

So now ICC_EOIR1_EL1 is setup and used to deactivate the IRQ and mark it as handled

frozen silo
#

did you also fix the icc ctlr defines that were for the el3 reg? I assume you also cleared the eoi bit in there (as if you don't then you have to write to both the eoi reg and the dir reg)

spring sonnet
#

Yes it should be good now

spring sonnet
#

I need to fix my bitmap allocator though. It seems that after enough allocations that I end up overwritting the bitmap it looks like. I needed to make adjustments anyways, but this gives me a perfect excuse for it.

unreal creek
spring sonnet
spring sonnet
#

Welp this is going to be a long day of rework. Looks like I broke some shit lol

spring sonnet
#

One thing I noticed that I never did do was to mark bits as in use for the bitmap itself.

spring sonnet
#

@unreal creek Hey, I've fixed some PMM stuff. I remember you wanted to have a look.

#

I also had to make sure upon setting up the bitmap, etc, that I was also reserving bits for the bitmap itself. That's something I previously wasn't doing

#

PMM should now be fixed. Now I ca get the slab allocator stuff going, but I'll probably spend most of my evening researching to understand not only how it works, but how it can be implemented. I was wanting to take a freelist approach on this as well.

#

Nope... still broken. It's weird because it get's used in the VMM when setting up and mapping page tables, etc. But when I try to call pmm_alloc() again after that I end up running into a level 3 page fault. So I'll have to spend some time figuring out wha'ts going on here.

#

GDB time!

spring sonnet
# 
(gdb) p/x hhdm
$1 = 0xffff000139f81000
(gdb) p/x hhdm->offset
Cannot access memory at address 0xffff000139f81008
(gdb)

Welp I found the issue for the crash. It seemes that after I've mapped my page tables, etc, and swapped them that this pointer ends up being NULL (or essentually unmapped).

spring sonnet
#

@frozen silo Hey, I noticed that when I try to clear interrupts through the EOIR that it says the IRQ isn't active? I was originally doing the ICPENDR for these

Screenshot_at_2025-03-28_19-27-18.png
#

But the IRQ's still fire it looks like.

spring sonnet
#

I do notice that I have to map each individual kernel section since trying to map the kernel and module region ends up with a translation fault

spring sonnet
#

I've decided to do slab sizes in a power of two starting at 8 bytes all the way to the max page size

unreal creek
spring sonnet
unreal creek
unreal creek
#

for if the irq is inferior to 32

spring sonnet
#

The EOIR thing I'm runnining into is probably due to how I configured it. I'll have a look though and get it fixed so it's out of the way

unreal creek
spring sonnet
#

Right now I'm trying to make sure I've configured things propely for the GIC to deactivate them after they've been fired

#

According to the GICv3 documentation here if I clear the EOI mode bit here it should allow ICC_EOIR1_EL1 to allow IRQ's to be priority dropped and deactivated which I assume is what I want here

Screenshot_at_2025-03-28_23-54-29.png
#

Otherwise I'd have to write to ICC_DIR_EL1 to deactivate the IRQ

#

Apparently that didn't end up working as expected. Both IRQ's get fired, acknowledged, and their respective handlers are called.

#

Hmm... I'll spend some time looking at this and see what's going on

spring sonnet
spring sonnet
#

Something I'm not understanding at all... the IRQ gets fired, but when I check the active status the IRQ in question isn't marked as in service (as the output in my terminal showed too).

#

I'm not sure if there's some configuration I missed or what, but that's kind of annoying.

spring sonnet
#

So the IRQ get's fired, I check if it's pending (it is), I read IAR to acknowledge the IRQ, and call the interrupt handler accordingly. After doing so I then wite EOIR1 with the same IRQ ID that read read from IAR to clear the IRQ.

#

But the IRQ is apparently never active even after acknowledging it....

spring sonnet
#

I could incorrectly be setting the groups and getting them backwards. I'm not sure if this has any effect on IRQ's not entering an active status though

frozen silo
spring sonnet
#

The issue I noticed though is that when acknowledging the IRQ from IAR or HPPIR it's never marked as active so I've been trying to figure out what especially might be causing this

spring sonnet
frozen silo
#

also when I tried to build it from the git but it looks like its missing VMM_RECURSIVE_BASE/VMM_RECURSIVE_IDX whatever those are

spring sonnet
spring sonnet
# frozen silo yeah the if is not really necessary at all, also in https://github.com/Professor...

When I had went back to read the documentation last night this is what I had ended up going with:

    // Set non-secure common binary point, end-of-interrupt, etc
    uint32_t ctlr = __icc_ctlr_read();
    ctlr &= ~(ICC_CTLR_CPBR_NS);        // ICC_BPR0_EL1 & ICC_BPR1_EL1 determines preemtion groups 0 & 1
    ctlr &= ~(ICC_CTLR_EOI_MODE);       // ICC_EOIR1_EL1 drops priority & deactivates IRQ
    __icc_ctlr_write(ctlr);
frozen silo
#

yeah that's fine

spring sonnet
#

Right now I'm debugging and seeing if there's anything I missed that would cause the read of IAR to not mark the IRQ as active

unreal creek
spring sonnet
unreal creek
spring sonnet
#

Then you have the 30 32-bit registers ranging from w0-w30

spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
#

Then I save the exception link register to the stack so I can return properly

unreal creek
spring sonnet
# unreal creek where ? 

.macro pusha null_arg
    stp x0, x1, [sp, #-16]!
    stp x2, x3, [sp, #-16]!
    stp x4, x5, [sp, #-16]!
    stp x6, x7, [sp, #-16]!
    stp x8, x9, [sp, #-16]!
    stp x10, x11, [sp, #-16]!
    stp x12, x13, [sp, #-16]!
    stp x14, x15, [sp, #-16]!
    stp x16, x17, [sp, #-16]!
    stp x18, x19, [sp, #-16]!
    stp x20, x21, [sp, #-16]!
    stp x22, x23, [sp, #-16]!
    stp x24, x25, [sp, #-16]!
    stp x26, x27, [sp, #-16]!
    stp x28, x29, [sp, #-16]!
    stp x30, xzr, [sp, #-16]!
.endm

.macro popa null_arg
    ldp x30, xzr, [sp], #16
    ldp x28, x29, [sp], #16
    ldp x26, x27, [sp], #16
    ldp x24, x25, [sp], #16
    ldp x22, x23, [sp], #16
    ldp x20, x21, [sp], #16
    ldp x18, x19, [sp], #16
    ldp x16, x17, [sp], #16
    ldp x14, x15, [sp], #16
    ldp x12, x13, [sp], #16
    ldp x10, x11, [sp], #16
    ldp x8, x9, [sp], #16
    ldp x6, x7, [sp], #16
    ldp x4, x5, [sp], #16
    ldp x2, x3, [sp], #16
    ldp x0, x1, [sp], #16
.endm

They get stored in pairs because the stack must always remain 16-byte aligned

unreal creek
spring sonnet
#

xzr is the zero register

unreal creek
#

15 is stack pointer ?

spring sonnet
#

Actually no x30 is the link register my bad

#

There's an sp register for stack pointer

#

Sorry my mad I'm still waking up

spring sonnet
unreal creek
#

and how i out to a port in aarch 64

#

sorry i'm new to arm

frozen silo
#

there are no io ports on aarch64 like there are on x86

#

only mmio

spring sonnet
#

In my case I've implemented a PL011 UART driver for the QEMU ARM64 virt board

spring sonnet
#

@frozen silo Well... I Just now managed to make the IRQ finally switch to an active state after a ton of debugging...

frozen silo
#

what did you change?

spring sonnet
# frozen silo what did you change?

https://pastebin.com/DQNXjREV

So I had read out HPPIR and IAR before and after the IRQ is fired. I see the first read is the IRQ ID which is expected, the next read after that is 1023 which that value after the read seems to always be there (apparently it's supposed to be zero?). But apparently after doing the two reads like that the IRQ's now enter an active state and can be cleared after servicing

Pastebin
void irq_handler(void) { int iar = __icc_iar1_read(); printf(...

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

#

I'm not sure why that worked, but it did

#

Because before this when I would check the the active status it would show it's still pending and was never moved to an active status

frozen silo
#

it looks to be working with just this thonk at least there aren't any more of those invalid irq messages

📎 diff.diff
spring sonnet
#

Thank you for the update as well on this. It's good to know it's working on your end at least so I was doing something incorrectly. I had suspected this to begin with

unreal creek
frozen silo
frozen silo
#

also the reason I removed the irq disable/enable in that patch was because its automatically done before the irq handler is called and re-enabled accordingly when you eret

spring sonnet
#

Ahhh that makes sense then; I thought I needed to handle this manually before I called the respective IRQ handler

spring sonnet
frozen silo
#

it might be that it's in there when you are handling some other irq and it happens at the same time or smth ig

#

but its going to be handled after you eret from the current one in that case

spring sonnet
#

That makes sense

#

I'll test with just handling IAR in this case then, and see if both IRQ's can be handled here since you mentioned that I don't really need to handle anything with the higher-priority IRQ register. Especially since PMR is set to allow all IRQ's

unreal creek
spring sonnet
# 
    int iar_irq_id = gic_ack_irq();

    if (iar_irq_id != 1023) {
        if (irq_handler_table[iar_irq_id] != NULL) {
            // Handle IRQ
            irq_handler_table[iar_irq_id]();
        } else {
            printf("IRQ ID %d has no handler registered!\n", iar_irq_id);
            return;
        }
    }

@frozen silo You're absolutely right! After making the adjustment it seems to work perfectly now.

#

I can also clear the EOIR and ht seems to properly drop the IRQ priority and deactivate it. The IRQ handler that was called though handles writing the EOIR with it's IRQ ID though. I'm not sure if I should do this within the IRQ exception handler itself instead of the IRQ handler for stuff like UART, timer, etc

frozen silo
#

doesn't really matter, ig it might be nicer to do it inside the irq exception handler after the actual irq handler has been called as you have the irq id from the iar available

spring sonnet
#

That's what my thought was as well. I think I'll just go ahead and do that instead

# 

void irq_handler(void) {
    int iar_irq_id = gic_ack_irq();

    if (iar_irq_id != 1023) {
        if (irq_handler_table[iar_irq_id] != NULL) {
            // Handle IRQ
            irq_handler_table[iar_irq_id]();
        } else {
            printf("IRQ ID %d has no handler registered!\n", iar_irq_id);
            return;
        }
    } else {
        printf("No valid IRQ ID's found!\n");
        return;
    }

    gic_clear_irq(iar_irq_id);
}

This is so much cleaner than the ugly shit I was doing before...

#

Works like a charm!

#

Time to commit the changes...

spring sonnet
#

Now that I have that fixed I can get back to the slab allocator...

spring sonnet
#

I'm really glad to have the IRQ handling in place haha

#

@unreal creek If you need to use it as a base to implement your interrupt handling feel free. Be sure to read the GICv3 docs though to understand what it is that each part does and how it works though...

#

It's not difficult to implement, but debugging can take some time haha

#

it wasn't until today I understood how the Linux vmalloc() functions...

#

Later down the road I'll want to implement that too

spring sonnet
#

The slab allocator is coming along well. Currently working on the first tests of allocation, and working out the bugs. I have no ability to free at the moment, but that will come likely tomorrow depending on how far I get tonight on this.

I also have a simple kmalloc(), kcalloc(), and kfree()

void *kmalloc(size_t length) {
    void *ptr = NULL;

    if (length <= PAGE_SIZE) {
        ptr = slab_alloc(length);
        assert(ptr != NULL);

        return ptr;
    }

    return NULL;
}

void *kcalloc(size_t length) {
    void *ptr = NULL;

    if (length <= PAGE_SIZE) {
        ptr = slab_alloc(length);
        assert(ptr != NULL);

        memset(ptr, 0, length);
        return ptr;
    }

    return NULL;
}

void kfree(void *ptr) {
}

As I said though no way to free tonight, but I planned to have these use the eventual vmm_alloc() for allocations that exceed the page size

#

Simple, and functional

unreal creek
spring sonnet
unreal creek
spring sonnet
#

It also supports legacy mode as well

unreal creek
spring sonnet
unreal creek
spring sonnet
# unreal creek okay

But my current implemenations works quite will for it's use case, and should handle IRQ's just fine

#

I've tested this with a simple timer and the PL011 UART device on QEMU ARM64 virt

spring sonnet
#

This is the best part of debugging in vscode is being able to readout struct and other variables with ese while being able to follow the code, and also read the disassembled view

Screenshot_at_2025-03-30_00-25-43.png
#

This has helped me to better catch issues in my code rather than do a blind run and hope it works

#

Like right now with a NULL pointer dereference.

Screenshot_at_2025-03-30_00-34-20.png
spring sonnet
#

After I fix this bug I'm going to bed.... it's been a long day of debugging and implementing of a slab allocator. It's quite simple too, but also doesn't really need to be complex

#

First successfull kmalloc() 🎉

Screenshot_at_2025-03-30_01-02-34.png
#

Make that two..

Screenshot_at_2025-03-30_01-06-17.png
#

Needs work of course, but this is pretty cool!

unreal creek
#

on what machine ??

spring sonnet
# unreal creek how do i run your operating system in qemu ?

Here's the run script that I use for it:

#!/bin/sh


OUT_DIR=${HELIX_BINARY_DIR}

EFI_ROM=${HELIX_BINARY_DIR}/efi_rom.img
EFI_VARS=${HELIX_BINARY_DIR}/efi_vars.img
HDD_IMG=${HELIX_BINARY_DIR}/helix_hdd.img

QEMU=qemu-system-aarch64

MACHINE=virt,gic-version=3
CPU=cortex-a53
SMP=1
MEM=4G



${QEMU} \
    -d guest_errors,cpu_reset,mmu \
    -M ${MACHINE} \
    -cpu ${CPU} \
    -smp ${SMP} \
    -m ${MEM} \
    -drive if=pflash,file=${EFI_ROM},format=raw,readonly=on \
    -drive if=pflash,file=${EFI_VARS},format=raw,readonly=off \
    -drive if=virtio,file=${HDD_IMG},format=raw \
    -usb \
    -device qemu-xhci \
    -device usb-kbd \
    -device usb-mouse \
    -net none \
    -device ramfb \
    -display gtk,zoom-to-fit=off
unreal creek
spring sonnet
unreal creek
#

i copied the BOOTAA64.EFI to the good place

#

but it don't do anything when i launch it

static shore
unreal creek
#

limine-bios.sys

#

and limine-uefi-cd.bin

static shore
#

well, did you run the cmd?

#

and do you have the limine.conf file?

unreal creek
unreal creek
static shore
#

limine bios-install <path to device/image>

#

this?

#

can you show your xorriso command too?

unreal creek
spring sonnet
# unreal creek yes 
#!/bin/sh



DISK_IMG=${HELIX_BINARY_DIR}/helix_hdd.img
EFI_IMG=${HELIX_BINARY_DIR}/efi_rom.img
EFI_VARS=${HELIX_BINARY_DIR}/efi_vars.img

HDD_BLOCK_SIZE=512

EFI_PARTITION_START=2048
EFI_PARTITION_OFFSET=$((${HDD_BLOCK_SIZE} * ${EFI_PARTITION_START}))






[ ! -e ${DISK_IMG} ] && qemu-img create -f raw ${DISK_IMG} 4G

[ ! -e ${EFI_IMG} ] && qemu-img create -f raw ${EFI_IMG} 64M && dd if=${HELIX_EXT_DIR}/qemu_arm64_efi.fd of=${EFI_IMG} conv=notrunc
[ ! -e ${EFI_VARS} ] && qemu-img create -f raw ${EFI_VARS} 64M



sgdisk \
    --new 1:0:+512M \
    --typecode 1:"ef00" \
    --change-name 1:"EFI" \
    ${DISK_IMG} >/dev/null 2>&1


mformat \
    -T 1048576 \
    -h 32 \
    -s 63 \
    -H 0 \
    -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ::/


mmd -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ::/EFI
mmd -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ::/EFI/BOOT

mmd -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ::/HelixOS
mmd -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ::/HelixOS/images
mmd -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ::/HelixOS/fonts

mcopy -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ${HELIX_EXT_DIR}/limine/BOOTAA64.EFI ::/EFI/BOOT/BOOTAA64.efi
mcopy -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ${HELIX_SOURCE_DIR}/limine.conf ::/EFI/BOOT/

mcopy -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ${HELIX_BINARY_DIR}/kernel/kernel.elf ::/HelixOS/
mcopy -i ${DISK_IMG}@@${EFI_PARTITION_OFFSET} ${HELIX_EXT_DIR}/images/default.bmp ::/HelixOS/images/boot_splash.bmp

This is my install script

static shore
#

that won't make an iso tho

spring sonnet
#

I boot directly off the disk itself rather than an ISO

static shore
#

oh

#

i prefer iso bc i can launch them on other hardware too

unreal creek
static shore
#

?

unreal creek
#

i use that : rm -f ${iso_out} xorriso -as mkisofs -R -r -J -b boot/limine/limine-bios-cd.bin \ -no-emul-boot -boot-load-size 4 -boot-info-table -hfsplus \ -apm-block-size 2048 --efi-boot boot/limine/limine-uefi-cd.bin \ -efi-boot-part --efi-boot-image --protective-msdos-label \ ${OUT} -o ${iso_out} ./limine/limine bios-install ${iso_out}

static shore
#

what is the out dir set to?

spring sonnet
#

If your booting ARM64 off an ISO you'll probably need to make adjustments for ISO booting. I just boot from the disk directly.

static shore
#

i can send you my qemu cmd for iso if you want

#

-cpu cortex-a76 -smp 10 -M virt -m 8G -usb -device qemu-xhci -device usb-kbd -device usb-mouse -device ramfb -device ahci,id=ahci -device ide-hd,drive=hdd0 -drive if=pflash,unit=0,format=raw,file=${OVMF_FILE} -drive file=${ISO_OUTPUT},if=none,id=hdd0

#

my qemu flags

unreal creek
#

btw

#

qemu show quest as not initialzed the display (yet)

static shore
#

oh

spring sonnet
static shore
#

do you have -s -S flags?

unreal creek
static shore
#

launch your qemu

#

and uncheck pause

unreal creek
#

qemu-system-aarch64 -M virt -cdrom cd.iso -device ramfb

#

this is my flags

unreal creek
static shore
#

try my flags

unreal creek
#

i can see the value of reg 15 is changing

unreal creek
static shore
#

i am using the ovmf bios btw

#

this one

unreal creek
#

i didn't give it a bios

static shore
#

ovmf is for uefi

unreal creek
static shore
#

?

unreal creek
# static shore ?

qemu-system-aarch64: device requires 67108864 bytes, block backend provides 33554432 bytes

static shore
#

run this cmd: dd if=/dev/zero of=${OVMF_FILE} bs=1 count=0 seek=67108864

#

this should increase the size to what it wants

unreal creek
#

now it boot

#

and show not bootable device was found

static shore
#

so the iso is not being recognised as bootable

#

can i have that iso pls

#

i'll try to boot it

spring sonnet
static shore
#

the dd is for the bios

#

i use it to increase the size of the file

spring sonnet
#

If your booting ARM64 UEFI you'll want to follow my script for creating the ROM disk images. The UEFI firmware you can pull from my repo but can also be installed via sudo apt install qemu-arm64-efi and just do:

dd if=<path/to/QEMU_EFI.fd> of=efi.img conv=notrunc

unreal creek
spring sonnet
unreal creek
spring sonnet
static shore
#

he increased it and it works, but the iso is not bootable somehow

unreal creek
#

and it boot

#

but once the UEFI boot

#

it say no bootable device found

spring sonnet
static shore
#

i tried to boot the iso but it is not recognising it

spring sonnet
unreal creek
static shore
#

fixed it

#

boots now

#

-cpu cortex-a76 -smp 10 -M virt -m 8G -usb -device qemu-xhci -device usb-kbd -device usb-mouse -device ramfb -device ahci,id=ahci -drive if=pflash,unit=0,format=raw,file=${OVMF_FILE} -cdrom ${ISO_OUTPUT}

#

@unreal creek

#

this will boot that iso

spring sonnet
#

Slab allocations are working quite well. Testing for different sizes including the largest allowed to test.

Screenshot_at_2025-03-30_11-48-15.png
#

Now to work on freeing.

#

This.... is going to be a long day

unreal creek
unreal creek
spring sonnet
spring sonnet
#

krealloc() works like a charm too!!

Screenshot_at_2025-03-30_14-22-21.png
spring sonnet
#

Finally have the ability to free objects from the slab...

Screenshot_at_2025-03-30_19-05-46.png
spring sonnet
#

Next up is a VFS and tmpfs

#

Maybe I'll make it ramfs

unreal creek
unreal creek
spring sonnet
#

I gotta fix some bugs in my slab allocator it looks like. Just tried to allocate a pointer within an allocated pointer and slab_alloc() just shit itself

spring sonnet
#

I'm glad I use asserts all over the place in my kernel. I found the bug, and noticed when I'm updating the pointer to the next free object in the existing slab cache I end up causing it to become NULL

#

Reworking the slab allocation from list.

worthy lynx
#

epic

spring sonnet
#

Fixed my slab allocator

unreal creek
#

that work

spring sonnet
#

I had pulled an all nighter debugging my slab allocator, and had to rework a few things. It works really well now, but before I had a bug where my freelist would become corrupted 😅

unreal creek
spring sonnet
#

I had ran into that issue at one point too

unreal creek
spring sonnet
# unreal creek yes i copied the struct 
typedef struct {
    uint32_t    uart_dr;                // 0x000 - UART data register
    uint32_t    uart_rsr_ecr;           // 0x004 - UART status/error clear register
    uint8_t     __reserved1[16];        // 0x008 - 0x018 reserved
    uint32_t    uart_fr;                // 0x018 - UART flag register
    uint32_t    __reserved2;            // 0x01C - reserved
    uint32_t    uart_lpr;               // 0x020 - UART low-power counter register
    uint32_t    uart_ibrd;              // 0x024 - UART integer baudrate register
    uint32_t    uart_fbrd;              // 0x028 - UART fractal baudrate register
    uint32_t    uart_lcrh;              // 0x02C - UART line control register
    uint32_t    uart_cr;                // 0x030 - UART control register
    uint32_t    uart_ifls;              // 0x034 - UART interrupt FIFO level select register
    uint32_t    uart_imsc;              // 0x038 - UART interrupt mask set/clear register
    uint32_t    uart_ris;               // 0x03C - UART raw interrupt status register
    uint32_t    uart_mis;               // 0x040 - UART masked interrupt status register
    uint32_t    uart_icr;               // 0x044 - UART interrupt clear register
    uint32_t    uart_macr;              // 0x048 - UART DMA control register
} __attribute__((packed)) uart_t;

The reserved value should be 16 bytes otherwise it's incorrectly offset

unreal creek
spring sonnet
unreal creek
#

i'm stupid

#

i forgot to apply hhdm

spring sonnet
#

So writes to them in Limines page tables causes a translation fault

unreal creek
#

mmm...

spring sonnet
unreal creek
#

after fixiing the two it still don't work

unreal creek
spring sonnet
# unreal creek yes

Then yes you'll need to remap the UART, GIC, and other stuff in your own page table to make use of them. Limine maps these as reserved initially, but once you remap them it'll work fine

unreal creek
spring sonnet
#

I've remaped my MMIO at the beginning of 0xffff00000000000000

spring sonnet
spring sonnet
unreal creek
#

anyway thanks

spring sonnet
# unreal creek anyway thanks

Of course! Also, yes I strongly recommend getting paging in place as soon as you can. Since Limine already has the correct configuration setup all you have to do is re-create your own page table. I just basically re-mapped everything, and mapped my kernel sections in

unreal creek
unreal creek
spring sonnet
spring sonnet
unreal creek
#

i have a vmm allocator

unreal creek
#

cause it load font from initrd

spring sonnet
#

Right now I'm getting the vfs, and ramfs in place. Then I can start working on a tiny ramdisk and implement /dev

spring sonnet
#

It's nice to see folks take interest in my project. It makes me feel pretty good haha

#

Anyone is welcome to fork it and make their own changes

spring sonnet
#

Looks like my slab allocator needs some work again. Right now larger sizes being allocated causes an exception so I need to debug and see why this might be happeneing...

#

Ahh I already see the bug. I need to fix not creating the other size nodes instead of just 8-bytes.

#

That's alright! I'll spend the day implementing that

#

Something I don't currently do is that if an slab is found, but the size isn't the correct size I'll probably want to handle creating a new slab of that object size and link it into the existing cache...

spring sonnet
#

Welp I fixed the new slab allocation. I was incorrectly caclulating the number of objects of the given length which resulted in it being zero

#

Yep that was the bug I had, and it's now fixed. Yay!!!

spring sonnet
#

Welp.... another bug manifested itself... it seems now when I try to aquire the spinlock during slab_alloc() it ends up in an infinite loop. Not kernel exception or anything...

spring sonnet
#

I think each slab cache should have it's own lock. Not sure if this approach is good at all, but we'll see...

#

Oh shit I seen that after creating a new slab, and linking it to the existing cache I wasn't releasing the lock....

spring sonnet
#

Okay I think it's fixed again 😂

#

Had to make sure I was releasing the lock before leaving the context of slab_alloc()

spring sonnet
#

VFS development can continue...

unreal creek
spring sonnet
# unreal creek good

Had to fix a bug in my slab allocator where I wasn't releasing the lock, and caused a deadlock

unreal creek
spring sonnet
unreal creek
#

vnode and that stuff

spring sonnet
#

Sort of.... it'll be it's own little implementation

unreal creek
spring sonnet
unreal creek
spring sonnet
# unreal creek okay 
typedef struct vfs_inode_opts vfs_inode_opts_t;
typedef struct vfs_file_opts vfs_file_opts_t;
typedef struct vfs_file vfs_file_t;


#define VFS_MAX_NAME_LEN    64




typedef struct vfs_dirent {
    const char          d_name[VFS_MAX_NAME_LEN];
    vfs_inode_opts_t    *inode;
    struct vfs_dirent   *next;
} vfs_dirent_t;

typedef struct vfs_inode_opts {
    uint64_t            uid, gid;
    uint64_t            atime, ctime, mtime;
    uint64_t            mode;
    size_t              length;
    uint64_t            ref_count;
    vfs_inode_opts_t    *parent;
    vfs_inode_opts_t    *children;
    vfs_inode_opts_t    *siblings;
    vfs_dirent_t        *dirent;
    int                 (*create)(vfs_inode_opts_t *root, uint64_t mode);
    vfs_inode_opts_t    *(*lookup)(vfs_inode_opts_t *root, const char *path);
} vfs_inode_opts_t;

typedef struct vfs_file {
    vfs_inode_opts_t    *inode;
    uint64_t            flags;
    uint64_t            offset;
} vfs_file_t;

typedef struct vfs_file_opts {
    vfs_file_t *(*fs_open)(const char *name, uint64_t flags);
    size_t (*fs_read)(vfs_file_t *file, void *buf, size_t length);
    size_t (*fs_write)(vfs_file_t *file, void *buf, size_t length);
    int (*fs_close)(vfs_file_t *file);
    int (*fs_seek)(vfs_file_t *file, uint64_t offset);
    int (*fs_truncate)(vfs_file_t *file, size_t length);
    int (*fs_mkdir)(vfs_file_t *file, const char *path);
    int (*fs_rmdir)(vfs_file_t *file, const char *path);
} vfs_file_opts_t;

typedef struct vfs_fs_type {
    char                *name;
    vfs_file_opts_t     *fs_opts;
    struct vfs_fs_type  *next;
} vfs_fs_type_t;

typedef struct vfs_mount {
    char                *name;
    vfs_inode_opts_t    *inode_opts;
    vfs_file_opts_t     *fs_opts;
    struct vfs_mount    *next;
} vfs_mount_t;

So this is the initial implementation I think I'll be going with.

#

I want to ensure that it supports unix path's, etc, but implemented in my own code of course.

#

Thank God I have a working kmalloc() for this lol

spring sonnet
#

Been trying to study and readup about the virtual filesystem design as well as inodes, etc.

#

I planned to implement a simple ramfs filesystem to get things going.

#

Just setting up the initial root vfs node...

spring sonnet
#

Wrapping my head around how this is going to work though is going to drive me a bit insane...

#

Mainly because I'm using linked lists for the VFS implementation

willow prism
unreal creek
spring sonnet
spring sonnet
unreal creek
willow prism
#

A tree based vfs is not bad but using vnodes as the nodes you link into the tree is bad

#

Because of hard links

#

A tree based vfs where the nodes are something else, usually called "cached directory entries" or "dirents" or whatever, and these merely reference the vnodes (and multiple dirents can reference the same vnode bc of hard links), is exactly the most modern design in use

#

You can see it in both xnu and linux

#

It's known as a dirent cache or name cache

spring sonnet
spring sonnet
# willow prism You will encounter issues 
typedef struct vfs_vnode {
    uint64_t            uid, gid;
    uint64_t            atime, ctime, mtime;
    uint64_t            mode;
    size_t              length;
    uint64_t            ref_count;
    struct vfs_vnode    *parent;
    struct vfs_vnode    *children;  // First child for directories
    struct vfs_vnode    *siblings;  // Next file in same directoy
    vfs_file_opts_t     *fs_opts;
} vfs_vnode_t;

typedef struct vfs_file {
    vfs_vnode_t         *inode;
    uint64_t            flags;
    uint64_t            offset;
} vfs_file_t;

typedef struct vfs_file_opts {
    vfs_file_t *(*fs_open)(const char *name, uint64_t flags);
    size_t (*fs_read)(vfs_file_t *file, void *buf, size_t length);
    size_t (*fs_write)(vfs_file_t *file, void *buf, size_t length);
    int (*fs_close)(vfs_file_t *file);
    int (*fs_seek)(vfs_file_t *file, uint64_t offset);
    int (*fs_truncate)(vfs_file_t *file, size_t length);
    int (*fs_mkdir)(vfs_file_t *file, const char *path);
    int (*fs_rmdir)(vfs_file_t *file, const char *path);
    int (*fs_create)(const char *path, uint64_t mode);
    struct vfs_vnode *(*fs_lookup)(vfs_vnode_t *root, const char *path);
} vfs_opts_t;

typedef struct vfs_mount {
    char                *name;
    vfs_vnode_t         *mounts;
    struct vfs_mount    *next;
} vfs_mount_t;

Here's a more cleaned up version of this, but using the vnode approach. Still a work-in-progress of course

unreal creek
spring sonnet
unreal creek
#

so you can do c ssize_t vfs_read(vfs_node *node,void *buffer,size_t count){ if(node->read){ return node->read(node,buffer,count); } return -EBADF }

#

trust me i will make everything easier

#

so for exmple your tmpfs function can look like ssize_t tmpfs_read(vfs_node *node,void *buffer,size_t count){ //normally the inode componoent is just a void * and is used to store fs specific info tmpfs_inode *inode = node->inode; //do the actual read ... }

#

@spring sonnet

spring sonnet
#

or in the filesystem call vfs_read()? That's the part I'm having a bit of a hard time with

unreal creek
unreal creek
# spring sonnet I'm still trying to wrap my head around how vfs_read works with other read imple...

if you want check my vfs it's really simple (not unix like it use mutliple root like window but with string instead of letter (eg : "cdrom:/file.txt") but most of it can be apply to a unix vfs (except for mount stuff))
https://github.com/tayoky/stanix/blob/main/kernel%2Ffs%2Fvfs.c

GitHub
stanix/kernel/fs/vfs.c at main · tayoky/stanix

An 64 bit OS made from scratch. Contribute to tayoky/stanix development by creating an account on GitHub.

spring sonnet
#

Sounds good

unreal creek
# spring sonnet Sounds good

using your struct (with the modification i described) you want something like that for all vnode function

ssize_t vfs_read(vfs_node *node,void *buf,size_t count){
    //.. what i described erlaier
}
#

then what you want for opening is just doing a bunch of lookup

#

you cut the path into an array

#

first you alaway save the root node

#

imagine you have /tmp/fol/test.txt

#

you cut into tmp fol and test.txt

#

you take the root node

#

and do lookup tmp on it

#

you get the vnode for /tmp then you do lookup fol on it and you get the vnode for /tmp/fol then you do lookup test.txt on it and voila you get your node

spring sonnet
#

Sounds good to me..

#

It looks pretty staright forward

#

I'm just having a hard time grasping how vfs_read(), vfs_write(), and the rest of the family are called/used if a filesystem registers it's down read, write, etc, functions

#

I'm sure I'll understand as I research more about it today

unreal creek
#

like that

#

and then ```c
ssize_t vfs_read(vfs_node *node,void *buffer,size_t count){
return node->opts->read(node,buffer,count)
}

spring sonnet
#

What would call vfs_read() in this case though?

#

Since you mentioned filesystems don't call vfs_read(), and the likes manually

#

I'll spend some time studying this...

unreal creek
#

and then you use the vfs like that ```c
vfs_node *file = vfs_open("/tmp/test.txt");
char buf[256];
vfs_read(file,buf,256);

#

@spring sonnet

spring sonnet
unreal creek
spring sonnet
#

Thank you good sir for helping me better understand how this should work

unreal creek
spring sonnet
#

Ahhh now it's making more sense to me now...

worthy lynx
unreal creek
#

but that when your register a vfs

spring sonnet
spring sonnet
#

Trying to decide how I shouild implement vfs_lookup(). Been giving myself a headache with this today.

spring sonnet
#

I'm considering moving the mount names into the vnode structure. Right now I'm having a difficult time trying to figure out how I'll imlement searching child's sibling nodes for sibdirectories, etc.

spring sonnet
#

This is a lot more tough than I expected. Since vnodes don't store their own names I have to use other methods of traversing child nodes siblings.

spring sonnet
#

A method I thought of is to have the file opts struct hold a directoy entry of the subdir path. Because I can't think of how else I'd get the next path of something or the name of a file without this. I'm aware to also check whether this path is a directoy, file, block special, etc.

# 

typedef struct vfs_dirent {
    char                name[VFS_MAX_NAME_LEN];
    uint64_t            mode;
    vfs_vnode_t         *vnode;
    struct vfs_dirent   *next;
} vfs_dirent_t;

The idea here is each node subpath from the initial mount path (e.g. /mnt, /root, etc) will have it's path stored here, and also have it's curresponding vnode stored here as well.

#

This is probably whay I should of been doing anyways, but hey better now than never.

tender acorn
#

is the node in another allocation?

spring sonnet
#

Oh absolutely! Linked lists was the way to go for this haha

tender acorn
#

for my "device tree" that i was making the device itself onto the same page allocation lol

#

just to waste less

#

well in this case it would make sense to have it on a different one tho

spring sonnet
#

For allocations I Just use my kmalloc() which in turn calls slab_alloc(), but the current slab_alloc() implementation is pretty good. If the size needed isn't already in the cache a new page frame from the PMM is allocated, and I diivide that space up into what size is needed, then link it to the cache... then the next time I need that same size request again I can just return an object from the cache

#

It tries really hard to be conservative about memory

willow prism
tender acorn
#

oh so it already has a granularity of 1 byte lol

spring sonnet
spring sonnet
tender acorn
#

im reworking my whole os lol

#

i deleted 99% of the code

spring sonnet
#

Holy shit man lol

tender acorn
#

and changed a shit ton of stuff

#

for example

#

now types will not use the EFI names

#

which are on all caps

spring sonnet
tender acorn
#

thanks

#

i kinda got confused getting info for making one lol

spring sonnet
#

I do need to improve upon it, but you're free to check it out and see how it's done

#

Slab allocators are not so bad to implement. It's when you start implementing the virtual file system is when shit gets real lmao

#

I better commit that change real quick for it to create the other slab objects in the cache instead of just the 8 byte one. That way other sizes exist too lol

#

But it's a great way to allocate byte size allocations and not have to waste a whole page of memory for a tiny allocation

spring sonnet
# tender acorn thanks

I permit you to use my code as your base if needed as well. Just don't forget to credit me haha

tender acorn
#

yeah ill probably roll out my own lol

#

i just have the tendency to make my functions have weird but fast and less abstracted inputs

#

so most of my stuff is highly incompatible with other code

spring sonnet
#

Sounds like a plan to me lol

spring sonnet
#

I've not a clear idea of how I'll handle linking subpath inodes to their mounts...

#

Need to take time to think about the design as well.

spring sonnet
#

Finally getting some progress made tonight on this VFS.

#

I've learned a hell of a lot so far, and had to make some adjustments for keeping track of subpaths, but so far it's going well.

#

I think making my VFS make use of linked lists was a good idea. Thinks can be created/removed dynamically

#

I wanted to make sure I kept the unix path support since I planned to eventully port programs, etc, but I wanted to have my very own implementation of this.

#

I also fixed my slab allocator once again not creating the other cache node sizes...

unreal creek
unreal creek
#

you have the root node (for exmple an tmpfs thing)

#

and if you want to mount something to /tmp for exmple

#

the filesystem give you a vnode that represent the local root

#

then you open /tmp,
and do mount on it

spring sonnet
#

I have a question... does the vnode need to store the directory entries at all? Because right now I really have no idea how I'd store those without making a seperate global variable for them

unreal creek
# 
int vfs_mount(const char *path,vfs_node *local_root){
    vfs_node *mount_point = vfs_open(path);
    if(!mount_point)return -ENOENT;
    return mount_point->opts->mount(mount_point,local_root);
}
vfs_node *local_root = new_tmpfs();
vfs_mount("/tmp",local_root);```
unreal creek
frozen silo
# 
vnode_t* tmpfs_lookup(vnode_t* node, const char* component) {
    tmpfs_node_t* tmpfs_node = (tmpfs_node_t*) node->data;
    tmpfs_node_t* next = tmpfs_node_dir_hash_map_lookup(tmpfs_node, component);
    if (next) {
        vnode_ref(next);
    }
    return next;
}

vnode_ops_t tmpfs_ops = {
    .lookup = tmpfs_lookup
};
``` could be how the lookup could be implemented on a tmpfs (and in a create() op you'd create a new vnode and insert it to the hash map of the parent tmpfs node with the given name)
unreal creek
#

@spring sonnet vfs_lookup is basicly just c vfs_node *vfs_lookup(vfs_node *node,const char *name){ if(node->opts->lookup){ retutn node->opts->lookup(node,name); } return -ENOTDIR; }

spring sonnet
# 

typedef struct vfs_dirent {
    char                name[VFS_MAX_NAME_LEN];
    uint64_t            mode;
    vfs_vnode_t         *vnode;
    struct vfs_dirent   *next;
    struct vfs_dirent   *prev;
} vfs_dirent_t;

typedef struct vfs_file {
    vfs_vnode_t         *inode;
    uint64_t            flags;
    uint64_t            offset;
} vfs_file_t;

typedef struct vfs_vnode {
    uint64_t            uid, gid;
    uint64_t            atime, ctime, mtime;
    uint64_t            mode;
    size_t              length;
    uint64_t            ref_count;
    struct vfs_vnode    *parent;
    struct vfs_vnode    *children;
    struct vfs_node     *siblings;
    vfs_file_opts_t     *fs_opts;
} vfs_vnode_t;

typedef struct vfs_file_opts {
    vfs_file_t *(*fs_open)(const char *name, uint64_t flags);
    size_t (*fs_read)(vfs_file_t *file, void *buf, size_t length);
    size_t (*fs_write)(vfs_file_t *file, void *buf, size_t length);
    int (*fs_close)(vfs_file_t *file);
    int (*fs_seek)(vfs_file_t *file, uint64_t offset);
    int (*fs_truncate)(vfs_file_t *file, size_t length);
    int (*fs_mkdir)(vfs_file_t *file, const char *path);
    int (*fs_rmdir)(vfs_file_t *file, const char *path);
    int (*fs_create)(const char *path, uint64_t mode);
    struct vfs_vnode *(*fs_lookup)(const char *path);
} vfs_opts_t;

typedef struct vfs_mount {
    char                *name;
    vfs_vnode_t         *mounts;
    struct vfs_mount    *next;
} vfs_mount_t;

This is the current structure I've been trying to work with. I thought I needed to keep track of subpaths with a directory entry.

#

RIght now I've not got some simple code in place to create nodes, and create a new mount. I've been trying to figure out a way to keep track of suubpaths from the root mount.

#

Unless I'm going about this the entirly wrong way.

lime sapphire
#

if you havent already, I recommend reading the vnodes paper, there's some good info in there

lime sapphire
spring sonnet
spring sonnet
lime sapphire
spring sonnet
#

Well I found the vnode paper I believe you mentioned

#

Oooo this paper loos good so far

spring sonnet
#

I think while I'm at it I might want to also work on my VMM a bit. I don't really want to be mapping the entire physical memory, and would like to allocate pages when needed.

#

I've been needing to do this anyways, and I also need to work on my exception handler as well.

#

So I think before I start thinking about the VFS stuff I better get this going first

#

I decided to also store my page tables at a reserved physical region and map them to a virtual address range. This will allow me to handle page table mappings, etc, a little easier than what I'm currently doing.

#

I need to also implement a page fault handler as well. I did mention that my exception handler needs work haha

spring sonnet
#

I'll still study the vnode stuff while I'm working on my VMM stuff. Just to gain some knowledge ahead of time

#

Alright changing to using a struct to keep track of page tables didin't break anything!!

#

I honestly expected that to break

#

I think I may also want to consider constructing my own memory layout too. And decide how I'll use some of my virtual addreress space for the kernel.

unreal creek
#

you don't need to

#

btw tve node don't need to keep track of parent or child

spring sonnet
unreal creek
# spring sonnet Well that will make things a bit easier to manage then. Right now I'm re-working... 
struct vfs_node;
typedef vfs_opts {
    struct vfs_node *(*lookup)(struct vfs_node *,const char *);
    ssize_t (*read)(struct vfs_node *,void *,size_t);
    int (*mount)(struct vfs_node *,struct vfs_node *);
//...
} vfs_opts_t;

typedef struct vfs_mount {
    struct vfs_node *root;
    struct vfs_mount *next,*prev;
} vfs_mount_t;

typedef struct vfs_node {
    vfs_opts_t *opts;
    void *inode;
    vfs_mount_t *mount_point;
} vfs_node_t;```
that the most basic stuff
spring sonnet
unreal creek
#

i can explain the rest

#

i'm pretty sure this thread as all the code to make a fully fonctionnal vfs

spring sonnet
#

My VMM is undergoing a serious overhaul...