#HelixOS (ARM64 OS, not dead project!)

1 messages · Page 3 of 1

spring sonnet
#

Welp the caused the virtio driver to shit itself... I needed to fix this anyways so this gave me a great excuse

spring sonnet
#

Well I now know what the bug is.... because of the face I've placed the heap at the end of the kernel, and map it to 0xFFFF800000000000, and the kernel is placed at -2GB... by HHDM offsetting no longer works in the traditional way. So I might want to tweak a few things or just place the heap elsewhere

spring sonnet
#

Going to work on a much needed proper virt2phys funciton to handle the job. I didn't like the other hackish idea I had for this...

hallow comet
#

Good luck!

spring sonnet
# hallow comet Good luck!

Thank you good sir! I've got that implemented, and now I get the proper physical address that I need instead of doing my semi-shitty HHDM virt2phys lol

covert kayak
#

I still don't, for me so far just slab allocator and a page frame allocator have been enough

#

chunk list allocator is best

willow prism
covert kayak
spring sonnet
# 
qemu: Guest says index 65535 is available

Go figure this starts to happen after I start deciding to use the heap....

hallow comet
#

of course XD

spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
#

My previous virt2phys was a little shitty lol

unreal creek
spring sonnet
# unreal creek okay

Have to fix the virtio driver to use this now too or really anything that makes used of kmalloc now that needs the physical base of something. Otherwise I get some ridiculous address value.

unreal creek
#

and then you work on the vfs ?

spring sonnet
#

Now that I've got a better allocator in place I feel better about starting the VFS

#

I got rid of the slab allocator too it served it's purpose

spring sonnet
#

I don't know what or why now all of a sudden QEMU is saying:

qemu: Guest says index 65535 is available

I've been debugging what might be wrong, and it seems to happen either after I signify the queue that in ready, and update the ring index.

#

And what really irritates me is the fact it decides to manifest after I switch to a heap allocator. I mapped a 16MB range just after the kernel an mapped it to 0xFFFF800000000000 (yes it's contiguous). Apparently it doesn't like that.

spring sonnet
#

So the rest of the virtio driver will use the kernel heap, but the queue itself for now is going to fall back to the PMM. At least till I can figure out why switching to the heap is causing this to fail

#

Annnnnnd new heap bugs have manifested..... apparently it thinks it's already out of memory....

hallow comet
#

Mmm, more bugs, everyone's favorite lol

spring sonnet
#

I think my issue though is my heap currnetly doesn't align anything... and one of the heap blocks that get allocated end up corrupted too

#

Oh God nooo

Screenshot_at_2025-04-24_09-40-31.png
#

Probably overwriting the metadata causing this to happen in the first place... so I need to make sure I'm aligning things properly

spring sonnet
#

I guess 16MB if heap clearly wasn't enough... so I increased it to 256MB

#

I need to work on making sure the returned address is correct for each block, and that I properly free, and coalesce the freed blocks to prevent fragmentation

unreal creek
spring sonnet
spring sonnet
#

Right now I'm trying to work on splitting available regions or just allocating whole blocks, and ensureing they're aligned properly

unreal creek
spring sonnet
# unreal creek mmm... okay

I barely started workin on the new heap yesterday too. The virtio's discriptor stuff don't like the heap so for now I had to fall back to the PMM for now

spring sonnet
#

I also will want to implement kmalloc_aligned().

spring sonnet
#

I have an idea... allocate alighed heap blocks of 4K, and use a slab allcoation method to allocate smaller chunks?

static shore
#

i managed to make mine work, so now i can alloc any size i want

#

(does not need to be aligned)

spring sonnet
static shore
#

you can check mine if you want

#

it should work

#

even for large allocs

spring sonnet
#

I was thinking to combine the two ideas... allocate 4K alighed chunks for large allocations, and smaller allocations I can just do kind of a slab allocation method

static shore
#

i did something similar

#

i alloc the normal 4096 pages

#

then subdivide those into chunks of the size i request

#

but in a buddy like system, so they can easily be merged back

spring sonnet
#

Ahhh I see then; I've got a region just after the kernel that I've decided to make use of for the kernel heap, and mapped it to my desired virtial memory region

static shore
#

i use diff pagemaps for each thread

#

so i can start from the hhdm on each of them

spring sonnet
#

Right now I've been testing a simple chunk allcoation method with allocations being 8-byte aligned and this is where I realize and see issues occur for large allocations....

#

I also implemented a proper virt2phys function in my VMM where I can finally properly get the physical address of page table entries, etc...

#

This was needed since the heap kind of branches away from the whole HHDM. And this a design decision on my part...

#

So now I don't need to use my hack-ish macro for the virt2phys calculation

spring sonnet
#

I kind of have an idea in mind of how to combine the two. I'm hoping this plays nice with the virtio driver as it needs a contiguous 4K aligned region.

spring sonnet
#

So far I'm really liking the idea. I'm in the process of getting things setup, but hope to get to a stage of being able to test allocations tonight. Otherwise I'll have to do that tomorrow

#

Feels nice to finally move onto a heap instead. The last allocators server their purpose to get the kernel going....

spring sonnet
#

I think a better approach would be to have the heap allocate page-aligned allocations, and have a slab allocator like before just split these as needed. So instead of using the PMM I just use the heap

#

Just kind of playing around with different ideas...

spring sonnet
#

So I thought of something different. Have kmalloc() allocate off the pre-reserved heap region, and have kmalloc() aligned allocate a page (or pages) when requested. I'm not sure if that's a very efficient approach, but was an idea I had.

spring sonnet
#

Starting off simple...

Screenshot_at_2025-04-24_21-19-17.png
#

Test #2 seems to have passed...

    void *ptr = kmalloc(1024);
    printf("ptr = 0x%lx\n", (uintptr_t)ptr);

    void *ptr2 = kmalloc_aligned(16 * 1024, PAGE_SIZE);
    printf("ptr2 = 0x%lx\n", (uintptr_t)ptr2);

    void *ptr3 = kmalloc(40 * 1024);
    printf("ptr3 = 0x%lx\n", (uintptr_t)ptr3);

    kfree(ptr2);
    kfree(ptr);

    void *ptr4 = kmalloc_aligned(100 * 1024, PAGE_SIZE);
    printf("ptr4 = 0x%lx\n", (uintptr_t)ptr4);

    kfree(ptr3);
    kfree(ptr4);
Screenshot_at_2025-04-24_21-27-26.png
#

Currently no heap expansion at this time, but plan to implement this tomorrow

#

Just a simple heap for now....

spring sonnet
#

I realize instead of placing the heap at the end of the kernel I should use some of the other usable/reclaimable regions to avoid issues.

#

Bits 55:58 are implementation-defined / free or at least from what I've read about. So I can use one of these to mark the heap when mapping, and just have my kernel handle this accordingly.

#

Seems I've got some currently unused usable regions that are around 50MB or so that I can use for the heap space.

#

This would probably be a better idea anyways... I just hope it plays nice with my virtio driver...

spring sonnet
#

Looks like the heap will get around 101MB of usable space.

#

Now, if I decide to throw in the reclaimable memory this might be increased somewhat

spring sonnet
#

I need to come up with how I'll fix this virtio driver since it required a physically contiguous memory region. Unless I take a chunk of the HHDM and reserve this for some kind of dma_alloc(), but right now that's not ideal as I was hoping to use the heap for these allocations.

covert kayak
#

😭

spring sonnet
covert kayak
#

well walls of text/code are against rules in the server so maybe there's a max message length?

spring sonnet
#

Ahhh that makes sense then

fast sable
spring sonnet
sly pebble
spring sonnet
#

And the heap is using the rest of the usable/reclaim region after the first largest chunk, and mapped to the heaps virtual address region

#

Just kind of reworking things in the kernel currently for allocations..

lime sapphire
lime sapphire
#

in the past I've implemented sub-page allocations as a page allocated from the pmm and accessed via the hhdm, and larger allocations falling back to the vmm.

#

you have to be careful with dependencies there though, as now anything that makes large allocations may depend on the vmm - and everything that depends on.

spring sonnet
lime sapphire
#

yeah for dma purposes you dont want to use the heap

spring sonnet
#

I had a feeling... I'll probably want the queue to use the PMM or implement something to allocate and mark page frames as DMA use only

lime sapphire
#

unless you're using the iommu devices will be using physical addresses, so you need a way to get those from whatever your code on the cpu is seeing.

#

so yeah, you can allocate pages from your pmm and access them via the hhdm on the cpu, makes it easy to work with them that way.

#

there's also MDLs, which are quite a handy primitive to have.

spring sonnet
spring sonnet
lime sapphire
#

yeah, the page tables only apply to the code running on that cpu core

#

mdl = memory descriptor list. You can implement them in your vmm. The idea is you say "hey, I want a list of pages that back this region of virtual memory" and you get them. If you support swapping/paging-to-disk then this would also pin those pages in memory, the intent being that those pages are now stable and wont be moved. Meaning they're safe to hand out to devices.

spring sonnet
lime sapphire
#

it allows your driver code to allocate a chunk of virtual memory, and when the device needs to use it, you get the mdl and tell the device about the physical pages in the list. The device will need to support scatter-gather if you're going to pass it > page-sized regions.

spring sonnet
#

This is quite interesting actually

lime sapphire
#

its pretty handy

sly pebble
#

Scatter gather lists are everywhere in io

spring sonnet
#

I think I'll want to implement this into my VMM in this case. That way stuff like the virtio driver's queue, etc, don't get paged/moved at all

lime sapphire
#

iirc virtio still requires contiguous physical regions for the queues, but it plays nicely with SGL stuff elsewhere

#

but you can just use a single page for each queue 😛

spring sonnet
#

Which will be nice... I just have to rework a few things 😅

#

Thank you for telling about this good sir! I'm going to go research more on this concept, and get started

#

This will be very handy for DMA-related stuff

lime sapphire
#

no worries and good luck 🙂

spring sonnet
#

Thank you! 😁

lime sapphire
unreal creek
# lime sapphire Lol

my connection show nothing i clikced on then it clicked two times so i relicked

#

my connection is realllllly slow

spring sonnet
#

Working on MDL's today

spring sonnet
#

Hmm... I wonder if I can do a combination of using the kernel heap to store the MDL data itself, but have the physical pages backed by the PMM. Sounds like a good staring point.

spring sonnet
# 
typedef struct {
    uintptr_t   virt_base;      // Virtual base of MDL
    uintptr_t   phys_base;      // Physical base of backing pages
    size_t      length;         // Length of MDL range
    size_t      page_count;     // Number of physical pages backed by this MDL
    bool        is_pinned;      // Prevent pages from being swapped
    void        *data;          // Raw pointer to memory
} vmm_mdl_t;

Simple starting point...

static shore
spring sonnet
static shore
#

oh ok

spring sonnet
#

This will be for my virtio driver, and other stuff that may need the HHDM. It also allows me to pin this memory so that it's not swapped or evicted when needing to free up pages

#

It's a new concept r4 told me about last night so I'm still learning about it

static shore
#

wdym by free up pages btw

spring sonnet
#

If pages are needed or can be paged to disk then they can be. This MDL part allows me to protect the allocated region so that it's not paged out if pages are needed by other processes, etc...

static shore
#

oh ok

#

i only free my pages if i call the free function

#

i may look into it too

spring sonnet
spring sonnet
# static shore i only free my pages if i call the free function 
virtio_queue_t *virtio_blk_queue_init(size_t queue_size) {
    vmm_mdl_t *virtio_mdl = NULL;
    virtio_queue_t *vq = NULL;
    size_t desc_size = VIRTIO_DESC_ALIGN(sizeof(virtio_desc_t) * queue_size);
    size_t avail_size = VIRTIO_AVAIL_ALIGN(sizeof(virtio_avail_t) + sizeof(uint16_t) * queue_size);
    size_t used_size = VIRTIO_USED_ALIGN(sizeof(virtio_used_t) + sizeof(virtio_used_elem_t) * desc_size);
    uintptr_t desc_offset = desc_size;
    uintptr_t avail_offset = desc_offset + avail_size;
    uintptr_t used_offset = avail_offset + used_size;
    size_t total_size = VIRTIO_ALIGN(desc_size + avail_size + used_size, PAGE_SIZE);
  


    // NOTE: Device drivers like virtio are DMA only so heap allocations don't work here
    virtio_mdl = vmm_mdl_alloc(total_size);
    vq = (virtio_queue_t *)((uintptr_t)virtio_mdl->data);

    vq->desc = (virtio_desc_t *)((uintptr_t)vq + desc_offset);
    vq->avail = (virtio_avail_t *)((uintptr_t)vq + avail_offset);
    vq->used = (virtio_used_t *)((uintptr_t)vq + used_offset);

    for (size_t i = 0; i < queue_size; i++) {
        vq->desc[i].next = 1 + i;
    }

    vq->free_desc = 0;
    vq->last_used = 0;

    return vq;
}

Yeah but for stuff like this you don't want to just have randomly freed from memory... in this case my virtio queue descriptor stuff... that would cause all sorts of issues lol

static shore
#

yeah XD

#

i still need to work on drivers

spring sonnet
#

Take your time! I wanted to get disk access going early on

spring sonnet
# static shore yeah XD

That would be bad to if I was doing a read from the disk, and got the queue freed mid-read 😅

spring sonnet
#

I think I may want to just do a direct mapping of the kernel heap instead of remapping it to a different region....

#

My vmm_mdl_alloc() calls to kmalloc() to store it's header accordingly, but of course the PMM is used to allocate the pages that are needed for the DMA stuff

#

Yeah I think I better just do that instead.... it'll me easier to manage the MDL allocations along with other things.

willow prism
#

Because there's no guarantee the page frames are physically contiguous

#

You need an array of page frame numbers

spring sonnet
spring sonnet
# willow prism You can't use a phys base and page count

The part I'm having a bit of a hard time with is thinking about how to provide the allocated frames for the MDL stuff. I plan to have it make use of the PMM of course since r4 did tell me this should be seperate from my kmalloc(), etc

#

At least I'm trying to think of a clean way to distribute allocated page frames.

lime sapphire
#

will beat me to it

lime sapphire
#

as for how to get the pfns, you get the page mapped for each page-sized chunk of the virtual region. If you're doing demand paging stuff, then take care of that first ofc.

spring sonnet
#

Right now I have all of the physical memory directly mapped into kernel space (like Limine does) so I know it'll be easy for me to just provide a set of page frames for this. But I'm wondering if I should keep a page count as well as the regions start and end address? Just for housekeeping, etc, for later of that DMA memory is no longer needed?

lime sapphire
#

keep what you need to get the job done

#

I think I have something like: addr_space, virt_base, offset, length, pages[].

#

length + offset can be used to calculate the number of pages, you can make a nice macro for yourself too

spring sonnet
#

I think that's what I'll end up going for in this case

spring sonnet
spring sonnet
#

Alright I have an idea of how to get started with this.

spring sonnet
spring sonnet
# 
#define VMM_MDL_PAGE_IDX(offset)       ((offset) / PAGE_SIZE)
#define VMM_MDL_PAGE_OFFSET(offset)    ((offset) % PAGE_SIZE)

Simple way to handle the page frames. Wtill a work-in-progress though...

spring sonnet
#

Got a good and simple way to get pages, etc...

spring sonnet
#

I'm starting to realize that maybe I should of kept my slab allocator still. I can always bring it back if I need it.

#

I just notice cases where I can allocate a slab for header information on DMA stuff and not waste a whole page on it

willow prism
#

why did you get rid of your slab allocator

spring sonnet
willow prism
#

You just like randomly deleted it or what

spring sonnet
#

No.. I had removed it when I was getting started on the heap thinking I wasn't going to need it. I was wrong, and learned that I should have different memory allocation API's for things...

#

I'm kind of realizing my mistake in removing the slab allocator

willow prism
#

But a slab allocator is just an implementation of a heap

#

You already had a heap

spring sonnet
#

This is true... removing that slab allocator was a bit of a mistake. I plan to bring it back, but most likely have the pages that the heap uses be remapped elsewhere

lime sapphire
#

and if I was being especially critical, do you really need that bool now? Save yourself the bytes until you get there, you might find other ways to do that 🙂

spring sonnet
# 
qemu: virtio-blk missing headers

Apparently virtio doesn't like the fact I'm trying to do this..

spring sonnet
#

I'll get it figured out

spring sonnet
spring sonnet
#

Okay, finally got my virtio driver working with this, but still need to adjust how I'm getting an allocated page frame from within the list.

#

Right now it works fine though

spring sonnet
#

This was a bit of a pain in my ass to get working correctly....

spring sonnet
#

I was thinking of haivng a part of memory used for DMA use

#

Nah the current method of allocation page frames is fine.

#

I guess since the reserved bitfields are implementation defined from what I understand I can also use that to mark pages as "pinned". And then just do a check for this when going to unmap I guess, but the VMM might need some slight work for this

#

Since I planned to reintroduce my slab allocator for kernel heap paging I was also considering remapping pages used by the heap into a different virtual address region.

#

I do use a bit to mark heap allocations.

spring sonnet
#

I know that request descriptors also must be 1:1 mapped (HHDM works too of course). So I will need to implement slab allcoation as before for handling smaller allocations.

#

Might have to consider a few allocation API's for this too.....

#

I guess I could have a slab allocator that can be used for DMA-related stuff, etc. And combile this with the MDL stuff, and then have my kernel's kmalloc() use it's own slab allocator

spring sonnet
#

From some information I've gathered I guess Linux does something similar to this as well. Kernel heap pages in my kernel can also be paged out when needed so this will be great. Motivates me to get that EXT2 driver going. I do need to work on the VFS design before I can continue though, but I'll do it in parts as I develop the filesystem.

spring sonnet
#

For now since I got the beginnings of that MDL stuff going I'll work on the kernel heap slab allocator. After this I'll start working more on the DMA slab allocator, and have this and the MDL stuff kind of together for this.

#

Got a hell of a lot of work to do for sure.

spring sonnet
#

I'll be doing a slab allcoator like before, but this will be for the kernel heap since it's a form of heap allcoator. I'll map the frames I get from the PMM to the heap virtual region, and any allocations larger than what the slab supports will just have full pages allocated to satisfy the allocation needs. After I'll work on the MDL DMA allocation API stuff.

#

Got a pretty good idea of what I want to do here

spring sonnet
#

Kernel slab heap allocator is coming along. Just kind of taking time to decide on how I want to handle allocations, etc. Any length that's not supported by the slab will just be backed by pages to satisfy the requested size. After freeing I'll just unmap that entry or if it's a slab allocation just return the allocated memory to the freelist...

spring sonnet
#

Well.... it wasn't until now I discovered a slab bug I had in my old code. The free list pointer I was setting ends up writing too far outside of it's page boundary, but I never noticed this due to the fact I was mapping all physical memory directly. Now that the heap handles setting it's own pages for isolation from other mapped pages the bug has manifested itself.... interesting

#

So basically an out-of-bounds write bug...

spring sonnet
#

Just noticed my in my VMM for the vmm_unmap() if it returns it never releases the lock. Need to fix this

#

Otherwise I'll hit a deadlock

spring sonnet
#

Well... apparently I spoke too soon because now when I acquire a lock in the VMM I deadlock 😂

#

I need to take a look and see where I'm forgetting to release the lock, etc...

#

Well, one issue I can see is that I try to take the lock twice when getting a physical address of the PTE. Not good...

unreal creek
#

@spring sonnet do you have any expérience with ps2 keyboard i'm on my ps2 driver for the last 2 week a.d it still don't work ?

spring sonnet
spring sonnet
#

Finally got my kmalloc() and MDL to play nice together. Far from perfect, but this is some really good progress.

#

The plan I have for larger page allocations is to just reuse the allocated pages and keep a free list of of them.

#

Lots of work being done on memory allocation for sure

sly pebble
#

Whats a kernel beyond a bloated memory allocator 🙏

spring sonnet
sly pebble
#

Sarcastic but a little bit serious, good and versatile allocator is great to have

spring sonnet
#

Oh, okay! I was a little worried there haha

#

It'll be much improved over time of course, but for now it'll work out...

spring sonnet
#

Welp... the new heap allocator mostly works... need to take a look at the slab allocations ensure the freed slab objects are actually probaly being placed back into the freelist. Or if out of objects to create new one, and allocate from that.

Screenshot_at_2025-04-27_22-06-16.png
spring sonnet
#

Ahh... turns out I forgot to handle when new objects needed to be allocated if they run out....

#

I think I might be hitting a deadlock somehere in my heap allocator or VMM....

#

Yup it's a deadlock....

#

Fok

#

I can see too were I forget to release the lock before returning a pointer to allocated memory......

#

Heeey that's certainly improvment!!! I'll continue this tomorrow....

Screenshot_at_2025-04-27_23-25-03.png
spring sonnet
#

After fixing a few bugs with not releasing the lock a few times, and an out-of-bounds read bug I finally got things working 👀

Screenshot_at_2025-04-28_00-33-15.png
#

My new heap allocator seems to be working fine too

#

Got one big ass commit to push tomorrow

worthy lynx
#

soooo good bro!

spring sonnet
worthy lynx
#

round robin is pretty easy

#

but idk if its good lmaoo

spring sonnet
#

That's actually what I was looking into doing haha

worthy lynx
spring sonnet
#

I'll take a look and read up on some stuff

spring sonnet
#

Forgot to remove that comment before committing... I think I'll keep it there for a nice reminder...

Screenshot_at_2025-04-28_11-13-51.png
unreal creek
spring sonnet
hallow comet
lost stream
spring sonnet
#

Seems from what I've seen about saving a task context on arm64 only x19-x30 (general purpose registers) need to be saved. Along with elr, and spsr according to the ABI

#

Kind of makes me wonder though about x0-x18 though.

#

I'll learn more as I research about this...

lost stream
spring sonnet
trail jolt
#

x0-x8 are arguments and out regs, x17 and x18 might be clobbered by linker veneers before the callee is even entered, i assume x9-x16 are just marked as caller-saved

sly pebble
#

Idk why esr is part of the context, you'd often reorganize esr into something that makes sense for your kernel

spring sonnet
spring sonnet
sly pebble
#

Yeah, the esr hold a lot of info but in exception handling you quickly get rid of the information

spring sonnet
#

That's what I had realized haha; I only really need that information when an exception occurs, and after that it'll end up discarded

spring sonnet
#

Was reading/learning about the round robin scheduling. Got a very very quick and dirty test going because I really wanted to test, debug, and expand my knowledge more.

Screenshot_at_2025-04-28_20-14-43.png
#

After some bug fixing it seems to work...

#

I'm going to spawn more tasks just to test things, but this is pretty cool. I guess it makes use of the timer for this so I'm glad I had that already implemented before hand.

unreal creek
spring sonnet
unreal creek
spring sonnet
#

I need to debug and see what caused the exception though. I'm pretty sure it's in my __context_switch() handler.

unreal creek
spring sonnet
spring sonnet
#

My ass forgot to load x1's context, and ended up doing x0 again so that really caused some weird bugs...

#

Hell one of them was the fact when I returned from the exception I ended up in kslab_alloc() lmao

spring sonnet
#

I'll be redoing that code anyways. I wanted to play around with an idea that I had to get a visual example of how it would work

spring sonnet
#

This is going to be brutal to implement.

spring sonnet
#

Trying to think of some design ideas for this round robin scheduler. I decided upon a free list queue, but something I was thinking about is since I planned to have the timer be used for this I don't like the idea of a schedule() function handling the timer reset since this is done in the timers IRQ handler. I know I'd need to keep track of the number of ticks per process so that if a process is finished before it's quantum is up then I can handle that accordingly or switch to the next task.

#

@unreal creek Didn't you implement a round robin scheduler? Or did you end up going with something else

unreal creek
#

i have a round Robin

spring sonnet
# unreal creek i have a round Robin

May I take a look at how you've implemented your scheduler to get some insparation? I want to better understand how I should handle the task setup, queue, and switching of tasks

unreal creek
#

the function yeld call the function schedule that tell to what process to switch to

#

and then it call context_switch (coded in assembly) to do the actual context switch

#

https://github.com/tayoky/stanix/blob/main/kernel/task/scheduler.c
https://github.com/tayoky/stanix/blob/main/kernel/arch/x86_64/pit.c
https://github.com/tayoky/stanix/blob/main/kernel/arch/x86_64/switch.s
if you want to take a look

GitHub
stanix/kernel/task/scheduler.c at main · tayoky/stanix

An 64 bit OS made from scratch. Contribute to tayoky/stanix development by creating an account on GitHub.

GitHub
stanix/kernel/arch/x86_64/pit.c at main · tayoky/stanix

An 64 bit OS made from scratch. Contribute to tayoky/stanix development by creating an account on GitHub.

GitHub
stanix/kernel/arch/x86_64/switch.s at main · tayoky/stanix

An 64 bit OS made from scratch. Contribute to tayoky/stanix development by creating an account on GitHub.

spring sonnet
#

Thank you good sir! I want to spend some time studying this before I get started

unreal creek
#

bye

spring sonnet
spring sonnet
#

I'm just now learning that I can make this preemtive multitasking work through the IRQ context. Didn't even think about that.

#

I guess it's also "standard" to re-enable the timer in a schedule() function just before a context switch? It doesn't feel right to me, but I guess if that's what I have to do.

spring sonnet
#

It's all starting to make sense to me now.

#

Now I'm understanding the flow of resetting the timer, and scheduling the next task.

# 

#define TASK_STACK_SIZE (16  * 1024)

#define TASK_QUANTUM_MS 20

#define TASK_STATE_READY    (1UL << 0UL)
#define TASK_STATE_RUNNING  (1UL << 1UL)
#define TASK_STATE_BLOCKED  (1UL << 2UL)
#define TASK_STATE_DEAD     (1UL << 10UL)



typedef struct {
    uint64_t    x19;        // 0x00
    uint64_t    x20;        // 0x08
    uint64_t    x21;        // 0x10
    uint64_t    x22;        // 0x18
    uint64_t    x23;        // 0x20
    uint64_t    x24;        // 0x28
    uint64_t    x25;        // 0x30
    uint64_t    x26;        // 0x38
    uint64_t    x27;        // 0x40
    uint64_t    x28;        // 0x48
    uint64_t    x29;        // 0x50
    uint64_t    x30;        // 0x58
    uint64_t    sp;         // 0x60
    uint64_t    elr_el1;    // 0x68
    uint64_t    spsr_el1;   // 0x70
    uint8_t     __pad[16];  // 0x80
    uint128_t   q[32];      // 0x80 - 0x280
    uint64_t    fpcr;       // 0x280
    uint64_t    fpsr;       // 0x288
} __attribute__((packed)) task_context_t;

typedef struct task {
    task_context_t  ctx;        // Task context state
    int             task_id;    // Unique task ID
    int             quantum;    // Amount of time task has run for
    uint64_t        task_state; // Current state of task
    size_t          stack_size; // Task stack size (default 16KB)
    void            *stack;     // Task stack pointer
    struct task     *next;      // Next task in queue
} task_t;

Currently what I have in mind for the task information design, etc.

#

For some reason trying to debug context switches in vscode in QEMU causes my system to freeze. This is twice in the last two days I had to reboot.

spring sonnet
#

It only happens in the while(1); loop in the next task I set, but I'm able to exit QEMU so it's no problem.

spring sonnet
#

Well, I can start a new task, but as soon as I yield shit breaks. This is going to be difficult to debug.

# 

    ldr x19, [x1, #X19_OFFSET]
    ldr x20, [x1, #X20_OFFSET]
    ldr x21, [x1, #X21_OFFSET]
    ldr x22, [x1, #X19_OFFSET]
    ldr x23, [x1, #X20_OFFSET]
    ldr x24, [x1, #X21_OFFSET]
    ldr x25, [x1, #X19_OFFSET]
    ldr x26, [x1, #X20_OFFSET]
    ldr x27, [x1, #X21_OFFSET]
    ldr x28, [x1, #X28_OFFSET]
    ldr x29, [x1, #X29_OFFSET]
    ldr x30, [x1, #X30_OFFSET]

Well...... I now see exactly why shit broke....

#

Whooooops

sly pebble
#

To make debugging a little easiest I suggest making snapshots and using them to rewind, look into qemu-img , and then in the monitor using savevm or loadvm

spring sonnet
#

But I will be making use of that feature...

#

Apparently using spinlocks in my schedule() function wasn't such a good idea. Even though I check the if the current task is the only one running, and return I make sure I'm releasing the lock, but the next time I take the lock I deadlock.

#

Removing them seems to fix that issue

sly pebble
#

It became a must have tool for me to debug some problems, esp if you attach a debugger on qemu

spring sonnet
#

That's going to be a really nifty tool to make use of for debugging then! I never knew QEMU even had such a feature haha

# 
    sched_create_task(&helix_init_stage2);
    sched_start_task();

void helix_init_stage2(void) {
    task_t *t = sched_get_current_task();

    printf("Hello from task: %d!\n", t->task_id);

    while(1) { sched_yield(); }
}

Doing a simple test....

#

Holy fuck I actually did multitasking!!! But I need to look into the crash

Screenshot_at_2025-04-29_19-52-19.png
#

Duuuude that's pretty cool!!!

sly pebble
#

Nice!

spring sonnet
# sly pebble Nice!

Thank you good sir!! You should of seen my face when I seen the output hahaha

#

It's the first time working with multitasking so this is really exciting

sly pebble
#

So after a little testing you want to progress to userland?

spring sonnet
#

I think the crash might be related to a stack overflow issue. I originally was giving each task a 16KB stack, and increasing it to 64KB seems to allow for it to run a bit longer, but of course the crash happens again. I'm not too sure how large I should make the stack, but I also need to check and see if I'm doing anything to cause this to happen in the first place

#

Yep it's a stack overflow

spring sonnet
#

Seems it's my first stack that's causing the overflow. Did a stack canary for testing, and printed the task ID.

spring sonnet
#

Going to spend some time on fixing that, but I'm really excited to test context switching

spring sonnet
#

I'm almost wondering it it's an issue in my allocator (kmalloc)

unreal creek
spring sonnet
#

I'm checking to see if my kmalloc() is doing any shenanigans that might be causing issues

#

But so far I have bare bones context switching and multitasking... Despite it being semi-broken lol

unreal creek
spring sonnet
unreal creek
spring sonnet
#

Looks like my kmalloc() has a few bugs I need to work out........

spring sonnet
#

@unreal creek I can't help but feel like my context switching somehow runs out of memory and eventully overflows the stack.

unreal creek
unreal creek
#

how big is your stack ?

spring sonnet
spring sonnet
unreal creek
spring sonnet
# unreal creek what is a translation fault

It's a page fault

https://pastebin.com/MkuSYsp9

This is my current scheduler setup. It's quite bare minimual, but seems to work fine for some time when doing multitasking, but after some time I end up running into a page fault. I'm not too sure yet if it's when I'm loading/storing the contexts states

Pastebin
#include #include #include #include

Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.

#

Yeah task #1 seems to run perfectly fine. It yields, and and because it's the only task running it continues. No crashes so far so I'm not doing something right here....

unreal creek
spring sonnet
unreal creek
#

sorry

spring sonnet
#

That's alright!! I'll keep debugging and see what I can find....

spring sonnet
#

Is this something you handle on x86_64 at all?

sly pebble
spring sonnet
#

Well that's one thing off the right off the bat I can tell is another issue

Screenshot_at_2025-04-29_23-26-34.png
#

Duplicate saving

unreal creek
spring sonnet
# unreal creek yes

I figured haha; I noticed some issues in my context switch code that I'll need to resolve...

#

One was the duplicate saving, and another was that I wasn't saving the fpcr and fpsr registers....

#

I'm also reloading them twice from the other context... wtf lol

unreal creek
#

is it working now?

spring sonnet
#

I'm still fixing a few things up, but I'll be testing soon

sly pebble
#

Does all of your tasks share the stack?

spring sonnet
#

Fuck... I spoke too soon... crashed lol

sly pebble
#

I suggest to log your stack after returning from yield, if I understand your test task correctly

#

Make sure its the same for each task, and not increasing

spring sonnet
#

That's my next plan; I also noticed something a bit odd here:

Taking exception 4 [Data Abort] on CPU 0
...from EL1 to EL1
...with ESR 0x25/0x96000004
...with FAR 0x2a0
...with ELR 0xffffffff8001c09c
...to EL1 PC 0xffffffff80010200 PSTATE 0x3c5

The faulting address (FAR) shows something completely invalid too

sly pebble
#

A fucked up stack could cause this

#

You can look at your disassembly for the elr and see the address register is loaded from the stack

#

If not then the context for that register is broken

spring sonnet
# sly pebble You can look at your disassembly for the elr and see the address register is loa...

I actually just took a look at something too...

aarch64-linux-gnu-objdump -d kernel/kernel.elf | grep -A20 8001c09c
ffffffff8001c09c:    f9415000     ldr    x0, [x0, #672]
ffffffff8001c0a0:    92400000     and    x0, x0, #0x1
ffffffff8001c0a4:    f100001f     cmp    x0, #0x0
ffffffff8001c0a8:    540000e1     b.ne    ffffffff8001c0c4 <schedule+0x6c>  // b.any
ffffffff8001c0ac:    f0000840     adrp    x0, ffffffff80127000 <irq_handler_table+0x1dc8>
ffffffff8001c0b0:    91090000     add    x0, x0, #0x240
ffffffff8001c0b4:    f9400000     ldr    x0, [x0]
ffffffff8001c0b8:    f9400fe1     ldr    x1, [sp, #24]
ffffffff8001c0bc:    eb00003f     cmp    x1, x0
ffffffff8001c0c0:    54fffe61     b.ne    ffffffff8001c08c <schedule+0x34>  // b.any
ffffffff8001c0c4:    f9400fe1     ldr    x1, [sp, #24]
ffffffff8001c0c8:    f9400be0     ldr    x0, [sp, #16]
ffffffff8001c0cc:    eb00003f     cmp    x1, x0
ffffffff8001c0d0:    540000a1     b.ne    ffffffff8001c0e4 <schedule+0x8c>  // b.any
ffffffff8001c0d4:    d2800280     mov    x0, #0x14                      // #20
ffffffff8001c0d8:    9400043e     bl    ffffffff8001d1d0 <timer_set>
ffffffff8001c0dc:    97fffe8b     bl    ffffffff8001bb08 <__daif_clr>
ffffffff8001c0e0:    14000018     b    ffffffff8001c140 <schedule+0xe8>
ffffffff8001c0e4:    f9400be0     ldr    x0, [sp, #16]
ffffffff8001c0e8:    f9415000     ldr    x0, [x0, #672]
ffffffff8001c0ec:    927ef801     and    x1, x0, #0xfffffffffffffffd

The very first load there it might be that I'm probably accessing an invalid field most likely

#

x0 seems to be NULL in that case too...

spring sonnet
sly pebble
sly pebble
spring sonnet
spring sonnet
#

Going to spend today reworking some stuff..

#

@unreal creek

    new_task->stack_size = TASK_STACK_SIZE;
    new_task->stack = kmalloc(new_task->stack_size);

    if (!new_task->stack) {
        printf("SCHED: Failed to allocate new task stack! (out of memory)\n");
        
        kfree(new_task, sizeof(task_t));
        spinlock_release(&s);
        return;
    }

    memset(new_task->stack, 0, new_task->stack_size);

    // NOTE: Stack must be 16-byte aligned
    uintptr_t sp = (uintptr_t)new_task->stack + new_task->stack_size;
    sp = ALIGN_DOWN(sp, 16);

    new_task->ctx.sp = sp;
    new_task->ctx.elr_el1 = (uint64_t)(uintptr_t)__start;
    new_task->ctx.spsr_el1 = __spsr_read();

Here's how I'm setting u pthe current stack. I do make sure it's aligned on 16-byte boundaries, but I'm not sure if my curent way of setting up the stack is causing issues. I don't see though how on even 64KB of stack space it stil eventually crashes unless my logic is messed up

unreal creek
#

does kmalloc take bytes as parameter and note a page count ?

spring sonnet
spring sonnet
#

Probably going to re-write all of the scheduler shit. I'm starting to get a bit irritated with the stack corruption and other tasks not properly running after the first two if they don't yield.

tender acorn
#

@spring sonnet i will switch to using sgdisk and qemu-img totally, mind telling me how to use sgdisk?

#

i saw i need to make a loopback file to the image file?

covert kayak
#

that seems unrelated to the thread 😭

tender acorn
#

i should have asked on misc tho

#

mb

spring sonnet
#

It's a pretty simple utility, and making use of qemu-img over dd is even better

#

I also understand how the stack overflow was occuring too during the context switch within the timer IRQ handler....

#

Right now I'm taking some time to learn how to handle such a thing...

#

Well, not just that but I'm also realizing other things about saving/loading the context, etc, and how changing the stack was kind of messing this up

#

At least for how I was handling things.

spring sonnet
#

Yes, when an interrupt (IRQ) occurs on an ARM64 CPU, the CPU saves the current execution context before jumping to the interrupt handler. This saved context is later restored when the interrupt handler completes, allowing the CPU to resume its previous execution.

This is is helpful to know. I thought I had to handle saving the current context myself, and this is where I started to struggle to understand about scheduling tasks, etc.

spring sonnet
#

Now it's making sense where the idle task comes into play.

#

Now I'm beginning to better understand with the more knowledge I gain.

spring sonnet
# 
void sched_create_task(void (*__start)()) {
    task_t *new_task = NULL;
    uint64_t *sp = NULL;


    new_task = kmalloc(sizeof(task_t));

    if (!new_task) {
        printf("SCHED: Failed to initialize new task context!\n");
        return;
    }

    // Initialize new task stack
    new_task->stack_size = TASK_STACK_SIZE;
    new_task->sp = kmalloc(new_task->stack_size);

    if (!new_task->sp) {
        printf("SCHED: Failed to initialize new task stack!\n");
        
        kfree(new_task, sizeof(task_t));
        return;
    }

    // Set temporary stack pointer to setup task context    
    sp = (uint64_t *)((uint8_t *)new_task->sp + new_task->stack_size);

    // NOTE: Registers are stored/loaded in pairs!
    for (uint64_t i = 0; i < 10; i++) {
        *(--sp) = 0xDEADB33F; // x19-x28
    }

    // x30 -> return address of new task
    *(--sp) = (uint64_t)__start;

    // x29 -> frame pointer
    *(--sp) = 0xDEADB33F;

    new_task->sp = sp;
    new_task->task_pid = task_pid++;
    new_task->task_state |= TASK_STATE_READY;

    if (!task_list) {
        // First task in queue
        task_list = new_task;
    } else {
        // Link task to end of queue
        task_t *prev = task_list;

        while(prev->next != NULL) {
            prev = prev->next;
        }

        prev->next = new_task;
    }
}

A very simple task setup function for now....

#

I also setup an idle task that calls another function to schedule the next task or just idle in a low-power state and wait for the next interrupt. This is a much clearner version of what I was doing too.

spring sonnet
#

I can't ever seem to get context switching right. I somehow end up in a loop in the context switcher itself or I return back to the sceduler.

covert kayak
#

oof. good luck, scheduler bugs can be a hell to fix

#

what's your context switch looking like?

spring sonnet
# covert kayak what's your context switch looking like? 
.global __context_switch
__context_switch:
    // x0 -> pointer to current task_t struct
    // x1 -> pointer to new task_t struct

    // Store callee-saved registers
    stp x19, x20, [sp, #-16]!
    stp x21, x22, [sp, #-16]!
    stp x23, x24, [sp, #-16]!
    stp x25, x26, [sp, #-16]!
    stp x27, x28, [sp, #-16]!
    stp x29, x30, [sp, #-16]!

    // Store SP in task context struct
    mov x4, sp
    str x4, [x0, #0x08]

    // Load next task SP
    ldr x4, [x1, #0x08]
    mov sp, x4

    // Restore callee-saved registers for next task
    ldp x29, x30, [sp], #16
    ldp x27, x28, [sp], #16
    ldp x25, x26, [sp], #16
    ldp x23, x24, [sp], #16
    ldp x21, x22, [sp], #16
    ldp x19, x20, [sp], #16

    // Return to new task (x30 holds return address)
    ret

It's rather simple where I pass in task structs to store/load the previous, and next tasks. It works with loading the idle task, then the next task (test task), but then when the test task yields it ends up in some kind of a loop in the context switcher or ends up returning to the scheduler. I've been having a helish time trying to figure out why this happens

#

I was thinking I might have to write the yield code in pure assembly

covert kayak
#

oh I didn't realise this project was for arm

#

nice

spring sonnet
sly pebble
#

Are there any interrupts?

#

If you dont keep elr you'd gonna end up in a very nasty place

spring sonnet
# sly pebble If you dont keep elr you'd gonna end up in a very nasty place

I do have the timer IRQ that increments a systick count, but the IRQ handler vector code saved the context (ELR, and SPSR included) and jumps to that code. Once it returns it gets restored but I feel like my logic for how I'm trying to handle this is a bit off. My context switcher doesn't currently handle saving ELR or SPSR as I thought this would cause issues when trying to return from the timer IRQ

#

I just notice that it works fine to idle till the systick time reaches the max quantum, then executes the next task in the queue, and when that task yields to give up it's CPU time that's when things seem to break

sly pebble
#

Yeah id wager its related to timer honestly, what do you do one getting a timer and decide its time to switch tasks?

spring sonnet
# sly pebble Yeah id wager its related to timer honestly, what do you do one getting a timer ...

So, here's how I handle incrementing the systicks witin the timer IRQ:

void timer_irq_handler(void) {
    uint64_t curr_time = timer_virt_get_time();
    uint64_t deadline = curr_time + saved_ticks;
    
    timer_disable();

    sys_ticks++;

    __cntv_cval_write(deadline);

    timer_enable();
}

And this is the task that checks for the max quantum

static void sched_idle_task(void) {
    while(1) {
        uint64_t curr_ticks = timer_get_sys_ticks();

        if (curr_ticks >= TASK_QUANTUM_MS) {
            // Reset sys_ticks ticks
            timer_reset_sys_ticks();

            // Run next task
            sched_run_next_task();
        }

        printf("Idling...\n");

        // Idle till we have another task to run
        wfi();
    }
}

This is the task that handles switching when the max quantum (10ms) has been reached and it's time to switch to the next task

# 
__attribute__((noreturn)) void sched_run_next_task(void) {
    task_t *prev = task_list, *next = task_list->next;

    // Look for next task to run
    while(!(next->task_state & TASK_STATE_READY) && next != task_list) {
        next = next->next;
    }

    next->task_state &= ~TASK_STATE_READY;
    next->task_state |= TASK_STATE_RUNNING;

    task_list = next;
    __context_switch(prev, next);

    __builtin_unreachable();
}

Then my very simple logic for switching to the next task in the queue....

#

It works for the first two tasks, but when returning from the 2nd task breaks. I can't really make it sit and wait for an interrupt because then it ends up blocking the exection too...

sly pebble
#

Okay so your timer doesn't preempt the task, ig that's fine for now

spring sonnet
sly pebble
#

I mean you'd end up in the mercy of the task otherwise, if I make a task that wouldn't call yield you can't preempt it otherwise (no multi cores shenanigans)

#

But the less code in your timer handler the simpler it is, so for start thats a good idea not to preempt

spring sonnet
#

That's what I thought as well. I wanted to keep the timer handler clean, and just increment a global variable that the scheduler code can get from there

#

I'll continue to debug and examine what's going on. I'll eventually figure out a solution to this

sly pebble
#

Im not sure I follow your code, how would you ever return to the idle task after the process finished his quota?

spring sonnet
#

That's something I was thinking about too. I was thinking I might need some kind of "trampoline" that can handle something like this

#

Sounds like a horrible design decision though

#

That or I might need to rethink how I'm doing this

sly pebble
#

Yeah I think the idle and timer relations are both a little bit broken, if your idle is a task then it doesn't yield but wait until itself get an interrupt and then decide its been too long until the other task (which one?) Ran for enough and it transform itself and would never return, and if it would then that's a problem (unreachable + no return).
A small note if you use a lot of unreachables, I'd make a macro that assert you never got there, something like panic followed by builtin unreachable

#

For instance if you disable timer you'd stuck in an endless loop in idle

spring sonnet
#

Yeah I think I better rethink how I'm doing this then...

#

I think I will implement a macro to asset and kernel panic for cases like this as well

sly pebble
#

Yeah I suggest taking a step back and thinking how do you want your timer to work, stuff like
What does it mean if a task finished its quota?
If a task kept yielding, does it need to be stopped through timer?
Is there one timer for the entire system or is it per task?

spring sonnet
#

I do plan to take a step back and think about this for a little bit...

spring sonnet
#

Taking some time to study this topic a bit more, and understand how preemptive multitasking should work

#

I think it's the right thing to do in this case

spring sonnet
#

Currently just researching about round robin context switching, etc, to learn more on this

spring sonnet
#

I'm glad I decided to take a step back and retink this a bit.

#

This looks interesting.

#

This register will be extremely useful.

sly pebble
#

Im away from laptop at the moment, but iirc gcc would use it for tls, be wary of using it yourself

#

I dont recall if it uses el0 or el1 register for the kernel

spring sonnet
sly pebble
#

When you use thread local, I think gcc compiles into read from tpidr, idr if it use el0 or 1

spring sonnet
sly pebble
#

I think it wouldn't really help you, and i advise against using arch specific mechanisms explicitly for generic stuff
I mean if gcc already use it then you'd get in trouble if you use it as well

spring sonnet
#

This is true....

frozen silo
#

usually tpidr_el1 is used to store either a pointer to a Cpu structure for the current cpu or a pointer to the current thread's Thread structure

#

tpidr_el0 you shouldn't use in the kernel because its used by userspace for tls (and can be read/written by userspace directly)

spring sonnet
#

I probably shouldn't mess with it then in this case...

#

I've also been learning about how Linux tackles the preemptive multitasking. It seems you can use the IRQ vector to restore to the next task context. I didn't know this could be done. Right now my IRQ vector code just saves the context, handles the IRQ, restores the current context, and returns from it

#

I was maybe considering adopting this approach

spring sonnet
#

Looks like I'll be doing so.

spring sonnet
#

Time to get working on this design a bit

spring sonnet
#

I finally understand how context switching is actually supposed to work. It's not as difficult as I remember, and my last attempt was almost there. I'm glad I took a step back to relook over this.

spring sonnet
#

Think today's task will be to design and implement task creation.

spring sonnet
# 
task_t *sched_create_task(const char *name, void (*__start)()) {
    task_t *t = NULL;
    uint64_t *sp = NULL;




    if (!name || !__start) {
        return NULL;
    }

    t = kmalloc(sizeof(task_t));

    if (!t) {
        return NULL;
    }

    strncpy((char *)t->task_name, name, sizeof(t->task_name));

    t->task_id = next_task_id++;
    t->task_resched = false;
    t->status = TASK_READY;

    t->ctx.elr_el1 = (uint64_t)(uintptr_t)__start;
    t->ctx.spsr_el1 = __spsr_read();

    t->ctx.fpcr = __fpcr_read();
    t->ctx.fpsr = __fpsr_read();

    t->stack_size = TASK_STACK_SIZE;
    t->stack = kmalloc(t->stack_size);

    if (!t->stack) {
        kfree(t, sizeof(task_t));
        return NULL;
    }

    // Temporary stack pointer
    sp = (uint64_t *)(t->stack + t->stack_size);

    // x19 -> Exception return address
    *(--sp) = (uint64_t)t->ctx.elr_el1;

    // x20 -> Saved program status register
    *(--sp) = (uint64_t)t->ctx.spsr_el1;

    // x19 -> FPU control register
    *(--sp) = t->ctx.fpcr;

    // x20 -> FPU status register
    *(--sp) = t->ctx.fpsr;

    // Set x19-x30 to clean state
    for (uint64_t i = 0; i < 10; i++) {
        *(--sp) = 0x00; // x19-x30
    }

    // Set current stack pointer
    t->ctx.sp_el1 = (uint64_t)sp;
    t->ctx.sp_el0 = 0x00;
    
    return t;
}

For now, a really simple task creation function. This of course is just to get started.

#

I don't want to start complicating things on myself early on.

#

It does help to understand how the stack work as well so that I can properly setup the task stack. That way when I go to restore the task's context everything will get loaded properly.

spring sonnet
# 
__aarch64_irq_handler:
    // Save callee-saved register contents
    stp x19, x20, [sp, #-16]!
    stp x21, x22, [sp, #-16]!
    stp x23, x24, [sp, #-16]!
    stp x25, x26, [sp, #-16]!
    stp x27, x28, [sp, #-16]!
    stp x29, x30, [sp, #-16]!

    // Save ELR and SPSR
    mrs x19, elr_el1
    mrs x20, spsr_el1
    stp x19, x20, [sp, #-16]!

    // Save FPU control/status registers
    mrs x19, fpcr
    mrs x20, fpsr
    stp x19, x20, [sp, #-16]!

    // Handle IRQ
    bl irq_handler

    // Handle task scheduling, etc
    b __ret_from_irq

__ret_from_irq:
    // Check if task needs to be rescheduled
    bl sched_maybe_reschedule

    // If no task needs to be recheduled return to previous context
    ldr w0, [x0, #TASK_CTX_RESCHED_OFFSET]
    cbz w0, __ret_to_prev

    // Perform context switch

__ret_to_prev:
    // Exit back to previous context (no context switch)

    // Restore FPU control/status register
    ldp x19, x20, [sp], #16
    msr fpcr, x19
    msr fpsr, x20

    // Restore ELR and SPSR
    ldp x19, x20, [sp], #16
    msr elr_el1, x19
    msr spsr_el1, x20

    // Restore callee-saved registers
    ldp x29, x30, [sp], #16
    ldp x27, x28, [sp], #16
    ldp x25, x26, [sp], #16
    ldp x23, x24, [sp], #16
    ldp x21, x22, [sp], #16
    ldp x19, x20, [sp], #16

    // Return to previous task
    eret

Still working on this, but kind of getting things ready for context switches in the IRQ handler.

#

Proceeding with the knowledge I have gained from looking over how such things are handled.

#

Well, looks like I might need to make a few changes already with how I'm braching in different conditions....

#

Somehow I broke the execution flow for after the IRQ handler returns. For some reason trying to branch to different lables is causing issues. This sucks

#

Oh God now I'm breaking things....

Screenshot_at_2025-05-02_21-39-10.png
#

Apparently I do need to also save x0-x18....

#

Welp...

#

That means I better make sure that they get saved during a context switch.....

sly pebble
#

Remember not to save the irq handler registers but the before you enter irq

#

I mean unless you want to return to the irq handler, which is also an option

spring sonnet
sly pebble
#

I don't recall on top of my head what's the abi, but if all of your asm follows abi the callee saved registers shouldn't change (irq_handler should save/restore them and so does sched_maybe_reschedule)

#

In the irq you must ensure all of the registers wouldn't change, if you decide to come back to the irq handler after your task been rescheduled (easier imo) then make sure all of your context switch follows the abi and irq handler ensures no register change

spring sonnet
#

I know during IRQ entry, all general purpose registers must be saved, but then this kind of starts throwing stuff off for me on context switches.

#

I thought I had a grasp on how the flow is going to work...

sly pebble
#

You can try looking from a prospective of a callee saved and a caller saved from the moment a process calld yield until it returned and make sure the value are the same

#

(If its a callee saved, caller saved can be trashed before/after yield)

spring sonnet
#

Mainly because when the task is interrupted I won't know what was being used so it's best to save things as it is...

spring sonnet
#

I'll want to make sure my timer does atomic operations as well when handling the systick count or resetting it. Otherwise things might get a bit hairy.

spring sonnet
# 
__aarch64_irq_handler:
    // Save general purpose registers
    pusha 0

    // Save ELR and SPSR
    mrs x19, elr_el1
    mrs x20, spsr_el1
    stp x19, x20, [sp, #-16]!

    // Save FPU control/status registers
    mrs x19, fpcr
    mrs x20, fpsr
    stp x19, x20, [sp, #-16]!

    // Handle IRQ
    bl irq_handler

    // Check if we need to rescheule
    bl sched_maybe_reschedule
    cbz x0, __ret_to_prev

    // Switch context

__ret_to_prev:
    // Return to current task

    // Restore FPU control/status register
    ldp x19, x20, [sp], #16
    msr fpcr, x19
    msr fpsr, x20

    // Restore ELR and SPSR
    ldp x19, x20, [sp], #16
    msr elr_el1, x19
    msr spsr_el1, x20

    // Restore general purpose registers
    popa 0

    // Return to previous task
    eret

A bit better improved version of what I was trying to do.. The macros for push and pop take a "null argument" so the compiler is happy (so I just pass zero in this case).

#

Still in the works of course, but so far it's looking good...

worthy lynx
#

btw @spring sonnet have you ever tested on real hw?

spring sonnet
spring sonnet
#

Going to get started on kernel threading.

spring sonnet
#

Going to spend today studying and implementing mutex(s). I realize I'll need these as well.

spring sonnet
#

This will be interesting learning about thread blocking, unblocking, etc.

spring sonnet
#

So, from what I've seen/leanred I can make use of the TPIDR register in EL1 since my kernel will be the only one to use this, but I'll want to avoid using EL0's TPIDR register since GCC will be making use of this. I've been studying different thread implementations, and decided this is probably the best way to go on setting the current thread, etc.

spring sonnet
#

Learning threading and mutex(s) has been a pain in the ass. It's a lot to take in.

sly pebble
#

Think of the bright side, as long as youre running qemu under x86 you wouldn't likely to face the very annoying memory coherency bugs and they would only pop up when trying to run in actual hw (ask me how i know 🙃 )

spring sonnet
#

😅

sly pebble
spring sonnet
sly pebble
#

Great, otherwise you could've used TPIDRRO_EL0

spring sonnet
#

I was thinking the EL0 version of this register was reserved for use in userspace only, so I opted to use the EL1 variant instead

sly pebble
#

Yeah, there's tpidr for el0 and el1 but also tpidrro for el0, a register that's read only from el0

spring sonnet
#

I wonder if I can make use of both of them from EL1? I'm making use of TPIDR_EL1, but not TPIDR_EL0 yet

sly pebble
#

Im quite sure you need to set tls for the thread when doing user mode, you can search tls_thread_switch() in linux for instance

spring sonnet
#

Sounds good to me! I'll get right on it

spring sonnet
#

The part I'm trying to think about is how I'm going to cleanly implement keeping track of a threads quantum. My original approach was just preempt a task when the sys_tick count reached the set limit, and would switch to the next task.

#

I'd like to try and do better than this, and implement something better, so I need to spend a little bit thinking about this.

spring sonnet
#

Well, I seem to have another allcoation bug manifesting.....

#

Need to debug why kmalloc() is returning NULL trying to allocate a thread struct

#

Oh no wait! It wasn't that! It's because I currently don't have a thread struct stored in TPIDR_EL1 right now, so the check for it fails. Whoopsie

#

I think I better just make a check or something for this so it doesn't assert

spring sonnet
#

Now I somehow broke my IRQ vector code, and it crashes on IRQ return again.....

#

Ahh turns out that wasn't it. It's an issue in my thread_create() function.

spring sonnet
#

I've got so much shit to work on......

spring sonnet
#

I'll come back to the thread and sheduling stuff another time. I'm going to work on the VFS and cleanup my EXT2 driver..

sly pebble
spring sonnet
#

Thank you! ^-^; I figured I best work on some other parts of the kernel for now, and just come back to the threading, etc

sly pebble
#

You're the most productive when working on stuff that's interesting to you at the moment, make sure to keep everything you're working on on a separate branch so you wont lose track and wont have to complete an entire task just to change tasks

spring sonnet
#

I absolutely agree! Right now the VFS and EXT2 driver stuff sounded a bit better to work on for now... I did make sure to setup my kernel in a dev branch rather than a release one, so that I can make changes when I need to. I still do try to make sure the kernel is buildable though, but changes are made that might break other things temporarily

sly pebble
#

Yeah i mean not a dev branch but rather branch for each task, threading and ext2 for instance

spring sonnet
#

That actually is a brilliant idea!

#

And then I can just switch between the two...

sly pebble
#

Yeah, i mean that's standard 😅

spring sonnet
#

Well, I think I better do just that... 😅; It'll help me keep better track of things as well

sly pebble
#

Yeah and you can make small commits that will help you figure out what you were just doing earlier, and before merge you'll squash them all into one

#

I dont know why hobby os don't normally do prs honestly, at least until they are large enough that an alternative is non substantial

spring sonnet
#

I really should start doing this in more projects as well...

sly pebble
#

Probably, I wish I've learnt how useful git is during uni

spring sonnet
#

There's still things about git that I learn about that I never knew existed 😅

unreal creek
#

use git pull --rebase 98% of the time it work

spring sonnet
spring sonnet
spring sonnet
#

I didn't know git had a way to stash untracked files, and now I get to re-write my VFS code from last night 😭

spring sonnet
#

Got some VFS work to do today, and have to rework my EXT2 driver for this as well.

#

EXT2 branch has been made, and can commit ext2-specific stuff there...

spring sonnet
#

God damn it I need to redo my run script.... Otherwise the EXT2 partition ends up invalid 😅

spring sonnet
#

Got everything back in order, etc, and am getting the VFS going. Need to figure out how I'm going to rework the EXT2 driver for this...

sly pebble
spring sonnet
#

At least now I know this for in the future, but that really was not fun to deal with lol

sly pebble
#

I don't think I understand, did you delete untracked files accidentally because you thought they were stashed?

spring sonnet
#

No, when I ran git stash, created the new branch, and switched to it I noticed that it had lost a few things. That's probably also due to the fact I ran git reset --hard HEAD~3 which was probably why that happened...

sly pebble
#

A hard reset can be undo by reflog to find the missing commits - hard reset only works on tracked files
A stash can be undo via git stash pop in your new branch

spring sonnet
#

I did try to do that before, but told me the repo was up-to-date. I'm sure I might of did something else that I'm not remembering for that to not work at all, but I was able to get back to where I was previously

spring sonnet
#

Now I can get started with populating the root node for the vfs...

Screenshot_at_2025-05-07_16-54-20.png
sly pebble
spring sonnet
#

I certainly will! This was a lesson learned for sure...

spring sonnet
#

Now I'm just getting irritated with this VFS design.

#

I can't think of any other way of implementing it either.

spring sonnet
#

Mostly irritated because I got told my VFS design is "wrong".

hallow comet
#

Oof, sounds rough

spring sonnet
#

I normally don't let it get to me, but tonight just wasn't a good night for that

#

I'm just doing some small rework to my vfs_mount() to better handle what I'd like for it to do.

spring sonnet
#

I'm redoing the entire vfs.

frozen silo
spring sonnet
#

Or at least that's what I'd like to do.

spring sonnet
#

Going to re-work some of the VFS today

spring sonnet
#

I'll be redoing my VFS. The current design is just crippling.

hallow comet
#

Good Luck!

spring sonnet
spring sonnet
# 
typedef struct vfs_superblock {
    vfs_dirent_t            *root;
    vfs_fs_type_t           *fs_type;
} vfs_superblock_t;

typedef struct vfs_fs_type {
    char                fs_name[VFS_MAX_NAME_LEN];
    vfs_superblock_t    *(*fs_mount)(const char *device, const char *path, uint64_t flags, vfs_fs_type_t *fs_type);
    void                (*fs_kill_sb)(vfs_superblock_t *sb);
} vfs_fs_type_t;

typedef struct vfs_node {
    uint64_t    inode;
    uint64_t    type, mode;
    uint64_t    atime, ctime, mtime;
    size_t      length;
    char        path[VFS_MAX_NAME_LEN];
    void        *fs_private;
} vfs_node_t;

typedef struct vfs_dirent {
    vfs_node_t          *node;
    uint64_t            type;
    uint64_t            mode;
    uint64_t            offset;
    uint64_t            length;
    struct vfs_dirent   *parent;
    struct vfs_dirent   *children;
    char                name[VFS_MAX_NAME_LEN];
} vfs_dirent_t;

typedef struct vfs {
    vfs_fs_type_t       *fs;
    vfs_superblock_t    *sb;
    struct vfs          *next;
} vfs_mount_t;

So, here's the new idea I have in mind. It's minimal for now, but considering my EXT2 driver currently only supports reads (no writes yet), I think this will be just fine.

spring sonnet
#

Okay, the design is slightly being changed...

#

Okay this design isn't any better. I'm starting to piss myself off with this.

#

I'll get it figured out..

covert kayak
#

isn't vfs design fun 😔

spring sonnet
spring sonnet
#

I'm looking at Klangs VFS design for some inspiration. Tomorrow I plan to rework everything once more....

unreal creek
spring sonnet
#

And since I already have a good start on EXT2 I can just rework what I have there too to get me started

spring sonnet
#

So, after studying Klangs VFS work for a little bit, I can see it's almsot similiar to how my original implementation was. This is really nice to see that my idea wasn't absolute shit.

#

I think I'll adjust the old design, and try to make things a bit more clean.

unreal creek
spring sonnet
unreal creek
spring sonnet
#

Thing is though I notice most of these design use tmpfs where I'm going to be mounting ext2 directly as root ("/")

unreal creek
spring sonnet
#

In my case I would like to mount the EXT2 partition as the root filesystem rather than having tmpfs right now and chrooting into it....

#

I can always use tmpfs on things like /dev when it comes time to implement tmpfs

#

I mean, that's good for stuff like userspace, but it's not an elegant solution for the kernel

spring sonnet
#

Well, I realize why I was having a hard time with my old design... I should be storing the dirent struct within my vfs_root, which in turn holds pointers to nodes, etc.

#

The way I was attempting to do this before was just storing a pointer to a node, and this started to complicate things on myself.

spring sonnet
#

@worthy lynx Hey, are you still working on your kernel?

worthy lynx
#

I started a new one a few days ago but I’m taking a break rn

spring sonnet
#

I completely understand! It's always best to take a break, and come back to it when you feel up to it

spring sonnet
# 

typedef struct vfs_dirent {
    vfs_node_t          *node;
    struct vfs_dirent   *parent, *first_child, *next_sibling;
    char                name[VFS_MAX_NAME_LEN];
} vfs_dirent_t;

typedef struct vfs_opts {
    vfs_dirent_t    *(*fs_lookup)(const char *path);
    vfs_dirent_t    *(*fs_mount)(const char *path);
} vfs_opts_t;

typedef struct vfs_node {
    uint64_t        inode;
    uint64_t        uid, gid;
    uint64_t        refcount;
    uint64_t        type, mode;
    uint64_t        atime, ctime, mtime;
    size_t          length;
    uint64_t        num_links;
    struct vfs_node *link;
    void            *fs_private;
    char            name[VFS_MAX_NAME_LEN];
} vfs_node_t;

typedef struct vfs_mount {
    char                fs_type[VFS_MAX_NAME_LEN];
    uint64_t            type;
    vfs_dirent_t        *root;
    vfs_opts_t          *fs_opts;
    struct vfs_mount    *next;
} vfs_mount_t;

So, this is still based off on the old design, but make some small changes. Right now, this should be fine for getting started, and I can always add/remove stuff later.

#

This feels just as crippled as the old design. I can't think of a better way to implement this.

spring sonnet
#

Going to take a look at some vnode docs.

spring sonnet
# 


typedef struct vfs_dirent {
    vfs_node_t          *node;
    struct vfs_dirent   *parent, *first_child, *next_sibling;
    char                name[VFS_MAX_NAME_LEN];
} vfs_dirent_t;

typedef struct vfs_opts {
    vfs_dirent_t    *(*fs_lookup)(const char *path);
    vfs_dirent_t    *(*fs_mount)(const char *path);
} vfs_opts_t;

typedef struct vfs_node_opts {
    // open(), create(), etc
} vfs_node_opts_t;

typedef struct vfs_node {
    uint64_t            inode;
    uint64_t            uid, gid;
    uint64_t            refcount;
    uint64_t            type, mode;
    uint64_t            atime, ctime, mtime;
    size_t              length;
    uint64_t            num_links;
    struct vfs_node     *link;
    void                *fs_private;
    vfs_node_opts_t     *fs_opts;
    char                name[VFS_MAX_NAME_LEN];
} vfs_node_t;

typedef struct vfs_mount {
    uint64_t            type;
    vfs_dirent_t        *root;
    vfs_opts_t          *opts;
    struct vfs_mount    *next;
} vfs_mount_t;

Think this is slightly better....

spring sonnet
# 

typedef struct vfs_dirent {
    vfs_node_t          *node;
    vfs_mount_t         *mounted;
    struct vfs_dirent   *parent, *first_child, *next_sibling;
    char                name[VFS_MAX_NAME_LEN];
} vfs_dirent_t;

typedef struct vfs_opts {
    vfs_dirent_t    *(*fs_lookup)(const char *path);
    vfs_dirent_t    *(*fs_mount)(const char *path);
} vfs_opts_t;

typedef struct vfs_node_opts {
    // open(), create(), etc
} vfs_node_opts_t;

typedef struct vfs_node {
    spinlock_t          s;
    uint64_t            inode;
    uint64_t            uid, gid;
    uint64_t            refcount;
    uint64_t            type, mode;
    uint64_t            atime, ctime, mtime;
    size_t              length;
    uint64_t            num_links;
    struct vfs_node     *link;
    void                *fs_private;
    vfs_node_opts_t     *fs_opts;
    char                name[VFS_MAX_NAME_LEN];
} vfs_node_t;

typedef struct vfs_mount {
    char                fs_type[VFS_MAX_NAME_LEN];
    vfs_dirent_t        *root;
    vfs_opts_t          *opts;
    struct vfs_mount    *next;
} vfs_mount_t;

I think this should do just fine... I don't feel irritated with it either, so that's good...

#

@hallow comet Does this look sane enough for a VFS? 😅

hallow comet
spring sonnet
#

This is true haha; It feels like a better design than how I was originally doing things, so this should be fine...

spring sonnet
# hallow comet Looks good to me (although tbh I don't know what a *sane* VFS is supposed to loo... 
vfs_dirent_t *ext2_mount(const char *path) {
    uint32_t ret = 0;
    vfs_dirent_t *v = NULL;
    ext2_inode_t *root_inode = NULL;
    uint32_t ext2_idx = gpt_partition_get_index("Rootfs");
    uint64_t ext2_offset = gpt_partition_get_offset(ext2_idx);




    if (!path) {
        return NULL;
    }

    v = kmalloc(sizeof(vfs_dirent_t));
    assert(v != NULL);

    strncpy(v->name, path, sizeof(v->name));

    sb = kmalloc(EXT2_SUPERBLOCK_SIZE);
    assert(sb != NULL);

    ret = virtio_blk_read((uint8_t *)sb, ext2_offset + 2, EXT2_SUPERBLOCK_SIZE);
    assert(ret == 0);

    assert(sb->ext2_signature == EXT2_SIGNATURE);

    root_inode =  ext2_read_inode(EXT2_ROOT_INODE);
    assert(root_inode != NULL);

    v->node = kmalloc(sizeof(vfs_node_t));
    assert(v->node != NULL);

    v->node->inode = EXT2_ROOT_INODE;
    v->node->ctime = root_inode->ctime;
    v->node->atime = root_inode->last_atime;
    v->node->mtime = root_inode->last_mtime;

    v->node->uid = root_inode->user_id;
    v->node->gid = root_inode->group_id;

    v->parent = NULL;
    v->child = NULL;
    v->sibling = NULL;

    strncpy(v->name, path, sizeof(v->name));

    v->node->fs_private = (ext2_inode_t *)root_inode;

    return v;
}

A bit of a scary mount funciton.... lol

#

Not completed yet, but currently will work fine...

hallow comet
#

I think I only understood 10% of that function lol. But imma take your word on it

spring sonnet
#

Lol shounds good to me

spring sonnet
#

So basically "mounting" on "/"

hallow comet
#

ohhhh, okay that makes sense

spring sonnet
#

The VFS & EXT2 are both in their early stages, so things may change over time design-wise....

#

For now, this will work just fine

spring sonnet
#

....this vfs design feels kind of ridiculous....

worthy lynx
spring sonnet
spring sonnet
#

Got some progress made, but also just made some small changes to the VFS idea.

#

Needed some insparation, so I took a look at how Vinix was handling this, and gave me the idea to have a set of registered file systems. This will make things a bit more clean, and I can keep my vfs_mount() the way that I wanted it.

spring sonnet
#

Finally understand how my VFS design is going to work, and will start getting things in place. For now, today's task will be to read the directory structure vfs the VFS.

#

Want to keep things simple, and steady.

spring sonnet
#

So far, this is feeling like a better design than before. It's alightly modified, but it's kind of inspired by how Vinix handles mounts, and file system registering, etc.

spring sonnet
#

Feels good to get some progress made finally.

spring sonnet
#

Currently trying to decide on how to implement look-ups. I was thinking to check if an existing node was already populated for the requested entry, and if not, locate the entry within the EXT2 file system. Then, I can populate a new node(s) as needed.

covert kayak
spring sonnet
unreal creek
spring sonnet
unreal creek
spring sonnet
spring sonnet
#

Path creation if not already cached is working quite well

Screenshot_at_2025-05-11_23-25-52.png
covert kayak
#

very nice

#

what file system is this on top of?

#

oh ext2 ic

#

nice

spring sonnet
#

Thank you!! Yes, I'm going with EXT2 ^-^

covert kayak
#

do you have a devfs or equivalent?

spring sonnet
#

Not currently....

#

I do plan to implement it eventually... I wanted to get my EXT2 filesystem mounted, then take it from there...

covert kayak
#

fair. You can always have a TempFS and have that support devices rather than having devfs+tempfs being separate (that's what I do)

#

imo tempfs is a better thing to build your vfs around initially but anyway

spring sonnet
#

That's actually something I was wanting to do was implement tmpfs, and just use that to support devices, etc...

#

It'll make life much easier lol

covert kayak
#

yeah sounds good

#

cos then you can also use that same tempfs to unpack your initrd onto etc

#

(well not the same tempfs, on a different tempfs instance at a different mountpoint, but ykwim)

spring sonnet
#

Oh absolutely; I did eventually plan to support ramdisk loading, and mounting the rootfs from there, but that's very much a work in progress..

covert kayak
#

fair enough. tmpfs dev can be a little tricky to get right honestly

spring sonnet
#

Seeing others that have been working on tmpfs, I expect it to be a tough beast like EXT2...

#

But I'm willing to take on the task

#

Being able to get the VFS going finally is nice. I was fighting bad with the design decisions lol

covert kayak
#

a tempfs is simple enough, but making it genuinely good and clean is hard

spring sonnet
#

For now, I think getting a functional implementation will be good enough, then I can worry about improving it later down the road

covert kayak
#

yeah

spring sonnet
#

I'm already having to fix my ext2's internal lookup. I had some bugs I overlooked with it ending up in an infinite loop

quasi token
#

Insane work on the vfs! I’m on my first attempt right now. You wouldn’t happen to sit on any resources? Having a hard time finding any, I’ve already read Sun’s vnode paper a couple of times already lol

spring sonnet
#

Right now, I've been working on doing path lookups, and can't decide if I should just not cache nodes in the vfs for now, and worry about that part later... lol

spring sonnet
spring sonnet
#

Well, I've moved my caching node logic into the VFS layer where it needs to be. Now, I can start calling my EXT2 lookup to locate the node if it's not already cached, and this will keep things fairly clean....

#

I still eventually need to handle indirect nodes for larger file sizes, but for now I won't worry about it too much

spring sonnet
#

Well, so far root directory lookups are working great. I need to try and work on subdirectory lookups now.

#

Each nodes fs_private field conatins a pointer to it's ext2 inode in memory, so that it can access later. So far, I'm off to a good start....

spring sonnet
#

Going to start implementing the ability to open/close/read files for now. Write support is planned later in the future...

spring sonnet
#

Very much a work-in-progress, but I can actually open files now....

Screenshot_at_2025-05-12_20-52-12.png
covert kayak
#

nice

spring sonnet
# covert kayak nice

Thank you!!! I do need to work on some stuff, and have it make calls to the filesystems actually open() function, but wanted to also test path look-ups...

#

So, today's task will be working on opening/closing/reading from a file, but doing so through the VFS

#

I also need to work on permission checks.... 😅

spring sonnet
#

Doing some simple permission checks, and ensuring the operation isn't trying to perform writes (for now...). I plan to implement write support later down the road...

spring sonnet
#

Pushes the new changes to the EXT2 development branch. Right now, it's quite simple, but I'm able to open files now, and do proper checks (somewhat) to ensure I'm not attempting to open a directory itself, and that if trying to create/write/append to files I return -EROFS. Very much a work-in-progress....

#

I think next up will be adding the ability to close a file, then work on reads...

#

I do also need stuff for doing stats on a file, etc... it'll all come as I go

#

Feels nice to get some progress made

spring sonnet
#

Let's fucking go!!!

Screenshot_at_2025-05-13_23-52-12.png
#

I've got simple reads working. Now, I do need to handle other cases for things like singly, doubly, and triply indirect blocks, but for now this works.

#

@unreal creek

#

I've also pushed the codem but of course this is all a work-in-progress. Let me know your thoughts!

#

I should test placing this under /home/helix/test.txt and really test my VFS capabilities

#

Yeah... I have bugs in the VFS to fix for those cases, but I'm glad I tested... otherise, it works!!

#

I'm at 12 starts on my repo too! It makes me happy to see folks taking interst in Helix

spring sonnet
spring sonnet
#

Had a few weird deadlock bugs to fix, and had to also make sure I was using spinlocks for reads, etc. Otherwise I'd run into race conditions it looks like.

#

Still have bug in the vfs_lookup() though. Appears to be undefined behavior, so I need to spend some time looking into that in the morning.

unreal creek
#

also vfs_mount should have three argument
the type
the path to mount to
and the thing to mount
so you can so
vfs_mount("/mnt/t","ext2","/dev/vda")

spring sonnet
spring sonnet
#

I realize I wasn't freeing buffers that I allocated too, and that's not great

#

This is what happens when you code at 3am lol

#

Well, and incorrect lengths when trying to free because I overlook things...

#

Since my kfree() takes a length parameter

unreal creek
spring sonnet
#

I see I have a bug in my lookup with linking nodes. I can only open a single file at a time, etc, otherwise it shits itself.

spring sonnet
#

I'm going to redo the lookup logic. I can't seem to track down the source of the bug because during debugging everything works, but when I run it without the debugger it crashed, so I think a redo is in order.

#

To be fair I did write that code at like 3am...

covert kayak
#

ok no good I've starred it already

spring sonnet
#

I can't seem to get path lookups working correctly. I trying to debug it everything suddenly works fine, but when I don't debug it asserts. Irritating.

#

Rewrote this function three times.

spring sonnet
#

I'm getting to the point now I just want to say fucking caching nodes for a while.

spring sonnet
#

Had to adjust how I was setting up my stack apparently. It was getting corrupt

spring sonnet
#

@unreal creek I see more of the memory leaks you were talking about. I had some in my ext2_lookup() as well..

#

Getting path lookups to work has been a pain in my ass

spring sonnet
#

I'm going to attempt to re-think my node linking logic. My previous logic was leading to some undefined behavior, and was really difficult to debug.

spring sonnet
#

Going to implement helpers to handle this task. Right now, I'll test my vfs_split_path() logic in userspace that bascially provides me an array of tokens of the split path, and can go from there.

#

I also need helpers to assist with linking/finding nodes, and then requesting from the EXT2 filesysstem for the uncached node.

spring sonnet
#

No new progress updates for the VFS.

#

Had about enough of trying to design the shit because of undefined behavior, and when I go to debug shit just magically works.

sly pebble
#

Things working instruction by instru tion might mean you're lacking memory barriers somewhere (in 1 core env anyways)

spring sonnet
sly pebble
#

Not necessarily, it might mean you don't put barriers when needed
If I were you Id look into your disassembly and see which instruction "breaks" and go from there

spring sonnet
#

That's actually a really good idea. I'll spend some time to see what's up

#

I wasn't aware I needed memory barriers in my VFS at all...

#

Well, now I know...

spring sonnet
spring sonnet
#

Adding memory barriers around calls to filesystem operations from the VFS does in fact seem to fix a lot of the issues I ran into before. I may need to adjust my lookup logic though.

#

Well, at least for trying to read out more than a single file.

#

.....I realize I might have some issues in my EXT2 driver I need to look at.

#

I just feel relieved to finally get some of these issues fixed.

spring sonnet
#

this shit give me a headache....

spring sonnet
#

Going to make a small list node lib for my kernel to better manage this VFS stuff.

#

I think it'll make things a hell of a lot more clean with storing nodes, etc, than what I'm currently trying to do.

spring sonnet
#

Probably going to say fuck the VFS for a while and go work on exception handling. I'm about sick of this bullshit. Apparently memory barriers did jack shit.

#

I'm just irrirated as fuck after spending multiple days debugging, and re-writing shit, and getting the exact same result. Debugging ends up somehow making it work, and I don't really get it,.

sly pebble
#

Have you pushed your code somewhere?

lost stream
spring sonnet
spring sonnet
# lost stream ub somewhere?

That's what I had came to the realization of is that I've got some undefined behavior somewhere. The issue is though is that it was really ridiculous to debug because as soon as I start stepping through code, things would appear to work fine, and on the next run I'd run into the same issue

lost stream
#

have you implemented ubsan

spring sonnet
#

I did find a tiny ubsan libaray I can maybe setup in the kernel...

#

Adding it into my kernel now. I have a feeling my kernel might shit itself after enabling it...

#

Suprisingly no

#

Well, time to add the old VFS code, etc, and see what happens.

lost stream
#

rdmsr is on this server, though I dont remember who it was lol

lost stream
spring sonnet
#

Well, so far I can see ubsan is working correctly

Screenshot_at_2025-05-18_12-46-45.png
#

My current registers being printed though looks a bit uhh.... not right, but that's not a high priority at the moment

spring sonnet
#

Need to now rework some things since I rolled back to an older commit where things worked. I have yet to figure out why I can't open and read more than a single file yet, but I'm sure it's just an issue with my logic or something...

spring sonnet
#

@lost stream I'm glad you had be setup ubsan in my kernel. This is actually helping me catch some NULL pointer dereferences that I otherwise would have missed, and completely overlooked

hallow comet
spring sonnet
#

It's caught a few NULL pointer dereferences so far that I ended up overlooking, so it does truly help. Only thing you need to implement is print(), but this can very well just be changed to use printf, etc, since it's a single file project

lost stream
#

yup, its very nice to have

spring sonnet
#

Decided to redo the VFS entirely.

#

I've been going back to reading the vnode paper once again, but also reading about the linux-style approach, and see about maybe combining the two like I wanted to do before. I never did do so like I wanted, and ended up with a pretty bad design, but would say it was a good first start.

unreal creek
spring sonnet
# unreal creek again ...

Yeah, old design I kept running into undefined behavior. I couldn't track the bug easily, because in the debugger everything seems to work fine.

#

A rewrite was in order after spending a week and a half at it.

#

I'd rather just redesign it than spend another week on debugging, and pissing myself off.

unreal creek
spring sonnet
#

I've got tinyubsan in the project now as well, and that's actually helped a bit

spring sonnet
#

Still working on it of course, but so far it seems like it's a good start.

spring sonnet
#

Been finding some good documentations around that will assist with the design ideas.

#

I'm working on filesystem registering, ete, right now though...

#

I have to also modify my EXT2 driver as well. I ended up moving some things I was using to get stuff like the block address as a macro instead.

unreal creek
#

you need a bit more operation

#

other than that i don"/ think the vfs has to care about superblock

hallow comet
#

I was actually gonna ask about that, a VFS is just an abstraction for an actual FS no? So why would it need a superblock? (I could be completely wrong, I don't know much about VFS design, I was just curious)

unreal creek
spring sonnet
#

My previous approach was a bit bad, and I was struggling to fix some nasty bugs, so a redesign was in order. So far, I'm quite happy with how everything is turning out, but will take some time to get the bare-minimum in place...

hallow comet
#

Ahh, okay, that makes more sense

spring sonnet
#

After studying the Linux VFS docs, etc, I became really interested in the design, so I wanted to implement my own version haha

spring sonnet
# hallow comet Ahh, okay, that makes more sense

So, I did a thing, and ended up making some of my EXT2 helpers as macros...

#define EXT2_GET_INODE_DATA_SIZE(inode) (((size_t)(inode)->hi_size_bytes << 32) | (inode)->lo_size_bytes)

#define EXT2_LBA_TO_OFFSET(lba, block_size) ((lba) * (block_size))
#define EXT2_OFFSET_TO_LBA(lba, block_size) ((lba) / (block_size))

#define EXT2_GET_INODE_SIZE(sb) ((sb)->inode_structure_size)

#define EXT2_GET_BLOCK_SIZE(sb) (1024 << (sb)->log2_block_size)
#define EXT2_GET_FRAG_SIZE(sb)  (1024 << (sb)->log2_fragment_size)

#define EXT2_GET_BLOCK_GROUP_COUNT(sb) (((sb)->total_blocks + (sb)->blocks_per_group - 1) / (sb)->blocks_per_group)

#define EXT2_GET_INODE_BLOCK_GROUP(sb, ino) ((ino - 1) / (sb)->inodes_per_group)
#define EXT2_GET_INODE_INDEX(sb, ino) ((ino - 1) % (sb)->inodes_per_group)

#define EXT2_SECTORS_PER_BLOCK(block_size, sector_size) ((block_size) / (sector_size))
#define EXT2_INODES_PER_BLOCK(block_size, inode_size) ((block_size) / (inode_size))
#

I felt this would be a bit more clean instead of calling to different functions to perform such simple tasks

hallow comet
spring sonnet
#

It's all up to you honestly; However you design it is up to you 🙂

spring sonnet
#

VFS work is coming along well...

spring sonnet
#

@hallow comet Well, a bug in my heap allocator finally manifested itself 💀

Screenshot_at_2025-05-23_13-29-22.png
hallow comet
spring sonnet
# 
    while(slab != NULL) {
        uintptr_t slab_start = (uintptr_t)slab + sizeof(*slab);
        uintptr_t slab_end = slab_start + (slab->total_objects * slab->object_size);

        if ((uintptr_t)ptr >= slab_start && (uintptr_t)ptr < slab_end) {
            // Place freed object back into freelist
            *(void **)ptr = slab->free_list;
            slab->free_list = (void *)ptr;
            slab->free_objects++;
        }

        // Move to next slab in list
        slab = slab->next;
    }

So, it looks like it's here... I think it also has to do with how I'm trying to allocate, then free the directory entry name....

#

Actually, I might need to check that...

spring sonnet
hallow comet
#

AY!!! 🎉

spring sonnet
# hallow comet AY!!! 🎉

Well, now to figure out why changing my shit to use vfs_inode_alloc() and vfs_dirent_alloc() now doesn't read the next three directories...

#

Lol

hallow comet
#

oh lord... XD

spring sonnet
#

I swear it's always something 😅

spring sonnet
hallow comet
#

XD

spring sonnet
#

Found it.....

    vfs_dir->d_node->fs_private = (ext2_inode_t *)ext2_inode;

😅

#

Fok

#

Ohhh I see what I did there... whoopsie

#

I set it from within the directory entry, and not the inode itself

spring sonnet
# hallow comet XD 

    vfs_dir->d_node->fs_private = (ext2_inode_t *)ext2_inode;
    vfs_dir->d_node  = vfs_inode;

Welp..........

#

I overlooked the hell out of this...

# 
    vfs_inode->fs_private = (ext2_inode_t *)ext2_inode;
    vfs_dir->d_node  = vfs_inode;

This is how it should have been initialized....

hallow comet
#

Ahh, yeah that would be an issue lol

spring sonnet
hallow comet
#

VFS design seems like a pain XD

spring sonnet
#

Oh God it's been painful 😅

spring sonnet
# hallow comet VFS design seems like a pain XD

Well, turns out I'm a little forgetful:

            if (strcmp(tmp_name, path) == 0) {
                ext2_dir = kmalloc(d->entry_size);
                assert(ext2_dir != NULL);

                memcpy(ext2_dir, d, d->entry_size);

                // kfree(tmp_name, d->name_length + 1);
                // break;
            }

My code was doing what it was supposed to do, and break when the target directory gets found... all I had to do was comment out those two lines, and now it prints everything 😅

hallow comet
#

Figures 🤣

spring sonnet
#

I feel like an idiot for that 💀

spring sonnet
# hallow comet Figures 🤣

Going to get the start of a tmpfs filesystem in place for /dev for now. At least now that my ext2 filesystem is mounted as "/" like I wanted

#

Now, I can begin the real fun stuff lol

spring sonnet
# hallow comet Figures 🤣

Well.... my lookup semi-works, but I need to figure out what I'm doing incorrectly for setting up the /dev node information...

Screenshot_at_2025-05-24_17-24-33.png
#

I also need to fix my shit, so that it properly used the tmpfs overlay

hallow comet
#

Well... that's better than it completely not working

spring sonnet
#

This is very true....

#

At least I know my code isn't complete shit lol

spring sonnet
#

The pain in the ass part is trying to implement a way to replace the original inode information for /dev with the tmpfs inode (to act as the new "root mount" on that directory....

#

I also have to make sure to preserve the original /dev inode data, so this ends up causing me to write some jank code to make it work lmao

hallow comet
#

That does sound annoying lol

spring sonnet
#

It's a pain in my ass....

#

I have to take a lot of time to re-review my code, and logic to make sure what I want to do is correct...

spring sonnet
# hallow comet That does sound annoying lol

Well, in my design too, I have a special entry in my directory entry struct that holds the "overlay" mount. So, when I go to do a lookup, if that exists, I just replace the current "root" with that overlay, and operate on that.... will this work as I expect it to? Probably not

#

It's worth a shot

spring sonnet
# hallow comet That does sound annoying lol

The issue I'm having now, is when I call the new filesystem mount, and set it's overlay, I don't exactly store the requested entry from the original filesystems lookup. So, when I leave the context of my vfs_lookup I don't have any way of accessing that directory entry...

#

Apparently I have this really bad habbit of writing crippling code without realizing until shit breaks.

hallow comet
#

I'm pretty sure 50% of the code I write for my OS is poorly written, so don't feel too bad

spring sonnet
#

I feel better knowing it's not just me...

spring sonnet
# 
void vfs_mount(const char *path, const char *fs_type, uint32_t flags) {
    vfs_dirent_t *root = NULL;
    vfs_filesystem_type_t *fs = NULL;
    vfs_dirent_t *dir = NULL;




    if (!path || !fs_type) {
        return;
    }

    list_foreach(item, fs_types) {
        vfs_filesystem_type_t *f = (vfs_filesystem_type_t *)item->data;
        assert(f != NULL);

        if (strcmp(f->fs_type, fs_type) == 0) {
            fs = f;
            break;
        }
    }

    if (!fs) {
        printf("VFS: Filesystem %s unregistered!\n", fs_type);
        return;
    }

    if (!fs->fs_mount) {
        printf("VFS: Filesystem %s has no fs_mount()!\n", fs_type);
        return;
    }

    if (path[0] == '/' && path[1] == '\0') {
        // Mount filesystem on rootfs
        
        root = fs->fs_mount(fs, path, flags);

        if (!root) {
            printf("VFS: Filesystem fs_mount() failed!\n");
            return;
        }

        if (!vfs_root) {
            vfs_root = root;

            printf("VFS: Mounted %s on %s\n", fs_type, path);
            return;
        } else {
            // Don't overwrite current rootfs
            printf("VFS: rootfs already mounted on %s\n", path);
            return;
        }

    } else {
        // Get original directory entry
        dir = vfs_lookup(vfs_root, path);
        assert(dir != NULL);

        // Mount new filesystem
        root = fs->fs_mount(fs, dir->d_name, flags);
        assert(root != NULL);

        // Setup new mount for overlay
        dir->d_mnt_overlay = root;

        printf("VFS: Mounted %s on %s\n", fs_type, path);
    }
}

You'll see what I mean for the part that handles sub-directory mounts

spring sonnet
# 
        list_insert(vfs_root->d_children, (vfs_dirent_t *)dir);

This should hopefully fix things.

#

My lookup is implemented to handle this, but this clearly isn't the corrent way of doing this.... oh well I just want my shit to work for now...

spring sonnet
#

yeah no that's not going to work.

#

oh my fucking christ.

hallow comet
#

oof

spring sonnet
spring sonnet
# hallow comet oof

Since right now I don't currently support unmounting of filesystems, I can can just directly overwrite the original inode with the new filesystem (tmpfs), and deal with the rest later

#

Not the most ideal solution, but it gets some progress made

hallow comet
spring sonnet
#

Going to just focus on my EXT2 driver instead...

#

bit buggy, but so far it works. i can start working on openeing, closing, and reading files. need to eventually figure out the whole devices thing, but will be later down the road.

spring sonnet
#

great after fixing an overflow bug now it only prints one directroy.

#

im so over this shit.

hallow comet
#

lord, you just can't catch a break

spring sonnet
#

To make matters worse, I was stepping through my lookup code in the debugger, and of course it just works, so something's getting messed up or I'm running into undefined behavior once again...

spring sonnet
#

I'm only irritated because I've already ran into this with the original VFS design

#

And now my EXT2 code is now doing the same 😭

sly pebble
#

I mean unless you find the source of the bugs re-writing wouldn't be too effective. Perhaps you're not waiting on devices, if I were to debug it I'd look into qemu and put prints on start and completion. Perhaps there's a trace event that might be helpful but idk because I rarely use those

spring sonnet
sly pebble
#

The problem with trace events I find is that you don't know what to expect unless you know qemu code, so idk how helpful it would be

spring sonnet
#

Welp.... I guess for now it couldn't hurt just to see if I get any clues. Otherwise I'll have to try and see what else I can do

spring sonnet
#

It seems I've fixed the issue. At least after ensuring I was using memory barriers where needed

#

Let's see if this still breaks trying to read other directories...

#

Yeah apparently trying to list more than two directories just breaks things.

#

I need to see why this is happening.....

#

Yeah I don't get what's happening or why it sometimes works, and then doesn't. I'll try and see if there's anything I can do to maybe track down if something is getting messed up. Apparently trying to use memory barriers still doesn't resolve anything.

#

ugh.

spring sonnet
#

k... time to spend today reasearching and looking at event traces. Or anything else I can find.

sly pebble
#

Looking at the qemu code for virtio-blk there's a trace for virtio_blk_req_complete and inside virtio_blk_handle_request.

Idk enough about virtio blk to say if its enough, but you can put a semihosting call in your code where you expect the transaction to complete and check if the order is blk start => blk end => your semihosting or is it blk start => your semihosting => blk end

spring sonnet
sly pebble
#

Its quite easy, run configure --prefix=<your install dir> --target-list=aarch64-softmmu and then build

spring sonnet
#

Sounds good to me!

sly pebble
#

Personally I add a couple of flags to add assertions but its irrelevant for you

spring sonnet
#

I'll enable all debug features that I can as well

spring sonnet
# 
austin@ryzen-beast:/media/austin/Scratch/HelixOS/build$ ./sysroot/usr/bin/qemu-system-aarch64 --version
QEMU emulator version 10.0.0 (v10.0.0)
Copyright (c) 2003-2025 Fabrice Bellard and the QEMU Project developers
austin@ryzen-beast:/media/austin/Scratch/HelixOS/build$

Host version is like 8.x.x, so this should be a bit better. Time to check some of the debug stuff

spring sonnet
# 
virtio_blk_req_complete
virtio_blk_rw_complete
virtio_blk_zone_report_complete
virtio_blk_zone_mgmt_complete
virtio_blk_zone_append_complete
virtio_blk_handle_write
virtio_blk_handle_read
virtio_blk_submit_multireq
virtio_blk_handle_zone_report
virtio_blk_handle_zone_mgmt
virtio_blk_handle_zone_reset_all
virtio_blk_handle_zone_append

BEAUTIFUL

#

Now, I can start debugging...

spring sonnet
# 
virtio_blk_req_complete 3899.175 ... req=0x646cbf5d7fb0
virtio_blk_req_complete 2958.317 ... req=0x646cbf5d7fb0
virtio_blk_req_complete 663.827 ... req=0x646cbf5d7fb0
virtio_blk_req_complete 859.735 ... req=0x646cbf5d7fb0
virtio_blk_req_complete 1717.276 ... req=0x646cbf5d7fb0

Noticed a series of completions on the same address, but otherwise no duplictes.

#

I think my virtio driver might actually have a bug that I don't know about yet...

sly pebble
#

I would've put a semihosting call after you assume the transaction completed, and see which one happens first

spring sonnet
sly pebble
#

Its quite simple, you put the semihosting function (for example print string =4) into x0 and the arg in x1, then do instruction for semihosting. In aarch64 its hlt #0xf000

spring sonnet
#

Wait, beacuse my host OS is x86_64, doesn't that mean semi-hosting isn't going to work in this case?

#

It just now dawned on me.....

#

ffs..........

sly pebble
#

Qemu emulate the semihosting

spring sonnet
#

Well, in that case, I'll get started

sly pebble
#

No not at all, it depends on the arm machine and weather it supports semihosting, and qemu does

spring sonnet
#

Thank God... I was worried I had to have an ARM64 host, but since that's not the case, I can get started...

spring sonnet
#

Yeah, my virtio driver is a bit buggy it looks like, so it's time to fix things up.

#

I did need to fix it, but "forgot"....

#

I had to move some things out of the request submission into the init stage of the device setup.

#

Some of what of this also suffered from potential race conditions....

spring sonnet
#

And now apparently headers are "missing" trying to fix these issues... Lovely.

spring sonnet
#

fixed.

#

managed to fix those issues. had to fix an issue were I was incorrectly setting the status for writes, needed to move queue selection and queue size selection in init, needed to make sure that when waiting for a status change, I relax the CPU, now the driver works.

#

knew my driver was buggy, but kind of "forgot" about it.

#

@hallow comet Well, now I'm able to read more than two directories with these fixes, so this is nice...

hallow comet
#

Nice!

spring sonnet
hallow comet
#

Glad you figured it out!

spring sonnet
#

I do need to fix my lookup function in my EXT2 code though... I can list directories under "/", but listing things under "/home/helix" ends up not working 😅

hallow comet
#

yeah, that would be helpful lol

spring sonnet
spring sonnet
#

@hallow comet My literal expression when I seen it working lmao

2020-05-02_15.33.53_cropped.png
hallow comet
#

I would probably have that face too lol

spring sonnet
#

Lol it's always nice when things finally start working after spanding weeks trying to debug...

neon oracle
spring sonnet
spring sonnet
#

Lol I was being complicated...

spring sonnet
#

HelixOS (ARM64 OS, not dead project!)

spring sonnet
#

Trying to fix the nested paths not being read/working has been completely irritating.

#

At least I fixed my virtio-blk bugs.

unreal creek
spring sonnet
#

I don't think anyone really had interest in my project anyways

unreal creek
unreal creek
spring sonnet
#

I'm just trying to fix my path lookups, so that things work properly, and can start moving forward

unreal creek
spring sonnet
# unreal creek okay

not really sure what else anyone would really be excited about here in that case. i was working on threading, was having major issues, then decided to shift my focus back to this, since i already had it semi-working.

#

Once I get this stuff finished, I can then work on file reads, etc...

#

I do need to work on exception handling

#

I understand if folks get sick of the vfs stuff here. It's something I really want to work on, so I'm going to till I'm satisfied with everything...

spring sonnet
#

So, my virtio driver needed some locking around the submission of queues. I also had to ensure a memory barrier got used here to ensure changes were seen. So far, this seems to fix a few more issues I had, and should also hopefully fix another race condition I was facing as well.

#

I believe my code that waits for a device status to be reported after the request should also have some locking added to it. This should prevent any race conditions or concurrent access, etc.

sly pebble
#

And + for doing whatever you feel necessary in your kernel until its at a point you want it to be

spring sonnet
spring sonnet
spring sonnet
#

apparently you can't store/load stomic variables with stdatomic because of some type of "trivially-copyable type". I don't understand how this even gets used at all.

sly pebble
sly pebble
spring sonnet
# sly pebble Nice, I recall saying you've had problems there a while back

Yes, I had figured out I had some nasty race condition bugs in my code, so this was causing some of my issues with reading from disk, and not getting correct results. Right now, I'm further improving this driver, by ensuring safe atomic operations during th status check after submitting a read/write request. I'm happy with how well it's performing now

spring sonnet
sly pebble
#

With smp you'd probably just hold a lock that signify you're using the device at the moment or something similar

#

(Which you shouldn't bother until implementing smp, any smp implementation without actually testing it wouldn't work)

spring sonnet
#

This is true... I thought it would be good to have a more safe status check operation either way to ensure I don't run into issues on concurrent accesses, etc...

sly pebble
#

You're going to have trouble with concurrent accesses if two threads would use the same status concurrently either way, you wouldn't be able to tell what status is for which core

#

Fixing smp problems is difficult when you're able to run smp, its impossible otherwise

spring sonnet
#

I'm looking into alternate ways of handling this via ring descriptor index, etc...

spring sonnet
#

Well, I did notice a few bugs in my EXT2 code. In the block group descriptor code I was allocating the temp descriptor the length of the block size (this is fine), but then later was trying to free it with the size of the struct... I"m really surprised that didn't manifest itself a lot sooner....

#

I'm also learning now, that I guess directories can span multiple blocks, so that might also be an issue of mine that I'm not currently handling.

#

At least I'm getting some bugs fixed in my virtio driver, and fixing my EXT2 code...

spring sonnet
#

Reworking my EXT2 code a bit....

spring sonnet
#

I realize some of what I was trying to do might not of been worked as I was expecting. Especially trying to break in a nested loop....

spring sonnet
#

Well, I managed to fix my issue of reading nested paths. Now I can read out /home/helix in my EXT2 filesystem. I still need to figure out though why trying to print more than three directories causes things to break though...

unreal creek
#

this is not a bug

#

this is a feature

#

to prevent malicious program to make a bunch of dir

spring sonnet
#

I just realized why my vfs_listdir() might be broken for nested path lookups.... I'm not splitting the path, and handling the lookup like I do in vfs_lookup(). Or even making the call for vfs_lookup()..... it took me this long to realize this.... wow

spring sonnet
#

So, I decided to change pace, and go back to implementing kernel threads for a while.

hallow comet
#

Ayyy, threading 😄

spring sonnet
#

I can always come back to the filesystem stuff later

tender acorn
#

runtime stuff is more important imo

#

filesystems can wait

#

or you can make the user implement them with a bad disk interface

spring sonnet
#

That way, when I go to start loading userspace programs, I'll have a proper way to run them

abstract saffron
#

my eyes is burn when i see arm64 instead of aarch64

029e21084518e860.png