EvalynOS

[rotund furnace]

the kernel has an address space

and each process has one

[hazy saddle]

how hard is it to check for allocated things in the rbtree for me to free up the entire shit

[rotund furnace]

not very

[hazy saddle]

and the reaper thread does all that anyways so its async

[hazy saddle]

okay let me commit my crap and then work on the ram block device

il prolly use a 2 layer 256 index per radix tree

because easy

and if your ram disk needs more than 256mib well tough cookies

[hazy saddle]

amazing

absolute cinema

[peak cloak]

theyre never up

[hazy saddle]

Okay I think for fork

I’m going to have two syscall paths

One that saves all registers

One that only saves the minimum ones and stuff

And then on fork() and others that need them all

It will use that one

Otherwise it’s a partial save

And then fork() will make the new process and change all the mappings properly

With CoW

And then il have a special thread function to kick start it that will do a full restore of all those registers that I saved from the original call

And once that one starts I return from the systemcall

And that should just work™

[hazy saddle]

Good thing I’m always up :3

Up as in awake at 3am

[hazy saddle]

i think im gonna put my rambd driver in src/kernel/drivers/block/

since its gonna be simple and two files

wait no i can do src/kernel/drivers/block/rambd and then have the abstraction shit in i think im gonna put my rambd driver insrc/kernel/drivers/block/

[hazy saddle]

well on the git server side i got captchas working with cloudflare because its free forever afaik

and restricted mode does not prevent you adding things so

il just manualy moderate it

🥀

the captcha will block most craps

[peak cloak]

cloudflare

[hazy saddle]

Dose @sterile plank cat flare have a captcha hosting

I’d like to use the better flare

The one of the cats

[peak cloak]

how would i know

[hazy saddle]

Your a cat right

You all share brain cells

Or something

Idk I’m a dragon

[peak cloak]

i say no

[hazy saddle]

Darn

[mortal bane]

thsi gives me an idea to impliment a function called strtroll

[hollow crater]

strtorickroll better
u give it a number and it outputs a random ascii encoded excerpt of never gonna give u up with the same amount of hex digits as the result long int

[hazy saddle]

Okay so

The rambd and ext2 and other stuff is gonna wait

Because @devout path has been working on it

And I had no idea that they got started

So I’m gonna wait til they can give me some info on the stuff they have done

And we can get it on dev branch or some such

Or work on it together

[hazy saddle]

well if i dont get a response tomorrow il do things ig

After the talk with qookie I know then general lay out of the block layer

The filesystem itself will cache blocks it’s self as needed

In a private section

And when you MMAP from there you get non complete pages as non present

Otherwise you get the page to use etc

If it faults the kernel tells the filesystem hey yo fill in this block / tell me what it is

And then it does and it resolves the fault

And all that info will be on page tables somewhere

So you can see if it’s a mmap region and where to go to resolve etc

And then later when I support OOM reclaiming

All of the cached pages will go into the place to be reclaimed etc etc etc

And that has a list of all the page caches with the physical ranges etc

And any user app referring to a page/region is on a list

So they can be unmapped all at once properly

So you MMAP a 2mb file

The pages making that thing up are uhh added to the reclaim stuff

And you as the only referer to it

If you just opened it

And if three address spaces have it they are all added

And it also links to your tracking for the mmap on the thread

[peak cloak]

is this a vfs or a manifesto

[hazy saddle]

So if you have to shootdown some pages

You can go through it and find the right things and mark then

Idk il need help when implementing this too

But I think I have a general idea

[peak cloak]

this is insanity

[hazy saddle]

Yesnt

[hazy saddle]

This is right though right

[peak cloak]

[peak cloak]

i sadly can not understand computer people speaking

probably

this looks way too insane for a vfs though

or whatever this is

i think you talked about making a vfs

[hazy saddle]

This is block layer

[peak cloak]

idk this looks right-ish(?)

[hazy saddle]

With caching

And filesystems

And MMAP working

😭

Idk maybe I can make this simpler

This will be a pain to implement

[peak cloak]

for my vfs i just want to do driver calls (sorta like system calls but for drivers) and if the fart doesn't respond i will execute them violently and they will never be seen again in true unix fashion

[hazy saddle]

Yes but a filesystem has to talk to a block devices

[peak cloak]

yeah

thought of that too

[hazy saddle]

And that needs page caches

[peak cloak]

what

[hazy saddle]

Block cache

When you read a block

You copy it into memory

And it’s cached

And that’s gonna be annoying to manage

[peak cloak]

thats a sector

[hazy saddle]

[peak cloak]

see i cant speak to computer people they speak complete jargon

[hazy saddle]

It’s a Whatever the fuck I don’t care you know what I mean

[peak cloak]

whatchamacallit

[hazy saddle]

Idk @devout path maybe had a plan for this

[peak cloak]

scenario:

• AWAKEN FILESYSTEM DRIVER
• find block device
• check block devs for file system
• no, commit death
• yes, proceed
• construct directory tree in vfs
• idk wait for user action, if a directory is accessed make more crap in the vfs, maybe unload some old crap

if the process dies, all the memory it allocated (the vfs nodes are allocated under the process) is evaporated into nothing and kvass continues as if nothing happened

[hazy saddle]

Yes but MMAP referring to those cached blocks

May need to be shot down

[peak cloak]

at least thats the plan for my thing

[peak cloak]

block device is called, it writes to some temporary buffer, and the buffer is copied to some memory that would be owned by the fs driver

if the hard disk driver writes some BS, then why would i care its the kernel, not a sentient being

the most complex thing would probably be allocating the stupid buffers and then mapping them under each process

[hazy saddle]

MMAP would refer to the same pages as the page cache of the filesystem it gets from the block device

So on fault I need to find the filesystem

And then have it get the block etc

And then something

[peak cloak]

what does mmap stand for

[hazy saddle]

Idk I’m sleepy

[peak cloak]

oh wait

brainfart

[hazy saddle]

https://man7.org/linux/man-pages/man2/mmap.2.html

[peak cloak]

booo linux

[peak cloak]

oh

uhh

scary crap

[hazy saddle]

@peak cloak yeah this entire block cache is the managarm system kinda

And I’m scared to implement this XD

[peak cloak]

managram

[hazy saddle]

https://tenor.com/view/phineas-and-ferb-major-monogram-i-hate-myself-phineas-ferb-gif-25887797

[mortal bane]

*managarm

[hazy saddle]

Oh wait the in ram block device kind of never needs caching

So I just won’t implement that

Later I can implement it when I have userspace block devices and fs drivers

But the in kernel ones are purely in memory

So I can design it a bit more for that

[hazy saddle]

Okay so I think I’m gonna go do the initrd today properly

With the in memory block device and ext2

This will be more of a separate block layer and filesystem

As the main sauce will be in userspave for filesystems and block devices

but the VFS is still in kernel

So I can route things properly and such

I’m gonna have to do kernel/filesystem/vfs

For vs shit

And then kernel/filesystem/fs.c and .h

For the abstraction shit

And then kernel/filesystem/rambd for the ram block device

And kernel/filesystem/ext2-rbd

For the ext2 driver that expects to talk with the ram bd

It’s still gonna be ext2 but lacking a lot of the other caching shit

That will be implemented with a mix of the ideas I got from the talks with managarm and @cyan bison

https://tenor.com/view/sunnyfication-gif-20299408

Time to cook

[hazy saddle]

Okay I’ve eaten food

Gonna drink some water

Play a bit of video games

Then get on this 🙏

Oh and take my fucking adhd meds 😭

But I think my kernel VFS file ops will be
open
close
read
write
seek
ioctl
mmap

And then the backing file/filesystem will handle these calls themselves

If they have to deal with files etc

[peak cloak]

[hazy saddle]

Wut

[peak cloak]

ok nevermind

i thought why does the vfs itself needs file operations

i forgot that device drivers live there

[peak cloak]

where's "new file"

[hazy saddle]

The VFS also has to route things to the right place and function pointers exist for that

[peak cloak]

nevermind i remembered the drugged up unix standard of "open with some flags is new file"

[hazy saddle]

You open it with a special flag

[peak cloak]

that part is stupid imo

i mean it makes sense

but idek

it makes total sense and i hate it

[hazy saddle]

When you make a file your still opening it too

[peak cloak]

true

make a file without opening it

[hazy saddle]

No

[hazy saddle]

welp time to get started

https://tenor.com/view/ghost-in-the-shell-keyboard-gif-7519694

[hazy saddle]

good enough for now

@cyan bison what you think about this? so ext2 for the initrd has a block like semantic thing to use properly

[cyan bison]

the initialization of idx1 and idx2 dont make sense

that, or the first if check is invalid

[hazy saddle]

idx1 points to a table with all the leaves and idx2 is the leaf

maybe the if check is bad

[cyan bison]

the if checks if idx1 and idx2 are zero

[hazy saddle]

[cyan bison]

and fails if they aren't

[hazy saddle]

i need to do a ~ dont i

[cyan bison]

~ is ambiguous

where would you add it?

[hazy saddle]

the 0xFFFF

[cyan bison]

num & ~0xffff makes sense

[hazy saddle]

yeah

this should be proper

thanks for catching that

[cyan bison]

yeah lgtm

[gentle void]

you might have to do integer promotion here?

[urban mulch]

What why

[gentle void]

as in ~(uint64_t)0xFFFF

not sure

[cyan bison]

0xffffUL

[gentle void]

same thing

[cyan bison]

except one isnt ugly

[hazy saddle]

well this works to be safe

now i can make ext2 ontop of this and then have the tarfs get dumps into there

[gentle void]

UINT64_C 🤓

[hazy saddle]

https://tenor.com/view/noelle-holiday-noelle-berdly-berdly-deltarune-deltarune-gif-13695066025043214185

what even is that btw lmfao

[urban mulch]

No, 0xFFFF is a signed integer constant and unless you're on 16-bit, the sign bit will be zero here and it will behave as expected. Including expanding to 0xffff_ffff_ffff_0000 after the implicit conversion in num & ~0xFFFF.

[gentle void]

macro that gives you the literal suffix for that type

[hazy saddle]

where does it come from

[urban mulch]

UINT64_C(1234)

Not 1234UINT64_C

[hazy saddle]

this is nasty

but works ig

i can skip it tho?

acording to you

[urban mulch]

I mean I'm making a C compiler, I'd hope I got this right

[hazy saddle]

[cyan bison]

is that what lily-cc is?

lol

[urban mulch]

Yes

[hazy saddle]

btw will your compiler support clang function overloadding in C

[urban mulch]

Eh

[cyan bison]

wtf is C function overloading

insanity

[hazy saddle]

clang supports that

and i love it

[cyan bison]

what

[urban mulch]

Wasn't thinking of doing it, but would be technically possible.

[cyan bison]

insane

[gentle void]

pls no

[hazy saddle]

i want my kernel to compile with lily CC one day and i plan to use that clang feature

[hazy saddle]

[gentle void]

don't do it

_Generic exists

C++ too

[urban mulch]

Ehhhhhhhh

[gentle void]

menix >>>> because it only needs gnu23

"""only"""

[urban mulch]

Yeah but Lily-CC isn't doing C++ within the next 10 years so

[gentle void]

good

tbh i don't really need c23

i could depend on like c11

[hazy saddle]

i mean i dont use it right nowww

[gentle void]

but i like the [[ attributes ]]

[hazy saddle]

but i do know my kernel got fucked up with GCC

idk why

[cyan bison]

https://tenor.com/view/i-saw-what-you-deleted-cat-gif-25407007

[urban mulch]

That means you did UB

[gentle void]

i always target gcc and clang

[cyan bison]

i use gcc to compile obos and then i use clang for the gh actions

[gentle void]

same

[cyan bison]

and it usually requires a prayer before each git push to ensure that the clang build boots

[urban mulch]

I use GCC for what little C remains in my kernel and clangd for the IDE

[hazy saddle]

uh o

[cyan bison]

i think it needs a plugin to handle LTO objects...

[urban mulch]

You did not add the LTO plugin to your linker

[hazy saddle]

how do i do that

[urban mulch]

Look it up 💀

[hazy saddle]

i am

duddbt wirj

[gentle void]

imagine not using a good build system

smh

[hazy saddle]

i addted flto here

[urban mulch]

Not how it works

[hazy saddle]

google lieddd

or reddit or some crap?

[urban mulch]

There's an actual plugin whose name I forget you need to add

-flto is at the compiler, not linker

[gentle void]

lol

reddit opinion

[urban mulch]

Also 10 years old, fat chance some random crap changed

[hazy saddle]

the fuck is it getting -m from gcc if i try and use it as the linker

[urban mulch]

But anyway yeah I use GCC as my linker. Or rather. GCC invokes the linker on my behalf.

[hazy saddle]

yeah it just straight up tripple faults on GCC

😭

right away

nothing gets printed

where is the UB

clang is better anyways

[cyan bison]

it's there

[hazy saddle]

im calling it a GCC issue

and ignoring it

[cyan bison]

im calling it a skill issue

[hazy saddle]

talk cheap send patches

[cyan bison]

are your requests volatile?

[hazy saddle]

yes

[cyan bison]

damn

[cyan bison]

step through a line or two with a debugger

[hazy saddle]

im too lazy

i wonder how dumb it would be for limine to install an IDT before the kernel for exceptions like this

[cyan bison]

very

[urban mulch]

Very

[hazy saddle]

it seems the linix tmpfs is also using page sized files

presumably for MMAP but id have to check source

Which means having a thing backed by regular blocks isn’t a horrid idea though I can do better than the dumb basic ram block device but that isn’t that hard to rework

And the only overhead on ext2 is a bit of the extra book keeping’s which again I can change it easily because it would still use the same VFS interfaces

Though I will say having things split between kernel and user could be a bit cursed but again Linux does support FUSE so

I think my continued research on this is telling me that it’s not that bad of an idea and still has some worth as even if I replace it I can reuse the code elsewhere and I still learn

Aswell the documentation on TMPfs refers to PAGE_SIZE in one spot so it does seem the backend of it uses page sized things

(On the Linux kernel)

[hazy saddle]

And i would potentially have to keep track of similar if i can using physical page backings properly though i should possibly look into how tmpFS handles things in source

[hazy saddle]

I mean I guess I could handle block sizes less than the page size by caching it fancy but that wastes memory and is annoying

Especially for a ramfs where I’d basically be leaving 3/4th of a block unused for 1kb block sizes

Unless also handled merging or some shit but that sounds complex

Needlessly atleast and most filesystems are natively block page sized anyways

Maybe I’m blinded by my own stupidity here but I can’t see why this system would cause major issues and wouldn’t have some benefit atleast in some way down the road

[gentle void]

it also does this for ext

a thing called page cache

[hazy saddle]

yeah the blocks of the filesystem should be page sized, how does it even deal with ones that are less than page sized

[gentle void]

wdym

like i said, page cache

[hazy saddle]

like you have 1 kb block filesystem will it manage to get mounted properly? how does it page cache with that?

[coarse oasis]

fill the page with multiple blocks?

or 0s if past eof

[gentle void]

yeah

[hazy saddle]

[gentle void]

page cache is a cache with kernel allocated memory

not device memory

[coarse oasis]

block size > page size is not allowed for ext though iirc

or rather, linux will refuse to mount

[hazy saddle]

the page cache is related to the file its self and not the actual backing blocks and is a cache of the entire file then?

i am dum

[gentle void]

that's why it's a cache

[hazy saddle]

though for a tmpfs it still makes sense to make them equal

[coarse oasis]

yeah the page cache reflects the actual file contents

[gentle void]

you can implement a tmpfs purely with a page cache

[coarse oasis]

this is important to keep in mind for ext2 for things like 0 blocks too

an ext file can contain a hole where the blocks in the middle use block number 0

which is not a real block but instead just memset(buf, 0, block_size)

[hazy saddle]

This suddenly makes way more sense

So then I need to look into a page cache proper

And how to map a file to the pages it contains

And other kinds of files

And then it’s create a vnode from every file and an inode for some basic tmpFS state to handle this

[coarse oasis]

$ mkfs.ext4 test -b 8192
Warning: blocksize 8192 not usable on most systems.
mke2fs 1.47.3 (8-Jul-2025)
mkfs.ext4: 8192-byte blocks too big for system (max 4096)
Proceed anyway? (y,N) y
Warning: 8192-byte blocks too big for system (max 4096), forced to continue
...
$ sudo mount test /mnt
mount: /mnt: wrong fs type, bad option, bad superblock on /dev/loop0, missing codepage or helper program, or other error.
       dmesg(1) may have more information after failed mount system call.
[40398.476137] EXT4-fs (loop0): bad block size 8192

[hazy saddle]

I had a miss understanding that the page cache represents the status of the block on the disk and not of the file

This makes alot more sense

😭

Okay what would be the proper data structure to hold the references of physical pages for such a page cache for the file

[gentle void]

rbtree

probably

[hazy saddle]

@rotund furnace time for me to licence your rbtree then

how do uhh do rbtress index though exactly

[coarse oasis]

managarm uses a radix tree :^)

[rotund furnace]

:P

[hazy saddle]

i only know how fix sized rafdix trees work with paging

i really should brush up on data structures at some point

[cyan bison]

use freebsd rb tree

[hazy saddle]

how hard is it to port

and whats the licence like

[cyan bison]

copy paste tree.h, done

license is BSD license idk

 Copyright 2002 Niels Provos <[email protected]>
 All rights reserved.
 
 Redistribution and use in source and binary forms, with or without
 modification, are permitted provided that the following conditions
 are met:
 1. Redistributions of source code must retain the above copyright
    notice, this list of conditions and the following disclaimer.
 2. Redistributions in binary form must reproduce the above copyright
    notice, this list of conditions and the following disclaimer in the
    documentation and/or other materials provided with the distribution.
 
 THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
 OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
 IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT,
 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT
 NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

that's the license in tree.h

oh and i meant openbsd not freebsd

all these BSDs are really confusing man

[hazy saddle]

https://github.com/openbsd/src/blob/b822cddf2b35c5483340e21b33714e1b40bf82bd/sys/sys/tree.h

this one

[cyan bison]

yes

[hazy saddle]

how do i comply with this

do i just like fucking uhh put thing in my credits.txt thing

Mm mm then I use this to index by the “virtual” pages in the file that makeup the cache

Then then I convert bytes of the file into that

And then it should just works

And then this makes a page cache for the filesystem

Which TMPfs is built upon

To have a real tmpfs

I think I get it now

Thank you once again @coarse oasis and @cyan bison and @gentle void I am no longer being dumb XD

[hazy saddle]

this fucking looks like black magic voodoo

how do you use this

its full of macros

control f shows no mallocing

[cyan bison]

because it doesn't

here's an example of how you define an rb tree:

struct shit {
  uint32_t key;
  // whatever...
  RB_ENTRY(shit) node;
};
static inline int cmp_shit(const struct shit* lhs, const struct shit* rhs)
{
  return (lhs->key == rhs->key) ? 0 : (lhs->key < rhs->key ? -1 : 1);
}
RB_HEAD(shit_tree, shit);
RB_PROTOTYPE(shit_tree, shit, node, cmp_shit);
// In a source file...
RB_GENERATE(shit_tree, shit, node, cmp_shit);```

to search the RB tree: RB_FIND(shit_tree, &tree, &key)
to insert into the RB tree: RB_INSERT(shit_tree, &tree, shit);
to remove from the RB tree: RB_REMOVE(shit_tree, &tree, shit);
and you can figure out the rest on your own

@hazy saddle

[hazy saddle]

Ahhh makes sense thank you

For VMA how should I handle these, i did notice there are some search functions

[cyan bison]

key can be anything fyi

[hazy saddle]

Could it be my mother

[cyan bison]

idk

she might be too fat to fit in RAM

[hazy saddle]

ERR OUT OF MEMORY

Exactly

[cyan bison]

wdym

[hazy saddle]

Like if I need to find something with any key but with a large enough range or some crap

[cyan bison]

#define in_range(ra,rb,x) (((x) >= (ra)) && ((x) < (rb)))
inline static int pg_cmp_pages(const page_range* left, const page_range* right)
{
    if (in_range(right->virt, right->virt+right->size, left->virt))
        return 0;
    if (left->virt < right->virt)
        return -1;
    if (left->virt > right->virt)
        return 1;
    return 0;
}
#undef in_range```

this is what i do for page range RB trees

[hazy saddle]

[cyan bison]

and the search is as shrimple as:

    page_range what = {.virt=addr,.size=OBOS_PAGE_SIZE};
    page_range* rng = RB_FIND(page_tree, &ctx->pages, &what);
    if (!rng)
    {
        OBOS_Debug("Fatal Paqe Fault: No page range found for target at 0x%p\n", what.virt);
        goto done;
    }```

[hazy saddle]

Wait that code looks strange

[cyan bison]

ok

it's worked for one and a half years

[hazy saddle]

I meant as in

I don’t know what what is

[cyan bison]

look at the example

there is a cmp_shit function which compares two struct shit for the rb tree

[cyan bison]

this does the same but for page_ranges

[hazy saddle]

ahh

[cyan bison]

for the page_range rb tree

u also said smth about the VMA, my VMA just does RB_FOREACH to iterate over the rb tree

to allocate shit

[hazy saddle]

but page_range what = {.virt=addr,.size=OBOS_PAGE_SIZE};
what does addr equal? i dont know what address im allocating? and i would fill the tree up with like TONS of data or some crap for every page

[hazy saddle]

oh

[cyan bison]

this is for the pf handler i should've mentioned

[hazy saddle]

Ohhh

And that’s to resolve virtual to physicals or some shit?

[cyan bison]

no

[hazy saddle]

I’m sorry I’m so stupid

[cyan bison]

it's my kernel not yours ur not stupid

the pf handler gets the page_range to find out why the fault happened

[hazy saddle]

Ahhh

So then my VMA could say have a range for every free bit of virtual address space?

[cyan bison]

sure

[hazy saddle]

Because I want to allocate a range of virtual addresss to like put like annon allocations on

Then I can go through the tree and iterate through it to find a free range?

But I’d also want to check used regions

[cyan bison]

yeah

[hazy saddle]

Would I use two RB trees?

So I track used regions too

[cyan bison]

no

one rb tree for used regions

why would you need a rb tree with free regions

[hazy saddle]

[cyan bison]

now this is what i do but there might be better approaches

[hazy saddle]

So if I only track used ranged

How would I find free ones?

Sounds complicated

[cyan bison]

https://github.com/OBOS-dev/obos/blob/master/src/oboskrnl/mm/alloc.c#L38

[hazy saddle]

Maybe because it’s on my phone and slightly sleepy I don’t understand this

But it’s basically iterate through the RB tree and because it’s self sorted you just stop when you find a gap or the end of the used stuff or something?

[cyan bison]

yeah

[hazy saddle]

Ok so basically to allocate I will look through the tree for gaps. Allocate within the gaps and then give them out which is O(n)?

And then to free I just remove it from the tree and then check the paging tables and remove them and invalidate pages in the TLB though would I have to check if that entire table is free? I don’t know how I could use this data structure to also handle that

[cyan bison]

wdym by "if that entire table is free"

?

but otherwise yeah that's what you do

[hazy saddle]

Like in page tables if I need to free a say PML2 or some such

When it’s empty

[cyan bison]

your mapping code stays the same.

[hazy saddle]

I mean for unmapping

Because I don’t handle that right now

[cyan bison]

bruh

[hazy saddle]

And the code I replaced did but it scanned the entire fucking thing

So I tossed it in the rewrite to come up with something better later

As I never unmapped anything before

[cyan bison]

just check if the PML2 is empty...?

[hazy saddle]

[hazy saddle]

Isn’t that slow though

[cyan bison]

if you want it to be

[hazy saddle]

To unmap a page I have to check up to 512*4 entries no

[cyan bison]

noi

[hazy saddle]

Depending on worst case

[cyan bison]

512*8

oh wait

i misread

but yeah that's what you do

[hazy saddle]

So

There really isn’t any better way lmfao

Welpppp

[cyan bison]

there could be

[hazy saddle]

Hm

[hazy saddle]

What better way oh lord oberrow

[cyan bison]

God knows

[hazy saddle]

Though I mean how offen do you unmap pages for it to become an issue

[cyan bison]

ask him

[hazy saddle]

God

[cyan bison]

....

[hazy saddle]

What

I could do it asynchronous actually

And just change the one PTE

Then put it in a queue to check

That sounds like a mess though

[cyan bison]

if ya like that better than a memcmp then sure

[hazy saddle]

Oh wait yeah it would just be a memcmp between a zeroed page and the PTE

Okayyy so reworked todo list

• port the RB tree thing and make it working and learn it
• support un mapping pages
• implement the VMA
• then finally make the proper tmpFS using the rb tree and proper page caching etc etc

Then redo the VFS

And the rbtree there can even be indexed by a structure that has the inode number AND the “block” number for the page cache

Why am I scared to start implementing this and I feel like I’m not going to be able to and fail

Maybe tomorrow il feel better

[peak cloak]

scary

[hazy saddle]

:O

first try btw

[hazy saddle]

kernel mode now works with FRED

now i just have to make userspace work right

its this simple actually too

to convert the frame

actualy

this may need to be changed

it will

crap

il need the reverse operation

ya know i should just make my interupt shit be off a table

so i can make a really simple one for both fred & normal idt

ehh thats later me

nah id have to convert somwhere

goal number 1 is to be number 1 ay this anywasys

so i can optmise later

[gentle quest]

Is there any real hw with fred?

[hazy saddle]

the new pantherlake chips

[gentle quest]

Cool

[hazy saddle]

they did of course had to ban iretq

and its now GPFing

when switching to ring3

on the eretu

i removed the extra eretu

and i had that code in reverrse order

same GPF

[hazy saddle]

ok here we go

i should prolly make a generic structure for these

or use a union

but eh

[rotund furnace]

pantherlake has it?

I thought only clearwater forest and diamond rapids

[hazy saddle]

lets fucking go

first kernel with working FRED

Well besides Linux

But I don’t think that counts

When Intel themselves

Made the patches

Lmfao

Right @rotund furnace

[rotund furnace]

windows :P

first hobby kernel

[hazy saddle]

True

Third kernel in the world then though

First hobby

[cyan bison]

now implement smth useful

jk that's nice

[hazy saddle]

Nah I am

But this was a fun side quest set out by @marble ridge

And I get a cool role

[hazy saddle]

Just don’t look at my shortcuts

Because I cut a few corners

Which I will figure out later

[rotund furnace]

minus the part where you completely forgot how stacks works :P

[hazy saddle]

-# shut up

Works on both FRED and non FRED systems

I could maybe make extra abstractions between the two drivers

For certain things

but

It’s not that bad and just a few extra if statements

Abstractions being function pointers like with my timer system

I will be formatting my patches soon @rotund furnace

🙏

Which means you can be the second hobby kernel to have this

https://www.youtube.com/watch?v=jl0JGChm5cI

i am so fucking happy at this lmfao

maybe more than i should be

but yeah lmfao

[hazy saddle]

i wonder what it is

:o

here we go

https://git.evalyngoemer.com/evalynOS/evalynOS/commit/8d8a0e60735a4a17b8fa86532f3b9a4271f4e497

[marble ridge]

yep

good job!

[rotund furnace]

figuring out how simics works was harder them impling fred

[peak cloak]

next step: call them "fred" from now on

[mortal bane]

ngl the blocky text and the sharp colors make it kind of hard to read and interpret

[hazy saddle]

https://youtu.be/abaHbYLEiZc

Perfect

[hazy saddle]

zoom in

but thats me getting a FRED interupt

btw this shit is so based

it handles swapGS

and a bunch og fucking bullshit

and you have a seperate function for ring3 and ring0 entires

[hazy saddle]

If you call me this I will curse you to use Intel simics for the rest of your life

[devout path]

hi

[hazy saddle]

Hi

[soft snow]

huh nice role

:D

[hazy saddle]

Thank you thank you

https://tenor.com/view/deltarune-chapter-ch-3-ant-gif-10438824724713779749

[hazy saddle]

Lmao

Didn’t I post that here

[hollow crater]

ig yea i stole it from u xD
I just found it somehow relevant

cool role btw owo
congrats for the fred-friendly kernel certificate

[hazy saddle]

Friendly?

The implementation is ass

Well somewhat

I need to fix it XD

Make it cleaner

But it working is what matters

Well I should get started on the rb tree things

With virtual memory allocations

And also make exit() work

So it cleans up the process

And frees the memory

[hollow crater]

lol well i was gonna examine the commit once im done with the things in hand rn
but I took a look at the spec yesterday and it was fairly straightforward soo
im not expecting to see anything super wild xD

[hollow crater]

[hollow crater]

I hate rbtrees
it took me way too long to get it right only for me to find some bugs later that slowly brought my impl back to be almost a 1:1 of that of clrs

😭

but yk it works™
at least i didnt vibe code it

[hazy saddle]

It’s more so my conversion of FRED frames to the other frames

It’s just a bit messy and I could make it better later

But it would need more reworks I CBA to do rn

[hazy saddle]

I’m uhhh

Just taking the openBSD one

I just need to figure out how to use it right

[hollow crater]

fair

[hazy saddle]

Once I get bash working I’m going to flesh out the fred stuff and other interupts to be nice

[hollow crater]

o I need to look it up
ive seen few implementations that loon nowhere like the one I have which kinda made me feel insecure about its performance lmao
but tbh I cant care less since i scrapped the only instance where I needed it only recently
I would eventually rely on it again for the scheduler and perhaps on the vmm too
but whatever eh

[hazy saddle]

The BSD one is fucking macro soup

[hollow crater]

Fred deserves a good treatment for sure 🙏

[hazy saddle]

Once VMA works well

[hollow crater]

damn

[hazy saddle]

I’m going to make a proper tmpFS

Then fork

And exec

And posix_spawn because that should be easy

Oh and il properly handle FD 0 1 and 2

[hollow crater]

I dont think im gonna do anything beyond the classic unix interface for starter :P

so nah no spawn 🥀

[hazy saddle]

spawn is posix sooo

And it’s a decent one 🥀

Like makes things faster

[hollow crater]

real

but I still cant care less

like not anytime soon

[hazy saddle]

Wait why the fuck does it still use fork()

[hollow crater]

id rather go and play around an aarch64 port than to actually implement more sophisticated functionality unironically

[hollow crater]

[hazy saddle]

Well

🥀

I can’t wait to make bash work

Idk how directory’s work tho

I know they are files

And contain the shit

But like how does the user program expect to get the data

Or some crap

But uhh

Yeah I’m
Gonna have a tmpfs

inode = 0 should be root or some shit

I think

So you can then find folders from there

[hollow crater]

o iirc theres a call for reading directory entries

[hazy saddle]

Yes

But how do I implement that

I mean I should read POSIX

I’ve just been beating around the bush for full research on that

https://man7.org/linux/man-pages/man3/readdir.3.html

I don’t think this is the systemcall

Maybe I could look at fucking sysdeps or some shit

From astral or other

But it seems it gives your a FD

You readdir on

And every call gives a seperate entry

Yeah this makes sense @hollow crater

[hollow crater]

yes
well first ig readdir is itself a wrapper around the syscall

wait a sec

ye
getdents ig
which works on a normal fd u get from opening the directory
instead of the directory stream stuff the wrapper defiens

[hazy saddle]

Yeah il have to look at sysdeps

Probably

I could also just make my kernel give the posix shit

But that would need me to malloc inside userspace?

[hollow crater]

ooo idk :P

[hazy saddle]

Maybe I

Should just ask somebody later

[cyan bison]

wait a sec

[hazy saddle]

What

[cyan bison]

there is basically one syscall for reading directories

[hazy saddle]

Hm?

[cyan bison]

I am looking for a code sample wait a sec

[hazy saddle]

True true

I presume my VFS should expose these too

Along with mkdir etc

[cyan bison]

https://github.com/OBOS-dev/obos/blob/master/src%2Foboskrnl%2Fvfs%2Fdirent.c#L569

[cyan bison]

up to you

[cyan bison]

this is what I do

[hazy saddle]

I mean is there a better way to do that

Maybe an IOCTL

[cyan bison]

uh no directory operations would definitely go in vfs

ioctl would just be a weird place to put imo

[hazy saddle]

I could just make everything an IOCTL

/s

And I could make every systemcall an IOCTL

Just IOCTL() is the only systemcall

[hazy saddle]

Looks easy enough

And mlibc gives me the buffer

[peak cloak]

yeah nerd

now you're actually a nerd

not even unironically

this crap is beyond me

[hazy saddle]

@cyan bison the VMA is just a used list inside an rb tree for you to find gaps in right

iterate through

and check gaps

and break when found or at end

[cyan bison]

yeah

[hazy saddle]

and then i can use that to also track of annon allocations? and shit right

or shold i do that diffrently

yeah il also have it store the phsyical pages of VMAs

[cyan bison]

yeah

[cyan bison]

what

[hazy saddle]

or what other way should i do it?

dig page tables?

[cyan bison]

no i dont understand that

[hazy saddle]

in the VMA allocation of the TB tree

also put in where it is physicaly

if possible

[cyan bison]

no

you get that from the PTs

[hazy saddle]

so yeah dig the ptes

of the region

[cyan bison]

yeah

[hazy saddle]

this will work

then i can have a proper exit()

💀

@cyan bison

wana know what my exit() is?

its literaly

while (1) {

}

[cyan bison]

bruh

[cyan bison]

not even like calling thread exit or anything??

[hazy saddle]

No

I don’t have any of that

Implemented

[cyan bison]

how

[hazy saddle]

Lazy

Processes never clean up memory either

Because of that

[cyan bison]

just like free the thread info and remove the thread from the scheduler queue

like a good three lines

you know what talk is cheap, time to send patches

[hazy saddle]

Yeah but the other things

Like the annon pages

And other shit

So I was lazy

[cyan bison]

yes but we are talking about threads

[hazy saddle]

But like I don’t have fork or exec yet

[cyan bison]

and yet those are not related to threads exitting

[hazy saddle]

But if it works don’t fix it

[hazy saddle]

When the program fully exits though

I have to free the pages and shit no

I just was lazy

And put the entire thing off

[cyan bison]

https://tenor.com/view/no-gif-23548728

[cyan bison]

pov me

[hazy saddle]

what if i needed two instances of a tree dynamicaly? or should i have one VMA tree and have part of the key stuff have a key for the PID etc?

[cyan bison]

if you need two trees for the same node then u make a new tree structure

[hazy saddle]

i need a tree for every process no?

[hazy saddle]

what is &tree here?

do i allocate the tree and how

[cyan bison]

a reference to the tree struct

[hazy saddle]

what

i did this

[echo cloud]

I’m doing that unfortunately

[hazy saddle]

https://tenor.com/view/guaton-computadora-enojado-computer-rage-gif-14480338

@rotund furnace Do you know how the openBSD tree works

[rotund furnace]

no

I wrote my own

[peak cloak]

no it aint

its

i cant even get doom into an infinite loop of nothing

the exit doesnt do anything

[cyan bison]

congratz

u have a data type u can use to define a tree

[hazy saddle]

yeah. what actualy makes the tree. like whats the way i allocate it?
is it ⁨rb_tree_test_tree⁩

[cyan bison]

yes

⁨c struct ⁨rb_tree_test_tree tree = {}; struct rb_tree_test test = {.key=5}; // Of course, here test would probably be dynamically allocated RB_INSERT(shit_tree, &tree, &test);⁩

[hazy saddle]

damn im dum

[cyan bison]

evalyn u know how u keep specs on ur file server

it would be cool if u also kept the RFCs

for

[hazy saddle]

you got the links

[cyan bison]

major protocols

[hazy saddle]

il throw em up

send them all my way

[cyan bison]

TCP
https://datatracker.ietf.org/doc/html/rfc9293

TCP Congestion Control
https://datatracker.ietf.org/doc/html/rfc5681

UDP
https://datatracker.ietf.org/doc/html/rfc768

IPv4
https://datatracker.ietf.org/doc/html/rfc791

IPv6
https://datatracker.ietf.org/doc/html/rfc8200

ARP
https://datatracker.ietf.org/doc/html/rfc826

ICMPv4
https://datatracker.ietf.org/doc/html/rfc792

ICMPv6
https://datatracker.ietf.org/doc/html/rfc4443

uh that is all i remember

for all of these if u go to info->other formats

u can download pdf

or just add .pdf to the end of the url

[hazy saddle]

https://files.evalyngoemer.com/share/manuals/inet/

How is this @cyan bison

[cyan bison]

good

ill keep u updated for other specs that are interesting enough to go there

[hazy saddle]

Fuck it I’m putting the amd64 manual back up even if it’s technically not allowed

What do you think of that

[cyan bison]

sure do it

[hazy saddle]

i added this to be safe

i also thew HTTP and IRC and nvme up there @cyan bison

[cyan bison]

yeah i see

[hazy saddle]

this is a really neat archive

i dunno if its good enough for #resources

[cyan bison]

also put RFC2018 for TCP selective ACK

nice

[hazy saddle]

i mean it would be fuckin based if the mods allowed it

You think this should go in #resources

[cyan bison]

yes i think it deserves to

[hazy saddle]

Any other manuals before I full send it

[cyan bison]

nope

[rotund furnace]

intel fred manual :P

[cyan bison]

it's already there

!

[rotund furnace]

damn

[cyan bison]

:!)

[hazy saddle]

:3

[cyan bison]

new emoticon unlocked

:..(

[hazy saddle]

Hey you! Are you having issues finding manuals for osdev or similar work?

I have a website that mirrors and archives a bunch of technical manuals useful for things.

https://files.evalyngoemer.com/share/manuals/

How’s this @cyan bison

[cyan bison]

lgtm

[hazy saddle]

Now can you ping a mod so I can have permission to do this

[cyan bison]

@gentle quest

it will be a bit, he is probably asleep

[hazy saddle]

Yeah I can wait

But this site is really kool

[cyan bison]

i agree

[hazy saddle]

Maybe I could get manuals for some network cards?

[cyan bison]

network cards are all over the place

[hazy saddle]

I mean atleast the one realtek and that one intel one

Those are the two popular ones

[cyan bison]

especially considering like 50 vastly different NICs all fall under the same umbrella term "e1000"

which is "that intel one"

[hazy saddle]

God damn

The e1000

[cyan bison]

same thing with "the one realtek" which is the "r8169" driver but is actually like 30 different NICs

[hazy saddle]

Nah fuck that shit

[cyan bison]

like im telling u it is fucking impossible to find specs for ts

[hazy saddle]

But I mean

That’s the point of the site

[cyan bison]

i didnt even write my e1000 driver

i took it from fucking freebsd

so did managarm

my r8169 driver works on exactly 1 (one) piece of hardware

that doesn't even use the rtl8169

[hazy saddle]

[cyan bison]

(which is also why i have no idea why I have the bug where i just get 4 byte garbage frames on the rx queue)

and when writing my r8169 driver

i only found the spec on the osdev wiki

[hazy saddle]

https://www.intel.com/content/dam/www/public/us/en/documents/datasheets/i210-ethernet-controller-datasheet.pdf

i can get manuals like these

[cyan bison]

yeah now get the other 49

[hazy saddle]

this isnt even an e1000

[cyan bison]

oh i lied apparently

[hazy saddle]

this is an intel i210

[cyan bison]

there are ||192||

[hazy saddle]

the nic my server uses

[hazy saddle]

fuck me

[cyan bison]

different variants of the e1000

at least, there are 192 different PCI device IDs for it

[hazy saddle]

https://tenor.com/view/family-guy-quagmire-glen-jaw-gif-22600043

[cyan bison]

luckily for us all, intel made their freebsd e1000 driver open source

[cyan bison]

oh and it's portable

u define kernel api kinda like how you define uacpi kernel api except not really at all

[hazy saddle]

thats good atleast

[cyan bison]

yea

[cyan bison]

all u need to do is implement the rx/tx queues

and handle IRQs yourself

the driver handles all the messy link stuff and whatnot

[hazy saddle]

http://realtek.info/pdf/rtl8139cp.pdf

i did find this

[cyan bison]

that's the rtl8139

which is a different device than the rtl8169

[hazy saddle]

[cyan bison]

it's older

and also does not fall under the same linux driver iirc

but it is quite similar ive heard

[hazy saddle]

why cant nics be standard

[cyan bison]

all NICs have different purposes

some faster than others

some slower

some with different features

many different companies who make them

it might not make the most sense for something like NICs to be standard like say, how XHCI is standard, imo

uh anyway what am i doing i need to implement congestion control

[hazy saddle]

Using my manual site :o

[hazy saddle]

Also why infy lol

[hazy saddle]

I mean more of a standard operating mode. And the goal of a nic is to well interface with the network. They could easily add a NIC spesific section of the spec or some crap

[cyan bison]

infy is my favorite mod

oh yeah fun fact i was the first to ever get uacpi to run on real hw

and the first kernel to ever have uacpi kernel api

of course this was in rewrite #3

[hazy saddle]

Oh damn

[cyan bison]

https://github.com/OBOS-dev/obos-old/blob/old_code3/src/drivers/generic/acpi/impl.cpp

[hazy saddle]

I am now the first kernel to ever get FRED working

I’m making a name for myself here

I just uhh

Have to fucking make the VMA

[cyan bison]

hey at least u got fred to support you while youre making your VMA

[hazy saddle]

I will sit upon this throne for the rest of my life

and never do anything

/s

Nah I really wana get bash and shit working

[cyan bison]

ofc yeah

having a network stack is so cool icl

like you are telling me

MY code

is communicating with someone on the other side of the goddamn world

[hazy saddle]

Real

I wonder if I should do TCP

Or USB first

[cyan bison]

network stack is a lot simpler

[hazy saddle]

I will also have to port python

[cyan bison]

and TCP was a good learning experience for me

[hazy saddle]

To run copy party

[cyan bison]

it's actually easy i found

[hazy saddle]

For my file server

[cyan bison]

(debated)

having a simple TCP/IP stack feels pretty good

[hazy saddle]

I wonder for FD 0 1 and 2 I should pre allocate them on proc create

[cyan bison]

i got a gdb stub running on UDP and once i was able to debug my kernel on real hw remotely from my PC it was very cool