Nyaux

[surreal path]

oh wait

im idiot

sorry

[kind root]

...

[surreal path]

yea works now

[kind root]

nice

[surreal path]

the io port is fixed io im guessing?

[kind root]

fixed or nomrla

normal*

[surreal path]

can be both?

[kind root]

either

[tawdry mirage]

i hope you are one of the first people to use gcc 14 once it's actually out

[surreal path]

gcc 14

whats the changes in gcc 14

[tawdry mirage]

as per the c99 standard, a bunch of what are currently warnings become hard errors

[surreal path]

oh no

[tawdry mirage]

including -Wincompatible-pointer-types

[surreal path]

scary

[kind root]

its not

[surreal path]

works

thanks uacpi!

[tawdry mirage]

https://uwu.foundation/7R9xhbk.png

[surreal path]

uwu.foundation

if uacpi doesnt find devices will it return not found?

[kind root]

dont think so

why not use a struct that you will pass to callback

and store resources there and whether a keyboard was found

[surreal path]

how will i know if a device isnt found

(in my callback found_ps2)

[kind root]

it wont get called?

[surreal path]

oh ok i can do a bool then

[kind root]

well u need to store resource somewhere as well

[surreal path]

why, i dont rlly need to

i already store the port number and int number

if i do find them

[kind root]

oh ok

[surreal path]

i override whatever port and int number

yea

time to learn about a vfs now!

[surreal path]

vfs makes my brain go boom

https://tenor.com/view/good-morning-love-gif-25692527

[desert haven]

whats causing the 🤯 ?

[surreal path]

everything

😭

[desert haven]

ok

gl

any specific questioins?

[surreal path]

yea uhhh

if an inode

represents a directory or file

how does the vfs know if its a directory or file

and like

what are v nodes

[tawdry mirage]

the inode stores it's type

[desert haven]

you have a 'type' field, which tells you what it is

damn, too slow

[surreal path]

okay and how would you get the next file?

sorry * the file in the directory

[cinder plinth]

inodes are not vfs things

[surreal path]

oh

[cinder plinth]

inodes are on disk fs

[desert haven]

theres usually a dedicated function for that, like readdir

[cinder plinth]

you have vnodes typically

[surreal path]

okay and what do vnodes represent

[cinder plinth]

an entry

in the vfs

inodes are entries in the filesystem

[desert haven]

so your vfs would be a tree of vnodes, which represent files/directories/links/whatever that come from a filesystem. Then the interfaces the kernel provides usually deal with vnodes, and a filesystem driver takes care of mapping vnodes to on-disk things.

assuming you're doing single-root (unix-style) vfs, but if you want to have multiple roots you just have multiple trees (think windows with C:/D:/E: drivers)

[surreal path]

how does the vfs typically work in unix like oses, how does one implment them, yes i get a vnode is an entry in the vfs but what do vnodes contain????? what do i need to do as a kernel developer right now. i have a lot of questions

sorry idk why that message was taking so long to send 😭 internet issues

[desert haven]

so your vnode would contain everything the kernel needs to provide the vfs interface. So that'd be something like:

• linkage within the tree (parent, children, maybe siblings).
• a lock to protect the vnode itself
• a type field
• an opaque pointer for the filesystem driver, this is where it can store any filesystem-specific data.
• a pointer or handle to access the file contents.
• maybe a reference count, depends on your design

you may want to include permissions here too

[cinder plinth]

I personally just copied netbsd's api

[desert haven]

probably smart

I read one paper and decided to fumble my way from there

[cinder plinth]

https://man.netbsd.org/VOP_OPEN.9

[desert haven]

and your workflow for doing something in the vfs would look like:

• traverse the tree, find your vnode.
• check if the operation makes sense
• ask the filesystem driver associated with the vnode to perform the operation.
  So you'd need a way to know which filesystem a vnode represents.

[surreal path]

okay ill start by making a vnode struct

[kind root]

Should honestly go into resource links

[cinder plinth]

https://man.netbsd.org/vnode.9

this is also useful

[surreal path]

why is there a thing called "vfs"

what is that

[desert haven]

that represents a particular filesystem

[surreal path]

oh

[desert haven]

so which filesystem the vnode comes from

[surreal path]

what flags should i make in my vfs

for the vnode

[desert haven]

Im actually not sure what the per-node flags are used for here

[kind root]

Add on per need basis

[surreal path]

okay dont need rn

so

not adding any yet

[surreal path]

i probs need a type field to describe if the vnode represents a file, directory or whatever

correct?

[desert haven]

yeah

[surreal path]

ok so something like

[kind root]

Why is it all caps lol

[surreal path]

funny

[kind root]

And I wouldn't store that as an enum in the struct

That consumes 4 bytes

[surreal path]

and?

[kind root]

Tbh in this setup it doesn't matter

Because padding

[surreal path]

and this will be the vfs node

[desert haven]

yeah have fun typing those enum values everytime lol

[surreal path]

[surreal path]

anyways should a vnode store its name or no

[kind root]

What if its a 32k char name

[desert haven]

usually the name is only fetched from the driver when its needed

for the above reason 😛

[surreal path]

so how will say someone find a file from a path? do they loop through every vfs node, traverse the tree. call the get name driver thing from the stored function pointers in the vfs node, check if the name matches, if not keep traversing? sorry bit confused

[desert haven]

thats the right direction. So first of all you'd want a function for querying if a node has a child with a specific name, and you'd also want to provide a general 'lookup' function. There's two ways of providing that:

• You could break the path down into segments (the things between the separators) and call the "do you have this child" function for each segment until you error out or find what you're looking for.
• You could also pass the path to the node's driver and ask it to resolve as much of it as it can for you, until it fails or maybe reaches a mountpoint of another filesystem. In that case you call that driver with the remaining path.

[cinder plinth]

that's what I do

that's on a per-filesystem basis

[surreal path]

okay im really confused but maybe if i write this down ill understand better, is this what a vfs is:

each vnode represents a file or directory, stated by its type field, each vnode contains a bunch of function pointers which points to its driver equivalent functions it may need to use

each vfs represents a mounted file system, each vfs holds a pointer to the first vnode in the tree it represents

say we want to resolve path '/hello/world.txt', we break the path up into things like '/', 'hello', 'world.txt' we lookup for the root vfs, and start traversing the tree. we select a vnode. use its driver interface to check if the filename matches any part of the path. if it does thats our vnode else we keep traversing. if we dont find it we select the next vfs and start traversing its tree essentially

am i understanding this correctly?

[desert haven]

yeah that sounds right

[cinder plinth]

yes

thats how i do it

[desert haven]

your vfs only really needs to support absolute paths, so you can always start at the root node

[cinder plinth]

I split up the path then call lookup() until it's reached the file you wanna do the OP on

[surreal path]

okay interesting

[desert haven]

so in your example it would be: check if root node has hello/, check if hello/ has world.txt. No need to special case the start of the path (by treating / as a node name).

I find that simpler imo

I included the trailing slash to indicate its a dir, but dont be confused by that (its not part of the name)

[cinder plinth]

yeah you need a proper path parser

[desert haven]

yeah for sure

[surreal path]

and then if the root vfs doesnt have hello/, goes to the next vfs and checks if the vfs name is hello. if so traverse the tree to find world.txt

correct?

[cinder plinth]

no

[surreal path]

oh

[cinder plinth]

if it doesnt have it ENOENT

[surreal path]

ENOENT?

[desert haven]

the root node is the top of the tree, if it doesnt have the first part of the path (hello/), it doesnt exist within that vfs.

[cinder plinth]

"No such file or directory"

[surreal path]

okay wait

i think im understanding

my brain is cooking

wait

[desert haven]

cooking is better than cooked

[surreal path]

IGNORE THE POORLY DRAWN DIAGRAM

[desert haven]

maybe its worth saying too, in case its causing confusion, the struct vfs in the paper I linked (and the struct you showed earlier) is what I've been refering to as a filesystem driver. One of these would be your root driver which has the root vnode.

[surreal path]

yes i understand that

[desert haven]

ok cool

[surreal path]

so im gonna need this

okay and

im gonna make a function pointers thingy rn

okay

[desert haven]

get_filename()
returns nothing

[surreal path]

lmao

probs should return char*

[desert haven]

const char* would be preferable imo

or you could have it return the name length, and take in a caller-supplied buffer (that way management of the name's memory is the caller's problem)

anyway, implementation details

[surreal path]

should vnode have a entry that points to a uint64_t address that could be the inode of whatever file system?

@desert haven ?

[desert haven]

a private pointer for the driver can contain that info

so stick a void* that the filesystem driver can access when dealing with that vnode

and the driver can store what it needs there, like the on-disk address of the file

[surreal path]

oh okay

void *data?

[desert haven]

yeah something like that

[surreal path]

okay

so now i kinda defined the structs

do i need to make a function that can parse a path or what should i do first

[desert haven]

yeah you could start with that

and then after that you'd be looking at writing your vfs and a tempfs driver, which is just a filesystem that stores everything in ram. It can super simple, but it allows you to test your vfs.

[surreal path]

kk

😎

[surreal path]

is it called tempfs or ramfs

[desert haven]

lol

or tmpfs

[surreal path]

/run

initrd

tarfs

[desert haven]

tar is a good way to populate an initial tempfs though 😛

by passing it as a bootloader module

[surreal path]

yea true but i have no way of understanding how to decode a tar file !

!!!!

[cinder plinth]

tar is not a filesystem

[desert haven]

its very easy

[cinder plinth]

its quite simple

[desert haven]

lol was waiting for this

[cinder plinth]

for some reason osdev says it is

iirc

[desert haven]

oh right

[surreal path]

man how am i struggling to write a path parser

💀

[finite summit]

it's simple

you tokenize path

then look at path in fancy ways

[surreal path]

im trying to

[finite summit]

https://github.com/oberrow/obos/blob/old_code/src/oboskrnl/vfs/cwd.cpp#L26

string_view is just a struct with a cstring+its length

[surreal path]

gonna steal string functions

from glibc

[finite summit]

Getting sued speedrun

[surreal path]

okay i got a path parser done

stole some strtok thing and then used that to do a funny like this

bit cursed but eh

[tawdry mirage]

fun fact sizeof char is always 1

[surreal path]

lol

anyways

[tawdry mirage]

like guaranteed by the c standard

so * sizeof char is just useless

[surreal path]

okay made a function that parsers a path, guessing i need to make a function that resolves the vfs node from the path?

[surreal path]

ok removed it lol

[tawdry mirage]

also that logic looks wrong

[surreal path]

wdym

[tawdry mirage]

assume parse_path("/a/b/c"):
strlen(path) = 6 => buf points to a 6 byte buffer
buf[0] = p; accesses bytes [0, 8) in the buffer, so accessing 2 bytes past the end

[surreal path]

my brain 💥

why accessing bytes 0, 8

??

[tawdry mirage]

what is sizeof(char *)?

[surreal path]

8

oh

right

what should i change in this case

should i make the buffer bigger or just completely change the logic

[tawdry mirage]

buf needs to dynamically grow as you find entries (or you can scan the string twice, once to collect the number of entries, then again to collect the entries themselves)

[surreal path]

strtok finds entries for me

which i didnt write

returning a char*

[tawdry mirage]

oh that's another thing, strtok writes into the input

[surreal path]

oh

[tawdry mirage]

so this is just gonna page fault on a write to a read only page

[surreal path]

ugh then i guess ill just straight up remove strtok

this is pain

i hate working with strings

[tawdry mirage]

the compiler should've warned about casting const away

when passing the argument to parse_path

[surreal path]

okay so how should parse_path be implmented

whats the "correct" way of doing this?

?

if i wanna like

make an array of char*

pointing to each token

like

i actually dont know what to do anymore

[desert haven]

do you need to know all of the parts of the path at the same time?

I iterate through the path one part at a time

[surreal path]

no not rlly

[desert haven]

so how about a function that takes the path, and the start of the current segment, and returns the start of the next segment (or null)?

could be done using strchar

[surreal path]

strchar?

[desert haven]

I think thats the name of it

ah strchr() https://en.cppreference.com/w/c/string/byte/strchr

[surreal path]

okay but what about paths that have more then one slash like /home/meow/nyaux/kernel/whatev

this function would return home/meow/nyaux/kernel/whatev but i would need just "home"

[desert haven]

ok maybe return the length as well (so the next /)

and you can use the strncmp and friends where you pass in the length as well

no need to rely on the null terminator

[surreal path]

what?

[desert haven]

so your 'get next part of path' function returns a const char* where the part starts, and a size_t for the length of that part.

[surreal path]

you cant return 2 types at the same time, unless its a struct

[desert haven]

sure

do that then

[surreal path]

alr

[surreal path]

and where does strncmp come into this

[desert haven]

for comparing the path segment with the name of the vnode

[surreal path]

oh yeah right

[surreal path]

@desert haven something like this?

[desert haven]

return strncmp(name, seg.seg, seg.len) == 0 lol

[surreal path]

lol

[desert haven]

but that looks fine, assuming you dont call this with a segment thats empty

[surreal path]

yea

anyways im gonna make a function that looks up the vnode from the path ig

[desert haven]

cool!

[surreal path]

and returns the vnode

will i make a global variable called root_vfs or smthin?

[desert haven]

yeah 🙂

[surreal path]

alr!

how to traverse the tree, since we have parents and siblings and children

whats the best way to do this?

i cant come up with something that will traverse this tree properly

im confused on whats the best way to do this

[desert haven]

alright so you can think of a tree has having levels right?

[surreal path]

yea

[desert haven]

cool, and your path is a bunch of segments

[surreal path]

yea

[desert haven]

so you're trying to match a segment per level of the tree

[surreal path]

right

[desert haven]

so you'd track the 'current' node (starting with the root) and level (starting at 0), and check if the current node has any children that match the segment for this level.

if it does, that child becomes the new current node

increment the level/get the next segment of the path

until you either dont find a matching child (node doesnt exist), or you perfectly match the end of the path (you've found your file)

[surreal path]

okay so what if the filesystem looks like this

vnode -> vnode
|
v
vnode -> vnode -> vnode
|
v
what we looking for

im guessing that once we searched through all the children of a level, we go back x levels

[desert haven]

no backtracking

[surreal path]

oh

[desert haven]

if the current node doesnt have a child with the current path segment, you cant continue

lets say you lookup /hello/world.txt, if the root doesnt contain a child with the name hello, then you're done - return an error

same idea with /hello/world/again.txt, if you find hello/ and dont find a child with the name world/, you dont go back up a level, that doesnt make any sense.

that would be something like /hello/../world/again.txt

[surreal path]

oh right i think i know what your saying

so we set the cur_vnode to the root vfs, check for the first path seg on each sibling. if found we get the next path seg and set cur_vnode to the child of the first path seg vnode we found

and we keep doing that

i get u

thats makes way more sense

[desert haven]

yeah exactly

[surreal path]

okay easy

[surreal path]

what i came up with

i THINK it will work

[desert haven]

test it and find out

[surreal path]

how 😭

bro i dont have

a file system

i dont have a ramfs lol

[desert haven]

you can manually stick some nodes into a tree though?

[surreal path]

ig

[desert haven]

just for testing

[surreal path]

this line causes page fault???

tf

oh wait

[surreal path]

@desert haven LOOK !!!

😱

[desert haven]

hahaha cool!

[surreal path]

very cool 😎

ok so now

uhhh

what do i need to implement now?

[surreal path]

👁️

👀

[steady flume]

YUH

[kind root]

bro

if (true) return true else return false

return strncmp(node->ops->fs_getfilename(node), seg.seg, seg.len) == 0

[finite summit]

Great to say some actually progress on this and not just hopeless debugging

[ionic jetty]

Indeed

I managed to get absolutely random allocator faults out of the blue

But i‘m 99,9% convinced my allocator is not the problem

[finite summit]

That's what everyone says

It's a sign of oberrow curse

[ionic jetty]

it can't be my allocator

i know it

i've checked the logic 10 times

i've done test runs in userspace with 20 threads each doing thousands of allocations

(and checked for overflows use after frees and stuff during the testing)

nononononononono

it was krealloc

So my kmalloc returns based on allocation size either buddy pages or slab objects

And my krealloc copies the memory from the old allocation to the new

And to figure out how much to copy, it checks the slab i allocated from and then copies MIN(new_size, old_size)

[leaden hinge]

what is that

[ionic jetty]

But I forgot the get old_size from the buddys pages when reallocing a buddy page, and always assumed its a slab page

[finite summit]

Allocator bugs

[ionic jetty]

Its broken allocators and UB and weird scheduler behavioir and broken IRQLs

[leaden hinge]

so just bugs

[ionic jetty]

Btw apologies for spamming your thread nyaosmaster

[ionic jetty]

Yes

That could have been caught with testing

[leaden hinge]

so just bugs

[tawdry mirage]

btw, get_filename doesn't make much sense for a vnode

consider a file with 2 hard links to it

what name should be returned for it

[unkempt relic]

I will be happy to discuss VFS design concerns on my return

Home at 7

[surreal path]

hm?

[surreal path]

what?

[surreal path]

alright

whats ur timezone?

[shut laurel]

inode level then?

[tawdry mirage]

no, inodes don't have names

[shut laurel]

where should it sit then?

[tawdry mirage]

directory entries associate a name with an inode

[surreal path]

what

[shut laurel]

wouldn‘t vnode level work though, as every hardlink is represented by a vnode in that case?

[surreal path]

whats a hardlink, why cant fs_getfilename be a file system operation???

what the fuck is going on

im completely lost

💥

[shut laurel]

hardlink = references to the same inode

iirc

[tawdry mirage]

why would you give each link it's own vnode though? if you update the mode of one, the change should be reflected in the other clones as well

[shut laurel]

oh right, thats true

[tawdry mirage]

which would mean you need to keep track of extra data for that

[surreal path]

i am very confused

[tawdry mirage]

so a file on an unix-like filesystem is actually made up of 2 parts

the directory entry, which gives it a name, and the inode, which tracks all the other information

and the directory entries just refer to inodes by number

[surreal path]

is the directory entry structure implemented by the x file system?

[tawdry mirage]

and a hard link is just such a reference, and you can have multiple hard links to the same inode

x file system?

[surreal path]

like

[kind root]

Twitter file system

[surreal path]

bruh

😭

i mean as in

should the vnode data be a pointer to this directory entry

which in turn points to the inode

or smthin

[tawdry mirage]

what you want is an operation to get a list of the directory entries (so ls can list the entries) and some operation to get a vnode for a child by name

[surreal path]

what

[shut laurel]

so the getfilename isn‘t really needed anymore as you do it the other way around basically?

[tawdry mirage]

getfilename is fundamentally broken because a vnode may have more than one associated name, or may not have any name associated at all

the latter case happens if you have the file open, but unlink it from the filesystem

[shut laurel]

right

[surreal path]

okay im like extremely confused so i think ill explain how i understand the "unix like" vfs as of now

vfs node represents a mounted file system, can have a enum type that explains what type of file system it is. it points to the first vnode it represents and the next vfs in the linked list

vnodes represent a file or directory, they have a field that points to its sibling, its parent, and child. vnodes have a data field that can point to the data structure of the file system which points to the actual data itself.

vnodes have an operations field that allow u to do operations on said field

how i've currently implmented the vfs is i have a function that looks checks if the vnode has the same fs name as the path segment. if so return the vnode, else return null. then i have a loop that traverses the tree until it resolves the vnode from the path. breaking the path down as it goes down the tree until it finds it

path = home/nyaux/kernel.bin

finds vnode with name "home" using fs_getfilename

path = nyaux/kernel.bin

finds vnode with name "nyaux" using fs_getfilename

path = kernel.bin

finds vnode with name "kernel.bin"

what is a directory entry and how does this differ from how i've implemented my vfs

?

[tawdry mirage]

path = /home/nyaux/kernel.bin
path starts with /, start looking from the root vnode
cur_vnode = this_process_root_vnode
cur_vnode = cur_vnode->find_child("home")
assuming this succeeds
cur_vnode = cur_vnode->find_child("nyaux")
assuming this succeeds
cur_vnode = cur_vnode->find_child("kernel.bin")

[surreal path]

oh so

vfs (represents info about mounted file system)
|
v
vnode (root vnode, name is / or whatever)
|
v
vnode (name is home) -> vnode (name is etc)
|
v
vnode (kernel.bin)

[kind root]

Vnodes dont have a name

[surreal path]

oh

[kind root]

They're pointed to by a directory entry

And that has a name

[surreal path]

directory entry?

???

[kind root]

Bruh

[tawdry mirage]

from the original vnode paper:

Path name traversal is done by the lookuppn routine (lookup path name), which takes a path name in a path
name buffer and returns a pointer to the vnode which the path represents.
[...]
Lookuppn traverses the path one component at a time using the vn_lookup vnode operation. Vn_lookup takes
a directory vnode and a component as arguments and returns a vnode representing that component.

[kind root]

You have a vnode thats current directory

[tawdry mirage]

a vnode does not know it's name, what knows it's name (or rather, it's name for it), is the parent directory

[kind root]

You read that directory to find if it has anything with the name

[tawdry mirage]

and the parent directory stores that name -> inode association in a directory entry

[surreal path]

so theres a directory entry that represents a directory, and it stores the name of its child and its ptr to its inode?

[kind root]

A directory is an array of entries

Each entry has a name

And an associated inode number

That inode might be another directory

Or maybe a file

[surreal path]

inode number?

[kind root]

Unique identifier for an inode

Any object on the fs is an inode

[surreal path]

okay im sorry again im rlly lost. im gonna undo everything i did in my vfs.h and vfs.c and rlly try to understand this

is there a resource that explains how this system works start to finish for someone who knows nothing about vfs's

[kind root]

Article on ext on osdev wiki

[surreal path]

which ext

ext3 or 2 or ?

ext2 seems most

[kind root]

https://wiki.osdev.org/Ext2

[unkempt relic]

Greenwich mean time

Well

British summer time now with dst

[leaden hinge]

BST should be year-round

[surreal path]

OOHOHHHH SOOO THE DIRECTORY STORES THE NAMES OF THE FILES IT HAS

[kind root]

Finally

[surreal path]

you could have like a structure

struct directoryentry
{
  char *name;
  int inodenum;
  struct directoryentry *next;
}

struct directory
{
  struct directoryentry *head;
}

[kind root]

Terrible naming but yes

[surreal path]

how should it be named

[kind root]

Thats better

[surreal path]

so we have a global table called a inode table

an array of inodes of MAXINODENUM

and the inodenumber in a directory entry is the index into this global table array

[kind root]

Pretty terrible idea

[surreal path]

oh

then i misunderstood everything

[kind root]

I can make a file with inode number 99999999

[surreal path]

so what is inode number then

if theres no global array to index to

how r u supposed to get the inode of a file

?

[tawdry mirage]

inode numbers are specific to the underlying file system

if i have 2 ext4 file systems on /foo and /bar, inode 3 on /foo is not gonna be the same file as inode 3 on /bar

[kind root]

inode numbers are only unique with two other ids that fstat gives you I think

[surreal path]

again im completely fucking lost

[kind root]

And on some synthetic fses they don't even have to be

[cinder plinth]

dont worry about inodes if you're not gonna support on disk filesystems for now

[surreal path]

no i need to worry about inodes

[tawdry mirage]

i think (st_dev, st_ino) is a unique value for every file

[kind root]

Ye, that

[surreal path]

i need to understand all of this

why is it so complicated

[kind root]

Its really not

Focus

[surreal path]

im trying to

[tawdry mirage]

you don't need inodes to leak out of the file system driver

the vnode layer can be mostly unaware of them

(saying mostly because there might be a case i'm not thinking of)

[surreal path]

https://tenor.com/view/stressed-head-bang-gif-14502957

im gonna take a break for today on osdev cause i have a presentation due tomorrow thats worth 20% of my summer grade

[leaden hinge]

that sounds pretty important

[surreal path]

yea it is

[kind root]

How you gonna do it from Linux

[surreal path]

power point web

[kind root]

The things people do to not use windows

[cinder plinth]

we just use google slides

[unkempt relic]

I will caution you against this approach (have you read some resource about ancient unix?) On my return

[surreal path]

i have not read any resource on ancient unix

[unkempt relic]

Global inode table is so ancient unix its practically reanimation the ghost of that guy whose name began with d

Denis ritchie

Anyway I'll discuss when I'm ho.e

[surreal path]

alright

[kind root]

Hoe

[unkempt relic]

well done

you got the whole channel laughing

[ionic jetty]

HHAHAHAHAH HOE THATS SO FUNNY

you got me there 🤣

[kind root]

Lol

[unkempt relic]

this is logically (but not literally; the inode table is usually divided into cylinder groups in filesystems like UFS and the ext series, which is just UFS redone) true of the structure of traditional unix filesystems' on-disk formats

but of the in-memory data structures used by most kernels nowadays, big global arrays with a hard limit are very much out of fashion

and given the very high number of inodes you may be dealing with, impractical....

if you are familiar with the structure of traditional unix filesystems then you can imagine the vnode as a virtualised inode

however when sun designed the vfs (the PDF was linked earlier) they made sure to define it sufficiently abstractly that other filesystems could work with it

there are some cases of impedance mismatch, like FAT and its unity of inode and dirent, but even that's no great problem - you will find most filesystems can easily provide APIs like those the vfs as described in the sun paper offers, excepting obvious things like no symlinks or hardlinks on fat

note the vnode described in the sun paper has no concept of name or parent because of what qookie said

the vnode is a virtualised inode there can be multiple paths to it

hard-links

the vnode generally contains a context pointer to some FS-specific data structure so that the FS driver knows what to do with it when asked to read/write to it, make a directory in it, whatever

it is possible to create an entire unix without adding anything else to the VFS than vnodes, the sun vfs paper shows how mountpoints are done there, but the cost would be that each filesystem would really want to include a cache for directory entries to avoid having to do harder work every time someone looks up a filename, even if it were already looked up

so there is a role for having a general namecache structure for lookups, mapping filenames in a folder to their own namecache structures, with namecache structures including a pointer to the associated vnode, and a pointer to the parent namecache structure

as a general rule i recommend referring to the sun vfs (the original) before the bsd vfs, the bsd vfs was an independent reimplementation but not as thoroughly integrated and full of ugly warts like having a VOP_LOCK op which locks a vnode (originally this was supposed to be for file locking, but it's used in BSD for general synchronisation - ugly!!) , and having specific requirements in vnode ops about having VOP_LOCK() locks held

in particular VOP_RENAME in bsd is bad

in Sun VFS it accepts two directory vnodes + two filenames; in BSD it accepts both directory vnodes + both file vnodes! it would be racy but it has requirements to enter on holding relevant locks. in any case it's ugly

[surreal path]

Sorry a lot of things passed through my head

I’ll read the sun paper tomorrow

[surreal path]

I had just finished my exam

😭

I’m getting back home from school, when I get back home I’ll get reading the sun VFS paper and get grinding 😎💪🔥

[shut laurel]

think before you write anything

less bugs

[kind root]

how did it go

[surreal path]

I think I did okay

I usually don’t have to like study too much to get good marks but it was certainly difficult ig

Just gotta hope I got at least 70% or my dads gonna take my devices away probs cause he upset with me about school

😭

[kind root]

zamn

[surreal path]

Zamn indeed

[kind root]

at least you're not wasting time playing garbage games like i did when i was your age lol

how old are u again

[surreal path]

17

[kind root]

Ye

[surreal path]

how old r u???

[kind root]

25

[surreal path]

woah

😭

[kind root]

lol

[surreal path]

lmao

[surreal path]

I was wasting time playing too much valorant at 16 but now I pretty much stopped playing video games all together cause of osdev

😎💪

[kind root]

i started programming at 18

then dropped out of college at 19

now im a lead dev

[surreal path]

I’ve been programming since 13

[kind root]

yeah i started super late

[surreal path]

I started off making shitty Roblox games

💀

[kind root]

lol

[surreal path]

lol

[surreal path]

u can do that and still get a job??

as a lead dev????

Don’t u need a degree or anything

[kind root]

i started as a junior

and no u dont if u have the skills

[surreal path]

zamn

😎

[ionic jetty]

Holy shit

Youre some kind of genius

[kind root]

How

[ionic jetty]

I always hear the people who started late complaining about how little chances they have and that they should have started earlier and whatnot

[kind root]

literally my thoughts

[hasty sedge]

Where u from?

[surreal path]

Ireland

[leaden hinge]

which county

[surreal path]

cork

[surreal path]

are u irish????

[leaden hinge]

half

my dads side of the family live in galway for the most part

[surreal path]

i was born in ireland but my parents are arabic

[surreal path]

my dad goes to galway to work

😭

[leaden hinge]

well all over ireland really but mostly galway

from cork?

[surreal path]

yea

3 hour drive lmao

[leaden hinge]

thats a fair distance

yeah

[surreal path]

yea

cant believe another osdev lives in ireland too

😱

[leaden hinge]

i dont live there

[surreal path]

oh

[leaden hinge]

just some of my family

[surreal path]

ah okay

[leaden hinge]

my dad was born in cork though

[surreal path]

cool

[kind root]

@ny

fuck

@surreal path do u speak irish

[surreal path]

no

[kind root]

how

[surreal path]

im not irish i was just born here

[kind root]

but still

dont people around you speak it

[leaden hinge]

a lot of irish people dont

[surreal path]

no

[leaden hinge]

its not the native language

the government keep trying to push it but everyone just speaks english

[surreal path]

lit ^

[leaden hinge]

except for a few people on the west coast islands

but they are dying out anyway

[surreal path]

yea

[kind root]

lol

[barren agate]

me too 🤝

[unkempt relic]

teaching of irish in ireland is notoriously shit

every irish adult beyond the age of 25 i meet here can't say anything those who finished high school more recently usually have a few gaelic phrases and little more

however every single one of them supposedly learnt it at school

[surreal path]

i avoided every single irish class

my brain is just gone

💥

bro

im lost

read the sun paper still dont get it

i dont know how to implement this

why am i so stupid

ugh

all i understand currently is that the virtual file system abstracts whatever file system and each vnode represents a directory or file and each vfs represents a mounted file system

---

|vfs| -> |vfs|

---

V -------------V
vnode (directory) -> vnode (directory, is a different file system)
V V
vnode (file) vnode (file)

i dont know how things like a directory entry relates into all this

should a directory entry be implemented by the file system, yada yada yada

a lot of questions

i wanna make sure I REALLY

understand what im doing

before i start writing any code

i need to make sure this is UNIX like, and this is the correct way to design and imeplment a vfs

fuckin hell

and i rlly dont know how inodes relate to this

i wish i wasnt this stupid

okay im reading this paper about "practical file system design" by the guys who made beos or whatever

has stuff about vfs shit here

[surreal path]

Okay so

if the vnode type enum is a directory and we are running the function vn_lookup(), vn_lookup is implemented by the file system and returns the child vnode of whatever it found from the path. FOR THIS CASE with a tmpfs file system, we can assume that the vnode is a directory structure in the data pointer and we can look through each array of dir_entry until we find our guy and the dir_entry could point to its vnode?

something like that?

???

[kind root]

yup

[surreal path]

oh easy!

now i understand !!!!

😄

😎

should vfs have a static variable that points to the root mounted VFS ????

[surreal path]

yea i get that!

[surreal path]

wait a process can have a root vnode???

tf

[kind root]

yes

https://man7.org/linux/man-pages/man2/chroot.2.html

[surreal path]

not gonna worry abt this for now

[surreal path]

what if path DOESNT start with /?

[tawdry mirage]

use the processes current working directory vnode instead of the root vnode

[surreal path]

wha

💥

[unkempt relic]

then thou proceedest from thine current working directory

[tawdry mirage]

the thing that cd changes

in your shell

[surreal path]

wait

so each process stores its current working directory?

[ionic jetty]

yes

[surreal path]

but how will u get the root vfs? if the path is absolute and has a '/'

how will u get the root file system

[unkempt relic]

you store a "root vnode" pointer in the process' structure

[surreal path]

oh

[kind root]

bro never used a relative path in his life

[unkempt relic]

also a global one typically

[tawdry mirage]

the global one is assigned to init's root vnode ptr, and all other processes inherit it

[surreal path]

oh

ohhhh

okay so one second im gonna go into my scheduler and change a few things

cause this is currently how i represent a process

i dont have a "process" struct

do i need to change how i've implemented this in order to continue?

[unkempt relic]

you can either define a process in distinction from a thread or you can put what would usually go into a process struct into independent structs which threads can share pointers to

[surreal path]

yea okay so what im gonna do is have it so a thread points to some common struct with other threads that describe the "root vfs" and anything per process related

but my scheduler will still see threads as "just processes to switch to"

i think this is a nice easy approach

[kind root]

yeah that doesnt make much sense

[surreal path]

wdym

[kind root]

this sentence

[surreal path]

oh yea fair

sorry lemme reword

my scheduler still sees everything as just a thread, it switches to. thats it. thats all it does, goes to the next thread in the linked list thats all it does. if you want process data to be shared between threads. threads can point to a shared structure describing info per process

[kind root]

thats usually how it works

[surreal path]

yea okay

[surreal path]

vnode? or vfs node?

[unkempt relic]

vnode, as chroot lets you set root to be a non-mountpoint-root

[surreal path]

alr

something like?

i probs should store pid in process info

[unkempt relic]

yeah, something along those lines

[surreal path]

alr

and ill make a function from my scheduler that gets the process info of the current executing thread

😎

[kind root]

where page table

[surreal path]

page table is stored in cpu_context_t

[kind root]

w..why

its a per-process thing

[surreal path]

along with the stack

oh right

i shouldnt be storing the rsp in this struct to be fair

as stackframe has rsp

can threads have separate stacks?

i think they can right?

[kind root]

what

[surreal path]

can threads have separate stacks

[kind root]

how can they not

just randomly overwriting each others data?

[surreal path]

alr

i should store the process name in process_info

[kind root]

threads can also have names

[surreal path]

fair

[leaden hinge]

nice naming convention

[surreal path]

_t

[kind root]

all over the place

[leaden hinge]

i mean the lack of it being the same

[surreal path]

if i have a function create_process, what should i set the cur root dir to?

?

okay for NOW im gonna store a global variable in vfs.c that is just the root vfs and set the process vfs initally to that (FOR NOWWWWW)

[leaden hinge]

the current working directory of the calling process

[surreal path]

i dont have that

[leaden hinge]

then yeah just have a default of like / or something

[surreal path]

as in i dont have user mode and i dont have any calling processes

[surreal path]

okay

[leaden hinge]

just some path that makes sense

[ionic jetty]

Kernel could get system root simply

[surreal path]

wha

[ionic jetty]

The root mount fs or whatever u wanna call it

[surreal path]

okay heres what im gonna do FOR NOW

gonna make vfs_resolve_path just use the root vfs if the start of path is '/' and return unimplemented otherwise

i just want something WORKING now

[leaden hinge]

you probably dont even need working directories right now

[surreal path]

working directories as in process working directories?

yea i dont need that rn

[leaden hinge]

yes

[surreal path]

does the vfs structure look like this
vfs (root mount)
V
vnode -> vnode -> vnode

or
vfs (root mount)
V
vnode (root)
V
vnode -> vnode -> vnode

lil confused

?

someone pls tell me

[kind root]

the latter

[surreal path]

should vnode root be of type directory?

[kind root]

what are the other options

[surreal path]

yea fair

[surreal path]

somethin like this

we can never expect v->type to be a file

[unkempt relic]

you want res to be a two-star parameter (⭐ ⭐) and to dereference it when you assign it (⭐ res = whatever)

[surreal path]

huh?

why???

[unkempt relic]

C doesn't have call by name your assignment is merely changing the local value of the pointer res

[surreal path]

oh wait i should return res

wait

wait wait

second

ur right

ok its a **res now

[unkempt relic]

and then *res = whatever; to assign it so that the caller can see it

[surreal path]

yep

ok and vnode shouldnt need to contain pointers to its parent or siblings or whatev

cause all we need is to read the directory entry in the vnode-> data

yea that makes sense

[tawdry mirage]

pointer to siblings and parent has the problem as having a name

[surreal path]

yea pretty much

a vnode still should point to its vfs tho, so we know information about its file system?

ig

[tawdry mirage]

yeah because you can't have hard links across file systems

[surreal path]

hard links are vnodes that point to the same file system node?

ig?

?

[kind root]

Yes

[surreal path]

yea alr

[tawdry mirage]

well at the file system level hard links are directory entries that refer to the same inode

[surreal path]

wha no i thought directory entries were like this

you check the type of the ptr_to_vnode and check if its a file or directory

[tawdry mirage]

for tmpfs sure, but on like ext2

the dentry is just a (name, inode number) pair

on disk

[surreal path]

can inode represent a file or directory?

[kind root]

Yes

So you would have struct inode that would have a vnode pointer

And inode would be stored in a dir entry

Like you have here

[surreal path]

should i make a tmpfs_inode or smthin?

[kind root]

If you want to support links

[tawdry mirage]

for tmpfs you can work on vnodes alone ig

[kind root]

Hmm

Probably

[surreal path]

um

i dont know what to do

what do u guys think i should do, implment tmpfs_inode or just work on vnodes alone

[kind root]

A hardlink would just increment the refcount for a vnode then?

[tawdry mirage]

yeah

[kind root]

I guess

[surreal path]

refcount?

[kind root]

Reference

Count

[surreal path]

why do u need to keep track of that

[kind root]

Because you might have multiple directory entries referring to the same node

And if one is deleted you dont wanna wipe the other

[surreal path]

why would deleting a directory entry cause wipe the other directory entry

it doesnt matter if they point to the same node

[kind root]

If you leak the nodes then it wouldn't

[tawdry mirage]

if the vnode is unused you want to deallocate it

[surreal path]

oh right, if ref count is zero u'd wanna deallocate it

yea fair

[tawdry mirage]

so you need to keep count of how many users still exist

[surreal path]

yea fair

testing the vfs is super mother fricking cursed

[kind root]

Lol

[frigid cliff]

Yeah, as someone who is writing a 100% rust OS, rust is kind of a pain for OS dev

[surreal path]

[leaden hinge]

viper when

[surreal path]

😎

😎

how do i make it so i dont have to manually create all these dir and dir entries and stuff

this is pain

[frigid cliff]

You write an FS driver 🤷‍♀️

[surreal path]

i did

well

a basic one

tmpfs

well not rlly

💀

just implemented the lookup function for the tmpfs fs

[frigid cliff]

lol

[leaden hinge]

a function

[surreal path]

what function?

[leaden hinge]

that does that for you

and you pass name or whatever

[tawdry mirage]

you add vnode ops to create files and directories

in a directory vnode

[surreal path]

im thinking sec

which vnode operation would it be

im reading the sun vfs paper

[tawdry mirage]

vn_create and vn_mkdir if you look at the vnode paper

[surreal path]

will nm be void *?

[tawdry mirage]

const char *? since it's a name

[surreal path]

oh

nm is a char *?

where is vn_read and vn_write?

[unkempt relic]

vn_rdwr

also strategy, bread, brelse overlap

but don't worry about those artefacts of old unix design

[leaden hinge]

i was thinking 🍞

[surreal path]

uiop?

[surreal path]

bread?

[leaden hinge]

b read makes a lot more sense

[tawdry mirage]

sunos-specific structure?

[surreal path]

what is uiop for

[tawdry mirage]

it is a paper about sunos after all

[surreal path]

wheres the offset in bytes field

😠

[unkempt relic]

https://youtu.be/VersLJjqRGY?t=79

[tawdry mirage]

uiop has the count and buffer ptr ig

[surreal path]

guess ill just make my own fields

😎

cause im a rebel! i deviate from STANDARDS

rebel sounds

[leaden hinge]

i just found out you could click on a youtube video inside of discord and watch it right there instead of opening the browser

[unkempt relic]

it's an old unix structure used for i/o particularly to/from userspace

[surreal path]

not gonna worry abt it and add my own fields

this is MY kernel afterall, it is unix like 😎

(so far)

first im gonna do the simple vnode ops

like vop_mkdir

if i implement my vfs and a tmpfs file system by TODAY

will that make me a cool kid?

will that make me not a beginner osdev officialy or will i still be considered some beginner osdev

😠

[unreal hill]

step 1 of being cool: don't ask if you're cool or not.

[surreal path]

😠

[leaden hinge]

there is no such thing

[steady flume]

i dunno, but i think it's pretty baller the speed at which ur developing this kernel

[surreal path]

its because i have no life

💀

[steady flume]

oh well

it's still cool to look at

[surreal path]

TODAY VFS AND TOMORROW USERLAND 😤

(real)

[leaden hinge]

depends on your definition of userland

could probably fairly easily get processes running in userspace tomorrow maybe even with a few basic syscalls

assuming you have various things done already

[surreal path]

we are very fucking close to userland

[surreal path]

yea i have a lot of stuff already

basic ah scheduler, pmm, vmm, slab allocator, apic driver, acpi, ps2 keyboard

[leaden hinge]

and vfs now

[surreal path]

yep!

and a tmpfs file system hopefully

🤞

[leaden hinge]

what are the vfs files currently

[steady flume]

i saw that

[surreal path]

wdym?

[leaden hinge]

the vfs is just an abstraction over several filesystems

so which one do the files you showed above belong to

[surreal path]

they dont belong to anything, i lit just made some empty vfs structs

[leaden hinge]

right

[surreal path]

and i made a tmpfs lookup operation

and a tmpfs directory and directory entry

but i need to make a few operations

now

[leaden hinge]

[surreal path]

mkdir
create
rdwr

i make these three operations for the tmpfs and i will have a basic ah tmpfs

and then i will try to parse a tar file

and populate the tmpfs

[leaden hinge]

what about lookup

[surreal path]

and use it

[surreal path]

done already

[leaden hinge]

how about remove

[surreal path]

uhhh

i will also do that ig

🤷‍♂️

[leaden hinge]

that isnt as important for a while really

[surreal path]

yep

with v_mkdir can we expect vnode to be a directory?

and not a file?

?

okay i will asume so

i will also create a new vnode?

and alloc that on the heap

ig

and ig tmpfs_dir_entry on the heap too

something like?

i think

oh and also for v_create

what is E?

what is M?

???

im so confused lmao

[unkempt relic]

read, write, that sort of thing

[surreal path]

what?.

[unkempt relic]

files can be opened for reading, writing, both

[surreal path]

so it makes it read only or write only or whatever?

or smthin

[unkempt relic]

it's for e.g. permission checking

[surreal path]

ok then i wont worry abt it

[surreal path]

wait it probs better if i allocate the name

cause name field would be invalid if i just did vn_mkdir(.., "directory1", ..)

yea that would be better, i will memcpy the null terminator as well to the name

[unkempt relic]

of note, sunos did a comprehensive integration of the VFS, while 4.4BSD didn't. all pipes in solaris are vnodes in the fifofs for example, while in BSD derivatives only named pipes are. so the open mode can be important in solaris: https://github.com/illumos/illumos-gate/blob/e2a8479967f5e2ae9ba0aa265cfa175d7874b057/usr/src/uts/common/fs/fifofs/fifovnops.c#L223 take this for example

[surreal path]

wha

sorry i dont understand

i dont even know what are pipes

💀

only that its those things echo "blah" | grep "blah"

or smthin

[unkempt relic]

i should clarify, the permission checking might actually be irrelevant because i think the access vop is used instead for this. the importance is around e.g. arranging that if someoen opens a pipe for reading, you wake up the writers who were waiting for something like this to happen

[surreal path]

im not gonna worry abt this for the moment

when i need pipes and stuff and when i understand what pipes are and how to implement them ill change my vfs and stuff

[unkempt relic]

fair and valid

you will understand the concerns aronud them in due course

[surreal path]

yep

[unkempt relic]

for now suffice it to say that in the BSD copy of the VFS, they didn't integrate it completely as it was in sunos, so not every read() and write() call from userland will go through a vnode. but this detail will be of interest only when you are dealing with pipes, device nodes, sockets, that sort of thing

[surreal path]

okay

oh and for vnode_create, is the function expected to actually create the tmpfs node for the file vnode? or no

[unkempt relic]

the create vnode op should create a file, according to whatever that means for a filesystem, and return a pointer to its vnode

so for tmpfs it should create a tmpnode

[surreal path]

alr

!

something basic

okay these two functions are done 😎

now to make the actual hard part

vn_rdwr

also what is vn_open and vn_close?

im not gonna worry abt that for now

guessing for vn_rdwr, 0 is read and 1 is write

for the read operation, if the file dont exist i gotta make it i guess?

i suppose

[tawdry mirage]

read can only be done on an already open filee

and you can't open a file that doesn't exist

[cinder plinth]

O_CREAT: hold my beer

[surreal path]

already open?

what

[tawdry mirage]

if you have a vnode then the file already exists

[surreal path]

what if the tmpfs_node.size is 0

meaning that the tmpfs_node doesnt have any memory backing it, there is nothing to read from

[elder shoal]

you read 0 bytes

[cinder plinth]

oh qookie while you're here

[surreal path]

[cinder plinth]

do you have a statically linked mlibc linux executable real quick

[tawdry mirage]

statically linked no

[surreal path]

do u need that for ur os?

[tawdry mirage]

i have some test binaries on hand, but they're dynamically linkeed

[cinder plinth]

I'm trying to run static glibc executables atm and I feel like mlibc would be easier first

[surreal path]

or wha?

[cinder plinth]

yea

[surreal path]

ah okay

[cinder plinth]

running linux binaries

[surreal path]

[unkempt relic]

you likely need to do nothing for a regular tmpfs file

[tawdry mirage]

if the file is empty read just returns 0

[surreal path]

ok

[unkempt relic]

/* ARGSUSED1 */
static int
tmp_open(struct vnode **vpp, int flag, struct cred *cred, caller_context_t *ct)
{
    /*
     * swapon to a tmpfs file is not supported so access
     * is denied on open if VISSWAP is set.
     */
    if ((*vpp)->v_flag & VISSWAP)
        return (EINVAL);
    return (0);
}

solaris' current implementation of the open vop for a tmpfs vnode

the only thing it does is ban swapon of a tmpfs file

[surreal path]