#Zinnia
1 messages ยท Page 11 of 1
@hoary cave lol i can reproduce the crash in qemu
if you set the ram size to like 4gb it page faults on the ap
oh noooooo
that's so stupid
what do you mean cr3 has to be a 32-bit value
:(((((
ap startup?
yes
only 32bit values
yep
ill make a reusable page table for 32 bit then
then i dont need to map the trampoline pages either
why cant you use your normal kernel page tables? the root of that should be under 4G?
also holy shit 1024 cpus
doesn't have to be under 4G
it has to be 32-bit
it does not
huh?
that won't work
why
you can't switch to kernel page table + jump to higher half in one instruction
you either need higher half mapped in the temporary <4G page table or something else
i was planning on doing that
having a link time higher half location to jump to that is also mapped in the temp table
yo actually
you can literally just
allocate a temporary page table below 4GB
copy over the entries from the proper one

and just use that before you switch to long mode
then just KERNEL_PAGE_TABLE.lock().switch_to(); in ap_entry
that literally just works
gl
you can also identity map the trampoline page in the kernel tables, then there's no need for a temporary set of tables.
that's what i did already
ah nice
that's the reason why i'm having issues with 32-bit
yeah honestly I dont get why
the kernel page table is a 64-bit address
so just allocate the root table within the 32-bit range?
you dont need paging enabled until you get to lmode, so you can use pae-style PTEs, its just the 32-bit limitation.
unless you have no pages available <4G, which would be interesting
but you can't mov cr3, rax in pmode
you can only mov cr3, eax
right?
yeah right
that would be impossible in a PC system
exactly
but if the pml4/pml5 address can be contained in 32bits, eax is fine
this would prevent smp bootup in general since the init vector is always <1M
i thought you said having pages below 4G would be impossible lmao
lol
oh lmao
exactly
just allocate it below 4G
as i said already
and just copy over data from proper pml4/5 before jumping to trampoline

i can allocate a temp page table no problem below 4G, but i can't do the same with the kernel page table.
when i first initialize mm it's using a bump allocator
allocate temp root page table, copy data from kernel root page table
no need to temp alloc the other levels
you don't have to
you can switch to it when your ap gets to long mode
yoooo
And then it page faults
yes
idk why yet
it does set the kernel page map
hm it still tries to access something in the lower half
Just map the lower half in the kernel pagemap h
oh nahhhhh
i know what's happening
i switched stack pointers and now rust is referencing the 32-bit stack
bruhhhh
Marvin do my ap startup as well 
prs are welcome
i needed another function that didn't use the stack
hell yes
ARL moment
oh god damn it i need to fix the tsc logic now
weirdly enough it refuses to boot on my hp laptop with ryzen 2600U
i really wouldnt test on real hw without supporting pci aml
god knows what half initialized state u leave it in
hm
no
ez
i will figure this out
fair
i kinda want mounting lol
yes nyaux does not support it
(or well, its repurposed for just initing the fs cause im a retard)
idk how to do mounting either so liek who's the retrard now
i think I'll do an actual initrd then (as a block device)
/dev/initrd0 goes crazy
insane
well that's how an initrd works lol
???
ramdisk
i mean sure
thats what the linux docs do
what do they do
initrd = block device
initramfs = tmpfs
i can give you a link to a block pist
holy shit /dev/initrd is a real thing
im reading docs.kernel.org
which is what many people here call an "initrd"
it's just a block device with initrd contents
yep
once the last process closes it then its gone apparently
me when i dont know what block devices are 
makes sense
/dev/sda
maybe someone can help me with figuring out mounting code
ilobilo told me to check out his code but it's just wrong
Tru
idk if davix does unix stuff
but keyronex definitely did
and i have a feeling that fadanoid did a good job with that stuff
their code was always pretty much high quality
no i mean i dont know how "block devices" differently from char devices
on linux they dont i think
only if u open with O_DIRECT
???
like with char devices u basically just memcpy shit to a buffer + some offset
page cache mentioned, death in 1 second
just a cache of either disk or file blocks
the way I do it: char devices can literally just override all the fd functions to whatever they want (i.e. they can supply file-description-local function pointers for open, seek, read, write, ioctl, mmap, etc.) while block devices only provide device-local read_block/write_block functions and the generic vfs code implements seek, read, write, mmap, etc on top of those
do u support O_DIRECT?
override fd functions?, you mean the vfs ops?
no

similar to how with regular files the generic vfs code implements seek, read, write, mmap, etc. on top of the fs-provided file data memory object
no
why not?
seems like more of a hassle than it's worth for a hobby os due to its interactions with the page cache
wdym "device-local", what does read_block and write_block take in as parameters and how does it "write a block"
oh so O_DIRECT skips the page cache?
well the request gets sent to the disk directly
but it must keep the page cache up to date
if theres a page for this block
I have 4 sets of "vfs ops": file description operations (seek, read, write, ioctl, mmap, etc.), inode operations (lookup, create, setattr, open, readlink, etc.), character device operations (just open for now), and block device operations (read_block and write_block)
i see, also why do u need an open op?
in inodes or in char devs?
to create an fd from an inode
in inodes it's only for directories, regular files are handled by generic vfs code
in char devs it's because open has device specific effects beyond just creating an fd
for example on terminal devices you might set the controlling terminal, or initiate a modem connection
why does that have to be vfs level lol, just have it so any syscall to open will just create a file descriptor pointing to a vnode like nyaux does
right makes sense
how does read_block/write_block work tho?
how do you open files in the kernel
why would u want to do that, if ur the kerenl just lookup and access the fuckin vnode directly
lol
okay?
because the thing that keeps track of the cursor is the file
not vnode
so if you access it directly, you can't seek
okay why would i need seek
to seek within a file
you pass it a buffer, an offset in blocks, the number of blocks to read/write, and it reads or writes
i mean, what im trying to say is cant u just read/write directly to the file in the kernel since u know the offsets beforehand anyway
how big is a "block"
that depends on the device
usually 512 or 4096
right makes sense
that still should operate on a file
so basically to read from a block you would just
// handle offset being wrong or bigger then the actual buf or file
...
memcpy(buf, filebuf + (offset * sizeof(BLOCK), calculatedsize * sizeof(BLOCK));
why tho?
open flags, multiple writers, locks, etc
you can't write to a node directly in user mode either
nyaux does not have locks on fds 
makes sense tho for those reasons
well on actual devices it'd involve writing to a bunch of registers and dma and everything like that but that suffices for a ramdisk yeah
ill change how things are done in nyaux kernel wise when i need to
ofc
i mean for a on ram impl or smt
lol
my rationale is that you need the operations on a user file anyways, so why make that distinction
๐คทโโ๏ธ
wait I can helps
basically have a vfs_mount pointer on your vfs node to say if it is a mountpoint or not
this will have a pointer to the root of the mounted directory
so when you mount a disk you pretty much add that mount to the node and other data and when you read the node you just check that field and redirect the read ๐ฅ
that's the jist of it
and then with lookups and stuff etc. just keep a table of mounted filesystems and their mountpoints and search it
shrimple as that
oooof
weak ptrs are very nice
How can the parent disappear if it has a child
20 cm long code
this is for entries
Hm?
dentries can become invalid
or rather, if i use an arc for both the parent and the children then there will be double refs
And u assume enoent if parent disappeared?
just any error if the parent entry can't be resolved
idk if errno is the correct one
Dunno
but afaict if an entry can't resolve it's usually because there's no entry
someone needs to get the milk
what the fsck is this?
how do you people manage to take something which is a bit tedious to do in C, yes, but not too hard, and write it in C++ or Rust or any other language with some amount of syntactic sugar in a way that makes it illegible to anyone but yourself?
let parent_node = parent.lock().clone().ok_or(ENOENT).upgrade().ok_or(ENOENT)
you better write a few really long comments in lookup functions etc. in your VFS that explain stuff like this
i don't see why
lock = lock mutex
clone = increment rc
upgrade = convert from weak to strong rc pointer
i do have a comment above it with what it does, but i think you're overreacting
yeah, not explaining that 'clone means increment refcount', rather to explain that "we are doing a lookup on foo, if condition then we know it is ENOENT, this cannot race against a concurrent unlink followed by a concurrent mknod because this"
i commented it with // Attempt to get the node of this entry
i could just make that a function
"get the node of this entry" you mean acquire a reference to the inode that this dentry points to, or return ENOENT if there is none?
yes
so a dentry is always known positive or known negative, and there is no "hasn't been looked up yet" state?
yes
you can revalidate it if it is negative
i mean
in what case would you have a dentry that is neither negative or positive
then you wouldn't have the dentry to begin with
e.g. when you create a dentry representing purely a path
like O_PATH or something
also when you create a dentry, do a lookup, and that lookup fails because of ENOMEM or EINTR or something, so you do not actually store an inode there
what's O_PATH
yup
well that's what the revalidation is for
i think
yeah so there is also a state where the dentry is neither positive nor negative, but "not yet looked up" (AKA unknown)?
either that, or you cannot cache ENOENTs
by having three dentry states:
- dentry is positive
- dentry is negative
- dentry is not looked up yet
1sec, let me reference the code
really positive and negative are substates of "dentry has been looked up" -- it is implied by the inode pointer being NULL or non-NULL which one it is
d_time is only relevant for NFS or FUSE
right
well in rust i can make this a typed enum
enum EntryState {
Positive(Arc<INode>),
Negative,
Unknown,
}
or i'll just keep using an Option<>
and a flag
yeah, this would work
i had that before but someone told me i shouldn't do it :P
the disadvantage is that there might be cases where you unnecessarily need to hold the dentry lock a bit extra
can you find that message and fwd it to me?
(it's not something I have said, I hope)
ok then it's definitely not something I have said
nah dw
the reason I'm asking is because I'm curious if there is a rationale for that
i'm just stupid and trying to piece everything together (horribly)
i'm going down a huge rabbit hole rn but i only want to have a vfs that doesn't suck horribly ๐ญ
no
you're not
i'm still stuck with the god damn mounting shit because i just can't piece it together
i keep confusing the struct names and idk how that interface should look like
in fact that's something which I think is very present in this discord server. you implement something and it kinda-works-but-also-kinda-doesn't, and then you see that there was some other approach you could've taken which you realise after the fact would make things much easier, and then you feel stupid for not thinking of that thing first.
the thing is
whoever came up with that other thing probably first got it wrong in many more ways than you did initially
okay so
@distant cypress does davix have a working mounting impl yet
because going straight after the way linux does it is not feasible for me
no
the plan is something like this:
- do scheduling (mostly done, just gotta fix a but with vmap/kfree_large that was discovered when doing this)
- implement RCU
- begin working on a VFS
now, considering that there is a decent chance that I might want to sprinkle in some hardware stuff (like proper interrupt handling, PCI, ACPI etc.) in between, that might postpone this a bit
and I also have a job now (it's a really good job), so that means less time to work on this
damn
yea my job drains like 80% of my time away now, and when i come home i rarely want to touch my PC again
i mean the only thing i don't understand about ilobilix' vfs is this struct
(I'm probably (one of) the lowest paid employee(s) at the company, but I'm also fresh out from high school, and it is a programming job, so what can I expect? and it's really good to have on the CV)
we don't need to pay lots of money to go to university here ๐ช๐บ
๐ช๐บ ๐ช๐บ ๐ช๐บ ๐ช๐บ
yeah
I can try to explain what the basic idea is, if you want
alright
ilo has no comments whatsoever 
when I write the Davix VFS, I will make sure to write large comments explaining everything which is not-immediately-obvious, showing why it works and providing rationale for it
anyway
so in the old days, Unix filesystems might've worked something a bit like this:
you have the struct inode and that's it.
all lookups (including . and ..) go via the filesystem
(this is probably part of the historical reason why . and .. are included in readdir)
anyway
so we want to cache lookups, because calling into the filesystem driver is slow
so what do we do?
(skipping ahead many many years, vnodes and such)
we have a struct dentry, and dentries form a hierarchical tree from the root of a filesystem
(not worrying about mounts yet)
iirc there is a comment in the Linux codebase explaining that the dcache is the "master" of the icache
what they mean by this is that dentries hold references to the inodes they point at, and not vice versa
this makes sense because when you do a lookup on some path, you want to walk the dentry tree, and then you want to look at the corresponding inode
anyway
now that we have a dentry tree, filesystem operations like a rename or such must happen both on-disk and in-memory
now, bringing mounts into the picture (but not bind mounts/mounting the same filesystem multiple times, just yet)
(I'm drawing another picture in gimp)
a dentry which is mounted over holds a pointer to the root of the filesystem that has been mounted there
this makes it very easy to "step into" a mount
you just
while (dentry->mounted_fs_root != NULL) {
dentry = dentry->mounted_fs_root;
}
(omitting refcounting and locking for brevity)
and stepping out of a mount is almost as trivial as walking ..:
while (dentry->is_mount_root && dentry->mountpoint) {
dentry = dentry->mountpoint;
}
(again, omitting refcounting and locking for brevity)
so now, how do we do bind mounts?
trick question: they are incredibly hard to do with this scheme
now what is the parent pointer of /foo/bar/ and /.../baz/?
because it cannot be both
one idea might be to allow there to be multiple dentry trees for the same filesystem
but this should only stay an idea
because remember this
if I have /dev/sda mounted on /a/b and on /c/d, and I do a rename("/a/b/foo", "/a/b/bar"), now I also have to move the corresponding dentries in /c/d
the consequence of this is that it is impractical to have multiple dentry trees for the same filesystem, so there needs to be a one-to-one mapping between filesystems and dentry trees
now we get to this type of structure
so if we have one dentry tree per filesystem, that is one parent pointer
this is not enough to walk ..
therefore the path needs to contain something other than a dentry too.
a struct mount is very convenient for this, because two things that we need to do very often become very simple.
walking the child of a dentry ("stepping into" a mount):
struct path lookup_child(struct path parent, const char *name)
{
mount *mnt = parent.mount;
dentry *dentry = parent.dentry;
again:
for (mount *child_mnt : dentry->mounted_here) {
if (child_mnt->parent_mnt == mnt) {
// "step into" the mountpoint
mnt = child_mnt;
dentry = child_mnt->root_dentry;
goto again;
}
}
// direct child lookup, not taking into account VFS
// mountpoints or anything like that
dentry = lookup_dentry_child(dentry, name);
return (struct path) { mnt, dentry };
}
walking .. (leaving a mount):
struct path lookup_parent(struct path path)
{
mount *mnt = path.mount;
dentry *dentry = path.dentry;
while (mnt->has_parent_mount && dentry == mnt->root_dentry) {
dentry = mnt->mountpoint.dentry;
mnt = mnt->mountpoint.mount;
}
dentry = dentry->parent;
return (struct path) { mnt, dentry };
}
these two operations become very trivial
and because we only have one dentry tree per filesystem, we only need to update one dentry tree for every filesystem operation like a rename or unlink or whatever
ok now I have dumped a lot of information here, but is anyone actually following along?
please do ask questions
my infodump shall be forever commemorated by that pin
gimme some time to read this, really appreciate this
this looks similar to my filesystem logic
I think I did it kind of correct then
nice
another thing with this is that there is no "correct"
however a scheme like this is apparently very common in "big boy OSes" like Linux and some BSDs
struct vfs_mount {
struct vfs_node *mount_point;
struct vfs_node *root;
struct vfs_ops *ops;
char name[256];
void *fs_data;
};
struct vfs_node {
enum fs_type fs_type;
bool open;
char name[256];
uint32_t flags;
uint16_t mode;
uint64_t size;
void *fs_data;
void *fs_node_data;
struct vfs_mount *mount;
struct vfs_ops *ops;
};``` so i basically have this, and mounting is as simple as
```c
enum errno vfs_mount(struct vfs_node *mountpoint, struct vfs_node *target) {
if (!mountpoint || !target)
return ERR_INVAL;
if (!(mountpoint->mode & VFS_MODE_DIR))
return ERR_NOT_DIR;
struct vfs_mount *mnt = kmalloc(sizeof(struct vfs_mount));
mnt->root = target;
mnt->mount_point = mountpoint;
mnt->ops = target->ops;
return ERR_OK;
}```
So this logic should work just fine, and upon any lookup, I can call a table that saves these mountpoints to check if said path is a mountpoint and then return the node of the root of the mounted FS rather than one representing the FS that the root is mounted on?
this is different from the scheme I have described
but it should work fine? or no
idk, it's your scheme, you either make it work or you don't
but one thing that I notice here is that mounts don't have a concept of a parent mount
you have struct vfs_node *mount_point;, which is not enough for walking ..
because walking .. also requires a parent mount, not only a parent dentry (or node in your parlance)
also, there is a struct vfs_mount *mount; member in your struct vfs_node.
this is if a node is a mountpoint
ahhh
it'll be NULL if it isn't
I would suggest naming it something like child_mount then
apart from mount_point of struct vfs_mount being a struct vfs_node * and not a ยซstruct vfs_mount *, struct vfs_node *ยป tuple, this looks mostly reasonable to me
so, I should still keep a table of mountpoints, and upon any lookup, check if the mountpoint path matches the lookup path and redirect it there?
there are many ways to do it
oh yea, I could make it a tuple good point thanks
or just extra field, whatever
but that is suboptimal with "my" scheme (which isn't "mine", I don't even know where it originally comes from)
look at this function
so with "my" scheme, you have all the mounts that are rooted at a certain dentry in a linked list in that dentry
so you iterate this list, and if you find a mount that has its parent mount as the current mount you are in, you step into that mount
oh ok, that's reasonable yea, I was thinking that using a table would be easier in the long run with not a huge performance hit, since I already have a fully functional ext2 filesystem driver, and I also am working on FAT and it would be simpler to just pass in the name into a lookup function that searches a mountpoint table for all lookups
I think I will be fine with this design for now
I'll just have a struct of mountpoint names and parents and just search a table of those ๐
thanks guys
ah i think this is one of the things i was missing
yea this is very understandable
a question though, how should i be passing the struct path around?
like what keeps track of this struct path
IMO: it's viable to just pass it around by value. reference counting is done on the mount and the dentry that it points to, not on the struct path itself.
just be careful not to confuse an "owning" struct path and a "non-owning" struct path
(where "owning"=the caller gave you their reference and it is now your job to unref it, and "non-owning"=you borrow the struct path from the caller.)
just normal refcounting shenanigans
well this is done for me by rust already :3
i can't really mess that up
so basically, instead of returning a dentry everywhere i should instead just pass a struct path?
if you're dealing with global paths, yes
it's useful to have a mental distinction between "global" paths, which fully specify a location in the full filesystem hierarchy with all its bind mounts and such, and "local" paths, which specify a location within a certain physical filesystem
okay so, let's say i want to open() a file
what would the call stack look like with what arguments
this struct path is really hard for me to model
imagine the user tries to open ./foo/bar.txt
how does this struct path spawn
maybe something like this:
int
sys_openat (int dirfd, const char __user *path, int oflag, int mode)
{
// ...
struct file *filp = kmalloc(sizeof(*filp));
if (!filp) return ENOMEM;
int errno = vfs_lookup(&filp->path, dirfd, path);
if (errno != 0) {
kfree(filp);
return errno;
}
// ...
}
now you have a struct file and a struct path within that struct file
(this struct path is useful if the file descriptor we create ever becomes a dirfd, and if someone ever looks at /proc/pid/fd)
the fuck is the difference between a vnode and inode
vnode = dentry
no
not in sun parlance
if a vnode has a pointer to a struct inode and there is a hierarchy of vnodes which you can traverse without calling on the filesystem driver, yes, such a 'vnode' is possibly the equivalent of a dentry
honestly I might be partially at fault for 'vnode's being used incorrectly
like mid 2022 or something, I studied VFS:es and also designed some stuff myself
back then, I used 'vnode's as a Linux dentry equivalent
right okay
what saved the struct path to begin with
like, how do i go from a dentry to a struct path?
am i retarded?
๐ญ
you don't usually need to
when would you need to get path from a dentry?
resolve function has enough information to do so (the parent path)
like, where do i keep track of struct mount and where do i go from a string path to a struct path
i cannot get this in my brain
I keep those in my equivalent of your struct FileSystem
string path always has a parent
processes have path cwd
otherwise it's root
so you're saying that cwd and root should also be a path?
that makes sense
are you sure about that? maybe i can't read c++, but i only see a name string in struct filesystem
check tmpfs.cpp
the list should be in the base class
I just haven't gotten around to changing it
this is a case of do as I say not as I do lol
i hold the opinion that the cwd should be stored as a handle to the current directory
you begin with a struct path
cwd is a struct path
so the static vfs_root is also a path then
-
because you can just pass it to all your syscalls and save some lookups
-
because you can determine the current directory's path (most of the time) by just walking up the tree
yeah, possibly
no
so it shouldn't be an Option<>
but you can possibly make something else root and make old root non-root
no
this is what pivot_root does in Linux
yea i know that systemd does that to mount the real root from the initrd
you can make ur os have the capability to run purely in memory with no on-disk root anywhere
I'm trying to see if I can make that happen for mine ๐ฅ
?
okay so
wdym ?
what is a dentry
everything is a vfs_node
oh directory entry
????????
like
๐
im confused a little but uhhh we'll see to it when we get to it
you'll need to rewrite your entire "vfs"
idk why that wouldn't work
hehe fat32 go brrrr (their things are a little cursed)
I am not having a blast implementing it
ext2 was fine and dandy but fat32 is just not it
nvm i know what it is
what?

crazy
do you plan to even have an on disk filesystem ๐ข
yes
hooray
be like me and turn every vnode into a com object 
i assure you that being able to read from a folder and opendir a file is essential to the computing experience
@sick flax tmpfs has been mounted
now i need to fight some scheduler memes
and i need to read the initramfs into the tmpfs
Pog
menix gamer
and then you need to test bind mounts and moving mounts and unmounting filesystems.
oof
btw
how does a lookup work on the vfs layer? i look at the parent dir entry -> inode, then call fs lookup?
a high-level description might be this:
- first, get a reference to a dentry with the name you are looking up that is in the dentry tree (or was at some point after the lookup began).
- next, look at the dentry state.
- if it is positive, you've got yourself a dentry and an inode.
- if it is negative, you've got yourself an ENOENT.
- otherwise (hasn't been looked up), you call
dir->i_ops->lookup(dir, dentry);. (dir is astruct inode *)
so what happens in the first step? let's say I'm looking for "/sbin/init", which doesn't exist. Do i just literally go to the root, then look for the "sbin" dentry, then look if i have a "init" entry?
do i create negative entries for the whole path?
so for a lookup on like a path consisting of multiple components
usually you iterate over components and do them one at a time
one thing to be careful with is the lookup on the final component -- sometimes you want it to be ENOENT, sometimes you want to create a file there because you're actually doing open(O_CREAT) or something.
no, you stop as soon as you get ENOENT
right
in this case, how do i propagate this to the calling function? do i first lookup everythin except the final component and then assert that the final lookup is ENOENT?
one way might be to have lookup return a path, which is the final mountpoint and dentry you arrive at, or ENOENT/etc. if anything on the way there failed
now it is caller's responsibility to lock the final dentry and look at its state.
the advantage is this keeps core lookup really simple
should i have a dentry function that looks up a whole (string) path, or do i do this component wise in the caller?
it's probably best to do it per component
the lookup_child etc. code snippets I have posted here and there, stepping into mounts and such, they walk a single component only
this is nice because when you've opened the inode, you don't need the dentry or mountpoint anymore
(except in some refcounting schemes where this is needed not to unmount the superblock)
i made a file struct which contains a ref to the inode and a fileops pointer
yeah
it is probably a good idea to have a 'path' field there which can optionally be present
for dirfd of openat and such
good point
and fexecve
ill look into this in the morning
I'm really glad for this infodump
if you can read rust code, i could tell you when i have my impl done and maybe you can spot some mistakes then
I'll do my very best (C++ brain) if I have time for it :P
my code should contain at least doc strings as to what i want to achieve
so you could fall back to those
@distant cypress so i'm following the lookup_child functions now, does that imply that a dentry struct always has a link to a dentry where it's mounted on?
this is what my structure looks like at the moment
If the mountpoint is reachable from just a dentry, bind mounts don't work
That's what struct path is for
you may need to change the parent of an entry
Yeah I'd swap the strength of parent and children
children need to be strong as well
Children a vec of weak refs, parent a strong ref
How so?
i think so at least?
hmm maybe not
wait wtf is that code lol
now that i read it more carefully it doesnt make much sense at all?
If the children are strong you will never be able to evict from the dentry cache
Because there's a strong ref to every root for obvious reasons, and all dentries are reachable from the roots
Regardless, the way I'd do things is strong ref to parent, weak refs to children, strong ref to inode
this is something that is kinda hard to express in rust
but what youd ideally have is that when the entry is removed, the parent forgets about it
without there being a refcounting/weak ref relationship
Yeah that's a weak ref
kinda, yeah
except that a rust weak ref still keeps some of the pointed-to memory allocated
Common rust L
i mean. a c++ weak_ref does too
Fair enough
I don't use either so I'm not too familiar with their implementation details
i'd have ```rs
struct CacheEntry {
/// Strongly owning the inode.
inode: Rc<Inode>,
/// If Some, then the parent cache entry. The parent may not be freed until this entry is.
parent: RWMutex<Option<NonNull<CacheEntry>>>,
/// If Some, then this is the set of the children this cache entry has. The child pointers are guaranteed to be valid.
children: RWMutex<Option<BTreeMap<Vec<u8>, NonNull<CacheEntry>>>>,
}
and then manage refcounts by hand
how so?
"being dropped" does not mean "freed"
i see this, but dropping a Weak should drop the allocation then?
i mean in theory i could just drop the weak if a call to upgrade fails
also you really should not have the children be a Vec<Arc<Entry>>
you should have them be in a tree
so that finding them is reasonably fast
you should never use a hashmap in a kernel without careful consideration
because entropy shit?
because they need lots of contignous memory
also here you want to be able to iterate through shit in order lol
fair
also, your cache should be exhaustive
i.e. you either have a live pointer to all your children, or you have no pointer to any of them
because otherwise you cant use it for caching directory walking
so you're saying a lookup_child function should populate all possible children in the btreemap?
1sec, I will be at my computer in a few minutes, and then typing won't be kanker
yes?
also need to read all the messages posted by others first
because you (a) read them together from disk anyway on most filesystems
and (b) it's the only good way to make readdir caching work
alright then
and its not terribly expensive
honestly
thank god im not using rust
ngl the only good way to express this in rust is to just use nonnull everywhere
and do it by hand
also
i dont think thats really true
solaris uses a bunch of hashmaps everywhere
huh
turnstiles, vmem and kmem
yeah so parent dentries holding strong references to their children but child dentries holding weak references to their parent, this is usually done in the opposite way (however that makes direct dentry child lookup a bit trickier, you might want to use RCU for this, and it might not be very pleasant to do in Rust)
one of the big reasons I see for having strong parent references is that if you do, if you hold a reference to a dentry, you know that it's ancestors all exist in the dentry tree
if you have some kind of struct Mount, it might look something a bit like this:
struct Mount {
// lots of members ...
ListHead mount_list_linkage;
// lots of members ...
struct Path mountpoint;
bool has_parent_mount;
};
now, the Dentry would look something a bit like this:
struct Dentry {
// lots of members ...
ListHead child_mount_list;
// lots of members ...
};
If I were to design a VFS, I would store a linked list of "filesystems mounted here" in each dentry, e.g. in Dentry::child_mount_list above (with Mount::mount_list_linkage being intrusive linkage for this list).
When looking for filesystems to step into, i.e. cd foo, you'd look at Dentry::child_mount_list and see if there is a Mount with mountpoint.mount == the mount you're currently in.
However when looking for mounts to step out from, i.e. cd .., you'd test if the dentry you're at is the root dentry of the mountpoint you're in.
If it is, you set the current path (both mount and dentry) to Mount::mountpoint
the problem is just rust honestly
rewrite in C
and you will fix it
(idk what problem you're having)
yeah, some of these ownership relationships are probably hard (but possible) to express in terms of Rust
no
i do not actually
yes you do
thank you
storing filesystems mounted here is not good
why not?
because of cache coherence across bind mounts
??
this is my mount struct
you may have one dentry that has multiple places in the FS
no
this idea of a path being a ยซmount, dentryยป tuple is the thing that fixes that
in practice you will not run into this
its a hobby os, of course you wont
if someone bind mounts the same dentry of the same filesystem in a hundred places, and then bind mounts filesystems on top of all of those, sure, it becomes slow.
you wont ever have any bind mounts tbh
but then it's the user's fault for doing weird shit.
did u read the last sentence? 
bro is late to everything
yeah yeah
i was packaging limine for debian
iretq@DESKTOP-TAQSOFJ:~/debian-limine/limine-9.3.3$ ls -l ..
total 2224
drwxr-xr-x 22 iretq iretq 4096 Jun 19 18:14 limine-9.3.3
-rw-r--r-- 1 iretq iretq 566679 Jun 19 18:15 limine_9.3.3-1_amd64.build
-rw-r--r-- 1 iretq iretq 8207 Jun 19 18:15 limine_9.3.3-1_amd64.buildinfo
-rw-r--r-- 1 iretq iretq 5694 Jun 19 18:15 limine_9.3.3-1_amd64.changes
-rw-r--r-- 1 iretq iretq 40900 Jun 19 18:15 limine_9.3.3-1_amd64.deb
-rw-r--r-- 1 iretq iretq 1184 Jun 19 18:14 limine_9.3.3-1.debian.tar.xz
-rw-r--r-- 1 iretq iretq 1395 Jun 19 18:14 limine_9.3.3-1.dsc
-rw-r--r-- 1 iretq iretq 414892 May 27 20:12 limine_9.3.3.orig.tar.xz
-rw-r--r-- 1 iretq iretq 109880 Jun 19 18:15 limine-bios_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 461716 Jun 19 18:15 limine-cd_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 24340 Jun 19 18:15 limine-dbgsym_9.3.3-1_amd64.deb
-rw-r--r-- 1 iretq iretq 15508 Jun 19 18:15 limine-dev_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 34484 Jun 19 18:15 limine-doc_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 29712 Jun 19 18:15 limine-pxe_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 94116 Jun 19 18:15 limine-uefi-aarch64_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 117712 Jun 19 18:15 limine-uefi-ia32_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 95328 Jun 19 18:15 limine-uefi-loongarch64_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 105020 Jun 19 18:15 limine-uefi-riscv64_9.3.3-1_all.deb
-rw-r--r-- 1 iretq iretq 112376 Jun 19 18:15 limine-uefi-x86-64_9.3.3-1_all.deb```
i wonder if this has a chance of getting into debian proper
i don't even really know what an rcu does
it lets you do stuff like:
dentry *d_get_direct_child(dentry *parent, const char *name)
{
rcu_read_lock();
dentry *child = /* some hashmap/btree lookup operation */;
if (child->refcount.increment_unless_zero()) {
rcu_read_unlock();
return child;
}
/* 'child' is being concurrently deleted by someone else, handle it. */
// ....
}
except it isn't one
so the thing with RCU is it also lets you do something like this:
void d_drop(dentry *entry)
{
again: // no locking yet
if (entry->refcount.decrement_reaches_zero()) {
// the refcount is already zero here
dentry *parent = entry->parent;
mutex_lock_exclusive(&parent->lookup_mutex);
remove_from_your_favorite_lookup_datastructure(entry);
mutex_unlock_exclusive(&parent->lookup_mutex);
rcu_free(entry);
entry = parent;
goto again;
}
}
i.e. you don't have to do any heavy locking for a large part of the filesystem operations
hm
i need to read into it
but now
i've heard different people say different things about how i should do the dentry stuff now
and i'm confused
yeah,
this doesn't help if i already didn't knew what i was doing before xddddd
so the thing with vfs design is that it is to some extent an open-ended question and there is no one true answer.
but generally there are some patterns that are common in good VFSes, e.g.:
- child dentries holding a reference to their parent dentries
- paths being a ยซmount, dentryยป tuple
- the dentry cache 'being the master of' the inode cache
- caching ENOENTs in the dentry cache
these are useful to at least have in mind when you design a VFS
eh i'll just do something that works for now and is relatively easy to implement
i have different things i want to do
do RCUs need any scheduler changes?
afaict the logic is all self contained inside an rcu struct
classic rcu just needs a function to be called on yield
and it needs yield (or, more specifically, that function) to be called periodically, otherwise idle CPUs will make rcu sync deadlock
depending on RCU variant, you need a few hooks here and there:
// RCU API:
void rcu_enable(void);
void rcu_disable(void);
void rcu_quiesce(void);
// in lowlevel userspace entry/exit code:
void after_entry_from_userspace(void)
{
// ... (before enabling interrupts)
rcu_enable();
}
void before_exit_to_userspace(void)
{
// ... (after disabling interrupts)
rcu_disable();
}
// in the scheduler:
void schedule(void)
{
// ...
rcu_quiesce();
// ...
}
// in the idle task:
void idle_task(void)
{
// ...
/*
* Also make sure someone calls rcu_enable when interrupting
* the idle task.
*/
rcu_disable();
}
this is at least the interface which I'm planning to use
so the idea with rcu_disable and rcu_enable is that you mark regions where no one will be using RCU for a long time, e.g. when a CPU is running in userspace context or the idle task is running.
what's rcu_quiesce
marks a quiet point
i.e. this
Oh, the rcu disable thing seems like a much more elegant solution to the problem than what I'm doing (I call rcu_quiet every millisecond)
Might yoink that
a point where you're sure you're not in an rcu critical section
so in classic rcu where you disable preemption in an rcu critical section that's in yield
@distant cypress what should i pass to File::open? I would assume struct Path doesn't make sense if i open with O_CREAT
string path
hm
i'm feel like my brain cells are just disappearing
@distant cypress sorry, but would it be possible for you to give me some function prototypes for everything i'd need? what you said gave me a better understanding but every time i try to start implementing it my brain shuts off and i just end up staring at my screen the whole day
here is what i have so far, but everything seems wrong
By File::open do you mean what the kernel ends up calling when syscall open() is called?
literally me three years ago
๐ข
i thought having a common interface to allow the kernel and user to open files
idk if that's useful
yeah it's probably useful for e.g. initrd unpacking
yeah... sysfs is tricky because you want to create and destroy nodes at runtime while also avoiding allocation failures
procfs is also tricky
i feel that
(I wouldn't worry about these until later)
i guess i just need an overview of functions that i have to impl to get stuff working
File *kernel_open(Path root, Path origin, const char *path, int oflag, int mode);
something like this would probably work for a very long time
it lets you do chroots via root, start the lookup from any directory e.g. cwd or a dirfd, you can implement openat, creat etc. in terms of this. like the only file creation syscalls you cannot implement in terms of this are mkdir, mknod, mkfifo, symlink, link etc. but those shouldn't be tricky, you just do something very similar
anyhow I don't have a lot of time today sadly, so cannot really help more than this
thanks tho
@vast lotus why exactly do you do mount_top here?
i thought root is already the target directory
uhh i think that's just something i forgot to remove from an earlier iteration
makes sense then
Are you saying hydrogen has bugs
๐จ
i like that hydrogen usually has the cleanest impl
Yep
hydrogen's vfs code is probably the buggiest part of the whole kernel because i haven't done any real testing
my 'testing' for vfs is just fixing whatever issues crop up when running applications
i'm only cross referencing stuff so it's not completely broken on my end
yeah fair enough
just a warning, the hydrogen vfs code probably has a bunch of deadlocks (esp. in stuff that has to take multiple locks like rename and such), race conditions (one that i'm aware of is a/b/.. being able to resolve to a/c if a/b is renamed to a/c/b), and performance issues
rust helps a bit here
sure it can't completely avoid deadlocks, but it makes it easy to not accidentally do it
i've been meaning to rewrite my vfs but i haven't gotten around to it
How do u even make an rcu in rust
can't be too hard
btw https://docs.kernel.org/filesystems/path-lookup.html is a pretty useful resource on how linux manages its dcache
did you get the log buffer working yet?
I've been just playing games tbh lmao
fair enough
Completed metro exodus yesterday
Also had a relatively busy work week but thats just excuses
bruh
its a great game
Ik
It really is
i got it when it was like 5โฌ
haven't touched it yet
i only read that it got mod tools
half of the dev team got killed in ukraine apparently
i don't even game tbh. i just stare at the screen and cringe
yeah
I do that in between gaming
Also there's a vr metro game
That came out recently
But idk if that's any good
VR games rarely are
Yup
hi
Yet you killed my ass in Xonotic ๐ญ
fried
Menix xonotic port
death stranding is so good
@sick flax
idk if it uses anything linux specifc
webkit
fair
Managarm chromium first
The build system was whack last I checked
fair
which
Surprise
what a stacktrace
the memory was unsafe
rust has good panics at least
why can't c++ have them as well
i cannot easily get the crash location
- stack trace is bad
lol
oh
that doesnt really give u the backtrace
but rust has something for cpu exception too, no?
real
wait what
he wanted the crash location
and backtrace
(not freestanding)
ah
just link libstdc++ into your kernel
doesn't frigg have a backtrace
well this also needs like libunwind i believe
by using stack frames it skips a lot of stuff
well
no
just link libunwind into your kernel as well

still misses inlines
i think i have that
inlines = zzz
#define inline
actually, does the C standard allow defining keywords?
yes
yeah, i have that
just compile with -finstrument-functions and make __cyg_profile_func_enter and __cyg_profile_func_exit push stuff to a stack that you can then print later 
just don't crash
my scheduler would like to disagree
it's fucking up my 0 queue and idk why
the 1 queue works
not the 0
how does this happen
i would like to know too
if only i could make it work on arch
but cmake wants to disagree too
XDD
how do i stop them from being dropped instantly
this doesn't make a lot of sense
oh nevermind it's referred to by a struct path
thread_local bool in_hook = false;
thread_local bool in_backtrace = false;
thread_local std::vector<void *> call_stack;
extern "C" {
[[gnu::no_instrument_function]] void __cyg_profile_func_enter(void *this_fn, void *call_site) {
if (in_hook || in_backtrace) return;
in_hook = true;
call_stack.push_back(this_fn);
in_hook...
lol
if you take call_site into account you can see the actual address of the call instruction instead of just the address of the function being called
and it even survives inlining
okay now i'm almost at a point where i can load CPIO data into the vfs
i'll just have to make it so tmpfs always populates inodes
can be if you use frame pointers only
bigger problem is that std::stacktrace_entry specifically uses std::string
std::stacktrace is templated on the allocator but std::stacktrace_entry isn't
i assume the former is for internal storage and not for entry strings
damn
i'm not a fan of this solution but eh, it's already in the standard
oh speaking of reflection got standardized today
i saw yeah, great news
also great news that apparently the gcc impl is already underway
zamn
that reminds me
i need to make sure stacktraces dont look at userspace
/usr 
Yeah
why does the entry have to exist if TMPFILE is specified?
i thought a tmpfile is an unnamed node
With O_TMPFILE the path specifies the directory whose filesystem it should be created in
we have gotos at home:
lol
gotos are actually so useful for this type of code. IMO it's insane some systems-level languages don't have them.
it's because rust has drop()
if you can just goto anywhere it messes up resource flow
but does it really?
at the compiler level, goto is just a manually-inserted edge between two control flow nodes.
I don't really mis gotos when I use Rust.
Mostly used them for error handling cleanup in C
stuff like error handling is done with ? instead of gotos
i miss the question mark propagation ๐
ts so good
"I should write a macro to do that!" 
stop
C macro generic data structures
#define _OK()
Me moments before deciding to finally start using Rust
for my next trick i will be switching to meson
#define Err(e) return e; type
why 
for trolling purposes
lol
well cargo isn't giving me the control i want over how the kernel is built
meson has built-in rust support
and can (somewhat) handle crates
Oh? How so?
i want built in and dynamic modules
like i had in the C version
right now everything has to be dynamic
as in, compiled as a dylib
What's stopping you from compiling them statically in also?
cargo
there's no way to encode that
also there's no post build actions
only pre-build
so are you manually manipulating the generated ELF or what
this sounds quite silly
wait so how would this work then
config
o ya
Can't link together two relocatable objects after the fact? Why not just use a Makefile to hide that? Then you'd have the kernel be a relocatable too and just link the last step manually
