#Zinnia
1 messages ยท Page 9 of 1
user space
okay jumping to user space works
now i just need to hook it up to the scheduler
or rather, i need to do the actual switch yet

[00:00:00.002433] [info] uacpi: successfully loaded 1 AML blob, 1705 ops in 0ms (avg 4373970/s)
what cpu is that?
because I get the exact same result on ryzen 5 4650g
and faster cpus usually produce larger numbers :P
265kf
can you try without -bios ovmf?
[00:00:00.002445] [info] uacpi: successfully loaded 1 AML blob, 1705 ops in 0ms (avg 5401158/s)
lol
someone said that intel lake is particularly not good at uacpi
yes
@somber solar can you test a new image?
sure but in a bit
qemu-system-x86_64 -m 2G -bios OVMF_x86_64.fd -no-shutdown -no-reboot -cpu host,migratable=off -accel kvm -machine q35 -drive format=raw,file=menix.img,if=none,id=disk -device nvme,serial=FAKE_SERIAL_ID,drive=disk
wtf
yea it need ovmf
that sucks ass
why is it so slow ๐
[ 0.001047] uacpi: successfully loaded 1 AML blob, 1729 ops in 0ms (avg 4492075/s)
i can't get past 4.5M
how do you tes the speed?
uacpi does
i think it has problems with my function then
that means it's returning 0 twice
huh
/shrug print the value
i think i will
oh wait
ik why
i enable my interrupts after uACPI loaded
so the PIT does not start counting
bruh
what cpu
something tells me the print function is the culprit
@hoary cave i finally got around to start using the MmioField abstractions
and they work as expected
this is so much nicer
the big endian stuff also works
soon
you should probably make it take the field/array as value instead of reference
true
you don't modify it and you get get rid of the redundant &
and llvm is smart enough to optimize it all out
not even the copy->ref but even into full specialization unroll considering it's all const
eah
you need to make the copy
that's expected bruh
how tf do i copy
??
do i need to derive clone
#[derive(Copy)]
oh
and clone i guess
yea
i like it better at least
same
without the annoying &s
with that i can actually do PCI config space stuff without cursed packed structs
you could probably put the regs in impl Hpet
then do Self::MAIN_COUNTER
but idk what you prefer
i think that makes sense
i like the mammogram way though
having a spec namespace
but it also makes sense to tie it to a struct
ugh
need to figure that out
i guess for multiple structs i can do smth like this?
struct SpecA;
impl SpecA {
const FIELD_A: MmioField::<u64> = MmioField::new(0);
}
struct SpecB;
impl SpecB {
const FIELD_B: MmioField::<u64> = MmioField::new(40);
}
lgtm
hm how should i do Stack MM
i was thinking of making a struct Stack
but that might be overkill
but then i could do stuff like safe stack manipulation for auxvals
or refcounting or whatever
Try mine too pls
i did
.
How many pts
[00:00:00.00309] [ info ] uacpi: successfully loaded 1 AML blob, 1726 ops in 0ms (avg 5416703/s)
how's yours so fast
do you stub anything?
obviously because I helped him write it ๐
No
Well you did but not the parts that make it fast
Yeah
I want something faster
But
Too lazy
Plus it's fast enough alr
? this is plenty fast aready
Yeah
Why not
you need to manage your own phys pages
you could use the same allocator for everything
except dma
makes kernel
doesn't use phys page allocator
just uses hhdm directly like a pro
Might go for bump allocator before initing uacpi and then switch to talc just for the points lol
that's kinda not the point lol
Ik
real
it'll be slower
I'm a pro
Fun fact: Literally BadgerOS' kernel heap right now
because bump isn't freed and isn't cached
alloc and "free" will take zero time, use of that memory will be slower
O yeah
^The memory will not be cached so you get full penalty of fetching main memory the first time you write into such an alloc.
Which takes over 200x as long as L1 cache on modern systems
yes
Better than talc?
If you think about it, a computer's memory is just cache all the way down.
- CPU registers
- L1 cache
- L2 cache
- L3 cache
- RAM
- Disk
probably
I use vram as ram
I don't use talc
You were supposed to say "pretty sure"
I don't memorise tv show scripts
You just finished watching it
Making me feel chronically online
It's probably the lack of features
^
it's not like the speed decreases with the amount of features
yea
More features means larger executable means lower chance of uacpi code having favorable cache locality, I suppose
But that should be an extremely minor factor
ah so that's why linux only gets like 1-2M points
that makes perfect sense
guys we should make our kernels less featureful
it does?
buddy moment
eh tbh i'm totally fine with that score
i shouldn't care that much about it anyways
not fully implemented of course
but it does namespace init i think
alongside acpica of course
The more people here say they shouldn't care that much about it, the more I want to port it to my OS just to see what exactly number I "shouldn't care that much".
yay
What font is that?
some pc font
Clone -> explicit cloning via .clone() with a hook into a optional custom clone function
Copy -> Makes the type use copy semantic instead of move semantic

okay break time is over
next week i'm on vacation so i'll have plenty of time to work on this
i really want to get vfs working again
@analog crane what exactly does a real time kernel entail? what do i have to do differently?
predictability and consistency
depending on how hard you go with it, you can be a soft real-time system or a hard real-time system, a soft real-time system is linux trying to be predictable for your video to not stutter, a hard real-time system is the contoller behind the moderating rods of a reactor
as to concrete things you might want to do, you want to avoid interrupts as much as possible, you want to have simple and predictable scheduling algorithms with quotas, maximum and minimum usage percentages, that stuff
Ironclad is capable of being something pretty close to strict hard real time under certain configurations, certain hardware, and certain task configurations
soft RT is a fine goal for a general purpose system and it has applications for audio/video, recording, that stuff
POSIX has facilities for it with SCHED_**
yea i'm trying to "market" it as a general purpose system
then be careful how you advertise the real time aspect, because when people say real time, they usually mean hard real time
so people will think you are being misleading
it won't be a major thing, just something to add to the readme
something like
- soft real-time scheduling constraints
yknow, to farm github stars
then be sure to have rocket emojis, people love those
๐ฆ ๐ blazingly fast scheduling
I prefer the wording capable of, thats how I talk about Ironclad's real time, because real time is a fluid thing, especially soft real time
no real time system is real time always and the same way
"Capable of providing soft real-time scheduling"?
yah
now to implement that...
its relatively easy compared with hard real time
depending on how your OS works its probably 30 mins
nice
:3
but what most people understand as soft RT as far as POSIX is concerned is SCHED_FIFO/SCHED_RR
just call it "dynamic scheduling policies"
i love scheduling
i'm definitely messing up the RSP
but idfk where
or how
ok nvm i got it
i was returning and corrupting the stack frame lol
lol
time to test mine too
okay i fixed scheduling from task 1 to task 1
i saved the old rsp to the task struct but didn't restore the now overwritten task
lol
slight oversight
dont worry i had that too
Yes thats right
i just if (old_task == new_task) return;
i guess that works
i second that
Written in asm for mine but same
okay done
now i can finally work on the vfs
this is going to be very interesting...
idk yet how to even approach this in rust
Wait is this a rust re-write of a formerly C project?
ye
ah
(pain)
in C i had the scheduler and vfs already working
i guess first step is doing initrd parsing
or vfs root
then initrd
I am seriously tempted to do Rusty BadgerOS
But to do so I would need Rust abstractions for all my stuff
you can also do a c kernel with rust ffi
That's basically what it'd probably be for a while
i already had that working in the legacy branch
FFI is pretty simple
especially if you build as a shared object
idk about staticlib
im tempted to make a fusefs driver
bro u dont even have userspace ๐
or a vfs
do those first!!!!
or im gonna beat u to it in my own kernel (i already have kernel threads)
cro
alright doing vfs now
i'll just slap Arc everywhere
bro i was talking about when i'm done with this

i did some stuff in my kernel but nothing interesting lol
yes from scratch
zamn
i don't do much anyway
i don't have a proper allocator
no scheduler yet
i'm gonna work on a dispatcher next
add timers
semaphores and mutexes too probably
i'm doing initrd rn
but i feel like i need to restructure my kernel
what's it like atm
not very interesting lol
private on github atm
i'll probably make it public when it can do something other than print a memory map
dam
oh damn i actually found a solution for the generic irq handler issue
instead of providing a function with a *mut () context i can just make a trait that saves self as the contexgt
then when i call the handler it'll automatically use self as the context without any pointer hacks and fully safe code on the impl side
should i have the UAPI bits in rust or in C?
@late bramble what do you do?
i could have a bindgen from C and then have rust wrappers around the uapi
but things like SEEK_WHENCE, etc
typically the kernel provides the headers with bits
i can do either way i think
i can generate C bits from rust code or the other way around
but tbh C bits make more sense
right?
the interface between user and kernel
like concrete values for SEEK_SET
or struct stat
that's in the libc, right?
yea but the kernel needs an equivalent structure
if it's different then the data will be broken
I rewrite them in Rust by hand
yeah
so i guess you follow linux abi
yes
I could probably use glibc too but as I said in my thread, there's a syscall it uses that I did not understand at the time 
glibc ๐คข
yeah, bloated
you could also use mlibc with linux 
never looked at mlibc tbh
surely this isn't going to explode ever
lol
idk
what's uapi
also why the hell not stdint.h
user kernel interface
ah so system calls and that kinda thing?
Unless you're 64-bit-only, long will be 32-bit or 64-bit depending on the architecture
There's also the LLP64 ABI which IIRC is 64-bit but has 32-bit long.
llp is stupid
do i have to care about it tho? can't i just say that i need lp64
ofc you can
I guess either way this is free Rust advertisement because it doesn't have this problem 
hm problem
iirc errno has to be a #define list
but rust bindgen doesn't care about defines
can i make it an anonymous enum?
Technically, yes, that is how you're required to define errno
Is it really the case bindgen doesn't do defines? I thought it did.
Either way, though, you'll probably be fine using anonymous enums.
Unless something actually #ifdefs on an errno value, an anonymous enum results in the same type: int.
i guess i could do both?
ah nvm bindgen is goated
it can actually understand value defines
yeah i wouldn't use enums for errno, lots of things use #ifdef on errno values since some errno values only exist on some systems
seems to work
And like can you #define ESOMETHING ERROR_SOMETHING where ERROR_SOMETHING is in an enum?
Or do you have to provide the integer value outright
I think theoretically you shouldn't do that.
Buuuuuuut
That's literally never gonna be a problem in a sane codebase

They need to be macros that expand to integer constant expressions, so no
Says who?
posix
posix
What if they dont?
The <errno.h> header shall define the following macros which shall expand to integer constant expressions with type int, distinct positive values (except as noted below), and which shall be suitable for use in #if preprocessing directives
not posix compliant
not posix compliant
the main issue you'll experience is that they won't be suitable for use in #if preprocessing directives
is the macro part a requirement? because an anonymous enum would behave the same way
i dont think it would
why not?
cause if you put it in an enum it doesnt get expanded to an int at the preprocessor stage
and you cant use #ifdef on it
yes, because an anonymous enum wouldn't expand to an integer constant expression
It is per spec
an anonymous enum expands to an enum constant expression with type int
so at the preprocessor stage it's still just an identifier
instead of an actual int
but why would someone need to do this
idk but posix requires it
In this case I would recommend making the macro definitions first, and for the Rust FFI, you make an enum along the lines of:
typedef enum {
ERRNO_LOOP = ELOOP,
} errno_enum_t;
or write your own preprocessor that runs before rustc and replaces E* with integer literals
i do something similar
i still define the errno values
for the rust impl i just wrap around this
why the fuck is it so hard to safely fill a fixed buffer with string data
๐ญ
bro this is lowkey impossible
there's no safe way of doing this???
which brainlet came up with this
this is ridiculous
@near tartan would it be fine if i DMed you about the logo?
ok
wtf is that code?
X && X
i < buf.len() && i< buf.len()
also a for loop would be better
oh, well for just i < buf.len() you could
oh, didnt know that
i probably didn't have to make it a const fn
but this is some stupid overhead i don't wanna deal with
@hoary cave turns out i probably can't use traits for FileOps, because those wouldn't be dyn compatible

nvm i guess i can do this
but that's a bit yucky ngl
ur supposed to
impl FileOps for files
this guy bruh
whar
why are u storing a vtable manually
wdymmmmm
::::::::::::::::::::::::::::::
rtgrwrgtbiuurib
marvfin password generator
it can't either????
why not
because that's also controlled by the file system
that's what linux vfs does at least
idk pain man
h
look at aero 
๐
maestro does it "wrong"
it stores the name right in the node
this looks like a struct dentry
me when i don't read the comment
Pub
or look at the structure contents
Wow look this struct is in a pub
atomic reference counted heap alloc
a std::shared_ptr :^)
yeah i was about to say
And Linux is the only objectively true good source for that?
no but i have no idea how to do it efficiently so i look for known good sources ๐คทโโ๏ธ
clearly what linux does is working out
Tbh I should really get to do a vfs so I can actually give out usefull responses to that topic
lol
nvm about maestro btw
it's just called differently
it has INode typedefd as a u64
and Node is the actual struct
i fucking hate github search
Just an idea, what about a graph datastructure?
go on?
Like a simple file system is like a tree and now you throw in symbolic links and multiple mount points which allow it to loop back into itself, at that point it's just a big mess of interconnected nodes, a graph
sure
but i mean it's not really a tree, right?
if you have a bunch of Arc's then it's basically a complicated linked list
i don't see the need for any special data structure
It's a tree where each node can have a arbitrary amount of child nodes
symlinks don't point to nodes
To be more specific I would say one can base a vfs on a sparse graph
symlinks just contain a "reference" to something, they don't point to the actual node
What is that something?
symbolic links are just path names, right?
symlinks are just blobs that are interpreted as paths by the os
yes
symlinks are regular files with a different file type
they don't point to anything themselves
your kernel decides what they point to
but the target node is not part of the actual vfs "graph"
and hard links are pointers to other nodes
but i don't think they allow you to create invalid cycles
eh should be easy enough
i'll just implement something that """works""" and fix it later
Famous last words
ok path traversal already kinda works
what youre doing is weird lol?
how so
a File is an open file descriptor
why do you have a dyn FileOps inside file, why not just impl FileOps for File?
because these functions are different depending on the underlying file system
oh actually that makes sense
Maybe use a Filesystem abstraction layer?
Also which for example?
read/write?
how is that different between two filesystems?
what?
different code required
ext2 does different things on write than tmpfs
isn't this super obvious?
by putting the functions in the File struct i can store an array off File's without having to care about what file system they belong to
i just have to call the trait impl
but they do have the same functions signature?
why does that matter
the implementation is different
that's the whole point of an abstraction
tbh i still dont get it how they are so different from the vfs level, sure what a fs does internaly may differe but shouldnt the interface be constant across different fs?
yes?
i really don't get the confusion
the interface is the same
what makes you think that it isn't?
@idle flower
what i wonder is why you dont just store like a trait fs_api object in the node, where the object represents a fs instance, that way the vfs trait doesnt need to be dyn
or do i miss something?
that would still have to be dyn
File is the abstract struct that represents an open file
because it's abstract, the compiler cannot know what to do when you call e.g. File::read
therefore the trait has to be a dyn
per definition
since it's essentially a vtable
yea
insane call stack
how do you cope with the vscode debugger
wdym
i've always found it to be really painful to use
i have a debugging config
it does all kinds of stupid shit at least when using gdb
ah maybe lldb is less ass
use codelldb
i've heard rust didn't bother writing a debugger
you can just use lldb
that they just use gdb as if for c
i dont suppose lldb has the same plugin api as gdb does
i have alot of gdb scripts that i dont particularly want to port lol
why do you need scripts
printing custom data structures, dumping hardware state, that sort of stuff
ah
there's this but definitely not the same api
rn i'm trying to figure out why openrc doesn't give me a /usr/sbin/init
i like the gdb commandline its just the vscode integration thats ass
you need to put -exec before every command you type
this is by design because whoever made the extension is a retard
the lldb extension is much better
kek
bro
they really gated openrc-init behind linux
wtf
but then they still have ifdefs?
they don't call that
incredible
you have to manually provide -Dos
leave it up to big projects to fuck up the most basic build configuration
i think this is on purpose
maybe because BSDs have their own init?
and openrc runs on top?
the symlink and get_option('sysvinit') stuff makes sense
but needing to provide the os as a build flag is dumb
at the very least ```py
target = get_option('override_target')
if target == ''
target = host_machine.system()
endif
oh i see why they gate it behind linux
duh
reboot is a linux syscall
makes sens
easy fix, i'll just patch menix in
i have linux reboot anyways
wtf
i guess not
AAAAAA this is an mlibc issue
noooo
don't tell me i didn't push that
fuck
@spark surge hi can you look at https://github.com/managarm/mlibc/pull/1368 pls
much โค๏ธ
damn mlibc doesn't implement updwtmp yet
time to do that then
i could also just stub it out smh
Does mlibc have rcu libs?
the what
The rcu library that stdlib has
which stdlib
rcu is c++26 btw
then again, mlibc is not a c++ stdlib
Lol
and C does not have rcu in libc
no stdlib has implemented rcu or hazptrs yet
theres implementations of both from the paper authors on github if you need them
folly has good implementations of both as well
honestly just put google and facebook on the standards committee and let them make the stdlib abseil and folly
merged
tyy
im gonna implement some linux function
idr what its called exactly
something with updw
updwtmp
that's it
idk what it does yet but it's missing
and openrc wants it
i love osdev
more vfs stuff today
in the next few weeks I'll be building my dual epyc server to optimize my smp logic
(I'll probably have to touch the allocator again for cpu local regions)
just found this thread and reminded me of a question I was gonna ask a few days ago, how inter-compatible with either glibc or musl is mlibc?
depends
some structs are slightly different
but new features are usually kept similar to musl
and mlibc has glibc extensions
motherboard has been obtained
sp3 mobo ๐
๐
nice, which cpus are you going to use in it?
damn thats a good deal
Is that basicly a ATX mainboard? Does it support redundant psu's?
Welp lga2011-3 can't compete with that
deep eatx
but yes
basically a PC
So no psu redundancy?
ye
i can't afford that anyways
if you wanted redundant psus you need to buy a server blade
You mean rack server or actual blade servers?
I have one Dell r730xd at home, loud af
that's why i didnt go for a server
im building a rack mount pc basically
because sp3 is physically compatible with tr4
i can just use off the shelf threadripper AIOs
Nice
Wtf is TruDDR4? Just saw that in the specs for Lenovo servers
It's always a good sign if there isn't a price tag on the product page
r/udimm?
Lenovo lists it as TruDRR5 4800 RDIMM
You gave me the idea to look for blade servers, the amd ones would outperform even my server but at the same time burn a hole in my pocket
@nocturne rampart i got more vfs shit done now
i think i might actually get it working today
Nice
yippee
๏ฟฝ๏ฟฝ
yes
??
i handle non-utf8 characters
because rust's String only accepts valid utf-8 strings
but vfs entries could be non-utf-8
makes things a tad bit more complex but whatever
doesn't rust have a path type for that?
guess why i'm not using it
I'm guessing you can't use it with no_std
ding ding ding
what do I get
but you can use rust strings?
yes
strings are part of alloc
ospath requires host support
because its definition/behavior is os dependent
Vec<u8>
close enough
and for display i convert into a lossy string
which replaces invalid utf-8 chars with the replacement char
makes sense
the last part that i'm struggling with is mounting
idk how that interface should look like
like from an API standpoint
BadgerOS on this topic:
// Try to mount a filesystem.
// Some filesystems (like RAMFS) do not use a block device, for which `media` must be NULL.
// Filesystems which do use a block device can often be automatically detected.
errno_t fs_mount(char const *type, fs_media_t *media, file_t at, char const *path, size_t path_len, mountflags_t flags);
I haven't done this yet but it'll determine the actual path by means similar to calling realpath
That makes the mounting have an API pattern basically the same as the other filesystem functions; at + path vector (bytes) identifies location.
how do you keep track of mounts
I have:
- A static struct for the root filesystem
- A struct inserted into the file object for non-root filesystems
And TODO but: - A table of real path to VFS structs (excluding root)
Doesn't support stuff like mount namespaces nor stacked mounts in this model but it keeps it nice and simple.
That last one is useful for having one overview of all mounts but isn't necessary for this model to work.
a nice way to represent mounts in the VFS is to let paths be a ยซmount, namecache entryยป-tuple.
pseudocode for mounting a filesystem:
errno_t
do_mount (struct path mountpoint, struct namecache_entry *fs_root)
{
struct vfs_mount *newmount = malloc(sizeof(*newmount));
newmount->parent_mount = mountpoint.mount;
newmount->parent_nc_entry = mountpoint.nc_entry;
newmount->root = fs_root;
// initialize other fields...
// I guess also properly refcount parent_mount, parent_nc_entry, root
// and obviously do proper locking too
mountpoint.nc_entry->child_mounts.insert (newmount);
}
pseudocode for entering mounts:
void
enter_mount (struct path &path)
{
again: // again, do proper locking and refcounting
vfs_mount *current = path.mount;
for (vfs_mount *child : path.nc_entry->child_mounts) {
if (child->parent_mount == current) {
path.mount = child;
path.nc_entry = child->root;
goto again;
}
}
}
purely from an API standpoint, these internal details of filesystem mounts shouldn't be relevant, though.
a ยซmount, namecache entryยป tuple is basically equivalent to the Linux ยซmount, dentryยป tuple which is used for paths
also, walking .. in such a system might look something like the following:
void
walk_parent (struct path &path)
{
// again, do proper locking and refcounting instead of this shite.
while (path.mount->root == path.nc_entry) {
path.nc_entry = path.mount->parent_nc_entry;
path.mount = mount->parent_mount;
}
path.nc_entry = path.nc_entry->parent;
}
I just realized I was doing this pattern without noticing
oh, and this does support stacked mounts and bind mounts
mount namespace-like functionality can be implemented with some extra ifs in enter_mount and walk_parent, probably
something like
if (child->parent_mount == current && child->mount_ns == current->mount_ns) {
path.mount = child;
path.nc_entry = child->root;
goto again;
}
FWIW BadgerOS makes a clear separation between the filesystem code and individual implementations. Said implementations only ever deal with calls qualified by either a file handle or a dir handle + name combo. The generic filesystem code deals with walking the filesystem and related while individual implementations need only worry about FS specifics. That makes the actual FS-specific code only see the media and flags like so:
// Try to mount a FAT filesystem.
errno_t fs_fat_mount(vfs_t *vfs) {
But you probably already do this.
(media and flags stored in the vfs_t struct)
I don't quite get why this is beneficial.
Like if you have a global linked list of all mountpoints (or a list per mount-namespace), and you can go from mountpointโpath รก la getcwd, then you can do this without a table of path to mountpoint
no I don't have anything resembling a VFS yet in my kernel
ehhh
Fair
I don't actually have that global collection yet and the vfs_t already stores the mount path so...
Is your current code up?
the old version didn't have mounts so I've no idea what I'm actually doing
I can probably help translate
on dev
๐
Nitpick: BadgerOS supports un- and re-mounting root, so using Once may not be needed.
wdym
no
once just means that it's uninitialized until it is explicitly set
it's kinda needed because Arc cannot be stored statically
since that needs a heap allocation
i could store it as an option
I'd just use an Option<Arc<...>>
yea fair
if you don't mind, note down all the things that you can find so i can look into it in the morning
Will do
Proposed changes:
- Consider using
Option<Arc<...>>for the root filesystem Filecurrently points to aFileOps,Inodeandposition. This is different from BadgerOS' model where the equivalentvfs_file_desc_tonly stores offset and a reference tovfs_file_obj_t, the latter being what actually specifies the file operations.- Assuming suggestion 2 is implemented, you should have
Filecall read and write with absolute positions onINodeusing shareable mutexes where the mutex is taken shared when reading/writing, but exclusive when the file is resized. This improves performance for concurrent accesses to one inode and is POSIX-compliant. - Consider making a relation between
INodeand the VFS (I think you useSuperBlockfor this) for non-root mounted filesystems. - Consider renaming
SuperBlockto something that indicates it is a mounted filesystem, e.g.MountedFSandFileSystemto something likeFSDriver.
Proposed additions:
- BadgerOS implements stuff like
/dev's files by creating facilities for special types of device files, which are supported by BadgerOS' RAMFS so that devices have a very simple API to register their device files:errno_t fs_mkdevfile(file_t at, char const *path, size_t path_len, devfile_t devfile);. - As per david's suggestion, consider having a collection of all mounted filesystems in a central location.
- While BadgerOS does not yet have file caches, consider making those at the
INodelayer (normal files+dirs only of course) on the generic side so it's not the VFS's problem to deal with caches.
-# more incoming
Separation of concerns:
BadgerOS separates the following concerns each into their own file or set of files:
- Filesystem media abstractions (block device or RAM); single API for using loopback files, RAM as media or an actual block device. https://github.com/badgeteam/BadgerOS/blob/99dc25357f46db45a4eb22936e63bde14aa56786/kernel/src/filesystem/media.c
- Caches for block devices (those are in the device subsystem); this also deals with byte-level accesses to media that doesn't support it. https://github.com/badgeteam/BadgerOS/blob/99dc25357f46db45a4eb22936e63bde14aa56786/kernel/src/device/class/block.c
- A specific filesystem implementation; it is only concerned about translating generic to filesystem-specific and the upward-facing API will look somewhat similar to the one for generic operations on file objects/inodes. https://github.com/badgeteam/BadgerOS/blob/99dc25357f46db45a4eb22936e63bde14aa56786/kernel/src/filesystem/fs_fat.c
- All filesystem walks and operations on file descriptors; the reads/writes with implicit offsets for example. https://github.com/badgeteam/BadgerOS/blob/99dc25357f46db45a4eb22936e63bde14aa56786/kernel/src/filesystem/filesystem.c
- All operations on file objects/inodes; mkdir, link, unlink, creation of new files, the reads/writes use explicit offsets here and resizes are also explicit. Caching for on-disk files would happen here is BadgerOS had it yet. https://github.com/badgeteam/BadgerOS/blob/99dc25357f46db45a4eb22936e63bde14aa56786/kernel/src/filesystem/vfs_internal.c
All of this structure comes from quite some time of me of figuring out how best to structure this all. I hope this can be of any help! If you more need help you can always ask me a question :3
regarding /dev, ill have a devtmpfs that handles this
That's mostly an implementation detail but BadgerOS' way of dealing with it is more flexible than being restricted to just one implementation that can hold device files, should I ever choose to do something of the sort.
But like I said implementation details. Not terribly consequential compared to the other points.
elaborate on point 2 of the proposed changes please
Basically you have the "file operations" tied to your File structs whereas BadgerOS only does this for FIFOs and devfiles, neither of which have offsets. The regular files (and also reading symlinks for example) go through the equivalent of your INode struct instead, where the mounted filesystem is asked to do an operation directly on the inode.
Note: For file-level caching, the cache would be in the inode and the filesystem implementation would not see the effect of a write until synced from this cache, which sends just the one write to sync it.
RE: proposed change 2
Assuming that File is a file descriptor object, it should be noted that Files can be acquired from other sources than the VFS, e.g. via socket(), connect(), etc.. These don't really have underlying filesystem objects (iiuc Linux uses a "fake" underlying struct inode for these), and so it makes sense to have FileOps such as read, write be separate from the INode. It should also be noted that this comes at no additional indirect call overhead vs. putting file operations like read/write in the INode, because you'd still do (at least) one call via function pointer in either case. Then there's also the case of open on a char device or block device. These have real underlying inodes that are part of a real filesystem, but they dispatch into something else (a device driver or a block storage driver).
if I had a nickel everytime you said "BadgerOS" i'd probs have a few dollars by now
i agree with this
it's more ergonomic to do that
or dont use the concept that everything is a file desciptor and go for something like everything is a resource handle and a file descriptor is just a specifc type of resource handle
that's what I do but you still need to be able to decouple file ops from inode ops to implement POSIX properly
like most io boils down to control io and bidirectional data stream, the data stream is basicly the same but control io is vastly different
e.g. if you open a named pipe you should get a file description but no operations on that file description should involve the actual filesystem you opened it from
thb i wouldnt call such a descriptor a file discriptor from the os perspective as it will inevitably lead to confusion, how posixs and other api's actually represent that descriptor is their problem
I'll look into it now in like an hour or two
im definitely gonna keep the file ops as part of the file struct
what's the difference?
a resource handle would represent a generic handle and specific handles such as network and filesystem would build on top of it, so its mostly a high level differentiation
sady no, im not in userspace yet
for example in linux everything is a file descriptor.
That means that i could do file operations on something that isnt a file, like a usb device or network connection
thats not sane and if you explicitly create different handles (where each one has its own set of possible operations) its clearly visible what you get and how to use it
so how do i abstract that
it would be like rusts type system a compiler thing
technically yes
hm
but only in the code where the file ops is actually called
well yes, but that would essentially mean that for file ops my resource is just an fd called resource
i dont get the semantical difference
well the fd would implement the resource handle trait for example, which defines operations that dont care about the specific type/are generic over all types (like close or get type)
trait ResourceHandle: Drop + Clone + Send + Sync {
fn close(self);
fn get_type(&self);
}
struct FileDescriptor {}
impl ResourceHandle for FileDescriptor {
fn close(self) {
// Close the file descriptor
}
fn get_type(&self) {
// Return the type of resource
}
}
surely there's more common functionality than close
probably, its just an example
I use this system in hydrogen, and yeah pretty much the only necessary common functionality is close (or, more accurately, free() when the last handle to an object is closed)
Although I do have some more stuff in the generic object vtable for event queues
hm
This is the common stuff
i could implement this in drop()
in rust i can have an external refcount
i guess i can have a dynamic drop impl
how do you use object_t?
this is my file description struct as an example
ops is a pointer to <file_ops_t>.base
verifying the object type is done during handle -> object resolution
yeah idk what the best way to model it with structs+traits would be
it's kinda meant to mimic a proper inheritance tree
except without language support
yea no inheritance in rust sadly
i guess i can have something like
struct Object {
ops: Box<dyn ObjectOps>,
}
trait ObjectOps {
fn drop_obj(&self) {}
fn get(&self) -> Arc<Object>;
}
struct File {
base: Object,
inode: Arc<INode>,
}
impl ObjectOps for File {
fn get(&self) -> Arc<Object> {
self.base
}
}
then i don't need to mess with ordering guarantees
no that kinda sucks, make use of the cool rust type system instead
we do the proxima way in dufay and it'd be much better with e.g actual classes or type traits
hm
idk how
i can't really have an enum because that can't be extended with modules
and traits are just vtables
Traits can inherit from one another though
Yea that's good but add a refcount maybe
true
the more i think about this i think i actually did that in the old version ๐ญ
yea i'll just do EResult<()>
it's very sensible
you can read() and write() a TCP socket fine
every real OS is having this
the resource idea was actually really good
that allows me to work with a single trait
is there anything i need besides r/w, seek and ioctl?
mmap/munmap?
oh true
hmm shouldn't seek be in the struct that's accessed with a file descriptor
I'm guessing your resource is like the one in vinix
tries to change file permissions of a network socket
It'll fail
Do you need that though
Probably only for devices that you could actually mmap like frame buffer devices
But the rest you can simply fill in pages by reading normally and memcpying into the destination page (and into your page cache if you have it)
you can mmap regular files wdym
yes you can
but the file system shouldnt provide its own implementation of mmap
unless you REALLY want to repeat code
instead, let the OS handle it, UNLESS the file is SPECIFICALLY allowed to provide its own physical pages
(such as an fbdev)
just have the repeated part as a separate function, why overcomplicate it?
void mmap(...)
{
if (file->can_provide_physical_pages())
mmap_those_physical_pages()
else
generic_mmap_commit_impl()
}```

boron has (will have) a BACKING MEM method which does exactly this
BSTATUS(*IO_BACKING_MEM_METHOD)(PIO_STATUS_BLOCK Iosb, PFCB Fcb)
(FCB is the equivalent of Vnode)
i just have a filesystem-generic file ops struct for regular files
and regular_file_mmap maps the page cache for the vnode (which is the only part the fs controls)
i mean yeah it's not for the sake of performance or something like that i just think it's cleaner
I suppose that works as well
Honestly my thing might be a bit less restrictive when it comes to mapping files that arent "regular files"
Since IO_BACKING_MEM_METHOD would just return the starting physical address of the device's exposed mappings
for that i just use a device-specific memory object
for example /dev/mem has a memory object that, when mapped, maps the portions of the mapping that aren't kernel-owned ram, and returns ENXIO when something tries to access an unmapped area
why would you expose physical memory as a file... at all?
linux lets you do funny stuff with that
userspace drivers
hydrogen isn't quite a microkernel because it does all the non-hw-specific stuff in kernel (vfs, processes, but also specific filesystems), but actual hw drivers run in userspace
also when doing this i am extremely careful to restrict access to actual ram that's owned by the kernel and not the firmware: doing so is only allowed through an fd that you get with an ioctl that allocates physical memory
ah i don't think I'll be doing that so I'm not sure how useful it is for me
linux also doesn't do that and it still has /dev/mem (though similarly restricted: you can't access kernel-owned ram at all unless you configure the kernel to allow it, otherwise it just zero-fills the buffer) so it must be useful for something
we at suse recently got rid of it because of security concerns
we were able to change uname to output windows
lol what
thats awesome
because there were so many issues with the phmem module
we just got rid of it completely
that's fair
also this
we were able to modify drm control structures
and we could crash the entire system by writing garbage to hpet and lapic mmio regs
i mean yeah being able to write to hardware registers in mmio through /dev/mem is to be expected
i think the worst part was that this could work without root
wait what
yea
wouldn't that require an exploit to bypass linux's regular file access checks
or do you mean by root chowning/chmodding it
yeah but i'd expect that part to be part of the generic vfs code
like if non-root tries to open a root-owned 0600 file the driver code should never even be entered right
ah nvm the base operation does require root
thank fuck
ah i remember what it is
you can do it with secure boot enabled
you can just ioremap random structures
that's still pretty bad
but yeah, otherwise i don't see the point in exposing phys mem to userspace
@vast lotus what is the event in struct object for?
you mean event_add, event_get, event_del?
yep
those are for my event queue support, it's similar to kqueue and it's what i use to implement poll/select
event_add validates the event type and registers the queue to listen to the given event, event_get gets some more data about the event and returns whether it's still pending, and event_del unregisters the event
and the object itself doesn't store anything regarding events?
kinda? for each event type there's an event_source_t somewhere, which maintains a list of listening queues
but event_add is responsible for finding that
it's not in object_t itself because objects can support multiple event types and each event type needs its own event_source_t (and the event sources might not be per-object, like with fifos, where they're per-inode instead of per-file-description)
for example this is where they are for fifos and pipes (this is per-inode data)
right
i finally got around to adding a stack trace
idk if i want to bother with demangling
Reason 92882 not to rust
you misspelled c++
Oh?

Isn't this rust
no
Why does it say rust
it was a joke ๐ญ
Lol
Yeah