#Zinnia

1 messages ยท Page 9 of 1

near tartan
#

user space

#

okay jumping to user space works

#

now i just need to hook it up to the scheduler

#

or rather, i need to do the actual switch yet

somber solar
#

@near tartan

near tartan
near tartan
#

[00:00:00.002433] [info] uacpi: successfully loaded 1 AML blob, 1705 ops in 0ms (avg 4373970/s)

somber solar
#

what cpu is that?

#

because I get the exact same result on ryzen 5 4650g

#

and faster cpus usually produce larger numbers :P

near tartan
#

265kf

somber solar
#

that is definitely better than mine

#

weird

somber solar
near tartan
#

[00:00:00.002445] [info] uacpi: successfully loaded 1 AML blob, 1705 ops in 0ms (avg 5401158/s)

#

lol

hoary cave
#

someone said that intel lake is particularly not good at uacpi

near tartan
#

:(

#

btw @somber solar your kernel crashes

somber solar
#

crashes?

#

there should be an error during module loading

near tartan
#

yes

somber solar
#

that's expected

#

(because lto)

near tartan
#

@somber solar can you test a new image?

somber solar
#

sure but in a bit

near tartan
#
qemu-system-x86_64 -m 2G -bios OVMF_x86_64.fd -no-shutdown -no-reboot -cpu host,migratable=off -accel kvm -machine q35 -drive format=raw,file=menix.img,if=none,id=disk -device nvme,serial=FAKE_SERIAL_ID,drive=disk
somber solar
#

maximum I was able to get

#

doesn't boot without ovmf

near tartan
#

wtf

#

yea it need ovmf

#

that sucks ass

#

why is it so slow ๐Ÿ’”

#

[ 0.001047] uacpi: successfully loaded 1 AML blob, 1729 ops in 0ms (avg 4492075/s)

#

i can't get past 4.5M

cyan nexus
#

how do you tes the speed?

near tartan
#

uacpi does

cyan nexus
#

i think it has problems with my function then

near tartan
#

you can't stub the counter duh

#

you need a timer at that point

cyan nexus
#

this is how i do it

near tartan
#

that means it's returning 0 twice

cyan nexus
#

huh

near tartan
#

/shrug print the value

cyan nexus
#

i think i will

#

oh wait

#

ik why

#

i enable my interrupts after uACPI loaded

#

so the PIT does not start counting

near tartan
#

bruh

cyan nexus
#

lol

#

a bit slow

#

lol

near tartan
#

what cpu

cyan nexus
#

something tells me the print function is the culprit

cyan nexus
#

qemu runs under wsl too

#

and i added xhci too

near tartan
#

@hoary cave i finally got around to start using the MmioField abstractions

#

and they work as expected

#

this is so much nicer

#

the big endian stuff also works

somber solar
#

no pci?

near tartan
#

soon

hoary cave
near tartan
#

true

hoary cave
#

you don't modify it and you get get rid of the redundant &

#

and llvm is smart enough to optimize it all out

near tartan
#

mfw

hoary cave
#

not even the copy->ref but even into full specialization unroll considering it's all const

#

eah

#

you need to make the copy

#

that's expected bruh

near tartan
#

how tf do i copy

hoary cave
#

??

near tartan
#

do i need to derive clone

hoary cave
#

#[derive(Copy)]

near tartan
#

oh

hoary cave
#

and clone i guess

near tartan
#

yea

hoary cave
#

idk if oyu need close

#

i think yuj might

near tartan
#

cant hurt

#

better?

hoary cave
#

i like it better at least

near tartan
#

same

hoary cave
#

without the annoying &s

near tartan
#

with that i can actually do PCI config space stuff without cursed packed structs

hoary cave
#

you could probably put the regs in impl Hpet

#

then do Self::MAIN_COUNTER

#

but idk what you prefer

near tartan
#

i think that makes sense

#

i like the mammogram way though

#

having a spec namespace

#

but it also makes sense to tie it to a struct

#

ugh

#

need to figure that out

#

i guess for multiple structs i can do smth like this?

struct SpecA;
impl SpecA {
    const FIELD_A: MmioField::<u64> = MmioField::new(0);
}

struct SpecB;
impl SpecB {
    const FIELD_B: MmioField::<u64> = MmioField::new(40);
}
hoary cave
#

nah

#

just use a module atp

near tartan
#

yea but how do you seperate the fields in a module

#

another module?

hoary cave
#

yup

#

or a prefix

near tartan
#

lgtm

#

hm how should i do Stack MM

#

i was thinking of making a struct Stack

#

but that might be overkill

#

but then i could do stuff like safe stack manipulation for auxvals

#

or refcounting or whatever

near tartan
#

i did

dense vine
#

.

dense vine
near tartan
#

[00:00:00.00309] [ info ] uacpi: successfully loaded 1 AML blob, 1726 ops in 0ms (avg 5416703/s)

dense vine
#

That's nice

#

Thanks

near tartan
#

how's yours so fast

dense vine
#

Idk

#

Lol

near tartan
#

do you stub anything?

somber solar
dense vine
#

No

dense vine
near tartan
#

odd

#

you just use talc for everything?

dense vine
#

I want something faster

#

But

#

Too lazy

#

Plus it's fast enough alr

near tartan
#

? this is plenty fast aready

dense vine
#

Yeah

near tartan
#

but it won't work in the long run

#

this is why i had to get rid of talc too

dense vine
near tartan
#

you need to manage your own phys pages

somber solar
#

except dma

near tartan
#

exactly

#

and at that point you need to manage your own

somber solar
#

having phys page allocator isn't a necessity

#

it's just better

hybrid island
dense vine
#

Might go for bump allocator before initing uacpi and then switch to talc just for the points lol

near tartan
#

that's kinda not the point lol

dense vine
#

Ik

hybrid island
dense vine
#

Isn't bump allocator the fastest?

somber solar
#

because bump isn't freed and isn't cached

near tartan
#

uacpi does a lot of allocs and frees

#

like 200+

somber solar
#

alloc and "free" will take zero time, use of that memory will be slower

dense vine
#

O yeah

hybrid island
#

^The memory will not be cached so you get full penalty of fetching main memory the first time you write into such an alloc.

dense vine
#

Ig I'll stick with talc then

#

For now

hybrid island
#

Which takes over 200x as long as L1 cache on modern systems

dense vine
#

@somber solar didnt u used to use friggs alloc?

#

Is it good?

somber solar
#

yes

dense vine
#

Better than talc?

hybrid island
#

If you think about it, a computer's memory is just cache all the way down.

  1. CPU registers
  2. L1 cache
  3. L2 cache
  4. L3 cache
  5. RAM
  6. Disk
somber solar
#

probably

dense vine
somber solar
#

I don't memorise tv show scripts

dense vine
#

You just finished watching it

somber solar
#

a month ago

#

or 2

dense vine
#

Making me feel chronically online

dense vine
#

It's probably the lack of features

dense vine
hoary cave
#

it's not like the speed decreases with the amount of features

near tartan
#

yea

vast lotus
#

More features means larger executable means lower chance of uacpi code having favorable cache locality, I suppose

#

But that should be an extremely minor factor

hoary cave
#

ah so that's why linux only gets like 1-2M points

#

that makes perfect sense

#

guys we should make our kernels less featureful

near tartan
#

buddy moment

#

eh tbh i'm totally fine with that score

hoary cave
near tartan
#

i shouldn't care that much about it anyways

hoary cave
#

not fully implemented of course

#

but it does namespace init i think

#

alongside acpica of course

ionic lava
near tartan
dense vine
near tartan
#

some pc font

idle flower
near tartan
near tartan
#

okay break time is over

#

next week i'm on vacation so i'll have plenty of time to work on this

#

i really want to get vfs working again

drowsy tendon
#

i might be willing to help out at some point

#

i'll see what i can cook up

near tartan
#

@analog crane what exactly does a real time kernel entail? what do i have to do differently?

analog crane
#

predictability and consistency

#

depending on how hard you go with it, you can be a soft real-time system or a hard real-time system, a soft real-time system is linux trying to be predictable for your video to not stutter, a hard real-time system is the contoller behind the moderating rods of a reactor

near tartan
#

hm

#

i think it would be a nice feature to have

#

is ironclad hard rt?

analog crane
#

as to concrete things you might want to do, you want to avoid interrupts as much as possible, you want to have simple and predictable scheduling algorithms with quotas, maximum and minimum usage percentages, that stuff

analog crane
# near tartan is ironclad hard rt?

Ironclad is capable of being something pretty close to strict hard real time under certain configurations, certain hardware, and certain task configurations

near tartan
#

nice

#

i think soft rt should be doable

analog crane
#

soft RT is a fine goal for a general purpose system and it has applications for audio/video, recording, that stuff

#

POSIX has facilities for it with SCHED_**

near tartan
#

yea i'm trying to "market" it as a general purpose system

analog crane
#

then be careful how you advertise the real time aspect, because when people say real time, they usually mean hard real time

#

so people will think you are being misleading

near tartan
#

it won't be a major thing, just something to add to the readme

#

something like
- soft real-time scheduling constraints

#

yknow, to farm github stars

analog crane
#

then be sure to have rocket emojis, people love those

near tartan
#

๐Ÿฆ€ ๐Ÿš€ blazingly fast scheduling

analog crane
#

no real time system is real time always and the same way

near tartan
#

"Capable of providing soft real-time scheduling"?

analog crane
#

yah

near tartan
#

now to implement that...

analog crane
#

its relatively easy compared with hard real time

#

depending on how your OS works its probably 30 mins

near tartan
#

nice

analog crane
#

its best effort

#

you get to define best effort

#

thats how soft RT works

near tartan
#

:3

analog crane
#

but what most people understand as soft RT as far as POSIX is concerned is SCHED_FIFO/SCHED_RR

near tartan
#

just call it "dynamic scheduling policies"

near tartan
#

i love scheduling

cyan nexus
#

lol

#

i hope mine works

#

i'll test it soon after i implement context switch

near tartan
#

i'm definitely messing up the RSP

#

but idfk where

#

or how

#

ok nvm i got it

#

i was returning and corrupting the stack frame lol

cyan nexus
#

lol

near tartan
#

anyways

#

finally

cyan nexus
#

time to test mine too

near tartan
#

okay i fixed scheduling from task 1 to task 1

#

i saved the old rsp to the task struct but didn't restore the now overwritten task

#

lol

#

slight oversight

near tartan
#

HAHHAHAHHAHA

#

lmfao

modern hamlet
#

Yes thats right

vast lotus
#

i just if (old_task == new_task) return;

near tartan
#

i guess that works

modern hamlet
#

i second that

hybrid island
near tartan
#

okay done

#

now i can finally work on the vfs

#

this is going to be very interesting...

#

idk yet how to even approach this in rust

hybrid island
#

Wait is this a rust re-write of a formerly C project?

near tartan
#

ye

hybrid island
#

ah

near tartan
#

(pain)

#

in C i had the scheduler and vfs already working

#

i guess first step is doing initrd parsing

#

or vfs root

#

then initrd

hybrid island
#

I am seriously tempted to do Rusty BadgerOS

#

But to do so I would need Rust abstractions for all my stuff

near tartan
#

you can also do a c kernel with rust ffi

hybrid island
near tartan
#

i already had that working in the legacy branch

#

FFI is pretty simple

#

especially if you build as a shared object

#

idk about staticlib

near tartan
#

im tempted to make a fusefs driver

hoary cave
#

bro u dont even have userspace ๐Ÿ’€

#

or a vfs

#

do those first!!!!

#

or im gonna beat u to it in my own kernel (i already have kernel threads)

near tartan
#

alright doing vfs now

#

i'll just slap Arc everywhere

near tartan
hoary cave
near tartan
#

i have plenty of time now

#

what were you doing yesterday anyways

#

@hoary cave

hoary cave
#

i did some stuff in my kernel but nothing interesting lol

near tartan
#

bruh

#

did you start from scratch or just work more on what you already had

hoary cave
#

yes from scratch

near tartan
#

zamn

hoary cave
#

i don't do much anyway

near tartan
#

:(

#

i'm stuck at this conference

hoary cave
#

i don't have a proper allocator

#

no scheduler yet

#

i'm gonna work on a dispatcher next

#

add timers

#

semaphores and mutexes too probably

near tartan
#

i'm doing initrd rn

hoary cave
#

but i feel like i need to restructure my kernel

near tartan
#

what's it like atm

hoary cave
#

not very interesting lol

#

private on github atm

#

i'll probably make it public when it can do something other than print a memory map

near tartan
#

dam

near tartan
#

oh damn i actually found a solution for the generic irq handler issue

#

instead of providing a function with a *mut () context i can just make a trait that saves self as the contexgt

#

then when i call the handler it'll automatically use self as the context without any pointer hacks and fully safe code on the impl side

near tartan
#

should i have the UAPI bits in rust or in C?

#

@late bramble what do you do?

#

i could have a bindgen from C and then have rust wrappers around the uapi

#

but things like SEEK_WHENCE, etc

hoary cave
#

typically the kernel provides the headers with bits

near tartan
#

i can do either way i think

#

i can generate C bits from rust code or the other way around

#

but tbh C bits make more sense

#

right?

late bramble
#

uapi?

near tartan
#

the interface between user and kernel

#

like concrete values for SEEK_SET

#

or struct stat

late bramble
#

that's in the libc, right?

near tartan
#

yea but the kernel needs an equivalent structure

#

if it's different then the data will be broken

late bramble
#

I rewrite them in Rust by hand

near tartan
#

oh right you don't use mlibc

#

you use musl

late bramble
#

yeah

near tartan
#

so i guess you follow linux abi

late bramble
#

yes

near tartan
#

hm

#

i think i'm just going to make them their own bindgen'd crate

late bramble
# near tartan you use musl

I could probably use glibc too but as I said in my thread, there's a syscall it uses that I did not understand at the time halfmemeright

near tartan
#

glibc ๐Ÿคข

late bramble
#

yeah, bloated

near tartan
#

you could also use mlibc with linux meme

late bramble
#

never looked at mlibc tbh

near tartan
#

you should

#

it's a very clean impl

#

though not complete obv

#

that was easy

near tartan
#

surely this isn't going to explode ever

hoary cave
#

lol

near tartan
#

idk

modern hamlet
#

also why the hell not stdint.h

near tartan
modern hamlet
#

ah so system calls and that kinda thing?

near tartan
#

more like errno

#

like the concrete values for posix

hybrid island
#

There's also the LLP64 ABI which IIRC is 64-bit but has 32-bit long.

near tartan
#

llp is stupid

hybrid island
#

I know

#

Still exists anyway

near tartan
#

do i have to care about it tho? can't i just say that i need lp64

hybrid island
#

ofc you can

#

I guess either way this is free Rust advertisement because it doesn't have this problem trl

near tartan
#

hm problem

#

iirc errno has to be a #define list

#

but rust bindgen doesn't care about defines

#

can i make it an anonymous enum?

hybrid island
#

Technically, yes, that is how you're required to define errno

#

Is it really the case bindgen doesn't do defines? I thought it did.

#

Either way, though, you'll probably be fine using anonymous enums.

#

Unless something actually #ifdefs on an errno value, an anonymous enum results in the same type: int.

near tartan
#

i guess i could do both?

#

ah nvm bindgen is goated

#

it can actually understand value defines

vast lotus
#

yeah i wouldn't use enums for errno, lots of things use #ifdef on errno values since some errno values only exist on some systems

near tartan
#

seems to work

modern hamlet
#

Or do you have to provide the integer value outright

hybrid island
#

I think theoretically you shouldn't do that.

#

Buuuuuuut

#

That's literally never gonna be a problem in a sane codebase

near tartan
#

sane

#

trust the user

hybrid island
vast lotus
near tartan
#

posix

vast lotus
#

posix

modern hamlet
#

What if they dont?

vast lotus
#

The <errno.h> header shall define the following macros which shall expand to integer constant expressions with type int, distinct positive values (except as noted below), and which shall be suitable for use in #if preprocessing directives

near tartan
#

not posix compliant

vast lotus
#

the main issue you'll experience is that they won't be suitable for use in #if preprocessing directives

somber solar
modern hamlet
#

oh so like #if ELOOP == 5?

#

that type of stuff?

somber solar
#

why not?

modern hamlet
#

cause if you put it in an enum it doesnt get expanded to an int at the preprocessor stage

#

and you cant use #ifdef on it

vast lotus
vast lotus
#

an anonymous enum expands to an enum constant expression with type int

#

so at the preprocessor stage it's still just an identifier

#

instead of an actual int

modern hamlet
vast lotus
somber solar
#

welp apart from #if directives

#

me omw to define errnos as inline constexpr int s

hybrid island
#

In this case I would recommend making the macro definitions first, and for the Rust FFI, you make an enum along the lines of:

typedef enum {
   ERRNO_LOOP = ELOOP,
} errno_enum_t;
somber solar
#

or write your own preprocessor that runs before rustc and replaces E* with integer literals

near tartan
#

i still define the errno values

#

for the rust impl i just wrap around this

near tartan
#

why the fuck is it so hard to safely fill a fixed buffer with string data

#

๐Ÿ˜ญ

#

bro this is lowkey impossible

#

there's no safe way of doing this???

#

which brainlet came up with this

near tartan
#

this is ridiculous

drowsy tendon
#

@near tartan would it be fine if i DMed you about the logo?

near tartan
#

ok

dense vine
#

X && X

#

i < buf.len() && i< buf.len()

#

also a for loop would be better

near tartan
#

i meant to do buf and s

near tartan
#

for loops are not supported

dense vine
#

oh, well for just i < buf.len() you could

dense vine
#

wdym?

near tartan
#

you can't do for loops

#

only while is allowed in const fns

dense vine
#

oh, didnt know that

near tartan
#

i probably didn't have to make it a const fn

#

but this is some stupid overhead i don't wanna deal with

near tartan
#

@hoary cave turns out i probably can't use traits for FileOps, because those wouldn't be dyn compatible

#

nvm i guess i can do this

#

but that's a bit yucky ngl

hoary cave
#

impl FileOps for files

#

this guy bruh

near tartan
#

whar

hoary cave
#

why are u storing a vtable manually

near tartan
#

wdymmmmm

hoary cave
#

::::::::::::::::::::::::::::::

near tartan
#

rtgrwrgtbiuurib

hoary cave
#

marvfin password generator

near tartan
#

real

#

but like

#

impl FileOps for File makes no sense

hoary cave
#

yeah no but

#

why store a vtable + inode

#

if the inode can impl FileOps

near tartan
#

it can't either????

hoary cave
#

why not

near tartan
#

because that's also controlled by the file system

#

that's what linux vfs does at least

hoary cave
#

idk pain man

near tartan
#

h

hoary cave
#

look at aero trollartistic

near tartan
#

๐Ÿ’€

hoary cave
#

or maestro

#

aero is surprisingly good

near tartan
#

maestro does it "wrong"

#

it stores the name right in the node

#

this looks like a struct dentry

eternal wharf
modern hamlet
eternal wharf
#

or look at the structure contents

modern hamlet
#

Wow look this struct is in a pub

near tartan
#

mfw

modern hamlet
#

What's an Arc?

#

/gen

near tartan
#

atomic reference counted heap alloc

modern hamlet
#

oh, i see

#

so arc is the acronym for atomic refcount

near tartan
#

yes

#

there's also Rc but that's not thread safe

eternal wharf
modern hamlet
idle flower
near tartan
#

no but i have no idea how to do it efficiently so i look for known good sources ๐Ÿคทโ€โ™‚๏ธ

#

clearly what linux does is working out

idle flower
#

Tbh I should really get to do a vfs so I can actually give out usefull responses to that topicmeme

near tartan
#

lol

#

nvm about maestro btw

#

it's just called differently

#

it has INode typedefd as a u64

#

and Node is the actual struct

#

i fucking hate github search

idle flower
#

Just an idea, what about a graph datastructure?

near tartan
#

go on?

idle flower
#

Like a simple file system is like a tree and now you throw in symbolic links and multiple mount points which allow it to loop back into itself, at that point it's just a big mess of interconnected nodes, a graph

near tartan
#

sure

#

but i mean it's not really a tree, right?

#

if you have a bunch of Arc's then it's basically a complicated linked list

#

i don't see the need for any special data structure

idle flower
#

It's a tree where each node can have a arbitrary amount of child nodes

hoary cave
#

symlinks don't point to nodes

idle flower
#

To be more specific I would say one can base a vfs on a sparse graph

hoary cave
#

symlinks just contain a "reference" to something, they don't point to the actual node

idle flower
#

What is that something?

hoary cave
#

something, or nothing

#

symlinks don't have to point to anything

near tartan
#

symbolic links are just path names, right?

vast lotus
#

symlinks are just blobs that are interpreted as paths by the os

hoary cave
#

yes

#

symlinks are regular files with a different file type

#

they don't point to anything themselves

#

your kernel decides what they point to

#

but the target node is not part of the actual vfs "graph"

#

and hard links are pointers to other nodes

#

but i don't think they allow you to create invalid cycles

near tartan
#

eh should be easy enough

#

i'll just implement something that """works""" and fix it later

idle flower
near tartan
#

ok path traversal already kinda works

dense vine
near tartan
#

how so

dense vine
#

this is my weird vfsnode trait

near tartan
#

a File is an open file descriptor

dense vine
near tartan
#

because these functions are different depending on the underlying file system

dense vine
#

oh actually that makes sense

idle flower
#

Maybe use a Filesystem abstraction layer?

near tartan
idle flower
#

how is that different between two filesystems?

near tartan
#

what?

dense vine
near tartan
#

ext2 does different things on write than tmpfs

#

isn't this super obvious?

#

by putting the functions in the File struct i can store an array off File's without having to care about what file system they belong to

#

i just have to call the trait impl

idle flower
#

but they do have the same functions signature?

near tartan
#

why does that matter

#

the implementation is different

#

that's the whole point of an abstraction

idle flower
#

tbh i still dont get it how they are so different from the vfs level, sure what a fs does internaly may differe but shouldnt the interface be constant across different fs?

near tartan
#

yes?

#

i really don't get the confusion

#

the interface is the same

#

what makes you think that it isn't?

#

@idle flower

idle flower
#

what i wonder is why you dont just store like a trait fs_api object in the node, where the object represents a fs instance, that way the vfs trait doesnt need to be dyn

#

or do i miss something?

near tartan
#

that would still have to be dyn

#

File is the abstract struct that represents an open file

#

because it's abstract, the compiler cannot know what to do when you call e.g. File::read

#

therefore the trait has to be a dyn

#

per definition

#

since it's essentially a vtable

idle flower
#

yea

near tartan
#

insane call stack

pine rock
#

how do you cope with the vscode debugger

near tartan
#

wdym

pine rock
#

i've always found it to be really painful to use

near tartan
#

i have a debugging config

pine rock
#

it does all kinds of stupid shit at least when using gdb

near tartan
pine rock
#

ah maybe lldb is less ass

near tartan
#

use codelldb

magic charm
#

i've heard rust didn't bother writing a debugger

near tartan
#

you can just use lldb

magic charm
#

that they just use gdb as if for c

pine rock
#

i dont suppose lldb has the same plugin api as gdb does

#

i have alot of gdb scripts that i dont particularly want to port lol

near tartan
#

why do you need scripts

pine rock
#

printing custom data structures, dumping hardware state, that sort of stuff

near tartan
#

ah

#

there's this but definitely not the same api

#

rn i'm trying to figure out why openrc doesn't give me a /usr/sbin/init

pine rock
#

i like the gdb commandline its just the vscode integration thats ass

#

you need to put -exec before every command you type

#

this is by design because whoever made the extension is a retard

near tartan
#

the lldb extension is much better

pine rock
#

vscode also tells gdb it supports ansi colour escapes

#

but then

near tartan
#

kek

#

they really gated openrc-init behind linux

#

wtf

#

but then they still have ifdefs?

pine rock
#

host_machine.system() doesnt even return Linux it returns linux

#

bruh

near tartan
pine rock
#

incredible

near tartan
#

you have to manually provide -Dos

pine rock
#

leave it up to big projects to fuck up the most basic build configuration

near tartan
#

i think this is on purpose

#

maybe because BSDs have their own init?

#

and openrc runs on top?

pine rock
#

the symlink and get_option('sysvinit') stuff makes sense

#

but needing to provide the os as a build flag is dumb

near tartan
#

why

#

oh yea you can do cross files

#

maybe they're actually braindead

pine rock
#

at the very least ```py
target = get_option('override_target')
if target == ''
target = host_machine.system()
endif

near tartan
#

oh i see why they gate it behind linux

#

duh

#

reboot is a linux syscall

#

makes sens

#

easy fix, i'll just patch menix in

#

i have linux reboot anyways

#

i guess not

#

AAAAAA this is an mlibc issue

#

noooo

#

don't tell me i didn't push that

#

fuck

near tartan
#

wait no

#

it exists

#

did i forget to fucking install it?

#

i did

#

๐Ÿ’”

near tartan
#

much โค๏ธ

near tartan
#

damn mlibc doesn't implement updwtmp yet

#

time to do that then

#

i could also just stub it out smh

cyan nexus
#

Does mlibc have rcu libs?

near tartan
#

the what

cyan nexus
#

The rcu library that stdlib has

near tartan
#

which stdlib

cyan nexus
#

Gcc

#

For c++23

near tartan
#

no

#

why c++?

#

mlibc is a c library?

cyan nexus
#

I think c has rcu too

#

I don't remember

near tartan
#

rcu is c++26 btw

cyan nexus
#

Ah

#

I tought it was 23

near tartan
#

then again, mlibc is not a c++ stdlib

cyan nexus
#

Lol

near tartan
#

and C does not have rcu in libc

pine rock
#

no stdlib has implemented rcu or hazptrs yet

#

theres implementations of both from the paper authors on github if you need them

#

folly has good implementations of both as well

storm bobcat
near tartan
#

dentry cache time

near tartan
#

tyy

#

im gonna implement some linux function

#

idr what its called exactly

#

something with updw

#

updwtmp

#

that's it

#

idk what it does yet but it's missing

#

and openrc wants it

near tartan
#

i love osdev

near tartan
#

more vfs stuff today

near tartan
#

in the next few weeks I'll be building my dual epyc server to optimize my smp logic

#

(I'll probably have to touch the allocator again for cpu local regions)

prime seal
near tartan
#

depends

#

some structs are slightly different

#

but new features are usually kept similar to musl

#

and mlibc has glibc extensions

near tartan
#

motherboard has been obtained

eternal wharf
#

sp3 mobo ๐Ÿ’€

near tartan
#

๐Ÿ’€

pine rock
near tartan
pine rock
#

damn thats a good deal

near tartan
#

it is

#

i actually only paid 225

#

with coupon codes

idle flower
idle flower
near tartan
#

but yes

#

basically a PC

idle flower
#

So no psu redundancy?

near tartan
#

ye

#

i can't afford that anyways

#

if you wanted redundant psus you need to buy a server blade

idle flower
#

You mean rack server or actual blade servers?

near tartan
#

either

#

there's 4x blades from Lenovo

#

but there's also 19" servers

idle flower
#

I have one Dell r730xd at home, loud af

near tartan
#

that's why i didnt go for a server

#

im building a rack mount pc basically

#

because sp3 is physically compatible with tr4

#

i can just use off the shelf threadripper AIOs

idle flower
#

Nice

#

Wtf is TruDDR4? Just saw that in the specs for Lenovo servers

#

It's always a good sign if there isn't a price tag on the product page

idle flower
#

You gave me the idea to look for blade servers, the amd ones would outperform even my server but at the same time burn a hole in my pocketmeme

near tartan
#

yes

#

they will also deafen you

#

because they're basically small 1u servers

near tartan
#

@nocturne rampart i got more vfs shit done now

#

i think i might actually get it working today

nocturne rampart
#

Nice

near tartan
#

yippee

somber solar
#

๏ฟฝ๏ฟฝ

near tartan
#

yes

somber solar
#

??

near tartan
#

i handle non-utf8 characters

#

because rust's String only accepts valid utf-8 strings

#

but vfs entries could be non-utf-8

#

makes things a tad bit more complex but whatever

eternal wharf
#

doesn't rust have a path type for that?

near tartan
#

meme guess why i'm not using it

somber solar
#

I'm guessing you can't use it with no_std

near tartan
#

ding ding ding

somber solar
#

what do I get

eternal wharf
#

but you can use rust strings?

near tartan
#

yes

#

strings are part of alloc

#

ospath requires host support

#

because its definition/behavior is os dependent

eternal wharf
#

makes sense ig

#

i assume you're just storing it as u8 slices then?

near tartan
#

Vec<u8>

eternal wharf
#

close enough

near tartan
#

and for display i convert into a lossy string

#

which replaces invalid utf-8 chars with the replacement char

eternal wharf
#

makes sense

near tartan
#

the last part that i'm struggling with is mounting

#

idk how that interface should look like

#

like from an API standpoint

hybrid island
# near tartan the last part that i'm struggling with is mounting

BadgerOS on this topic:

// Try to mount a filesystem.
// Some filesystems (like RAMFS) do not use a block device, for which `media` must be NULL.
// Filesystems which do use a block device can often be automatically detected.
errno_t fs_mount(char const *type, fs_media_t *media, file_t at, char const *path, size_t path_len, mountflags_t flags);
#

I haven't done this yet but it'll determine the actual path by means similar to calling realpath

#

That makes the mounting have an API pattern basically the same as the other filesystem functions; at + path vector (bytes) identifies location.

near tartan
#

how do you keep track of mounts

hybrid island
#

Doesn't support stuff like mount namespaces nor stacked mounts in this model but it keeps it nice and simple.

#

That last one is useful for having one overview of all mounts but isn't necessary for this model to work.

distant cypress
#

a nice way to represent mounts in the VFS is to let paths be a ยซmount, namecache entryยป-tuple.

pseudocode for mounting a filesystem:

errno_t
do_mount (struct path mountpoint, struct namecache_entry *fs_root)
{
        struct vfs_mount *newmount = malloc(sizeof(*newmount));
        newmount->parent_mount = mountpoint.mount;
        newmount->parent_nc_entry = mountpoint.nc_entry;
        newmount->root = fs_root;
        // initialize other fields...
        // I guess also properly refcount parent_mount, parent_nc_entry, root
        // and obviously do proper locking too
        mountpoint.nc_entry->child_mounts.insert (newmount);
}

pseudocode for entering mounts:

void
enter_mount (struct path &path)
{
again:  // again, do proper locking and refcounting
        vfs_mount *current = path.mount;
        for (vfs_mount *child : path.nc_entry->child_mounts) {
                if (child->parent_mount == current) {
                        path.mount = child;
                        path.nc_entry = child->root;
                        goto again;
                }
        }
}
#

purely from an API standpoint, these internal details of filesystem mounts shouldn't be relevant, though.

distant cypress
#

also, walking .. in such a system might look something like the following:

void
walk_parent (struct path &path)
{
        // again, do proper locking and refcounting instead of this shite.
        while (path.mount->root == path.nc_entry) {
                path.nc_entry = path.mount->parent_nc_entry;
                path.mount = mount->parent_mount;
        }
        path.nc_entry = path.nc_entry->parent;
}
hybrid island
#

I just realized I was doing this pattern without noticing

distant cypress
#

mount namespace-like functionality can be implemented with some extra ifs in enter_mount and walk_parent, probably

#

something like

if (child->parent_mount == current && child->mount_ns == current->mount_ns) {
        path.mount = child;
        path.nc_entry = child->root;
        goto again;
}
hybrid island
#

FWIW BadgerOS makes a clear separation between the filesystem code and individual implementations. Said implementations only ever deal with calls qualified by either a file handle or a dir handle + name combo. The generic filesystem code deals with walking the filesystem and related while individual implementations need only worry about FS specifics. That makes the actual FS-specific code only see the media and flags like so:

// Try to mount a FAT filesystem.
errno_t fs_fat_mount(vfs_t *vfs) {
#

But you probably already do this.

#

(media and flags stored in the vfs_t struct)

distant cypress
distant cypress
hybrid island
#

Fair

#

I don't actually have that global collection yet and the vfs_t already stores the mount path so...

near tartan
#

hm

#

now i need to see how i can translate this to rust

hybrid island
#

Is your current code up?

near tartan
#

the old version didn't have mounts so I've no idea what I'm actually doing

hybrid island
#

I can probably help translate

near tartan
hybrid island
#

๐Ÿ‘

near tartan
#

but it's nowhere near complete

#

you wanna look in kernel/src/generic/posix/vfs

hybrid island
#

Nitpick: BadgerOS supports un- and re-mounting root, so using Once may not be needed.

near tartan
#

wdym

hybrid island
#

in mod.rs

near tartan
#

no

#

once just means that it's uninitialized until it is explicitly set

#

it's kinda needed because Arc cannot be stored statically

#

since that needs a heap allocation

#

i could store it as an option

hybrid island
#

I'd just use an Option<Arc<...>>

near tartan
#

yea fair

#

if you don't mind, note down all the things that you can find so i can look into it in the morning

hybrid island
#

Will do

hybrid island
#

Proposed changes:

  1. Consider using Option<Arc<...>> for the root filesystem
  2. File currently points to a FileOps, Inode and position. This is different from BadgerOS' model where the equivalent vfs_file_desc_t only stores offset and a reference to vfs_file_obj_t, the latter being what actually specifies the file operations.
  3. Assuming suggestion 2 is implemented, you should have File call read and write with absolute positions on INode using shareable mutexes where the mutex is taken shared when reading/writing, but exclusive when the file is resized. This improves performance for concurrent accesses to one inode and is POSIX-compliant.
  4. Consider making a relation between INode and the VFS (I think you use SuperBlock for this) for non-root mounted filesystems.
  5. Consider renaming SuperBlock to something that indicates it is a mounted filesystem, e.g. MountedFS and FileSystem to something like FSDriver.

Proposed additions:

  1. BadgerOS implements stuff like /dev's files by creating facilities for special types of device files, which are supported by BadgerOS' RAMFS so that devices have a very simple API to register their device files: errno_t fs_mkdevfile(file_t at, char const *path, size_t path_len, devfile_t devfile);.
  2. As per david's suggestion, consider having a collection of all mounted filesystems in a central location.
  3. While BadgerOS does not yet have file caches, consider making those at the INode layer (normal files+dirs only of course) on the generic side so it's not the VFS's problem to deal with caches.
    -# more incoming
#

Separation of concerns:

BadgerOS separates the following concerns each into their own file or set of files:

#

All of this structure comes from quite some time of me of figuring out how best to structure this all. I hope this can be of any help! If you more need help you can always ask me a question :3

near tartan
#

regarding /dev, ill have a devtmpfs that handles this

hybrid island
#

That's mostly an implementation detail but BadgerOS' way of dealing with it is more flexible than being restricted to just one implementation that can hold device files, should I ever choose to do something of the sort.

#

But like I said implementation details. Not terribly consequential compared to the other points.

near tartan
#

elaborate on point 2 of the proposed changes please

hybrid island
# near tartan elaborate on point 2 of the proposed changes please

Basically you have the "file operations" tied to your File structs whereas BadgerOS only does this for FIFOs and devfiles, neither of which have offsets. The regular files (and also reading symlinks for example) go through the equivalent of your INode struct instead, where the mounted filesystem is asked to do an operation directly on the inode.

#

Note: For file-level caching, the cache would be in the inode and the filesystem implementation would not see the effect of a write until synced from this cache, which sends just the one write to sync it.

distant cypress
# hybrid island # Proposed changes: 1. Consider using `Option<Arc<...>>` for the root filesystem...

RE: proposed change 2

Assuming that File is a file descriptor object, it should be noted that Files can be acquired from other sources than the VFS, e.g. via socket(), connect(), etc.. These don't really have underlying filesystem objects (iiuc Linux uses a "fake" underlying struct inode for these), and so it makes sense to have FileOps such as read, write be separate from the INode. It should also be noted that this comes at no additional indirect call overhead vs. putting file operations like read/write in the INode, because you'd still do (at least) one call via function pointer in either case. Then there's also the case of open on a char device or block device. These have real underlying inodes that are part of a real filesystem, but they dispatch into something else (a device driver or a block storage driver).

storm bobcat
near tartan
#

it's more ergonomic to do that

idle flower
#

or dont use the concept that everything is a file desciptor and go for something like everything is a resource handle and a file descriptor is just a specifc type of resource handle

vast lotus
#

that's what I do but you still need to be able to decouple file ops from inode ops to implement POSIX properly

idle flower
#

like most io boils down to control io and bidirectional data stream, the data stream is basicly the same but control io is vastly different

vast lotus
#

e.g. if you open a named pipe you should get a file description but no operations on that file description should involve the actual filesystem you opened it from

idle flower
#

thb i wouldnt call such a descriptor a file discriptor from the os perspective as it will inevitably lead to confusion, how posixs and other api's actually represent that descriptor is their problem

near tartan
#

I'll look into it now in like an hour or two

#

im definitely gonna keep the file ops as part of the file struct

idle flower
# near tartan what's the difference?

a resource handle would represent a generic handle and specific handles such as network and filesystem would build on top of it, so its mostly a high level differentiation

near tartan
#

I'm having a hard time with modeling this in my head

#

any concrete pseudo code

idle flower
#

sady no, im not in userspace yet

#

for example in linux everything is a file descriptor.
That means that i could do file operations on something that isnt a file, like a usb device or network connection

#

thats not sane and if you explicitly create different handles (where each one has its own set of possible operations) its clearly visible what you get and how to use it

near tartan
#

so how do i abstract that

idle flower
#

it would be like rusts type system a compiler thing

near tartan
#

the resource still needs a reference to the file ops to act as an fd

#

right?

idle flower
#

technically yes

near tartan
#

hm

idle flower
#

but only in the code where the file ops is actually called

near tartan
#

well yes, but that would essentially mean that for file ops my resource is just an fd called resource

#

i dont get the semantical difference

idle flower
#

well the fd would implement the resource handle trait for example, which defines operations that dont care about the specific type/are generic over all types (like close or get type)

near tartan
#

hm

#

can you write a really simple struct how that looks?

#

I'm so lost

idle flower
#
trait ResourceHandle: Drop + Clone + Send + Sync {
    fn close(self);
    fn get_type(&self);
}

struct FileDescriptor {}

impl ResourceHandle for FileDescriptor {
    fn close(self) {
        // Close the file descriptor
    }

    fn get_type(&self) {
        // Return the type of resource
    }
    
}
near tartan
#

surely there's more common functionality than close

idle flower
#

probably, its just an example

vast lotus
#

I use this system in hydrogen, and yeah pretty much the only necessary common functionality is close (or, more accurately, free() when the last handle to an object is closed)

#

Although I do have some more stuff in the generic object vtable for event queues

near tartan
#

hm

vast lotus
#

This is the common stuff

near tartan
#

i could implement this in drop()

#

in rust i can have an external refcount

#

i guess i can have a dynamic drop impl

#

how do you use object_t?

vast lotus
#

this is my file description struct as an example

near tartan
#

like, how do you go from object_t to file ops

#

ah

vast lotus
#

ops is a pointer to <file_ops_t>.base

#

verifying the object type is done during handle -> object resolution

near tartan
#

hm that's going to be hard to replicate in rust

#

at first glance

vast lotus
#

yeah idk what the best way to model it with structs+traits would be

#

it's kinda meant to mimic a proper inheritance tree

#

except without language support

near tartan
#

yea no inheritance in rust sadly

#

i guess i can have something like

struct Object {
    ops: Box<dyn ObjectOps>,
}

trait ObjectOps {
    fn drop_obj(&self) {}
    fn get(&self) -> Arc<Object>;
}

struct File {
    base: Object,
    inode: Arc<INode>,
}

impl ObjectOps for File {
    fn get(&self) -> Arc<Object> {
        self.base
    }
}
#

then i don't need to mess with ordering guarantees

storm bobcat
#

no that kinda sucks, make use of the cool rust type system instead

#

we do the proxima way in dufay and it'd be much better with e.g actual classes or type traits

near tartan
#

hm

near tartan
#

i can't really have an enum because that can't be extended with modules

#

and traits are just vtables

hybrid island
#

Traits can inherit from one another though

near tartan
#

i can just have a resource trait

#

with no extra fields

#

then impl Resource for File

storm bobcat
#

Yea that's good but add a refcount maybe

near tartan
#

is that needed?

#

most objects are inherently Arc'd

#

like files

storm bobcat
#

Then it's fine

#

Might be easier to arc it instead of having a refcount

near tartan
#

yea

#

thanks to rust the lifetime is automatically handled in 99% of cases

storm bobcat
#

Do that, have methods for resources

#

Like read write map

#

open

#

ioctl

near tartan
#

true

#

the more i think about this i think i actually did that in the old version ๐Ÿ˜ญ

storm bobcat
#

If it's not supported by that resource just return enoimpl or something

#

enosys

near tartan
#

yea i'll just do EResult<()>

magic charm
#

you can read() and write() a TCP socket fine

#

every real OS is having this

near tartan
#

the resource idea was actually really good

#

that allows me to work with a single trait

#

is there anything i need besides r/w, seek and ioctl?

somber solar
near tartan
#

oh true

somber solar
#

I'm guessing your resource is like the one in vinix

near tartan
#

idk if seeking on a resource makes sense

#

probably not

idle flower
magic charm
modern hamlet
#

Probably only for devices that you could actually mmap like frame buffer devices

#

But the rest you can simply fill in pages by reading normally and memcpying into the destination page (and into your page cache if you have it)

somber solar
modern hamlet
#

but the file system shouldnt provide its own implementation of mmap

#

unless you REALLY want to repeat code

#

instead, let the OS handle it, UNLESS the file is SPECIFICALLY allowed to provide its own physical pages

#

(such as an fbdev)

somber solar
modern hamlet
#
void mmap(...)
{
    if (file->can_provide_physical_pages())
        mmap_those_physical_pages()
    else
        generic_mmap_commit_impl()
}```
somber solar
modern hamlet
#

boron has (will have) a BACKING MEM method which does exactly this

#

BSTATUS(*IO_BACKING_MEM_METHOD)(PIO_STATUS_BLOCK Iosb, PFCB Fcb)

#

(FCB is the equivalent of Vnode)

vast lotus
#

and regular_file_mmap maps the page cache for the vnode (which is the only part the fs controls)

modern hamlet
#

thats pseudocode

#

and i have a similar struct

vast lotus
modern hamlet
#

tbh all it does is save one branch

#

compared to my method

vast lotus
#

i mean yeah it's not for the sake of performance or something like that i just think it's cleaner

modern hamlet
#

I suppose that works as well

#

Honestly my thing might be a bit less restrictive when it comes to mapping files that arent "regular files"

#

Since IO_BACKING_MEM_METHOD would just return the starting physical address of the device's exposed mappings

vast lotus
#

for that i just use a device-specific memory object

#

for example /dev/mem has a memory object that, when mapped, maps the portions of the mapping that aren't kernel-owned ram, and returns ENXIO when something tries to access an unmapped area

modern hamlet
near tartan
#

linux lets you do funny stuff with that

vast lotus
#

hydrogen isn't quite a microkernel because it does all the non-hw-specific stuff in kernel (vfs, processes, but also specific filesystems), but actual hw drivers run in userspace

#

also when doing this i am extremely careful to restrict access to actual ram that's owned by the kernel and not the firmware: doing so is only allowed through an fd that you get with an ioctl that allocates physical memory

near tartan
#

ah i don't think I'll be doing that so I'm not sure how useful it is for me

vast lotus
#

linux also doesn't do that and it still has /dev/mem (though similarly restricted: you can't access kernel-owned ram at all unless you configure the kernel to allow it, otherwise it just zero-fills the buffer) so it must be useful for something

near tartan
#

we at suse recently got rid of it because of security concerns

#

we were able to change uname to output windows

vast lotus
#

lol what

near tartan
#

hold on let me try to find it

#

also note the awesome riscix arch

vast lotus
#

thats awesome

near tartan
#

because there were so many issues with the phmem module

#

we just got rid of it completely

vast lotus
#

that's fair

near tartan
#

also this

#

we were able to modify drm control structures

#

and we could crash the entire system by writing garbage to hpet and lapic mmio regs

vast lotus
#

i mean yeah being able to write to hardware registers in mmio through /dev/mem is to be expected

near tartan
#

i think the worst part was that this could work without root

vast lotus
#

wait what

near tartan
#

yea

vast lotus
#

wouldn't that require an exploit to bypass linux's regular file access checks

#

or do you mean by root chowning/chmodding it

near tartan
#

exploit

#

the phmem shit is completely broken

vast lotus
#

yeah but i'd expect that part to be part of the generic vfs code

#

like if non-root tries to open a root-owned 0600 file the driver code should never even be entered right

near tartan
#

ah nvm the base operation does require root

vast lotus
#

thank fuck

near tartan
#

ah i remember what it is

#

you can do it with secure boot enabled

#

you can just ioremap random structures

#

that's still pretty bad

#

but yeah, otherwise i don't see the point in exposing phys mem to userspace

near tartan
#

@vast lotus what is the event in struct object for?

vast lotus
#

you mean event_add, event_get, event_del?

near tartan
#

yep

vast lotus
#

those are for my event queue support, it's similar to kqueue and it's what i use to implement poll/select

#

event_add validates the event type and registers the queue to listen to the given event, event_get gets some more data about the event and returns whether it's still pending, and event_del unregisters the event

near tartan
#

and the object itself doesn't store anything regarding events?

vast lotus
#

kinda? for each event type there's an event_source_t somewhere, which maintains a list of listening queues

#

but event_add is responsible for finding that

#

it's not in object_t itself because objects can support multiple event types and each event type needs its own event_source_t (and the event sources might not be per-object, like with fifos, where they're per-inode instead of per-file-description)

#

for example this is where they are for fifos and pipes (this is per-inode data)

near tartan
#

right

near tartan
#

i finally got around to adding a stack trace

#

idk if i want to bother with demangling

grave peak
#

Reason 92882 not to rust

near tartan
#

you misspelled c++

grave peak
#

Oh?

near tartan
grave peak
#

Isn't this rust

near tartan
#

no

grave peak
#

Why does it say rust

near tartan
#

it was a joke ๐Ÿ˜ญ

grave peak
#

Lol

near tartan
#

but yea c++ also has the mangling problem

#

demangling should not be too hard

grave peak