Managarm and related projects

[trim eagle]

Oh yeah

Porting strace should be easy

Getting it working is where the pain is at

[vestal sapphire]

I mean for strace we'd just need to impl some hooks that request handlers could call to fill in some info

but some people hated that ¯_(ツ)_/¯

[lament grove]

LMAO I just looked at your RISC-V impl and you just disregard paging entirely

[trim eagle]

RISC-V is a stub

AArch64 can boot into weston on supported platforms

AMD64 can, without much issues, run the shit I flex with here

[lament grove]

I can tell

[wild stone]

i still want to work on the risc-v impl at some point but uhh time + brain

[lament grove]

If I didn't already have too many projects :P

[wild stone]

basically same

A-GPS and ubuntu touch things and other stuff and even more stuff and 3d printing things

[lament grove]

Ah yes

I have 1 OS project, 1 firmware project, 1 CPU project (all three of which I organise) and then some

[trim eagle]

For me, real life is a bitch

[wild stone]

i have some A-GPS/SUPL things going on which are stupid enough that i looked at actual legal insurance

[trim eagle]

This KDE meme I was working on today was this quick as I already had like 95 percent laying around. Just had to upgrade it to 6.3.0 over 6.0.0

[lament grove]

It's amazing how little code you need to run a demo in comparison to how much you need to a minimum viable product

[carmine current]

ah yes, hate because managarm does stuff that actually works

even if you disregard all the stubbed stuff, managarm still does more than 99% of all hobby OSes 😉

[fervent zodiac]

i mean sure

but 99% of hobby oses are hello worlds

[trim eagle]

even disregarding hello worlds

[fervent zodiac]

fair

[trim eagle]

speaking of new stuff

do see a small issue with bash launching here

[upbeat smelt]

tomorrow theyre gonna release the 24.05.2 tarballs

[trim eagle]

of course they do

[quaint fern]

thor appears to use a hash table to store the map of handle numbers to object descriptors - why?

(not a bad why just a curious why)

it seems to date back to very early days, even to the 19th commit to managarm back in 2015

[upbeat smelt]

at a guess because we just allocate handles via an incrementing counter

without reusing them

yep ```cpp
Handle Universe::attachDescriptor(Guard &guard, AnyDescriptor descriptor) {
assert(guard.protects(&lock));

    Handle handle = _nextHandle++;
    _descriptorMap.insert(handle, std::move(descriptor));
    return handle;

}```

[quaint fern]

this touches a bit on what i was thinking earlier in my thread (as goes: if a handle namespace doesn't have to conform to posix requirements to do first-free allocations, you can make handle management simpler and more scalable; i have no good answer to how you could scale handle allocation if you make a first-free guarantee)

but what terrible retribution might you bring down upon your heads if someone allocates about 4 billion handles a second for about a hundred years in one process!

[upbeat smelt]

handles are signed 64-bit

so you'd need 2^63 handles

[quaint fern]

50 years then

[lament grove]

Because I am a nerd, I calculated it's actually 68 years at that rate

[cinder trench]

managarm has the best hobby kernel

For now.

[fervent zodiac]

the managarm kernel doesnt do all that much though

[forest bear]

That's kind of the point

[fervent zodiac]

and dont they still have broken oom handling

[native prairie]

You mean none or asserts

[trim eagle]

Yeah we still need to bubble that up

That might be something I can tackle

[trim eagle]

Video player acquired

[dusk crystal]

nice

[dense umbra]

now add audio

gj tho!

[native prairie]

Is this on real hw?

[trim eagle]

nope

qemu

[sly turret]

how common is ppl misreading/misspelling this project's name as "managram"?

[cedar cosmos]

181 results here

[shrewd grove]

you never get it wrong if you saw the cppcon talk on it

[worn rampart]

Me every time I search for it on Google, don't find it, and remember it's spelled differently

[spring sentinel]

is it publicly available anywhere? I‘d watch it in the background

[sly turret]

its on youtube

[trim eagle]

Certainly. It’s on YouTube

[sly turret]

I watched it

[spring sentinel]

found it, ty

[trim eagle]

https://youtu.be/BzwpdOpNFpQ?si=tRZFARp3sWTqSbZK

[spring sentinel]

Some people really don’t appreciate others making stuff

just don’t comment at all if you’re not interested

[shrewd grove]

90% of your c++ code should be coroutines... otherwise you're likely doing something strange

love it

[quaint fern]

1 and 2 are obvious misunderstandings (every kernel is "asynchronous" to the extent that interrupts are, no man has ever denied this, but that's obviously not what is meant by 'asynchronous' in managarm), 3 is a hostile question, and 4 is silly

i don't believe cppcon is usually a place where proposed solutions to the hard problems of computer science are presented

[shrewd grove]

not at all, its just c++ self indulgence

[spring sentinel]

i think some people just feel good hating on others, no matter what they do

it may not be a groundbreaking revolutionary kernel but can someone not just do what their interested in

[odd schooner]

this guy is both pretentious and clueless

why the fuck are you looking for number 4 in cppcon!?

number 1, 2, and 3 would probably get answered by watching the fucking talk

[lament grove]

These kind of people do nothing but complain on the internet all day about things that don't even affect them.

Best to just ignore it.

[storm plover]

Zelensky was so wide the music failed to play

[trim eagle]

we don't do audio yet so that's intended

[storm plover]

yeah ofc

Good job!

I really want to have time to take a look at Managarm

[graceful temple]

https://tenor.com/view/megamind-gif-26070434

no usb audio driver?

[trim eagle]

Not yet no

[vestal sapphire]

The USB audio driver is probably the first one we'd implement tho

Just yet PCM at the endpoint lol

(let's ignore the part with isochronous transfers)

[trim eagle]

True

[coarse rune]

Just plug it in? so overcomplicated 🙄

[storm plover]

just use AI? so overcomplicated 🙄

[civic vortex]

just sing the music yourself? so overcomplicated 🙄

[trim eagle]

couldn't be left behind in the multi user / cooked mode terminal department

so Managarm has proper user login now (PRs pending)

now we just need to add checks everywhere

[worn rampart]

Managarm is managram in my heart

[silent oak]

I should make my own fork of Managarm called Managram

[median wharf]

we'll make our own managarm with blackjack and hookers

[storm plover]

anyone said gambling?

[trim eagle]

You mean this one?

Are those hookers cancellable tho?

[median wharf]

no :(

[jaunty charm]

Someone mention hookers?

[wary wing]

mammogram

cc @vestal sapphire

[vestal sapphire]

no lapfed

no

[wary wing]

[dense umbra]

https://github.com/freebsd/freebsd-ports/tree/main/games/xblackjack

[median wharf]

perfect

[trim eagle]

Bookmarking for further use

[trim eagle]

finally starting to implement permission checking

[native prairie]

nice

[nocturne tide]

why is it owned by root

[trim eagle]

small issue of the file creation kind I guess

[nocturne tide]

cool stuff though

ah fair

[trim eagle]

let's see

[trim eagle]

fixed that, but now chown is kinda not working

[trim eagle]

chown is a stub

so that makes sense

[lament grove]

lmao default perms 666, WCGW

[trim eagle]

Umask is a stub

[lament grove]

Much to do still I see?

[trim eagle]

The entire permission stuff is kinda wack. I did do some checking for setuid and friends when I implemented that

But file perms are, not really adhered I guess

[trim eagle]

So yes, always

[lament grove]

Like all good projects, the work never ends :P

[dense umbra]

it is rewarding work in the end

[trim eagle]

Sure is

I plan more work between 11:00 and like 15:00 (maybe later) CEST related to permissions

Ideally I’d get proper perms working on tmpfs that day

Possible bonus being chown

[kind seal]

I am always deeply amused by the things managarm lacks while it has a working webkit browser

[upbeat smelt]

any% speedrun

[trim eagle]

It’s a funny thing yes

[vestal sapphire]

We had no network drivers other than virtio for the longest of times

[nocturne tide]

i mean, realistically, thats all you need to test most of the stack

[vestal sapphire]

Yes but you can't use it outside of a VM

[nocturne tide]

yeah, but you dont need to in order to get a working browser :^)

[vestal sapphire]

Now, you can run most USB-Ethernet adapters, Realtek NICs, and MBIM cellular moderns (while Intel NIC drivers are an open PR)

[nocturne tide]

i dont even plan on testing my shit on real hardware until i am considerably far into development

like unti l i have xorg running or whatever

[fierce mist]

But its better to test often to find bugs earlier rather than later

[shrewd grove]

If I may (lol), do it early and often.

[native prairie]

ever get the desire to rewrite managarm to be PhD free and monolithic

[dense umbra]

I'll rewrite managarm to be degree free

[safe glen]

yeah that's what Lyre was (rest in peace)

but tbh a PhD free monolithic managarm would not be managarm

managarm's identity is "pragmatic microkernel written by a PhD"

[native prairie]

and your new secret kernel also?

[safe glen]

yes

[trim eagle]

You’d need to ask @carmine current

[native prairie]

nice

[safe glen]

i am actually still working on my secret kernel

[native prairie]

when is it going to be unveiled

[safe glen]

when it's Vinix tier in terms of completeness but also stable

[native prairie]

damn

[trim eagle]

When udev

[safe glen]

why would i?

udev is a linux thing

[trim eagle]

I’m joking

[native prairie]

drm

[safe glen]

that, maybe

[trim eagle]

I know you won’t do udev and the other highly Linux specific interfaces

[safe glen]

yeah because it's stupid

it's the uncanny valley between a unix-like OS and a linux reimplementation

Maestro is on the reimplementation side

Ironclad, Astral, Lyre, Vinix, secretOS are on the unix-like side

managarm is in the uncanny valley

[trim eagle]

I guess that’s almost bound to be if you start implementing Linux interfaces but you’re fundamentally different than Linux (microkernel)

[safe glen]

you like porting things to see managarm become a usable OS

the microkernel PhD asynchronous IO whatever part of managarm doesn't even enter the equation

it's like a black box in the end

without using native interfaces

[dense umbra]

the only part of managarm I ever dare to touch is mlibc, anything beyond that is magic and my tiny brain wont handle

for I trust the phd gods

[safe glen]

mlibc is hardly "part of managarm" as i said before

like sure, perhaps it de jure is

but it is de facto its own project

[wary wing]

uhuh

[dense umbra]

hows the secret mintsukios going

[nocturne tide]

lapfed trying to be a smartass like always

[safe glen]

going

[dense umbra]

nice

[safe glen]

yeah and failing like always

given i did say mlibc is a managarm part de jure

[carmine current]

Mint i think you're overestimating the complexity of managarm lol

In the end it's just another kernel

Some parts are a bit tricky but being tricky is not a goal in itself it's sometimes just necessary for perf or whatever

And we usually try to abstract away the tricky parts

like, nobody is using hel IPC syscalls directly, we're always using the more high level helix:: stuff

[nocturne tide]

i did say "trying", thought that failure would be implied :')

[wary wing]

I was just joking lol

[nocturne tide]

what's the joke

[quaint fern]

it's a language thing

managarm is militantly C++ and that makes it inscrutable to those who are not fluent in C++

[safe glen]

^ this

this too

[wild stone]

-# i fail to see the issue here

[full slate]

i mean the complexity is there for a reason

eg you could write the equivalent coroutines in C as a manual state machine but that sucks

[quaint fern]

it goes without saying, the idea of managarm in C is unimaginable

and when you already all but need co-routines from C++, a brand new feature, why would you ever shy away from templates and r-value references and other C++ stuff?

managarm's distinguishing characteristics are after all:

1. a pragmatic microkernel,
2. written in modern C++,
3. profoundly asynchronously

[full slate]

I'm not really sure what the argument is over tbh

[quaint fern]

i wasn't aware there was one

i thought i was just noting that managarm is a c++ project and not a c project and a reasonable working knowledge of modern c++2b or whatever it's called now is necessary

[upbeat smelt]

we're mostly on c++20 since many 23 things would require upgrading to gcc 14

and i think it might be worth it to just wait it out until gcc 15 releases

[quaint fern]

that C++ is mostly a superset of C i think gives people false expectations about how much they will be able to work with C++ projects

[carmine current]

Is there any reason not to just upgrade to gcc 14?

[vestal sapphire]

effort

[trim eagle]

No real need for any of those features either yet right?

Usually need drives toolchain upgrades

[carmine current]

Is it really a lot of effort to rebase the patches?

[trim eagle]

Can be. Depends on the changes and how long ago it was last done. And it’s time one usually has limited amounts of, it’s 4 gcc rebuilds after all

[worn rampart]

What patches do you use?

[median wharf]

it's really not that bad, i think the idea "managarm is extremely complicated" is sort of a myth

the managarm paper actually does a really good job explaining all of the "magic" subjects

[carmine current]

patches to enable managarm and mlibc targets

[worn rampart]

Ah, makes sense

[upbeat smelt]

i generally agree that it's not all that complicated but saying "read the paper" to understand how it works kinda feels like the opposite of simple

although the paper itself is not complicated either, it just sounds funny to me

[safe glen]

fwiw Vinix/Gloire/Lyre etc have gcc 14

the fact that managarm now uses a special repo for gcc makes it very not obvious what the necessary patches for it are

[dense umbra]

@trim eagle I wonder how faster you could get factorio to run with llvmpipe if it used smp multicore

IIRC it calls sysconf to find the number of processors and then spawns threads for it

[trim eagle]

I mean that’s an easy test

[native prairie]

probably not fast at all due to ipc issues

[trim eagle]

Change the hardcoded 1 to whatever you pass to qemu

[native prairie]

looking at glxgears

[trim eagle]

(Yes proper solutions are in the works)

But I personally think we need to get that thread migration stuff sorted in the scheduler

@wild stone speaking of thread migration

[wild stone]

yeah yeah i know lol

[trim eagle]

Related, managarm PR 575 can be used to implement sysconf cores online / available right @wild stone ?

[native prairie]

is managarm actually able to run factorio now?

thats cool ig

[wild stone]

PR 575 needs changes (aside from being old and probably full of merge conflicts lol) to use kerncfg

[trim eagle]

Ish? if I recall correctly @brittle mauve had some memes with it.

[trim eagle]

Ahhhhh right

[native prairie]

i thought factorio needed like 9999 libraries

[brittle mauve]

yeah it uses that to get the number of render threads to spawn if you don't specify it manually

[native prairie]

yeah like sound and shit

that stuff is implemented on managarm?

[trim eagle]

Hmm yeah I think that shouldn’t be too hard. We can do similar memes that the getmemoryinfo kerncfg does

It’s ported at least (we have no audio server)

But alsa-lib is straight in upstream

And idr if I ever pushed pulseaudio

[brittle mauve]

for pulseaudio I just copied the libs from the host because I don't think its upstream yeah

[trim eagle]

Rest is upstream for sure

[trim eagle]

Understandable

[native prairie]

oh nice

[trim eagle]

I might have that recipe locally

[native prairie]

that audio stuff on linux is probably worse than drm even

and atomic drm is not even that cancerous

[trim eagle]

We’ll find that out when someone starts on audio

[brittle mauve]

alsa stuff is kinda annoying yeah

[native prairie]

i know a guy

[trim eagle]

Surprisingly same

[brittle mauve]

it has some good parts but then it has some annoying parts like the parameter refining stuff

[native prairie]

last time i looked this HDA output tree stuff looked a lot more scarier than DRM encoder/connector/etc stuff

[brittle mauve]

well yeah its kinda weird but its not that bad

and ofc there is stuff like figuring out what outputs can be used at the same time if you want to support that

[dense umbra]

and the audio card deadlocking

[brittle mauve]

yeah lol

I have only had that happen in passthrough when I wasn't correctly stopping the card before doing all the initialization, then the controller would init fine but when you try to send commands to the codec it just wouldn't ever respond

[trim eagle]

Sounds like nothing you can’t handle

[brittle mauve]

maybe after the astral driver is in an usable state if nobody else has started doing it (though the current bugs like the card hanging that matthew had when rebooting into windows might be annoying to figure out)

[trim eagle]

That would be highly based if you can do that

[carmine current]

what ipc issues?

[native prairie]

.

I thought it was slow because ipc

[carmine current]

i don't think anybody benchmarked it properly

so we don't know where the bottlenecks are

[brittle mauve]

disk io is also a thing that should somehow be benchmarked because its pretty bad eg. when writing a lot of decently sized files like wine does (the initial startup time takes like 2s on linux and on managarm it takes multiple minutes)

idk how would you do that in a good way tho because it goes through the different servers

[dense umbra]

Does managarm not cache stuff and do writeback over time

Idk how thatd even work in a microkernel

[upbeat smelt]

we do that

[dense umbra]

How does it work like from page fault to the page getting mapped in
Im curious cuz microkernel and all

[quaint fern]

for managarm it may differ, but for example, mach as of 1987

[upbeat smelt]

for each file the fs driver creates a new managed memory object

then, when a page fault occurs and it's related to a page in it, the kernel sends a notification to the driver, the driver fills the page, and then tells the kernel that it's done

and likewise to evict memory the kernel sends a notification, the driver writes it back to disk, and tells the kernel that it's done

[native prairie]

Whats the latency for that roundtrip?

[upbeat smelt]

https://github.com/managarm/managarm/blob/master/drivers/libblockfs/src/ext2fs.cpp#L863-L904

[dense umbra]

I see

[upbeat smelt]

well it's certainly worse than if the fs driver was in the kernel, but i have a feeling the disk read/write latency still higher

[native prairie]

Why is it worse than if it was in the kernel?

Nvm im rarted

[upbeat smelt]

not accounting for the fs server blocking while waiting for the device read: user -> kernel -> fs server -> kernel -> user

vs user -> kernel -> user

[native prairie]

Yeah makes sense

The price to pay for microkernel stuff

[upbeat smelt]

at least the way we do it the block device driver and fs server are together, since the fs stuff is a shared lib that block drivers link against

[trim eagle]

Did some looking, my current plan for this is a sysconf request to posix, and for the core count a kerncfg request from posix. Kerncfg just needs to call getCpuCount and return that back upstream. Later I’ll probably work over the same thing to implement memes for cpuinfo

[median wharf]

point taken but a paper is not by definition complicated to read

[trim eagle]

aight that's a thing now but I get a frigg assert

debugging....

hmm

infoLogger() << "thor kerncfg: handleRequest 1" << frg::endlog;
        auto [acceptError, lane] = co_await AcceptSender{boundLane};
        if(acceptError != Error::success) {
            infoLogger() << "thor kerncfg: handleRequest acceptError" << frg::endlog;
            co_return acceptError;
        }

this error pops

I'd almost say that this is broken

auto [kerncfgSendResp, kerncfgResp] = co_await helix_ng::exchangeMsgs(
                getKerncfgLane(),
                helix_ng::sendBragiHeadOnly(kerncfgRequest, frg::stl_allocator{}),
                helix_ng::RecvInline{}
            );

where getKerncfgLane() is

namespace {
    helix::UniqueLane kerncfgLane;
};

helix::UniqueLane &getKerncfgLane() {
    return kerncfgLane;
}

[fervent zodiac]

do you not initialize it?

or am i missing some sort of secret managarm dark magic

[trim eagle]

async::result<void> enumerateKerncfg() {
    auto filter = mbus_ng::Conjunction{{
        mbus_ng::EqualsFilter{"class", "kerncfg"}
    }};

    auto enumerator = mbus_ng::Instance::global().enumerate(filter);
    auto [_, events] = (co_await enumerator.nextEvents()).unwrap();
    assert(events.size() == 1);

    auto entity = co_await mbus_ng::Instance::global().getEntity(events[0].id);
    kerncfgLane = (co_await entity.getRemoteLane()).unwrap();

    auto procfsRoot = std::static_pointer_cast<procfs::DirectoryNode>(getProcfs()->getTarget());
    procfsRoot->directMkregular("cmdline", std::make_shared<CmdlineNode>());
}

should be initialized

[fervent zodiac]

you sure that definitely happens first?

[trim eagle]

yep, this is called before posix-init is even launched

async::detached runInit() {
    co_await enumerateKerncfg();
    co_await clk::enumerateTracker();
    async::detach(net::enumerateNetserver());
    co_await populateRootView();
    co_await Process::init("sbin/posix-init");
}

[fervent zodiac]

also you know you dont have to put "ng" after a namespace

[trim eagle]

you do if this is the "new" api and the old one still exists under the non "ng" name

[fervent zodiac]

you can rename the old one to v1

or migrate everything to the new api

[trim eagle]

I believe the plan is to migrate and nuke the old one

[fervent zodiac]

but fair

[trim eagle]

almost everything is migrated iirc?

[fervent zodiac]

is helix not ng a thing?

[trim eagle]

yes

[fervent zodiac]

ok ig

[trim eagle]

but you really should not use that anymore

the "new" interface is so much nicer

[fervent zodiac]

managarm: we have legacy code in a hobby os somehow!

and its about as stable as windows 95

[trim eagle]

there are probably a few requests still using old helix

[fervent zodiac]

i.e. not

[trim eagle]

yeah yeah

[trim eagle]

@carmine current do you have any clue? See the message I replied to and the few below there

[fervent zodiac]

what's the specific error?

[trim eagle]

logging that for you...

[fervent zodiac]

🤦‍♂️

[carmine current]

This needs offer() to match the accpet() in the kernel

[fervent zodiac]

first step in debugging an error

what is the error?

[trim eagle]

yeah yeah I didn't log it yet

[fervent zodiac]

should be the first thing you log

[trim eagle]

not wrong

transmissionMismatch seems to be the one

aka what korona said

hmm

hold up

might see something

progress!

[carmine current]

Nice

[trim eagle]

yeah it was missing an offer, and therfor the GET_MEMORY_INFORMATION request is also broken

but we never noticed that as that's not used

woot works

PRs made

I do wonder if glxgears will spawn more threads now. if I recall correctly it queries this shit

[trim eagle]

small issue

can't test glxgears

as my tmpfs permission work broke weston boot

[11:29:29.503] failed to create lock file /tmp/.X0-lock: Access denied (EACCESS)

[dense umbra]

no sticky bit? :megaman:

[trim eagle]

uhhh

I highly doubt that we handle that at all

[dense umbra]

Yeah youll need that

In astral I have /tmp and some X.org folder as sticky

mkdir -p /tmp
mount /tmp tmpfs
chmod 1777 /tmp

chmod 0755 /dev/pts

# required to be owned by root and sticky for rootless Xorg.
mkdir -p /tmp/.X11-unix
chmod 1777 /tmp/.X11-unix

[trim eagle]

yeahhhh

that's gonna be a small init change

[dense umbra]

Just finish the systemd port smg

Smh

[trim eagle]

yes yes

that's also still in progress

and then I can let systemd-tmpfiles handle this

yes

[dense umbra]

I tried porting openrc but dear lord so many linux apis

[trim eagle]

just port systemd once I'm done with it

I'll nuke the really annoying API's out of it

(capabilities, namespaces, cgroups)

"just" implement timerfd, memfd, epoll, signalfd, netlink (netlink maybe optional if no udev?)

this list is not checked, nor inclusive

[dense umbra]

Wonder if anyone ever ported system v init

[trim eagle]

it's not that hard I think

fd = open(lockfile, O_WRONLY | O_CLOEXEC | O_CREAT | O_EXCL, 0444);

[dense umbra]

I think I compiled it in astral once for shits and giggles but never tried to boot with it

[trim eagle]

0444 is r--r--r-- right?

[dense umbra]

I think so

[trim eagle]

yes

it is

[dense umbra]

Its what made me talk about that getconf or something command in #mlibc-dev

[trim eagle]

but you want to open the file for writing?

[dense umbra]

Its a lock file I dont think so?

Wait

[trim eagle]

O_WRONLY is write only

[dense umbra]

Bruh

[trim eagle]

that's kinda dumb

[dense umbra]

Im assuming it does that just to check for the file existance or something??

[trim eagle]

idk

[dense umbra]

And so it never actually opens it

[trim eagle]

but this seems stupid

[dense umbra]

But can return EEXIST

[trim eagle]

fd = open(lockfile, O_WRONLY | O_CLOEXEC | O_CREAT | O_EXCL, 0444);
    if (fd < 0 && errno == EEXIST) {
        fd = open(lockfile, O_CLOEXEC | O_RDONLY);
        if (fd < 0 || read(fd, pid, 11) != 11) {
            weston_log("can't read lock file %s: %s\n",
                lockfile, strerror(errno));
            if (fd >= 0)
                close (fd);

            errno = EEXIST;
            return -1;
        }

        /* Trim the trailing LF, or at least ensure it's NULL. */
        pid[10] = '\0';

        if (!safe_strtoint(pid, &other)) {
            weston_log("can't parse lock file %s\n",
                lockfile);
            close(fd);
            errno = EEXIST;
            return -1;
        }

        if (kill(other, 0) < 0 && errno == ESRCH) {
            /* stale lock file; unlink and try again */
            weston_log("unlinking stale lock file %s\n", lockfile);
            close(fd);
            if (unlink(lockfile))
                /* If we fail to unlink, return EEXIST
                   so we try the next display number.*/
                errno = EEXIST;
            else
                errno = EAGAIN;
            return -1;
        }

        close(fd);
        errno = EEXIST;
        return -1;
    } else if (fd < 0) {
        weston_log("failed to create lock file %s: %s\n",
            lockfile, strerror(errno));
        return -1;
    }

[trim eagle]

yes

[dense umbra]

Or maybe it expects to run as root

[trim eagle]

it runs as root

[dense umbra]

IIRC root ignores the mode check for read write

Think that might be it

It expects to be root and so opens expecting for the write open to be succesful on the readonly file

[trim eagle]

hmm

interesting...

with that changed it does boot

and indeed glxgears seems to make 4 threads for it

(can't talk perf as my cpu is currently limited to 2.0GHz as to not imitate a F16 taking off in class

[dense umbra]

Coward

[trim eagle]

well then it would be obvious that I'm doing something !class in class

I have a few spare minutes in like 25 minutes

and I can unleash, test and reign the cpu back in in that time

[dense umbra]

I think llvmpipe scales to 8 cores

[trim eagle]

Managarm runs with smp 4 here

so 4 threads it is

[vestal sapphire]

based

[dense umbra]

Now get factorio and see how faster it is

[trim eagle]

I'll try glxgears and call it a day

@vestal sapphire fixed the items you mentioned

quick initial test shows 81 FPS

that's terrible

idk

but this is absolute dogshit

let's see what happens when I hardcode 1 again

lol it's still making 4 threads

and higher FPS count

averages 99.75 FPS

what

imma retry the sysconf version

82,501 FPS with the changes

I lose perf

that's like what

17.3% loss?

what the actual fuck

@dense umbra @carmine current @vestal sapphire you might be interested in this

[dense umbra]

Perf loss with smp?

[carmine current]

there are many reasons why this could happen

[dense umbra]

I imagine itd be because on glxgears the bottleneck is more so IPC speed?

[trim eagle]

perf loss with sysconf reporting 4 cores online

both are actually running with 4 cores (smp 4 passed to qemu)

[vestal sapphire]

glxgears is just an IPC benchmark

[carmine current]

• without thread migration, the threads may run on the same CPU and decrease perf (unless it uses pinning)
• the overhead of multithreading may be too high compared to the speedup for glxgears
• the bottleneck may simply be somewhere else
• benchmarking artifacts: just measuring once is not enough to get a reliable result. what is the standard deviation of different runs?

[trim eagle]

it's still interesting that I get a noticable perf loss just by sysconf reporting 4 cores instead of 1

[carmine current]

• a non-one value for the number of CPUs may cause to use a suboptimal code path somewhere

[trim eagle]

I ran twice with and twice without the sysconf changes (aka early EINVAL return inserted in the sysdep) and the perf diff was around the same, maybe +/- 1 FPS over 30 seconds measured

[carmine current]

does it spam the cpu count request or does it only call it once?

[trim eagle]

once

at startup

[carmine current]

how do you know that?

[vestal sapphire]

common wisdom

[trim eagle]

yeah, however, I see 4 threads being created regardless of the sysconf changes being enabled or not (related to the first point)

so it seems to always use 4 threads

[dense umbra]

On astral I sped up glxgears from 60 fps to 250 by just optimizing my ipc

[carmine current]

aka uneducated guessing

[trim eagle]

it's probably a good guess

but without measurements you can't be sure

[carmine current]

I did perf optimization for 5+ years professionally, if I learned one thing it's that guessing is often wrong

and one needs to do actual benchmarks to see what's happening

[trim eagle]

that's true

[carmine current]

If we want to improve perf on managarm, we should write and/or port some benchmarks

[trim eagle]

we should do more benchmarking yes

[carmine current]

and we should start with something simpler than glxgears

something as stupid as:

• I/O on tmpfs on linux with mlibc-lfs vs. I/O on tmpfs on managarm

[quaint fern]

implement DTrace4Managarm

[carmine current]

the problem with glxgears is that there are too many moving parts

[trim eagle]

literally

[carmine current]

even if ipc is the problem, we cannot see what kind of ipc is slow

since glxgears involves at least 4 processes

[trim eagle]

yeah glxgears isn't the easiest to start

[carmine current]

some benchmark that only involves posix and no hardware I/O would be a good place to start

[trim eagle]

ig we can time how often one can call getpid

[carmine current]

i think it'd better to check how often you can write X bytes to a tmpfs file

and then we can vary X between 1 byte and 128 mib

or so

[trim eagle]

yeah that would be interesting

[trim eagle]

and merged, we'll defer the perf memes to later

[trim eagle]

Good news from the network stack: we merged our port of the FreeBSD e1000 Ethernet driver giving us access to a lot of Intel NICs

[shrewd grove]

wow thats cool

was it much work to port the bsd driver?

[trim eagle]

@vestal sapphire knows more about that, I believe it wasn’t too bad actually

He might also be able to tell us what the next network driver is, I believe it’s Realtek that’s up next

[native prairie]

uE1000 by FreeBSD

[trim eagle]

[native prairie]

its like 40k lines right

[worn rampart]

igb/igc when

Aka the drivers actually needed for a machine from the last 10-15 years :^)

[trim eagle]

The 2.5 gigabit stuff?

Maybe soon maybe not

[worn rampart]

Igb is 1gb

Igc is 2.5gb

Igb are nics like i210/i211/i350

I guess technically an e1000 driver also works for them tho, so it might already work

[vestal sapphire]

igc more like 3 y but ok

igc seems to be as much effort as e1000, so once I have enough time it should be pretty quick

[vestal sapphire]

you basically only need to write bindings, that is implement a few osdeps (mostly PCI stuff), and the actual binding which is calling the functions of the imported code

for which you can take a look at what freebsd, fuchsia et al already do

[worn rampart]

I210 which is an igb card is from like 2013

[trim eagle]

Update from ACPI land: Managarm can now read battery information and that is exposed to userspace, and we can have upower use that too!

Oh and it works on real hardware too

Meanwhile, from the mlibc LFS project, work has begun on supporting sanitizers for mlibc (at least on the Linux platform). The initial goal here is libasan and libubsan. Work will continue next week but it’s looking promising

[shrewd grove]

a managarm screenshot with asserts featured

but thats pretty cool, gj

[native prairie]

thats how u know its authentic

[trim eagle]

True

But glib asserts are not Managarm asserts

[quaint fern]

Everything's a bloody critical assertion failure in glib

It's a wonder the gnome slop runs at all on any computer

[trim eagle]

It’s fucking worse than Managarm and we live on asserte

[worn rampart]

Yeah scammed where are the managram asserts

[fervent zodiac]

just because it's a critical assertion failure doesn't mean you have to crash, though

of course!

[vestal sapphire]

https://forum.osdev.org/viewtopic.php?p=349733#p349733

https://tenor.com/view/lets-go-lets-goo-lest-gooooooooooooooooo-gif-19416648

[native prairie]

wait theres a sortix post right above

is it resurrected

[trim eagle]

Apparently

[dense umbra]

Does sortix still not support networking nor smp

[dusk solstice]

WebKitGTK support is insane lmao

[trim eagle]

Yeah I’m quite happy with that

[dusk solstice]

Is it possible (and not too much harder) to port mlibc to a non-unix-like system? Still using an everything-is-a-file idea, but for example using only event queue rather than signals (but I can probably write a wrapper for that) and using a lettered vfs?

[carmine current]

Managarm implements all UNIX "syscalls" using IPC

So what you want to do sounds similar

[upbeat smelt]

you might have some trouble if you want to enable the posix option etc but ansi shouldn't be that much trouble

since for example things like realpath assume paths are unix-like (forward slashes, absolute paths begin with a slash)

[dusk solstice]

okay thank you. I'm considering taking some existing libc and modifying it to my system a bit. rn it's kinda between mlibc and musl. idk mlibc's license yet tho so I'll have to look at that. I just wanna know what will be the easiest to port atm tho.

[upbeat smelt]

well musl is linux-specific

with no layer between the libc functions and syscalls

[dusk solstice]

hm yeah okay

ty

I kinda just figured I could kinda make a fork of it that's more specific to my design, but it may be tricky

[dusk solstice]

just checked, it seems to be under mit which is nice

[upbeat smelt]

well you could go through all functions that issue syscalls and modify them or change the definition of the syscall macro to dispatch to a custom function to handle them instead

[dusk solstice]

yeah that's what I intended to do

I may try out mlibc. I'll need to see but rn I'm kinda just thinking about libc to procrastinate on more important stuff lol

so I think I'll leave this for a bit anyway

[vestal sapphire]

I never get how musl is even a consideration when it literally hard-assumes linux, so you get to manually patch every single function that might issue a syscall

[nocturne tide]

because it's not glibc

and obviously nothing besides linux exists anyway

so why would you even care?

[safe glen]

the fact that people think musl is easier to port than glibc is but a testament to how unportable glibc really is

[vestal sapphire]

I'm honestly surprised that people try to port musl as much as they do, despite the pain

even in the face of multiple other libcs that are significantly easier to port, mlibc being only one of them

[safe glen]

mlibc is where its at

newlib is also fairly portable, but it's still harder to do so than with mlibc and its completeness is questionable

[full slate]

I know a company that patches musl for their shenanigens, and it probably does make more sense than using mlibc

since mlibc is a bit incomplete for production use-cases, and patching musl is not really that bad (for certain usecases)

[quaint fern]

musl's biggest demerit is that it looks like it was written by aliens

i think they also skip implementing some traditional additions found in both the bsd and gnu libcs

[full slate]

average libc implementation

anything written by systems programmers looks like it was written by aliens

probably because we are

[dusk solstice]

I've noticed that the code style is also... a big mix of different styles lmao

[icy verge]

Clang format to the rescue

[quaint lotus]

Can Managarm be used as the OS for a development machine used to work on another OS project as of now? Do most Linux programs work out of the box?

[vestal sapphire]

depends™️

[median wharf]

mostly, no

[carmine current]

While most development tools work, you'd probably face stability issues (e.g. oom)

[quaint lotus]

I see. Well I look forward to the day when I can use it for this.

[trim eagle]

not on Managarm, but on Linux with mlibc as the C library of choice, we're running systemd now

it's still a bit hacky, but it does boot

really happy with this, the plan is to clean this up and start upstreaming fixes in the coming week

eventually the goal is to run systemd on Managarm itself

[native prairie]

Nice

Very important step

[dusk solstice]

is initware no longer in development?

[dusk solstice]

I was trying out managarm today. Works pretty nicely but Vim had some weird bugs just running it.

not sure exactly how to explain it, I'll send a screen recording.

[trim eagle]

Sounds good

[dusk solstice]

Oops sorry forgot to upload it. Looking back the bugs aren't actually that hard to explain lol.

I haven't tested it as root, I just assumed that it wouldn't make a difference

it's the same issue within wayland in the terminal emulator

so yeah just massive delays, that red thing at the bottom, and it doesn't really clear the screen when vim is started or show the "VIM - Vi Improved" and version info message on start

[trim eagle]

Hmm. Yeah that’s scuffed. The red thing is setitimer sysdep not being implemented (the logs in the background give that one away)

The rest, good question, something to investigate for sure

[dusk solstice]

(this is from yesterday's daily compressed image btw)

[vestal sapphire]

yeah this simply is a setitimer not being implemented issue afaict

[wild stone]

setitimer not being a thing breaks a surprising amount of things

[vestal sapphire]

zsh

[median wharf]

For anyone who missed it, syscall cancellation work is in its final stages, with it finally working. Exiting cat coming to a mainline managarm near you soon!

https://cdn.discordapp.com/attachments/678613590646849576/1317881024213946418/2024-12-15_16-46-44.webm?ex=67639799&is=67624619&hm=17fc85ea3c46057e5f04ab3ea21679ee13f22afad0bb7b87334d7348102dc4a8&

[upbeat smelt]

is the last week of 2024 the week of the managarm desktop

booted off of an usb stick, using an usb mouse, and an usb nic

there's still a bug somewhere in the pci mbus code since some devices fail to get picked up by posix-subsystem (initially it wouldn't even boot into weston, after adding some debug prints it boots, but one pci device is still missing from lspci)

[trim eagle]

Nice

Also update xbps plz

You should be able to do that from the running Managarm instance

And then one can do actual package manager shit

[upbeat smelt]

i more just wanted to see if the network connection works fully

i dont think i have curl or wget installed

nor ping

but ig since dhcp worked it should've been good enough

[native prairie]

how did debug prints help booting btw?

[trim eagle]

But just to test, xbps-install -u xbps?

[upbeat smelt]

probably the bug is timing related

[native prairie]

ah

i wonder how many things would break if u strip all printfs from managarm

[carmine current]

probably nothing?

[native prairie]

hopefully

[brittle mauve]

removing asserts also doesn't really break anything apart from a few things in the graphics drivers

https://github.com/managarm/managarm/blob/master/drivers/gfx/bochs/src/main.cpp#L122 https://github.com/managarm/managarm/blob/master/drivers/gfx/plainfb/src/main.cpp#L119 https://github.com/managarm/managarm/blob/master/drivers/gfx/virtio/src/main.cpp#L175

[lament grove]

BadgerOS actually has a concept for this! There's assert_dev_drop (assert something is true, drop in release builds), assert_dev_keep (assert something is true in debug builds, run but do not check in release builds) and finally assert_always (always run and check).

[carmine current]

We should fix these

[dusk solstice]

which NICs do managarm support?

[upbeat smelt]

rtl8169, e1000, virtio-net, anything with usb cdc ecm/ncm/mbim

the nic i have uses cdc ncm

[graceful temple]

managarm: load bearing asserts lmaooo

[quaint fern]

that reminds me to start properly using the new distinction i've introduced between kassert_dbg() and kassert()

that's "drop if release build" and "assert regardless". but your intermediary "do but don't assert" would be a useful addition too

[carmine current]

In my experience the perf advantage of compiling without assertions is minimal

[fervent zodiac]

v8 calls these DCHECK and CHECK respectively iirc

[fervent zodiac]

that depends on how you use assertions, though

if you use them to put possibly expensive consistency checks all over the place, it can be significant

[native prairie]

Just do serenity ASSERT vs VERIFY

[carmine current]

i usually guard expensive checks behind global constexpr bool debugFoo;

[quaint fern]

which one's which though?

[native prairie]

assert is like normal assert, aka compiled out for release

verify is permanent

[quaint fern]

i wonder why capitals for that matter

serenity has such an odd style

[native prairie]

not sure

[quaint fern]

they also took the $ from OpenVMS which is used to denote system services there, but they couldn't tell me why

[native prairie]

i think the idea was to easily distiguish syscall handlers from normal functions

[quaint fern]

do you know is serenity still alive or has it atrophied without the commanding leadership of klinge?

[native prairie]

there's still 2 or 3 people that try to do stuff, but it's more of the same, adding new tech debt on top of previous tech debt, plus ladybird cherry picks

[trim eagle]

neofetch kinda overrated, so we do fastfetch now

a bunch of features are still missing, working on that

[silent oak]

?

[trim eagle]

unlucky no cpu info for us yet

as it needs /proc/cpuinfo on linux

@vestal sapphire oh sysfs master I humbly request a few additions

    ffGetSmbiosValue("/sys/devices/virtual/dmi/id/bios_date", "/sys/class/dmi/id/bios_date", &bios->date);
    ffGetSmbiosValue("/sys/devices/virtual/dmi/id/bios_release", "/sys/class/dmi/id/bios_release", &bios->release);
    ffGetSmbiosValue("/sys/devices/virtual/dmi/id/bios_vendor", "/sys/class/dmi/id/bios_vendor", &bios->vendor);
    ffGetSmbiosValue("/sys/devices/virtual/dmi/id/bios_version", "/sys/class/dmi/id/bios_version", &bios->version);
    if (ffPathExists("/sys/firmware/efi/", FF_PATHTYPE_DIRECTORY) || ffPathExists("/sys/firmware/acpi/tables/UEFI", FF_PATHTYPE_FILE))
        ffStrbufSetStatic(&bios->type, "UEFI");
    else
        ffStrbufSetStatic(&bios->type, "BIOS");
    return NULL;

[vestal sapphire]

Fuck No

I'm not doing smbios

[trim eagle]

if we add these items we get bios support in fastfetch

[trim eagle]

we only need to expose it to userspace no?

oh no we probably need to parse some bs

how bad is this?

(while you're at it)

const char* ffDetectBoard(FFBoardResult* board)
{
    if (ffGetSmbiosValue("/sys/devices/virtual/dmi/id/board_name", "/sys/class/dmi/id/board_name", &board->name))
    {
        ffGetSmbiosValue("/sys/devices/virtual/dmi/id/board_serial", "/sys/class/dmi/id/board_serial", &board->serial);
        ffGetSmbiosValue("/sys/devices/virtual/dmi/id/board_vendor", "/sys/class/dmi/id/board_vendor", &board->vendor);
        ffGetSmbiosValue("/sys/devices/virtual/dmi/id/board_version", "/sys/class/dmi/id/board_version", &board->version);
    }
    else if (ffReadFileBuffer("/proc/device-tree/board", &board->name))
    {
        ffStrbufTrimRightSpace(&board->name);
    }
    else if (ffReadFileBuffer("/proc/device-tree/compatible", &board->vendor))
    {
        uint32_t comma = ffStrbufFirstIndexC(&board->vendor, ',');
        if (comma < board->vendor.length)
        {
            ffStrbufSetS(&board->name, board->vendor.chars + comma + 1);
            ffStrbufTrimRightSpace(&board->name);
            ffStrbufSubstrBefore(&board->vendor, comma);
        }
    }
    return NULL;
}

[vestal sapphire]

It's just a world of pain to obtain everywhere, pass to userspace, decode, and expose over sysfs

[trim eagle]

plz?

[trim eagle]

sadge

I guess I'll just revert board and bios back to nosupport for now

[vestal sapphire]

👍

I'm already hating sysfs enough for doing NVMe

Speaking of, stream in 5

[trim eagle]

how about efivarfs then ?

[vestal sapphire]

Writable?

[trim eagle]

disabling that too...

[vestal sapphire]

Problem with efivars is having to preserve some memory map shit, update EFI's memory map, and then call runtime services

[trim eagle]

yeah that's meh

[vestal sapphire]

We don't even have that info in Thor currently lol

[trim eagle]

I'm nuking a bunch of shit that we don't support yet lol

including all the cpu info memes

as we don't do /proc/cpuinfo (on the TODO list!)

ahhh there goes physical disks as we don't have /proc/mounts

[upbeat smelt]

we could do what linux does and use an identity mapping for calling the efi rt stuff

or at least efistub doesn't move any of the runtime memory map entries

it just sets the flags and then calls the function

[trim eagle]

@vestal sapphire you know where I am (gf) so this location info is way off but it's funny! also fixed the initsystem stuff

@vestal sapphire /dev/input/by-path is a thing? fastfetch wants it if we want keyboard items

do you know anything about that?

(and /sys/class/input needs some love too for mouse handling)

[vestal sapphire]

yeah but I'm not certain whether that's something that udev or we set up

[trim eagle]

noted

ahhhh I see why some stuff is off

one moment more cmake patching....

mlibc: Unexpected DRM ioctl with, number: 0xad (raw request: c01c64ad) @vestal sapphire

fastfetch doing DRM nasties

for now the best I can do

[carmine current]

SMBIOS is simple btw

[safe glen]

bruh no way

holy shit

[trim eagle]

Starting the year strong, first actual new port that wasn’t just me fixing xbbs is a good one

[safe glen]

streaks wanted to port it

we took a look at the source and went like "nah"

[trim eagle]

I gotchu

[safe glen]

like it's a lot of work

[trim eagle]

Not really

It wasn’t too bad

[safe glen]

at least it seemed like a lot of work

[trim eagle]

Bootstrap 433 if you want to take a look

[trim eagle]

That is true

[native prairie]

its literally an array of structs yeah

[trim eagle]

But some copy paste later and a bunch of replace _linux with _nosupport gets you the basics

[safe glen]

bootstrap 433?

[trim eagle]

Then I took a Quick Look at all the modules and if I figured we can run it I switched it out

[trim eagle]

Bootstrap-Managarm PR 433

[safe glen]

ah

ty

[trim eagle]

Hmm. Maybe one day then (or can I interest @native prairie in making a PR for it?)

[native prairie]

currently busy with uacpi 1.0

[safe glen]

SMBIOS is piss easy

[trim eagle]

After that, no rush here

[trim eagle]

Leo said it sucks lol

[safe glen]

anyways @reef steeple should i interest you with https://github.com/managarm/bootstrap-managarm/pull/433

[safe glen]

i said it's easy, not that it's good

[trim eagle]

Sucks to implement

[safe glen]

i and Leo worked on it together for Limine remember?

[trim eagle]

That it isn’t good is almost given (and it’s exposed over sysfs and I know you don’t like that)

[trim eagle]

Oh fuck that’s true yeah

[reef steeple]

problem is that they use the linux compat

because managarm is a linux distro

I would have to rewrite a lot more

[safe glen]

i mean not necessarily

[reef steeple]

and whatever is not linux they just disable it :P

[wild stone]

i wonder how often the word unimplemented is present in managarm code

[safe glen]

you can just nosupport stuff

and then implement it

[upbeat smelt]

i'm pretty sure by-path is made by udev

[reef steeple]

yes, problem is not that it wont compile until its ported, its that its useless until you hit a critical mass of stuff

[safe glen]

it will

that's what i am saying

that's what the *_nosupport.c files are for

[reef steeple]

yeah im just telling you thats not the issue

[safe glen]

what's the issue?

[reef steeple]

that I would need to implement a ton of code for it to have the same features as neofetch does right now

[safe glen]

yeah i mean sure but you can at least get it to build lol

[reef steeple]

whats the fun on having something compile if you cannot use it

its why I bang my head for hours fixing df

[safe glen]

sure whatever

it's okay i'm too tired to do it today anyways

[reef steeple]

if you want to write the bindings for feature parity with neofetch you couldnt physically either

I can port it without the bindings as a "woah it builds" kind of thing if you want that

[safe glen]

unironically the bindings can literally be exec calls to programs just like neofetch does if you want

[reef steeple]

its not that easy, but sure

I just hope someone implements the bindings at some point properly because I cannot, my hands are a bit too full already

[safe glen]

don't ask me

i think we reached our human limit

[reef steeple]

then ill just leave this at the spot it was in my TODO, and get to it when time comes, thanks for bringing it to my attention

[trim eagle]

Maybe you support one of the bsd interfaces

[reef steeple]

in these regards (information gathering) ironclad is not like bsd or linux, with their sysctl and /proc respectively

its actually really interesting you guys dont have /proc/meminfo

why is that

[trim eagle]

I never added it. Same for cpuinfo

I might look at that tomorrow (at least to stub it somewhat and fill what I can reasonably fill

But I also want to look at fixing webkitgtk

[reef steeple]

fair enough

another person with a bit too much work

[trim eagle]

Yeah

But with the new accounting PC (which is now at my room temporarily hopefully permanently) I get more compile speed

So WebKit should be less painful

Which is good

[reef steeple]

neat

[vestal sapphire]

DRM_IOCTL_MODE_GETFB

should be easy to implement tho?

have fun dennis

[trim eagle]

I feel like we already have a way to get a framebuffer? Just not wired to mlibc?

Or am I thinking too simple here

[vestal sapphire]

I mean ig getproperties is similar

but we should implement that ioctl properly to keep atomicity requirements etc

tldr don't try emulating it in mlibc sysdep land

[trim eagle]

I’ll take a look at getproperties and work from there (look now, work tomorrow)

[vestal sapphire]

getting the info shouldn't be hard

from libdrm source

basically just look at what drmModeGetFB does

[trim eagle]

@vestal sapphire we have GETFB2 implemented

So I guess I can yoink that ish?

Yeah a bit different in what info it gets

But that’s probably a good start

No weird shit either just some uint32’s

Yeah I can do that I think

[vestal sapphire]

probably getfb2 is just a superset?

didn't check

[trim eagle]

I’ll check

[vestal sapphire]

ah not quite

[trim eagle]

Partially it is tho

So I can lift that straight out of the other one

[vestal sapphire]

on another note, I'm getting closer to nvme-cli support

I not need to rename sdaX if they're nvme drives to the nvme format

[trim eagle]

No rename needed? So no nvme0n1p1 memes?

[vestal sapphire]

nah, instead of providing "sdaX" we should be providing nvme0n1p1 instead

[trim eagle]

If we’re on nvme then yes

[vestal sapphire]

ofc

[trim eagle]

Yeah obviously

[vestal sapphire]

re the nvme naming

[trim eagle]

Based!

[vestal sapphire]

now ig I need to expose a nvme0n1 as well

otherwise nvme-cli complains

[trim eagle]

If that matches Linux then yes

[vestal sapphire]

yeah

it wasn't even too large of a hack to make it work

[trim eagle]

Nice

[vestal sapphire]

ah shit I forgor about the correct major numbers

[upbeat smelt]

exposing the full block dev shouldn't be that difficult i think

since we can expose the raw partitions and they're readable etc

[vestal sapphire]

it isn't but the wiring around it is a bit annoying

esp when dealing with nvme memes

that is forgetting about the character device memes for nvme

[trim eagle]

I wonder, should I attempt a real installation of Managarm on the fancy PC today. I have nvme room to spare and the Ethernet card is one that you have Leo if I recall correctly. So that could be actual funnies

I want to say I’ll retry from usb first but I think xhci still fucked there

[vestal sapphire]

oh yeah about the rtl 2.5 GBit cards

I haven't touched that

[trim eagle]

No rush

There is higher priority stuff

[vestal sapphire]

nah I really wanted to get it going

[trim eagle]

Speaking of, EOY blog post?

[vestal sapphire]

I even set up my PC so that I have redundant connections

[upbeat smelt]

what does that need? i assume it just handles some ioctls?

[vestal sapphire]

did discord just eat my screencap

try 2

the fuck

[dense umbra]

need managarm eoy post

[upbeat smelt]

both arrived here

[vestal sapphire]

it allows for passing through commands to the NVMe from userspace

[upbeat smelt]

btw are we exposing multiple namespaces correctly?

[vestal sapphire]

hopefully yes

I can't think of a reason why not but I never tested it

[trim eagle]

I need to write that piece of crap

[vestal sapphire]

I don't even remember what I worked on in 2024

I'll have to look it up ig

ah there was the multitouch thing

what else

[trim eagle]

I did some KDE memes

But they broke

Korona was gonna look into that vfs meme but I prefer load balancing first

[vestal sapphire]

libblockfs: Disks don't currently serve requests

FUUUUUUUUUUUUUUU

[upbeat smelt]

i don't see why that's a big problem

i think on the libblockfs side you should be able to mostly ^C^V the partition request handling

[trim eagle]

One big TODO lol

[vestal sapphire]

yeah except the small minor details that it assumes ext2

[upbeat smelt]

it should probably get refactored at a later point to unify the partition and device handling

yeah but only if you try to mount it

iirc we don't try to parse the ext2 data until a mount request

[vestal sapphire]

ok I'll try

[upbeat smelt]

and besides, unpartitioned media is a thing

a bit dated (floppy disks etc) but still

[trim eagle]

When fat32 and ext4

[vestal sapphire]

whenever someone manages to untangle libblockfs

[trim eagle]

Figured

Maybe 2027

[upbeat smelt]

i mean it doesn't seem like it'd be that big of a job, and i'd do it, but i don't want to promise anything since this month is quite packed for me

[trim eagle]

No rush. I’ll remind you after I’m back in the land of the living past FOSDEM

[upbeat smelt]

[lament grove]

Almelo
Almelo is a dutch city

/gif ge

[trim eagle]

Your statement is correct

And the VOC will come to get you

[lament grove]

I can't help myself when I notice another Dutch person online

[trim eagle]

G E K O L O N I S E E R D, MAKKER!

[lament grove]

Oh Kut

[trim eagle]

Valid response

[lament grove]

Wait does Managarm not do FAT yet?

Does it at least support multiple mounts?

Maybe I can make a bit of a FAT driver for Managarm when I'm done with BadgerOS' one.

[trim eagle]

Nope

[trim eagle]

Considering we mount a bunch of tmpfs all over, I’d say yes

[trim eagle]

Highly based, but be warned, there be demons on the road and libblockfs needs some love

[lament grove]

I'm reading some of it, looks like some parts of the code assume EXT2.

BTW: Is your tmpfs just EXT2 on a RAMdisk?

[trim eagle]

Correct

[trim eagle]

Dont think so? @upbeat smelt

[upbeat smelt]

no our tmpfs is an actual tmpfs

in posix-subsystem

we just have objects for nodes and links, and file data is stored in memory allocated using the kernel apis (so that it can be mmaped)

and it uses the same api as other file systems in posix-subsystem (incl externfs which is used to talk to ext2 on different servers)

[lament grove]

What protocol does externfs speak with the servers?

[upbeat smelt]

https://github.com/managarm/managarm/blob/master/protocols/fs/fs.bragi

but do note that not all file requests go through posix

for externfs

[lament grove]

So that's a custom protocol

[upbeat smelt]

for most operations the flow is more like ```
client posix fs server
| open | |
|-------->| |
| | open |
| |---------->|
| | ipc lane |
| |<----------|
| ipc lane| |
|<--------| |
| read | |
|-------------------->|
| | read data |
|<--------------------|

[lament grove]

I see.

How does this handle paths that traverse different filesystems?

E.g. a symlink from one FS to another

[native prairie]

Is IPC lane just an fd?

[upbeat smelt]

in this case basically, each process just has a table that maps fd -> ipc handle for things like read/write etc

[upbeat smelt]

posix-subsystem is what manages the vfs and path resolution (hence why open goes through it), and for path resolution we have a request that asks the fs to resolve up to N components of the path, either until it's done, or it reaches a symlink, or it reaches an obstruction placed by posix-subsystem (e.g. a mountpoint)

[lament grove]

Ok, makes sense

[native prairie]

So the kernel syscall to talk to these operates on the handles?

[upbeat smelt]

so e.g. if you resolve /foo/bar/symlink/baz, it asks the fs server for / to resolve the whole path, the fs server resolves up to /foo/bar/symlink/, posix-subsystem then follows the link, and asks the appropriate fs server for the rest of path

[lament grove]

How does it go from there to the open function ?

[upbeat smelt]

the kernel syscall for this is just the generic ipc syscall

[native prairie]

Why does there have to be a translation layer from IPC handles to fds?

[upbeat smelt]

and you give it a handle to the ipc lane to send the messages on

the kernel is unaware of fds

[native prairie]

What are handles tho, not integer values?

[lament grove]

Something to do with microkernels

[upbeat smelt]

they are integers, but posix requires that certain invariants about fds hold, for example open and dup always use the lowest free fd

invariants that we don't necessarily satisfy with our handles

[native prairie]

Ah ok

[upbeat smelt]

and our handles are 64-bit not 32-bit so they could potentially not fit

[native prairie]

Fair

[upbeat smelt]

the translation is pretty thin though

it's just a page with an array of 512 handles (yes currently it's fixed in size)

it's specifically a page because it's shared between the process and posix-subsystem (on open, posix-subsystem writes the handle into the page)

[lament grove]

I'm looking at the open function and I can't tell what the contract for calling this function is.

Does it get something already initialized? It seems so.

[upbeat smelt]

https://github.com/managarm/managarm/blob/master/protocols/fs/src/server.cpp#L1634

and the request comes over ipc from https://github.com/managarm/managarm/blob/b38a4ef5e8553dbc7d9be805c470e8e6ef7a5fb4/posix/subsystem/src/extern_fs.cpp#L285

the open function on the libblockfs side is there to create a new ipc stream for handling file requests

and it sends back the handle to one lane of the ipc stream

[lament grove]

Ohhhh

That isn't actually the file open function

No, it writes atime, it has to be the open function

Where does the inode information enter that object?

[upbeat smelt]

the inode objects in libblockfs are created when an fs link is resolved

i.e. when posix-subsystem resolves a path to a specific inode on disk

and then when we open the file, the open request is sent

[lament grove]

So they're created by libblockfs's path resolving function

And passed around by the posix-subsystem

[upbeat smelt]

well it's handled in the generic protocols/fs code

https://github.com/managarm/managarm/blob/master/protocols/fs/src/server.cpp search for calls to serveNode

or well, the responsibility is shared

in libblockfs, getLink etc create the ext2-specific inode object, and in the protocols/fs server code, it starts the request handling machinery

[lament grove]

Riiiight

So libblockfs.cpp would have to be refactored such that it doesn't reference ext2 but rather an abstraction layer that lets you also use different filesystems (like FAT).

[upbeat smelt]

yeah, i'm thinking we just make an abstract base class that the ext2 inode and superblock classes inherit from

and make the open,getLink, etc functions work on the base classes instead

[lament grove]

libblockfs.cpp doesn't seem to reference the superblock - or at least not by that name.

[upbeat smelt]

ah the class is called ext2fs::FileSystem

that has the superblock operations (e.g. creating a new node, mounting, etc)

[lament grove]

Either way, though, I think virtual inheritance is warranted here. That would mean taking just the API from ext2fs.h that is used by non-ext2 files, make abstract classes for them and then make the ext2 classes inherit from them.

After that, the FAT filesystem shenanigans can finally begin.

But first, I'll make FAT work in BadgerOS.
And sleep. That's probably also useful :P

[upbeat smelt]

it would be nice to have some other simple unixy file system to test this with

before we have to force a square peg (fat) into a round hole (unix-style vfs)

[trim eagle]

When ext4

[upbeat smelt]

ext4 is not that good a good choice for this because 1) it's more complex than ext2 2) we can just massage the ext2 driver to support ext2/3/4 instead

[trim eagle]

Yeah I know

So then what

Zfs? Btrfs?

Xfs?

[upbeat smelt]

that's what im wondering too

none of them are particularly simple

[trim eagle]

Nope

[upbeat smelt]

i wouldn't be surprised if there was no good choice

[trim eagle]

There’s a reason everyone does ext2 and maybe fat32

[upbeat smelt]

and i suppose we could just go for fat32 as the second fs

[trim eagle]

My alternative would be that isofs meme? Iso9660?

[reef steeple]

if you want a unixy simple FS, you could do UFS, it has no practical application in the real world though

[trim eagle]

Cdrom moment

[upbeat smelt]

hmm iso9660 might not be that bad of an idea?

although i don't know if it's closer to fat vs unixy fs in structure

[lament grove]

FWIW, ISO9660 is another filesystem on my BadgerOS TODO list

[upbeat smelt]

iso9660 would be nice for managarm too since then we could make iso images instead of hdd images

and people seem to prefer those for testing stuff

[lament grove]

I think SquashFS could be useful since it's the way basically all live ISOs work for Linux distributions.

[trim eagle]

Squash might not be a bad idea either

[upbeat smelt]

that's true too

but squashfs would first require loopback devices i think

and iso9660 is a prereq for isos even with squashfs

[lament grove]

That's true

[trim eagle]

Fair

[upbeat smelt]

loopback devices should be relatively simple though now that i think about it

we could just link libblockfs into posix-subsystem, and have that create fake block devices from files

or we could have a dedicated server that handles them ig

[vestal sapphire]

that would also tie in nicely with the USB DVD drive I have here

[upbeat smelt]

[vestal sapphire]

but yeah FAT32 would be nice as a first addition

[upbeat smelt]

it'd allow us to mount /boot

[vestal sapphire]

I have some old btrfs work around

I might want to upstream that at some point

[reef steeple]

didnt know managarm does not support fat

you absolutely want that priority number 1

[trim eagle]

Yeah fat32 is important

[forest bear]

managarm is LEAN

[serene portal]

cooked

libblockfs

[trim eagle]

And if I recall correctly with a small amount of work one gets fat12 and fat16 too then right?

[trim eagle]

(We’ll ignore vfat and exFAT for now)

[upbeat smelt]

yeah it's just some logic to properly index into the FAT etc

based on the amount of bits per entry

[vestal sapphire]

reminder about #678613510237978625 message

[upbeat smelt]

vfat is just fat32 with long file names i think?

not 100% sure but i think i've read that somewhere

[reef steeple]

its weird that it has wayland and browsers yet no fat32, feels weird priorities

[upbeat smelt]

i think the fat lfn patent expired too so we should be safe

[trim eagle]

That shit is still golden

[serene portal]

who even uses fat smgh

[reef steeple]

you

[serene portal]

fat users need to eat less

[vestal sapphire]

it's not like I haven't tried FAT before, it's just that untangling libblockfs is horrible

[reef steeple]

that I cannot argue

[quaint fern]

ufs should be an easy implementation

ext2 is inspired by it

or rather almost everything in ext2 is directly cognate with something in UFS

[carmine current]

yeah, i agree. it's probably not that difficult. mostly, it's just adding a few vtables and separating "generic inode" from "ext2 inode" and similar

considering that FSes are quite different under the hood, the other way around may also be viable / better:

get rid of the generic classes and add a few templated helper functions to add the IPC boilerplate to an Ext2Inode

similar to how we do the arch-generic stuff in the kernel

basically, we'd have a concept Inode and a helper

template<Inode I>
protocols::fs::NodeOperations makeNodeOperations();

[trim eagle]

bool bus_socket_auth_needs_write(sd_bus *b) {
        log_debug_elogind("elogind bus_socket_auth_needs_write for %d", 1);
        unsigned i;

        if (b->auth_index >= ELEMENTSOF(b->auth_iovec)) {
                log_debug_elogind("elogind bus_socket_auth_needs_write for %d", 2);
                return false;
        }
        log_debug_elogind("elogind bus_socket_auth_needs_write for %d", 3);

        for (i = b->auth_index; i < ELEMENTSOF(b->auth_iovec); i++) {
                log_debug_elogind("elogind bus_socket_auth_needs_write for %d", 4);
                struct iovec *j = b->auth_iovec + i;

                if (j->iov_len > 0) {
                        log_debug_elogind("elogind bus_socket_auth_needs_write for %d", 5);
                        return true;
                }
                log_debug_elogind("elogind bus_socket_auth_needs_write for %d", 6);
        }
        log_debug_elogind("elogind bus_socket_auth_needs_write for %d", 7);

        return false;
}

hmm this seems to return false in the if (b->auth_index >= ELEMENTSOF(b->auth_iovec)) branch. I honestly have no clue if that makes sense

(pager)[44]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy (lies but we're gonna do cgroup2 anyway soon I think)
(pager)[44]: (DEBUG) src/basic/cgroup-util.c:738:cg_pid_get_path: Searching for PID 0 in "/proc/self/cgroup" (controller "_elogind")
(pager)[44]: (DEBUG) src/basic/cgroup-util.c:1543:cg_pid_get_owner_uid: Shifted PID 0 to (null) (result -3)
(pager)[44]: sd_pid_get_owner_uid() failed, enabling pager secure mode: No such process (ESRCH)
posix: Cancel passthrough on fifo WriterFile::handleClose()
loginctl[43]: Pager executable is "less", options "FRSXMK", quit_on_interrupt: yes
loginctl[43]: kanker 0
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:111:bus_socket_auth_needs_write: elogind bus_socket_auth_needs_write for 1
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:115:bus_socket_auth_needs_write: elogind bus_socket_auth_needs_write for 2
loginctl[43]: kanker2 0
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:136:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 1
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:160:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 2
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:621:bus_socket_read_auth: elogind bus_socket_read_auth 1
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:623:bus_socket_read_auth: elogind bus_socket_read_auth 2, k: 58

idk

loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:136:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 1
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:160:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 2
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:173:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 3
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:198:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 4
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:209:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 5
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:222:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 6
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:227:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 7
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:236:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 8
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:242:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 9
loginctl[43]: Bus n/a: changing state AUTHENTICATING → HELLO
loginctl[43]: (DEBUG) src/libelogind/sd-bus/bus-socket.c:249:bus_socket_auth_verify_client: elogind bus_socket_auth_verify_client for 11
loginctl[43]: meme 3: Operation not permitted (EPERM)

the problem is that we EPERM at the end

loginctl[43]: Sent message type=method_call sender=n/a destination=org.freedesktop.DBus path=/org/freedesktop/DBus interface=org.freedesktop.DBus member=Hello cookie=1 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a
loginctl[43]: meme 4: Operation not permitted (EPERM)
loginctl[43]: meme 4: Unknown error code (?)
loginctl[43]: Got message type=method_return sender=org.freedesktop.DBus destination=:1.0 path=n/a interface=n/a member=n/a cookie=1 reply_cookie=1 signature=s error-name=n/a error-message=n/a
loginctl[43]: Bus n/a: changing state HELLO → RUNNING
loginctl[43]: meme 4: Operation not permitted (EPERM)
loginctl[43]: Sent message type=method_call sender=n/a destination=org.freedesktop.login1 path=/org/freedesktop/login1 interface=org.freedesktop.login1.Manager member=ListSessions cookie=2 reply_cookie=0 signature=n/a error-name=n/a error-message=n/a
loginctl[43]: Got message type=error sender=org.freedesktop.DBus destination=:1.0 path=n/a interface=n/a member=n/a cookie=3 reply_cookie=2 signature=s error-name=org.freedesktop.DBus.Error.ServiceUnknown error-message=The name org.freedesktop.login1 was

so that's fucked