Managarm and related projects | Operating System Development | Page 5

trim eagle Jan 6, 2025, 12:07 PM

#

wouldn't be surprised if something in our socket code is just fucked

carmine current Jan 6, 2025, 12:13 PM

#

what's this?

trim eagle Jan 6, 2025, 12:13 PM

#

elogind

#

I want to try and get a proper display manager up and running and they all need elogind or systemd-logind

carmine current Jan 6, 2025, 12:16 PM

#

Does dbus work now btw?

trim eagle Jan 6, 2025, 12:16 PM

#

I think so?

#

at least ish?

#

still not entirely sure

native prairie Jan 6, 2025, 12:18 PM

#

trim eagle I want to try and get a proper display manager up and running and they all need ...

LETSFUCKINGGOOOOOO

reef steeple Jan 6, 2025, 12:19 PM

#

Wayland ones do

quaint fern Jan 6, 2025, 12:28 PM

#

Many of them working fine on bsds without systemd or logind

reef steeple Jan 6, 2025, 12:31 PM

#

I guess that's true

vestal sapphire Jan 6, 2025, 12:37 PM

#

trim eagle wouldn't be surprised if something in our socket code is just fucked

I mean a lot of socket code works fine, including dbus, so I'm not really inclined to blame that

#

haven't really looked into this tho

trim eagle Jan 6, 2025, 12:38 PM

#

yeah never say never but socket code should work but idk

vestal sapphire Jan 6, 2025, 12:39 PM

#

I can now run NVMe admin commands from userspace let's goooooo

trim eagle Jan 6, 2025, 12:40 PM

#

nice

vestal sapphire Jan 6, 2025, 12:41 PM

#

ok ig that's enough for a prelim PR

#

I'll deal with NVMe-oF later

#

if that still boots tho meme

trim eagle Jan 6, 2025, 12:42 PM

#

meme

vestal sapphire Jan 6, 2025, 12:42 PM

#

ah fuck

#

it doesn't

#

christ

trim eagle Jan 6, 2025, 12:45 PM

#

oof

#

I'm not making much progress with elogind here either

vestal sapphire Jan 6, 2025, 12:46 PM

#

minor details, time to also wire up NVMe-oF info to sysfs

trim eagle Jan 6, 2025, 12:46 PM

#

fucking useless dbus elogind piece of shit

trim eagle Jan 6, 2025, 12:46 PM

#

vestal sapphire minor details, time to also wire up NVMe-oF info to sysfs

based

vestal sapphire Jan 6, 2025, 12:46 PM

#

I don't have to do the character devices yet tho

#

there's also the minor detail that NVMe ioctls are absolutely undocumented and the code is unreadable

trim eagle Jan 6, 2025, 12:46 PM

#

meme

#

minor details

carmine current Jan 6, 2025, 12:53 PM

#

vestal sapphire I can now run NVMe admin commands from userspace let's goooooo

does this do a passthrough command?

vestal sapphire Jan 6, 2025, 12:54 PM

#

yeah, it uses the ioctl interface to run commands on the admin queue

#

we should probably keep a whitelist or blacklist of commands for that tho

carmine current Jan 6, 2025, 12:55 PM

#

very nice

vestal sapphire Jan 6, 2025, 12:55 PM

#

given that the point of the interface is to allow for using commands in userspace that are not supported by the driver, the blacklist would be better tbh

#

I suspect the original point was vendor-specific commands, but it's also nice for managing drives in a SAN env

#

where you might have a pretty custom software stack

#

I don't think this is relevant to us tho, as you'd probably want to use io_uring for such a use-case anyways

vestal sapphire Jan 6, 2025, 12:58 PM

#

carmine current does this do a passthrough command?

you can test it on your host, too

#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <sys/types.h>
#include <linux/nvme_ioctl.h>

int main(int argc, char** argv) {
    int fd = open(argv[1], O_RDWR);
    if (fd < 0) {
        perror("open");
        exit(1);
    }

    char buf[4096] = {0};
    struct nvme_admin_cmd mib = {0};
    mib.opcode = 0x6; // identify
    mib.nsid = 0;
    mib.addr = (__u64) buf;
    mib.data_len = sizeof(buf);
    mib.cdw10 = 1; // controller

    int ret = ioctl(fd, NVME_IOCTL_ADMIN_CMD, &mib);
    if (ret) {
        fprintf(stderr, "ioctl failed: %d\n", ret);
        exit(1);
    }

    printf("SN: %.20s\n", &buf[4]);
    printf("SN: %.40s\n", &buf[24]);
    printf("FW: %.8s\n", &buf[64]);
    printf("Status: 0x%04x\n", mib.result);
    return 0;
}

vestal sapphire Jan 6, 2025, 1:44 PM

#

success

carmine current Jan 6, 2025, 1:44 PM

#

very nice

vestal sapphire Jan 6, 2025, 1:47 PM

#

also

#

ok let's goooo

#

I'm not quite sure how to implement some of the attrs yet but details

#

also, how should some of the attrs be queried?

#

for instance, we need to expose stuff like model, serial and a per-queue LBA size attr

#

putting all of that into mbus attrs doesn't make too much sense to me

#

for model and serial you could make the case that you might want to match that with mbus filter, but the rest doesn't quite work

#

esp for capacity data that would change all the time

carmine current Jan 6, 2025, 1:52 PM

#

what does "change all the time" mean?

#

this changes only rarely, right?

#

in principle there are just two options: mbus properties or requests

#

requests probably require more boilerplate than mbus properties

#

so what's the issue with mbus properties?

trim eagle Jan 6, 2025, 1:56 PM

#

hmm I think this is dbus being a dumdum or us launching elogind retardedly, as it's complaining about org.freedesktop.login1 not being provided by any .service files

carmine current Jan 6, 2025, 1:58 PM

#

carmine current in principle there are just two options: mbus properties or requests

maybe we should add some generic sysfs attribute infrastructure? such that mbus properties with names sysfs.foobar are automatically turned into sysfs attributes called foobar?

trim eagle Jan 6, 2025, 2:04 PM

#

busctl gets killed by signal 18 (that's SIGCONT meme )

carmine current Jan 6, 2025, 2:04 PM

#

well, that's something we should implement

#

maybe open an issue

trim eagle Jan 6, 2025, 2:05 PM

#

pager moment, if we run it with --no-pager I get output

#

no login1 on the bus tho

carmine current Jan 6, 2025, 2:09 PM

#

SIGCONT needs a gh issue if there is none already

trim eagle Jan 6, 2025, 2:09 PM

#

yeah I'll open one

carmine current Jan 6, 2025, 2:09 PM

#

SIGSTOP + SIGCONT are relatively easy to implement

#

can be done entirely in posix, without touching the kernel

vestal sapphire Jan 6, 2025, 2:12 PM

#

carmine current so what's the issue with mbus properties?

afaict there are properties that return the "occupied" size of a namespace, so that would be updated often

#

among other things

#

I'm slowly building up the commits here: https://github.com/no92/managarm/tree/nvme-sysfs

GitHub

GitHub - no92/managarm at nvme-sysfs

Pragmatic microkernel-based OS with fully asynchronous I/O - GitHub - no92/managarm at nvme-sysfs

trim eagle Jan 6, 2025, 2:16 PM

#

honestly elogind also feels partially like a cgroup issue

#

or rather, the lack of cgroups

carmine current Jan 6, 2025, 2:20 PM

#

vestal sapphire afaict there are properties that return the "occupied" size of a namespace, so t...

ah. if it's something that changes multiple times per second, it should be a request

#

you could also do a generic "get sysfs attribute" request

#

or integrate it with this idea: #1091278035686268928 message

#

for example: if there is a sysfs.dynamic-attr.foobar, you add a foobar file and do a showSysfsAttribute request when it's read

trim eagle Jan 6, 2025, 2:22 PM

#

trim eagle or rather, the lack of cgroups

relevant logs attached

elogind[41]: (DEBUG) src/login/elogind.c:348:elogind_startup: Daemonizing elogind...
elogind[41]: (DEBUG) src/login/elogind.c:114:elogind_daemonize: Double forking elogind
elogind[41]: (DEBUG) src/login/elogind.c:115:elogind_daemonize: Parent PID     :    41
elogind[41]: (DEBUG) src/login/elogind.c:116:elogind_daemonize: Parent SID     :    38
elogind[41]: Successfully forked off 'elogind-forker' as PID 42.
elogind-forker[42]: (DEBUG) src/login/elogind.c:128:elogind_daemonize: Child PID      :    42
elogind-forker[42]: (DEBUG) src/login/elogind.c:129:elogind_daemonize: Child SID      :    38
elogind-forker[42]: (DEBUG) src/login/elogind.c:146:elogind_daemonize: Child new SID  :    42
elogind-forker[42]: Successfully forked off 'elogind-daemon' as PID 43.
elogind-forker[42]: (DEBUG) src/login/elogind.c:155:elogind_daemonize: Fork 2 returned:     1
elogind-forker[42]: (DEBUG) src/login/elogind.c:156:elogind_daemonize: Grandchild PID :    43
elogind-daemon[43]: (DEBUG) src/login/logind.c:1347:run: Creating manager...
elogind-daemon[43]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy (lies but we're gonna do cgroup2 anyway soon I think)
elogind-daemon[43]: Mounting cgroup2 to /sys/fs/cgroup of type cgroup2 with options nsdelegate,memory_recursiveprot.
elogind-daemon[43]: Failed to mount cgroup2 at /sys/fs/cgroup: No such file or directory (ENOENT)
elogind-daemon[43]: Mounting cgroup2 to /sys/fs/cgroup of type cgroup2 with options nsdelegate.
elogind-daemon[43]: Failed to mount cgroup2 at /sys/fs/cgroup: No such file or directory (ENOENT)
elogind-daemon[43]: Mounting cgroup2 to /sys/fs/cgroup of type cgroup2 with options n/a.
elogind-daemon[43]: Failed to mount cgroup2 at /sys/fs/cgroup: No such file or directory (ENOENT)
elogind-daemon[43]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy (lies but we're gonna do cgroup2 anyway soon I think)
elogind-daemon[43]: Found cgroup2 on /sys/fs/cgroup/, full unified hierarchy (lies but we're gonna do cgroup2 anyway soon I think)
elogind-daemon[43]: Failed to read $container of PID 1, ignoring: No such process (ESRCH)
elogind-daemon[43]: Found container virtualization none.
elogind-daemon[43]: Failed to create '/run/systemd/inaccessible', ignoring: File exists already (EEXIST)
elogind-daemon[43]: Failed to create '/run/systemd/inaccessible/reg', ignoring: Invalid argument (EINVAL)
elogind-daemon[43]: Failed to create '/run/systemd/inaccessible/dir', ignoring: File exists already (EEXIST)
elogind-daemon[43]: Failed to create '/run/systemd/inaccessible/fifo', ignoring: File exists already (EEXIST)
elogind-daemon[43]: Failed to create '/run/systemd/inaccessible/sock', ignoring: File exists already (EEXIST)
elogind-daemon[43]: Failed to create '/run/systemd/inaccessible/chr', ignoring: File exists already (EEXIST)
elogind-daemon[43]: Failed to create '/run/systemd/inaccessible/blk', ignoring: File exists already (EEXIST)
elogind-daemon[43]: (DEBUG) src/basic/cgroup-util.c:738:cg_pid_get_path: Searching for PID 1 in "/proc/1/cgroup" (controller "_elogind")
elogind-daemon[43]: Cannot determine cgroup we are running in: No such process (ESRCH)
elogind-daemon[43]: (DEBUG) src/login/logind.c:124:manager_unref: Tearing down all references (manager_unref) ...
elogind-daemon[43]: Failed to allocate manager object: No such process (ESRCH)

vestal sapphire Jan 6, 2025, 2:22 PM

#

I'm not sure converting all sysfs attributes to a request thing would be beneficial, but it might be nice for some of them

carmine current Jan 6, 2025, 2:22 PM

#

i'm not saying that all sysfs attrs should be requests

#

only the dynamically changing ones

vestal sapphire Jan 6, 2025, 2:22 PM

#

a lot of attributes can be determined from static data by posix, and I'd keep that

carmine current Jan 6, 2025, 2:22 PM

#

or rather, only the very frequently changing ones

#

like multiple changes per second

vestal sapphire Jan 6, 2025, 2:23 PM

#

ig we'd need to keep a list of dynamic attrs per subsys as we need to realizeAttr them

carmine current Jan 6, 2025, 2:23 PM

#

for example, if it's something that only changes as a result of user integration, it's fine to update it in posix

carmine current Jan 6, 2025, 2:24 PM

#

vestal sapphire ig we'd need to keep a list of dynamic attrs per subsys as we need to realizeAtt...

i'd explore the idea of letting driver declare these via mbus properties

#

that sounds easier to extend

vestal sapphire Jan 6, 2025, 2:24 PM

#

hmm might be nice

#

if we put the sysfs side of that in drvcore, then integrating that into sysfs code would mean a single-line diff per subsystem that supports that

#

this might actually be a very convenient solution tbh

#

esp now that we have mbus array types meme

carmine current Jan 6, 2025, 2:28 PM

#

i could imagine mbus properties like

#

sysfs.static.foo = 21
sysfs.static.bar = 42
sysfs.dynamic = [baz]

resulting in 3 files foo, bar, baz where the last one uses a request and the others are just static data

vestal sapphire Jan 6, 2025, 3:40 PM

#

hmm I hit a bit of a problem

#

I may be stupid however

#

but

#

I should attach NVMe-oF controllers under /sys/devices/virtual/nvme-fabrics/ctl/nvmeX, but that should also be a nvme-class device

#

I basically need to inhibi the emission of the nvme in the path before nvmeX

#

how do we solve that

carmine current Jan 6, 2025, 3:42 PM

#

Is that the realpath on linux?

#

Or is nvmeX a symlink?

vestal sapphire Jan 6, 2025, 3:43 PM

#

realpath /sys/class/nvme/nvme1
/sys/devices/virtual/nvme-fabrics/ctl/nvme1

carmine current Jan 6, 2025, 3:43 PM

#

Hmm maybe check how it works in the linux kernel

vestal sapphire Jan 6, 2025, 3:44 PM

#

I tried but I really don't quite understand what's happening there

#

or more like

#

how a nvme-fabrics controller is being exposed over sysfs

#

nvmf_init inits the nvme-fabrics/ctl device tho

#

can't follow it from there really tbh

carmine current Jan 6, 2025, 4:39 PM

#

the logic in linux is: class devices only get a glue directory if the parent device is not also a class device

#

and ctl is a class device

vestal sapphire Jan 6, 2025, 5:06 PM

#

okay I think I implemented that correctly now

#

but booting with NVMe-oF gives me

udevd[22]: worker [34] /devices/pci0000:00/0000:00:02.0 is taking a long time
udevd[22]: worker [27] /class/input/event0 is taking a long time
udevd[22]: worker [33] /class/input/event1 is taking a long time
udevd[22]: worker [31] /class/input/event2 is taking a long time
udevd[22]: worker [30] /class/input/event3 is taking a long time

#

I have no idea what the problem is

#

afaict the NVMe-oF appears as wanted in sysfs and stage2 is entered

#

also after a while

udevd[34]: timeout: killing '/usr/bin/runsvr run /usr/lib/managarm/server/gfx-plainfb.bin' [35]
udevd[34]: slow: '/usr/bin/runsvr run /usr/lib/managarm/server/gfx-plainfb.bin' [35]

#

I'm really not sure why tbf

#

weirdly enough it works on regular NVMe boot

#

ah the problem is probably udevd[22]: sender uid=4280484760, message ignored

#

yeah it was a me issue, as always

#

#

#

let's goooo

#

damn this makes a solid chunk of nvme-cli's commands just work™️

trim eagle Jan 6, 2025, 5:26 PM

#

Based

vestal sapphire Jan 6, 2025, 5:26 PM

#

ig we're missing some rarely used ioctls for stuff like I/O commands (pls don't) and the character devices

#

this ought to be enough for the first PR tho

#

ig the nvme-fabrics char dev would be next

#

after that, I would deem the nvme work good enough for the foreseeable future

#

small diff that only spans 35 files

#

ok, small correction, 33

#

32

upbeat smelt Jan 6, 2025, 5:29 PM

#

damn very nice

trim eagle Jan 6, 2025, 5:32 PM

#

Nice

carmine current Jan 6, 2025, 6:07 PM

#

very nice

carmine current Jan 6, 2025, 6:08 PM

#

vestal sapphire ah the problem is probably `udevd[22]: sender uid=4280484760, message ignored`

how did this happen though?

#

i guess one useful addition would be supporting configfs

#

or wait

#

is that actually needed to connect to nvme-of?

#

or does it use the char device?

vestal sapphire Jan 6, 2025, 6:36 PM

#

carmine current is that actually needed to connect to nvme-of?

no that's only for hosting a controller

vestal sapphire Jan 6, 2025, 6:37 PM

#

carmine current or does it use the char device?

/dev/nvme-fabrics

#

requesting reviews on https://github.com/managarm/managarm/pull/812

GitHub

NVMe sysfs support by no92 · Pull Request #812 · managarm/managarm

vestal sapphire Jan 7, 2025, 12:07 PM

#

okay I'm struggling to figure out what happens to /dev/input/eventX with systemd udev

trim eagle Jan 7, 2025, 12:10 PM

#

Always fun stuff

carmine current Jan 7, 2025, 12:16 PM

#

i still need the relevant patches / branches such that i can debug it meme

#

But my gut feeling is still that it can't be that hard to figure out what happens by logging in posix-subsystem

#

/dev/ is a tmpfs, so adding all tmpfs-related calls (and mount to check that we're not mounting over it) must yield something useful

#

Maybe we should even log the tmpfs readDir calls that we do during ls to check that we're reading from the right tmpfs

vestal sapphire Jan 7, 2025, 12:40 PM

#

I've done request logging, I didn't see any remove/rename/rmdir etc in there

#

the only mounts happening are in stage1

carmine current Jan 7, 2025, 1:03 PM

#

yea, I'm not talking about request logging but about adding logs to tmpfs.cpp

#

These can maybe even be committed (disabled by default through a constexpr bool)

#

If all else fails, we could dump the entire tmpfs on every modification

vestal sapphire Jan 7, 2025, 2:03 PM

#

alright, I'll see

vestal sapphire Jan 7, 2025, 2:37 PM

#

one thing I noticed is that our sysfs representation for event devices is really not what linux does

#

the eventX devices should be under a inputX device afaict

#

both sharing the input subsystem

#

and inputX being a class device of the input class

#

trying to hackily wire that correctly doesn't seem to help either

#

yeah now kmscon doesn't seem to pick up the input devices, great

#

however, of note, eudev seems to somehow keep the char devices around

#

unlike systemd udev

carmine current Jan 7, 2025, 2:55 PM

#

Well, the first question we should answer is not why are the devices deleted but how are they deleted

#

Since last time we looked at this we didn't even know how they get unlinked

vestal sapphire Jan 7, 2025, 3:53 PM

#

even when logging tmp_fs no unlink of eventX takes place

#

there is no matching rmdir, unlink or renameat

trim eagle Jan 7, 2025, 4:06 PM

#

Bruh

carmine current Jan 7, 2025, 4:19 PM

#

carmine current Maybe we should even log the tmpfs readDir calls that we do during `ls` to check...

What if you do this ^

carmine current Jan 7, 2025, 4:20 PM

#

carmine current If all else fails, we could dump the entire tmpfs on every modification

And this ^?

#

I wonder if we're even reading from the right tmpfs

vestal sapphire Jan 7, 2025, 5:06 PM

#

carmine current What if you do this ^

I logged readDir too, it doesn't even call it on /dev/input

trim eagle Jan 7, 2025, 5:06 PM

#

Interesting

vestal sapphire Jan 7, 2025, 5:06 PM

#

honestly I'm feeling a bit stuck rn

carmine current Jan 7, 2025, 5:17 PM

#

Does it call it on dev?

#

Is dev/input somehow replaced by a symlink

vestal sapphire Jan 7, 2025, 5:18 PM

#

ig I can run ls -la /dev after the ps2 driver inits

#

drwxr-xr-x 0 root root        0 Jan  1  1970 char
c--------- 0 root root   1,   7 Jan  1  1970 full
c--------- 0 root root   1, 255 Jan  1  1970 helout
d--------- 0 root root        0 Jan  1  1970 input
c--------- 0 root root   1,  11 Jan  1  1970 kmsg
c--------- 0 root root   1,   3 Jan  1  1970 null
c--------- 0 root root   5,   2 Jan  1  1970 ptmx
d--------- 0 root root        0 Jan  1  1970 pts
c--------- 0 root root   1,   8 Jan  1  1970 random
b--------- 0 root root   8,   0 Jan  1  1970 sda0
b--------- 0 root root   8,   1 Jan  1  1970 sda1
d--------- 0 root root        0 Jan  1  1970 shm
b--------- 0 root root 240,   0 Jan  1  1970 ttyS0
c--------- 0 root root   1,   9 Jan  1  1970 urandom
c--------- 0 root root   1,   5 Jan  1  1970 zero

carmine current Jan 7, 2025, 5:19 PM

#

Maybe check the inode numbers

vestal sapphire Jan 7, 2025, 5:21 PM

#

and ls -la /dev/input/ gives

total 1
drwxr-xr-x 0 root root 0 Jan  1  1970 by-path

#

that is after /dev/input/event0 is installed by posix tho?????????????

#

like right after

#

ig one possible issue is the fact that systemd want a way to resolve an fd to a path

#

and it uses the linux cancer that is /proc/self/fd/

#

I'm wondering if we're messing up stuff there

#

maybe we should just implement a different mechanism for that for now, and path systemd to use that

#

gottem

#

posix: MKDIRAT /dev
posix: FSTATAT request
posix: Fix tmpfs getStats()
mlibc: mkdirat() ignores its mode
posix: MKDIRAT /dev/input
posix: FSTATAT request

#

that is deep in udev code

trim eagle Jan 7, 2025, 5:28 PM

#

vestal sapphire maybe we should just implement a different mechanism for that for now, and path ...

Or we should fix proc/self/fd?

vestal sapphire Jan 7, 2025, 5:28 PM

#

that is way too advanced shit for me

trim eagle Jan 7, 2025, 5:29 PM

#

Understandable

#

But we’re gonna need that anyway

carmine current Jan 7, 2025, 5:35 PM

#

There must be some vfs bug

#

If the device is added to /dev/input but never shows up

#

Hmmm

#

Maybe udevd creates /dev/input before posix creates it and that somehow causes the bug?

#

https://github.com/managarm/managarm/blob/80d02436c3c59f9c4d68e7a6a6c6bd91db8fd8b5/posix/subsystem/src/device.cpp#L96

GitHub

managarm/posix/subsystem/src/device.cpp at 80d02436c3c59f9c4d68e7a6...

Pragmatic microkernel-based OS with fully asynchronous I/O - managarm/managarm

#

This comment looks sus

#

Is this path even executed?

#

This code path

vestal sapphire Jan 7, 2025, 5:53 PM

#

yeah, the input dir is mkdir'ed once

#

when setting up the first eventX

#

it also succeeds

carmine current Jan 7, 2025, 6:00 PM

#

And mkdev is also called? For all events?

vestal sapphire Jan 7, 2025, 6:01 PM

#

yeah, and it succeeds

#

tested by inserting a print after the while(true) loop

carmine current Jan 7, 2025, 6:02 PM

#

could you log whether the dir that device.cpp sees is the same that ls sees?

#

Possibly by logging the tmpfs dir ptr

vestal sapphire Jan 7, 2025, 6:08 PM

#

huh

#

I don't get that

#

maybe that's something for vc

#

ig I could also supply you with my current changes

carmine current Jan 7, 2025, 6:16 PM

#

that'd be nice

#

I can also join vc in an hour or so and we can debug it together

vestal sapphire Jan 7, 2025, 7:16 PM

#

certified mammogram moment

fervent zodiac Jan 7, 2025, 7:29 PM

#

lmao what

vestal sapphire Jan 7, 2025, 11:35 PM

#

okay I don't quite know why but systemd-udev performance is almost acceptable now

vestal sapphire Jan 7, 2025, 11:52 PM

#

https://github.com/no92/managarm/tree/systemd-udevd is the managarm/managarm side of things to get systemd-udev working

GitHub

GitHub - no92/managarm at systemd-udevd

Pragmatic microkernel-based OS with fully asynchronous I/O - GitHub - no92/managarm at systemd-udevd

#

I'll probably break some of the commits out into #760, and keep the rest for a follow up PR

#

the bootstrap and mlibc changes required should then be merged with the second PR ig

#

and our waitid implementation doesn't quite respect the flags we provide, that's annoying

trim eagle Jan 7, 2025, 11:55 PM

#

Implement it?

vestal sapphire Jan 7, 2025, 11:56 PM

#

yeah I'm considering that

vestal sapphire Jan 8, 2025, 12:34 AM

#

ok yeah I didn't implement all of them, but this should probably be a small decrease in logging spam, a small correctness fix and maybe even a microscopic performance gain (didn't benchmark it tho meme )

#

get in here to ~~hate~~ dunk on it https://github.com/no92/managarm/commit/4129c7a4709fa42fa3d89845a81073cf4077c303

GitHub

posix: introduce flags for specifying `Process::wait()` behavior · ...

vestal sapphire Jan 8, 2025, 12:55 AM

#

vestal sapphire https://github.com/no92/managarm/tree/systemd-udevd is the managarm/managarm sid...

https://github.com/no92/mlibc/tree/eudev is the corresponding mlibc side

GitHub

GitHub - no92/mlibc at eudev

Portable C standard library. Contribute to no92/mlibc development by creating an account on GitHub.

#

ok small problem persists ig, weston crashes meme

#

also I can't compile weston for shit

trim eagle Jan 8, 2025, 1:13 AM

#

Yikes

vestal sapphire Jan 8, 2025, 1:14 AM

#

the weston-scanner path is fucked again

vestal sapphire Jan 8, 2025, 1:44 AM

#

the weston patch is basically prog_scanner = find_program('wayland-scanner', dirs: dep_scanner.get_variable(pkgconfig: 'wayland_scanner'))

#

it's retarded on weston's side

#

find_program's first param doesn't take absolute paths, yet they dgaf

vestal sapphire Jan 8, 2025, 2:08 AM

#

vestal sapphire https://github.com/no92/mlibc/tree/eudev is the corresponding mlibc side

https://github.com/managarm/bootstrap-managarm/pull/436 is the bootstrap side

GitHub

systemd-udevd by no92 · Pull Request #436 · managarm/bootstrap-mana...

#

these three branches should be enough to get systemd udev working

#

I really need to offload some stuff from my local bootstrap branches

#

I can't keep rebasing 30+ commits any longer

trim eagle Jan 8, 2025, 3:01 AM

#

PR in what you want; I’ll merge later today or tomorrow if they look good @vestal sapphire

vestal sapphire Jan 8, 2025, 3:01 AM

#

already done

trim eagle Jan 8, 2025, 3:02 AM

#

That one PR was the only thing?

#

Cuz then I’ll look at 435

vestal sapphire Jan 8, 2025, 3:02 AM

#

the rest is WIP

trim eagle Jan 8, 2025, 3:02 AM

#

Noted

vestal sapphire Jan 8, 2025, 3:02 AM

#

you can take a look at the systemd PR too meme

#

but just taking evtest + mtview off the rebase list is a good start too

trim eagle Jan 8, 2025, 3:03 AM

#

No(t now). But I’ll review the qemu one too

vestal sapphire Jan 8, 2025, 3:03 AM

#

the rest of the infra is ModemManager or NetworkManager stacks, WIP GPU stuff, WIP USB passthrough stuff

#

and some random shit, too

trim eagle Jan 8, 2025, 3:06 AM

#

Reviewed and merged some shit

vestal sapphire Jan 8, 2025, 3:07 AM

#

Based

vestal sapphire Jan 8, 2025, 9:37 PM

#

apparently I got iperf3 working

#

cc @trim eagle

#

and with a USB ethernet adapter I get this

upbeat smelt Jan 8, 2025, 9:48 PM

#

vestal sapphire apparently I got iperf3 working

250Mbps on localhost?

#

or is it client on managarm in qemu and server on linux

carmine current Jan 8, 2025, 9:49 PM

#

latter

#

the usb ethernet is over a real network

upbeat smelt Jan 8, 2025, 9:55 PM

#

ah

#

that's pretty good

trim eagle Jan 8, 2025, 10:55 PM

#

That’s good stuff yeah

magic marsh Jan 9, 2025, 12:53 AM

#

ooh, nice
I wonder what's holding them up from saturating the full speed though (I imagine those are gigabit?) thonk

carmine current Jan 9, 2025, 6:55 AM

#

Rn we only submit a single packet to the NIC at a time

#

that's an obvious optimization

trim eagle Jan 10, 2025, 12:42 AM

#

systemd[1]: systemd 256.8-gd8299e3^ running in system mode (-PAM -AUDIT -SELINUX -APPARMOR -IMA -SMACK -SECCOMP -GCRYPT -GNUTLS -OPENSSL -ACL +BLKID -CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC -KMOD -LIBCRYPTSETUP -LIBCRYPTSETUP_PLUGINS -LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -TPM2 -BZIP2 -LZ4 -XZ -ZLIB -ZSTD -BPF_FRAMEWORK -XKBCOMMON -UTMP +SYSVINIT -LIBARCHIVE)
systemd[1]: Hello from systemd Managarm!

#

It’s slowly coming alive

dense umbra Jan 10, 2025, 12:49 AM

#

real

native prairie Jan 10, 2025, 7:32 AM

#

LETSFUCKINGGOOOOOO

serene portal Jan 10, 2025, 10:11 AM

#

real

vestal sapphire Jan 11, 2025, 1:51 PM

#

let's make sure we can have as many plusses as possible

carmine current Jan 11, 2025, 1:52 PM

#

let's first make sure that it actually works :^)

vestal sapphire Jan 11, 2025, 1:53 PM

#

I'll look at the udev PR tomorrow

trim eagle Jan 11, 2025, 1:54 PM

#

vestal sapphire let's make sure we can have as many plusses as possible

Yes, but not now

carmine current Jan 11, 2025, 1:56 PM

#

I think the udev PR should be broken up

#

some parts are mature and can be merged / reviewed right away, some definitely need some work

trim eagle Jan 11, 2025, 1:56 PM

#

It’s also actively in progress as I’m pushing work for systemd boot to it

#

So yes; we should

#

Same goes for the mlibc side (not a PR yet)

#

Or the bootstrap part

carmine current Jan 11, 2025, 1:57 PM

#

for example, the socket changes can go to their own PR

#

maybe: socket changes for udevd / other changes for udevd (two separate PRs) -> changes for systemd (separate PR as well)

trim eagle Jan 11, 2025, 1:58 PM

#

Yeah that’s likely better

#

I should try elogind too with the socket changes btw

vestal sapphire Jan 11, 2025, 1:59 PM

#

carmine current some parts are mature and can be merged / reviewed right away, some definitely n...

So rather 3-4 smaller PRs?

trim eagle Jan 11, 2025, 1:59 PM

#

I have a feeling the socket memes fixed whatever fucked that

vestal sapphire Jan 11, 2025, 1:59 PM

#

Might be

trim eagle Jan 11, 2025, 2:00 PM

#

In which case KDE might be on the menu

#

(KDE proper that is)

vestal sapphire Jan 11, 2025, 2:00 PM

#

In general we fixed a whole bunch of virtually undebuggable memes, it might be worthwhile to retest a bunch of ports

#

Not just for the udev thing

native prairie Jan 11, 2025, 2:01 PM

#

trim eagle I have a feeling the socket memes fixed whatever fucked that

What was the bug?

trim eagle Jan 11, 2025, 2:01 PM

#

Bunch of fixes in general, including some netlink fixes and CMSG truncation fixes and support

native prairie Jan 11, 2025, 2:02 PM

#

Nice

#

LETSFUCKINGGOOOOOO kde support

trim eagle Jan 11, 2025, 2:02 PM

#

Hmm. Do I start CGroups next week?

#

Or do I switch to elogind

#

And get stuff like SDDM and KDE?

carmine current Jan 11, 2025, 2:05 PM

#

I think top prio should be getting systemd-udevd in

trim eagle Jan 11, 2025, 2:06 PM

#

That can be done if we split out the PRs and address some bootstrap comments

vestal sapphire Jan 11, 2025, 2:07 PM

#

trim eagle Bunch of fixes in general, including some netlink fixes and CMSG truncation fixe...

Honestly I wouldn't even be surprised if KDE hangs were related to that

#

I'll be home tomorrow

trim eagle Jan 11, 2025, 2:08 PM

#

The issue with KDE was elogind not working

#

But that might work now

serene portal Jan 11, 2025, 2:58 PM

#

vestal sapphire let's make sure we can have as many plusses as possible

selinux trl

trim eagle Jan 11, 2025, 3:08 PM

#

serene portal selinux <:trl:829316641191886888>

AppArmor, smack, seccomp, libcryptsetup moment

#

And no, PAM isn’t that hard, locally I have it

#

So that will turn into a plus I think

vestal sapphire Jan 11, 2025, 5:13 PM

#

Honestly seccomp sounds like a useful idea

trim eagle Jan 11, 2025, 5:13 PM

#

Sure does

#

But not now lol

vestal sapphire Jan 11, 2025, 5:13 PM

#

Especially given that I see quite the overlap with implementing strace stuff

vestal sapphire Jan 11, 2025, 5:14 PM

#

serene portal selinux <:trl:829316641191886888>

That needs a special netlink protocol on top of whatever you need in terms of functionality

#

Let's just say no

serene portal Jan 11, 2025, 5:14 PM

#

chad

trim eagle Jan 11, 2025, 5:14 PM

#

Maybe one day. Never say never

vestal sapphire Jan 11, 2025, 5:15 PM

#

Now that I think about it, I should really start implementing strace stuff

#

It's waaaay too useful to me to just not do it

trim eagle Jan 11, 2025, 5:16 PM

#

True

vestal sapphire Jan 11, 2025, 5:16 PM

#

Another idea I had is to write a small utility that enables POSIX request logging for the invoked program

#

So you could call something like posix-requests ls and get request logging for ls

#

That would mean some more complexity on the POSIX end but shouldn't be too bad

#

And even in general having a way to enable request logging by patching a port would be super useful

#

We could just have a header under /usr/include/managarm/ for that

trim eagle Jan 11, 2025, 5:41 PM

#

Sounds good

carmine current Jan 11, 2025, 5:46 PM

#

I don't see how you'd do strace without a lot of effort

#

and I don't think strace and seccomp overlap either

vestal sapphire Jan 11, 2025, 8:17 PM

#

I mean having hooks placed at the requests goes a long way into integrating both features

carmine current Jan 11, 2025, 9:21 PM

#

for strace you probably want the hooks in mlibc though and for seccomp you want them in posix/other servers

carmine current Jan 12, 2025, 8:49 PM

#

CPU #3 enters load balancing
CPU #3 has load 630
CPU #2 enters load balancing
CPU #2 has load 633
CPU #1 enters load balancing
CPU #1 has load 585
CPU #0 enters load balancing
CPU #0 has load 600
Total system load is 2448 (ideal load: 612)
Moving thread with load 40 from CPU 2 to CPU 1
thor: 0xffffc000024b4030 is moved to CPU 1

Load balancing PR incoming for Managarm 😄

native prairie Jan 12, 2025, 8:50 PM

#

damn nice

trim eagle Jan 12, 2025, 8:50 PM

#

Pog

wild stone Jan 12, 2025, 8:59 PM

#

carmine current ``` CPU #3 enters load balancing CPU #3 has load 630 CPU #2 enters load balancin...

yo that looks far more advanced than my lost copy of when i tried it lol

#

i just kinda randomly pulled threads from cpus with low idle time to ones with high idle time

#

at least thats what i think i recall doing

carmine current Jan 12, 2025, 9:00 PM

#

it's not very complicated though

#

it did requrie a bit of design work but the actual implementation is quite short

wild stone Jan 12, 2025, 9:00 PM

#

yeah but you have skill and i do not

#

mine was a turbo bodge lol

#

i recall there being reasons i never pr'd it in although i cant remember what they were

#

i think one was it randomly shitting itself and hanging? cant remember

trim eagle Jan 12, 2025, 11:57 PM

#

carmine current ``` CPU #3 enters load balancing CPU #3 has load 630 CPU #2 enters load balancin...

Did you by chance try zlib -j4 with this? I know ar will fuck us but still

carmine current Jan 13, 2025, 9:27 AM

#

No, I could try that once it actually works meme

vestal sapphire Jan 13, 2025, 4:44 PM

#

hmm, systemd seems to only use malloc_info in two places, and musl doesn't have it

#

instead, it patches them out

#

maybe we should do that, too?

carmine current Jan 13, 2025, 4:46 PM

#

if there is a working patch, can we just re-use that one?

vestal sapphire Jan 13, 2025, 4:47 PM

#

it's a+5 diff

#

so I'm dropping the malloc_info stub then

trim eagle Jan 13, 2025, 5:25 PM

#

Sounds good

trim eagle Jan 13, 2025, 5:26 PM

#

carmine current No, I could try that once it actually works <:meme:575445094589661215>

Fuck. That’s more getopt debugging (or I’ll try the getopt patch as is while booting systemd-udevd and see if that boots lol)

vestal sapphire Jan 13, 2025, 8:53 PM

#

ah okay

#

the issue for weston seems to be that we don't support O_NOFOLLOW for openat yet

#

OOOOOOOOOOOOOH it's the combination of O_NOFOLLOW and O_PATH

trim eagle Jan 13, 2025, 9:32 PM

#

Oof

vestal sapphire Jan 13, 2025, 10:10 PM

#

ok I have more bad news

#

I have now implemented both O_NOFOLLOW and O_DIRECTORY and not even kmscon boots with this

#

what seems to happen is that the rules don't get executed??????????

#

I have no clue why they aren't tho

trim eagle Jan 13, 2025, 10:12 PM

#

Bruh???

carmine current Jan 13, 2025, 10:13 PM

#

What paths are actually opened using nofollow?

vestal sapphire Jan 13, 2025, 10:15 PM

#

no that many

#

only sysfs stuff afaict

#

given that commenting the O_NOFOLLOW flag stuff makes it work again I'll assume it expects something to be a symlink that isn't?

#

not sure about it

carmine current Jan 13, 2025, 10:17 PM

#

Do we ever return ELOOP?

vestal sapphire Jan 13, 2025, 10:17 PM

#

no

carmine current Jan 13, 2025, 10:17 PM

#

vestal sapphire given that commenting the O_NOFOLLOW flag stuff makes it work again I'll assume...

but then it'd have been broken before right?

vestal sapphire Jan 13, 2025, 10:17 PM

#

worked by coincidence ig

carmine current Jan 13, 2025, 10:18 PM

#

No, i suspect that it's not that it expects ELOOP somewhere but rather that something else is behaving differently now

#

since we never returned ELOOP before, missing ELOOP cannot be the problem

#

Maybe O_PATH now sometimes refers to a different file and that breaks stuff?

#

What files are opened using O_PATH?

vestal sapphire Jan 13, 2025, 10:24 PM

#

just sysfs stuff

#

if does openat("/sys", O_PATH|...)

#

then openat("/sys/devices", O_PATH|...)

#

etc

#

until it just stops, and that's it

carmine current Jan 13, 2025, 10:28 PM

#

Where does it stop?

#

Does it also pass nofollow here?

vestal sapphire Jan 13, 2025, 10:28 PM

#

not all the time

vestal sapphire Jan 13, 2025, 10:29 PM

#

carmine current Where does it stop?

it hangs when it doesn't pick up the DRM device

#

but that is because the udev rules are never run

#

by that I mean no server gets started

#

dunno why

carmine current Jan 13, 2025, 10:30 PM

#

Does it walk the entire sysfs though?

vestal sapphire Jan 13, 2025, 10:30 PM

#

once, yes

#

but it doesn't do a second pass

#

ok I have a function that's failing unexpectedly

trim eagle Jan 13, 2025, 10:40 PM

#

The plot thickens

vestal sapphire Jan 13, 2025, 10:44 PM

#

the friggin chaseat function is the bane of my existance right now

#

jfc I hate this so much

trim eagle Jan 13, 2025, 10:49 PM

#

Understandable

vestal sapphire Jan 13, 2025, 11:01 PM

#

oh it comes back to fail in the ... openat?

trim eagle Jan 13, 2025, 11:01 PM

#

Bruh?

vestal sapphire Jan 13, 2025, 11:03 PM

#

I'm as confused as you

#

I have a bad feeling it's trying to do a pass through /proc/self/fd/

#

ok good, it doesn't

#

phew

trim eagle Jan 13, 2025, 11:08 PM

#

Yeah that would’ve been bad™️

vestal sapphire Jan 13, 2025, 11:08 PM

#

I'm literally being premium trolled rn

#

no matter what I do, my fprintf is not changed

trim eagle Jan 13, 2025, 11:10 PM

#

A classic

vestal sapphire Jan 13, 2025, 11:27 PM

#

ah I think what's happening here is that it tries to openat paths that are ".."

#

and somehow that fails

#

unsure why and how tho

trim eagle Jan 13, 2025, 11:33 PM

#

Fuck more vfs path memes

#

I ran into items related to “..” paths in KDE kcalc recently too

vestal sapphire Jan 13, 2025, 11:34 PM

#

oh boy

#

I hate vfs code with a passion ngl

vestal sapphire Jan 14, 2025, 12:06 AM

#

tldr seems to be

#

udev does an fd1 = openat(fd0, "..", <flags>), followed by fd2 = openat(fd1, "..", <flags>)

#

however, printing the relative_to paths in the openat requests.cpp revleas that fd0 and fd1 are treated the same

vestal sapphire Jan 14, 2025, 12:47 AM

#

ok I am not so sure about that story any more

vestal sapphire Jan 14, 2025, 1:05 AM

#

ok I have found a solution

#

it's not a good one

#

anyways I have weston with systemd-udev

#

for a terminal I need a treeLink impl for /dev/pts/0 tho

#

I'd be glad if someone could help me out with that

#

VFS code is not my strong point

carmine current Jan 14, 2025, 9:16 AM

#

treeLink() just returns a directory's own directory entry

#

so it's needs to return the pts::Link https://github.com/managarm/managarm/blob/master/posix/subsystem/src/pts.cpp#L470 that points to itself

GitHub

managarm/posix/subsystem/src/pts.cpp at master · managarm/managarm

Pragmatic microkernel-based OS with fully asynchronous I/O - managarm/managarm

#

wait, /dev/pts/0 is not a directory, so it doesn't need to implement treeLink()

#

treeLink() should only ever be called on directories. so it can be called on /dev/pts but not /dev/pts/0

vestal sapphire Jan 14, 2025, 11:32 AM

#

hmm it might be called from ViewPath's getPath?

#

not sure

#

might be due to some debug logging evn

#

indeed

carmine current Jan 14, 2025, 11:33 AM

#

callers of treeLink() should first check that the node is a directory

#

if getPath() doesn't do that, it's a bug

vestal sapphire Jan 14, 2025, 11:33 AM

#

alright

#

maybe that should be documented somewhere, that would be super duper helpful

#

along with some of the other VFS concepts, it's kinda hard to work with if you don't have a good mental model (like me) or a reference

#

same goes for the PathResolver flags ig

carmine current Jan 14, 2025, 11:34 AM

#

You can add a comment to treeLink() meme

vestal sapphire Jan 14, 2025, 11:35 AM

#

anyways, the udev not working with weston meme seems to be that it compares inode numbers for dirfds, however we report 0 in sysfs meme

carmine current Jan 14, 2025, 11:35 AM

#

but actually getPath only calls this on FsLink::getOwner() which should always be a directory

vestal sapphire Jan 14, 2025, 11:35 AM

#

ah ok

carmine current Jan 14, 2025, 11:36 AM

#

are you sure if this crash is caused by getPath() calling treeLink() on /dev/pts/0?

#

because that'd mean that we have a FsLink that returns a non-directory a containing directory meme

vestal sapphire Jan 14, 2025, 11:36 AM

#

vestal sapphire Jan 14, 2025, 11:37 AM

#

carmine current are you sure if this crash is caused by getPath() calling treeLink() on /dev/pts...

I'm not sure what it was exactly, but I just commented out a bunch of my debugging logs and that made the meme go away

#

it failed with an uncaught exception, so you'd notice if you ever run into that

carmine current Jan 14, 2025, 11:38 AM

#

alright

#

did these debug logs just do getPath()?

trim eagle Jan 14, 2025, 11:38 AM

#

Nice work Leo on the terminal

carmine current Jan 14, 2025, 11:38 AM

#

nice that works now 😄

vestal sapphire Jan 14, 2025, 11:38 AM

#

carmine current did these debug logs just do getPath()?

more than that but that seemed like the obv source of calls to treeLink()

carmine current Jan 14, 2025, 11:39 AM

#

getPath() should just work on all paths though

#

so maybe it'd be good to find out which debug log breaks it

#

or to get a stack trace

vestal sapphire Jan 14, 2025, 11:42 AM

#

and it broke again ...

carmine current Jan 14, 2025, 11:42 AM

#

and open an issue

vestal sapphire Jan 14, 2025, 11:42 AM

#

and it works again

#

wtf

#

it was a git add -p meme where it had the hunk diff be weird lol

#

yooo let's goo

#

ok but how would we allocate inode numbers for sysfs?

#

how do we allocate inodes here then? just have an id_allocator?

carmine current Jan 14, 2025, 11:55 AM

#

yes

vestal sapphire Jan 14, 2025, 12:09 PM

#

yoooooooo it works

#

okay time to make a nice commit out of this then

#

letsgo

trim eagle Jan 14, 2025, 12:16 PM

#

Based

vestal sapphire Jan 14, 2025, 2:05 PM

#

vestal sapphire how do we allocate inodes here then? just have an `id_allocator`?

ig we also free them in the destructor then?

carmine current Jan 14, 2025, 2:10 PM

#

yes

#

While we're at it, we can also do this for SpecialLink::EmbeddedNode

vestal sapphire Jan 14, 2025, 2:16 PM

#

for sysfs, I can just place the inode allocator in the superblock I assume?

#

ig we can do the same for the EmbeddedNode stuff too

#

that currently reports a hardcoded inode num of 1

carmine current Jan 14, 2025, 2:20 PM

#

yes, placing it in the superblock makes sense

vestal sapphire Jan 14, 2025, 2:43 PM

#

https://github.com/managarm/managarm/pull/820 contains the fs memes

GitHub

systemd-udevd filesystem fixes by no92 · Pull Request #820 · managa...

vestal sapphire Jan 14, 2025, 2:43 PM

#

carmine current While we're at it, we can also do this for SpecialLink::EmbeddedNode

I haven't done that yet, not sure if I get to that today

carmine current Jan 14, 2025, 2:50 PM

#

I'll review it in the evening

#

Btw why is this thread now used for Managarm development? :^)

vestal sapphire Jan 14, 2025, 3:06 PM

#

there was a lot of other chat going on in voice-0 so I just switched here at some point lol

#

I just didn't think much of it

#

about the CMSG truncation thing, I'll probably write another helper for writing out truncated data

#

so that you could do something like this:

        auto [truncated, remaining_space] = ctrl.message(SOL_NETLINK, NETLINK_PKTINFO, sizeof(info));
        remaining_space -= ctrl.write_truncated(info, truncated, remaining_space);

carmine current Jan 14, 2025, 3:13 PM

#

vestal sapphire there was a lot of other chat going on in voice-0 so I just switched here at som...

Sure but there's also the Managarm server :^)

vestal sapphire Jan 14, 2025, 3:13 PM

#

switching servers is effort

#

meme

serene portal Jan 14, 2025, 3:18 PM

#

vestal sapphire there was a lot of other chat going on in voice-0 so I just switched here at som...

also funny that managarm dev happens in voice-0 meme

vestal sapphire Jan 14, 2025, 3:19 PM

#

hmm I'm not sure about how CMSG truncation works any more

#

I used to think that the cmsg headers are only placed as a whole, but the data might be truncated

#

but now I think they are treated as an inseperable unit, and the MSG_CTRUNC flag is set if not all CMSGs fit in the receive buffer for them

#

I'm not 100 % sure which way it is tho

#

man 7 unix seems to suggest that some CMSG types get their data truncated, but others are written as whole?

carmine current Jan 14, 2025, 3:29 PM

#

This is easy to test on a linux host though, isn't it?

vestal sapphire Jan 14, 2025, 3:29 PM

#

yeah but I gtg now, I'll pick this back up tomorrow

#

from what I can tell though, headers are assumed to be complete or missing

#

ig the details depend on how the CMSG_* macros work

#

but CMSG_NXTHDR should return a nullptr for a truncated (even if partially copied) header

#

the one exception being the list thingies that are explicitly stated to drop parts of the list, which basically amounts to the list being truncated if the cmsghdr fits

#

but that just reports an appropriate CMSG_LEN, which a regular header would not

#

even if we partially copy descriptors in, the CMSG_LEN should state the regular length so that CMSG_NXTHDR picks up on it being incomplete

#

if a cmsghdr would only partially fit, it should not be copied over, and CMSG_NXTHDR should pick up on the fact that the remaining space in the control buffer is insufficiient for a cmsghdr, and return a nullptr

#

okay I think I have this sorted out

#

my currentt impl isn't entirelly correct but it's close enough ig

#

I just shouldn't copy in partial cmsg_data (except for the above lists) and keep the CMSG_LEN to what it should be, not what would fit

carmine current Jan 14, 2025, 3:42 PM

#

How does SCM_RIGHTS interact with truncation and MSG_PEEK?

#

In particular, when are the fds installed?

#

This should probably be checked using a linux test program

carmine current Jan 14, 2025, 3:44 PM

#

carmine current How does SCM_RIGHTS interact with truncation and MSG_PEEK?

Maybe @quaint fern knows

vestal sapphire Jan 14, 2025, 3:44 PM

#

#

not sure about MSG_PEEK though

quaint fern Jan 14, 2025, 3:45 PM

#

carmine current Maybe <@391791686881247234> knows

until fairly recently there was a bug on linux where some FDs could be leaked in this case, and the bsds were also problematic in some way or another

vestal sapphire Jan 14, 2025, 3:47 PM

#

vestal sapphire

worst case is that we'd need to cache the fds from when we attached them during a MSG_PEEK call

#

hmm no, by then the data would have been just discarded nvm

#

our current logic should be simplle to adapt to this ig

#

something like this should work?

        if(!packet->files.empty()) {
            auto [truncated, payload_len] = ctrl.message(SOL_SOCKET, SCM_RIGHTS, sizeof(int) * packet->files.size());
            for(auto &file : packet->files) {
                if(payload_len < sizeof(int))
                    break;

                ctrl.write<int>(process->fileContext()->attachFile(std::move(file), flags & MSG_CMSG_CLOEXEC));
                payload_len -= sizeof(int);
            }

            packet->files.clear();
        }

#

hmm weston doesn't like this

carmine current Jan 14, 2025, 3:59 PM

#

Apparently MSG_PEEK installs the fds and recv installs them again?

#

This should be double checked by compiling a test program though

#

But it seems to be what the linux code does

vestal sapphire Jan 14, 2025, 4:00 PM

#

cursed

carmine current Jan 14, 2025, 4:01 PM

#

Which means that peek + recv could return different fd numbers afaict

fervent zodiac Jan 14, 2025, 4:01 PM

#

holy shit what

carmine current Jan 14, 2025, 4:02 PM

#

https://elixir.bootlin.com/linux/v6.12.6/source/net/unix/af_unix.c#L2498

af_unix.c - net/unix/af_unix.c - Linux source code v...

Elixir Cross Referencer - source code of Linux v6.12.6: net/unix/af_unix.c

vestal sapphire Jan 14, 2025, 4:03 PM

#

        if(!packet->files.empty()) {
            auto [truncated, payload_len] = ctrl.message_truncated(SOL_SOCKET, SCM_RIGHTS, sizeof(int) * packet->files.size());
            for(auto &file : packet->files) {
                if(truncated && payload_len < sizeof(int))
                    break;

                ctrl.write<int>(process->fileContext()->attachFile(std::move(file), flags & MSG_CMSG_CLOEXEC));
                payload_len -= sizeof(int);
            }

            if(truncated)
                reply_flags |= MSG_CTRUNC;

            if(!(flags & MSG_PEEK))
                packet->files.clear();
        }

#

works with weston™️

#

where message_truncated sets the correct CMSG_LEN for the truncated fd list

#

anyways see you tomorrow

vestal sapphire Jan 14, 2025, 4:56 PM

#

vestal sapphire ```c++ if(!packet->files.empty()) { auto [truncated, payload...

Until tomorrow, does this impl look reasonable?

trim eagle Jan 14, 2025, 10:41 PM

#

@carmine current did some digging in the statx man page

stx_mnt_id
              The mount ID of the mount containing the file.  This is the same number reported by name_to_handle_at(2) and corresponds to the number in the first field in one of the records in
              /proc/self/mountinfo.

so I guess we should also implement /proc/self/mountinfo? That was something on the TODO list for me anyway so not a really big detour (and gives us working mount calls to list all mounts which is nice)

carmine current Jan 14, 2025, 10:45 PM

#

Does systemd use that field?

trim eagle Jan 14, 2025, 10:45 PM

#

the mount id?

#

yes

#

and it would eliminate another hack in dev_is_devtmpfs

#

which parses mountinfo

trim eagle Jan 14, 2025, 11:28 PM

#

trim eagle the mount id?

I’ll be rechecking that, I might’ve mislooked and they might use a different field

trim eagle Jan 15, 2025, 2:44 AM

#

Looks like it’s way simpler

#

So scratch that field

#

I think it’s just the type field

#

Easy stuff

carmine current Jan 15, 2025, 7:20 AM

#

yeah that can be implemented entirely in terms of FsNode::getStats() I guess

#

We should probably still add a statx request such that it's easier to extend in the future

trim eagle Jan 15, 2025, 10:26 AM

#

Yep

#

But I fully plan on lifting logic from getStats here

carmine current Jan 15, 2025, 10:36 AM

#

Btw, can you promote the last libasync version into stable?

#

Then I can restart the GHA ci on my PR

trim eagle Jan 15, 2025, 10:41 AM

#

On it

#

Done

vestal sapphire Jan 15, 2025, 10:52 AM

#

trim eagle <@358943271151468555> did some digging in the statx man page ``` stx_mnt_id ...

Our name to handle at is stubbed with ENOSYS tho

#

At least in my udev work

trim eagle Jan 15, 2025, 10:53 AM

#

vestal sapphire Our name to handle at is stubbed with ENOSYS tho

Same here

#

But statx is picked over name_to_handle_at

#

At least here

carmine current Jan 15, 2025, 10:55 AM

#

statx is way easier to implement than name_to_handle_at

vestal sapphire Jan 15, 2025, 10:55 AM

#

Yeah I hope

carmine current Jan 15, 2025, 10:55 AM

#

statx is basically just stat() with a couple extra fields

vestal sapphire Jan 15, 2025, 10:56 AM

#

But the fact that they are linked is annoying tbh

carmine current Jan 15, 2025, 10:56 AM

#

while name_to_handle_at needs special handling in all fses that support it

carmine current Jan 15, 2025, 10:56 AM

#

vestal sapphire But the fact that they are linked is annoying tbh

Linked?

vestal sapphire Jan 15, 2025, 10:56 AM

#

Field corresponds etc

vestal sapphire Jan 15, 2025, 10:57 AM

#

trim eagle <@358943271151468555> did some digging in the statx man page ``` stx_mnt_id ...

Referring to this

trim eagle Jan 15, 2025, 10:57 AM

#

Yeah that’s meh

#

But we’ll need that field if we want proc self mountinfo anyway

#

And we do

#

So I don’t care about name to handle at there for now

carmine current Jan 15, 2025, 10:58 AM

#

Well it's just that some of the data that they return is identical

trim eagle Jan 15, 2025, 10:58 AM

#

Yep

carmine current Jan 15, 2025, 10:58 AM

#

But they don't depend on each other

trim eagle Jan 15, 2025, 10:59 AM

#

Thank god for that

carmine current Jan 15, 2025, 11:04 AM

#

Hmmm i fear that for correct handling of the mount point attribute of statx() we need to change the type that associatedLink() returns to ViewPath instead of just FsLink

#

which is not terribly difficult but a lot of search and replace meme

#

#userland message at least if the answer to this question is "yes"

trim eagle Jan 15, 2025, 11:08 AM

#

carmine current which is not terribly difficult but a lot of search and replace <:meme:575445094...

sadmeme

reef steeple Jan 15, 2025, 11:35 AM

#

couldnt you just add the statx fields to your default stat and thats it, maybe make some sort of frankestein of both and do some processing in software

#

not even necessarily talking managarm, just like, in general

carmine current Jan 15, 2025, 11:47 AM

#

yeah

#

that's probably the ways to go

vestal sapphire Jan 15, 2025, 6:14 PM

#

okay let's gooooo

trim eagle Jan 15, 2025, 6:51 PM

#

Pog

vestal sapphire Jan 15, 2025, 10:50 PM

#

managarm, booted into weston, using systemd-udev, booted compeltely diskless with PXE and NVMe-over-fabrics

#

cc @trim eagle

dense umbra Jan 15, 2025, 10:52 PM

#

2025 is the year of the managarm desktop

#

~~eoy blog post when~~

fervent zodiac Jan 15, 2025, 10:57 PM

#

vestal sapphire managarm, booted into weston, using systemd-udev, booted compeltely diskless wit...

no btop? /j

#

anyway very very cool

#

highly based

#

now do iSCSI

#

or nvme over infiniband

vestal sapphire Jan 15, 2025, 10:59 PM

#

fervent zodiac or nvme over infiniband

that is a consideration ngl

#

iSCSI is effort so no

fervent zodiac Jan 15, 2025, 11:00 PM

#

NVMEoIB is a lot more effort

native prairie Jan 15, 2025, 11:01 PM

#

vestal sapphire managarm, booted into weston, using systemd-udev, booted compeltely diskless wit...

Insane

#

Are we close to kde

vestal sapphire Jan 15, 2025, 11:08 PM

#

fervent zodiac NVMEoIB is a lot more effort

yeah but I don't have to implement the entire command set

#

just the RDMA transport binding spec and mellanox drivers meme

vestal sapphire Jan 15, 2025, 11:09 PM

#

fervent zodiac no btop? /j

jokes on you, I'm trying to port it rn

trim eagle Jan 15, 2025, 11:12 PM

#

dense umbra 2025 is the year of the managarm desktop

Yeah yeah

trim eagle Jan 15, 2025, 11:13 PM

#

fervent zodiac no btop? /j

That’s probably rookie shit

native prairie Jan 15, 2025, 11:13 PM

#

dense umbra 2025 is the year of the managarm desktop

Only after a chromium port trl

vestal sapphire Jan 15, 2025, 11:17 PM

#

vestal sapphire jokes on you, I'm trying to port it rn

ok it fails due to no /proc/stat support

trim eagle Jan 15, 2025, 11:18 PM

#

native prairie Only after a chromium port <:trl:829316641191886888>

DONT FUCKING TEMPT ME!

trim eagle Jan 15, 2025, 11:18 PM

#

vestal sapphire ok it fails due to no `/proc/stat` support

Noted

fervent zodiac Jan 15, 2025, 11:18 PM

#

vestal sapphire ok it fails due to no `/proc/stat` support

just make it always return ```
cpu 10710711 4467 3001155 80734813 26252 614687 137502 0 0 0
cpu0 1830304 362 418882 9279482 3131 263737 60564 0 0 0
cpu1 2182626 716 389194 9172304 4945 91249 22156 0 0 0
cpu2 2129518 528 392976 9229754 5084 89892 17169 0 0 0
cpu3 2074664 473 388940 9294103 4673 88469 25097 0 0 0
cpu4 826870 937 399498 10673104 2525 25826 5777 0 0 0
cpu5 557527 535 215947 11133139 2197 24475 2518 0 0 0
cpu6 615486 492 394391 10926874 1985 12031 1567 0 0 0
cpu7 493714 419 401322 11026051 1709 19005 2652 0 0 0
intr 1150324246 0 60882860 494004957 0 0 0 178571 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 369216291 0 67889441 154 404182 0 0 0 0 0 0 0 3825 0 28242241 0 4296883 0 0 0 0 7122515 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 50140 0 0 0 0 0 0 0 0 0 37702310 0 0 0 0 0 0 564 72560952 5672 29941 5672 0 4485389 60 0 0 0 0 0 0 7045 0 90 286 0 3233613 592
ctxt 1565363764
btime 1736630431
processes 412261
procs_running 2
procs_blocked 0
softirq 305882733 1556 23603978 338 12657688 3804303 0 38122174 107634865 4393 120053438

#

what even is correctness

#

this is the real managarm style

#

hardcode random values unrelated to reality

vestal sapphire Jan 15, 2025, 11:19 PM

#

we can prob provide reasonable values for that even

#

0=user, 1=nice, 2=system, 3=idle, 4=iowait, 5=irq, 6=softirq, 7=steal, 8=guest, 9=guest_nice

trim eagle Jan 15, 2025, 11:19 PM

#

We probably can

#

I only added proc/self/stat

#

Not global stat

#

But shouldn’t be too hard

vestal sapphire Jan 15, 2025, 11:20 PM

#

but yeah, porting btop seems easy on the porting side

#

it wants a few things we don't have implemented yet tho

trim eagle Jan 15, 2025, 11:20 PM

#

It has been added to the list for tomorrow
W

vestal sapphire Jan 15, 2025, 11:20 PM

#

like getloadavg and pthread_timedjoin_np

#

and /proc/stat

trim eagle Jan 15, 2025, 11:21 PM

#

VC tomorrow is guaranteed good, even earlier than before

dense umbra Jan 15, 2025, 11:21 PM

#

trim eagle DONT FUCKING TEMPT ME!

do it no balls

fervent zodiac Jan 15, 2025, 11:21 PM

#

you can patch out getloadavg

#

i think

vestal sapphire Jan 15, 2025, 11:21 PM

#

  - name: btop
    labels: [aarch64, riscv64]
    architecture: '@OPTION:arch@'
    source:
      subdir: ports
      git: 'https://github.com/aristocratos/btop.git'
      tag: 'v1.4.0'
      version: '1.4.0'
    tools_required:
      - host-cmake
      - system-gcc
    pkgs_required:
      - mlibc
    revision: 1
    configure:
      - args:
        - 'cmake'
        - '-GNinja'
        - '-DCMAKE_TOOLCHAIN_FILE=@SOURCE_ROOT@/scripts/CMakeToolchain-@OPTION:[email protected]'
        - '-DCMAKE_INSTALL_PREFIX=/usr'
        - '-DBTOP_GPU=false'
        - '@THIS_SOURCE_DIR@'
    build:
      - args: ['ninja']
      - args: ['ninja', 'install']
        environ:
          DESTDIR: '@THIS_COLLECT_DIR@'
        quiet: true

trim eagle Jan 15, 2025, 11:21 PM

#

dense umbra do it no balls

Been there done that

fervent zodiac Jan 15, 2025, 11:21 PM

#

its part of the linux backend

trim eagle Jan 15, 2025, 11:21 PM

#

Lemme find the picture

vestal sapphire Jan 15, 2025, 11:21 PM

#

fervent zodiac you can patch out getloadavg

I just stubbed it to return 0

fervent zodiac Jan 15, 2025, 11:21 PM

#

lmfao

vestal sapphire Jan 15, 2025, 11:21 PM

#

which is how I got to the proc memes

fervent zodiac Jan 15, 2025, 11:21 PM

#

the managarm way

serene portal Jan 15, 2025, 11:22 PM

#

fervent zodiac the managarm way

where's ur udev

fervent zodiac Jan 15, 2025, 11:22 PM

#

i dont have udev :^)

#

or anything else

fervent zodiac Jan 15, 2025, 11:22 PM

#

vestal sapphire like `getloadavg` and `pthread_timedjoin_np`

you can add yourself to the list of systems without timedjoin_np

odd schooner Jan 15, 2025, 11:22 PM

#

fervent zodiac the managarm way

where's your graph-based <insert something here>?

fervent zodiac Jan 15, 2025, 11:22 PM

#

src/btop.cpp:317, add || defined __managarm__

fervent zodiac Jan 15, 2025, 11:23 PM

#

serene portal where's ur udev

see, i have no need for udev because i have a significantly better device composition model based on the abstract beauty of ansi colors

#

and also some macro memes i threw together

#

i wonder how bad it would be to try making a hobby os which tries to hit the apple and/or BSD code paths instead of the linux code paths

trim eagle Jan 15, 2025, 11:26 PM

#

dense umbra do it no balls

#678608576151027734 message or

#

Current status: broken, sure

#

But it worked way back when

native prairie Jan 15, 2025, 11:27 PM

#

Damn

trim eagle Jan 15, 2025, 11:27 PM

#

And that’s what counts

native prairie Jan 15, 2025, 11:27 PM

#

Revive it LETSFUCKINGGOOOOOO

trim eagle Jan 15, 2025, 11:27 PM

#

So @serene portal got your shit

trim eagle Jan 15, 2025, 11:27 PM

#

native prairie Revive it <:LETSFUCKINGGOOOOOO:1003833035688513606>

It’s in the pipeline

#

But systemd gets priority

native prairie Jan 15, 2025, 11:28 PM

#

Didnt you get it working?

trim eagle Jan 15, 2025, 11:28 PM

#

It bitrotted away

native prairie Jan 15, 2025, 11:29 PM

#

Systemd?

trim eagle Jan 15, 2025, 11:29 PM

#

So my priorities are keeping xbbs working -> simple port requests or similar -> systemd -> fixing webkit -> chromium atm

serene portal Jan 15, 2025, 11:29 PM

#

trim eagle So <@173337436989030401> got your shit

what did i do

trim eagle Jan 15, 2025, 11:29 PM

#

serene portal what did i do

You dared me. I fulfilled it

trim eagle Jan 15, 2025, 11:29 PM

#

native prairie Systemd?

Chromium

serene portal Jan 15, 2025, 11:30 PM

#

trim eagle You dared me. I fulfilled it

when

#

i have dementia

native prairie Jan 15, 2025, 11:30 PM

#

Yeah but you posted a screenshot of systemd booting in managarm right

trim eagle Jan 15, 2025, 11:31 PM

#

native prairie Yeah but you posted a screenshot of systemd booting in managarm right

That was Linux with mlibc

trim eagle Jan 15, 2025, 11:31 PM

#

serene portal when

Ah fuck. @native prairie dared me

#

Whoops

native prairie Jan 15, 2025, 11:32 PM

#

trim eagle That was Linux with mlibc

Ah

trim eagle Jan 15, 2025, 11:35 PM

#

native prairie Ah

Managarm is next tho

native prairie Jan 15, 2025, 11:35 PM

#

Letsgo

trim eagle Jan 15, 2025, 11:36 PM

#

Literally today after uni I’ll be fixing systemd until stuck, then webkitgtk until Korona and Leo are back, then back to systemd

vestal sapphire Jan 15, 2025, 11:36 PM

#

trim eagle Literally today after uni I’ll be fixing systemd until stuck, then webkitgtk unt...

I might be around all day actually

#

depends

#

gimme an ETA and I'll have my alarm set for that -20 mins

trim eagle Jan 15, 2025, 11:37 PM

#

vestal sapphire gimme an ETA and I'll have my alarm set for that -20 mins

Let’s say 1pm?

vestal sapphire Jan 15, 2025, 11:37 PM

#

sure

dense umbra Jan 16, 2025, 12:03 AM

#

trim eagle https://discord.com/channels/678608576151027731/678608576151027734/1026247505249...

based

trim eagle Jan 16, 2025, 12:09 AM

#

vestal sapphire sure

Aight. Noted

trim eagle Jan 16, 2025, 12:10 AM

#

dense umbra based

I will bring it back this year and upstream it

#

The year TODO list is basically: systemd -> KDE (or GNOME, both are blocked on logind in the end) -> Chromium

dense umbra Jan 16, 2025, 12:31 AM

#

soystemd kringe de crymium

#

based

vestal sapphire Jan 16, 2025, 2:25 AM

#

ok given that I wrapped up some nice stuff, time to do a small thing I've been meaning to do for a while

#

it's going pretty ok so far

#

it's a secret project tho meme

trim eagle Jan 16, 2025, 6:24 AM

#

Nice

carmine current Jan 16, 2025, 6:58 AM

#

vestal sapphire yeah but I don't have to implement the entire command set

The entire command set of iscsi to run libblockfs is probably 3 commands or so though

vestal sapphire Jan 16, 2025, 2:08 PM

#

one thing that we haven't implemented yet is when a process dies, we should move it's children to the dying process' parent

wild stone Jan 16, 2025, 2:11 PM

#

vestal sapphire one thing that we haven't implemented yet is when a process dies, we should move...

We don't do that? Lol

#

Also isn't there some meme regarding threads as well

vestal sapphire Jan 16, 2025, 2:11 PM

#

I don't think so

vestal sapphire Jan 16, 2025, 2:11 PM

#

wild stone Also isn't there some meme regarding threads as well

what meme

wild stone Jan 16, 2025, 2:11 PM

#

vestal sapphire what meme

Wasn't it that PID == TID and that killing one thread didn't kill everything else or something

fervent zodiac Jan 16, 2025, 2:12 PM

#

thats also a problem

carmine current Jan 16, 2025, 2:23 PM

#

vestal sapphire one thing that we haven't implemented yet is when a process dies, we should move...

Is this what's supposed to happen?

#

I don't think it is

#

When a process dies, it's children are moved to init

#

I'm not sure if the ppid changes

#

Ah yes, ppid will change, see https://man7.org/linux/man-pages/man2/getppid.2.html

vestal sapphire Jan 16, 2025, 2:59 PM

#

ah yes

#

so ig we just move to some PID?

#

don't know which one tho

#

the lowest available?

trim eagle Jan 16, 2025, 3:02 PM

#

https://man7.org/linux/man-pages/man2/PR_SET_CHILD_SUBREAPER.2const.html

#

plz?

carmine current Jan 16, 2025, 3:08 PM

#

vestal sapphire so ig we just move to some PID?

init is always pid 1

#

We may also have a pointer to the init Process struct somewhere in global namespace? Not sure

#

ah yes and subreapers would take precedence over init

vestal sapphire Jan 16, 2025, 3:33 PM

#

secret project screenshot

#

damn it really wants setitimer

carmine current Jan 16, 2025, 3:53 PM

#

Icmp?

#

Is the secret project?

vestal sapphire Jan 16, 2025, 3:54 PM

#

yeeeeeeeeeees

carmine current Jan 16, 2025, 3:55 PM

#

very nice

#

Once we have icmp, we can also implement path mtu discovery

vestal sapphire Jan 16, 2025, 4:13 PM

#

that was my thinking

#

what I did not figure out yet though is how we would determine when to do path MTU discovery

carmine current Jan 16, 2025, 4:19 PM

#

each tcp socket should do it individually i guess

vestal sapphire Jan 16, 2025, 4:19 PM

#

on connect?

carmine current Jan 16, 2025, 4:19 PM

#

this can happen simultaneously with normal operation

quaint fern Jan 16, 2025, 4:20 PM

#

carmine current ah yes and subreapers would take precedence over init

Yes systemd will want this

#

However that's only necessary for systemd user service manager

#

The system service manager if it runs as pid 1 no problem

carmine current Jan 16, 2025, 4:23 PM

#

Makes sense

trim eagle Jan 16, 2025, 4:46 PM

#

quaint fern Yes systemd will want this

yeah we figured that. My issue now is cgroups and the fact that it actually wants shit meme

#

so we'll attempt to tackle that this evening

#

meanwhile, webkitgtk upgrading is in progress.....

fervent zodiac Jan 16, 2025, 7:53 PM

#

okay i tried building managarm

#

im gonna try contributing because im bored

trim eagle Jan 16, 2025, 7:54 PM

#

based

fervent zodiac Jan 16, 2025, 7:54 PM

#

it only took me like 10 seconds to find a defect so

trim eagle Jan 16, 2025, 7:54 PM

#

what did you find?

#

the lack of iommu? meme

fervent zodiac Jan 16, 2025, 7:54 PM

#

what is a umask

#

nah iommu is hard

#

and requires effort

trim eagle Jan 16, 2025, 7:54 PM

#

umask is indeed not implemented

fervent zodiac Jan 16, 2025, 7:55 PM

#

yeah

#

it sounds quite useful to have

trim eagle Jan 16, 2025, 7:55 PM

#

yep

fervent zodiac Jan 16, 2025, 7:55 PM

#

lets see how linux does it

#

mask = xchg(&current->fs->umask, mask & S_IRWXUGO); okay seems easy enough

carmine current Jan 16, 2025, 8:34 PM

#

there are lots of low hanging fruits regarding POSIX compliance

fervent zodiac Jan 16, 2025, 8:34 PM

#

hostnames meme

fervent zodiac Jan 16, 2025, 9:28 PM

#

wow my new changes look great

carmine current Jan 16, 2025, 9:28 PM

#

thonk

fervent zodiac Jan 16, 2025, 9:28 PM

#

but, the umask does get changed

carmine current Jan 16, 2025, 9:29 PM

#

.. is resolved in the vfs so it's a bit strange that this returns ENOENT

fervent zodiac Jan 16, 2025, 9:29 PM

#

posix: FsNode::getStats() returned illegal mode of 8624

#

this isnt worrying or anything

#

that is 0o20660

carmine current Jan 16, 2025, 9:30 PM

#

ah, posix handles mode and type in different fields and merges them only at the end

fervent zodiac Jan 16, 2025, 9:31 PM

#

that makes sense i guess

#

i was chmoding a directory to have weird mode flags

#

still nope

#

but this time its not that

#

i'm uh

#

not sure if it is even my fault

carmine current Jan 16, 2025, 9:41 PM

#

what exactly are you doing?

fervent zodiac Jan 16, 2025, 9:41 PM

#

i added a new UmaskRequest, and i and the mode given by the client with the one stored by the last umask request

#

https://paste.sr.ht/~pitust/4df97aa625ad22d3fd5a32a4de1299e0983b9744

#

and a corresponding patch to mlibc

#

can i turn off logging to framebuffer?

carmine current Jan 16, 2025, 9:43 PM

#

I don't think so, why would that be useful?

fervent zodiac Jan 16, 2025, 9:44 PM

#

its slow

#

i think that's slowing down running it, anyway

carmine current Jan 16, 2025, 9:44 PM

#

I don't think it's that slow

fervent zodiac Jan 16, 2025, 9:44 PM

#

hmm maybe not

carmine current Jan 16, 2025, 9:45 PM

#

serial is probably way slower than framebuffer

fervent zodiac Jan 16, 2025, 9:46 PM

#

on TCG though

#

also, serial AND framebuffer is definitely slower than just serial

carmine current Jan 16, 2025, 9:46 PM

#

why don't you test aarch64 with kvm?

#

:^)

fervent zodiac Jan 16, 2025, 9:46 PM

#

carmine current why don't you test aarch64 with kvm?

does that work?

carmine current Jan 16, 2025, 9:47 PM

#

i guess if you've already built x86, it's annoying to build again

fervent zodiac Jan 16, 2025, 9:47 PM

#

also, wouldn't that take another hour to build

carmine current Jan 16, 2025, 9:47 PM

#

yes, I think it does CC @upbeat smelt

fervent zodiac Jan 16, 2025, 9:47 PM

#

oh no a faster development cycle might be useful

#

i only care about base tbf

#

i have built some of weston-desktop, and gave up at 200/600

#

because my m1 is kinda fast but not THAT fast

upbeat smelt Jan 16, 2025, 9:47 PM

#

aarch64 with kvm should work, i recall testing it on my pi4 and fixing some bugs that cropped up in eir-virt that way

fervent zodiac Jan 16, 2025, 9:48 PM

#

okay then

upbeat smelt Jan 16, 2025, 9:48 PM

#

vm-util should also automagically start using kvm ```py
have_kvm = False
if not args.no_kvm:
# Make sure we have KVM, and are going to run the same architecture
if os.access("/dev/kvm", os.W_OK) and os.uname().machine == args.arch:
qemu_args += ["-enable-kvm"]
have_kvm = True
else:
print("No hardware virtualization available!", file=sys.stderr)

carmine current Jan 16, 2025, 9:49 PM

#

uh actually I think you're right that this is not your fault

#

the .. bug

#

that's incredibly weird, i need to debug this in the vfs code

fervent zodiac Jan 16, 2025, 9:51 PM

#

okay thanks

carmine current Jan 16, 2025, 9:51 PM

#

ls -la works but ls -la dir/ breaks

fervent zodiac Jan 16, 2025, 9:51 PM

#

since it is not my fault

#

i will try debugging the rest of the code then

#

okay how do i change the target ISA

carmine current Jan 16, 2025, 9:53 PM

#

https://github.com/managarm/managarm/blob/master/ci/bootstrap-site-aarch64.yml put the define_options: into bootstrap-site.yml

fervent zodiac Jan 16, 2025, 9:53 PM

#

ok thanks cool

carmine current Jan 16, 2025, 9:53 PM

#

and probably nuke the tool-builds / pkg-builds and similar dirs

fervent zodiac Jan 16, 2025, 9:53 PM

#

ugh

upbeat smelt Jan 16, 2025, 9:54 PM

#

also want the labels: part maybe, although it only really matters when you're messing with installing packages

fervent zodiac Jan 16, 2025, 9:55 PM

#

okay this is gonna be fun (not)

#

../../../src/mlibc/meson.build:38:1: ERROR: Unknown compiler(s): [['aarch64-managarm-gcc']]
``` okay that was fast

carmine current Jan 16, 2025, 9:56 PM

#

hmm

#

also nuke the tools and packages dir I guess

fervent zodiac Jan 16, 2025, 9:56 PM

#

more rm rf needed

#

yeah

#

may i recommend having target specific packages :^)

#

so you have host-@@ARCH@@-gcc

#

instead of host-gcc

carmine current Jan 16, 2025, 9:57 PM

#

you could also just create a second build dir for this purpose

fervent zodiac Jan 16, 2025, 9:57 PM

#

i could have

#

i probably should have

carmine current Jan 16, 2025, 9:57 PM

#

fervent zodiac so you have host-@@ARCH@@-gcc

hm yeah something like that would definitely be needed if we want to support multiple archs from the same build dir

#

i wanted to add this feature in the past because it saves building llvm multiple times

upbeat smelt Jan 16, 2025, 9:58 PM

#

shouldn't be too hard to generate that with an y4 macro :^)

fervent zodiac Jan 16, 2025, 9:58 PM

#

carmine current i wanted to add this feature in the past because it saves building llvm multiple...

oh god

#

please save me from this

#

btw

#

do you build a single-target llvm?

carmine current Jan 16, 2025, 9:58 PM

#

no

fervent zodiac Jan 16, 2025, 9:58 PM

#

you should

#

it is way faster to build

upbeat smelt Jan 16, 2025, 9:59 PM

#

you could've kept {tools,tool-builds}/host-llvm-toolchain ig

fervent zodiac Jan 16, 2025, 9:59 PM

#

i only rm -rf'ed tools/*gcc

carmine current Jan 16, 2025, 9:59 PM

#

hm, might be a good idea actually

upbeat smelt Jan 16, 2025, 9:59 PM

#

we only build llvm for x86, aarch64 and riscv64

fervent zodiac Jan 16, 2025, 9:59 PM

#

ah okay

#

so thats fine anyway

#

3 targets and 1 target is about the same

native prairie Jan 16, 2025, 10:01 PM

#

when 32 bit managarm

fervent zodiac Jan 16, 2025, 10:01 PM

#

did you mean: managarm that is even slower than it is now somehow

native prairie Jan 16, 2025, 10:01 PM

#

lol

upbeat smelt Jan 16, 2025, 10:01 PM

#

i'm not touching 32-bit

fervent zodiac Jan 16, 2025, 10:01 PM

#

yeah no

trim eagle Jan 16, 2025, 10:01 PM

#

native prairie when 32 bit managarm

never

fervent zodiac Jan 16, 2025, 10:01 PM

#

i wouldnt want to either

#

it would just make the performance problems a lot worse

trim eagle Jan 16, 2025, 10:02 PM

#

I'd rather chop my balls off very slowly

fervent zodiac Jan 16, 2025, 10:02 PM

#

btw did anyone try profiling managarm?

#

that might be useful to do as well

native prairie Jan 16, 2025, 10:02 PM

#

it could use that monkuous' profiler

fervent zodiac Jan 16, 2025, 10:02 PM

#

any profiler will work

#

but yeah

#

i think llenotre's profiler might work too?

trim eagle Jan 16, 2025, 10:03 PM

#

please do

upbeat smelt Jan 16, 2025, 10:03 PM

#

we already have a rudimentary profiler

#

that uses pmc nmis

fervent zodiac Jan 16, 2025, 10:03 PM

#

what does it tell you?

upbeat smelt Jan 16, 2025, 10:03 PM

#

where IP is after N retired insns

fervent zodiac Jan 16, 2025, 10:03 PM

#

i mean

#

what are the hot spots lol

upbeat smelt Jan 16, 2025, 10:04 PM

#

ah

fervent zodiac Jan 16, 2025, 10:04 PM

#

upbeat smelt where IP is after N retired insns

also you may not want to use N retired insns

#

but N cycles if thats a thing, idk if it is

#

or N uops

upbeat smelt Jan 16, 2025, 10:05 PM

#

hm on intel we do clock cycles

#

don't know what the amd pmc code is doing because i don't know what the msrs are

#

but i'd guess clock cycles as well actually?

carmine current Jan 16, 2025, 10:08 PM

#

yeah

upbeat smelt Jan 16, 2025, 10:09 PM

#

fervent zodiac what are the hot spots lol

about that, no idea actually

#

might be a good idea to check :^)

fervent zodiac Jan 16, 2025, 10:09 PM

#

lmfao what

#

you guys have a profiler

#

and haven't thought to uh

#

USE IT

upbeat smelt Jan 16, 2025, 10:10 PM

#

i haven't personally looked

fervent zodiac Jan 16, 2025, 10:10 PM

#

ah okay

carmine current Jan 16, 2025, 10:10 PM

#

it tells us that the kernel does not dominate the overall running time meme

#

and profiling userspace is a bit harder

fervent zodiac Jan 16, 2025, 10:10 PM

#

ahhhhh

#

okay

carmine current Jan 16, 2025, 10:10 PM

#

profiling userspace would also be very useful

upbeat smelt Jan 16, 2025, 10:11 PM

#

upbeat smelt i haven't personally looked

and i guess i won't be looking today ```
thor: Fix exiting fibers
thor: Fix exiting fibers
tallocating 2
qemu: terminating on signal 15 from pid 153102 (python3)
Traceback (most recent call last):
Exception ignored in: <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>
Exception ignored in sys.unraisablehook: <built-in function unraisablehook>
Exception ignored in: <function BaseSubprocessTransport.del at 0x7fd757aabba0>
Traceback (most recent call last):
File "/usr/lib/python3.12/asyncio/base_subprocess.py", line 126, in del
File "/usr/lib/python3.12/asyncio/base_subprocess.py", line 104, in close
File "/usr/lib/python3.12/asyncio/unix_events.py", line 568, in close
File "/usr/lib/python3.12/asyncio/unix_events.py", line 592, in _close
File "/usr/lib/python3.12/asyncio/base_events.py", line 794, in call_soon
File "/usr/lib/python3.12/asyncio/base_events.py", line 540, in _check_closed
RuntimeError: Event loop is closed
hor: Fix exiting fibers
xbstrap: Action run of task qemu failed

fervent zodiac Jan 16, 2025, 10:11 PM

#

do you have ASLR?

carmine current Jan 16, 2025, 10:11 PM

#

i guess we could try to port something like gprof to start with

#

no

fervent zodiac Jan 16, 2025, 10:11 PM

#

carmine current i guess we could try to port something like gprof to start with

you want a system profiler

#

not gprof

carmine current Jan 16, 2025, 10:11 PM

#

upbeat smelt and i guess i won't be looking today ``` thor: Fix exiting fibers thor: Fix exit...

hm?

fervent zodiac Jan 16, 2025, 10:12 PM

#

carmine current no

okay then, is there a way to make it always place each library at a fixed place?

#

meme

carmine current Jan 16, 2025, 10:12 PM

#

not without hacks i guess?

fervent zodiac Jan 16, 2025, 10:12 PM

#

fervent zodiac okay then, is there a way to make it always place each library at a fixed place?

one unique to that library

fervent zodiac Jan 16, 2025, 10:12 PM

#

carmine current not without hacks i guess?

okay then time to write an ugly hack

#

oh on arm at least i get frame pointers

upbeat smelt Jan 16, 2025, 10:12 PM

#

carmine current hm?

vm-util dies during boot (i assume when we try to dump the ring buffer with the traces?)

fervent zodiac Jan 16, 2025, 10:12 PM

#

and i have linux perf kvm

upbeat smelt Jan 16, 2025, 10:13 PM

#

upbeat smelt vm-util dies during boot (i assume when we try to dump the ring buffer with the ...

looks like an exception when trying to decode data as utf-8?

carmine current Jan 16, 2025, 10:13 PM

#

uh

#

is this the newest commit of bootstrap-managarm qookie?

upbeat smelt Jan 16, 2025, 10:15 PM

#

uh no meme pulled bootstrap-managarm and now it works

fervent zodiac Jan 16, 2025, 10:16 PM

#

lmfao

upbeat smelt Jan 16, 2025, 10:19 PM

#

top few results ```
2.96% (5328 samples) in:
thor::FbDisplay::setBlanks(unsigned int, unsigned int, int, int) in 0
3.00% (5397 samples) in:
frg::slab_pool<thor::KernelVirtualAlloc, thor::IrqSpinlock>::allocate(unsigned long) in 0
3.36% (6029 samples) in:
thor::Stream::Submitter::run() in 0
4.02% (7225 samples) in:
helSubmitAsync in 0
4.07% (7314 samples) in:
thor::IrqSpinlock::lock() in 0
4.85% (8711 samples) in:
thor::restoreExecutor(thor::Executor*) in 0
5.25% (9434 samples) in:
thor::sendPingIpi(thor::CpuData*) in 0
179701 (= 65.55% of all samples) in the kernel
100.00% of all kernel samples could be resolved

#

from just booting into weston and running some commands in the terminal

carmine current Jan 16, 2025, 10:19 PM

#

the number of ping IPIs could definitely be reduced

#

we could be a bit smarter about that

fervent zodiac Jan 16, 2025, 10:22 PM

#

upbeat smelt top few results ``` 2.96% (5328 samples) in: thor::FbDisplay::setBlanks(unsi...

4% of hits in thor::IrqSpinlock::lock is rather worrying

#

i think i know one source of your problems

#

that word "ticket" over there

#

uh

#

yeah please dont

#

this isnt very impactful on perf, but:

#

no inline fast path???

fervent zodiac Jan 16, 2025, 10:24 PM

#

fervent zodiac yeah please dont

in my testing, they face a large performance penalty

upbeat smelt Jan 16, 2025, 10:25 PM

#

what do you suggest instead, just regular test and set a bit ones?

fervent zodiac Jan 16, 2025, 10:25 PM

#

yeah

#

normal ttas

carmine current Jan 16, 2025, 10:26 PM

#

i guess it doesn't hurt to try that

#

re inline fast path: that's a good point. also we could enable LTO

upbeat smelt Jan 16, 2025, 10:27 PM

#

wonder how much ub that'd uncover meme

fervent zodiac Jan 16, 2025, 10:28 PM

#

carmine current re inline fast path: that's a good point. also we could enable LTO

LTO wouldnt help

#

llvm cant half inline a function

native prairie Jan 16, 2025, 10:30 PM

#

upbeat smelt wonder how much ub that'd uncover <:meme:575445094589661215>

💀

native prairie Jan 16, 2025, 10:30 PM

#

fervent zodiac llvm cant half inline a function

Wdym

upbeat smelt Jan 16, 2025, 10:30 PM

#

fervent zodiac llvm cant half inline a function

the spinlock is simple enough that i think it'd just inline it wholesale

fervent zodiac Jan 16, 2025, 10:30 PM

#

native prairie Wdym

you can either inline everything

#

or inline none of it

upbeat smelt Jan 16, 2025, 10:31 PM

#

with lto even if the slow path was in a separate fn

fervent zodiac Jan 16, 2025, 10:31 PM

#

what you want is a single fast path

fervent zodiac Jan 16, 2025, 10:31 PM

#

upbeat smelt the spinlock is simple enough that i think it'd just inline it wholesale

no thats why you mark the branch cold

upbeat smelt Jan 16, 2025, 10:32 PM

#

fair enough

fervent zodiac Jan 16, 2025, 10:34 PM

#

btw libguestfs mounts are broken on fedora

carmine current Jan 16, 2025, 10:34 PM

#

nooo

fervent zodiac Jan 16, 2025, 10:34 PM

#

qemu-system-aarch64: can't apply global host-arm-cpu.migratable=no: Property 'host-arm-cpu.migratable' not found
``` also this is exciting

upbeat smelt Jan 16, 2025, 10:35 PM

#

yeah libguestfs is kinda a pain

fervent zodiac Jan 16, 2025, 10:35 PM

#

fervent zodiac btw libguestfs mounts are broken on fedora

or in managarm idk

#

testing is hard

carmine current Jan 16, 2025, 10:35 PM

#

isn't libguestfs mainly sponsored by redhat? meme

upbeat smelt Jan 16, 2025, 10:35 PM

#

i recall it was also broken on arch a while ago and i had to manually build it to use it from an older PKGBUILD

carmine current Jan 16, 2025, 10:35 PM

#

but yeah libguestfs is quite unstable

fervent zodiac Jan 16, 2025, 10:35 PM

#

i think host-arm-cpu is just straight up gone

#

oh wait

#

no

#

wtf

upbeat smelt Jan 16, 2025, 10:36 PM

#

ig migratable=no is x86-specific?

fervent zodiac Jan 16, 2025, 10:36 PM

#

it is yeah

#

okay how do i specify a custom qemu path?

#

actually nvm my build wouldnt work either because its riscv only and not kvm

upbeat smelt Jan 16, 2025, 10:37 PM

#

you can install the host-qemu tool via xbstrap to have a locally-built qemu

#

and vm-util should pick it up automatically

fervent zodiac Jan 16, 2025, 10:38 PM

#

fuck.

carmine current Jan 16, 2025, 10:38 PM

#

wait M1 doesn't support 4K pages?

fervent zodiac Jan 16, 2025, 10:38 PM

#

also that sounds wrong

fervent zodiac Jan 16, 2025, 10:38 PM

#

carmine current wait M1 doesn't support 4K pages?

it... does

#

it is a managarm skill issue

#

i wanted to only send userspace patches, not thor patches too meme

native prairie Jan 16, 2025, 10:39 PM

#

Maybe because apple native is 16k it only propagates that via kvm

fervent zodiac Jan 16, 2025, 10:40 PM

#

no

carmine current Jan 16, 2025, 10:40 PM

#

kernel/eir/arch/arm/arch.cpp <- the check is here

#

and it reads id_aa64mmfr0_el1

fervent zodiac Jan 16, 2025, 10:40 PM

#

i can see it

#

i have grep too

native prairie Jan 16, 2025, 10:40 PM

#

Lol

fervent zodiac Jan 16, 2025, 10:41 PM

#

my id is 19868771418113

#

0x0000_1212_0f10_0001

#

okay

#

i found the bug

#

0xf is a signed int

#

0xf << 28 is still signed

#

and gets improperly sign extended

upbeat smelt Jan 16, 2025, 10:43 PM

#

trolled again

fervent zodiac Jan 16, 2025, 10:43 PM

#

you're welcome

upbeat smelt Jan 16, 2025, 10:43 PM

#

i keep getting trolled by sign ext on aarch64 in vmm-related code

fervent zodiac Jan 16, 2025, 10:43 PM

#

that will be 15k in consulting fees

#

this is where it gets to now

carmine current Jan 16, 2025, 10:44 PM

#

looks like SMP startup is broken

#

is the BSP == cpu zero?

#

IIRC there's a TODO in the aarch64 code that we assume this

fervent zodiac Jan 16, 2025, 10:46 PM

#

i think so

#

this time its this

#

i dont think its smp

carmine current Jan 16, 2025, 10:48 PM

#

you can try to boot without SMP

#

but yeah seems that this time it didn't get to SMP init

#

weird

fervent zodiac Jan 16, 2025, 10:48 PM

#

it gets here a lot

#

either here or the smp thing

#

qemu pins one core to 100%

#

so

#

i think it is running SOMETHING

#

#

i believe the m1 has a vgic

upbeat smelt Jan 16, 2025, 10:51 PM

#

cc @brittle mauve

carmine current Jan 16, 2025, 10:54 PM

#

If it's in eoi it's probably getting an irq storm?

fervent zodiac Jan 16, 2025, 10:54 PM

#

yeah i think so

carmine current Jan 16, 2025, 10:55 PM

#

you can set logEveryIrq = true to confirm this

fervent zodiac Jan 16, 2025, 10:55 PM

#

yes.

#

its an irq storm

#

then stack exhaustion, most likely

upbeat smelt Jan 16, 2025, 10:56 PM

#

hm is irq 30 the generic timer?

carmine current Jan 16, 2025, 10:56 PM

#

Is it the timer irq?

fervent zodiac Jan 16, 2025, 10:56 PM

#

yeah

upbeat smelt Jan 16, 2025, 10:56 PM

#

inb4 the math for computing the deadline overflows

#

i should port it over to the new helper class for doing the math

carmine current Jan 16, 2025, 10:58 PM

#

checking whether that's the case would be easy with __builtin_mul_overflow

upbeat smelt Jan 16, 2025, 10:58 PM

#

true but i didn't consider it when i first wrote it

#

and the helper is already there so

#

i think i didn't think of it as an issue because for qemu-virt without kvm and on the pi4 the frequency of the timer is <50MHz or something like that

carmine current Jan 16, 2025, 11:01 PM

#

true

fervent zodiac Jan 16, 2025, 11:01 PM

#

i suspect the frequency on m1 is very much more

carmine current Jan 16, 2025, 11:02 PM

#

The helper class that uses the 128 bit fixed point math can probably be used just like in the rv code

fervent zodiac Jan 16, 2025, 11:02 PM

#

where is the frequency stored?

upbeat smelt Jan 16, 2025, 11:02 PM

#

fervent zodiac where is the frequency stored?

CNTFREQ_EL0

#

i think it's an el0 reg

fervent zodiac Jan 16, 2025, 11:02 PM

#

wtf

#

do you guys not like read it???

carmine current Jan 16, 2025, 11:02 PM

#

https://github.com/managarm/managarm/blob/master/kernel/thor/arch/riscv/timer.cpp#L91

upbeat smelt Jan 16, 2025, 11:03 PM

#

oh FRQ not FREQ

carmine current Jan 16, 2025, 11:03 PM

#

^ that's how rv does it

upbeat smelt Jan 16, 2025, 11:03 PM

#

trolled again

upbeat smelt Jan 16, 2025, 11:03 PM

#

fervent zodiac do you guys not like read it???

thor/arch/arm/timer.cpp line 88

fervent zodiac Jan 16, 2025, 11:06 PM

#

on m1 its the standard m1 reference clock

#

24mhz

#

same as the watchdog timer, fwiw

upbeat smelt Jan 16, 2025, 11:06 PM

#

that shouldn't be overflowing then

#

hmmmm

upbeat smelt Jan 16, 2025, 11:10 PM

#

carmine current <https://github.com/managarm/managarm/blob/master/kernel/thor/arch/riscv/timer.c...

btw i was thinking we could probably abstract the whole logic for multiplexing the preemption timer and alarm timers into one hw timer into generic code?

#Managarm and related projects