#pmOS - microkernel OS for RISC-V, x86, and LoongArch

1 messages Β· Page 10 of 1

carmine nacelle
#

I feel like I have to improve lapic/tsc calibration first

#

probably AMD?

#

like they had the whole cpu cores unlocking thing

twilit talon
#

CPUs can be software-disabled for many reasons

carmine nacelle
#

intel with avx...

#

Can there be repeated entries?

twilit talon
#

yes

carmine nacelle
#

In MADT

twilit talon
#

ofc

carmine nacelle
#

but I mean two x2apic entries with the same id

twilit talon
#

yes

carmine nacelle
twilit talon
#

u have to be able to deduplicate them

carmine nacelle
#

and what if there's only x2apic entry for id < 255

#

or is this too insane?

twilit talon
#

why does it matter

#

u pretend they dont exist at all

carmine nacelle
#

fun

twilit talon
#

(if lapic entries also exist)

carmine nacelle
#

you get less cpu cores sadmeme

twilit talon
#

u dont

#

u dont ignore them if there arent any lapic entries

carmine nacelle
#

CPU usable is flags & ACPI_PIC_ENABLED or flags & ACPI_PIC_ONLINE_CAPABLE, right?

twilit talon
#
static bool cpu_usable(u32 flags)
{
    if (flags & ACPI_PIC_ENABLED)
        return true;

    if (!s_online_capable_bit_usable)
        return false;

    return flags & ACPI_PIC_ONLINE_CAPABLE;
}
carmine nacelle
#

oh no

twilit talon
#

nah its simple

#
static void setup_online_capable_bit(void)
{
    struct acpi_fadt *fadt;
    uacpi_status st;

    st = uacpi_table_fadt(&fadt);
    BUG_ON(st != UACPI_STATUS_OK);

    /*
     * Online capable bit was introduced in ACPI 6.3, prior revisions might set
     * it erroneously so don't look at it for those versions.
     */
    s_online_capable_bit_usable =
        (fadt->hdr.revision > 6) ||
        (fadt->hdr.revision == 6 && fadt->fadt_minor_verison >= 3);
}
carmine nacelle
#

How many helpers are there in uACPI? nooo

twilit talon
#

lol

carmine nacelle
#

(I forgot what was the function)

twilit talon
#

uacpi_table_fadt returns an internal version that doesnt need ref/unref

carmine nacelle
#

So uacpi just stores it in its buffer

twilit talon
#

yeah, since the raw copy requires a lot of sanitization

carmine nacelle
#

And not checking if I'm on bsp

twilit talon
#

looks good

twilit talon
#

Wdym?

carmine nacelle
hexed acorn
#

ah no i misread the check

#

i thought the latter part was == 3

twilit talon
#

The current version is 6.6 I think

carmine nacelle
#

I have 6.5 in my osdev folder

twilit talon
#

It came out in like December

carmine nacelle
#

Small update: I've started publishing PCI devices to pmbus as well, but it's currently broken because of broken puts() or whatever Rust is using nooo

#

Maybe I should just finally switch to mlibc? thonk

#

Though I wanted to implement futex first coolmeme

#

I think I'm going insane, why is my todo list so large nooo

carmine nacelle
#

This is cool

#

I think this can all fall nicely into place

#

I don't have energy to do it right now, but I think I can add handles to the interrupts in kernel (which can use iommu as well), which can act as a permission system

#

Then also do this, with a different handle

carmine nacelle
# carmine nacelle This is cool

(anyway, now I can match AHCI devices in a better form, I think I'm going to make my ahci driver use this, then do the disk thing, then hopefully I can have disk?)

carmine nacelle
#

It just loaded, and died

carmine nacelle
#

I think the kernel OOMed

carmine nacelle
#

Ok, this looks extremely promising (it's failing because I haven't implemented that syscall yet), and it found a mouse and a keyboard, and tried to start the service twice (even though i8042 should have only started once), so I can probably make it restart servers as well with very little work

#

I can probably even forgo dependencies for servers for now, and make it start servers as the things get published in pmbus

#

Because the idea is to have devicesd launch and publish pci devices -> this will start ahci server which will publish the disk -> this will start the disks server, which will parse the partition tables (iirc I had code for that already) -> which will start the filesystem drivers, which can finally start the vfs or whatever

#

(I need to decide how to handle the mountpoints)

carmine nacelle
#

Idk, I have so many ideas for this

#

(I really have to study though, so I don't have time for this now)

#

(famous last words)

carmine nacelle
carmine nacelle
#

I am going insane

#

My yaml parser is trolling me

twilit talon
#

lol

carmine nacelle
#

It's working, but I'm exhausted

#

Though I'm planning to make 64 bit port bootable with Hyper as well

#

After this

dense carbon
carmine nacelle
#

(I've used libyaml, but I think I'm doing it wrong, since the code I'm writing is horrible)

dense carbon
#

yaml is famously complicated

carmine nacelle
#

yeah, it seemed really simple though

#

Anyway, this is not big, but it's working

twilit talon
#

cool

carmine nacelle
#

And this is my limine file, which can hopefully make Hyper work as well

twilit talon
#

yeah thats the thing about yaml, if u actually use advanced features its ridiculously complicated

carmine nacelle
#

My i686 port might be broken

#

I haven't tested it in like a year

#

And I've changed a bunch of x86 stuff recently

#

(I mean I unified more of x86_64 and i686)

twilit talon
#

yeah that definitely looks better than shoving everything into name

left dew
#

does the init server do device matching to launch the servers?

carmine nacelle
#

with the pmbus

left dew
#

yeah i figured that much

carmine nacelle
#

and then it just starts them and gives them a handle to the device

twilit talon
#

Btw what if I have multiple ps2 keyboards

carmine nacelle
#

Then that is supposed to publish two ps2 port object (which I haven't implemented yet)

twilit talon
#

IMO ALWAYS_ONCE should be implied by the lack of a match specifier

carmine nacelle
#

Then that launches the PS/2 server

carmine nacelle
twilit talon
#

Wdym

carmine nacelle
#

I also though about it, and the stuff like POSIX server can interact with the bootstrapd directly

carmine nacelle
#

(I also wanted to add a mechanism to restart them, and so on)

twilit talon
#

Right

carmine nacelle
#

Also, I need to "urgently" fix my build system trl

#

(recompiling everything for a different arch is a mess)

twilit talon
#

How do u not recompile everything if its a different arch

carmine nacelle
#

I mean I do

#

But I have to recompile libc, then reinstall llvm's libs, and then recompile the rest

#

But like it's not automatic

#

Cool, the i686 kernel doesn't compile

twilit talon
carmine nacelle
#

For a dumb reason actually

#

Missing includes...

twilit talon
#

πŸ’€

carmine nacelle
#

and missing using namespace ultrameme

#

What even is this error? πŸ’€

#

Cool, it gets to userspace, and crashes after that...

#

(In i686 port)

#

(I mean the userspace crashes)

#

I think I'll fix it tomorrow

#

Which probably implies the kernel fully works

twilit talon
#

Thats cool

#

Is that the hyper version

carmine nacelle
#

Yeah, ofc

twilit talon
#

Cool

carmine nacelle
#

Just qemu-system-i386 -hdd pmos-hyper.img -serial stdio

twilit talon
#

Thats like 128 megs of ram

carmine nacelle
#

who cares trl

#

Maybe I should fix it now?

#

I'm too sleepy

twilit talon
#

1 am is rookie time

carmine nacelle
#

16 is syscall_create_right

#

which shouldn't be too hard to fix, whatever it is

#

And -ESRCH means that it can't find the task group?

twilit talon
#

I hope 15 is create_left trl

carmine nacelle
#

SYSCALL_SET_INTERRUPT trl

twilit talon
#

Nooo

carmine nacelle
carmine nacelle
#

Which might be suboptimal

#

(I mean I removed it from limine as well, because of SMP considerations, but since I do it myself, the other cores shouldn't be touching it anyway, so there shouldn't be race conditions with it)

#

Yeah, the fact it's doing this makes no sense...

#

I'll do it tomorrow

twilit talon
carmine nacelle
#

Failing to create a right

#

It's failing because the rights namespace (a.k.a. the current task group) doesn't exist, which makes no sense

#

At least from my recollection of code that I wrote like a year ago

twilit talon
#

Interesting

carmine nacelle
#

What actually pagefaulted was syscall_return???

#

And the stack was unmapped for some reason

carmine nacelle
#

I've no idea what's going on

#

Or rather, how did 64 bit port work?

#

Huh?

#

I think I just "forgot" to set it on pthread entry

#

Because for some reason I did it in assembly??

#

This looks more promissing...

carmine nacelle
#

Since when are uint64_t s are 8 bytes aligned on x86??

#

Or wait

#

Wait, it's not it

#

bruh

#

Also, since when is movq valid on ia32???

#

Cool, the stack is off...

#

I think the kernel is ignoring the fixed flag

twilit talon
#

thats because by default clang assumes a higher baseline

#

so it uses sse unconditionally even for 32 bit

carmine nacelle
#

It's not the problem with sse, but that my stack is not where it's supposed to be bruh

twilit talon
#

yeah im just saying movq and sse is because of that

carmine nacelle
#

Ofc it's available with i686 target

#

I think movq should be fine with misaligned uint64_t though

twilit talon
#

yes

carmine nacelle
#

I think I'm getting somewhere

carmine nacelle
#

Which I've changed for 64 bit pointers, but forgot for 32 bit ones

carmine nacelle
#

Though why didn't it print the rest of the stuff?

#

Now it just decided to break for no reason...

#

It must be something stupid

carmine nacelle
#

The log server isn't starting

#

I found the error trl

#

Ok, i686 is kinda working

#

(what's broken is syscall return...)

#

I think I want to port x86_64 to hyper as well

twilit talon
#

I will add a cmdline field to modules soon but I guess u don't need it anymore

carmine nacelle
#

Yeah...

twilit talon
#

I mean a config is better anyway

carmine nacelle
twilit talon
#

Lol why

toxic torrent
#

lol

carmine nacelle
#

I'm tired and want to fix i8042 driver first

twilit talon
#

How did you break it

carmine nacelle
carmine nacelle
#

Like the logic probably works, but I want it to use pmbus, etc.

twilit talon
#

Ah

carmine nacelle
#

Because it's 2 1k loc files, that are different for 2 protocols I support...

carmine nacelle
#

I think I need a new C helper for pmbus...

#

(why am I writing C?)

carmine nacelle
#

Also, maybe I should finally implement futex??

carmine nacelle
#

i8042 is trolling me (in QEMU!)

carmine nacelle
#

This is concerning...

carmine nacelle
#

Or maybe it's actually fine

#

But my CPU cooler is like at 100% for some reason

#

Which tells me it's not trl

#

I'm dumb

carmine nacelle
#

I think my rbtree impl is trolling me nooo

carmine nacelle
#

Now I kinda want to properly do interrupts stuff, like it's done on managarm in kdb server, but I kinda don't have energy for it

carmine nacelle
#

Should I just finally do proper interrupt rights? thonk

#

(a.k.a. handles)

carmine nacelle
#

Talking of which, I have decided that I will make my rights also be messages in the kernel, so that they can be pushed onto the ports

#

(since then it lets the kernel send messages without having to allocate memory for them)

carmine nacelle
#

I know this is basically just a bunch of userspace stuff, but why is it so cool to see it do this? (devicesd -> i8042 -> ps2d, all launching new drivers instances through pmbus)

#

(the ordering here is from async IPC and the fact that I think I don't have SMP enabled in qemu...)

carmine nacelle
#

What's concerning now is that my ns16550 driver breaks half the times, for some reason

#

(Which I think is because of broken interrupts)

#

Cool, the keyboard is fixed

#

(totally not overengineered)

#

Time to figure out evdev?

twilit talon
#

why do u need evdev even

#

i thought u werent doing linux compat

carmine nacelle
#

how do I represent keyboard and mouse data then?

twilit talon
#

thats up to you to design

carmine nacelle
#

Like I though it was just some standard structs + char devices at /dev/input

twilit talon
#

standard for linux yes

carmine nacelle
#

Since Managarm and freebsd have it, why not do it as well

twilit talon
#

managarm has it because its aiming for linux compatibility

#

although at api not abi level

#

and freebsd literally has linuxlator so yeah

carmine nacelle
#

what do I use then??

#

to not reinvent the wheel

twilit talon
#

depends on your goals

#

if u dont want to reinvent the wheel do linux compat

carmine nacelle
#

I don't want to deal with input tbh

#

All I wanted was drivers

#

Also, I have no idea how human input and terminals and GUIs work

carmine nacelle
#

I see Wikipedia mentions STREAMS when talking about line discipline trl

carmine nacelle
#

I have decided, I'll do my tty server in C++ trl

#

I like coroutines, and people have said that the Rust's ones are too complicated

plain jungle
#

who said that?

#

Rust async/await is fine

left dew
#

yeah, no, it’s fine

#

probably easier and better integrated than c++ coroutines

#

tho that comes with a few drawbacks, mainly lifetimes that are wack

plain jungle
#

yeah, if you just compare the mechanism, C++ coroutines are probably a bit easier than rust async/await

#

but with rust you can use a lot of existing libraries

left dew
#

tbh i have never even taken a look at c++ coroutines, so that's why i think rust futures are easier :^)

carmine nacelle
#

Another detail is that I already have stuff which uses C++ coroutines, I just haven't bothered yet to make them handle rights (I mean this kinda also was the reason to implement them in the first place)

carmine nacelle
#

Wtf is this

#

My IPC in a nutshell

carmine nacelle
#

Ok, my C++ insanity has almost worked first time

#

And is now trolling me nooo

carmine nacelle
#

what the fuck

#

Ok, this was actually not a C++ bug

#

This is very promissing (timers are broken again nooo)

carmine nacelle
#

Yeah, it's actually kinda mostly working...

carmine nacelle
carmine nacelle
#

My AHCI is hardcore trolling me

#

It's just calling exit() somewhere, with no error messages

#

Also, I'm fed up with my timers

#

It just breaks half of the time

#

I'm done

carmine nacelle
#

It was a very dumb double free

carmine nacelle
#

thonk why does limine not load with 128mb of ram?

twilit talon
#

Does hyper work

carmine nacelle
#

it was last time I checked

twilit talon
#

Big

carmine nacelle
#

though hyper is still i386 only

#

huge todo list nooo

carmine nacelle
#

(I think I finally need to implement shared libraries though)

lunar island
#

this is still smaller than a rust hello world trl

carmine nacelle
#

pmbus is in rust KEKW

lunar island
#

damn

carmine nacelle
#

I think the time has come to fix the timers...

#

The only issue is that I haven't decided what interface I want to use

carmine nacelle
#

I hope I don't get AI psychosis trl
But I've talked to a clanker, and I think I have a very good solution trl

carmine nacelle
#

This is insightful

twilit talon
#

sycophancy final boss

#

llm are trained to do that so

carmine nacelle
#

I need to think about what it said

#

But I think I'll add timer capabilities, which send messages as programmed (so I can just tread them as recieve rights for messages on the other side), and it's like what I have already, but better. And I was worried about the concurrency, but it suggested me that I can just IPI other cores to invalidate the timers on them (since my timers are per-CPU, but multiple threads have access to capabilities).

(It also said to look at QNX, Mach and L4, which is like reasonable...)

carmine nacelle
#

After some (passive) thinking, I think it just repeated what I said to it

#

So this doesn't count

#

Why do I even bother with asking AI

carmine nacelle
#

I think I'm in the mood for fixing timers...

#

Though I won't redo the timers interface for now...

#

Just make it absolute, and universal

carmine nacelle
#

Though while I'm at it, what's stopping me from making the kernel fully preemptible? trl

#

I just won't reschedule if I'm preempted...

carmine nacelle
#

I think I'm going insane trl

#

What if I just make it use HPET for scheduling? thonkthonkthonk

manic girder
#

I mean could be worse, iirc lemonOS uses (and only has support for) the PIT.

twilit talon
#

Damn lemonos

carmine nacelle
#

My new timer infra is having the time of its life

#

(It looks like it overflows?)

carmine nacelle
carmine nacelle
#

I think the ACPI PM time source is working now...
The only issue is that it seems that it's a bit faster than it should be in QEMU (but not by too much...)

twilit talon
#

how did u hook it up into the kernel

carmine nacelle
carmine nacelle
twilit talon
#

try on real hw

carmine nacelle
#

I would need find the said real hw

#

And the com modem cable

twilit talon
#

dont u have display output

carmine nacelle
twilit talon
#

like fbcon

carmine nacelle
#

It's in a different branch trl

twilit talon
#

bruh

#

how old is your project again KEKW

carmine nacelle
#

since 2022

twilit talon
#

its crazy that u have so many features yet a black screen lmao

carmine nacelle
#

No, I can just rebase on it

#

It should probably work

#

no, the driver is in userspace

#

it's not black, it's just that if the kernel doesn't boot, there is no screen

#

I think it should shough

carmine nacelle
twilit talon
#

yeah u should

carmine nacelle
#

I didn't boot on my test machine trl

#

But I would imagine it just doesn't have acpi pm

twilit talon
#

i dont think ive seen an x86 box without a pm timer

carmine nacelle
#

It's a mac

twilit talon
#

anyway its easy to check via acpidump

carmine nacelle
#

Idr if I ever sent you its dump...

twilit talon
#

i have like 4 dumps from u

#

not sure which one is which tbh

carmine nacelle
#

I have like 10 machines

#

It's an intel mac

twilit talon
#

lmao

#

i mean acpi dumps dont exactly tell u any hardware information

carmine nacelle
#

My kernel is trolling me

#

I'm building the kernel-fb branch...

#

Can you get the dump from OS X?

twilit talon
#

most likely yes

#

but idk if it can be done via acpidump

carmine nacelle
#

if not, I can just boot ubuntu on it, or something

twilit talon
#

i mean even windows supports dumping it

#

i think osx should have a way too

carmine nacelle
#

tbh I should make a flantem in kernel a define instead of a separate branch

twilit talon
#

yes

#

do kconfig

carmine nacelle
#

Or like just have it at all time, since it useful for panics

#

Since now I just print it to com

#

Yeah, it's a broadcast IPI dip skill issue

#

I can already see it's very accurate

#

(the timer)

#

I mean, why wouldn't it be

#

And shutdown still works

#

The keyboard had a skill issue though, for some reason

#

Though knowing it's a mac, it probably has a usb or i2c keyboard, or some weirdness like that

#

AHCI is busted though

#

Ok, too large of a todo list

#

I'll do HPET, then TSC (deadline), then the KVM thing probably

twilit talon
#

its a common theme

#

because back in the day ids had to be unique

#

but it doesnt actually program that id in

#

btw are those found cpu lapic logs only for APs?

carmine nacelle
#

But it prints it when it finds them in madt

twilit talon
#

why isnt there a print for BSP

carmine nacelle
#

(Need to fix the typo)

#

It's all very simple trl

twilit talon
#

Coildn't

#

☠️

#

i like how u have for_each and then a handrolled iteration loop

carmine nacelle
#

Yeah, because you said that there was for_each after I did that loop, and I didn't bother with rewriting it

twilit talon
#

lol

carmine nacelle
#

I mean it works

carmine nacelle
#

thonk do I want to bother with hpet interrupts?

carmine nacelle
#

(No)

#

HPET is refusing to cooperate nooo

#

Which I think is a skill issue with units trl

#

Yeah, HPET in qemu also runs a little bit faster than it should

#

(though I'm using qemu in wsl2, so it's double virt, so it's kinda not out of the question=

#

Also, my keyboard and AHCI drivers now work consistently (with either of them)

#

I want to do TSC...

carmine nacelle
#

Why is APIC timer code on oasdev wiki in assembly??

plain jungle
#

how much faster?

carmine nacelle
#

Like the same as ACPI timer

plain jungle
#

they should all run at the proper frequency

carmine nacelle
#

And I can see it's a bit faster

#

And it was seemengly very accurate on the phys hardware

manic girder
carmine nacelle
plain jungle
#

that's almost certainly an issue on your side

carmine nacelle
#

Yeah, I think it's an issue with WSL2

plain jungle
#

56/60 deviation is huge

carmine nacelle
#

Because ACPI PM's frequency is fixed, and my kernel uses deadlines based on it, I don't think the code can be bad

#

I want to implement kvm clock and see if it improves

carmine nacelle
#

(so even if my lapic calibration is bad, it shouldn't matter)

carmine nacelle
#

Cool, it's tripple faulting if I use tsc deadline on wrmsr nooo

#

Wtf

#

I think I'll just trust that it's working and not test it trl

#

it found a ps/2 keyboard -> timers are not fucked

carmine nacelle
#

wait, the pvclock is like really nice actually?

#

it gives me the tsc frequency and time in nanoseconds, and just have to lapic dedline it and that's it??

#

No calibration or weirdness needed

left dew
#

that's kind of the whole point

twilit talon
#

What did you expect lol

carmine nacelle
#

idk, the typical x86 jank

carmine nacelle
#

I take my words back

#

I don't understand it

carmine nacelle
#

Oh no, loongarch actually has really bad timer nooo

#

Or actually not?

carmine nacelle
#

nevermind

#

loongarch is like tsc + lapic (without tsl deadline), but it's like architectural, with a fixed frequency that's given to you in cpuid

carmine nacelle
#

Apparently the timers weren't working there because I never actually called the initialization function trl

carmine nacelle
#

thonk coild the limine smp init code for risc-v be broken?

carmine nacelle
#

RISC-V is trolling me nooo

plain jungle
#

why are you switching between some many different tasks all the time lol

carmine nacelle
#

I'm just fixing the broken stuff

#

since I switched away from using limine mp request (on x86), I had to reimplement it on risc-v

#

and I kinda wanted to have smp

#

(I mean maybe I'll implement smp loongarch as well)

#

cool, it's working

#

the only broken thing now is acpi power button interrupt, for some reason

#

(also, smp trampoline on risc-v is like super easy)

carmine nacelle
#

I have to stop, since I have exams

#

(I'm disappearing till friday)

carmine nacelle
#

I think I'll add clock source, calibration source and interrupt source to kernel as an abstraction on x86

#

This way I can support hpet and kvm clock elegantly...

#

(as an interrupt source, with the kvm clock handling, etc.)

carmine nacelle
#

I think I either won't support PIT in the kernel, or make it a cmdline flag

#

Fuck me

carmine nacelle
#

Cool, I think my HPET calibration produces more or less the same frequency as ACPI PM

carmine nacelle
#

(anyway, my timer infra is proper now, and I don't use PIT anymore at all)

#

It's ACPI PM, HPET and CPUID 0x15 TSC

#

(or like the first two for calibration)

#

I think I can live without KVM timer for now...

#

(I'll maybe do it this weekend)

#

But I'm happy and this feels complete

carmine nacelle
carmine nacelle
#

I need to look at how people do POSIX servers. because what I have feels weird

carmine nacelle
#

thonk should I seek contributors?

twilit talon
#

How would that even work

#

Where would u look

carmine nacelle
#

idk

#

but like now all of the code is mine

#

(except the libraries and stuff like that)

#

idk how to say what I want to say

#

I can't keep doing it alone forever

light basin
#

I would love to contribute when I get to a point of not being utterly incompetent at C/C++ pepeflower

carmine nacelle
#

I mean I have Rust in userspace as well coolmeme

toxic torrent
#

on that note, how do you build your std?

#

my problem is that I don't know what functions i need to add to the libc crate that aren't needed for the unix group

#

actually

#

where tf IS your rust std

#

nvm

#

hm

#

did you just add random headers to the libc crate until it worked?

carmine nacelle
#

Kinda, yeah

#

I think I've just defined my os as unix, then added headers by looking at other operating systems, then implemented everything that was needed for it to link

toxic torrent
#

k

carmine nacelle
#

bruh I've tried to implement smap and smep, but qemu is trolling me

#

This doesn't detect it for some reason

#

the kernel works, but my userspace breaks with smap πŸ’€πŸ’€

toxic torrent
#

do you fault on something?

carmine nacelle
#

My mmap syscall fails πŸ’€

#

(or like not the actual mmap, but mapping memory objects)

hexed knot
#

so basically your code is broken?

carmine nacelle
#

yeah

hexed knot
#

where does it fail

carmine nacelle
#

i've no idea

#

it just fails

hexed knot
#

see this is why i put tracing probes all over my kernel so you can enable them to get debug prints all over the place πŸ™ƒ

#

(and yes they are not terribly fast, but they are very cool)

carmine nacelle
#

I though it was a copy from user thing, which didn't have smap, but apparently it's not?

#

anyway, I have an exam in a few hours and I'm just procrastinating

#

I though it would be a quick thing since I already had it on risc-v?

#

I think that my copy_from_user catches the pagefault incorrectly or something?

hexed knot
carmine nacelle
#

Yeah, I am pretty sure it's the copy_from_user stuff, because why would it fail otherwise

carmine nacelle
#

Cool, I have smap and smep on amd64

#

it was some assembly skill issue

#

in my copy to user...

carmine nacelle
carmine nacelle
#

My i686 port is trolling me nooo

#

Not anymore trl

#

Do I fix my build system?

carmine nacelle
#

Do I go with xbstrap or jinx thonk

left dew
#

totally unbiased answer: jinx

carmine nacelle
#

So I can't eventually self-host it? thonk

left dew
#

unless you want to implement linux features, no

carmine nacelle
#

Xbstrap also talks about containers...

toxic torrent
#

I don't remember the outcome of the discussion

#

being able to self host the distro is something I'd like to do at some point

plain jungle
#

xbstrap can run without containers although I wouldn't recommend that

carmine nacelle
#

though I think I'm still quite far from self hosting

carmine nacelle
#

why are all the build systems so confusing?

#

this is cool

carmine nacelle
#

what the fuck

carmine nacelle
carmine nacelle
#

Cool, it's compiling clang

toxic torrent
carmine nacelle
#

The image didn't send

#

The first result for make on Google is some ai slop?

toxic torrent
#

cursed

carmine nacelle
#

What am I supposed to do?

twilit talon
#

sudo !!

carmine nacelle
#

I know

#

But I'm trying to make jinx compile it

modest thistle
#

patch it maybe?

#

or change install dir

carmine nacelle
#

no

#

it's a host recipe

#

I did this

#!/bin/sh

name=clang
revision=1
from_source=clang-src
# hostdeps="cmake"

configure() {
    cmake -S ${source_dir}/llvm  -DCMAKE_BUILD_TYPE=Release -DLLVM_ENABLE_PROJECTS="clang;lld;lldb" -DLLVM_TARGETS_TO_BUILD="X86;RISCV;LoongArch" -DCMAKE_INSTALL_PREFIX=${prefix} -DLLVM_TARGET_TRIPLE=x86_64-pmos -DCLANG_DEFAULT_CXX_STDLIB=libc++ -DCLANG_DEFAULT_LINKER=lld
}

build() {
    make -j ${parallelism}
}

package() {
    make install
}
#

Cool, I think it worked

#

didn't know llvm also took DESTDIR

modest thistle
carmine nacelle
#

for some reason, it's not finding the directory?

carmine nacelle
#

I'm too dumb for this

#

How do I make it find clang?

#

If it's not in the path

carmine nacelle
#

Why

#!/bin/sh

name=libc
version=0.0.1
revision=1

source_dir=lib/libc
deps="libc-headers"
hostdeps="clang"

configure() {
    cmake -S ${source_dir} -DSYSROOT=${sysroot_dir} -DTARGET_ARCH=${JINX_ARCH} -DINSTALL_DIR="${dest_dir}/usr/lib"
}

build() {
    make -j ${parallelism}
}

package() {
    INSTALL_DIR="${dest_dir}/usr/lib" make install
}
left dew
#

DESTDIR

#

for make install

carmine nacelle
#

This is my CMake

cmake_minimum_required(VERSION 3.22)
set(CMAKE_C_STANDARD 17)

set(TOOLCHAIN_PREFIX "${TARGET_ARCH}-pmos")

set(CMAKE_C_COMPILER "clang")
set(CMAKE_ASM_COMPILER "clang")
set(CMAKE_AR "llvm-ar")

SET(CMAKE_C_COMPILER_TARGET ${TOOLCHAIN_PREFIX})
SET(CMAKE_ASM_COMPILER_TARGET ${TOOLCHAIN_PREFIX})

set(CMAKE_TRY_COMPILE_TARGET_TYPE "STATIC_LIBRARY")
set(CMAKE_ASM_OUTPUT_EXTENSION_REPLACE 1)
set(CMAKE_C_OUTPUT_EXTENSION_REPLACE 1)

set(CMAKE_C_FLAGS "-Wall -Wextra -O3 -g -nostdlib -fPIC --sysroot=${SYSROOT} -fno-omit-frame-pointer -mno-omit-leaf-frame-pointer -flto -ffat-lto-objects")

if (${TARGET_ARCH} STREQUAL "riscv64")
    set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -march=rva20u64 -mabi=lp64d")
endif()

set(CMAKE_ASM_FLAGS ${CMAKE_C_FLAGS})

project(libc VERSION 0.1 LANGUAGES C ASM)

set(SRC_FOLDER "src")

file(GLOB_RECURSE GENERIC_SRC "${SRC_FOLDER}/generic/*.c" "${SRC_FOLDER}/generic/*.S")
file(GLOB_RECURSE ARCH_SRC "${SRC_FOLDER}/arch/${TARGET_ARCH}/*.c" "${SRC_FOLDER}/arch/${TARGET_ARCH}/*.S")

list(FILTER ARCH_SRC EXCLUDE REGEX ".*/crti\.S$")
list(FILTER ARCH_SRC EXCLUDE REGEX ".*/crtn\.S$")
list(FILTER ARCH_SRC EXCLUDE REGEX ".*/crt0\.S$")

set(SRC ${GENERIC_SRC} ${ARCH_SRC})

# GCC shenanigans
# https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67618
# https://github.com/zephyrproject-rtos/zephyr/issues/64941
# https://github.com/zephyrproject-rtos/zephyr/pull/65082
# https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html
#
# Apply -fno-inline-malloc so GCC doesn't optimize calloc to itself, creating infinite recursion
set_source_files_properties(${SRC_FOLDER}/generic/malloc/malloc.c PROPERTIES COMPILE_FLAGS "-fno-builtin-malloc -fno-builtin-free -fno-builtin-calloc -fno-builtin-realloc")
set_source_files_properties(${SRC_FOLDER}/generic/string.c PROPERTIES COMPILE_FLAGS "-fno-builtin")

add_library(csrc OBJECT ${SRC})
add_library(c STATIC $<TARGET_OBJECTS:csrc>)
# add_library(c_static SHARED $<TARGET_OBJECTS:csrc>)

# set_target_properties(c_static PROPERTIES OUTPUT_NAME c)

# crti and crtn
add_library(crtstuff OBJECT ${SRC_FOLDER}/arch/${TARGET_ARCH}/crti.S ${SRC_FOLDER}/arch/${TARGET_ARCH}/crtn.S ${SRC_FOLDER}/arch/${TARGET_ARCH}/crt0.S)

# set(INSTALL_DIR "${CMAKE_CURRENT_SOURCE_DIR}/../../sysroot/usr/lib" CACHE PATH "Installation directory")
install(TARGETS c ARCHIVE DESTINATION ${INSTALL_DIR})
# install(TARGETS c_static DESTINATION ${INSTALL_DIR})

install(FILES $<TARGET_OBJECTS:crtstuff> DESTINATION ${INSTALL_DIR})
#

I don't get it

#

It worked the second time...

left dew
#

i'm pretty sure cmake just handles DESTDIR for you out of the box

#

no need for all that INSTALL_DIR stuff

carmine nacelle
#

Or not

#

I had to do this??

#!/bin/sh

name=libc
version=0.0.1
revision=1

source_dir=lib/libc
deps="libc-headers"
hostdeps="clang"

configure() {
    cmake -S ${source_dir} -DSYSROOT=${sysroot_dir} -DTARGET_ARCH=${JINX_ARCH} -DDESTDIR="${dest_dir}/usr/lib"
}

build() {
    make -j ${parallelism}
}

package() {
    DESTDIR="${dest_dir}" make install
}
#

Ok, I've made it work...

left dew
#

you don't need DESTDIR when configuting

carmine nacelle
#

yeah

carmine nacelle
#

Jinx's trolling me again nooo

#
#!/bin/sh

name=kernel
version=0.0.1
revision=1

source_dir=kernel
deps="libc-headers libcxx"
hostdeps="clang"

configure() {
    cmake -S ${source_dir} -DTARGET_ARCH=${JINX_ARCH} -DSYSROOT=${sysroot_dir}
}

build() {
    make -j ${parallelism}
}
left dew
#

yeah? you don't have a package step bro

#

you're trolling yourself

carmine nacelle
#

If I add it it still does it

#
#!/bin/sh

name=kernel
version=0.0.1
revision=1

source_dir=kernel
deps="libc-headers libcxx"
hostdeps="clang"

configure() {
    cmake -S ${source_dir} -DTARGET_ARCH=${JINX_ARCH} -DSYSROOT=${sysroot_dir}
}

build() {
    make -j ${parallelism}
}

package() {
    DESTDIR="${dest_dir}" make install
}
left dew
#

well i didn't know what the error was

#

clearly it has nothing to do with the step missing

#

try to rebuild libcxx

carmine nacelle
#

I tried twice

#

And renamed it from libc++ to libcxx

left dew
#

idk i dont have the brain capcity to even think of a possible solution

#

maybe it's the rc in the name

#

who knows

#

rindex is failing for some reason

#

it was added libcxx to the pool

carmine nacelle
#

It doesn't have version

#

Wouldn't it infer it from the source?

left dew
#

well that'd do it

#

uh yes

#

it would

#

show recipe pls

carmine nacelle
#
#!/bin/sh

name=libcxx
revision=1
from_source=clang-src
deps="libc-headers libc"
hostdeps="clang"

triplet="${JINX_ARCH}-pmos"

configure() {
    cmake -S ${source_dir}/runtimes \
        -DLLVM_ENABLE_RUNTIMES="libcxx;libcxxabi;libunwind;compiler-rt" \
        -DCMAKE_BUILD_TYPE=Release \
        -DCMAKE_INSTALL_PREFIX=/usr/ \
        -DLLVM_TARGET_TRIPLE=loongarch64-pmos \
        -DCMAKE_CXX_COMPILER_TARGET="${triplet}" \
        -DCMAKE_C_COMPILER="clang" \
        -DCMAKE_CXX_COMPILER="clang++" \
        -DCMAKE_C_COMPILER_TARGET="${triplet}" \
        -DCMAKE_ASM_COMPILER_TARGET="${triplet}" \
        -DCOMPILER_RT_BUILD_BUILTINS=ON \
        -DCMAKE_C_FLAGS="--sysroot=${sysroot_dir} -nostartfiles" \
        -DCMAKE_CXX_FLAGS="--sysroot=${sysroot_dir} -nostdlib" \
        -DLIBCXXABI_USE_LLVM_UNWINDER=true \
        -DLIBCXX_ENABLE_SHARED=OFF \
        -DLIBUNWIND_ENABLE_SHARED=OFF \
        -DLIBCXXABI_ENABLE_SHARED=OFF \
        -DCOMPILER_RT_BUILD_BUILTINS=ON \
        -DCOMPILER_RT_BUILD_ORC=OFF \
        -DCOMPILER_RT_BUILD_SANITIZERS=OFF \
        -DCOMPILER_RT_BUILD_MEMPROF=OFF \
        -DCOMPILER_RT_BUILD_XRAY=OFF \
        -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
        -DCOMPILER_RT_BUILD_PROFILE=OFF \
        -DCOMPILER_RT_INSTALL_LIBRARY_DIR="/usr/lib" \
        -DCOMPILER_RT_BUILD_CTX_PROFILE=OFF
}

build() {
    make -j ${parallelism}
}

package() {
    DESTDIR="${dest_dir}" make install
}
left dew
#

should be good

carmine nacelle
#

I've added version and revision to libcxx and rebuilt it

#

If I remove libcxx, it starts to build it (but obviously can't because of the missing headers)

#

And like idk, there shouldn't be anything wrong

#

And like also it had built clang fine from the same source

#

yeah, I gues it really doesn't like the -

#

I've changed clang to 19.1.0_rc3 and got this

#

Ok, the kernel is compiling...

#

I guess the userspace shoudn't be too difficult

#

I just need to figure out how to package it into a disk file, and generate bootloader configs

carmine nacelle
#

Is there something to do conditional building?

#

I think I'll make a clanker generate a script trl

#

I think that it inherits the version, but not revision, from the source recipe?

carmine nacelle
#

Time to figure out how to compile rust

carmine nacelle
#

But tbh it's much nicer than my make insanity so far

#

(Also, I've switched everything to CMake trl)

#

I'm too flat brained to figure out out of tree gnumake compilation

carmine nacelle
carmine nacelle
#

I'm too dumb for this nooo

#

(does anyone know how to fix it?)

#

I think it still doesn't fix my issue gdtdoesnotwork

left dew
#

it would be nice if you shared the recipe

left dew
#

lmao what

#

just rm -rf sources/rust-src*

#

and see if that fixes it

carmine nacelle
#

I think it really doesn't like - in names...

#

and _

left dew
#

it should be fine

#

example: any project using jinx

carmine nacelle
#

I've changed the recipe to just rust and now it seems to be compiling?

#

Though my another complaint is that it fails easily on a flaky wifi connection

left dew
#

well that doesn’t sound like a jinx issue?

carmine nacelle
#

Everything else retries

#

anyway

#
running: /build_dir/host-builds/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --jobs=default --manifest-path /base_dir/sources/rust/src/bootstrap/Cargo.toml -Zroot-dir=/base_dir/sources/rust -Zwarnings
warning: unused config key `build.docs` in `/build_dir/.cargo/config.toml`
error: failed to run `rustc` to learn about target-specific information

Caused by:
  process didn't exit successfully: `/build_dir/host-builds/rust/build/x86_64-unknown-linux-gnu/stage0/bin/rustc - --crate-name ___ --print=file-names -Zallow-features= -Wrust_2018_idioms -Wunused_lifetimes --target x86_64-unknown-pmos --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=split-debuginfo --print=crate-name --print=cfg -Wwarnings` (exit status: 1)
  --- stderr
  error: error loading target specification: could not find specification for target "x86_64-unknown-pmos"
    |
    = help: run `rustc --print target-list` for a list of built-in targets

Traceback (most recent call last):
  File "/base_dir/sources/rust/x.py", line 53, in <module>
    bootstrap.main()
    ~~~~~~~~~~~~~~^^
  File "/base_dir/sources/rust/src/bootstrap/bootstrap.py", line 1413, in main
    bootstrap(args)
    ~~~~~~~~~^^^^^^
  File "/base_dir/sources/rust/src/bootstrap/bootstrap.py", line 1363, in bootstrap
    build.build_bootstrap()
    ~~~~~~~~~~~~~~~~~~~~~^^
  File "/base_dir/sources/rust/src/bootstrap/bootstrap.py", line 1030, in build_bootstrap
    run(args, env=env, verbose=self.verbose, cwd=self.rust_root)
    ~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/base_dir/sources/rust/src/bootstrap/bootstrap.py", line 237, in run
    raise RuntimeError(err)
RuntimeError: failed to run: /build_dir/host-builds/rust/build/x86_64-unknown-linux-gnu/stage0/bin/cargo build --jobs=default --manifest-path /base_dir/sources/rust/src/bootstrap/Cargo.toml -Zroot-dir=/base_dir/sources/rust -Zwarnings
carmine nacelle
#

No, this is weird

#

there is no sources??

#

wait

#

It's weird

carmine nacelle
#
#!/bin/sh

name=rust
revision=1
from_source=rust
allow_network=yes

configure() {
    cat > config.toml <<EOL
[llvm]
download-ci-llvm = false
targets = "X86"

[build]
target = ["x86_64-unknown-pmos", "x86_64-unknown-linux-gnu"]
build-dir = "${base_dir}/host-builds/rust"
docs = false

[install]
prefix = "${prefix}"
sysconfdir = "${prefix}/etc"
EOL
}

build() {
    CARGO_HOME="${build_dir}/.cargo" ${source_dir}/x.py build --stage 2 -j${parallelism} --verbose 
}

package() {
    CARGO_HOME="${build_dir}/.cargo" DESTDIR="${dest_dir}" ${source_dir}/x.py install -j${parallelism}

    
    find ${dest_dir} -name "*.old" -delete

    strip_command=strip
    post_package_strip
}
#!/bin/sh

name=rust
version=1.0.0
revision=1
git_url=https://gitlab.com/mishakov/rust-pmos.git
commit="39dedc845fe6715741cf9a7d425cd389f9e26f0f"

early_prepare() {
    git submodule update --init --recursive --depth=1
}
#

ok, nevermind

carmine nacelle
#

I think it's ignoring my bootstrap.toml?

#

I think I've figured it out...

toxic torrent
#

TIL

carmine nacelle
#

I think it compiled?

#

what's post package strip anyway?

toxic torrent
carmine nacelle
#

no, i mean where is it coming from?

toxic torrent
#

Jinxfile

#

did you yoink this recipe from somewhere

carmine nacelle
#

maybe trl

#

I mean I yoinked it from 5 places

toxic torrent
#

well you have this thing called a Jinxfile

carmine nacelle
#

i know

toxic torrent
#
post_package_strip() {
    if [ -z "$strip_command" ]; then
        strip_command="${OS_TRIPLET}-strip"
    fi

    for f in $(find "${dest_dir}"); do
        if file "$f" | grep 'not stripped' >/dev/null; then
            echo "* stripping '$f'..."
            stripped_file="$(mktemp)"
            ${strip_command} "$f" -o "$stripped_file"
            chmod --reference="$f" "$stripped_file"
            mv -f "$stripped_file" "$f"
        fi
    done
}
carmine nacelle
#

it's just a bunch of shell scripts

toxic torrent
#

this is what i have in there

carmine nacelle
#

I'm too dumb for this

#

it didn't build cargo??

toxic torrent
#

isn't cargo a separate repo

carmine nacelle
#

Idk

#

Clanker said it isn't

#

But I don't get it

#

The recipes that I've seen just run cargo install on cargo

#

But like where is it coming from?

toxic torrent
#

i have cargo and rustc

carmine nacelle
#

yeah, I've seen it

#

Imagedeps???

#

Also, is there a way to get the host tripple?

#

I wanted to run the build server on an arm board

modest thistle
carmine nacelle
#

idk

#

maybe?

carmine nacelle
#

I don't get it

#

I yoinked the cargo recipe, and it should be working fine, but it still can't find cargo

toxic torrent
carmine nacelle
toxic torrent
#

did you add the cargo to hostdeps

carmine nacelle
#

yes

toxic torrent
#

show

carmine nacelle
#

it's just ignoring it

toxic torrent
#

my cargo recipe is different

carmine nacelle
#

find install thing doesn't fail, so it should be fine

toxic torrent
#

cargo install?

carmine nacelle
#

no

#

I can't find where I stole it from trl

toxic torrent
#
#! /bin/sh

name=cargo
from_source=cargo
revision=1
imagedeps="build-essential libssl-dev pkg-config cargo"
hostrundeps="gcc rust"
allow_network=yes

build() {
        CARGO_HOME="${build_dir}/.cargo" \
    cargo install \
        --target x86_64-unknown-linux-gnu \
        --target-dir "${build_dir}" \
        --path "${source_dir}" \
        -j${parallelism} \
        --root "${build_dir}/out"
}

package() {
        CARGO_HOME="${build_dir}/.cargo" \
    cargo install \
        --target x86_64-unknown-linux-gnu \
        --target-dir "${build_dir}" \
        --path "${source_dir}" \
        -j${parallelism} \
        --root "${dest_dir}${prefix}"

    # Strip binaries
    strip_command=strip \
    post_package_strip
}
#

this is mine

carmine nacelle
#

I'm rebuilding it

carmine nacelle
#

this has to be something dumb

#

If I add ls /usr/local, it's empty

#

Even though it does install it

carmine nacelle
#

Yeah, it's ignoring hostdeps for some reason...

carmine nacelle
#

It works if I comment source_dir out?????

carmine nacelle
#

Yeah, it breaks as soon as I add it...

carmine nacelle
#

Just if I remove .jinx-cache and sources and build directory, and run ../jinx build pmbus with this recipe, it just gets the debian image, and doesn't even try to compile the compilers and anything

#!/bin/sh

name=pmbus
version=1.0.0
revision=1

source_dir=userspace/ps2d
deps="libc pmos-rust-lib"
hostdeps="cargo rust"
allow_network=yes

prepare() {
    CARGO_HOME="${build_dir}/.cargo" \
    cargo fetch --target ${JINX_ARCH}-unknown-pmos
}

build() {
    CARGO_HOME="${build_dir}/.cargo" \
    cargo build --target ${JINX_ARCH}-unknown-pmos --jobs ${parallelism}
}

package() {
    CARGO_HOME="${build_dir}/.cargo" \
    DESTDIR="${dest_dir}" \
    RUSTFLAGS="-C link-arg=--sysroot=${SYSROOT} -C linker=clang++ -C link-arg=--target=${JINX_ARCH}-unknown-pmos -C link-arg=-lunwind -C link-arg=-static" \
    cargo install --target ${JINX_ARCH}-unknown-pmos --root ${dest_dir} --path ${source_dir}
}
left dew
#

because the jinx cache isnt where the built tools are

#

make sure the cargo host tool is built correctly

#

i think the directory is called host-pkgs

#

there should be cargo in there

#

and it should be installed under $prefix, so /ust/local for tools

#

so the final path for the executable should be host-pkgs/cargo/usr/local/bin/cargo

carmine nacelle
left dew
#

ls host-pkgs/cargo

#

it probably built but is incomplete somehow

#

maybe your recipe is messed up

carmine nacelle
carmine nacelle
left dew
#

where is ANYTHING

carmine nacelle
#

if I comment source_dir out then it starts doing some stuff

left dew
#

where is binutils even

carmine nacelle
#

like what it's set to

#

and shell scripts are too complicated for me idk

left dew
#

how is shell complicated πŸ’€

#

anyway let me clone and try building

#

or just take a look but on pc this time

#

which jinx version are you using?

carmine nacelle
#

the latest from main

#

tbh I need to make a makefile for it

left dew
#

i see you didn't check in the jinx script and you don't have a makefile target to download it

#

yeah

#

that'd be nice

carmine nacelle
#

0.8 I think

left dew
#

ok time to build this thing

carmine nacelle
#

do you have a fast PC?

#

it gets to fail quickly on my 5900X

left dew
#

yeah

left dew
#

ok it took me way too long to find out

#

but

#

you are running cargo in prepare

#

that's part of the source recipe, not the recipe itself

#

so you need source_hostdeps="cargo rust"

#

after changing source_dir to pmbus:```
[iretq@raptor build]$ ../jinx build pmbus

  • building package: pmbus
    error: failed to get pmos as a dependency of package pmbus v0.1.0 (/base_dir/userspace/pmbus)

Caused by:
failed to load source for dependency pmos

Caused by:
Unable to update /base_dir/userspace/rust/pmos

Caused by:
failed to read /base_dir/userspace/rust/pmos/Cargo.toml

Caused by:
No such file or directory (os error 2)```

carmine nacelle
#

Thanks a lot

toxic torrent
carmine nacelle
#

More trolling gdtdoesnotwork

carmine nacelle
#

(I'll probably figure it out tomorrow)

carmine nacelle
#

Maybe I should just retire Rust trl

carmine nacelle
#

I think it's finally building it

carmine nacelle
#

I've no idea what I'm doing

carmine nacelle
#

I don't have dynamic linker

#

I'm not using mlibc and my libc is too dumb

toxic torrent
#

bruh

#

πŸ’€

carmine nacelle
#

Bruh, I think it's working

#

just need to figure out cargo install

toxic torrent
#

holy

carmine nacelle
#

Time to recompile rustc for the millionth time...

#

Also, I need a script to assemble all of this into an image

carmine nacelle
#

It takes forever on my laptop...

twilit talon
#

Thats why ill never do rust shit

#

This toolchain situation is miserable

toxic torrent
#

i like rust as a language

#

but jfc tooling is horrible

carmine nacelle
#

Nah, I think I've gotten it to work

#

It's just because nobody's using rust with Jinx, and it's not very well documented?

carmine nacelle
toxic torrent
#

why

carmine nacelle
#

config.toml didn't work for me

#

idk, I've made mine use bootstrap.toml

carmine nacelle
carmine nacelle
#

Jinx is trolling again (totally not because shit build scripts done by me)

carmine nacelle
#

What am I looking at?

#

Why does everything build limine as a normal recipe and not host recipe

#

Time to downgrade limine trl

twilit talon
#

Why do u need such an old revision

carmine nacelle
#

Or upgrade it

#

I yoinked the 10.6.0 recipe

#

Cool, I've managed to build the system image...

#

The only issue is that userspace doesn't work for some reason...

carmine nacelle
#

Cool, it fully works

carmine nacelle
#

everyone, not everything

toxic torrent
#

only i build it as a normal recipe

carmine nacelle
#

I can't speak today

carmine nacelle
toxic torrent
#

astral for example doesn't

#

i do it because i have no need for limine's host tool

#

and it's more like a normal package for my os

carmine nacelle
#

This is very nice of flanterm...

plain jungle
toxic torrent
#

cp

#

i don't support bios boot anyways

#

i just copy the BOOTX64.EFI to the esp

#

that's it

mellow sleet
carmine nacelle
#

bruh I need to figure out risc-v and other arches now...

#

But I haven't seen $ARCH be used anywhere?

#

Is it normal that there are 2 variables?

carmine nacelle
#

bruh Ubuntu's repos strike again

carmine nacelle
#

This is fun

#

cool, the latest SBI doesn't even boot

vale hinge
carmine nacelle
vale hinge
#

really?

twilit talon
toxic torrent
carmine nacelle
#

Time to set up the build server...

carmine nacelle
#

lol, who even uses docker (coudflare) on weekends...

carmine nacelle
#

(it's blocked by my isp again)

modest thistle
carmine nacelle
#

I guess I'll just wait

carmine nacelle
#

I think I'll merge it to main today

#

(I can send patches)

carmine nacelle
#

You can now actually build everything with make qemu-x86

#

(if someone wants to try it for some reason)

carmine nacelle
#

Bruh, I want to do disk...

carmine nacelle
carmine nacelle
#

I think it's a printf skill issue trl

carmine nacelle
#

I think I should finally add rights to memory objects?
But I don't really know how to handle them (like do I do weak/strong references??)

carmine nacelle
carmine nacelle
#

This is nice...

carmine nacelle
#

Also, I think I figured out how I should do multithreaded servers?

#

I'll just make programs (e.g. libc) explicitly request rights that correspond to the right cores

#

The kernel doesn't have to do anything

carmine nacelle
#

I think I'd need my own runtime for my IPC?

plain jungle
#

yes

#

a runtime is really small in rust though

carmine nacelle
#

I wonder how fucked my userspace would be if I introduce blocking send() thonk trl

#

Which I think I kinda have to do for Rust...

plain jungle
#

Hm?

#

Which I think I kinda have to do for Rust...
^ this sounds weird

#

Rust doesn't force you into a particular IPC model?

carmine nacelle
#

My kernel only lets me get the message that's at the front of the queue, and I think I'll have to have a future per right, so I'll have to get messages eagerly and push those the the future's queue (so there aren't deadlocks), so someone can easily DOS my rust servers by sending a lot of messages and making it OOM

#

So the solution is limiting how many messages can be sent

#

Idk, I still don't quite get what I have to do

plain jungle
#

how do you avoid this in C?

#

there should be no need to have a separate queue that buffers stuff

#

Rust's future design certainly doesn't require it

carmine nacelle
#

I mean maybe it doesn't matter

#

It's only an issue if you don't immediately await on the future

#

I mean unbounded queues are problematic in general?

plain jungle
#

Rust doesn't require you to add an unbounded queue

carmine nacelle
#

they are unbounded in the kernel

plain jungle
#

that sounds like a problem indeed

#

but anyway, I don't see how Rust futures make it more (or less) problematic

carmine nacelle
#

Yeah, but I might as well fix it

carmine nacelle
plain jungle
#

why does that matter?

#

you can convert callback based code to Rust futures or vice versa

carmine nacelle
plain jungle
#

i don't get it. you can have a rust future that registers a callback and completes the future on completion

#

you just have some kind of control block that stores the future's current waker

#

or similar

carmine nacelle
#

I want send to make new futures

#

It's not a problem with the callbacks/futures, but with the order of doing things being important if I carry on with unbound queues

#

in kernel

#

Since I'd have to buffer everything in userspace, and it's bad

#

(infinitely)

#

So I think I'll make it bounded

#

This would also solve the problem of not needing to allocate on every send

#

Like if I make it bounded, then the buffers can be preallocated, and it can't fail on OOM or weirdness like that, which is kinda important for drivers

#

(Since I can't just block them on oom)

plain jungle
#

there is no need to buffer anything in user space

carmine nacelle
#

But still, it's problematic

#

Like you create 2 futures, then if await on only one of them, the second one can deadlock everything

plain jungle
#

why?

carmine nacelle
#

Also, I think my IPC is a bit different from Managarm's?

carmine nacelle
plain jungle
#

you want the future to do the actual work at await time, not at future creation time

carmine nacelle
#

Idk if I'm not explaining it well

#

But I have (recieve) ports, which then have many recieve rights (bound to a port)

#

You can only get messages from the front of the port

#

I want to make futures wait on rights

#

So if there is a message for the given right, and I can't move it to the future (it's not being polled), I have to block

#

So it creates a deadlock

#

Or I'll have to throw messages away

#

Or I have to buffer them indefinitely in userspace

#

Or you have to wait on all of the futures immediately

carmine nacelle
#

The Stream/Waker would be polling the message queue

#

I think managarm just lets you wait on any stream, no?

plain jungle
#

wdym by "on any stream"? Managarm allows you to submit multiple IPC ops concurrently

carmine nacelle
#

My kernel doesn't

plain jungle
#

and wait for their completion

#

But messages on each stream are strictly ordered

carmine nacelle
#

Yes, but can you wait on several ops from one thread at the same time?

plain jungle
#

yes. but if you can't do that, why are you trying to use an async runtime anyway

carmine nacelle
#

I don't know how to explain it

#

It works if you buffer it

#

My AHCI driver is fully async in C++

left dew
#

why does it matter which operation completes in what order?

#

i don't quite understand what your problem with rust's async is

#

ah i think i see your problem

carmine nacelle
#

Let's say you have rights A and B, both can recieve multiple messages
You receive a a b in this order (which is buffered in kernel)
You would get a, move that to the future, then you would get a again. You either have to throw it away, or buffer it, or block on it. If you block on a, but you're awaiting on B before a, you can't make progress, since you can't recieve b because a is at the front

#

I mean I had an argument with a clanker and it really insisted that I don't do that

#

If you buffer a a in stream, then you can recieve b and there is no issue. But since my queues are unbounded, and there is no way to stop new messages, someone can just send a lot of messages, and it might DoS my server

left dew
#

i would have to look at your ipc model more closely but it really doesn't sound too bad, do you have a kernel api that lets you receive something? you're saying "recv A, recv A, recv B" - what syscalls are involved in this process? what are A and B in this case?

#

actually lemme take a look at the ipc link you posted

carmine nacelle
#

I think my async C++ thing is also somewhere in there

#

Like I can just ignore it and keep the order in mind and it will be fine, but it won't be "fearless concurrency"

left dew
#

sounds to me like you need a select-like syscall

carmine nacelle
#

yes

#

or that

left dew
#

or recv-many or whatever

carmine nacelle
#

But my queues are just a list/deque of messages

left dew
#

same idea i guess

carmine nacelle
#

So I would need to redo it

#

Or, if you make it bounded, then you can preallocate the buffers in userspace, and just do it in there

plain jungle
#

You get a, process it, you create another future that gets the next a, you process it, you get b

left dew
#

that's the "i can just keep the order right myself" thing

#

seems like that's also how it works in c++

carmine nacelle
#

It's not really a future, but a Stream in Rust

carmine nacelle
plain jungle
#

I think your issue stems from confusion about push and pull

carmine nacelle
#

It would be both I guess?

plain jungle
#

Rust futures/streams pull things from an object

#

It seems that you want to make it push based

carmine nacelle
#

I guess?

plain jungle
#

which doesn't work

#

So the fix is to not make it push based

carmine nacelle
#

But my kernel doesn't have the API to make it pull based

#

And I would need to move the message queues to the rights

#

If I want to be able to pull on rights instead of ports

carmine nacelle
#

I wanted to add a field that says that it's not the last message in the queue

#

So you can push several messages into the futures

#

Also, it doesn't matter, it's an abstraction over IPC

#

If something builds on top of it, it would be pull based

plain jungle
#

Ah i think i get your problem now

#

You want to distribute messages from a common receiver to multiple consumers but you don't know the consumers in advance

carmine nacelle
#

yes

plain jungle
#

Yeah that sounds like a design issue indeed

carmine nacelle
#

Or if I know them in advance, then I have to buffer

plain jungle
#

But one that you'd also have with callbacks

carmine nacelle
#

yeah

#

that's why I said C++ is also problematic

#

But with C/C++ you can just say "it's on you to do the things in the right order"

plain jungle
#

Well one thing that you can do is have a stream that returns (message, target) pairs

#

And have higher level logic deal with this issue

carmine nacelle
#

I don't want to do that

#

Then why would you want async in the first place

#

If you can only wait on one port

carmine nacelle
plain jungle
#

That's true

carmine nacelle
#

I mean that higher level logic is what I'm trying to write

plain jungle
#

But higher level logic may be better equipped to solve it (maybe. Depends on your design)

carmine nacelle
#

But unbounded queues are also problematic on other abstractions/stuff that builds on top of it, e.g. Unix pipes

#

Which have to block if they're full

#

Or not...

#

No, with pipes, you'll just wait for the reply

#

Idr, there were a bunch of issues with having no way to set a boundary on how many messages can be in the queue

#

But with my ports/rights thing, it should be somewhat straightforward to be able to add limits on rights

#

(with the biggest issue being the scheduler/blocking, which needs improving)

#

Which I also need for futex

carmine nacelle
#

thonk What if I do VFS in Rust?

#

which would be a test for my async thing...

carmine nacelle
#

Though I think my issue is that I don't quite understand VFS

#

Namely how the directory tree should be, what to do with mountpoints and hardlinks...

carmine nacelle
carmine nacelle
#

Small progress update -- I ended up doing it in Rust

#

And I'm currently fighting the borrow checker and stuff like that

carmine nacelle
#

This insanity works??
(create_named_stream does a request to bootstrapd)

#

(also, I think mut isn't needed here...)

carmine nacelle
#

Anyway, since I can't be bothered with writing a proper posix_spawn, I think I'll be launching filesystem drivers through bootstrapd

#

Namely, I think I'll first be launching drivers to probe the partitions, and then doing it again on mount

#

I need to look into the other operating systems thonk

carmine nacelle
#

The clanker just repeated what I said to it

plain jungle
#

whenever somebody says that AI will fully replace software engineers in the short term I am reminded of stuff like this

carmine nacelle
#

Wait, are all filesystems all inside of libblockfs and POSIX server in managarm?

hexed acorn
#

on-disk fses are handled by libblockfs yes

#

but its a dso that is linked into storage drivers, not a central server

carmine nacelle
#

I wanted to make my disk drivers and filesystems be separate servers...

carmine nacelle
plain jungle
#

I don't think separate servers are substantially more complicated