#Astral

1 messages · Page 3 of 1

prime juniper
#

sshd needs tcp server side

#

ssh had some funnies with resolving which port to connect to

#

so doesn't work (yet)

grand shadow
#

I have tcp server side so I should see if it works once I make my tcp stack less bad

split gazelle
#

minecraft bedrock is written in c++ and has microsoft code in it

lean kettle
#

minecraft bedrock is basically a shared library

grand shadow
#

for all the haters who doubted me

#

tcp can now download at a miraculous rate of ~300 kb/s

#

technology is awesome

#

time to build https support into this thing

prime juniper
#

Nice job

lean kettle
#

now im interested in finding out how fast managarms TCP/IP stack is

#

i dont think its actually that slow

grand shadow
#

I'm going to port openssl and try to download some big file and see how fast this can get

robust geyser
#

common monolithic kernel W

#

(for astral obviously)

robust geyser
# grand shadow

is the bottleneck now your connection? the fs? or still the netstack?

lean kettle
#

i tried seeing if i could install speedtest-cli with pip on managarm but i realised that

  1. we dont ship pip
  2. venv doesnt work lmao
robust geyser
#

i kinda want to port my xhci driver+usb stack to astral, it could be a fun challenge

lean kettle
#

i just got 550 kb/s

#

common managarm W

robust geyser
#

maybe it was related to the level of logging dennis had configured

#

i think he was logging quite a lot

lean kettle
#

probably more related to dennis iirc still running his vm with a single core and without kvm because reasons

robust geyser
#

nah hes running multicore with kvm

lean kettle
#

although i am probably also logging a bit less than he is

#

i am logging all posix requests tho

#

without logging requests i get between 700kb and 1000kb/s

grand shadow
analog berry
lean kettle
#

as much as i know we have bigger problems, i kinda want to optimize the net stack now halfmemeright

grand shadow
#

wtf is this build system agh

lean kettle
#

currently trying to make managarm system compile with -Ofast instead of -O2

robust geyser
#

yeah openssl is terrible lmao

lean kettle
#

we need a victory

robust geyser
#

let me build astral

lean kettle
#

although astral is also super based and impressive

grand shadow
#

I will rename astral to tcpstral and focus all my effort on the netstack

robust geyser
#

ill get that going at a gigabit soon

#

:^)

grand shadow
#

I don't even have tcp upstreamed yet

lean kettle
#

all hobby os's that can get shit working are automatically based (unless they are written in rust)

grand shadow
#

and the code is bad tbh

robust geyser
#

:^(

robust geyser
#

let me try and build astral anyway

grand shadow
#

do it

robust geyser
#

jinx, nice

grand shadow
#

honestly I will just take heavy inspiration™️ from the managarm recipes for this I'm not gonna spend the next few hours figuring this out

#

mammogram wins again

analog berry
#

Does that make it based

robust geyser
#

i recommend those instead ;^)

grand shadow
#

does vinix have openssl

robust geyser
#

of course

#

or at least

#

it should

#

maybe not...

grand shadow
#

it does

robust geyser
#

damn

#

i wonder what it needs it for

grand shadow
#

apparently taisei and libzip

robust geyser
#

hmm fs/tmpfs.c: In function ‘tmpfs_link’: fs/tmpfs.c:248:23: warning: comparison between pointer and integer 248 | if (node->vfs == V_TYPE_DIR) | ^~ [CC] io/block/vioblk.o [CC] io/pci.o io/block/nvme.c:10:10: fatal error: event.h: No such file or directory 10 | #include <event.h> | ^~~~~~~~~ compilation terminated.

analog berry
#

Wait why is vfs pointer storing the file type

grand shadow
#

wait wtf

#

how the fuck did I not notice that

#

aaaaaa I thought I nuked all includes of event.h

#

that had a nasty lost wakeup problem

#

@robust geyser pull now

robust geyser
#

yeah that seems to have worked, thanks

grand shadow
#

I do have like 12381292183129 unstaged changes rn so once I get openssl ported I will probably upstream them

robust geyser
#

now building the rest of the distro

#

funnn

grand shadow
#

well https is here

robust geyser
#

based

grand shadow
#

gonna see that certificate stuff

wild marsh
#

that's what debugoptimized is

robust geyser
#

i notice that sometimes astral is eating my keystrokes

#

when typing fast in the terminal sometimes it skips a key or two

grand shadow
#

weird, that doesn't happen here

#

is it on fvwm or the console

robust geyser
#

fvwm

#

i just boot using the default option

grand shadow
#

what is your qemu version

robust geyser
#

uhhh

#

8.2.4

grand shadow
#

hm then that weird bug isn't supposed to happen

#

maybe its because of the mouse focus thing

#

that never happened to me

robust geyser
#

doesnt happen anymore, weird i agree

#

also nvme boot not working, is that expected?

lean kettle
#

i checked by adding an #error to a file and checking the command

#

also compiling as release + removing a couple logs gives me a consistent 1MB/s

#

http is also slower than https for some reason, lmao

robust geyser
#

__assert(devfs_getbyname(root, &backing) == 0); in kernel_entry fails when using any of the nvme boot options

#

im assuming the nvme device just isnt there

wild marsh
robust geyser
#

which would make sense, i dont see anything on the qemu cmdline regarding that

wild marsh
#

not -Og -g

#

my bad

lean kettle
#

im going to try -Ofast now meme

wild marsh
#

Ofast won't help for managarm

grand shadow
lean kettle
grand shadow
#

there isn't even an nvme drive attached to the vm

wild marsh
#

or well, it won't be any better than O3

grand shadow
#

you can use vioblk as well (and change the root in limine.cfg)

robust geyser
#

also, have you thought about shipping a clang-format file?

prime juniper
wild marsh
grand shadow
lean kettle
prime juniper
robust geyser
#

basically a tool to format your code

#

you can configure it using a .clang-format file

prime juniper
robust geyser
#

a lot of people like to format code to keep it consistent, and a clang-format file would be nice for other people to have a way to follow the code style of astral

prime juniper
robust geyser
#

with all due respect

#

jinx > xbstrap

#

and you know it :^)

prime juniper
lean kettle
#

no

grand shadow
#

I should get lynx ported so I don't have to manually type in the download link of stuff every time

prime juniper
grand shadow
#

its not webkitgtk but its something

prime juniper
#

But I have a text mode webbrowser in managarm

robust geyser
# grand shadow hmmmmm

i mean i dont want to force you to do that, it would be nice but it may take some time to get everything tweaked so it formats the code exactly the way you want it to

#

and even then it wont be perfect

#

for that you just need to // clang-format: off + // clang-format: on

#

or learn how to tame it with strategic // at the end of line :^)

grand shadow
#

do I go for lynx or links

#

hmmm

robust geyser
#

managarm does links

grand shadow
#

I mean links is graphical so I might as well do that

prime juniper
#

Also supports non graphical if you run it without -g

#

if I recall correctly

grand shadow
#

oh perfect then

robust geyser
#

also... include/x86-64/arch/ist.h

#

why do you call the tss "ist"

prime juniper
#

There must be a reason why I didn’t do lynx. Maybe shit build system or some weirdness

grand shadow
#

doesn't really matter

robust geyser
#

i thought it was the "task state segment"

grand shadow
#

oh wait it is tss

#

oh well I'll rename it later

robust geyser
#

it contains the ist, but the entire structure is called the tss :^)

#

no worries

#

im just snooping around

#

this just caught my eye in a particular way

#

im trying to pick up the coding conventions, see how things work

#

so much code that could be simplified meme

#

but i must resist the urge

prime juniper
#

Local man gonna work on astral?

#

What happened to your own OS iteration 69420

robust geyser
#

yea, maybe an xhci driver or something

#

i dont have motivation

#

and i dont know how to do shit

prime juniper
#

Classic

prime juniper
prime juniper
#

You can do it

robust geyser
#

yeah i can

#

i mean i know i can

#

but i just have to do so much shit

#

i want to do stuff properly

prime juniper
#

As one should

robust geyser
#

yea but idk how 😭

prime juniper
#

From the start

robust geyser
#

lol ur so right

prime juniper
#

Get bash and coreutils and friends

#

Then probably /dev and /sys

robust geyser
#

yea let me port them to kernel space real quick

prime juniper
#

I hear udev

prime juniper
robust geyser
#

yea but i need so much shit for that

#

i dont want to speedrun a shit scheduler + vfs again

prime juniper
#

Then get to work meme

winged basin
#

keyronex networking is drastically slowed by printfs in the code

#

that's one to watch out for

robust geyser
#

its just too much stuff to handle

#

without a clear path

winged basin
#

ordinarily networking is prioritised over everything else because the stack below the sockets themselves run in DPCs

prime juniper
#

Speaking of keyronex related items, I should upstream the InitWare changes

winged basin
#

but with printfs in the way the DPCs get hung

#

i have no idea if astral prints during networking but it's a low hanging fruit

prime juniper
#

and tell @winged basin to continue work on it, when new udev and tmpfiles

robust geyser
#

i feel like im never gonna get my shit together and write anything good

prime juniper
#

Also incorrect

robust geyser
#

every time i sit down and try to do something useful i just get overwhelmed by how much shit i have to do

prime juniper
#

Take it slow. One item at a time

#

I fell and fall for that trap too often still

prime juniper
#

Just in a different scope

robust geyser
#

tbh its easier said than done, i wish i could plan things out but i dont have enough knowledge on what to actually do

prime juniper
# winged basin some day...

Feared you’d say that. Maybe I can entice you with Managarm support PRs for InitWare meme (for real tho, I need to get this old ass udev to behave while I’m waiting on new udev)

robust geyser
#

i want to have a nice vmm, robust interrupt system, i have no idea how everything is tied together just yet and that annoys and discourages me

prime juniper
lean kettle
grand shadow
#

think its get sock opt

#

either way I'm probably gonna at least upstream all of these changes I have here jesus christ

grand shadow
#

couldn't get the graphical version to work

#

nevermid it was google that didn't work in specific

#

epic

grand shadow
#

I mean hey its not too bad to use

#

the tcp ip stack doesn't immediatelly combust

#

the only thing I ran into was an oom but I was also running at 256 mb of ram meme

#

hm the issue was that it was default to ipv6

#

or not

#

idk I'm tired today was productive

prime juniper
#

Nice work

neon crane
#

dont confuse!

#

i mean it's about 2.4mbps which is still slow as hell but at least it's not 56kbps 😄

neon crane
grand shadow
grand shadow
#

I wanna do something fun today and get an ssh server on astral

grand shadow
#

The sys futex wait signal interruption stuff still hasnt been upstreamed into mlibc damn its like an 8 line diff

craggy tide
#

PR?

grand shadow
#

#1068

craggy tide
#

also can i trust you that this is correct?

grand shadow
grand shadow
wild marsh
#

i restarted the ci jobs

#

freedesktop seems back up? at least the gitlab instance loaded in my browser

#

as for the code i looked at it and it seemed fine

#

could you run a rebase to get rid of the merge commit though?

grand shadow
#

I will once Im back home

#

In a few hours

wild marsh
#

we could also just squash right now ig

#

when merging

grand shadow
#

Yeah I dont think squashing would be a problem since its just a single commit anyways

wild marsh
#

okay ci passed now

grand shadow
#

Im surprised you didnt run into this when doing mlibc on linux since FUTEX_WAIT is interruptible there

wild marsh
#

inb4 linux returns EAGAIN if futex_wait is interrupted by a signal :^)

#

hm no it doesn't ```
EINTR A FUTEX_WAIT or FUTEX_WAIT_BITSET operation was interrupted by a signal (see signal(7)). Before Linux 2.6.22, this error could also be returned for a spurious wakeup; since Linux 2.6.22, this no longer happens.

grand shadow
#

What kept dying because of this was fvwm iirc and fvwm does start new processes once in a while

#

So I think it was a sigchld interrupting it

wild marsh
#

i guess geert already merged it

#

without squashing meme

grand shadow
#

Yeah lmao

grand shadow
#

hmmmmm

#

I will need socketpair for sshd

#

yeah localsocket socketpair doesn't sound so bad to implement

grand shadow
#

damn sshd is making me implement all of the socket system calls I was too lazy to implement

robust geyser
#

chroot is pretty simple

grand shadow
#

yeah I literally just copied my chdir and changed a single thing

#

I already had the root stuff implemented just never the chroot syscall itself

robust geyser
#

ahh nice :^)

#

did you try it after that change?

grand shadow
#

yeah I'm stubbing some more stuff the mlibc sysdeps now

robust geyser
#

wtf

grand shadow
#

I don't handle uid/gid stuff so its probably complaining that it can change the gid because I just stubbed those

#

I can probably just hack something together lmao

#

or actually

#

disable that in openssh

#

WHAT DOES IT WANT FROM ME I DON'T HAVE PASSWORD SUPPORT

#

wait there isn't even a mathewnd user

#

oh

analog berry
#

lol

#

Does this mean I can p4wn astral now

grand shadow
#

if you can connect to it because not even I can lmao

#

I need to figure out how to make this work

#

I either implement crypt(3) or stub it to return the bullshit I put in /etc/passwd

#

random bullshit go

analog berry
#

ssh speedrun

grand shadow
#

I mean its an mlibc panic because of a function I don't even know how to implement being stubbed

#

I will just make it so that like if the first input is some specific string it will return another string

#

and otherwise do the normal unimplemented

grand shadow
wild marsh
#

yeah if you have libxcrypt you can just disable the mlibc crypt option

#

likewise with libiconv

grand shadow
#

ohhhh

#

I should do that then

wild marsh
#

most things should pick the dedicated libraries

#

libxcrypt should definitely work because not even glibc provides crypt on it's own nowadays

#

(or at least distros use libxcrypt)

grand shadow
#

well ty for the pointer

bright kernel
#

ur trying to make https work?

#

or is it some other internet stuff

robust geyser
#

sshd

grand shadow
bright kernel
#

ah I see, insane!

grand shadow
#

I'm just going after sshd now

bright kernel
#

yeah getting https ported is one of my OS goals, but I'm still a long long way from it

#

and then other stuff it may need which just adds to the complexity

#

great stuff on ur OS tho!

craggy tide
#

when it comes to ports

#

i could even do it for you like i do it for Gloire

#

i don't mind

grand shadow
#

you don't have to, I'm probably gonna have to take a day to do port maintenance sooner or later anyways

craggy tide
#

sure

#

just ask if you want tho

grand shadow
#

I ported firefox to astral meme

robust geyser
#

what a madman

grand shadow
robust geyser
#

great work

wild marsh
grand shadow
#

I should try

robust geyser
#

yeah that would be amazing

analog berry
#

Good job

analog berry
grand shadow
#

I just got libxcrypt ported and then remembered I could let ssh use empty passwords

grand shadow
robust geyser
#

man

#

every day i need to do more and more to catch up

#

:^)

#

i should start doing something

obsidian pivot
#

but we both gotta catch up

#

I'll start catching up

#

uhhh

#

the 22nd

robust geyser
#

i probably wont lmao

obsidian pivot
#

my arm with a cast is itchy

#

but the itch

#

is bad

#

like

#

it needs to be itched

grand shadow
#

idk if x forwarding doesn't work with wsl2, if it doesn't work with astral or if I'm just too dumb to set it up

#

whatever I have openssh thats what matters :3

robust geyser
#

should work on wsl, at least it worked last time i tried it

#

but great work nonetheless :3

grand shadow
#

seems pretty stable on ssh too, was able to run links and stuff

#

I should do the ssh from one astral vm to the other

robust geyser
#

u should host a public astral vm :D

grand shadow
grand shadow
bright kernel
robust geyser
#

that is so god damn based

analog berry
#

but nice thats really cool

grand shadow
#

And unstub all those functions from mlibc

analog berry
#

no no dont do that

#

mount your host fs into astral and expose over ssh

#

and host on a public ip

errant zephyr
#

perfect

errant zephyr
#

and run printf "flag{%s}" <(head -c 12 /dev/urandom | xxd -ps) > /flag

#

and see how long it takes for someone to recover that

grand shadow
grand shadow
analog berry
#

u can mount any directory as a fat disk

errant zephyr
warped plinth
winged basin
#

i found the easiest thing is to implement virtio-9p

errant zephyr
#

i mean

winged basin
#

and if you are prudent when you implement it, you can then implement networked 9p without changing the actual 9p driver (just the transport driver, which should be separate)

errant zephyr
#

its plan9 so

winged basin
#

so it's much less pain

errant zephyr
#

syscall_hostname assumes a terminator and then performs a strcpy into a buffer that is too small

#

oh and

#

PLEASE enable SMAP and SMEP

#

hostname also has a kernel read which is cool

#

oh and write

grand shadow
#

I dont really have a read from user implemented yet lmao

#

Old astral had it but Im too lazy to backport that

errant zephyr
#

its a major security feature

grand shadow
#

Yeah

#

Guess I will add it together with credentials

#

Since everythijg still runs as root

errant zephyr
grand shadow
#

I dont lmao

#

My scheduler works with smp but its bad

#

Actually wait

errant zephyr
#

ah i see

grand shadow
#

Yeah that can be vary bad oop

#

I should probably add a getrunningthread func or something

analog berry
#

this is serenity os level stuff

#

why cant vfs_write lock user pages until its written?

#

you dont need this memcpy bs

#

also u have an interesting code style

#

no underscores nor camel

#

itsjustonewordcase

#

veryhardtoreadbutwhatever

wild marsh
neon crane
#

was about to say that

#

just bail on an invalid page fault

analog berry
#

thats not even what im talking about

#

im saying he straight up allocates a new kernel buffer

#

so it copies the entire range from user, then does DMA from that range, then frees the range

#

thats extremely expensive

neon crane
#

my guess is he doesn't want it to crash WITHIN vfs_write

analog berry
#

copy_from_user is for like tiny structs

#

not disk writes

wild marsh
#

i mean just copy_from_user into the page cache

neon crane
#

does he have such a thing

analog berry
#

if he has one then yeah

neon crane
#

and it bails in the middle of writing

analog berry
#

u return the page back to cache

winged basin
neon crane
#

and then you've half-written the file already

wild marsh
#

undefined

#

return EFAULT and carry on

winged basin
analog berry
#

for example

#

u cant half write

winged basin
grand shadow
analog berry
#

how is it impossible?

#

for page in pages
if page.lock() < 0: return EFAULT

neon crane
winged basin
#

it is legal to have, let's say, 1 tib of data in virtual memory, and to write that whole lot to a file

neon crane
winged basin
#

with only, say, 32 gib of ram

analog berry
#

sooo how does that make it impossible to lock pages

#

so that unmap hangs until the data is copied

wild marsh
#

you can't lock all 1TB into memory at once

analog berry
#

u lock page by page

wild marsh
#

locking implies it can't be paged out

winged basin
analog berry
#

if any fails u unlock all of them

#

by lock i mean the virutal address cannot be unmaped by user

#

it doesnt have to be present

#

u just say this virtual address cannot be unmapped for the duration of the operation

winged basin
#

in that case possible but requires a little work

analog berry
#

u have to have a similar mechanism for paging out stuff anyway, while in a page fault handler you shouldnt allow a different thread to unmap said page

winged basin
#

you need to inhibit files for which some mapping exists from being truncated, for example

#

it also breaks a bit on some networked filesystems where mmap() can be risky in general (because the file may be remotely truncated and then what should you do? provide zero-filled pages or fault?)

analog berry
#

probably doesnt really matter because this is implementation defined behavior

winged basin
#

my feeling is that if someone tries to write data part of which is virtually unmapped then it is fair to punish them by only writing the virtually mapped part of it

neon crane
analog berry
#

fair enough but what i described above is not that difficult to implement

winged basin
#

for write() style interface anyway, for asynchronous writes not a virtual but a physical wiring of the pages is necessary

grand shadow
analog berry
#

if its present it will be present

analog berry
#

which is why we lock stuff

#

or you just bump the page reference count if u have struct page

winged basin
analog berry
#

what does physically wired mean

winged basin
#

in keyronex this is the use v.s. reference distinction

analog berry
#

so like ref count?

winged basin
#

a page use can be e.g. file shared, anonymous, pml3, or deleted; a page is freed when its use is "deleted" at the same time as its reference count is 0

analog berry
#

i mean yeah that works

winged basin
#

and a reference count > 0 = physical wiring because the page is not subject to either freeing or reuse

grand shadow
#

Hm another thing that phases me more than the performance aspect of it does is that it needs to allocate the memory for writing all of that especially if its like a huge file which is dumb yeah

#

Whatever Ill deal with it later Id need user memcpy for that anyways

#

It was just the easiest solution at the time

analog berry
#

Well since u have no cache see my solution above

#

Just dma from those pages

grand shadow
#

I do have a cache?

#

I will just user copy into it like I already do in vfs_write

analog berry
#

oh then im even more confused

#

why not memcpy into the cache directly

grand shadow
#

Because I wrote that before I had the page cache

#

I didnt implement it with my vmm I did it much later because I had no idea how to

#

I did this because it was the simplest thing to do at the time

analog berry
#

I see

grand shadow
#

Anyhow for today I will probably go around the kernel doing a few todos here and there while seeing how I will be implementing the credential stuff

#

Maybe even the user copy stuff too

#

Since thats part of the todos lmao

winged basin
#

kauth

grand shadow
#

I got a friend to ssh into astral all the way from poland so serverside stuff defo works over the internet too

robust geyser
#

can i try aswell

#

im polish too :D

wild marsh
#

POLAND MOUNTAIN 🇵🇱

robust geyser
#

its pride month

#

i can confidently say that im polish now

#

🇵🇱

grand shadow
grand shadow
# robust geyser its pride month

my family is trying to get italian citizenship and the hearing is in the 24th I really hope we get it and get it this month so I can come out as italian during pride month lmao

#

its probably not happening because bureocracy is slooow but one can hope

prime juniper
#

Nice work

neon crane
grand shadow
neon crane
#

I didn't say it because of that

#

I said it because since you live in Brazil, you most likely speak Portuguese

grand shadow
#

I mean isnt portugal an eu country

craggy tide
#

it is

serene parcel
spark sentinel
grand shadow
#

I have been very productive these past days yes I love productivity

#

I ported shadow (still not 100%, passwd works with the putpwent pr and login will work once I get to add the handling of a few wtranscode errors in more functions)

#

that way once I start working on stuff related to different users I can quickly test stuff out

#

and today I didn't do a lot other than add a commit to that pr and try to build gcc inside astral for fun (which during a mmap test in the configure the makefile calls ran into a page refcount issue which I still have to find the cause of, neat)

#

this weekend I really should get going again

warped plinth
#

u got this

robust geyser
#

also, if you start working on filesystem permissions you might notice jinx conveniently makes everything owned by $yourhostuseruid, which probably doesn't map cleanly into the astral vm :3

#

that's because of the unshare+uid/gid mapping trick it uses to gain fake root in the containers without running as root outside the container

#

which also sucks because shit inside the container (install scripts or whatever) can't set the uid/gid of certain files/directories, which might be needed for some things

errant zephyr
#

you can just fakeroot tar tho

robust geyser
#

but the files on the filesystem wont be owned by root

#

?

#

thats the problem

#

you are root inside the container

#

but on the outside you aren't

errant zephyr
#

so tar thinks that its all root

robust geyser
#

yeah but its not actually root

#

so the files on the outside are owned by your user, not root

#

thats the problem

errant zephyr
robust geyser
#

but linux maps uid 0 inside the container to your user's uid on the outside

#

so what's written in the fs metadata is actually different depending if you look at it from inside or outside the container

#

at least if you look at it through linux apis

#

the actual values written there are what you'd see outside the container

craggy tide
#

if it didn't work like that it would be a security issue

#

what you can do is just adjust the ownership of the files when you pack the initrd

#

and no it's not just a Jinx issue, it would be the same with xbstrap

#

unless you run it as root

prime juniper
#

Yes that is correct

errant zephyr
#

and then no problem

#

or with fakeroot which is equivalent

craggy tide
#

^

grand shadow
craggy tide
#

Gloire manually runs chown -R root:root on the sysroot as root for similar reasons

#

but yeah you can also pack the initrd in a fakeroot as pitust said, assuming you use an initrd

grand shadow
#

I was going to do that as I barely use the initrd to test in my machine since its faster to just boot from disk if Im doing some random testing

craggy tide
#

fair

grand shadow
#

That as in what gloire does

craggy tide
#

i mean there is no other way

#

other than running as root

#

unless you manually edit the filesystem inodes from userland to be all root

#

:^)

#

maybe guestfish can do something? idk

#

but guestfish is an abomination

errant zephyr
#

mksquashfs can, idk about mke2fs or tar

supple hatch
#

congrats on mod staging btw mathew

#

also keep up the good work with astral

grand shadow
grand shadow
supple hatch
#

thats actually rlly cool

#

must have taken a lot of work to get working

grand shadow
#

It was mostly having a working tcp impoementation

#

Also theres a web browser.now that works for some pages

supple hatch
#

woah

#

niceeee

proud vine
#

bro is the definition of "diy"

neon crane
#

I just hope that the OS doesn't turn out messy out the other end like (iirc) SCAL/UX was

grand shadow
#

eithre way

#

boom thats one bug fixed, no longer dies because of the page refcount

#

lets see how far into gcc compilation this goes

#

waitpid deadlock memes at the end of intl configure

obsidian pivot
#

does astral say, boot on real hardware?

grand shadow
#

I tried once like last year and it got to a bash prompt but the keyboard didn't work

#

I should try again someday

#

I think I already know whats happening and it should be an easy fix

#

just some lock ordering oops I did in my signal code

#

I'm gonna leave it building gcc while I go out for lunch

#

it began

#

anyways hopefully it doesn't crash and burn when I come back

#

oh lol

lean kettle
#

ill take this as a challenge on managarm

prime juniper
#

Configure won’t work without at least one hack I have locally and something something fcntl.h check

lean kettle
#

tar -xvf also fails with weird EBADF errors?

craggy tide
#

that's honestly really impressive matt

obsidian pivot
#

I, for one, could only dream of getting to that point

#

so to speed that up

#

I'll stop talking for now

#

I'll be back

grand shadow
#

testing user copy stuff going on well lmao

#

I think I fucked up really badly

warped plinth
#

i'd love to run astral on my laptop

grand shadow
grand shadow
#

time to go through all of the 70 system calls and switch what they use

grand shadow
#

I'm curious, is xhci super difficult to implement?

lean kettle
#

cc @wild marsh

wild marsh
#

conceptually not really, but i've been having trouble getting stuff working on real hardware

#

it's just a bunch of ring buffers for submitting commands and transfers and receiving events (incl completions), and some context data structures that hold the device and endpoint states

wild marsh
robust geyser
#

xhci itself is pretty simple, i was considering implementing it in astral for a while but i couldnt really find any motivation to read and learn the codebase lol

#

i think the hard part is making it work with different controllers on real hardware

#

but that might just be because my scheduler is absolute dog poop, the xhci driver did seem to work fine for the most part

#

usb is pretty simple too, it just builds on top of xhci and it gives you so much for free basically

wild marsh
#

i mean linux defines like 46 quirks for xhci :^)

wild marsh
robust geyser
#

yeah, s/xhci/hcd basically

#

dont they work pretty similarly? i have not looked into anything else at all so i wouldnt know

wild marsh
#

well for the older ones you need to build the schedule for packets to send within a microframe yourself

#

and for ehci you need to manage generating split transactions for high speed usb hubs

#

thankfully xhci manages that on its own

obsidian pivot
#

run curl http://ascii.live/can-you-hear-me

grand shadow
#

gimme a sec I'm making execve work with copy from user

grand shadow
#

bro

#

mfw ported curl just to get rick rolled

obsidian pivot
#

tbh I thought you already had it

grand shadow
#

I had wget

#

oh well now I have curl AND wget

obsidian pivot
#
HTTP request sent, awaiting response... 417 Expectation Failed
2024-06-09 18:33:06 ERROR 417: Expectation Failed.```
#

I get that with wget

grand shadow
#

On astral?

obsidian pivot
#

no

#

wsl ubuntu

#

I couldn't be bothered to compile astral

grand shadow
#

Oh lmao

obsidian pivot
#

last time I tried

#

it needed to build gcc

#

I ain't goin' through all that

craggy tide
#

as if that's the biggest bother

#

wait til you see LLVM

grand shadow
#

I really should get around to seeing if I could unassert a few wtranscode errors like someone else did in another function

#

it gets really close to being done but it doesn't actually get done

prime juniper
#

based

#

doing it without passwords and passing the user on the command line does work (with some hacks)

#

debug logging is elogind for session management

grand shadow
#

with the sysconf pr you're now able to actually pass the login user in the prompt

prime juniper
#

only if we implement cooked mode I think meme

grand shadow
#

oh lmao

#

and hopefully with me handling the illegal_input mbrlen error thing it should be able to log in with a password

prime juniper
#

based

#

I patched pam to just ignore password and return ok meme

#

but that's only because input is fucked

grand shadow
prime juniper
#

I want to fix it but I have negative braincells so idk how well that goes lol

grand shadow
#

idk how this wchar stuff works

#

well thats fun

#

uhhh

#

making that return an illegal_input instead of an assert failure kinda worked

#

and with kinda I mean

#

wtf is this bash prompt

#

I'm still root uid though since I don't have a setuid or anything like that

#

maybe its that

#

this is really weird though

#

it seems like its not being treated as a login shell

craggy tide
#

i suggest reading the patches/recipes/extra files from Vinix

grand shadow
#

lemme see

prime juniper
#

Seems like it indeed

grand shadow
#

still doesn't explain the weird bash name

#

in xterm it works

#

maybe that wchar stuff gets called from bash and not from login

#

yeah bash is the one that calls mbrlen etc

#

gonna compile bash with the vinix configure arguments and see if that helps

#

ah well whatever

#

I'll look into this tmr

grand shadow
#

Im a pussy and I really dont feel like debugging bash so Ill drop this login thing for now and instead go fix some things I have on my backlog and work on fat32

robust geyser
#

i remember having a similar issue on linux but i dont exactly remember how it was fixed

#

does it ever segfault for you when using login?

grand shadow
#

Nope, all that happened was a wtranscode error

#

That I patched out

#

I genuinelt have no idea wtf bash wants from me, I think its thinking its own name isnt just ascii or smth

bright kernel
grand shadow
#

I'm thinking of buying a domain and a vps or something to host the packages for astral once you can install it to disk

#

tho I have no idea how that stuff works since I've never looked into it too much

#

other than buying a domain a few years ago because I was going around a bit and wanted to ssh to my computer at home

bright kernel
grand shadow
#

the issue mostly is the hosting part of things, I could self host it but I really don't feel like exposing my home network that much nor do I have any spare machines to leave on 24/7
with that out of the way its just a matter of doing it

bright kernel
grand shadow
#

which provider though

#

I searched in this server and one called "tornado vps" caught my attention

#

idrk atp I'll probably leave that to future me

grand shadow
#

I can barely find anything about it online

bright kernel
bright kernel
bright kernel
#

yo @grand shadow did you need to recompile the WSL kernel for KVM to work?

obsidian pivot
#

I didn't

#

if that helps

bright kernel
#

huh

#

how did yall get kvm working then

#

I tried adding myself to the group

#

and staring up the virt manager (and what it needs)

#

but it still didnt work

#

and checking with kvm-ok it said my CPU didnt support it (it ran fine on linux)

neon crane
obsidian pivot
#

probably for the astral gcc toolchain

bright kernel
grand shadow
#

yeah my laptop uses it

bright kernel
#

oh that's why

#

windows 10 doesnt support nested virtualization

#

dang

#

I might need to reinstall windows 11 or just dualboot

#

(I don't have much space for dualboot)

fervent fjord
neon crane
#

how come

#

im pretty sure it does

bright kernel
#

and running kvm-ok it said "this CPU does not support KVM run with sudo for more info.."

winged basin
bright kernel
winged basin
#

oh, very bizarre

bright kernel
#

and other people facing the same issue on gh reported it

#

and most answers were that windows 10 doesnt support it

#

apparently you could enable it using windbg

#

but it's not safe

grand shadow
bright kernel
grand shadow
#

I did it

#

I'm gonna see if I can download a minimal chroot with bash and dependencies

analog berry
wild marsh
#

a web browser

analog berry
#

Does it load youtube

grand shadow
#

don't think so

analog berry
#

What can it load

grand shadow
#

it could load github

#

it doesn't load a lot of things and what it does load is us quesitonable

#

also rip

#

got close

analog berry
#

Allocator bug?

grand shadow
#

hmm

#

double free

#

might be some vnode refcounting memes

#

well the download part of it worked lmao

#

averaged like between 600 and 700 kb/s

#

not that good but better than 30 kb/s

bright kernel
grand shadow
#

well I added an assert ( assertos reference) to my VOP_HOLD to see if I can catch that the first time it happens

grand shadow
#

Time to get my ass back on track again today

#

I got a list of todos from the source and I will try to do at least half today

#

My friend is also making a simple landing page for astral-os.org because I know nothing about web dev

warped plinth
#

hell yeah

#

ngl i love the bare html

neon crane
#

Also perhaps you should have your ISR handlers be like:

isr_X:
    if x != 8 && (x < 10 || x > 14)
    push qword 0
    endif
    push qword X
    jmp  isr_common

isr_common:...
grand shadow
#

It was one of those things that has been on my todo list for so long but it has never mattered enough for me to do it

#

Ill add it to the list of stuff I wanna do today tho

#

Im gonna work on a bunch of stuff I left behind before I do fat32

neon crane
#

and it saves quite a bit in binary size

neon crane
grand shadow
neon crane
grand shadow
#

Oops yeah I think I forgot that

#

I guess it never really mattered that much because all my checks are done on the cs

neon crane
#

It's about using the CPU in an inconsistent state I think

#

Maybe it doesn't matter if DS and ES have a CPL of 3 while CS and SS, a CPL of 0

#

but for completeness

grand shadow
#

yeah

#

my system call handler also suffers from that from what I can see

obsidian pivot
#

Is Astral cursed on the inside

#

i.e., bugs that are pretty big, but a worked around instead of fixed

grand shadow
#

I hope not lmao

#

I usually try to fix bugs when I find them

#

well those 2 fixes (the segment thing and interrupt code size) took like less than 10 minutes

#

goddamn if I werent so lazy

analog berry
obsidian pivot
#

and you guys say my kernel is cursed ||(it is)||

#

is the tar corruption

#

for the ramdisk?

analog berry
obsidian pivot
#

cpu0: oops

#

best error message ever

grand shadow
#

I probably should soon

#

I am just lazy (and work has been taking up quite a bit of my energy)

grand shadow
#

dead project xd

#

I think I did burn out a bit from working way too much on this

lean kettle
#

You don't have to constantly work on the same project, that's fine

grand shadow
#

I should port an irc client

warped plinth
#

:^)

grand shadow
#

best port ever

robust geyser
#

unironically

warped plinth
#

oh my god yes that's awesome

#

i genuinely look forward to the day i can use astral on my hardware

analog berry
#

like, today then

warped plinth
#

smp would be nice, but beggars can' be choosers

analog berry
#

it does have smp

warped plinth
#

well, bugless smp, i suppose

iirc there's still some problems with the current impl

grand shadow
#

you can pass nosmp to the kernel commandline in limine and it will boot without smp

#

the biggest smp issue rn is that my tlb shootdown is painfully slow

#

and some things still arent 100% smp safe and racey

warped plinth
#

oh fuck yeah, well i'd love to check it out then

prime juniper
grand shadow
#

perhaps...

grand shadow
#

fixed the certificate memes finally

grand shadow
#

I do wonder if it could post in the osdev forums, it can log in and navigate around just fine

tawny oak
#

haha wow, very nice

prime juniper
#

Never tried it from there

#

Works fine from WebKitGTK tho meme

grand shadow
#

is there anything in specific you needed kernel side for webkitgtk

#

well sysdep side since managarm is a microkernel

obsidian pivot
#

when smp fix?

#

even obos runs with SMP

#

except not with 128 cores

#

ok fixed

grand shadow
#

when I stop being lazy and fix a race that affects like 90% of the files in the kernel and make the tlb shootdown not horrible

obsidian pivot
#

that reminds me

#

I need tlb shootdown

grand shadow
#

that also happened out of smp but were more common with smp

obsidian pivot
#

lol

#

OBOS is better than Astral until SMP on Astral

grand shadow
#

soon™️

#

smp on astral has always been "almost there but never really quite there"

obsidian pivot
#

SMP is surprisingly the most stable thing on OBOS

grand shadow
#

it just adds thrill to the computing experience

obsidian pivot
#

(difficulty: impossible)

grand shadow
#

allocator....

obsidian pivot
#

no that was rewrite 4

#

now it's the timer interface

obsidian pivot
#

hmm

grand shadow
#

I want astral to be self hosting enough to be able to:
open osdev wiki on browser
follow barebones from compiling an i686-elf gcc cross compiler cuz why not
run qemu and the barebones and post in #osdev-misc-0 asking how do paging

lean kettle
#

if you manage all of this im donating 10€ to a (good) charity of your choice

bright kernel
lean kettle
#

by good i mean morally decent if that makes sense

grand shadow
#

Before I add new things I still gotta finish my todo list though

#

Tomorrow Im gonna go after 2 of them where a thread does nonstop yield waiting for I/O because my old event thing was bad, so Im gonna write another one that doesnt suffer from that nasty lost wakeup

#

Its gonna be more general than the poll system thing

#

So just an attach etc

grand shadow
prime juniper
# grand shadow is there anything in specific you needed kernel side for webkitgtk

Well, considering I always lie to the programs and say that (if needed) just use Linux code paths, probably some Linux stuff if you follow that route. I know we had to do some netlink shit cuz glib. I know there is a BSD backend which uses a different interface. I know I told GTK just assume we’re Linux, which means you may or may not run into some memes there too. My advice? Just try it. If you want to be Linux like, there is a good chance you’ll need some of the following (list is in random order and not guaranteed you need anything from it, list might be incomplete): eventfd, timerfd, signalfd, inotify, netlink, DRM. Most, if not all, can probably be substituted for something else, if you claim that you aren’t Linux but more like a BSD for example. Best way to find out is to try

prime juniper
#

Cuz the only thing in this list we can’t do yet is compile GCC

#

Qookie wrote a bootsector on Managarm and ran it in Qemu, I have had discord conversations from Managarm

prime juniper
lean kettle
grand shadow
#

shadow really doesn't like it when you don't have a valid shell

grand shadow
#

seems like the weird bash name isn't in bash but rather in login, makes things easier to debug I think

#

I wont have to read bash code

#

so it could be that exec

#

what the fuck?

#

this can't be right

#

am I going insane or

#

I think this was supposed to be argv[0] = argv0?

#

idk

#

actually wait

#

lemme try thios

#

this was taken from musl so it must be right

#

gonna see whtats in that argv[0]

#

yeah no this is wrong

#

from musl source

#

fucking hell

#

another mlibc bug it seems

#

that was why bash was acting all weird

#

execle is broken in mlibc

eager copper
#

be the change you want to see™️

grand shadow
robust geyser
#

lapfed slow as usual trolley

grand shadow
#

My list of things to fix is getting smaller, soon I will actually be able to progress again

warped plinth
#

woohoo!

grand shadow
#

if I stop being lazy soon otherwise soon®️

fleet osprey
#

skill issue tbh troll
I've been working on the same project daily for the past 2 months or more, I don't even remember anymore
legitimately not 1 day that I haven't worked on it

#

I don't think this is healthy

grand shadow
#

now I'm just slowly grinding away at a todo list of random fixes and improvements

grand shadow
#

did a thing

#

I should try to compile it meme

#

well the kernel

#

jinx Is Not Going To Work

#

oops yeah its missing flanterm and stuff

#

but hey git works

craggy tide
#

damn that's fucking crazy

#

not even managarm has working git iirc

warped plinth
#

oh fuck yeah, nice

grand shadow
#

still another step towards self hosting, which would be pushing its own code into the internet

#

I don't know if pushing works and I'm too lazy to configure the credentials and stuff

#

I will only consider astral truly self hosting once I can compile gcc though

#

because what good is for that it can compile itself if it cant compile the tool to compile itself?

prime juniper
# craggy tide not even managarm has working git iirc

I worked on it years ago. Back then, netstack was the biggest issue. So no, technically we don’t, but I’m pretty sure it’s a recompile away. Minor fixes at most (at least for git clone). If I dig I can probably find screenshots of git doing stuff

#

Yep. Got a screenshot from the 15th of April 2021 running git log

#678608576151027734 message

#

Still, getting git clone working is massive for self hosting. Amazing job

craggy tide
#

ah nice

obsidian pivot
#

but does it run doom

#

?

grand shadow
obsidian pivot
#

Based

charred sinew
regal stone
#

is astral based on a kernel?

#

cause if not

#

holy shit this is impressive

prime mulch
#

it is based on a kernel. but its a custom kernel

#

and yea it is

#

impresive

regal stone
#

linux?

prime mulch
#

the astral kernel

regal stone
prime mulch
#

yep

regal stone
#

wow

winged basin
#

though maybe someone could legitimately do something on top of Se.L4

#

or w/e

neon crane
#

@grand shadow i suggest you consider porting sm64 to Astral

#

would be cool

obsidian pivot
#

port astral to the 68k

grand shadow
# charred sinew What syscalls do you need to support X11?

the core of it is giving it somewhere to draw like using a xf86-video-fbdev driver and a way for applications to talk between them and the server like unix sockets, so a good enough posix support w/ the socket stuff gets you a long way

grand shadow
grand shadow
regal stone
wild marsh
#

i think it just needs sdl2 and opengl?

#

or something thereabout anyway

grand shadow
#

I don't have mesa and I'd need to port llvm for that and I wanna fix up some other stuff first

fleet osprey
#

can it run QEMU?

grand shadow
#

Idk never tried it

wild marsh
#

qemu should be simple enough i think

#

on the outside you might want sdl2 or gtk for the ui, on the inside it wants pthreads

#

and it does some signal trickery (sigaltstack + sigsetjmp & siglongjmp) to switch stacks for coroutines but that's also not that bad

fervent fjord
#

llvm-pipe

winged basin
#

screw llvmpipe

#

use softpipe

fervent fjord
#

bit slow

#

but its usable

grand shadow
neon crane
fervent fjord
#

great name

grand shadow
#

I do wonder something
eventually I will need a better init system than just running a shell script at boot and all
how easy would it be to port openrc, hm

#

its not something I really need atm (or for the close future anyways) but I am curious

errant zephyr
#

chromeos uses upstart fwiw

warped plinth
#

runit, perhaps?

sacred gyro
#

nah, port systemd

grand shadow
neon crane
grand shadow
#

Make init a symlink to bash smh

errant zephyr
neon crane
winged basin
#

Will discuss when home

errant zephyr
#

usually it also deals with service managment

#

because some services always orphan themselves

#

and so you have to be pid1 to get a notification

prime juniper
#

I should try that under LFS

fleet osprey
#

I wonder if it can run my "debugging init"

#

give me a sec to clean it up so it has a chance, and I'll drop it here

#

this is much more a test of how much of a generic userspace you have and less of a C library test (it's literally a script), but still

#

made this to get some info as to why somebody's OS wasn't booting for (non-osdev, it's Linux) project I maintain, just to gather some basic system info

#

cleaned it up a little for here, since I threw it together in like 60s over there

#

and it was meant for recording the screen with a phone camera, hence reading stuff out slowly

#

curious to see what they all give, since I know neofetch was being pretty funky in regards to RAM, and didn't even list a CPU on Astral last I saw it

grand shadow
#

gonna try it

#

actually wait

#

free -h doesnt work, I don't have /proc, I don't have dmesg, I don't have an /etc/os-release

robust geyser
#

i don't think /etc/os-release is necessary

#

but procfs is definitely something you need if you want neofetch to pick up your cpu/memory info

grand shadow
#

yeah I will have to look into it sometime

grand shadow
#

Dead project xd

#

Argh I need to finish fixing stuff so I can work on fat32 already

analog berry
#

Dont make the mistake of only supporting fat32

#

Support all types