#Astral

1 messages · Page 2 of 1

grand shadow
#

and signals probably to make it actually usable

#

though these should be hopefully not that hard to implement

craggy tide
#

because despite what your god (hyenasky) tells you, LoC is not a metric of anything if not just jerking off about how many lines of code you have in and of itself

#

if anything the less lines of code you have, the less room for error, and the generally more understandable your code is

#

you should always aim to implement whatever it is you're implementing in as few lines of code as possible without sacrificing functionality or readability of course

errant zephyr
#

see literally ironclad which trades off more loc (and development time ofc) for safety proofs

#

or any other verified os

robust bobcat
#

i love assert!

craggy tide
#

you know full well i am not talking about those exceptions

errant zephyr
#

no i dont think this premise is true in general

analog berry
#

Just collapse every expression into one line of code

errant zephyr
#

lines of code are objectively a bad measure

#

that is totally meaningless

analog berry
#

Yup

#

It can be used to get an approximate measure

errant zephyr
#

but a shorter program != a better program

craggy tide
#

i never even said that

craggy tide
#

you already know what i think of suckless

#

yes

#

that's not the same thing as "shorter program better"

analog berry
#

What was the idea of suckless

craggy tide
#

shorter program better

analog berry
#

Doesn't it suck

errant zephyr
craggy tide
#

it sucks

#

they boast about how they have like a 200 loc browser which pulls in a 5 million loc library

#

it's a joke

errant zephyr
#

lol

errant zephyr
#

i mean dwm is the "recompile to change config" wm right?

analog berry
craggy tide
craggy tide
#

surf

robust geyser
#

ah

#

ty

craggy tide
# errant zephyr what.

in suckless basic features are offered in separate patches so they can keep the loc down

errant zephyr
#

okay fair enough kinda

craggy tide
#

it's very stupid and purposely made unpackageable

errant zephyr
#

but yeah impossible to deal with

craggy tide
#

because config is also compile time

errant zephyr
#

and test

#

and do anything else with

analog berry
#

Test is the main thing

errant zephyr
#

and update

analog berry
#

How do u test every combination

errant zephyr
#

you cant

craggy tide
#

they do not have tests afaik

#

tests are bloat amirite

errant zephyr
#

yeah!

analog berry
#

Who needs those

#

Tests suck and are thus outside of scope of the suckless project

craggy tide
#

i should come clean though

#

i do use dmenu because it's handy

errant zephyr
#

dmenu is actually pretty good

#

that said i use wofi

#

because wayland

craggy tide
#

beats having to open my start menu to dig for a program or open up a terminal tab just for that

errant zephyr
#

i think thats a thing

#

hm

#

is it

craggy tide
#

isn't that just a dmenu frontend

errant zephyr
#

no idea

robust bobcat
#

rofi

spark sentinel
#

rofi fr

supple hatch
#

in fact I think judging kernels based of loc is super dumb

analog berry
supple hatch
#

yea I’m just stating that like I don’t boast about that lmao

analog berry
#

I was mostly talking about small kernel + huge library

supple hatch
#

ohhhh

#

Then that’s my kernel alright 😎

#

proves how portable uacpi is

#

thanks to your hard work !!!

analog berry
#

Lol thanks

supple hatch
#

np !!! 😎

grand shadow
#

running with more than 4gb of ram makes qemu complain and nvme not work

#

pci_nvme_err_addr_read

#

I'm probably passing a 64 bit address where I shouldnt somewhere

grand shadow
#

runs fine on vmware with 8gb?

#

wt

supple hatch
#

lucky

grand shadow
#

pci_nvme_err_addr_read addr 0x101000 like what the fuck?

#

I'm not going through qemu source again

#

virtio-blk is just fine with 8gb on qemu

#

I'm not even sure what could begin to cause this

#

welp turns out I forgot to enable the bus master thing in the nvme pci config

#

oopsie!

#

that was it I think

grand shadow
#

I need to go and revisit the page cache I was working on, especially now that I will have multiple applications running with the same shared libraries which will add up

#

Thatll go into the todolist along with signals

#

Its kinda silly how I did all that work to redo how I handle some parts of memory management for it but never actually took it out of paper

grand shadow
#

a short break from implementing signals

#

fvwm has some python scripts so once I implement shebangs it should be more complete too

#

also xterm carriage return and newline memes

supple hatch
#

so cool

grand shadow
#

slowly getting there

#

just noticed

#

signbal

winged basin
#

sign of baal

#

very satanic

#

signals are quite obnoxious good luck with them

grand shadow
#

it has been a whole lot of information to keep track off for the initial implementation but once thats done it should be easier to maintain

grand shadow
#

now I just need a sigreturn system call

grand shadow
craggy tide
#

hey could you share that signal tester? i would like to test Vinix's signal handling since it's been a while

#

btw nice job!

grand shadow
obsidian pivot
#

bruh just like *(uint8_t*)0xdeadbeef

#

and see if you sigsegv

grand shadow
#

I dont raise sigsegv in my pagefault handler yet I will have a specific pending sigset for the "urgent" thread signals like sigill and sigsegv that are raised by the kernel itself

grand shadow
#

idk why bash isn't saying the funny message but it does work

#

I will clean this stuff up tomorrow

wild marsh
grand shadow
#

I think it does but Ill doublecheck tmr

#

I used to just have it hardcoded to the exit status << 8

#

But I changed it now

supple hatch
#

very beautiful

obsidian pivot
#

ASCII escape sequences

supple hatch
#

funn

spark sentinel
grand shadow
#

I have spent the past few hours trying to fix this stupid fucking bug

#

and as it turns out

#

when a signal happened

#

I wasn't returning system call values corretly

#

it was returning from the system call with the old complete context rather than having the return values in rax and rdx

#

now fvwm is working (again)

grand shadow
#

I think I also forget to sync _cpu()->intstatus somewhere

#

or not

#

huh

#

an assert I forgot about in pipefs_newpipe just saved me headache

#

dang

#

its entering syscall_pipe2 with _cpu()->intstatus being false

#

probably something to do with the signals

#

bingo

#

I just directly switch context with interrupts disabled in syscall_sigreturn

#

so intstatus is never set true again

#

xterm closes now yippie

grand shadow
#

hey that works now

#

except its just default xterm

#

implementing shebangs now, I want the fvwm python scripts to run

warped plinth
#

woo!

grand shadow
#

it got the spirit it just needs a lil push

grand shadow
grand shadow
#

think I'm gonna finish job control next

grand shadow
#

actually I should probably just finish tcp once and for all

warped plinth
craggy tide
#

very nice progress

fleet osprey
#

Yea, this is quite impressive

grand shadow
#

I will do some more fixes today

#

my next major step will be one of these:

  • finish job control and tty stuff
  • finally get a page cache going
  • fat32
  • tcp
#

first thing I'll take care of will be finding out why the system gets slow when I close xterm after running a command

grand shadow
#

the xterm issue seems to be its just never returning from signal_signalproc

#

or rather

#

jobctl_signal

#

since its signaling the process group of bash

#
syscall: pid 24 tid 25: kill: pid -26 signal 1 (1823356 free pages)
kill
signaling thread
not ignored```
#

yeah it was just some linked list memes

#

fixed

analog berry
grand shadow
#

idk how I would even approach that

analog berry
#

well if you're interested i have an example on the osdev wiki on how to make acpi a bus in your system

#

it really aint rocket science

obsidian pivot
#

otherwise it kinda is

analog berry
#

well yeah from scratch its just a lot of work

winged basin
wild marsh
grand shadow
#

still got a lot to do on my tty line stuff

#

but finally the system wont be stuck if you run cat lol

#

going to miss the no job control in this shell message :(

grand shadow
#

I guess when its running with job control off it doesn't print signals that killed processes?

#

still need to get bash to run with job control inside xterm though

fleet osprey
#

nice!

#

this is really getting pretty good

#

I could only imagine my own OS getting close to that far

grand shadow
#

ty!

grand shadow
#

with a few pointy edges

#

that I will probably just end up rewriting a few functions to take care of

grand shadow
#

the more I think of it the more I realise I should go and stop procrastinating the page cache

#

if I wanna keep adding ports I need to make it not horrible to run from disk because using an initrd is unsustainable and running from disk without caching is sloooooooow

#

I could go the lazy route and do block cache since all disk accesses from the filesystem drivers go through the same device vnode stuff but like it'll be effort thats worth it grrrr I just need to stop procrastinating

#

I was doing the same thing to sessions/program groups before I just sat my ass down and did them

#

I guess I just need to go and read my old notes

#

yeah thats what I will do after I clean up signals and stuff

grand shadow
#

aaaaand it was an issue with xterm's build system

#

after porting it I get why they put that warning in the readme

#

no more "no job control in this shell"

#

now its just silence :(

grand shadow
#

once I finish the page cache I should just do a full playthrough of quake inside this to test the system lmao

#

I have an old install of full quake somewhere

grand shadow
#

I added a release to github with the full and minimal isos

warped plinth
neon crane
grand shadow
#

I mean I dont see why not Id just need to port an emulator

grand shadow
#

I've started work on the page cache already, right now Im working on getting the tmpfs to run completely out of it

#

so far so good I think

bright kernel
#

like I'd believe you'd cache at either map or when accessing a page

#

but how would the OS catch when you access a page

grand shadow
bright kernel
#

so it caches when the page isn't mapped already?

obsidian pivot
#

I believe so

#

A data structure commonly used for this "cache" is an RB-Tree

#

Now if you're anything like me, you probably don't want to implement an RB-Tree

#

you can use freebsd's tree.h for that

bright kernel
#

ah I see

#

I'll take a look into that later on

#

seems interesting

#

thanks!

obsidian pivot
#

well it's less of a cache and more of a structure to store info about pages in general

grand shadow
#

it makes implementing CoW much easier

#

wait I think thats just bash and ls trying to write at the same time bleh

#

I don't have the sigttou stuff

#

I did find a deadlock

#

woops I think it was because I took a spinlock and then slept on a mutex

grand shadow
#

got a good tmpfs integration with it now I think

#

and if I remove /usr/bin/bash and exit from bash more pages get released

#

yeah it seems to be working fine

#

I still need to add a case for device mappings oops I removed these

#

I'll do that tomorrow

grand shadow
#

I fixed that and it can boot to fvwm from the new tmpfs

#

Today Im gonna hook it to ext2 and add a few optimisations in ext2 for it too

#

Itll also show if my sync thread actually works decently since VOP_PUTPAGE is a noop for tmpfs

obsidian pivot
#

Le very cool

grand shadow
#

aand I think it works with ext2 too!

#

bootng from ext2 into console was already much quicker but you can also see gcc taking long to run for the first time and then doing it quickly the second time

#

my implementation of nvme and ext2 isn't nescessarily good and quick but this is defo an improvement

#

syncing isn't working 100% though

grand shadow
#

and thats fixed, I was forgetting to unset the pages as dirty from the writer thread

#

gonna try it with a full distribution now

#

actually that is the full one just gonna make it start to x

#

well xterm doesn't like this

#

hm

#

I reaaally don't like this

bright kernel
# grand shadow

what DE + Theme is that? it looks like the one on managarm

winged basin
#

that's what managram uses

bright kernel
bright kernel
#

wayland on wsl cool 😎

bright kernel
grand shadow
#

yeah it works

bright kernel
#

-enable-kvm? or some other accel?

grand shadow
#

-enable-kvm

bright kernel
#

huh interesting

grand shadow
#

why xterm in particular is the unlucky one for this to happen I have no idea

#

maybe that codebase really is cursed

#

something is wrong with the nvme/ext2fs side of things here

#

since this works from the tmpfs

grand shadow
#

something is just straight up overwriting xterm

#

like I think before it even has a chance to get loaded into memory

#

bruh

#

wtf

#

not just xterm

#

welp

#

ugh I think I have a refcounting issue

winged basin
#

so if the contents of /usr/bin/ls are retained in the page cache, and haven't been evicted

#

then it seems likely that they have been corrupted

grand shadow
#

I found the issue I think

#

I had forgotten to remove all pages from the page cache once a vnode has been removed from memory

#

so when the same address had been used again it was still using the same values as before

#

that was exactly it

#

page cache seems to be working fine for ext2 now wooooooooo

#

its actually usable running from disk now

#

the only thing I need to do is implement rename for ext2 since x.org needs that to rename a log file and refuses to start if it can't

robust geyser
#

mount tmpfs on /var/log :^)

#

jk, anyway it's some great progress keep it up!

warped plinth
#

that's some awesome stuff

grand shadow
#

ty!

#

the last thing I want to do today is fsync

#

I already have a blocking sync() and I just need fsync to be happy

grand shadow
#

and I also have to start thinking about how I'm gonna approach the metadata side of things on ext2 to optimize memory usage and disk activity
right now it just uses the disk file to r/w which works but I don't want to have to sync the whole disk after an fsync (which kinda just makes it a fancier syncfs)

I'm thinking of having a separate special metadata vnode that will be 1:1 to the disk but cached separately from it (so when metadata is being synced and VOP_PUTPAGE/VOP_GETPAGE is called it'll just do an uncached vfs_write/vfs_read to the disk file for that page)

and I will also be doing uncached reads/writes for actual file data because right now the page cache holds duplicate data from the disk and file pages

in these cases by uncached I mean checking if its in the cache so the data on the disk and memory doesn't diverge but not bringing pages to memory if they aren't already

winged basin
#

you can easily avoid writing back the metadata block in the regular-file PUTPAGE logic since you know there what blocks exactly you want to write back

#

but the opposite is trickier

#

it wouldn't be so tricky, but most people treat directory files as containing metadata not data because, well, they are metadata

grand shadow
#

hm that is true

winged basin
#

there are ways around this but i can't remember the details

grand shadow
#

I'll figure something out eventually, its not that high of a priority to cache the metadata separately right now as much as it is to not have duplicate file and disk data in memory

grand shadow
#

trying to run configure is going to be a decent way to stress test this thing

#

already managed to get a kvm error lol

#

think I'm hitting an interrupt after a swapgs to userspace

grand shadow
#

disabling the writer thread seems to make the issue go away

grand shadow
#

well all of this seems to be *some* issue with nvme

#

I ran configure with virtio-blk and it seems to works fine

#

nvm

#

ran it again and didn't work

high whale
#

honestly im seriously impressed by this

#

this project may actually become serious with a few more compilers ported over

#

(serious as in mainstream)

grand shadow
#

ty! there are other projects here that have a bigger chance of that though, like managarm

grand shadow
#

so it might not be actually related to the page cache but more so related to a disk race condition or w/e

#

if its that then I guess its an issue on both nvme and virtio blk

grand shadow
analog berry
high whale
high whale
craggy tide
#

no seriously, keep up the good work, i firmly believe you have bigger chances than managarm

grand shadow
#

just add an assert that checks if there has been a commit to the project in the current week smh

analog berry
high whale
#

its written in go

#

not sure if its easy to port

craggy tide
#

the only good thing is that mlibc exists though

#

but tbh that more so exists in its own bubble now than as some sort of core part of managarm

analog berry
#

yeah

craggy tide
#

in Astral and Vinix the only managarmware we use is mlibc

grand shadow
grand shadow
#

maybe something will come up that will give some idea of whats happening

#

since it only happens deep in the configure script and randomly

#

not even neofetch crashes

#

I will work on a "don't do caching" flag for file data

grand shadow
#

alright that cut in half the page usage for the cache as expected

craggy tide
#

you should add Astral

grand shadow
#

bet

grand shadow
analog berry
#

is this with smp?

winged basin
grand shadow
#

smp is still kinda borked sadly

#

I should fix the smp bugs already actually

#

my next goal will be self compilation from disk with smp

winged basin
#

what goes wrong with SMP?

grand shadow
#

something with pending interrupts and an assert fail in the scheduler where I think the thread running flag doesn't get cleared somewhere

#

so far the two bugs I have found

#

the second one is very apparent with like 8 cores

grand shadow
#

the pending bug is related to the dpc stuff like I had thought but like what

#

pending = true

high whale
#

that 0xffffff.........

grand shadow
#

its a function pointer

#

it points to the dpc isr

winged basin
grand shadow
#

I didn't even catch it with a watchpoint I'm just doing the backtrace from the assert fail

#

the assert failed but pending is true which is so weird

winged basin
#

set pending after the assertion failed one presumes?

grand shadow
#

oops I am an idiot

#

I made dpcisr a global but its supposed to be cpu local

#

aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa so much time wasted

#

its always the little things

winged basin
#

"its' a stupid mistake but thank god it's not an extreme subtle mistake that would have taken me weeks to figure out"

grand shadow
#

yeah true

#

thats one down

#

time to go after this one

robust geyser
#

oops

#

lmao

grand shadow
#

ugh I really need to optimise my tlb shootdowns

winged basin
grand shadow
#

well it built with 8 cores

#

is it stable on smp? no

#

does it at least boot now? yeah

#

I ran into all sorts of weird bugs like one where rax magically corrupted getting the cpu struct
like in gdb gsbase was fine, the self pointer was fine, but the return value from rax was 0xff

#

still no idea what caused that

#

since it never even touched the stack

#

and optimising the tlb shootdown is like desperately nescessary if I want any performance when doing smp

#

time to put this vm to rest meme

#

why is it working

#

there we go it crashed

#

got kinda far into configure with 8 cores

#

anyways thats enough smp debugging today for my sanity

robust bobcat
#

attempting self-hosting?

grand shadow
#

yeah, I wanna work towards that

#

I was able to compile the kernel under itself and now I'm messing around with autoconf stuff

#

somehow these smp fixes seem to have helped fix the issue with the configure breaking

#

I was able to run configure a few times

#

I do wonder why it doesn't run make though

#

from the tmpfs it ran just fine

craggy tide
#

bruh do you not sleep anymore? lol

grand shadow
#

Its 10 pm I will sleep soon lmao

grand shadow
#

*something* I fixed for smp was also causing this

grand shadow
high whale
#

can you build the linux kernel in astral

#

🤔

#

would be damn cool if you could

#

since gcc seems to be 100% ported

#

you can try porting git and some other stuff

grand shadow
grand shadow
high whale
#

a develop and coding OS

high whale
errant zephyr
#

just port libslirp :^)

high whale
#

specifically arch's pacman which is written in python and its compatible with cython (afaik cython is 100% gcc compiled)

errant zephyr
#

not really no

#

and also pacman isnt written in python

high whale
#

ohhh its in C, well if TCP works then there you go pacman works too

#

its a very good idea in that case

errant zephyr
#

i mean

#

it might not be easy to port

high whale
#

well atleast git, curl and any programming language package manager will be compatible

errant zephyr
#

it could be

#

idk

high whale
#

tho i can see like cargo, nuget and go's pkg mgr being ported over to astrel

errant zephyr
#

how do you know

errant zephyr
#

because rust, mono and go are way hard ports

high whale
errant zephyr
#

also as this is a hobby os any thinking about users is completly unwarranted

high whale
# errant zephyr because rust, mono and go are way hard ports

i suppose its better to make like a linux structure emulator kind of thing, so like it will have functions that are relied by these pkg managers and then those functions in that emulating library will just convert into current kernel instructions

errant zephyr
#

what.

#

please uh

#

read

high whale
errant zephyr
#

its not

high whale
errant zephyr
#

and it fundamentally misunderstands how things work, and also its mega hard to do

high whale
#

see its like if you use the gcc std libs for a hello world program

#

if you write your own C compiler that doesnt have printf

errant zephyr
#

not how it works

high whale
#

you can just make a mid-way library that has printf but does smth native so its like a function pointer

high whale
spare sky
#

Or source compatibility?

high whale
errant zephyr
#

brain damage compatiblity :^)

high whale
#

so its easier to port source

spare sky
errant zephyr
#

so basically already what they are doing?

spare sky
#

the libc acts as a layer between the kernel and the program

#

it's literally how everyone here does it

high whale
#

basically libs in between of source codes where it is made only for linux

#

like linux-only code reliant on linux drivers or some specific linux thing

spare sky
high whale
#

these libs will act as a mid way from these linux specific codes to a more compatible in this case astral programs

spare sky
#

as in, extensions in the libc

high whale
spare sky
#

TCP?

high whale
spare sky
#

yeah, but userspace doesn't interact directly with drivers

high whale
#

functions in linux that are only in linux by default

errant zephyr
#

theres actually really little linux-only stuff, most things are supported by at least a couple other oses

spare sky
#

^

#

yeah

errant zephyr
#

(or with small changes, linux epoll -> bsd/macos kqueue)

#

linux pidfd -> macos port fd to a task port? kinda?

spare sky
high whale
spare sky
#

anyways

#

you can still implement epoll and whatever in your userspace libraries

#

e.g. Managarm implements epoll in userspace

spare sky
#

i.e. implement interfaces that are somewhat source compatible

#

ofc it still requires some patching

#

but that's how porting works

high whale
#

nvm

spare sky
#

that's kinda required

#

unless you want full binary compatibility

#

you gotta at least patch the build system or whatever

high whale
#

idk why people just dont port ELF

#

if they do its heavily modified

spare sky
#

most people here use ELF

errant zephyr
#

wdym port elf

high whale
#

but only select few fully do

errant zephyr
#

elf is a format

#

like "zip"

spare sky
#

most fully support ELF, whatever that means

analog berry
errant zephyr
#

why dont people port zip files to their oses smh

high whale
high whale
errant zephyr
spare sky
#

most do port the GNU toolchain

errant zephyr
#

or llvm

high whale
#

but other than this server most people make their own virtual formats

#

like llvm

spare sky
#

?

errant zephyr
#

llvm is just elf too

analog berry
spare sky
#

^

errant zephyr
#

its like "gcc"

analog berry
#

Some people do PE but that's very few and its fine as well

errant zephyr
#

its a neat format

high whale
analog berry
errant zephyr
#

from me or from someone else

analog berry
#

Like 5 different people

errant zephyr
#

ah okay

analog berry
#

How it can be multiarch?

#

And stuff

#

I think

spare sky
#

anyways, there's more to binary compatibility than just using ELF and the GNU toolchain

analog berry
#

Your binary format is almost unrelated to binary compat tbh lol

errant zephyr
#

theoretically

jovial moat
#

guys

#

stop fighting with him

errant zephyr
#

i wonder if lipo would let you do that actually lmao

jovial moat
errant zephyr
#

most curious

analog berry
jovial moat
#

qwhar?

analog berry
#

They just realized they said something stupid

jovial moat
#

that was 17:45

errant zephyr
#

i mean it could be a troll

jovial moat
#

what?

lean kettle
analog berry
#

Doubt

jovial moat
#

what in the skibidi

#

how can i prove it

#

lol

errant zephyr
#

where is that from?

jovial moat
#

dms

errant zephyr
#

ah yes the best source of unfalsifiable evidence

charred sinew
#

Live screenshare of discord

errant zephyr
#

nah

analog berry
#

I believe it lol, I just don't think it was honest

errant zephyr
#

you can fake that easily

#

that said

charred sinew
errant zephyr
#

if you know what you are doing

lean kettle
#

its quite easy to do in 5 min

jovial moat
#

iim not faking

lean kettle
#

honestly its not that hard to do in 10 seconds

jovial moat
#

istg

analog berry
#

Source: trust me bro

jovial moat
#

blud

errant zephyr
#

but its also not that hard

#
// Options for the observer (which mutations to observe)
const config = { attributes: true, childList: true, subtree: true };

// Callback function to execute when mutations are observed
const callback = (mutationList, observer) => {
  for (const mutation of mutationList) {
    // you also have to walk the whole subtree here iirc
    if (mutation.target.children.length == 0 && mutation.target.innerHTML.includes('not  faking')) mutation.target.innerHTML = 'i love faking dms'
  }
};

// Create an observer instance linked to the callback function
const observer = new MutationObserver(callback);

// Start observing the target node for configured mutations
observer.observe(document.body, config);
analog berry
#

JavaScript experts enter the chat

errant zephyr
#

basically this

jovial moat
#

#lounge-0 message

#

i cant send from phone to a thread

analog berry
#

By annoy he just meant like talking

#

You can tell its not serious

high whale
#

i wonder whats going on

spare sky
high whale
#

actually i feel bad for doing this

#

im sorry, no offense ❤️

spare sky
#

ok

#

nobody gets banned today

high whale
#

i prefer to act dumb so i get more knowledge

high whale
spare sky
#

trolling is bannable

#

so see this as a final warning

high whale
#

i mean i read half the rules

#

but then it was 3 am so i dont remember anything

spare sky
#

well, I'd kinda consider that common sense

grand shadow
#

I'm gonna be seeing with how little memory it can boot

#

tried with 100mb and it boots to fvwm3 just fine and also shows my "take standby page from page cache" logic does work so far

#

ran gcc and it gpf'd

#

oh huh i/o error made it die

#

because I passed the wrong address to a pmm_release oops

#

but yeah that was probably an oom error

#

yup, it ooms

#

so 100mb is enough to boot to fvwm3 but not a lot else

#

50mb isn't enough even for limine to start

#

64mb isn't enough for fvwm to start

#

but bash does start

#

let me see how many pages it has free

#

9262

#

thats 37mb usable

#

so in theory it should be able to boot to bash with like 30mb of memory

#

I should add an oom killer or something

#

and also flush dirty pages etc

winged basin
#

no point in "heavily modifying" it, it's alraedy a working format and "heavily modifying" it would ruin the advantage of having all the tooling available

winged basin
#

though i suppose if an initrd is involved that imposes some restrictions

high whale
#

i dont think theres any significant program that does it

#

but its like a low-level bytecode interpreter concept

#

its not even running on the real CPU but its more like an instruction interpreter on a set of virtual registers

#

ig its unique enough

winged basin
high whale
winged basin
jovial moat
#

also, what is a elv

winged basin
#

you can see mach-o as a series of instructions to run on an interpreter, whose instructions are things like "map this there", and the result is a loaded executable

winged basin
#

there's only elf and elves

high whale
# winged basin mach-o is a bit like that

see the idea is that it will have a regular header but then you have custom registers and the way it works is your push data to those and then run a kernel checking syscall where the kernel interprets said register data as instruction

#

i think you are starting to get it but the core major plus point to this is that no program can have a memory leak

#

since all registers are temproary and ran only for that program, theres no chance of memory leaks or mismatched pointers

errant zephyr
#

and since the format is called ELF

#

just like you have one dwarf and multiple dwarves

errant zephyr
#

is it like ebpf?

grand shadow
#

stress testing gwah

#

256 mb of rma

grand shadow
#

I was able to fit in quake before it failed an assert on something which I think is unrealted to caching

grand shadow
#

yeah found the issue

#

forgot to set a timer entry as fired in a check function

grand shadow
#

I think there seems to be some sort of corruption of the scheduler stack which is causing issues

grand shadow
#

yeah that was funky

#

it handled signal stuff with the scheduler stack

#

including thread death

#

and process death

#

which uh

#

requires calling into an allocator

#

so it enables interrupts

#

which corrupts the scheduler stack which isn't supposed to be preempted at all

#

hey it did it at the same time

#

hopfully that was the source of the issues

#

I do need to optimise the order in which I take standby pages from the cache though

#

oh boy another catch

#

what is it this time

#

also I need to go even further than this if it sleeps in the signal check it will end up reusing the scheduler stack later which is bad bad

#

I could just add an arch_context_switch to the end of the trampoline since its what would end up happening once the external sched_usercheck function returns anyways

high whale
# errant zephyr is it like ebpf?

Not sure what that is but imagine an interpreter that gives instructions to kernel directly and the syntax is like assemblyish but in hex format, it’s like a virtual format that is interpreted on virtual registers (which is int variables)

robust geyser
#

so like ebpf

high whale
#

ill check and let you know

robust geyser
#

The Berkeley Packet Filter (BPF; also BSD Packet Filter, classic BPF or cBPF) is a network tap and packet filter which permits computer network packets to be captured and filtered at the operating system level. It provides a raw interface to data link layers, permitting raw link-layer packets to be sent and received, and allows a userspace proc...

high whale
analog berry
#

this is bpf, not ebpf

#

ebpf is general purpose unrelated to network

robust geyser
#

yeah yeah but it works similarly, no?

analog berry
#

probably

robust geyser
#

there is an ebpf section on that wikipedia page too

high whale
#

although ebpf is still not entirely like what i said

#

cuz theres no packets, clients or servers

robust geyser
#

well it's a bytecode format that can do all sorts of things

#

it's pretty much what you described

high whale
#

just imagine it as a python interpreter except it deals with fake assembly instructions and all data is stored on integers rather than parts of CPU

robust geyser
#

yeah

high whale
#

its an unique idea ngl

#

and thats why i like to call it "immutable execution"

#

not sure if it already exists

#

but yeah no memory leak would be possible in such an executor

robust geyser
#

linux can run native x86 programs and not leak memory too

high whale
#

unless you purposefully brick the registers by corrupted integers that is

high whale
#

thats the advantage

#

you still need to optimize code and what not but on a bare level you can never cause a kernel panic or a cpu halt since everything goes through a check by the interpreter

robust geyser
#

we have a tool for that

high whale
#

like overflow data, incorrect syscalls, accessing used memory (segfaults) and all

robust geyser
#

its called cpl 3

high whale
robust geyser
#

ring 3??

#

also if you want to do all that it's going to be terribly slow

#

just let everyone do what they want to do

#

you can add checks at syscall boundaries

#

make your kernel robust

high whale
robust geyser
#

dont write more and more shit just to prevent someone from passing in bad pointers or whatever

#

just make sure they arent bad in the first place

high whale
high whale
robust geyser
#

also you should probably make your own thread

#

and not hijack astral's :^)

high whale
high whale
# robust geyser also you should probably make your own thread

just a last thing to say, such an executor will always have pre-byte code that is readable before it is hex-ified and sent as an executable, its like compression except the checking is done then, you can pass bad pointers or whatever with like separate commands and what not but the interpreter doesnt check in real time, the pre-byte -> executable is when the checks happen, to bypass that you can add a flag, I am working on such an executor in C++ and in C# (for COSMOS toolkit, check it out) and these executors are still very fast at execution for example, programs that are 1-2k lines long when is compiled into an executable and run that on bare hardware it is about the same speed as this interpreter, for example I tested a simple bare calculator porgram in C and the same in a pre-byte format that I am making, the times were almost same with the C program coming at 0.29 seconds and the interpreter coming at 0.73 seconds. The only thing making it slow is that the interpreter is converting instructions and sending it to hardware (i.e using stdlib functions)

#

theres still a lot of limits to this but yeah I find this method better since at the loss of speed you trade it in for convenience of not having to deal with memory unless deliberately wanted

#

case closed.

grand shadow
#

I fixed the issue that prevented it from building

#

it also make installs but gnu hello fails cuz of some wprintf assert on mlibc

#

oh wprint is stubbed

#

still

#

thats a step towards self hosting, being able to build an autoconf program

#

from disk

#

still slowly fixing stability issues over time

grand shadow
#

my ext2 implementation also has some issues

#

after compiling both astral and gnu hello at the same time e2fsck gave me some errors

grand shadow
#

finally got around to making signals other than sigsegv be raised so sigbus sigill sigfpe can now be encountered by the applications

grand shadow
#

watching /dev grow as you add more drivers and things is so satisfying tbh

grand shadow
#

I'm going to implement setitimer and for that I will make an abstraction over my cpu timer_t

#

its going to work mostly like the normal application interval timer but not its not going to be restricted to just application use

#

I should be able to implement real, virtual and prof timers with this

grand shadow
#

also I can defo see an improvement in stability now, its able to configure while building itself and running neofetch

#

still can't say much about smp stability tho meme

#

actually I do wonder if these last fixes I did helped smp stability

#

maybe one of the todos for today should be a better tlb shootdown too

#

right now I send an ipi per page unmapped which is bad bad bad bad bad

#

like wtf bad

grand shadow
#

very specific weird thing that happens is when you ctrl c out of make it segfaults

#

I should implement named pipes simply because make jobserver

#

yeah I think I fixed those single core instability bugs, I've been able to constistenly run configure and build stuff and all for testing

#

that was painful but worth

eager copper
errant zephyr
#

not necessarily

grand shadow
#

I'd still need to do a few things to support them so

#

also astral hit 100 stars woo

flat copper
#

yayy

grand shadow
#

apparently mlibc doesn't like if sys_futex_wait is interrupted

#

it randomly failed and made fvwm(I think) die

#

@winged basin after searching the managarm discord it seems you ran into the same issue, in the end did you just make sys_futex_wait not interruptible?

robust geyser
#

this should definitely be fixed on the mlibc side

#

EINTR is allowed to happen on FUTEX_WAIT

winged basin
#

i forgot whether it looked like there would be lots of work to fix

#

but i would imagine it would actually be reasonably straightforward to fix

grand shadow
#

I should see if I can make a pr for it

#

I am not entirely sure how it will be handled in mlibc internals but I will try nonetheless

grand shadow
#

hm I think thats it

#

wasn't that difficult

#

I forgot I've been running with 256 mbs until quake OOM'd in my face oops

robust geyser
#

is it just not enough to start? or does astral leak memory?

#

or rather, not enough to start and keep running*

grand shadow
#

nono astral can start to fvwm with 128 mb and boot to bash with theoretically 30 mb its just that quake has a big heap

#

which causes it to oom as I didn't really allocate a lot of memory to the vm

#

actually wait it was probably the sys_futex_wait thing killing fvwm or something nvm

#

almost got the pr done I just gotta look at one more thing

#

fixed an issue with pthread_once while at it

#

well I have opened the pr

grand shadow
#

now do I do fat32 so it is actually installable to disk from inside itself or do I do tcp..

grand shadow
#

I think I'm gonna do tcp just to have the netstack done at this point

#

fat32 and installing to disk comes later

robust geyser
#

do tcp so you astral can download and install itself from within itself

serene parcel
#

Do u host it somewhere?

grand shadow
grand shadow
errant zephyr
grand shadow
#

ty

#

the issue was my header len being wrong

#

and then I spent a few hours struggling against the compiler which had decided to optimise out the fucking checksum function for some god unknown reason

#

but at least I get a SYN+ACK back

errant zephyr
#

your window size shouldn't be the current mtu

grand shadow
#

Yeah I alr changed it to 64k

grand shadow
#

thats something

#

I need to do the closing logic

#

also still don't have retransmissions

#

but that shouldn't be too hard

#

ephemeral port allocation does seem to be working though, I can rerun the program many times

#

and each time it uses a different port (since I still don't handle closes properly)

grand shadow
#

alright thats the client side of things working as far as I can see

#

only untested codepath is TCP_STATE_CLOSING but that should also work anyways

#

I should port something like wget and see if I can download something

grand shadow
#

well its struggling to dns

#

hm I do get the dns query response

#

recvmsg just doesnt return

#

djhgsdjghdgkshshmsdhnfjshdsnnfdfnaskdkfjdjafsjdkdsa

#

its supposed to be pollin

#

there w ego

#

now its waiting at poll

#

more tcp checksum issues I am going to cry

#

msg_peek apparently

grand shadow
#

oh wtf

#

thats returning enomem

#

oh its trying to

#

read size 0

#

it segfaults ugh

#

wait

#

I don't think its

#

supposed to be reading that many bytes

#

oops it was a miscopy on the msg_peek thing

grand shadow
#

tcp is fucking SLOW and I need to compile wget with openssl

#

but it does work

#

and thats astral build gnu hello but it was downloaded at runtime

#

and some shitty minesweeper gameplay because it sucks to play with a laptop touchpad

grand shadow
#

thats for tomorrow I don't want to even touch anything related to porting today

#

yeah it really cannot keep up lmao

#

tbf I still don't support receiving or transmitting out of order packets so

robust geyser
#

but damn good job man

#

impressive pace

grand shadow
#

tyty

#

tomorrow I will implement tcp server side and then probably optimise this because it despearetly needs it

robust geyser
#

sshing into astral from another astral vm would be based asf :^)

#

dropbear should be pretty simple to port

#

for the ssh server part

#

for the ssh client you could do busybox

#

idk how hard openssh is to port, but i think that would give you both so you may try that instead

grand shadow
#

yeah I really wanna port ssh so I can just go and tell my friends "hey can you ssh here" and have them be confused

robust geyser
#

i think you might need to port login for that to work tho

#

/usr/bin/login i mean

#

idk what package provides that sadly

grand shadow
#

from the ubuntu install in wsl its shadow-utils

robust geyser
#

yeah that should be easy then

#

in case of doubt ping the @ portmaster btw

grand shadow
#

did managarm ever port openssh

robust geyser
#

yep

#

i dont think it works as a server though

grand shadow
#

oh then it should be fine the mlibc side of things should work fine

robust geyser
#

but i think you can ssh from managarm

grand shadow
#

hmm

bright kernel
#

also the rate at which things get implemented in this OS is insane

grand shadow
#

I've had a good amount of free time this month

#

usually I have barely any so the urge to osdev ends building up

#

and then when I do have free time I spend like 10h straight doing this shit

bright kernel
#

ah yeah I see

#

same here

#

this entire week I only went to school for 1 day

#

since the rest of the week were free (at least here in Brazil)

#

and so I worked on my OS a lot more doing small fixes

#

and ported NES emulators and cowsay lmao

grand shadow
bright kernel
grand shadow
#

ya

bright kernel
#

shiiiiiiiiiiiiiiiiiiiiii damn

#

where u from? SP?

grand shadow
#

RS

bright kernel
#

oh damnn

bright kernel
#

I met francis from kiddie OS

#

nelson from the sirius OS

#

Fred from Gramado OS

grand shadow
#

I've seen the gramado OS posts on reddit before

bright kernel
grand shadow
#

thats a healthy tcp conversation

#

wait wtf

#

someone put the github link to astral on some random russian forum

craggy tide
#

it's called being popular

#

:3

#

i made cregfix.com and some Russians took it and made a VXD version

#

it happens, i don't even mind, it was 0BSD

grand shadow
#

I mean hey it did pull in more traffic than anything else so I'm not complaining

analog berry
grand shadow
#

github analuytics thing

analog berry
#

Damn

grand shadow
#

I just need TCB_STATE_SYNRECEIVED now and I'll have the basic part of tcp implemented (except retransmissions but who needs reliability anyways meme )

grand shadow
#

I just need accept

grand shadow
#

now I need to add retransmission and clean this code up

#

wait wtf I just noticed that mss

#

oops I was passing the one of the listening socket

robust geyser
#

holy fuck tcp server side support on astral?

#

good shit

grand shadow
#

Soon I will be able to host the worlds (slowest) http server

grand shadow
#

Yeah this code really needs a cleanup

#

Once I implement retransmission Im gonna do some cleaning

grand shadow
#

what the fuck is listening on 10.0.0.123

#

what the fuck

#

its a ghost

#

spooky

#

I'm trying to test retransmission by connectign to an address which I thought nobody would even be at to respond but I guess I was wrong and summoned the rfc 793 ghost

#

its gonna be a longer wait before it times out like that

#

starting at 1 sec until the rto is 64

#

once that rto of 64 expires it will reset

#

my tcp stack is luck based if you're lucky you do 200 kb/s if youre unlucky you do 22kb/s

grand shadow
#

for today I will be doing the handling of the peer sending me an RST packet, the queueing of several outgoing segments at the same time as I only send and retransmit one and the buffering of some out of order segments

grand shadow
#

once I have tcp going faster and also fat32 I wanna set up a local http server and try downloading and installing astral to disk from it as if I were downloading the packages

#

like tar up a bunch of packages and leave them there so I can wget from inside astral

#

might do just the basic stuff for bash and wget first so then I can install the rest from inside the installation.....

neon crane
#

fat32 isn't exactly like unix

robust geyser
#

yeah, extfs might be a better choice given it isn't any harder than fatfs and is actually more in line with unix concepts

grand shadow
#

I do have ext2 and I do run from it as a rootfs

#

Its just that limine cant install to it lol

eager copper
grand shadow
robust geyser
#

oh you're right lmfao

#

i forgot limine can't do extfs ☠️

#

can you mkfs from within astral?

grand shadow
#

Yeah I have e2fs tools

robust geyser
#

very nice

grand shadow
#

I dont have an fdisk tho

robust geyser
#

thats util-linux

#

shouldnt be too bad

grand shadow
#

Hm yeah true

#

With fat32 and an fdisk port I could properly install limine to disk and then download the packages (or cp -r everything from the install media lmao) and have a perfectly running astral system

#

That sounds epic

#

Still gotta finish tcp and have it not suck that much

neon crane
#

nvm I'm dumb

#

(I thought it could because nanoshell supported ext2, but I realized all my successful limine boots were from ISOs)

#

but wasn't ext2 support implemented in limine

robust geyser
#

it got dropped in 7.x i think

neon crane
robust geyser
#

in limine?

#

version 7.x

grand shadow
#

I have been quite busy with life today, very little progress has been done on astral

obsidian pivot
#

it seems pretty active

prime juniper
# analog berry any project does, imagine not accomplishing your 2020 goals after 4 years

Just catching up here, sorry for the necro and all that. That blog post you’re referring to (same for mint) is written in 2019. Two of the more active contributors now weren’t there or not as involved yet at that time and thus had nothing to do with these goals. While we didn’t port a webbrowser in 2020 (I personally think Firefox in 2020 wasn’t reasonable considering where the project stood purely in terms of ports in April 2020), we currently possess a capable webbrowser that can run discord so we have done that. The netstack, while not fully done yet and absolutely missing server side, has received a lot of work and there are driver PRs open (and usb Ethernet merged) as we speak. We ditched LAI for the better option of uACPI, which didn’t exist back then, and while we’re certainly still AssertOS and there are still things that shouldn’t trip like it does, surely it has improved in the past years and I can say that I’m currently gathering bug reports from the glib test suite so we can fix more crashes. Real hardware support remains a thing and probably always will, we’re not Linux, and neither are any of our projects here. The block stack, while not to my liking, mentions AHCI and NVMe, which we have had for a while now. Self hosting, the last item I’ll address from that post, has improved. Performance is still bad, especially if I’m screencapping it, but I recently showed you I can get network with dhcp, grab a tarball, extract it, configure it and build it. I severely dislike how it’s always managarm bad, managarm dead Project from some people here. Unfortunately I don’t have unlimited free time and neither do the rest of the people working on it. I know mint said that he’s only doing it to get us back to work but it’s demotivating as fuck for me. Makes me not want to come here and hang out. And I want to hang out here cuz I enjoy that a lot. So can we stop with the Managarm dead project bullshit? Yes it’s slow at times, and not as slow at other times. It’s not dead. Never was. If you want it to go quicker, be the change you want to see

#

CC: @craggy tide ^

#

Sorry for hijacking your thread for this rant @grand shadow but I had to say it

analog berry
#

be the change you want to see
trying to do my part caret

prime juniper
#

And I appreciate that. Your uACPI work (and related items inside Managarm) are immensely helpful

analog berry
#

thanks

#

im just mostly frustrated that its not moving at a faster pace but i appreciate the work you and other people are doing on it ofc

prime juniper
#

Trust me, that frustrates us all (at least me and Leo, but I assume that goes for everyone)

#

Doesn’t make the project dead. Sometimes I also just don’t have motivation and real life can be a bitch. I’d have a lot more time if I stopped doing some real life shit but something something going out more and meeting people IRL is a good thing or something like that meme

analog berry
#

true caret

craggy tide
#

i will only mention mlibc because it might as well be its own project

prime juniper
#

you can mention projects all you want, point out broken items or flaws whatever you want, please do. But it hurts to see people talk about a project I poured my soul in for the past 4 years in this way. Especially when it's not (entirely) true

#

I hope you get that

craggy tide
#

you get too personal with managarm

#

you take "attacks" on managarm as attacks on you

#

i don't think that's super healthy

#

but i am not your psychologist so i won't bother hurting you anymore by criticising managarm

prime juniper
#

they kinda are, calling it dead implies that everyone doesn't do shit

#

I am included in the set of everyone

#

I might take it too personal tho

#

I'll take that along, you might be right there

craggy tide
#

compared to the goals set in 2020, it is kinda dead, like sure, some shit has been done, but it's also been 4 years, and there are big hiatuses in development where truly no one really does shit - and i understand that people are busy, i am not criticising people's lives when i say managarm kinda dead, again, you take it personally that way

#

i hope you understand

prime juniper
#

the goals set for 2020 (in 2019) were not entirely reasonable, I agree (firefox 2020 was never happening). The real hardware thing will always be there, as we're not Linux

#

I agree that the pace of the project isn't as good as I'd like it to be

craggy tide
#

if i may add, managarm is way overambitious

#

every time there is a new problem that needs solving, the hardest way is always seemingly chosen

#

which, while commendable, certainly doesn't help with the project's pace

#

given the lack of manpower and time from contributors

prime juniper
#

can't disagree with that, dunno if it's always, but we certainly seem to like the hardest way

craggy tide
#

i wish i could contribute but 1. i do not really like that approach and 2. i am too stupid to understand managarm code

prime juniper
#

understandable on 1, same on 2

#

which is why I port shit meme

craggy tide
#

yeah, which is amazing

#

your work has been useful way beyond managarm

#

i have based many of my jinx recipes (especially dependency chains of the likes of Xorg) on your work

prime juniper
#

yeah I'm happy to see that people use it / reference it

craggy tide
#

if you wanted to you could probably be everyone's portmaster :^)

prime juniper
#

probably meme

#

but I'd need about 3 days in a day for that meme

craggy tide
#

yeah that's understandable

#

i maintain Vinix's and Gloire's ports and that's already a lot of work

#

i have thought about centralising ports

#

for mlibc-based hobby OSes

#

like, have a single repo with ports and then have each OS import whatever packages they want

#

it can be reasonably done

#

99% of ports are shareable code

prime juniper
#

probably yeah

craggy tide
#

if we end up doing that, you are certainly much appreciated for input/contributions

prime juniper
#

of course

grand shadow
#

Minecraft on managarm when

lean kettle
#

maybe soonish meme

prime juniper
grand shadow
#

Running the client when

#

I wanna play minecraf at 1 fps

#

I do wonder if running an older minecraft client is possible

lean kettle
# grand shadow Running the client when

requires a loop back interface + maybe upstreaming things into/patching that java lib minecraft uses to determine what OS it runs on that i forgot the name of

#

other than that, we should technically basically have everything needed to run minecraft

grand shadow
#

That stuff shouldnt be too hard

prime juniper
#

the native libs is gonna be interesting

lean kettle
prime juniper
#

outside that it's indeed only the loopback interface

#

afaik

lean kettle
#

haiku ported MC for example

prime juniper
#

oh

#

then we can do it too

lean kettle
#

and they had to do weird shit because i think it runs of their native display protocol

lean kettle
#

it also run at like 7 fps but we dont talk about that

grand shadow
#

7 fps is better than not running

prime juniper
#

which means ours will run at 0.7 fps

lean kettle
#

most of it has to do with mesa and llvmpipe

prime juniper
#

true

lean kettle
#

i got between 7 and 12 fps on 1080p running natively on my laptop

grand shadow
#

Write a gpu driver to play minecraft meme

craggy tide
#

just get a decent cpu

#

that can run basic 3d stuff at decent fps

#

with llvmpipe

#

my cpu can probably run minecraft at a playable framerate

lean kettle
#

well, install haiku and try i guess halfmemeright

craggy tide
#

could

#

i ran supertuxkart with llvmpipe

#

it was playable

#

and that is much more graphically complex than minecraft

lean kettle
#

minecraft is written in java and has mojang code in it

#

supertuxkart is written in C and has decent code in it

craggy tide
#

eh fair

grand shadow
#

@prime juniper also did you ever port ssh and sshd to managarm

prime juniper
#

I have it locally