Astral

[grand shadow]

but its nice enough that using it for everything wont hurt (and itd be nescessary anyways)

and with copying buffers I mean like directly copying to the page cache etc etc etc

it also has the nicety of not having to keep track of how deep you are into the buffer anymore

a nice effect of this is that you can create say a list of
region 1 size 32
region 2 size 12
region 3 size 32

and read say 40 bytes from device a and then 40 bytes from device b
and it all gets copied and handled transparently by the iovec_iterator abstraction

and if you want to just pass a buffer and lenght and not worry about it, there are the socket_{write,read} and vfs_{write,read} helpers

I should probably rename the socket ones to send and recv to make them more in line with the socket ops

[grand shadow]

alright, thats done now

all I/O goes through the iovec interface and copies directly from user to where it needs to go

another step in the unshittification of astral

[grand shadow]

I'm trying to port some funny x11 program and damn I have never heard of this function before

[wild marsh]

sigvec sounds like a precursor to sigaction

[grand shadow]

also the build system is so horrid I literally am just calling the compiler myself because fuck that

[wild marsh]

i wonder why it doesn't use signal unconditionally, that's in standard c afaik

[grand shadow]

this program is old old old

[analog berry]

What does it do

The program

[grand shadow]

you'll see in a sec

its beautiful

[fleet osprey]

[analog berry]

Bruh

What the hell is secure mode on this

[grand shadow]

dont ask me

[analog berry]

Fish cant escape the tank ig

[wild marsh]

what's up with the main and datetime windows?

[grand shadow]

I think its the fvwm sidebar just not wanting to load lmao

[fleet osprey]

well that's certainly... interesting

[grand shadow]

it puts it above all other windows, I guess as a type of screen saver

did a kill -10

[wild marsh]

screen locker

[grand shadow]

yeah I cant go back

[wild marsh]

to unlock you walk to the sysadmin's office and ask him to send it sigusr1 again :^)

[grand shadow]

this is literally the whole recipe

name=xfishtank
revision=1
from_source=xfishtank
imagedeps="base-devel imake"
hostdeps="xgcc xbinutils libtool pkgconfig"
deps="base imlib2"

build() {
        # the build system sucks, so we will directly call the compiler and ignore it!
        cp -rp ${source_dir}/* .

        gcc makeh.c -o makeh

        ./makeh fishmaps/?*.h > xfishy.h
        x86_64-astral-gcc -Dlinux -I./fishmaps -lX11 -lImlib2 -lm xfish.c medcut.c -o xfishtank
}

package() {
        mkdir -p ${dest_dir}/usr/bin/
        cp -v xfishtank ${dest_dir}/usr/bin/xfishtank
} 

it uses fucking imake so nope fuck off I will just do it myself

[grand shadow]

@prime juniper you should see if this runs on managorm

youre probably better off downloading it and building it inside managarm lmaooo

[fleet osprey]

btw, you still have imake in the deps

even thought you bypass it

[grand shadow]

shit true

[obsidian pivot]

good thing debian has this pckg

[prime juniper]

Nah I don’t even start on this shit

[grand shadow]

alright

soon I will rewrite the scheduler

first, though, some cleanup

removing proc and thread from sched

[grand shadow]

thread and proc now have their own files

and a scheduler folder in sys

enough for today

[fleet osprey]

🔥
cleanup is always nice

[deft oak]

holy shit this is amazing

[neon crane]

just remembered this discussion. wait what

like you can override a preexisting mapping with another?

[carmine swallow]

yes or eg. a smaller range that is in the middle of an existing mapping

[neon crane]

NGL that's kind of stupid

but i have no choice but to support it

if i want posix compatibility

[grand shadow]

first thing I will do will be make a cpu hierarchy

[carmine swallow]

btw did you add the function to have a semaphore with a max count?

[winged basin]

a very useful thing

after all it makes sense to aggressively work-steal between hyperthreads but less so between cores less shared

[flat copper]

it doesn't always make sense to work steal within hyperthreads, as that impacts the single threaded performance heavily

[grand shadow]

not yet, I can add it in a bit

[carmine swallow]

also tyrquake does work pretty decently now, the problem was that I unconditionally waited on the semaphore inside close for some reason

[grand shadow]

@carmine swallow pushed

do keep in mind I am using a new way of passing a buffer to drivers now

but switching from the old way to the new way should be as simple as switching out a function call

(and making sure you can take a fault)

[grand shadow]

does this mean that its not like huge delay now

[carmine swallow]

the iovec iterator stuff? I already switched to it a moment ago

[carmine swallow]

yes at least as far as I tested

also re the card dying inside windows, what was the exact sequence on how you got from astral to windows? because there are only really two things that I can immediately think of, either the card/codec ia not properly power cycled before windows tries to use it (which is needed because the astral driver doesn't properly stop the card on eg. a reboot) or maybe its the weird realtek quirk

[grand shadow]

yeah that

[grand shadow]

epicc

[grand shadow]

uhh I ^C the cat, tried to cat it again, astral gets stuck in the cat and then I reset

[carmine swallow]

ah

[tribal dust]

i decided to build and explore astral

anything specific that you are proud of and I should check out?

[grand shadow]

Nothing really stands out too much tbh

Some parts are pretty solid, some need to be better

[analog berry]

The ps2 driver

[grand shadow]

Nahhh 💀

Thats one of the "it needs to burn" parts

[tribal dust]

ok here's my first issue

• building package: astral
  /home/sanana/src/os/Astral/.jinx-cache/rbrt: mount() failure at line 309: Operation not permitted

[grand shadow]

What distro are you using

If you dont want to have to build it I have some precompiled binaries in my server

[tribal dust]

nixos

where does this thing try to mount stuff?

[grand shadow]

@craggy tide

Does jinx work on nixos

[tribal dust]

and it's a binary i think? that's no good for nix (I have nix-ld so it should be ok tho

[gritty shore]

day #239829382 of asking for docker support in jinx

would work anywhere

[tribal dust]

rootless podman

[gritty shore]

yeah podman could work too

[tribal dust]

port docker to astral

[grand shadow]

@tribal dust

[gritty shore]

I'm working on a podman jinx-like thing

[grand shadow]

I updated the image there yesterday

[tribal dust]

i'll try it

[tribal dust]

i'm working on nixstrap

wow so many alternatives

[gritty shore]

oh yeah that looked cool

idk how well it works on non-linux tho

my goal is to make it work anywhere podman runs

[tribal dust]

it should be OK for darwin other than nixpkgs issues

which I really couldn't fix

but often nixpkgs has darwin in badPlatforms because lack of maintainers

[gritty shore]

my main motivation is doing osdev on macOS

I tried like 5 different ways of getting jinx to run and it just wouldnt

[tribal dust]

yea jinx wouldn't run

you need some container runtime or something like nix

[gritty shore]

main problem was FS issues

I was basically running jinx which runs its own container in a linux container in a linux vm

[tribal dust]

lmao

[grand shadow]

fakeroot fakechroot /usr/sbin/chroot

[tribal dust]

my eventual goal for nixstrap is to eventually replace cppnix with something else, either something that really is not nix or a nix reimplementation that is non-unix friendly

but that is not today

today it runs everywhere nix runs

nixstrap shouldn't need much other than cppnix and libguestfs

yea i really can't think of anything that would really be needed other than stuff that disko requires (depends on what filesystems you want in the disk image) and libguestfs + cppnix

when I say nixstrap dependencies I mean required dependencies

for example the limine nixpkgs package does not build on darwin I think, so if you use the nixstrap limine module, then it wouldn't build

but you could definitely roll your own, nixstrap is modular

downloading mathewnd's astral build

[neon crane]

What kind of software uses readv/writev? I assume chiefly network stuff?

I wonder if its emulatable with a series of read()/write()s for a temporary solution

[grand shadow]

Its expected to be atomic Im pretty sure

[grand shadow]

Network stuff prefers to use readmsg/recvmsg

Which uses an iovec

[neon crane]

ah so that's a "no" unless you are the only reader/writer and can ensure that fact

[grand shadow]

Not even for that

[neon crane]

but thats kinda fucked up

[grand shadow]

Say you want to send datageams

Datagrams

Itll send many datagrams instead of one

[neon crane]

cause normal read() and write() dont have such atomicity themselves do they?

wrong reply

[neon crane]

.

[grand shadow]

More so in, its treated like one big write/read

So say you writev to a pipe, as far as its below pipe_buf on the total size of the write all the iovs will be written atomically

[neon crane]

ah

i see

[grand shadow]

Do keep in mind about this pipe write atomicity stuff, its not often implemented in hobby kernels

[grand shadow]

@tribal dust did it work

[grand shadow]

150 stars, nice

[analog berry]

nice, L for uACPI tho

[grand shadow]

soon uacpi will surpass acpica I trust

[analog berry]

Current state against ACPICA is:

• Way way better portability (drop-in cmake, meson, easy to tell which files to include, easily overridable platform layer)
• Way way better NT compatibility
• Way way better at using the stack (stuff that would recurse in ACPICA doesnt in uACPI)
• Way way better at handling bogus AML (no currently known fuzzer input that segfaults it unlike ACPICA, which dies if u blow on it wrong)
• Way way better CI and tests (complete lack thereof for ACPICA with half abandoned test suite)
• Lacks some minor convenience API
• Lacks thread safety in a lot of places
• Way less battle tested
• Lacks cool features like object cache, user-provided buffers for return values, a few other things here and there

A few of these im going to solve before 1.0

The rest maybe later

and well obviously ACPICA also provides a compiler, a disassembler and a debugger

which uacpi is not going to provide ever

[errant zephyr]

not in linux

:^)

[analog berry]

which point

oh like

• not in linux

yeah right

forgot that one

i can also say its a downside, since it has a lot of linux-specific (aka only in the linux source tree) patches, and no one knows how many of these are bugfixes that never made it to ACPICA upstream

thats the bad thing about being the bespoke linux acpi layer instead of trying to be a generic thing

[grand shadow]

write uacpi patches for linux

[analog berry]

well like i mentioned before, its not hard at all to make linux work with uacpi but it has 999999 places using ACPICA API

like most of that api maps 1 : 1 to uACPI but its still code that must be manually rewritten

so its almost impossible

aka would take months/years

[grand shadow]

I mean would something like that even be accepted

[analog berry]

of course, as long as there's a demonstrable improvement in some area it would absolutely be accepted

which is technically possible but years of work and talking to upstream

and well, uACPI polish as well, which it lacks in some aforementioned areas at this point

Like ACPICA is on life support right now, nobody is working on it

[grand shadow]

one could dream that if it really becomes nescessary multiple people would work on it

[winged basin]

it depends on whether you'd want to have your project imported mercilessly into the linux source tree never to make it out again

and hacked apart by randomers, and any changes you want to commit needing signed off

[analog berry]

yeah they dont do submodules so it would end up being a bespoke linux layer as well

[grand shadow]

damn

[winged basin]

drm used to be platform independent too and look what happened

[analog berry]

yup

[winged basin]

embraced, extended, extinguished

[analog berry]

was gonna say

[errant zephyr]

classic MSFT

oh wait

its anti msft sorry

[analog berry]

the closest possible thing is reactos, which had some interest as well

[errant zephyr]

why is it that everyone loves EEE so much

[winged basin]

it's a push but at least one of the BSDs may consider it

[analog berry]

yeah probably

[winged basin]

macos is out (do they use acpica anyway?), freebsd, dragonfly, net are possibilities, openbsd has their own acpi

[obsidian pivot]

I kinda want (doubt I ever will though) to try and port uACPI to linux and get it to boot (but without any drivers)

[analog berry]

macos rolled their own acpi interpreter

[winged basin]

that's fair enough for them

[errant zephyr]

macos doesnt have anything anymore now

[analog berry]

they also expect very specific aml with very specific objects so not like it matters

[errant zephyr]

especially since they don't even need to deal with hardware quirks

[winged basin]

they don't have to deal with countless aml insanities churned out by clowns

[analog berry]

yup

[errant zephyr]

yeah was gonna say

[analog berry]

they just

if it doesnt match what they expect

[wild marsh]

considering the new apple silicon computers don't use acpi at all that might be completely out of the question sooner or later :^)

[analog berry]

yeah they use the apple device tree thing

[winged basin]

funnily enough on that

they synthesised a device tree on the i386 port in the past

[errant zephyr]

i figure apple is commited to an ADT

no reason to go ACPI now

[analog berry]

especially since arm acpi is at not-so-good stage rn

good thing NT is pushing for it tho

so its not going away any time soon

[winged basin]

i can't remember if ARM ServerReady Extreme System Environment! or whatever it's called mandates ACPI or just uefi

[wild marsh]

the one for workstations does mandate uefi + acpi iirc

at least

[analog berry]

good

[wild marsh]

i think only the embedded one wants device trees

[obsidian pivot]

funny how this is the astral thread

[wild marsh]

and rest are either acpi-only or both

[analog berry]

only embedded people that dont have the time nor expertise to write acpi

or any sort of firmware to run it ig

[obsidian pivot]

isn't an aml interpreter also really big

[analog berry]

yeah true

[obsidian pivot]

so like acpi might be outta the question

[analog berry]

probably

[winged basin]

i wonder now whether riscv's version of ARM ServerReady Extreme System Environment!, which given riscv tradition is probably called "rvsbscbaippaktbnsdaiohsd", does mandate acpi itself, or even uefi

[wild marsh]

i mean you're already running linux anyway

most likely

[analog berry]

lmao

[obsidian pivot]

the static lib I compile for uacpi is 2.5 mib

[winged basin]

ACPI is the required mechanism for the hardware discovery and configuration. Server platforms are required to adhere to the RISC-V ACPI Platform Requirements Specification [21]. Platform firmware must support ACPI and the runtime OS environment must use ACPI for device discovery and configuration.
yes it does

[analog berry]

cool

[obsidian pivot]

and my kernel binary is 2.7 mib

[analog berry]

well if u do UACPI_HARDWARE_REDUCED or whatever its gonna be cut in half

[obsidian pivot]

fair

[winged basin]

surprisingly it's not even an alphabet soup name

it's called "RISC-V OS-A Server Platform Specification"

[obsidian pivot]

and if the linker is any good it removes code symbols that aren't used (at least I think linkers do that?)

[wild marsh]

what does that even disable?

[analog berry]

all registers and event management and shit like that

[wild marsh]

linkers usually deal with sections and not individual symbols

there is --gc-sections

but you need to compile with -ffunction-sections to have each function go into it's own section

[obsidian pivot]

I see

[analog berry]

uhh so if u just have .text it wont clean up dead code?

[errant zephyr]

yes

[analog berry]

wtf

[obsidian pivot]

In my experience, if you have a static lib which you use no symbols from no code is included

[wild marsh]

yeah linkers gc objects that don't have any references to them

(they also can do that for shared libraries with --as-needed)

[analog berry]

so which one is it

[errant zephyr]

one or both or neither

or some combination of the above

[analog berry]

💀

[wild marsh]

if you feed the linker foo.o but none of it's symbols are referenced in relocations from any other object file that object is just discarded

[analog berry]

that could be an interesting project

[winged basin]

i think it's worthy, a good stress test

[analog berry]

ohh by objects u mean .o

i thought it was some linker terminology

[gritty shore]

One day I will finish my linker

[obsidian pivot]

By objects I thought you meant symbols

[analog berry]

indeed

[wild marsh]

i worded that poorly

my bad

[obsidian pivot]

I wanna look into mlibc's dynamic linker and implement unimplemented relocations

[winged basin]

that was a nightmare for me once, trying to figure out the best way to get objc categories defined within a static library to load

[errant zephyr]

gotta love linkers

[obsidian pivot]

Yes.

Especially the m68k one

[winged basin]

there is a way to do it in apple ld but i don't know about gnu ld

[winged basin]

how goes your database anyway

[errant zephyr]

slowly

[winged basin]

and your orchestration tool

[errant zephyr]

im only working on it half of the time

[errant zephyr]

on hold until the db is done

so basically never

[analog berry]

ur making a db?

[errant zephyr]

it's semi working but i want to make it use the DB

[obsidian pivot]

Why it be deep-frying my executables

[winged basin]

i would love to implement a database but i don't have the prerequisites yet

[errant zephyr]

me neither, the difference is im just writing one anyway :^)

[winged basin]

but the problem domain bears a suspicious resemblance to filesystems and their interaction in terms of caching with memory so it's not a million miles away from osdev

[analog berry]

will it have sql like SELECT whatever FROM?

[errant zephyr]

its a nosql so no

because it has to be webscale

(still doing raft)

[obsidian pivot]

Bruh this is the astral thread

[analog berry]

yeah thats uh

beyond my knowledge

[wild marsh]

same, one day i will make mine support multiple input files, and make it actually lay out phdrs instead of only generating sections :^)

[errant zephyr]

all that means is that no

its just the same thing

but with a different syntax

[analog berry]

i see

[gritty shore]

My goal was to leverage Rust's async and "fearless concurrency" to try and beat mold

But I gave up because rust lol

[wild marsh]

i was writing mine in scheme, for my 65816 project https://github.com/qookei/paaliaq/blob/master/toolchain/reloc.scm

[gritty shore]

Speaking of which I kinda wanna build a breadboard computer with that chip

I mean ideally I'd have a fricking 386 but that won't work on a breadboard

[wild marsh]

actually

[errant zephyr]

rust async probably wouldnt help

you want rayon

[wild marsh]

https://www.youtube.com/playlist?list=PLw_043zkzQMoJCsi9rAPtaRf8Crzq7GEA

[gritty shore]

How slow does it run lmao

Breadboards can do about up to 20mhz

And that's really pushing ot

Also newer CPUs aren't dip so you need a breakout board I think they're called

[gritty shore]

Yeah but dunno how that compares to mold's parallel library

[wild marsh]

i mean that's right around the speed the (cheaper) 386s and 486s run at anyway no?

[gritty shore]

I don't think mold would be that hard to beat tho

[errant zephyr]

idk onetbb sounds really impressive

[gritty shore]

Mhm yeah I guess the slower i386s ran around 12mhz

can always use the embedded ones

Which ran as slow as you wanted

I mainly want a "modern" cpu because paging is just so nice

[gritty shore]

Yeah exactly

[obsidian pivot]

This reminded me to check ben eater's channel

I know it's not ben eater's project

[grand shadow]

thats on my watchlist

[obsidian pivot]

But anyway

[grand shadow]

when I finally had money to do breadboard computer stuff I wasnt too interested anymore (and when I tried I managed to hurt my fucking fingers wiring stuff up lmao)

I still have the z80 and stuff

[obsidian pivot]

Send them to me :^)

The breadboards

I should port obos to the i686 then make my own bread board i686 computer

[gritty shore]

If I ever do a breadboard computer I'd like to build some kind of macintosh thing but with a more modern cpu

[obsidian pivot]

Or i386

Might be smaller

[gritty shore]

Then print it on a pcb

Kinda worried about graphics and stuff

No idea how breadboard-able that is

[obsidian pivot]

I mean if ben eater made a breadboard gpu, what's stopping you

It was dead simple

Though

[grand shadow]

fuck it get an early amd64 processor

ascend

idk how hard itd be or if itd even be possible

[gritty shore]

That's shitty for real use

Ideally It'd be something like vga

or TIGA

Like the TMS34010

I think that's fairly breadboardable

[gritty shore]

You'd need to design a pcb

https://m.youtube.com/watch?v=TerraI_xMbg

Sounds promising

[obsidian pivot]

How would you get pci though

I guess no pci

[gritty shore]

no need

PCI didn't exist when 386 was first made

[obsidian pivot]

Wonder if I would be able to get seabios running

[gritty shore]

I'd just write my own

[obsidian pivot]

Probably not, my thing would be too obscure

[gritty shore]

simpler

[obsidian pivot]

Yeah

[gritty shore]

I already wrote half of one for my 5150 emulator :^)

[obsidian pivot]

Or make obos be the firmware

[grand shadow]

firmware for something that old surely wouldnt be too hard

[obsidian pivot]

And if it's homebrewed it might be easier since you know what you need to init and where everything is

[gritty shore]

Yeah 5150 wasn't too hard, you just have to have a ton of bios calls

[obsidian pivot]

I'll write ASL for it

[gritty shore]

You don't even need BIOS

wait

Just implement like 20, the ones needed to run limine

Then that's it

you can hardcode most of them too

Like e820

Or video modes

[obsidian pivot]

I can probably find an i386 chip on ebay for cheap

[gritty shore]

yea

[obsidian pivot]

And breadboard are easy

To fidn

[gritty shore]

High quality breadboards are like 10$ each

I reckon you'd need atleast 5

[obsidian pivot]

Ye

I'd need some sort of

[gritty shore]

I'd say 7-8 should be fine theoretically

[obsidian pivot]

memory address decoder

and ram

and rom probably too

[gritty shore]

ram and rom are cheap

On digikey

[obsidian pivot]

because idk how to make a dynamic ram refresh thingy

I'd just use sram

expensive but easier to work with

*more expensive

[gritty shore]

Why

There's a builtin mmu

[obsidian pivot]

mmio

[gritty shore]

You don't need a chip for that tho?

[obsidian pivot]

I mean

[gritty shore]

I don't think so

[obsidian pivot]

some sorta system

to do it

as in some and chips

to know whether to set the ram EN pin high

[gritty shore]

there's gotta be a way by mapping the address line pins or something

[obsidian pivot]

so you know how chips have the ENABLE pin

iirc they're used for stuff like this

[gritty shore]

yea

[obsidian pivot]

for example if bits 0xffff0000 are set, you'd set ROM.ENABLE to high (I think it's LOW to enable in a lot of cases, but whatever)

otherwise it could set RAM.ENABLE to hgih

[obsidian pivot]

or whatever the rom address was

I'd need to read the sdm for that

I'd add a uart

to it

what was the chip called

that had the ps/2 controller

and a bunch of other stuff

Intel 82091AA

unless that's too big

or I can't find it

but anyway

let's stop literring the astral thread

[gritty shore]

yeah you'd probs need a pic and pit too

Getting an OS to run on that would be an exercise in optimization

[obsidian pivot]

Yes.

[obsidian pivot]

so an Intel 8253/8254 and an intel 8259

(PIT/PIC)

along with the aic

[obsidian pivot]

(this)

and the i386

ram

rom

and other stuff

that I'm probably forgetting

[gritty shore]

Yeah that's it

Wiring it up is another story tho lol

you need a video chip too

[obsidian pivot]

indeed

[obsidian pivot]

that too

[wild marsh]

video chip: uart

:^)

[obsidian pivot]

tbh sounds good

I don't wanna deal with video timing and shit

and dma

and whatnot

[formal patio]

xbstrap with extra steps

[gritty shore]

Honestly yeah

except no yaml

[formal patio]

the yaml is about to change a bit btw

[gritty shore]

json instead

[formal patio]

korona is cooking

[gritty shore]

phdml when

[wild marsh]

it's still yaml no? but now there are macros that can transform it

[formal patio]

shhhhhhhh

yaml but easier

I mean already as-is, yaml is easier to type than json tbh

[gritty shore]

Yeah but formatting errors are annoying

Honestly jinx shell scripts are the best

[formal patio]

my problem is that I professionally hate maintaining shell scripts

[gritty shore]

I mean aren't your yaml files just shell scripts but in strings lol

[formal patio]

it's mostly program invocations

[gritty shore]

Yeah that's what I meant

[formal patio]

so not shell, but execve

[gritty shore]

jinx recipes are basically just invocations

[obsidian pivot]

maybe I should make a qemu target for this first

idk

I definitely will be getting a gdb stub though

[gritty shore]

well

if you're using relatively standard pc components no need

[obsidian pivot]

yeah actually fair

I could probably just do -M none and add the components myself

also for the funnies I will be implementing debugcon in real hw

(port 0xe9)

[obsidian pivot]

*aip

[gritty shore]

id just do serial

[obsidian pivot]

if anything it'll just redirect to port 0x3f8

which is com1

[gritty shore]

problem with hardware is you need to do polling and stuff

you can't do 0xe9 with that

[obsidian pivot]

tru

I'll just not do anything

as in

not do debugcon

if I end up doing this

anyway, since this my project I'll move to the obos thread

[grand shadow]

I wonder if you could like

send an ipi by writing to the funny msi address

[obsidian pivot]

I mean

yeah

but actually no

[grand shadow]

this is 0x8 when running on linux but 0x10 when booting off astral

what the fuck?

[obsidian pivot]

magic

[grand shadow]

wait nvm I forgot that wsl does shit

I need to check what it returns on windows

this is a 4 core 8 thread cpu it makes no sense for the value there to be 0x10

bruh?

I'm probably reading something wrong wrt to the spec then

hmmm I think I am

its talking about the physical package

not the core

so the range for core is insde the range for the logical processor in the physical package

man I'm tired

finally

ok time to test on real hw

an then go to sleep and clean out this path tomorrow

Still doesnt work on my laptop

[craggy tide]

why wouldn't it?

as long as the kernel is Linux, of course

[grand shadow]

Maybe its the old ass version I use or something

[craggy tide]

oh definitely

[tribal dust]

yep

[grand shadow]

Epic

[grand shadow]

alright I got the cpuid outputs and played around and seems like I got it right now

just gotta implement it in the kernel and test if it will work

[grand shadow]

my thinking is right

it just seems like cpuid leaf 0x4 returns 0 on all registers?

[grand shadow]

Quick google search tells me I have to use __get_cpuid_count instead of just __get_cpuid when using leaves that can have ecx set, oops!

[grand shadow]

pushed x86 topology detection stuff, still havent written the part which the scheduler will interface with though

also pushed a change idfk why I wasnt doing yet

on anonymous CoW with page refcount == 1, I just map the page now directly instead of copying it to a new page

really simple change, I really dont know why I didnt do it before

[tawny oak]

Oh cool, what are your plans for topology aware scheduling?

[grand shadow]

Something super simple and based on ule:
If it needs to be scheduled on a cpu, it goes there
Otherwise, it will triy to put it back on the cpu it has affinity on (such as the last one it was executed on), if it is not too overloaded
If that was not possible, it will walk back the topology trying to find a close cpu that can run it
Failing that, it will just find the least loaded cpu in the system and put the thread there

Also, I wonder how fucked up a CDE port would be

[supple hatch]

hi

[grand shadow]

Hello nyauxmaster 5.0

[supple hatch]

hi mathew

[grand shadow]

@prime juniper mr portmaster mr portmaster between 0 and WHY what do you think it'd take to port cde

[gritty shore]

Would be epic

[obsidian pivot]

what dat

[prime juniper]

Ngl but the fucks a CDE?

[grand shadow]

https://en.wikipedia.org/wiki/Common_Desktop_Environment

[prime juniper]

Oh

That

[obsidian pivot]

just port kde

what could possibly go wrong

[prime juniper]

Where sources

[grand shadow]

https://sourceforge.net/p/cdesktopenv/code/ci/master/tree/cde/

[prime juniper]

But considering it’s based on X11 and targeting Unix

[grand shadow]

I assume itd be horrifying because I had troubles getting it working on linux lmao

though once I rewrite the scheduler I can find out

[gritty shore]

Honestly might make my own cde-like DE if I ever get to that point

Something about old Unix guis...

[prime juniper]

Euhhhh

Why is libpam-dev listed

That better be optional

It is

Good

[errant zephyr]

i think its portable

OpenVMS is not unix

its vms

[analog berry]

ngl id kill for astral with kde

in minecraft

[prime juniper]

Why is it depending on libtirpc for the Linux backend? My usual path is say you’re Linux, disable Pam and go

[prime juniper]

Aight time to revive the KDE on Managarm project (tbf systemd on managarm is part of that cuz logind session memes)

[analog berry]

that would be huge

[grand shadow]

ikr

[prime juniper]

I have the entire stack ported (at least to a basic plasma desktop iirc)

[gritty shore]

I'd add my own twist tho, probs add a global menubar

Like macOS

[prime juniper]

But I’m running into startup memes

[analog berry]

yeah i mean thats the hardest part

systemd shit

[prime juniper]

Related to sessions and shit

[prime juniper]

No no I did that with elogind

[grand shadow]

I should also see if I can see wtf is causing fvwm to fuck up the side bar

[analog berry]

ah

still all of shit like dbus, sessions, etc very very borign and complex af

[prime juniper]

Yeah it’s a thing

[analog berry]

and its needed to run most complex software

is the worst part

[prime juniper]

Yup

But I think we have dbus relatively going now

[analog berry]

yeah thats very cool

[prime juniper]

Tho who knows what happens once we do users properly

Access control might be a bitch

[analog berry]

astral is very far from kde at least because it has basically 0 drm or anything like that

[grand shadow]

just run everything as root forever

yeah I dont drm

[prime juniper]

Yeah that’s gonna be some patches and that still doesn’t eliminate the session stuff

But yes

[analog berry]

i will fix that at some point ™️

[grand shadow]

uGPU.........

[analog berry]

stuck in uacpi scope creep forever

i want to finish it properly before working on other stuff

so that its an actual production ready library

[grand shadow]

by then I (hopefully) will have fleshed out astral's driver interface more

I was going to do that but I got sidetracked by something arguably more important

(which is making the scheduler actually good)

[analog berry]

both are important tbh

[grand shadow]

true

uni break is over but that just gives me something to scribble and think about when slacking off in class

[analog berry]

yeah some time to read about algorithms and stuff instead of coding stuff directly

[robust geyser]

you can provide a drm interface for a plain fb

idk if linux does tho

efifb maybe?

[analog berry]

yeah ofc, thats what managarm does

its still kind of tons of code

[robust geyser]

yeah drm is pain

[analog berry]

its definitely grown over the years too

[grand shadow]

I will do it

I will do the unthinkable

I will finally do it

invariant tsc for timekeeping

hpet too slow

especially since I need a fast and reliable and nonshit time source for the ULE interactivity metric

and also qemu is faster this way

after around what year is invariant TSC guaranteed to be a thing?

[obsidian pivot]

not 2013

[grand shadow]

ah well doesnt matter, I'll still have the hpet as a fall back

so if it doesnt have invariant tsc tough luck, buy a better computer next time and bear with the slowless for now

[grand shadow]

Another thing I will also do is rwlocks

[tribal dust]

port ideas: neko (maybe oneko) and minesweeper

[grand shadow]

minesweeper and xneko

Already ported

[prime juniper]

Now port libremines

Do it

(Audio optional)

[tribal dust]

omg :3

[grand shadow]

I like the penguin doing faces on ace-of-penguins minesweeper

[prime juniper]

Been there done that (never tried that minesweeper tho, I ported that for solitaire lol)

[grand shadow]

I ported it cuz I saw it on some fucked up linux video

And thought itd be funny

[prime juniper]

It is funny

[grand shadow]

https://youtu.be/ZaSIQU1kAAc?si=zgVhAYe6pjp8ILGC

[fleet osprey]

what the hell
this is..... uhhh

I have no words

[grand shadow]

lol I did *something* that increased fps in glxgears by 10

[obsidian pivot]

scheduler

[grand shadow]

I havent even began to rewrite it 😭

[grand shadow]

does wsl2 not pass the invariant tsc to the linux vm bruh

[analog berry]

Broke the clock reporting?

[grand shadow]

I didnt change much other than making it more precise so meh

dont think it was that

might be more related to me changing something about cow I think but probably now

[obsidian pivot]

I think after my vmm rewrite I will finally get to implement swap in the astral kernel

this time I don't plan on doing anything cursed

[obsidian pivot]

so hopefully in a day or two

[grand shadow]

yeah its just straight up not reported by cpuid

going insane

[analog berry]

-cpu host?

Oh like wsl2 itself is not invariant tsc?

[grand shadow]

yea

[analog berry]

Ah

[grand shadow]

tsc: Marking TSC unstable due to running on Hyper-V

[winged basin]

hyper-v is known to have fucked up tsc

whether it will behave properly if you actually have an invariant tsc on the host i don't know

[grand shadow]

I do all this programming on this laptop and I cant install linux because eventually someone else uses it and she has stuff here aaaaaaaagh

I do have invariant tsc on my host cpu

[winged basin]

maybe it will just work under hyper-v then

[grand shadow]

I should find a way to tell linux that its okay then

or just straight up patch the kernel

[winged basin]

as in hyper-v would have to deliberately fuck things up in order to make it so that the tsc were "unstable"

funny thing, apparently there are some bioses whose SMIs fucked with the tsc in an attempt to make it appear as the SMI never ran!!

absolutely unbelievable

[grand shadow]

funnily enough it seems like its just straight up disabled by it in the cpu

so cpuid 0x80000007 returns 0 in the edx bit 8

virtualbox with hyper-v does report an invariant tsc

so it is a config specific to wsl

[robust geyser]

czapek@raptor-wsl:~$ dmesg | grep -i tsc
[    0.000000] clocksource: hyperv_clocksource_tsc_page: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[    0.000001] tsc: Detected 3417.599 MHz processor
[    0.006855] TSC deadline timer available
[    0.040487] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x31433fb2e38, max_idle_ns: 440795229393 ns
[    0.134735] clocksource: Switched to clocksource tsc-early
[    1.310970] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x31433fb2e38, max_idle_ns: 440795229393 ns
[    1.313251] clocksource: Switched to clocksource tsc
czapek@raptor-wsl:~$ cat /proc/cpuinfo | grep -i tsc
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ss ht syscall nx pdpe1gb rdtscp lm constant_tsc rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single ssbd ibrs ibpb stibp ibrs_enhanced tpr_shadow vnmi ept vpid ept_ad fsgsbase tsc_adjust bmi1 avx2 smep bmi2 erms invpcid rdseed adx smap clflushopt clwb sha_ni xsaveopt xsavec xgetbv1 xsaves avx_vnni umip waitpkg gfni vaes vpclmulqdq rdpid movdiri movdir64b fsrm md_clear serialize ibt flush_l1d arch_capabilities```

interesting

i assume "tsc deadline timer available" means ive got invariant tsc?

this is wsl2 too

🤔

[grand shadow]

invariant tscp is nonstop_tsc iirc

which you do have

what wsl2 ver are you running on

[robust geyser]

idk, how do i check? lmao

[grand shadow]

and what distro

wsl --version

[robust geyser]

debian

[grand shadow]

o

[robust geyser]

WSL version: 2.0.14.0
Kernel version: 5.15.133.1-1
WSLg version: 1.0.59
MSRDC version: 1.2.4677
Direct3D version: 1.611.1-81528511
DXCore version: 10.0.25131.1002-220531-1700.rs-onecore-base2-hyp
Windows version: 10.0.22631.4169

[grand shadow]

I am on ubuntu

hm

[robust geyser]

yeah no that doesnt matter

distro i mean

its just a different rootfs

[grand shadow]

I'm on wsl 2.2.4.0

[robust geyser]

i do have my own kernel compiled tho

idk if that makes a difference

[grand shadow]

couold it maybe be that?

[robust geyser]

i would send it over but thats weird so i wont lol

compiling it is fairly easy but it takes time

[robust geyser]

czapek@DESKTOP-EPBQ7CO:czapek$ type .\.wslconfig
[wsl2]
kernel=C:\\Users\\czapek\\wsl\\vmlinuz
networkingMode=mirrored
firewall=false

[experimental]
sparseVhd=true```

thats how you enable a custom kernel

.wslconfig in user directory

[grand shadow]

@robust geyser does dmesg have hyperv_clocksource_tsc_page for you

[robust geyser]

czapek@raptor-wsl:~$ dmesg | grep -i hyperv_clocksource_tsc_page
[    0.000000] clocksource: hyperv_clocksource_tsc_page: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns```

[grand shadow]

Starting with Windows Server 2022 Hyper-V, Hyper-V uses hardware support for TSC frequency scaling to enable live migration of VMs across Hyper-V hosts where the TSC frequency may be different. When a Linux guest detects that this Hyper-V functionality is available, it prefers to use Linux’s standard TSC-based clocksource. Otherwise, it uses the clocksource for the Hyper-V synthetic system clock implemented via the shared page (identified as “hyperv_clocksource_tsc_page”).

could it be that my windows is too old for this to be enabled

its been installed since like 2021

no

2022

early 2022

I think

or

my cpu doesnt support tsc scaling or w/e

well

considering I dont plan on doing live migration cant I just disable it complaining??

aghhh

[robust geyser]

my winver

[grand shadow]

yeah its the same

its probs the scaling thing

idk

but its a hyper-v issue since it reports it as being unavailable in the cpuid bit

but only in wsl

in virtual box it reports it correctly

which wtf?

[robust geyser]

hyperv vbox?

[grand shadow]

yeah

[robust geyser]

interesting

maybe too old of a kernel?

maybe they changed something

[grand shadow]

I have no idea atp

[robust geyser]

what kernel do you have?

[grand shadow]

5.15.153 on wsl

[robust geyser]

czapek@raptor-wsl:~$ uname -a
Linux raptor-wsl 6.1.21.2-microsoft-standard-WSL2+ #1 SMP Tue Mar 19 10:13:35 CET 2024 x86_64 GNU/Linux

ohhh

wsl.exe is lying

[grand shadow]

gonna install the debian one and see if that does anything? I doubt it

[robust geyser]

nope

try updating the kernel

either aka.ms/wsl2kernel

and download the update package

or try compiling one yourself

[grand shadow]

xd

I think I'll have to compile it

[robust geyser]

yep

dont forget to checkout the right branch

there are some instructions somewhere how to compile the kernel

[grand shadow]

yeah I found them

[robust geyser]

good luck

or rather... have a good time

takes like 3-4 minutes on my machine, fortunately

i gotta update it

i estimate around 25 minutes on your computer

complete guess

[grand shadow]

we will see

I will do a make with time

[robust geyser]

dont forget to -j`nproc`

or -j$(nproc), whichever you prefer

linux-msft-wsl-6.6.36.6

im building this shit !!!!

[grand shadow]

@robust geyser you were off by a bit

[robust geyser]

i am still building

idk why i need like 90% of these drivers

but whatever

you do you msft

gpu drivers on wsl!!! yeah i need that for sure...

[grand shadow]

inb4 they allow passthrough to wsl

[robust geyser]

ah there

linking

nvm building some more shit..

oh those are just modules

[grand shadow]

crying

[robust geyser]

[ 0.000000] Linux version 6.6.36.6-microsoft-standard-WSL2+ (czapek@raptor-wsl) (gcc (Debian 14.2.0-4) 14.2.0, GNU ld (GNU Binutils for Debian) 2.43.1) #1 SMP PREEMPT_DYNAMIC Fri Sep 27 02:54:33 CEST 2024

czapek@raptor-wsl:~$ dmesg | grep -i tsc
[    0.000000] clocksource: hyperv_clocksource_tsc_page: mask: 0xffffffffffffffff max_cycles: 0x24e6a1710, max_idle_ns: 440795202120 ns
[    0.000000] tsc: Detected 3417.599 MHz processor
[    0.006792] TSC deadline timer available
[    0.038356] clocksource: tsc-early: mask: 0xffffffffffffffff max_cycles: 0x31433fb2e38, max_idle_ns: 440795229393 ns
[    0.087189] clocksource: Switched to clocksource tsc-early
[    1.125985] tsc: Refined TSC clocksource calibration: 3417.599 MHz
[    1.126299] clocksource: tsc: mask: 0xffffffffffffffff max_cycles: 0x31433fb2e38, max_idle_ns: 440795229393 ns
[    1.129983] clocksource: Switched to clocksource tsc```

¯_(ツ)_/¯

[obsidian pivot]

just change the linux kernel

[robust geyser]

really odd

works on 6.1.21.2 and 6.6.36.6 for me

[grand shadow]

ig it really is a hyperv meme or smth

[robust geyser]

just use tcg

honestly

[grand shadow]

but slow

[robust geyser]

right

[grand shadow]

I mean the only reason I was even going to implement the tsc was for faster timekeeping

I could try to botch something together

[grand shadow]

since scheduler needs that

fuck it

kvmclock time

[grand shadow]

ah fuck it I think I should just try to dual boot linux in this thing

what will I be arsed more to do

kvmclock or install linux

[robust geyser]

linux

[grand shadow]

honestly

yeah

[obsidian pivot]

install debian

[grand shadow]

for one specific reason

passing through my ethernet card for when I want to develop a driver for it

[obsidian pivot]

join me, myself, and I using debian

[grand shadow]

I was probably gonna install artix but then again I dont have time to juggle university work osdev and babysitting a linux distro

[obsidian pivot]

exactly

just install debian

anyway, brb

[robust geyser]

czapek@raptor-wsl:~/sources/zig-kernel$ dmesg | grep -i kvm
czapek@raptor-wsl:~/sources/zig-kernel$ echo $?
1```

cool, broken kernel

back to 6.1.whatever i guess

[dim arrow]

arch is better

[robust geyser]

he just said why he doesn't want arch tho

(artix is basically arch)

[dim arrow]

ah

yeah, arch can be a lot of work

[fleet osprey]

I'm an arch user, and tbh, I don't really have that many problems with it

most of the time it just works

[dim arrow]

same here

but when I started, it was a lot of work

it just depends on how you set it up

[robust geyser]

what is "hardware assert support" and why does managarm not have it implemented yet?

[grand shadow]

@prime juniper pls fix

[robust geyser]

have you perhaps looked into the Virtually Contiguous PFNDB™️?

[grand shadow]

windows is not helping and not letting me resize the partition enough grrrr

[robust geyser]

real windoze moenmt

[grand shadow]

I have thought of implementing it yeah

just putting everything in a huge region and praying theres one big enough isnt exactly good

[robust geyser]

yeah, no

i was trying to implement it today but i've hit a few roadblocks and i wanted to see how other people solved my issues

or if those are exlucisve to me being slow

but if you haven't worked on it (yet) then no worries

[grand shadow]

not yet

originally I was going to have each usable region keep track of its own pfns until vmm init

so like at the start of each regions the struct pages for each page would be allocated

[robust geyser]

yep thats what i did in my old kernel

but iterating over memory regions to find the page you want isnt ideal

it should just be pfndb[pfn>>4]

[grand shadow]

which is why then you make it virtually contigous later

[robust geyser]

but thats like double the work

ugh its annoying me that i cant find a better solution even though i know there is one and its not that fucking hard

tomorrow i’ll give your idea a try

even though it pains me i have to implement two memory managers just to throw one away immediately 💀

[grand shadow]

I think you would still need to take care of a few things

[grand shadow]

I mean like

[robust geyser]

necessary evil or something

[grand shadow]

have it at the start of each range and then just map it later on and have a function pointer or w/e

[robust geyser]

not very evil

just stupid

[robust geyser]

i cant have them be mapped contiguous after that

because there might be gaps between thrm after all

[grand shadow]

youd probably still need some calculation for it to put it in the right place in the page

[robust geyser]

page structs for a given region might not fill up an entire page

or like, idk how to word that

but i hope you know what i mean

[grand shadow]

yeah

uhhhhh

[robust geyser]

i turned wifi on and discord teleported me to the first message in this thread, nice

[grand shadow]

I might actually think of how to solve this tomorrow while bored at work if I remember

its too late and me tired

to think

[robust geyser]

i can throw my idea in here tomorrow

[grand shadow]

sure

[robust geyser]

i just have a small issue that im not sure how to fix

maybe you’ll have an idea

sleep now tho, gn

[grand shadow]

gn

[neon crane]

i hate hyperv

worst VM software to ever grace mankind

not only can it not run old operating systems good

but it RUINS the performance of my OTHER vm software

it's also why I use wsl1 all the time and never wsl2

i think it might make my computer slower too but that last one may be just placebo

[prime juniper]

Yep will do. Managarm must assert!

[neon crane]

boron has a functional virtually sparse page frame database

you can look there for inspiration

it doesn't have support for a buddy allocator or allocations bigger than a page of physically contiguous memory but I haven't needed that feature yet

[tawny oak]

I have it implemented if you want to take a look, nothing too fancy to look at.

ah I was not the first to respond 😆

https://github.com/DeanoBurrito/northport/blob/kernel-rewrite/kernel/arch/x86_64/Hat.cpp#L206-L213 where it happens, EarlyPmAlloc is implemented here https://github.com/DeanoBurrito/northport/blob/kernel-rewrite/kernel/interfaces/loader/Limine.cpp#L331, its just a bump allocator from the memory map.
HatInit() is one of the first things run when the kernel starts (theres some hooks for setting up any no-alloc log outputs, global ctors and an arch-layer hook for setting up interrupt handling stuff).

it could definitely be more efficient with not mapping holes in the memory map and using 2M pages where applicable, but i'll get there

[sharp hound]

You need to enable KVM not as a module because wsl with custom kernels does not load them

I've had the same problem and enabling KVM not as a module fixed it

[grand shadow]

These solutions are interesting but I was thinking of making something that doesnt depend on some separate early mapping code idk

Maybe it is faster to just have some separate paging code that takes out of the memory map than using a function pointer or an if else later on

Probably

Func ptr is indirection and fucks up compiler inlining and if else is a branch

[tawny oak]

yeah that sounds unpleasant, especially when its only needed for a short period of time

you could always have it call into another TU and then patch in a jmp over the early code after its not longer needed

could be fun 😅

[grand shadow]

I need to focus on the scheduler rewrite I need to focus on one thing otherwise I will never get anything done

So I will look into it soon

[tawny oak]

haha understandable

[obsidian pivot]

Wait what's whole convo about? PFNDBs?

[grand shadow]

My biggest trouble atm is deciding how I am going to proceed wrt the tsc, since hyperv hates me but I cant really afford to read the hpet on every wake up and interruptible sleep (at least not on fast machines)

[grand shadow]

Yeah

[obsidian pivot]

Hmm

that's like the array of struct pages right

lemme see how I did it...

[grand shadow]

I mean if windows allows me to shrink the partition over 9 gbs itd be easy but nope cant let you install linux nuh uh

[obsidian pivot]

my pmm alloc function returns a uintptr_t

which is the physical address

and struct page allocation is done later

if needed

then there is a separate vmm allocator which is kinda special in that it uses the pmm+hhdm to allocate memory

then the vmm might later allocate a struct page for that physical page and emplace it into an rb tree

trust me I'll push that code soon

[robust geyser]

ideally, you should return a struct page* when allocating memory

and accept that when freeing

i mean not that it matters if your pfndb is virtually contiguous, but if it isn’t then finding the page struct using the physical address isn’t O(1), but getting the ohysical address from a page struct is

[neon crane]

You should have functions to directly manipulate any page table

Separate from VAD lists and stuff like that

[robust geyser]

basically what i was trying to do, but 2MiB pages are giving me a headache

[tawny oak]

thats why I put a TODO

[robust geyser]

i am trying to calculate how much memory i need for the PFNDB, then take the biggest memory entry and steal that much memory from it and leave the rest for 4KiB pages (page tables)

but the biggest memory entry can be misaligned, so if i align up i lose a bunch of 4KiB pages

[grand shadow]

I do yeah thats the arch_mmu_* functions

But they dont have an erly pmm alloc or anything

[tawny oak]

how do you feel about steeling one of the mmap entries from another type? if you dont care about reserved entries, repurpose it for a usable one that has the misaligned part of the entry you want to use.

its not particularly elegant

[robust geyser]

i just thought of an idea

[tawny oak]

ah we should probably stop distracting mathewnd at some point too

[robust geyser]

what if instead of stealing a part from the start of the memory entry

i just yoink it from the end

lol

[tawny oak]

you could still waste pages

[grand shadow]

Idm if its useful discussion and not people just fighting over meaningless shit

[robust geyser]

i will use whats before that for 4KiB pages

so i dont waste the pages before that in the memory entry

and whats after i can account for too, because i know where the 2mib “region” starts and how big it is

[neon crane]

I believe that huge pages are nothing more than a slight optimization which can be focused on later

I wouldn't focus on them at the beginning

[tawny oak]

I agree

[robust geyser]

i dont really want to allocate hundreds of megabytes using 4KiB pages

it seems slooooow

[neon crane]

nop

[tawny oak]

but I also get wanting to have it done right at the beginning

[neon crane]

you don't allocate hundreds of megabytes

[robust geyser]

for a 32GiB memmap you do

[neon crane]

It's about 4 MB for the PFNDB on a system with 16 GiB

[robust geyser]

you dont get that too often but yeah

[neon crane]

Wait nvm no

[robust geyser]

i dont want it to be slow on a system like that

[neon crane]

If your pfdb entry is 32 bytes then its like 128 MB

Can be done in at worst a second

[robust geyser]

a second is a lot

especially when all i expect is a crash

[neon crane]

nop

[robust geyser]

yeah ok maaybe it isnt, but it can be instant!

and i figured it out 😎

[neon crane]

also 2MB and 1GB pages wont even solve your speed problem

[robust geyser]

i will implement it in a bit when i get up

[obsidian pivot]

I do have helpers for that

[neon crane]

well, they will barely

[robust geyser]

less memory to allocate and map will definitely improve boot speeds

[grand shadow]

Just allocate enough for each range at the start of it and map that appropriately?

[robust geyser]

250 allocations vs 100k allocations

[grand shadow]

Not 100% efficient but works

[neon crane]

barely

you still need to initialize that 128 meg database with a free list

[robust geyser]

i guess thats true too

[neon crane]

you can do that if you are hellbent on using huge pages

[neon crane]

why

why not return the page number?

[robust geyser]

i explained it next why i did that in my kernel

basically pfn->page* wasnt free

[neon crane]

it should

[robust geyser]

but page*->pfn was free

[neon crane]

ideally it should be a simple Pfndb[Pfn]

[robust geyser]

yeah yeah it should

but it wasnt

now it will be

[neon crane]

if its virtually contiguous then you might as well have an easy association of pfn->page

also the pfn can easily be converted to a PTE with a single bitshift, there isn't a level of indirection

[robust geyser]

PFN to PTE? interesting

how do you do that?

i know NT has a bunch of bullshit mappings but i don’t know how they work

[grand shadow]

I think he meant like, how its expected to be in the PTE? Like pfn << 12

[robust geyser]

i guess

[gritty shore]

Maybe just check if regions are aligned to 2mb and if so use 2mb for those

Or if they're not and they're like 2mb + 8192 then I'm sure you can figure out a way to do 2mb page + 2 4k pages

[neon crane]

?

the PFN is the physical address of the corresponding page shifted right by 12

you just shift it left by the PFN's position in the PTE

set the relevant bits such as P

boom

[robust geyser]

i thought you meant something else

[neon crane]

were you thinking of VPN to PTE address?

[robust geyser]

that one is obvious

[neon crane]

NT does use recursive mapping which allows a direct conversion

[robust geyser]

yeah i believe NT has some mechanisms for that

but im not sure how that works

ah that makes sense

[neon crane]

you can set one of the top level PTEs to the same pfn as the top level itself

that's basically how it's done.

and it'll be a virtually sparse but linear mapping of all of the page table levels in memory

i have yet to implement this in boron myself

[gritty shore]

how did I get Linux running while using less memory than that

like it was using <40mb on a 8gb system

[neon crane]

Out of how much exactly

[gritty shore]

40mb/8gb

[robust geyser]

probably a pretty tight memory map

but idk honestly how small it would have to be

[neon crane]

Exactly

Not "8gb"

Down to the page

[robust geyser]

just post dmesg | grep e820

[gritty shore]

Idk

[gritty shore]

I don't even have that laptop anymore

I don't think so atleast

[winged basin]

perhaps it just doesn't account the page structs

[gritty shore]

using du -h

no

whats the name

free -h

[grand shadow]

interesting

I need to see if that cpuid bit is set on wsl

if so I can try just patching the kernel

writing to that msr say in linux early boot and praying it actually works

if it does I will actually scream lol

[analog berry]

Whats the current command to build miminal astral from scratch?

[grand shadow]

change makefile to minimal

make

ez

[analog berry]

make DISTROTYPE=minimal?

[craggy tide]

update jinx to latest tbh