#shkwve: an incomplete kernel/maybe bootloader/UEFI thing/maybe something else if i get bored again

1 messages ยท Page 4 of 1

untold basin
#

wtf

#

i dont even know anymore

#

im gonna blame qemu pl011 emulation

#

this doesnt break on 8250 uart

untold basin
#

ill charge the m1

#

and fix the linux boot header

untold basin
#

okay

#

it hard reset really quickly

#

fuck

#

no logs

#

oh wait

#

im not running it on hv

#

damn!

#

ok good that i have a usb reset

#

in hv i get this error

#

i guess

#

damn

#

ah

#

caches!

#

i need to turn those on

#

yeah its an atomic to log::STATE

#

IMPLEMENTATION DEFINED fault (Unsupported Exclusive or Atomic access).

#

wtf

untold basin
#

tbh

#

i think im gonna switch to libfdt instead of my homebrew fdt code

#

later

trim seal
#

why?

untold basin
trim seal
#

make a uFDT

untold basin
#

that would involve writing C code

#

libfdt is already pretty good too

trim seal
#

well not necessarily

#

u can make C bindings for a rust library

untold basin
#

that would suck ass

#

and i would never recommend anyone use that

trim seal
#

lol

#

why does your code suck tho, arent fdts simple

untold basin
#

yes

#

but it is one of my core skills to write bad code for simple things

trim seal
#

lol

untold basin
#

in this case, the issue is rust ownership

#

the dtb is global

#

actually that doesnt matter i think

#

does it

sacred spire
#

just throw an rc around everything bro

untold basin
#

lmao

glad bough
untold basin
#

but theres a lot of big ugly there

untold basin
#

okay time to replace every single place that does any DMA with a proper dma buffer allocation

#

and proper cache flushes

#

i should add backtracing from exceptions on arm64

#

okay first crash

#

fdt parsing is fucked

rose sonnet
#

there's a nice library by r4

untold basin
#

hmm

#

C though

#

also it needs allocation

#

also i need write

open osprey
untold basin
#

plus, limine requires me to remove memory nodes

untold basin
#

okay progress update

#

im on fwcfg now

untold basin
#

okay, now pci

#

and now im getting a weird L1 translation fault

#

wtf

#

ah its oob

#

oookay so on qemu the pci window is outside of the low 128 gigs

#

fucking hell

#

okay

#

new plan

#

i'll construct a page table with a single PTE

#

which maps just the loader

#

then i parse the FDT

#

i'll assume it lives within a single 1 gig contingous area

#

so it can just get its own PTE

#

actually im overcomplicating it, i'll just parse the FDT in a prekernel, clean up the memory map, then map my loader and be done with it

#

and then build the prekernel with -mstrict-align

#

i'll call it "cmodule" instead of "prekernel" though

#

and reuse it for a bunch of other stuff

#

e.g. SMP spinning

#

hmm or maybe not

limber abyss
#

I sometimes look in this thread and wonder what the hell is happening

untold basin
#

lmao

lunar rock
#

lmao

agile nexus
# untold basin also i need write

fwiw it does do write, its opt-in behind a define and can operate with a static buffer if you're using it pre-allocator (its a little gross but it works).

#

afaik the only thing its missing is being able to set the reserved memory entries, which I guess you probably do care about

#

ah and good docs

#

but yeah, cant help you with the 'being written in c' part ๐Ÿ˜›

untold basin
#

i might use it then idk

#

since using C FFI is a famous strategy of escaping rust UB

#

have rust UB? just put the UB in C code and its not UB anymore

limber abyss
#

have rust? just put the rust code in C and it's not rust anymore

untold basin
#

lol

untold basin
#

okay i think i'll do a two stage bootloader

#

to maintain my sanity

untold basin
agile nexus
#

It can make a new fdt if that's what you mean

untold basin
#

ah yeah

#

thats nice

#

i probably still want rsvmem support though

agile nexus
#

Mm I think that requires an allocator though, for the buffer to write into

agile nexus
untold basin
#

unless i just binpatch your FDT to add a rsvmem region

#

wait do you not expose rsvmem at all???

agile nexus
#

Lol or that

untold basin
#

you need to consider it to construct a valid memory map

#

always

agile nexus
untold basin
#

ah you just use it with limine?

agile nexus
#

Its a long story, in the cpp version I did, but since making smoldtb I'm yet to need it

untold basin
#

fair enough

agile nexus
#

Pretty much

#

The CPP version was before limine had rv support

#

So it had those features

untold basin
#

ahhh

agile nexus
#

Honestly, if you're willing to give me a weekend I'll add them

untold basin
#

apart from the fact that it requires allocations, it looks quite nice

#

also why uintmax_t

#

i'd prefer to be given raw bytes

#

there's like a bunch of places where i need the raw cells too

agile nexus
# untold basin also why uintmax_t

its just what I wanted at the time. I was using size_t's for everything but got bitten by that somewhere (I forget where now). uintmax_t is hilariously overkill I know but solved my issue.

agile nexus
# untold basin i'd prefer to be given raw bytes

yeah fair, could be easily added. You could get that via read_string(0), which willl return a char* to the beginning of the property data. actually dtb_stat_prop() returns the base + length of the cells.

#

I should implement be an api for that though, maybe also returning the number of cells. thanks.

untold basin
#

aah stat_prop works thanks

agile nexus
#

I will eventually finish the docs, sorry

untold basin
#

no worries

#

okay the new new plan is to bring back a preloader

#

i'll map it with two 2mb pages, then give it 2 more 2mb pages for mapping the FDT and a 4k page for mapping whatever physical memory

#

running off of the stack that i'll later reuse as the normal shkwve stack

untold basin
#

and also map all of physical RAM

#

and then device memory (e.g. PCI BARs) need to be mapped later by device drivers, on demand

lunar rock
# agile nexus its just what I wanted at the time. I was using size_t's for everything but got ...

About that...

/home/blanham/projects/rooster/build/_deps/smoldtb-src/smoldtb.c:350:34: error: passing argument 3 of 'dtb_read_prop_1' from incompatible pointer type [-Wincompatible-pointer-types]
  350 |         dtb_read_prop_1(prop, 1, &handle);
      |                                  ^~~~~~~
      |                                  |
      |                                  size_t * {aka unsigned int *}
In file included from /home/blanham/projects/rooster/build/_deps/smoldtb-src/smoldtb.c:1:
/home/blanham/projects/rooster/build/_deps/smoldtb-src/smoldtb.h:90:70: note: expected 'uintmax_t *' {aka 'long long unsigned int *'} but argument is of type 'size_t *' {aka 'unsigned int *'}
   90 | size_t dtb_read_prop_1(dtb_prop* prop, size_t cell_count, uintmax_t* vals);
      |                                                           ~~~~~~~~~~~^~~~
/home/blanham/projects/rooster/build/_deps/smoldtb-src/smoldtb.c:360:34: error: passing argument 3 of 'dtb_read_prop_1' from incompatible pointer type [-Wincompatible-pointer-types]
  360 |         dtb_read_prop_1(prop, 1, &handle);
      |                                  ^~~~~~~
      |                                  |
      |                                  size_t * {aka unsigned int *}
/home/blanham/projects/rooster/build/_deps/smoldtb-src/smoldtb.h:90:70: note: expected 'uintmax_t *' {aka 'long long unsigned int *'} but argument is of type 'size_t *' {aka 'unsigned int *'}
   90 | size_t dtb_read_prop_1(dtb_prop* prop, size_t cell_count, uintmax_t* vals);
      |                                                           ~~~~~~~~~~~^~~~```
untold basin
#

lmfao

lunar rock
glad bough
#

lol

agile nexus
#

rip

agile nexus
lunar rock
agile nexus
#

That's a bit embaressing lol

lunar rock
agile nexus
agile nexus
rose sonnet
#

yeah i need to update it

untold basin
#

okay now i'm working on the shkwve first stage loader(tm)

#

so far i have an FDT parser

#

and a naming scheme that is truly horrifying

#

oh and a print function straight out of c++

#

i've heard that C doesn't have function overloading

#

but clearly anyone who says that has never used clang(::overloadable)

#

why yes i would like to have C++ name mangling in my C program

glad bough
#

if you have a very broad definition of overloading

#

_Generic() meme

untold basin
#

you can't even define an overload for void* and have it catch every kind of pointer

glad bough
#

you can afaik?

untold basin
#

which is boring

#

overloadable is way better

glad bough
#

overloads are boring

untold basin
#

its also decentralized so including more headers can add more things

#

so for example including my endianness support header makes my print function know how to print le32 and be32

#

hmm i wonder

#

should i respect /chosen/stdout-path

#

or should any use every serial i find as the stdout

#

probably the former tbh

untold basin
rose sonnet
#

terrifyingly neutral

untold basin
#

it is actually quite nice

#

if you dont want all of C++ but you want just one thing or whatever

rose sonnet
#

clang is not all C

untold basin
#

name one other compiler

rose sonnet
#

idk if you want C++ features why not just use C++

untold basin
#

and exposes me to C++ naysayers

#

also it would probably make me do a lot of c++ crap i dont want to have

rose sonnet
untold basin
#

idk

untold basin
rose sonnet
#

gcc is not clang

untold basin
#

you said icc

#

icc is clang

rose sonnet
#

ok clang is almost all C

untold basin
#

ok fine

untold basin
#

so irrelevant

rose sonnet
#

I think this server need to write its own C compiler

#

There's a shortage of them

untold basin
#

not really

#

we have clang

#

clang is already the best C compiler

rose sonnet
#

but only having 1 compiler is boring

untold basin
#

true

untold basin
#

so thats more progress ig

#

okay next steps

#

uh

#

memory map

untold basin
#

perfect

#

now time to clean it up

untold basin
#

memory map cleanup!

#

and now i have 4k alignment

#

tbh in rust the binary for boot0 would be like 50x bigger

#

and not 44k

#

which is mostly bss

zenith lion
#

you're rewriting in c?

untold basin
#

no

#

i'm adding a C first-stage loader

#

because i need it for bootstrap

#

hmm maybe i should rewrite in C

zenith lion
#

ah

#

M-mode?

untold basin
#

ill think about it later

untold basin
#

so no

zenith lion
#

ah arm true

untold basin
#

yeah arm

#

for now

#

ill be booting with it on riscv too

#

actually now that you said it

#

ill defo use it in M mode if im gonna be supporting split S/M

#

and if im gonna be supporting returning to EL2 etc etc

zenith lion
#

if it's gonna hold the function of an SPL, do M mode

#

if it's not do it in S mode

untold basin
#

it wont do load

#

no

zenith lion
#

because otherwise you're gonna lose SBI :3

untold basin
#

so it will be hard tied to the rest of the logic

untold basin
#

would be a lot of effort but its totally doable

zenith lion
#

there's a rust sbi implementation i think it's by xvanc?

untold basin
#

yeah but why

#

:^)

#

also now that you said it maybe i should rewrite in C

#

idk

#

ill think about it

zenith lion
#

rust is fine

#

this loader might be constrained by size tho

#

so C for this and rust for the rest makes perfect sense

untold basin
#

the entire thing is gonna be one big binary

zenith lion
#

rust for the rest:^) rhymes

untold basin
#

baked together by some build type process

untold basin
#

lol i dont want to write two disk drivers

zenith lion
#

i thought this was gonna be an SPL essentially so stored on rom

untold basin
#

nah

#

its to set up page tables and shit

#

and standardize the environment and such

zenith lion
#

ah

untold basin
#

because i dont want to compile everything without aligned access support

#

and also rust without paging on arm64 isnt really feasible

zenith lion
#

ah ok i understand

untold basin
zenith lion
#

btw i really hate the fact that u-boot is basically the only SPL option in risc-v

untold basin
#

i see but also im not gonna write stupid amounts of disk driver code

#

it would easily double the code size and require me to write lots of careful handover code

zenith lion
#

maybe not coreboot, coreboot's cool

untold basin
#

okay the new version of the shkwve boot0 can now set up paging too

#

now i'll do handover ig

untold basin
#

hmm i saw how linux does iommu stuff

#

ill probably steal it tbh

untold basin
#

hmm now im wondering if i should do like

#

rust for drivers and C for everything else or something idk

trim seal
#

why is that

glad bough
#

rust for everything

trim seal
#

i have a better idea

#

C for everything

untold basin
trim seal
untold basin
#

lmfao

trim seal
#

they also consume terabytes of stack

untold basin
#

and their ub rules are insane

glad bough
untold basin
#

im seriously considering using more C

#

especially because C codegen is a lot better lul

devout swift
untold basin
#

i can lto and strip C code too lol

glad bough
#

it's fine tbh because you have nostd anyways

#

with std i would agree

untold basin
#

im using C already for the first stage boot logic

#

but yeah idk maybe not

untold basin
glad bough
#

then doing it in rust was a mistake to begin with KEKW

untold basin
#

fair lmao

trim seal
#

why do u have multiple stages now?

untold basin
#

i need to have some paging setup

#

and the paging setup is nontrivial because it needs to allocate and parse FDT and shit

#

and i need it to not be rust

#

and i need it to be a separate TU

untold basin
#

two copies are a bit excessive

glad bough
#

dynamic driver modules!

#

have a dynsymtab the size of the whole binary!

zenith lion
#

e.g. Box::new() and stuff that doesn't first allocate on stack

#

it's been an open issue for decades

trim seal
#

nah id rather use a language like zig or c

zenith lion
#

:(

#

i would be so happy to have new-in-place

#

and a syntax for new-in-place too (like C++'s new-in-place

trim seal
#

be the change u want to see meme

zenith lion
#

sure but there's lots of bikeshedding

#

i wish rust wasn't as big as it is and we could do changes faster

#

at some point I'll return to langdev

#

I would really enjoy a language that only I can use

#

that would allow me to not care about other people's opinions

#

lzcunt software license

Definitions

Licensor means lzcunt and lzcunt only, if you're not her you're not licensed to use this software

open osprey
untold basin
open osprey
#

Because a good driver API is an abstract one. So the drivers don't have to deal so much in the problems rust can't answer

zenith lion
#

that's why it's a license

untold basin
#

anyway

rose sonnet
#

Why not Go

untold basin
#

no.

untold basin
#

okay so how should i implement my new shitty driver model

#

hmm maybe i should use message based async

#

what if i did C++

#

and used C++ coroutines

#

because C++ has implicitly shit perf instead of explicitly shit perf

#

and i have some C code for boot0 already

untold basin
#

holy shit who came up with this crap

trim seal
untold basin
#

OOM handling isn't a thing there anyway

sacred spire
untold basin
#

well

#

maybe

#

but consider this

#

its not rust

sacred spire
#

true

rose sonnet
untold basin
#

lol

tawdry knoll
#

What are you doing to make your Rust binaries so large? I see this brought up a lot, but never experienced it myself, so I am wondering whats causing it?

tawdry knoll
#

Also really cool project btw, I like the way your register dump looks lol

zenith lion
#

what I mean is like guaranteed new in place

#

e.g. in debug mode

zenith lion
#

this is just enabling an LLVM optimization for this

#

there's no rust guarantee that the stack allocation will be removed

#

so your max allocation is still limited by your stack size kinda

#

you can overflow the stack when creating huge strings or similar

sacred spire
#

Does it just not put string contents on the heap?

zenith lion
#

it does but heap allocations are first on the stack

#

e.g. take Box as an example

#

Box::new(123) will stack allocate 123 then memcpy it to the heap

#

LLVM can and will optimise that out but in debug builds and stuff it might not

sacred spire
zenith lion
#

so if you have a huge boxed array for example

zenith lion
#

new() is a function so it's arguments are created as they normally would

#

so 123 is created as it normally would be

#

then passed

#

tho 123 will likely be assigned a register so imagine something more complicated

sacred spire
zenith lion
#

nope it takes ownership of the value it can't take references

#

the solution to this problem is to guarantee that temporaries passed in this way are initialised in place

#

so guaranteeing the optimisation

#

and sometimes you want to do it manually as well which is new-in-place in C++

#

new (ptr) Foo();

#

rust doesn't have an equivalent

#

it used to have box expressions as an equivalent but it was removed

sacred spire
#

That's a shame

#

Is there an RFC or smth tracking this?

zenith lion
#

the rust "equivalent" is *rawptr = whatever;

zenith lion
#

and there's lots of bikeshedding

sacred spire
zenith lion
#

it's basically copy elision but you're eliding a stack allocation

#

getting rid of the temporary

#

bc the temporary can sometimes not fit in the stack

#

e.g. it's hard to create huge boxed arrays

#

not impossible mind you

untold basin
#

Box::new is a language item

#

and the impl is literally fn new(x: T) -> Box<T> { Box::new(x) }

zenith lion
#

I thought the impl was // compiler implemented or something and it was #[rustc_box]ed

untold basin
#

oh yeah maybe

#

nvm

#

but its the same thing

#

but its inline(always) anyway

tawdry knoll
#

Ahh I see. I guess you always have the fallback option of allocating a zeroed box and then repeatedly moving small amounts of data into it after it's already constructed. I don't remember ever encouraging a stack issue however, I do agree that it should just behave like that by default.

tawdry knoll
#

The thing in my eyes is that, yes -- Rust has a lot of growing pains (especially around osdev) and is kinda difficult to use when navigating around them. However, when it works it works really well, and for most things (that I care about at least) are actively getting worked on. In the grand scheme of things, Rust is still a baby language. I mean yeah obviously you have Zig and others that might not have some of the same annoyances, but I find that Rust's rules are much more complex than Zig. I cannot go back to not having proc_macros and not having fmt etc... lol

tawdry knoll
# untold basin i don't really know, tbh

Something you could look into is using the defmt crate. Its a much lighter version of fmt which is quite large for low level stuff. I've seen fmt consume ~30k of executable size alone, and without it you should notice much smaller executables. I think its like this because dyn has quite a bit of executable complexity, and the default formatter is optimized for zero branching. Oh, and you should disable stack unwinding. You can still get stack traces, just that Rust cannot recover from a panic and unwind its stack like C++'s exception can. Its not impossible to get executable sizes really tiny, just a bit of work.

untold basin
#

i dont have panic-unwind anyway

tawdry knoll
#

Ahh okay good

untold basin
#

i have my own unwinding logic though, for stacktraces

#

and i build with frame pointers on

tawdry knoll
#

yeah thats fine, shouldn't contribute much to large executables

untold basin
#

the tables are actually pretty big tbf

#

but its not THAT much

#

i might look at defmt yeah

tawdry knoll
#

like are you including your debug symbols in your executable?

untold basin
#

no

#

i am taking dwarf tables

#

doing custom magic

#

then including the results of custom magic in the final executable

#

the tables are a lot smaller than dwarf

#

still quite big but not ruinously big

#

isn't defmt serialization-only?

#

which kinda sucks

tawdry knoll
#

ahh I was about to say that Rust's symbols are really large lol, I have seen ELFs >2Mib for a <100Kib final executable

#

Anyway, hopefully you end up figuring out something that works. Good luck on the project!

#

Oh and one last thing, if you haven't already, you can make custom compile profiles in your Cargo.toml file to enable some optimizations even in debug mode. Because in debug mode Rust will stub out every BinOp to a function to ensure it doesn't overflow, this gets optimized out in Release mode. You can disable this check without enabling other optimizations, so it's still easier to debug

untold basin
#

yeah i know

#

i have it at opt-level=1

tawdry knoll
#

But do you have overflow-checks=false

untold basin
#

no

#

i want overflow checks though

#

they are useful for debugging

tawdry knoll
#

Yeah unfortunately they are pretty big, and I also leave them on were I can... Idk I wouldn't stress about debug's executable size too much unless it's causing tons of issues.

untold basin
#

even the release ones are massive though

#

which kinda is a problem

zenith lion
#

idk if you already do all these

untold basin
#

i dont have location details off

tawdry knoll
untold basin
#

and fmt::Debug

#

because they both break shit quite aggressively

zenith lion
#

true but

#

why rely on debug by default??

untold basin
#

i dont rely on it

#

other than like

#

actually maybe i dont at all

zenith lion
#

for release build where you need no debug

#

it makes perfect sense

untold basin
#

ehh

#

not really

untold basin
#

@vapid echo please tell me more

vapid echo
#

idk if you remember the library with rust-like functions

untold basin
#

sounds interesting

vapid echo
#

the one with the hamt and stuff

#

that

untold basin
#

i see

#

.hpp tho

vapid echo
untold basin
#

oh also you need the std header crap

vapid echo
#

yeah

untold basin
#

i dont have it

vapid echo
#

so add it meme

untold basin
#

why?

#

i have c++20 vomit instead

vapid echo
#

freestanding headers good

untold basin
#

including these thingies

untold basin
vapid echo
#

concepts thingies

#

and standard types

untold basin
vapid echo
#

yes

untold basin
vapid echo
#

the only thing you really need is like

#

<concepts>

#

I think

untold basin
#

i have my own thing

#

that i use

#

idk

vapid echo
#

whatever idrc I just thought it might be cool to use

#

    auto v = arr.iter().collect<Vec<int>>();
#

literally rust

untold basin
#

that looks cool

#

ill think about it

vapid echo
#

idk if u were thinking about using frigg

#

but it's basically it but better meme

untold basin
#

i was not

#

your thing definitely looks better than frigg

#

idk about the names but like

#

the rest is fine

#

also i dont like std

vapid echo
untold basin
#

like why have StringView

#

etc

#

tbh personally i prefer just using Span<const u8> for strings if possible

vapid echo
#

String is an allocated string and StringView is well, a string view

#

lol

untold basin
#

but yeah whatever it doesnt matter

vapid echo
untold basin
#

true

#

idk

#

ill think about itโ„ข๏ธ

vapid echo
#

ah I may have forgotten

#

I have it in slices though

#

but yea whether you end up using it or not, I'll probably look into contributing to your thing cuz it would be cool as fuck to boot my OS using a custom firmware

untold basin
#

trueee

untold basin
#

okay i think im gonna have to do bootloader stuff for shkwve now

zenith lion
zenith lion
#

make it rust FFI-able

#

it would probably need a fork of the rust stdlib tho

#

because rust does not have a standard struct layout in memory

#

you'd need to make rust stuff #[repr(C)]

#

but it would be cool to be able to pass any rust stdlib type (that your thing also defines in C++) over an ffi boundary

untold basin
#

im doing it all in C++ now

#

fuck rust :^)

zenith lion
#

DON'T FUCK RUST

#

YOU MIGHT GET TETHANOSIS

untold basin
#

looks like im targetting c++2c then

#

ugh "const unsigned char" is not """similar""" to "const char"

#

i have -funsigned-char too lol

#

so its like. literally the same type

untold basin
#

okay so lets get shkwve++ booting

lofty hearth
untold basin
#

it has the same size, alignment, signdness, layout

lofty hearth
#

std::is_same<unsigned char, char> || std::is_same<signed char, char> is always false

untold basin
#

yeah i know i know

untold basin
untold basin
#

okay now i have a custom "span" helper type in my C++ template goop

#

okay wtf

#

how do i make my initializer_list not weird

devout swift
#

those are the only variants of it that are supported

untold basin
#

yeah i figured

#

and the begin pointer must be const T*

#

lol

#

now i have a Vec<T>

vapid echo
#

atlas::Vec<T> though ๐Ÿ˜ญ

#

nah its ok I forgive you

untold basin
vapid echo
#

what I did for my fmt thing is I have a Formatter<T> class that types can extend

#

so you can do like

#

struct Formatter<Vec<T>> {}

#

then do a display function

untold basin
#

thats cool but consider this

#

i don't have one

#

:^)

#

i probably could add one tbh

untold basin
#

ah thats ugly too

#

i just have global state

#

for printing

vapid echo
#

cringe

untold basin
#

its a bootloader

#

i do not need nonglobal printing anyway

untold basin
vapid echo
#

well in my kernel

#

I just have a global sink

untold basin
#

ah yeah that makes sense

#

tbh

#

passing around a sink doesnt produce good codegen tho

vapid echo
#

I checked

#

and it wasnt that bad

#

a hello world was roughly the same size as frigg (or smaller? I forgor)

untold basin
#

ok makes sense

#

okay i have SmallVec

#

okay now im gonna add a refcounted pointer i think

#

okay now i have Box and Ref

#

now i will create a String type

#

a totally unique idea i did not steal from anyone else!

vapid echo
#

I called my Ref<> Rc<> meme

untold basin
#

okay im gonna use a Formatter type

#

its definitely a better approach

#

tho it has some disadvantages, especially around overload resolution

#

but its probably a better approach

vapid echo
#

literal plagiarism

untold basin
#

noooo

#

im not writing to a format buffer

#

also my impl puts __thunk on every function which is why its totally unique!

#

#define __thunk [[gnu::always_inline]] [[gnu::artificial]] inline

vapid echo
#

I just use inline

#

I let the compiler inline stuff if it deems it fit

untold basin
#

yeah probably a good idea

#

i should stop putting it on like everything lol

#

i now have this distinction too

#

which prints as expected

glad bough
#

hi

#

what are you doing

untold basin
#

oh and other things like this

untold basin
glad bough
#

what rewrite

untold basin
#

yeah im rewriting shkwve!

vapid echo
untold basin
#

in c++

glad bough
#

๐Ÿ’€

#

bro

#

at least don't use go for building then

untold basin
#

oh yeah good point

#

ill still keep the go stuff

#

ill just make it generate ninja for depfile handling

#

because having symbolized stacktraces is like. a cool thing

vapid echo
#

I think my way is cleaner

#

its very strange tho

untold basin
#

what is?

#

my old shit?

vapid echo
#

my way

untold basin
#

ah yeah i see

#

nah i like my impl

vapid echo
#

bunch of decltype stuff

untold basin
#

oh you mean for collect?

#

or like in general

#

tbh i like the iterator vs cursor distinction

#

but yeah anyway maybe i should make it run on bare metal

vapid echo
#

in general

#
 return Iterator<decltype(next_func)>(next_func);
#

this kind of stuff

untold basin
#

im kinda out of things to implement

zenith lion
#

does your formatter support {:x} and stuff

#

i did that for mine in C++

#

it was tough

untold basin
#

also i produce better codegen than rust does

zenith lion
#

because i was using very recent C++ features I was just learning

zenith lion
#

example?

#

for the formatter?

untold basin
#

i dont do loop stuff

#

or like slice stuff

#

uh

#

rust produces really bad codegen for println! lol

zenith lion
#

since when is my tabs :D

untold basin
#

it means >100

#

lol

#

my codegen looks more like this

zenith lion
#

it was โ™พ๏ธ before

untold basin
#

the Formatter<Span<int32_t>>::debug stuff has a bunch of logic etc

#

(putchar is temporary its gonna turn into real debug io stuff later)

#

but its all fully monomorphised

#

and i parse format strings at compile time

untold basin
#

@vapid echo lmao atlas doesnt even pass its own test suite

#

oh and now i managed to induce a crash

#
  TEST_CASE("collide") {
    struct FakeHash {
      uint64_t operator()(uint64_t a, size_t gen = 0) const {
        (void)a;
        return gen;
      }
    };

    Hamt<uint64_t, uint64_t, TracingAllocator, FakeHash> h;
    h.insert(1, 1);
    h.insert(2, 2);
    h.insert(3, 3);
    h.insert(4, 4);
    h.remove(1);
    h.dump();
    if (Option<uint64_t> v = h.get(4)) {
      CHECK(v.unwrap() == 4);
    }
  }
#

you should prooobably handle this kind of endless hash collision

#
    struct FakeHash {
      uint64_t operator()(uint64_t a, size_t gen = 0) const {
        return gen ? a : 0;
      }
    };

    Hamt<uint64_t, uint64_t, TracingAllocator, FakeHash> h;
    h.insert(1, 1);
    h.insert(2, 2);
    h.insert(3, 3);
    h.insert(4, 4);
    h.remove(1);
    if (Option<uint64_t> v = h.get(4)) {
      CHECK(v.unwrap() == 4);
    } else {
      CHECK(false);
    }
#

and this fails in the CHECK(false)

#

i wanted to see how you dealt with these kinds of collisions and the answer is that you do not KEKW

#

tbh i might externally allocate entries

#

its less efficient

#

but its easier too so

#

actually fuck it i dont need a hashmap for now

#

or any map

#

i might port frigg's rbtree

#

or implement an AA tree or something

#

idk

#

okay it is time to make it do os stuff

vapid echo
#

All passes for me

#

You probably need to run it in the proper directory

#

I read an elf file

untold basin
#

sounds like a broken test suite

#

but anyway i foud two different breakages with your HAMT too so

#

lol

vapid echo
#

well

#

Those issues are the hash's fault

#

Not mine

untold basin
#

the first one yes

#

the second one no

#

the second issue is your 100% fault

untold basin
#

yes

#

its a valid hash

#

this is the case if you get a first-level hash collision

#

which can happen

vapid echo
#

I did extensively test collisions

untold basin
#

your testing was clearly crap

#

since it took me 5 minutes to break it

vapid echo
#

bro I tried like a 4mb word dictionary

#

There were collisions

#

And it handled them

untold basin
#

idk man

#

my tests fails

#

and it very much shouldnt

vapid echo
#

I'll see what I can do

#

But it's weird considering I did test collisions like I said

untold basin
#

its called testing using brain instead of using brute force meme

#

also i almost got my shit build!

#

:^)

#

(almost)

#

progress

#

it links

vapid echo
untold basin
vapid echo
#

yea so to run tests you need to call ./build/tests directly

#

ninja -C build test wont work

untold basin
#

i did ./tests

vapid echo
#

because of the elf thing I told you

untold basin
#

lol

#

okay yeah whatever

#

same thing

#

i commented out that test and it worked

#

anyway time to fix arm bullshit

#

hmm weird it should work fine

#

wtf

vapid echo
#

ok i fixed it

#

this seemed to be the problem

#

just removing that condition fixes it

untold basin
#

esr suggests L3 translation fault

#

the ptes have bits 0x403 except for the last level which is 0x401

vapid echo
untold basin
#

403 is ACCESS|VALID|TABLE

vapid echo
#

I tested with the huge word dictionary

untold basin
#

401 is ACCESS|VALID

vapid echo
#

and it doesnt get hit once

untold basin
#

does my test pass now?

vapid echo
#

yeah

untold basin
#

cool

lofty hearth
untold basin
#

VALID is 0

lofty hearth
#

or bit 1

#
inline constexpr uint64_t kPageValid = 1;
inline constexpr uint64_t kPageTable = (1 << 1);
inline constexpr uint64_t kPageL3Page = (1 << 1);
#

from mammogram

untold basin
#

huhhh

lofty hearth
#

i think the only case bit 1 is unset in a valid entry is 2M/1G/512G blocks

untold basin
#

okay thanks!

#

yeah that would make sense i guess

vapid echo
#

I should probably make the elf test suite better

#

so it doesnt not work when calling ninja -C build test

lofty hearth
#

also page tables don't need ACCESS set

#

iirc those bits are just ignored for non-leaf entries

untold basin
#

i see

#

still nope tho

#

okay yeah now its crashing with FAR=0x18

#

and an L1 translation abort

vapid echo
#

thanks for the bug report btw i wouldnt have caught it

untold basin
#

okay next problem

#

i need to map peripherals

#

okay i have boot0 working (tho with a massive hack)

#

okay now the hack is gone too

untold basin
#

boot1!

#

the bad news is that weird things are happening

#

like it shouldnt reach MISC_abort_slow

#

ever

untold basin
#

yeah it works at -O0

#

and i get a garbage elf at -O1

#

wtf

#

oh

#

oh god

#

i hate LLVM's "lets just optimize out random """empty""" loops" pass

#

it optimized out my panic code

#

oh!

#

its a clang bug

#

fixed in 19

#

fedora only packages 18

#

grumble

#

and then i get to an allocator failure which is expected

#

cool

quartz storm
untold basin
#

the standard is stupid here

#

newer clang fixes it

#

C mode fixes it too

quartz storm
#

afaik for(;;) { /* some code with no 'break' */ } cannot be removed

#

fear the Big Bad Optimizing Compiler

untold basin
#

by clang++ versions 18 and older, yes

#

also thats not true

#

for (;;) {} cannot be removed

#

for (;;) { /* any statement */ } can

quartz storm
#

๐Ÿ’€

lofty hearth
#

not that i expect the compiler to actually change optimizations based on -std=c++XY but i'm curious

devout swift
#

and clang was basically the only compiler that optimized it so aggressively

untold basin
#

lmao

lofty hearth
#

fair enough

#

can't wait for c++2d

untold basin
#

lol

lofty hearth
#

hmm there is no way to fit more than 4 standards into a decade right?

#

hmm no ... 20 23 26 29 32 35 38 41 44 47 50 ...

lofty hearth
#

shhhh

untold basin
#

the max you can do is obviously 00, 03, 06, 09

#

although

#

09 12 15 18 21 would work if they were really slow for 09 and really fast for 21

lofty hearth
#

i was more interested in calendar decades, so that you get c++X{a,b,c,d,e} or whatever

#

but that obviously can't happen

untold basin
#

ahh ahh

untold basin
#

handoff

#

memory map from C++ (boot1)

#

and from C (boot0)

#

the 0x980 LoaderImage is zeros for padding

#

okay now it even runs with TCR_EL1.EPD0=1

#

it crashes on no malloc atm though

#

and its ugly as hell

untold basin
#

okay i think its time to write a shitty malloc

untold basin
#

and by "write" i mean "steal"

#

and ill probably have to write a new one because im stupid

#

but its fine

#

okay so now that i have a stolen completely novel memory allocator, what's next

#

@agile nexus i hit that incompatible uintmax_t type thing too btw

#

also. you should stop vomiting global state into the public symbol namespace with the state global

agile nexus
untold basin
#

tbh i forgot where because i just typedefed them together lol

#

also the global is the only way of getting the fdt root

#

other than path stuff ofc

agile nexus
untold basin
#

okay heres my new device subsystem tree thingy

quartz storm
untold basin
#

its test stuff

#

yes it will be used for things

#

when i do more impl shit

#

laterrrr

quartz storm
# untold basin its a link

but like... what is the point of nextLetter? if this is part of the filesystem hierarchy (like devfs or sysfs), code can just iterate over readdir

untold basin
#

lol

#

like its a symlink type of thing

quartz storm
#

yeah

#

but what does it permit software to do that it cannot without it?

#

also, assuming a usual VFS, is it not better to have the symlink contents be ../b instead of /b? the latter would lookup b in the filesystem root

#

or maybe it is implemented as a 'magic symlink' and it does not matter, idk.

untold basin
untold basin
#

like for modelling relations between devices

#

or whatever

quartz storm
#

but is there any meaningful relationship between two "adjacent" devices on a bus, other than belonging to the same bus?

untold basin
#

the "nextLetter" entry is just a test thingy

#

to make sure that this mechanism works correctly

quartz storm
#

ah

#

ok

untold basin
#

okay im back from implementing a shitty raytracer

#

and i decided to redesign the device system yet again

#
// idl, defining a kind of device
class Foo
    poke(u8 thingy) -> void

// a thing that is this kind of device
struct AFoo : Device(Foo) {
    void poke(u8 thingy) { print("poked: {}", thingy); }
};

// user API
Dev<AFoo> root = Dev<AFoo>::make(new AFoo{});
root->poke(0x41_u8);

Dev<Foo> lower = root.downcast<Foo>();
lower->poke(0x42_u8);
#

basically its rust traits except in C++

open osprey
#

don't forget to insert a STREAMS implementation as well

untold basin
#

i'm definitely planning to add a network stack

untold basin
#

now i have an event thingy as well: ```c++
// subscribtions
static void someHandler(evt::FooRelatedEvent* relev) {
print("handle a foo-related event");
}
static void someHandler2(evt::FooLocation* loc) {
print("handle a foo-located event");
}
EventHandler(someHandler);
EventHandler(someHandler2);

// creation
evt::FooLocation::dispatch(true, false);
evt::FooRelatedEvent::dispatch(true);

// idl
event FooRelatedEvent
bool isRealEvent

event FooLocation : FooRelatedEvent
bool requestCreation

#

with some cheap type casting logic which handles the simple single-inheritance case

#

(i form a tree of the events, then do a dfs over all of the events and assign IDs as i go, so typecheck is just min <= id && id < max)

#

my plan is to allow for events to be "claimed" by device drivers as my mechanism for creating new devices

vapid echo
#

concepts

untold basin
#

no not concepts

untold basin
vapid echo
#

Traits are better tho

untold basin
#

okay i think i'll implement STREAMS now

#

maybe

#

for uart

#

less priorities, because i do not need them

#

ig

#

and also other things probably, because looking things up is hard too

untold basin
#

okay i have probing going again

#

still need to do clk and pll torture

#

and implement a gic + pl011 driver

untold basin
#
#include <fdt.hh>
#include <irq.hh>
#include <option.hh>

// generic protocols
class InterruptController
    resolveFDTInterrupt(fdt::Cells cells) -> Option<u32>
    bindIRQ(u32 id, InterruptSlot* slot) -> void

// arm64-specific devices
#if arm64
class ARM64PlatformIC
    handleIRQ()
    handleFIQ()
#endif

// devices
event GICv2Found
    fdt::Node fdtNode

event FDTNode
    fdt::Node fdtNode
    // TODO: also add vmspace when i add that as an abstraction

``` it has a bunch of new maximally cursed features now
#

like #if and #include support

#

which get lowered to C++ as needed

zenith lion
untold basin
zenith lion
#

well it could be

untold basin
#

first of all, not really

#

second of all, why would i do that

zenith lion
#

if you make your idl lowered to headers you could include idl and headers

untold basin
#

those definitions arent compatible with my idl parser

zenith lion
#

ah ok

untold basin
#

okay i implemented a very shitty pl011 "driver"

untold basin
#

okay now i need to implement mapping pages

#

and a better allocator

untold basin
#

totally different from mangarm's

#
[[gnu::artificial]] [[gnu::always_inline]] static inline void memstore(u8*& dst, word value) {
    __builtin_memcpy_inline(dst, &value, wordSize);
    dst += wordSize;
}
[[gnu::artificial]] [[gnu::always_inline]] static inline word memload(const u8*& src) {
    word dst;
    __builtin_memcpy_inline(&dst, src, wordSize);
    src += wordSize;
    return dst;
}

but hey i have this impl which is a lot nicer than the weird enum tricks managarm uses

#

also i mess around with the impl to induce ldp/stp on arm

glad bough
#

real

#

what is that white line on that sourcehut link

#

a ruler?

untold basin
#

aka stupid line

#

yes its super distracting

glad bough
#

cant you turn it off?

untold basin
untold basin
glad bough
#

lol

untold basin
#

okay i think its time to acutally write a reasonable memory allocator

untold basin
#

okay i have my shitty malloc now

#

and i have page table logic working i think

#

kinda

#

still need to improve it a bit

untold basin
#

okay i have mapping pages down i think

untold basin
#

i think i have a bit too many debug assertions

#

also wtf is clang's optimizer doing

#

ahh yes, now that is true clang

#

if (x) { if (x) { do stuff(); call a noreturn function() } if (x) { call a noreturn function() } call a noreturn function() }

#

does clang just not do noreturn analysis at -O1?

untold basin
untold basin
#

tbh it would be cool if c++ had something that worked like #[track_caller]

#

and on one hand i could write a clang plugin to add like, [[track_caller]] or w/e

#

but on the other hand

#

that sounds really fucking hard

#

so i wont

#

clangir would make this way easier tho

#

or maybe not idk

#

actually it probably would yeah

#

unfortunately its also really fucking broken

untold basin
#

okay now i init my pl011+gic driver more better

#

and map pages

#

next step

#

interrupts

#

ugh

#

in newer qemu i get alignment faults

#

in the pre-vm logic

untold basin
#

basically assertboot meme

#

okay i routed an irq to the pl011 driver!

#

okay now im handling the dpc

untold basin
#

okay now i have serial

#

with interrupts!

#

time for shitty STREAMS

#

or maybe i wont

#

hm

#

okay lets start by entering the hell that is C++20 coroutines

untold basin
#

one thing i have to give to rust's async system props for

#

it is way easier to understand

untold basin
#

hmm i think i have a race condition somewhere

#

and now i'm reading from a different coroutine via an MPMC

untold basin
#

my new set of async primitives is basically just spamming out semaphores tbh

#

i should switch to a proper round robin style scheduler

trim seal
untold basin
#

and also theres a bunch of other things that go into my interrupt handling because (a) arm moment and (b) pitust shitcode

#

also also i initialize the idt as the second instruction in the proper loader ("boot1")

trim seal
#

boring, they should've called it IDT_EL1

untold basin
#

ok sorry 5th

#

the first thing is setting up the stack

#

and handoff info

#

the third thing is vbar_el1

#

so things i have to do:

  • a real round robin scheduler
  • actual drivers
  • idk all the other thinsg
#

okay i think ill do more drivers

#

hmm. maybe i can do clk and pll torture

untold basin
#

or device selection or something idk

#

okay lets actually do something useful

#

like an fwcfg driver

random elbow
#

I like the fact it's written in Go that's pretty cool!

untold basin
#

its not written in go

#

lol

#

the go is only the build system

#

the build entrypoint is basically ```sh
#!/bin/sh

set -e

mkdir -p obj

fence_rev=r22
if ! [ -e obj/x ] || ! [ -e obj/fence-buildsrc-$fence_rev ]; then
rm -rf obj
mkdir -p obj

go get ./build-src
go build -o obj/x ./build-src
touch obj/fence-buildsrc-$fence_rev
fi

obj/x "$@"```

#

obj/x is just the main builder program

rose sonnet
untold basin
#

"too many" just means that i cant show off my cool status code system

#

:^)

untold basin
#

okay that's fwcfg file listing working

#

huh i wonder what the tpm log is

#

sounds interesting but also really boring

untold basin
#

okay i could do ramfb

#

or i could write a shitty virtio driver

#

actually ig pci first