#Retro Rocket OS - A BASIC powered Operating System (started 2009!)
1 messages · Page 4 of 1
probably on forums.osdev.org
ah ok
i do still post there every so often
but nobody ever reples to any of the osdev pictures thread lol
i used to be a regular, but quit when the spam control went overboard
i used to try to respond to the pictures thread, but i could only really appreciate the graphical ones which people said weren't so good under the surface, so i didn't know what to say lol
yeah good is subjective
many people dont like retro rocket
because it doesnt do the standard memory protected rings and stuff system
with binary programs and preemptive intrrupt driven tasks
yeah, i'm not looking forward to dealing with those people
for one thing, i had a good think about cooperative multitasking and don't see too much of a problem with it.
i do <3
glad you do! 🙂 i think pretty much everyone here in this thread does
omg theres a fucking riot outside my house
not joking, at least 6 drunk and very angry idiots having a proper barney outside on the street, cop cars pulled up blues and twos blaring
the joys of living in town centre
its silly because we live literally across the street from the town police station and council office
omg i think its a mugging
"get off your fucking bike"
brb
my road's nice in cold weather, but when it gets warm enough for drunks to walk home, there's sometimes a bit of trouble
i think they arrested someone
the cops were yelling at someone to get off their bike, had him pinned down and he was shouting abuse calling them pigs etc
probably was up to no good in the cop shop car park
like 6 cops strode out the back door torches on to come deal with him
if drunks are drunk enough, its always warm enough to walk home

ok back to improving randomness in retro rocket!
csprng for tcp isns here we come!
new isn generator 😄
uint32_t get_isn(uint32_t local_addr, uint32_t remote_addr, uint16_t local_port, uint16_t remote_port) {
if (!isn_init) {
uint64_t seeds[2];
if (!csprng_fill(seeds, sizeof(seeds))) {
preboot_fail("Unable to initialise TCP ISN generator");
}
isn_hash_seed0 = seeds[0];
isn_hash_seed1 = seeds[1];
isn_tick_base = get_ticks();
isn_init = 1;
}
uint64_t tuple[2] = {
((uint64_t)local_addr << 32) | (uint64_t)remote_addr,
((uint64_t)local_port << 48) | ((uint64_t)remote_port << 32),
};
uint32_t delta = (get_ticks() - isn_tick_base) * 250; /* 250k/sec */
uint32_t base = (uint32_t)hashmap_sip(tuple, sizeof(tuple), isn_hash_seed0, isn_hash_seed1);
return base + delta;
}
decided to move the isn_init part into the tcp_init function
a lot cleaner now
void tcp_init() {
uint64_t seeds[2];
tcb = hashmap_new(sizeof(tcp_conn_t), 0, 6, 28, tcp_conn_hash, tcp_conn_compare, NULL, NULL);
tls_fd_table_init(FD_MAX);
if (!csprng_fill(seeds, sizeof(seeds))) {
preboot_fail("Unable to initialise TCP ISN generator");
}
isn_hash_seed0 = seeds[0];
isn_hash_seed1 = seeds[1];
isn_tick_base = get_ticks();
proc_register_idle(tcp_idle, IDLE_BACKGROUND, 1);
}
uint32_t get_isn(uint32_t local_addr, uint32_t remote_addr, uint16_t local_port, uint16_t remote_port) {
uint64_t tuple[2] = {
((uint64_t)local_addr << 32) | (uint64_t)remote_addr,
((uint64_t)local_port << 48) | ((uint64_t)remote_port << 32),
};
uint32_t delta = (get_ticks() - isn_tick_base) * 250; /* 250k/sec */
uint32_t base = (uint32_t)hashmap_sip(tuple, sizeof(tuple), isn_hash_seed0, isn_hash_seed1);
return base + delta;
}
nah, too much work
it would need a completely different expression parser, and most of the features of retro rocket wouldnt work in it, plus youd have to always have line numbers and BBC BASIC programs are stored tokenized
and BBC BASIC has weird numeric formats
40 bit 5-byte floating point?
plus, it would have to support BBC graphics, that is 7 graphics modes, some of which arent on a PC, and would need me to do mode switch
added support for TCP retransmission queues! This means the TCP implementation is much more realistic now, and doesnt just assume things get sent
also added proper random selection of dns request ids, and validation that dns replies come from the same ip that they were requested from
adding a telnet client!
telehack has some weird, old stuff
allowed for INPUT and SOCKREAD to accept an array subscript or whole-array input specification, this was left out for some reason a couple of years ago when i added arrays
also added the old type-in demo program MASTERMIND often seen in magzines. i used the one i found in telehack as a basis, only needed to change a couple of lines and remove GOTOs to modernise it and it ... just worked!
Fixed a bug that i introduced at some point where if you did A$ = @ it would get stuck in an infinite loop
i heard u leik teh BASIC so i did BASIC in ur BASIC 
added SOUND FADE streamhandle, milliseconds, SOUND REPEAT [ON|OFF] soundhandle
also improved performance of the interpeter again by a huge margin, watch how fast it rotates 😄
hi :) coming from the details you mentioned in #showing-off (on keyword parsing), perfect hashing (perfect hashing (https://en.wikipedia.org/wiki/Perfect_hash_function) is probably a good way to go further with this
In computer science, a perfect hash function h for a set S is a hash function that maps distinct elements in S to a set of m integers, with no collisions. In mathematical terms, it is an injective function.
Perfect hash functions may be used to implement a lookup table with constant worst-case access time. A perfect hash function can, as any has...
you could probably get away with some mixing of first few letters and length to generate lookup index in a pretty small lookup table
already went further!
GENERATE_ENUM_STRING_NAMES(TOKEN, token_names)
GENERATE_ENUM_STRING_LENGTHS(TOKEN, token_name_lengths)
uint16_t key = prefix16(ctx->ptr);
int start = keyword_prefix_offsets[key];
int end = keyword_prefix_offsets[key + 1];
for (int kt = start; kt < end; ++kt) {
int tok = keywords[kt];
size_t len = token_name_lengths[tok];
/* First two characters already matched by prefix16 bucket.
* Compare from byte 2 onwards; <=2 length is already a full match.
* Ordering is preserved within the bucket, so early-exit still works.
*/
int comparison = len > 2 ? strncmp(ctx->ptr + 2, token_names[tok] + 2, len - 2) : 0;
if (comparison == 0) {
const char *backup = ctx->nextptr;
ctx->nextptr = ctx->ptr + len;
bool next_is_varlike = (*ctx->nextptr == '_' || isalnum(*ctx->nextptr));
if (!next_is_varlike || keywords[kt] == PROC || keywords[kt] == FN || keywords[kt] == EQUALS) {
/* Only return the token if what follows the token is not continuation of a variable-name or keyword-name like sequence, e.g. "END -> ENDING"
* Special case for PROC, FN, =, as PROC and FN can be immediately followed by the name of their subroutine. e.g. PROCfoo
*/
return tok;
} else {
ctx->nextptr = backup;
}
} else if (comparison < 0) {
/* We depend upon keyword_tokens being alphabetically sorted,
* so that we can bail early if we go too far down the list
* and still haven't found the keyword.
*/
break;
}
}
im using a 16 bit bucket now, and able to bail early if the token is <= 2 chars long, e.g. IF, TO, ON etc
and this allows me to cut the first two chars off the strncmp
meaning that every lookup is now constant time O(1), buckets only on average contain 1.7 keywords (mean)
[288]: Building keyword buckets...
[289]: Keyword bucket summary:
[290]: keywords : 121
[290]: buckets used : 69 / 65536
[291]: empty buckets : 65467
[292]: avg bucket size : 1.753
[293]: max bucket size : 7 (key=5245 'RE')
thanks for the docs though, definitely going to read them 😄
also if youre curious:
/**
* @brief Return a 16-bit lexicographic prefix for a string.
*
* Encodes the first two characters as (c0 << 8) | c1 so that the numeric
* ordering of the key matches the alphabetical ordering of the keyword table.
*
* Assumptions:
* - Strings are at least one character long (second byte may be '\0')
* - Direct byte access is safe (x86-64, no alignment restrictions of concern)
* - Endianness is known and controlled (explicit shift/or avoids host order)
*
* Portability is not a goal; this is tuned specifically for x86-64.
*/
static inline __attribute__((always_inline)) uint16_t prefix16(const char *p)
{
return ((uint16_t)(unsigned char)p[0] << 8) | (uint16_t)(unsigned char)p[1];
}
thats the prefix16 function, its really simple intentionally
the neat part of perfect hashing is that you don't get any collisions without having an enormous table (2^16 slots for this feels unreasonably large)
it takes some brute-forcing to find a good hash function for that though
here you could use first three letters and just take their lowest 5 bits (which uniquely identify uppercase letters) and use that to index array of size 2^15...
or try some random shifts/xors/adds/muls until you get some small table (for e|ample < 256 slots) without collisions
yes, but "perfect" is purely academic
this is as close to perfect as I need to get rationally, before complexity and maintenance tuning of a hash function start taking all my time for something that is now incredibly quick
you'd need to get at least the first 6 letters for a truly perfect hash, if not more, because of a bunch of keywords that start SPRITE
2^16 bits for buckets is tiny tbh. it's like 128k, which is once, for the whole system. compared to each instances gc and heaps and stuff, a running basic program can go into megabytes easily, this isn't for tiny machines and 128k isn't going to break the bank.
bearing in mind all basic programs share that hash bucket table, it isn't a 128k table per program
ive moved builtin int/string/real functions into hashmaps. this will also greatly improve performance. now looking at caching common strings too
then you can use like just 1st, 3rd and 8th letter...
and although it takes some constant bruteforcing, perfect hashing is for example used in chess engines (magic bitboards) for architectures without bit extract instruction
but yes, it is a bit of added complexity (not much tho), you just seemed to be interested for better techniques for this problem :)
now made a separate hash store for symbolic constants e.g. variable names so they don't pollute the GC. they don't need to be collected.
stonking along so fast the gif is a blur and there are 4 or 5 ghosted copies of the cube per 60hz refresh interval
I really need to make a BASIC profiler
tried it out (it was fun :) ), and it doesn't work as well as I hoped, but probably could work ok-ish
with your keywords this seems to pretty stably (over seeds) generate 10bit/11bit lookup tables for hash function from word length + 8 lookups+shifts (doesn't seem to work with 7 :( ) in about half a second on my 3rd gen i5 with gcc and O3
perhaps, but I go for readability over perfection and it's fast enough.
also retro rocket always runs in -O0
my philosophy is, i can't hide behind the compilers optimisations if I pick bad algorithms
and it makes it very easy to debug, debugability and ease of understanding are very important to this OS
do I dare run the kernel profiler lol... that thing bogs down the system real bad
yeah, probably better here :)
oops :D, I forgot to check with a sanitizer :D
seems to work well with 7 or 6 operations -> 12/11-bit tables and takes about 8s for me with O0 (and it's about 80 lines which isn't that much)
lmfao the churn, the chuuuurn
0.8fps for one of my simpler programs, and list running dog slow... thats profiler build for ya lol
sooo worth it though
how did you do this, dump over serial?
yes
the profile data is built up in linked lists and arrays then a special key combo attached to the keyboard interrupt directly dumps out the callgrind.out over serial
then qemu can save it
I see, very nice
Added support for UDF optical media, and improved the github readme 😄
adding support for mounting and reading ADFS images as first-class data format!
@light ice let me know if theres anything i can do to help you with trying to diagnose the fred stuff 🙂
i'll likely get my own copy built some point soon, not tonight its nearly midnight here
I lowkey have zero clue
💀
@charred rampart is the one who made the clanker patches
I just know the FRED spec mostly
Maybe add a command to dump the full CPUID to port E9 in a JSON format?
So I can try and see what the fuck is going on
uhh the run.sh does nothing

qemu-fred-system-x86_64 -machine q35,accel=tcg -s -monitor stdio -cpu max,fred=on,lkgs=on -m 4G -drive id=disk,file=../../harddisk0,format=raw,if=none -device ahci,id=ahci -device ide-hd,drive=disk,bus=ahci.0 -drive file=rr.iso,media=cdrom,if=none,id=sata-cdrom -device ide-cd,drive=sata-cdrom,bus=ahci.1 -boot d -vnc 0.0.0.0:10 -debugcon file:debug.log -netdev user,id=netuser,hostfwd=tcp::2000-:2000,hostfwd=tcp::2080-:80 -object filter-dump,id=dump,netdev=netuser,file=dump.dat -device e1000,netdev=netuser
i added fred to it
and it wants a disk file that dosnt exist
and if i remove that stuff
it just opens the monitor
just take out -drive id=disk,file=../../harddisk0,format=raw,if=none
or create that drive
that isnt mandatory
also take out the -vnc 0.0.0.0:10
if you want it to run on standard display not vnc
by default it runs on vnc
youll have to pull nigtly again anyway
the way i was defining the fred vectors was wrong
sure
yeah this was completely wrong
fred_entry_page:
/* FRED SPEC:
* ----------
* FRED event delivery uses two entry points, depending on the CPL at the time the event
* occurred. This allows an event handler to identify the appropriate return instruction
* (ERETU or ERETS).
* Specifically, the new RIP value that FRED event delivery establishes is
* IA32_FRED_CONFIG & ~FFFH for events that occur in ring 3 and
* (IA32_FRED_CONFIG & ~FFFH) + 256 for events that occur in ring 0.
*/
/* CPL3 entry (unused in this kernel) */
.quad fred_unused_entry
/* pad to exactly 256 bytes */
.fill 0x100 - 8, 1, 0
/* CPL0 entry (all kernel interrupts arrive here) */
.quad fred_common_entry
i fixed it to match the spec
basically i filled it like a vector table before (oof)
is this the new one?
isnt this proper?
same thing isnt it?
hmmm, but the spec doesnt say jmp
the spec says it loads rip with the addresses at those offsets
RIP is set to FRED_CONFIG
or + 256 for CPL0
which spec version did you read?
8.3.1.1 Determining the New RIP Value
FRED event delivery uses two entry points, depending on the CPL at the time the event occurred. This allows an
event handler to identify the appropriate return instruction (ERETU or ERETS).
Specifically, the new RIP value that FRED event delivery establishes is IA32_FRED_CONFIG & ~FFFH for events
that occur while CPL = 3 and (IA32_FRED_CONFIG & ~FFFH) + 256 for events that occur while CPL = 0.
If the new RIP value is not paging canonical, FRED event delivery causes a general-protection fault (#GP).1
I think this is pretty clear

gonna match mine to yours then
/* CPL3 entry (unused in this kernel) */
jmp fred_unused_entry
/* pad to exactly 256 bytes */
.fill 0x100 - (. - fred_entry_page), 1, 0
/* CPL0 entry (all kernel interrupts arrive here) */
jmp fred_common_entry
i hate GAS
give me 6502 any day
Atleast with a good assembly like NASM
nasm is nicer yeah
doesnt have a % obsesion
ok, so if you refetch nightly, it has that fix
well, we made some progress
what do you want to see from the cpuid etc
waitttttt......
im so dumb
What
i just wasted all our time, and i really appreciate you spending time on this
i'll show you the diff, and you can gladly eslap me for my dumbness
diff --git a/src/init.c b/src/init.c
index c2126757..4830fc93 100644
--- a/src/init.c
+++ b/src/init.c
@@ -7,7 +7,7 @@ spinlock_t debug_console_spinlock = 0;
init_func_t init_funcs[] = {
validate_limine_page_tables_and_gdt, init_heap, init_console,
- init_acpi, init_idt, boot_aps, init_pci, init_realtime_clock,
+ init_acpi, init_interrupts, boot_aps, init_pci, init_realtime_clock,
init_devicenames, init_keyboard, init_ide, init_ahci, init_nvme,
init_virtio_block, init_filesystem, init_devfs, init_iso9660, init_udf,
init_adfs, init_fat32, init_rfs, init_modules, network_up, audio_init,
@@ -16,7 +16,7 @@ init_func_t init_funcs[] = {
char* init_funcs_names[] = {
"gdt", "heap", "console", "acpi",
- "idt", "cpus", "pci", "clock",
+ "interrupts", "cpus", "pci", "clock",
"devicenames", "keyboard", "ide", "ahci",
"nvme", "virtio-block", "filesystem", "devfs",
"iso9660", "udf", "adfs", "fat32",
it was bypassing the version that even LOOKED for fred

atleast we found the other bugs lmfao
Hopefully you did the rest of the spec proper

i was bypassing the init function that checks for fred lol
Yup XD
and just going right to the idt version
should be good to try nightly now
it should detect fred... and it may asplode
What’s your code for demuktiplexing the vector etc?
tbh the only other bit i could get wrong is extracting the interrupt no from the fred struct
.type fred_common_entry, @function
fred_common_entry:
mov %rsp, %r8
MPUSHA
mov 0x28(%r8), %rsi
mov 0x08(%r8), %rdx
PUSH_SENTINEL
call Interrupt
POP_SENTINEL
MPOPA
ERETS
void fred_ring0_entry(fred_frame_t* frame) {
uint8_t vector = (frame->cpu_frame.ss >> 32) & 0xFF;
printf("GOT VECTOR 0x%02x VIA FRED!!!\n", vector);
dispatch_interrupt(&frame->regs, &frame->cpu_frame, vector);
}
this is how i do it
all in C
typedef struct irq_saved_regs {
uint64_t r15, r14, r13, r12, r11, r10, r9, r8;
uint64_t rbp, rdi, rsi, rdx, rcx, rbx, rax;
} irq_saved_regs_t;
typedef struct irq_cpu_frame {
uint64_t error, ip, cs, flags, rsp, ss;
} irq_cpu_frame_t;
typedef struct fred_frame {
irq_saved_regs_t regs;
irq_cpu_frame_t cpu_frame;
uint64_t fred_event_data;
uint64_t fred_reserved;
} fred_frame_t;
Make sure you check that before I test
im just using offsets from %r8 which should hold the cpu frame
I’m about to eat lunch
but i could be wrong
mov %rsp, %r8
MPUSHA
mov 0x28(%r8), %rsi
mov 0x08(%r8), %rdx
taken from the sp
yeah i just move it into r8 before i push stuff
if 0x28 and 0x08 is the right offset, sure
You could also do mov whatever [rsp + val]
if they arent i'll get given the wrong interrupt numbers
is that syntax valid in gas?
Well it’s x86 assembly
So yes if you convert it
But if you push all the GPRs
Those offsets are wrong?
Ahh
right?
Is r8 being clobbered then?
shouldnt be
.macro MPUSHA
push %r15
push %r14
push %r13
push %r12
push %r11
push %r10
push %r9
push %r8
push %rdi
push %rsi
push %rbp
push %rbx
push %rdx
push %rcx
push %rax
.endm
MPUSHA just pushes stuff it doesnt touch any regs
You push after
oh wait yeah

If you interrupt anything
R8 is donezo
Push then use rsp relative loads
And do the processing in C
🙏
It’s way cleaner
And the compiler can do its magic
The assembly should just be a stub to save and restore and set the registers right to pass the frame
fred_common_entry:
MPUSHA
lea 120(%rsp), %r8
mov 0x28(%r8), %rsi
mov 0x08(%r8), %rdx
PUSH_SENTINEL
call Interrupt
POP_SENTINEL
MPOPA
ERETS
something like this perhaps
MPUSHA pushes 120 bytes
Do you even need the r8 here atp?
You can just do it right off rsp
What I do is have seperate C functions for fred that convert the frame etc
Since it’s also a different layout
Every so slightly
yeah i could
this vs
extern void dispatch_interrupt(irq_saved_regs_t* regs, irq_cpu_frame_t* frame, uint64_t vector);
then i have this function the IDT calls with
mov rdi, rsp ; irq_saved_regs_t* ; also same as lea rdi, [rsp + 0]
lea rsi, [rsp + 128] ; irq_cpu_frame_t
mov rdx, [rsp + 120] ; uint64_t vector
cld
call dispatch_interrupt
cli
and FRED calls it like this so it can do its FRED spesific parsing
It’s FRED Stub - JMP > Fred ASM - CALL > Fred C - CALL > common interrupt handler
fred_common_entry:
MPUSHA
mov 0x98(%rsp), %rsi /* 120 + 0x28 = 0x98 */
mov 0x80(%rsp), %rdx /* 120 + 0x08 = 0x80 */
PUSH_SENTINEL
call Interrupt
POP_SENTINEL
MPOPA
ERETS
Which is very little overhead which dosnt matter since FRED is wayy faster than IDT etc
i wont notice the speed difference at all
im doing this because intel is pushing this hard, and i dont want to be left behind
Yeah lmfao. Chances are on real hardware it will be faster esp on diskIO
Linux saw some good benchmarks
I’m pushing it hard too 
I have the role for a reason
I was the first hobby kernel to get it
My new kernel dosnt have CPL3 rn so
And it dosnt matter much for FRED
It’s still much faster due to how it operates on 64byte aligned stacks
And it pushes 64 bytes always
So it’s always on a cache line perfectly
Among other improvements
You will fault BTW if you don’t handle this right
pushing the r8 clobber fix
i use the stack that limine gives, if it isnt 64 byte aligned i may have to move my stack
send link?
sorry was checking some FRED spec stuff to make sure i wanst mucking stuff up
My IDT path wasn’t properly setting interrupts so any handler couldn’t get interrupted

My gates are all set to clear IF for stack pushes etc and then letting it happen on C code and disabling on exit
And I forget to have that 
FRED also sets RFLAGS to 0x2 so it’s cleared too along with direction flag etc
ty
yw
how far through?
it did
just wasnt there long enough
[0]: Interrupts enabled!
[0]: Initialisation of interrupts done!
ahh
happens between these two
it triple faulted at the first interrupt, i think
perhaps like you said, the stack isnt aligned 64 bytes
FWIW i use the limine stack too
its just the RSPx that has that requirement
otherwise things would be funky
so its not that
i really dont have the energy to back and forth debug this at 1am 🙁
Got to sleep
😭
I may be preaching the word of FRED more than a jahovas witness but I still respect sleep
lol
if you feel like poking around my code, i really really do appreciate PRs
retro rocket isnt hard to build, just has some odd deps
Il see what I can do
I will probably end up moving more logic into C though
And unifying things like my kernel
Because then I can control c control v a lot more of it

And it will be cleaner regardless
so long as the path for interrupts still handles the same for idt as it does for isr; it should go through the global functions Interrupt and IRQ with the same parameters
other levels shouldnt care about fred or idt
No they won’t
It’s just since the frames differ I need to deliver the stuff a lil differently
To make it easy for both
also, dont change the idt stuff 🙂
In my old kernel I had some nasty conversion logic
The only thing I’d have to change is argument passing
Like here
i dont really want to change idt stuff, this is well tested on real hw
FWIW this is behavior that wouldn’t break between real hardware and emulation
But I understand
but you dont need to change Interrupt() and IRQ(), just put your stuff below it and call into the existing stuff
May just be way harder to make FRED work
i have asm stubs that call into the C
https://git.evalyngoemer.com/evalynOS/evalynOS-old/src/commit/e469e9581f5c0583e3618935ff4e936a88cdacd8/kernel/src/drivers/x86_64/fred/fred.c#L164
you just need to add this 
if you dont wana change IDT code
i dont need all that
its for converting a frame?
its irrelevant to isr handlers in retro rocket
what do ISR handlers take in?
this is all i need for isr
void Interrupt(uint64_t isrnumber, uint64_t errorcode, uint64_t rip)
and this for IRQ:
void IRQ(uint64_t isrnumber, uint64_t irqnum)
ahh then its easy enough to not touch ^^
i dont need complex frame stuff
mine needs to edit the stuff from the stack for stuff like fork() later signals reg dumps, redirecting from faults etc etc
the Interrupt thing can build entire stack traces just from rip (its the pre-empted rip)
yeah, its a very simple system
which is why i was able to test it quite well on real hw
the existing code is at the bottom of asm/loader.S and src/idt.c
and yeah as i said i was planning to move the fred stuff to a new fred.c
but that can wait till it works
unless you get the urge lol
all i need to do is intead make this call a fred C function then invoke Interrupt(vector, fredframe->error, fredframe->rip)
would be nice if it lived in a fred.h i guess
i can prolly just inline it in the .c tbh
i like your fred flintstone btw lol
ahh dang
i ended up having to cut down the size on mine
ASCII codeblocks actualy end up being a ood bit smaller for the detail ratio
i fit an entire ralsei properly
check this thingy if you wana generate it
https://dom111.github.io/image-to-ansi/
i just noticed you configure the msrs for stack addresses, i dont
uint64_t star = ((uint64_t)(0x18 | 3) << 48) | ((uint64_t)0x08 << 32);
wrmsr(STAR, star);
wrmsr(FRED_CONFIG, (uint64_t)fred_ring3_entry_asm_stub);
wrmsr(FRED_RSP0, (uint64_t)&fred_kernel_stack + sizeof(fred_kernel_stack));
wrmsr(FRED_RSP2, (uint64_t)&fred_df_stack + sizeof(fred_df_stack));
wrmsr(FRED_RSP3, (uint64_t)&fred_nmi_stack + sizeof(fred_nmi_stack));
wrmsr(FRED_STKLVLS, 0);
thats only used for stack levles
which since its zero
its unised
RSP0 is from cpl3 to 0
the others are for stack levels
IIRC
my new kernel leaves rsp0 unset
i'll let you have a dig through my code 🙂 gonna go rest
thanks so much for looking
il try and do some stuff tomorrow
o7
btw @willow ginkgo where has intel been pushing this lol
i'll do other stuff away from the idt/fred suff
it was more about this
toms hardware really do a half ass job of explaining it
but if its coming to zen 6, i dont have to worry about "its intel only code" any more
Evalyn Goemer's Personal Website
it will eventually be everywhere
nice
I haven't really looked more into any FRED stuff
mainly been looking into acorn DFS today
install deps listed at bottom of readme
cd retrorocket; mkdir build
cmake ..
make -j${NPROC}
expects gcc only, isn't designed to be portable
and yeah it's that simple, when it's done it outputs iso and usb.img
this is the repo yeah
yup
if you're wondering php is required for image builder scripts, I prefer it to bash
how are you building it
can you show me the whole log
it looks like it's ignoring the include paths
your xmmintrin is weird....
why is it not standalone @light ice
idk, works fine for me in Ubuntu
no idea, something isn't right about your setup
it builds fine in github actions too

I am lowkey kinda cursed tho
I can’t build astral either
Everything I do works for me tho
yeah something ain't right
And never has had strange issues
😭
I do it a bit more freestanding tho and stuff so it’s less environment dependent
My commit is up for you to go look at fred shit tho and compare
you don't need a special cross compiler for retro rocket if that's what you got
Where'd you stash it
try a docker container of Ubuntu 24.04
your os interrupt setup is a bit different to mine
I CBA to set that up rn
try using gcc 13 instead
or 14
I'm using 14 and that's fine
I don’t think I can downgrade safely on arch
can't you install side by side
I have like gcc 10 thru 14 all installed at once
then CC=gcc-14 cmake ..
you could try this quick hack
make a file called mm_malloc.h in the include dir of retro rocket
no contents
then try
it can't make it worse lol
yeah that will fix it
did you explictly disable it?
no nor did I enable it!
because alot of compilers have it on by defau;t
add the no stack protector to the cmake cflags
the limine template needs all of these
yes
I'm going to change the cmakelists so you can pull
there, pull now
no thats what im saying
its RIGHT THERE lol
what the hell have you done to your system
yup
its still trying to do stack checks
and did you rerun cmake
i really dont know wtf youve done
is your os putting -fstack-protector-strong in?
hmmmm
its only IN the asmflags
pull and redo cmake
hopefully we can get it down to just the zlib
so we only got zlib problem
where are all these warns coming from that i didnt enable
that more annoyingly are in third party dependencies
mbedlts isnt my code...
Same with QEMU etc
its probably from -Wextra
as you have a newer gcc and they continually add new warnings in there
i'd ignore the warnings, they arent in my code
im not sure why it doesnt like zlib
go to lnclude/zlib/zlib.h
change #include <zconf.h> to #include "zconf..h"
likely wont fix it, im cluthing at straws
but im wondering if your systems zconf is weird and its picking it up
also what do you get from:
env | grep -E '^(CFLAGS|CXXFLAGS|CPPFLAGS|LDFLAGS|CC|CXX)='
no
just a return code of non zero
can you show me the whole build output pls
ty
nm: 'CMakeFiles/kernel.bin.dir/src/zlib/inflate.c.o': No such file
if i do from build dir
what.....
im not familar with cmake that much
ok so
i ran from wrong foldeer
its not adding the z_ prefix
strange
lmfao
how do i do incremental builds here?
btw what editor do you use
KATE
i have it all set up for clion if you use it, if you dont its ok
micro for terminal
as in, comes with .idea dir
it just gives syntax highlighting for .rrbasic and stuff
i get LSP and stuff dw
thanks soooo much for having the patience to work through this with me
asm/loader.S and src/idt.c
nowhere else rn, i didnt get around to splitting it up yet
i was also going to add a boot flag you can specify to turn fred on and off, thats what the bool is for, but rn that does nothing
it shouldnt even be called loader.S 
it used to be a 32-64 trampoline for grub years ago
it used to do basic identity map paging, long mode, and idt
now it just does idt stubs
is there a helper to enable and disable the IF flag?
interrupts_on()
and interrupts_off()
while FRED is being set up, interrupts are off
they arent enabled for the first time till thats done with
ahh so its fine to not diddle that in the enable_fred_for_this_cpu?
im yeeting cpu_serialise because mov CRx is already considered as such
same for reading so itd redundant
and the exita bits set in FRED looked off
and do you set IA32_STAR?
im not sure if its needed unless it does CPL changes though
thought i did
but i know it can do corruption if thats not valid
i dont ever change CPL, everything retro rocket is ring 0
but it also does CPL3 things
so it should be fine?
just wanting to note this just incase
nothing is CPL3
if you did set up CPL3 interrupts they would never be called
so IA32_STAR should be set? it wouldnt be set anywhere but idt.c
should be fineee
its like just the way its handling the incoming interrupts thats bad, im sure youll figure it out real quick
its 120 bytes pushed by the MPUSHA macro
i dont remember the order of the regs pushed, youd have to check the macro
its in the same file
yes
oh and in retro rocket we dont have reentrant interrupts
so all interrupts run with interrupts off and dont re-enable them
it makes things much simpler
i just realised this doenst pass stuff to IRQ, only Interrupt
Interrupt and IRQ are different places, Interrupt for exceptions, and IRQ for the rest
see src/interrupt.c
ahh yeah il 100% wana put this into C then
after i made all the logic in ASM

il just make more in asm

yeah if that works for you, and its more readable for me
so long as we dont change the IDT path for now
bur yeah depending on the isr number etc it needs to be routed to either IRQ or Interrupt and IRQ takes 2 params Interrupt takes 3
yes i can handle this
yup
irq 0 is isr 32
and i have 64 upwards reserved for msi
they still go through IRQ as normal
if i remember isrnumber is the raw number irqnum is the isr - 32
as in these macros
IRQ 0, 32
IRQ 1, 33
IRQ 2, 34
IRQ 3, 35
IRQ 4, 36
IRQ 5, 37
IRQ 6, 38
looks like it
.macro IRQ num byte
.global irq\num
.type irq\num, @function
irq\num:
MPUSHA
mov $\byte, %rdi
mov $\num, %rsi
jmp irq_stub
.endm
been a while since i looked at that or ever used the isrnumber parameter
yes
its a sentinel value that marks when we move into an irq
its purely for stack traces
ahh
btw on the C side, you can use dprintf() to output to the qemu console
kprintf to print to the screen
dprintf works right from early boot
kprintf works as soon as flanterm is up
gonna add a bit like this
i should be able to
it wont really need to be used elsewhere?
true, i just like to separate out structs
fair fair
yeah try dprintf
these remind me of page tables in the fb for some reason
hmmm
not sure why
perhaps something too broken
if you need to, you should be able to attach gdb to the kernel
is that the dprintf?
yes
tripple fault without the print

if i dont push the sentinal
it dosnt crash
but does the courrption?
hmmmm
is it trying to pull the sentinel as a return address or something
the sentinel is just the value -1
maybe?
im just gonna do this without the macros and use more of my OG code to be safe

atleast to debug
its important to push the regs though in MPUSHA
as those stop the interrupt handler trampling it
im sure youll figure it out
its just these?
im just inliining it
also i dont think you need all of these
with the sysv ABI

as it makes the C code do some saving its self
etc
wtf
maybe its the crappy patches
let me try SIMICS?
oh wait theat dosnt work

only if your CPU has it
but i dont see any official announce
they have it in KVM
wait is the kernel build with SSE?
it dosnt work
i have no idea why
here is what i got so far
wait
GPF?
its IA32_STAR
its expecting its ring0 sections too
because how its assuming segments
was gonna say
so you need to set IA32_STAR, and what, the kernel stack for ring 0 ints?
we dont call ERETU
yes
we call ERETS
hmmmm i really really dont want to go as far as changing my entire gdt
RED event delivery loads the CS and SS selectors with values
derived from IA32_STAR[47:32]
you may not
ifn its in a valid layout
its a copy of the limine one, copied to where i can control and alter it, but that has effects way outside of fred
uhh
src/gdt.c
so you dont know the exact layout?
it also does page table stuff
i think this one is your gob to deal with the GDTness
it clones the limine page table, so i can keep limine's identity map but identity map new pages as i wish for driver use
cursed
i dont want to change the GDT
💀
that effects stuff outside FRED
FRED should stay separate
dont you just need to set ISTAR to the selectors in the gdt?
this is the GDT limine gives
not change the GDT?
yes but its cursed af how it does it
yes and thats the one i take copy of, i dont change it
because x86
BUT
if you only do ring0 64bit
i can make it work

maybe
hopefuly

i thought this is all about interrupts and getting rid of legacy jank
so why is it making me change my gdt
because its how syscall/sysret work
didnt you say youre using limines stuff too?
but i dont USE syscall/sysret
but my own GDT
i dont have any need for syscalls
they USE THE SAME FORMAT. and its needed for RING0 stuff still
let me find the docs on this
what even does this mean
okay so
the IA32_STAR msr
it takes thoees bits from it
and puts it into the segment registers
but it assumes an offset
for SS
nope
uint64_t star = ((uint64_t)(0x18 | 3) << 48) | ((uint64_t)0x08 << 32);
wrmsr(MSR_STAR, star);
ok, so in my case its what 6th and 7th entry?
only difference from what it says in the limine docs is the gdt is no longer "in bootloader reclaimable memory"
as i take a copy and move it
but that matters not
hmmm wait i dont move it any more ignore that
like i said it matters not, it doesnt get reclaimed
it dosnt seem to help
uint64_t star = ((uint64_t)(0x28) << 48) | ((uint64_t)0x28 << 32);
wrmsr(IA32_STAR, star);
it cant hurt to set it though
is it because we dont set up the stacks
yours does
the segmebts should be right
true, but its worth trying yeah
no worries
Only thing I know of
Is the patches don’t support certain edge cases
With the GDT
And will constantly flood you with GPFs
Like how it is now

If the GDT is fucked up enough to not be standard
how is a simple limine gdt an edge case
ive been working on something completely different tonight btw
this
^ it literally is a real production fs from the 80s that is pretty much like that array
double sided double density DFS
nah that one only happens on eretu.
but its when star dosnt match the cs/cs thats expected so it has to manually read the GDT
so its not that
i managed to get simics to work on windows
i applied your FRED patches
it still breaks if:
__attribute__((target("general-regs-only")))
void fred_ring0_entry(fred_frame_t* frame) {
/*uint8_t vector = (frame->ss >> 32) & 0xFF;
kprintf("eeee %x\n", vector);
if (vector <= 0x1F) {
Interrupt(vector, frame->error, frame->ip);
} else {
IRQ(vector, vector - 0x20);
}*/
}
so, the crash isnt in the C side
just locked up here. no triple fault
i figured out why kprintf isnt working here i think
so it does run to the first int
What was the issue?
it prints to a backbuffer, the backbuffer is flipped by the timer interrupt
without interrupts on it cant flip
so i temporarily put rr_flip() inside printf
Ah
i can come up with a better solution later
idk yet
im adding a shit ton of debuggin
i also set IA32_STAR, and i set up all 4 stacks to eliminate that being a problem
btw simics is ass, what were intel thinking
hmmm
no crash, continually firing the timer interrupt...
hmm wait no thats isr 0
not 32
wtf is that doing
How do you calculate the vector?
uhm why did you add: attribute((target("general-regs-only")))
Debugging
To disable SSE on that function
ok
To make sure it wasn’t clobbering before it could get saved etc
that may break some interrupts
That only affects the one function?
The other functions would be able to use FPU after
i dont think it matters rn
but something is fishy
void fred_ring0_entry(fred_frame_t* frame) {
uint64_t vector = (frame->ss >> 32) & 0xFF;
kprintf("fred int %lu\n", vector);
if (vector <= 0x1F) {
kprintf("Interrupt: %lu %lu %lx\n", vector, frame->error, frame->ip);
Interrupt(vector, frame->error, frame->ip);
} else {
kprintf("IRQ: %lu %lu\n", vector, vector - 0x20);
IRQ(vector, vector - 0x20);
}
}
this is getting 0
Not a runtime error if it did
and thats from:
fred_ring0_entry_asm:
MPUSHA
PUSH_SENTINEL
movq %rsp, %rdi
call fred_ring0_entry
POP_SENTINEL
MPOPA
ERETS
and:
ah it doesnt have the calculations for rdx etc
That’s what C does
You just need to put the stack pointer into rdi
But if the sentinel is there
You need to move the movq up
Yes because it’s calculating the vector now properly
Because the stack isn’t misaligned
Also the sentinel may be making the stack not follow sysv
FRED pushes 64 bytes. Then you push 120. Then an extra 8 bytes and then you have the 8 bytes from call
Good good
im tracing down the stack with debug jank:
/* Both the Interrupt() and ISR() functions are dispatched from the assembly code trampoline via a pre-set IDT */
void Interrupt(uint64_t isrnumber, uint64_t errorcode, uint64_t rip) {
__attribute__((aligned(16))) uint8_t fx[512];
__builtin_ia32_fxsave64(&fx);
kprintf("Interrupt %lu\n", isrnumber);
for (shared_interrupt_t *si = shared_interrupt[0][isrnumber]; si; si = si->next) {
/* There is no shared interrupt routing on these interrupts,
* they are purely routed to interested handlers
*/
if (si->interrupt_handler) {
kprintf("Call handler\n");
si->interrupt_handler((uint8_t) isrnumber, errorcode, rip, si->opaque);
wait_forever();
}
}
if (isrnumber < 32) {
/* This simple error handler is replaced by a more complex debugger once the system is up */
aprintf("CPU %d halted with exception %016lx, error code %016lx: %s at rip 0x%lx.\n", logical_cpu_id(), isrnumber, errorcode, error_table[isrnumber], rip);
wait_forever();
}
entropy_irq_event();
local_apic_clear_interrupt();
__builtin_ia32_fxrstor64(&fx);
}
i expect one int call then wait_forever()
kprintf("Interrupt: %lu %lu %lx\n", vector, frame->error, frame->ip);
Interrupt(vector, frame->error, frame->ip);
doesnt actually seem to call that function at all
attepting to call Interrupt causes the exception 13
fxsave in the interrupt handler crashes it @light ice
I wonder if it’s to do with stack alignment?
maybe
#GP(0) If a memory operand is not aligned on a 16-byte boundary, regardless of segment. (See the description of the alignment check exception [#AC] below.)
on fxsave
__attribute__((aligned(16))) uint8_t fx[512]; __builtin_ia32_fxsave64(&fx);
looks aligned to me?
to align you should subtract 8 bytes from the stack before calling i think
You could also allocate an extra 8 bytes in the struct and add 8 bytes to the pointer?
you are breaking the ABI; the abi requires that the stack is aligned in a specific way at entry
there is an attribute to make the compiler align the stack
or at least to make clang align it
[[gnu::force_align_arg_pointer]]
hmmm taking out fxstor allows it to boot
Shouldn’t this be aligned? FRED pushing 64 bytes. Then 120 for the registers plus the 8 for the call?
If you remove the sentinel
there are other issues.... which are simics being simics i think, this vm isnt set up properly for rr
the call pushes 8 bytes tho
idr what the exact requirement is, though
no sentinel, already took that out
maybe that is fine
It’s 16 byte alignment for the stacks
I know
(64 + 120 + 8) % 16 = 0
Should be correct?
after the call, the stack has to be congruent to 8 mod 16
everything should already be aligned to 16 bytes yes
just use the stack aligning attribute
which call? the call to void Interrupt() or the call to the isr stub?
the call in the asm
ofc
it already is aligned
I guess not considering it’s blowing up
clang knows better how to do alignment tbh
well, it only blows up if we attempt to do fxstor, in a function 3 funcs removed from the asm
yeah but it can maintain align on its own

