Techflash OS | Operating System Development | Page 2

hexed citrus Jul 15, 2023, 10:24 PM

#

but yeah 1 sec, I'm fixing a random error

ebon reef Jul 15, 2023, 10:25 PM

#

in fact

#

you could have that be a macro

#

ROUND_UP_TO_PAGE_ALIGNMENT or something

#

so its obvious what its doing

hexed citrus Jul 15, 2023, 10:25 PM

#

#define ALIGN_PAGE(x) ((x + 4095) / 4096)

?

ebon reef Jul 15, 2023, 10:25 PM

#

yeah ig

hexed citrus Jul 15, 2023, 10:25 PM

#

alright, testing it now

ebon reef Jul 15, 2023, 10:26 PM

#

also its lazy to have your small block allocator just allocate pages

#

hugely wasteful

#

but this will work for now

hexed citrus Jul 15, 2023, 10:26 PM

#

yeah, I know it is lol, but it works
that's more than I can say about that last one

inland plinth Jul 15, 2023, 10:27 PM

#

https://godbolt.org/z/GPTYWzbcq none of em!

hexed citrus Jul 15, 2023, 10:27 PM

#

not looking to really mess with it until I can verify it works

ebon reef Jul 15, 2023, 10:28 PM

#

inland plinth <https://godbolt.org/z/GPTYWzbcq> none of em!

it is turning the last one into mine but the cmovns surprised me a little and im guessing it has to do with signed division semantics

inland plinth Jul 15, 2023, 10:29 PM

#

yeah at the end it tries to go your way but idk what cmov is doing either

#

weird

hexed citrus Jul 15, 2023, 10:29 PM

#

that's nothing like what mine generated, this is a lot of asm for 1 line, I should check my compiler settings, hang on

ffffffffffe046e5 <malloc>:
ffffffffffe046e5:       55                      push   %rbp
ffffffffffe046e6:       48 89 e5                mov    %rsp,%rbp
ffffffffffe046e9:       48 83 ec 20             sub    $0x20,%rsp
ffffffffffe046ed:       48 89 7d e8             mov    %rdi,-0x18(%rbp)
ffffffffffe046f1:       48 8b 05 e8 99 00 00    mov    0x99e8(%rip),%rax        # ffffffffffe0e0e0 <__stack_chk_guard>
ffffffffffe046f8:       48 89 45 f8             mov    %rax,-0x8(%rbp)
ffffffffffe046fc:       31 c0                   xor    %eax,%eax
ffffffffffe046fe:       48 8b 45 e8             mov    -0x18(%rbp),%rax
ffffffffffe04702:       48 05 ff 0f 00 00       add    $0xfff,%rax
ffffffffffe04708:       48 c1 e8 0c             shr    $0xc,%rax
ffffffffffe0470c:       48 89 c7                mov    %rax,%rdi
ffffffffffe0470f:       e8 68 fa ff ff          call   ffffffffffe0417c <PMM_Alloc>
ffffffffffe04714:       48 8b 55 f8             mov    -0x8(%rbp),%rdx
ffffffffffe04718:       48 2b 15 c1 99 00 00    sub    0x99c1(%rip),%rdx        # ffffffffffe0e0e0 <__stack_chk_guard>
ffffffffffe0471f:       74 05                   je     ffffffffffe04726 <malloc+0x41>
ffffffffffe04721:       e8 f7 e2 ff ff          call   ffffffffffe02a1d <__stack_chk_fail>
ffffffffffe04726:       c9                      leave

ebon reef Jul 15, 2023, 10:30 PM

#

ffffffffffe04702:       48 05 ff 0f 00 00       add    $0xfff,%rax
ffffffffffe04708:       48 c1 e8 0c             shr    $0xc,%rax

#

nope it did generate it

hexed citrus Jul 15, 2023, 10:30 PM

#

oh yeah lol it's on a debug build

#

no wonder

#

let me build the optimized one

#

wtf

#

it literally inlined malloc

ebon reef Jul 15, 2023, 10:31 PM

#

lmao

hexed citrus Jul 15, 2023, 10:31 PM

#

the pains of LTO 😔

#

let me hunt around and see if I can find it in places that I know it should be

#

wtf it inlined like, everything

#

there's no way I'm finding it

#

pffffft I think it optimized a little too hard

#

as soon as it boots it just triple faults

ebon reef Jul 15, 2023, 10:34 PM

#

that kind of LTO sounds like the kind of thing that everything was already doing in 1965 and then it took the *nix ecosystem until 2047 to reinvent it and proclaim it as a great innovation

hexed citrus Jul 15, 2023, 10:35 PM

#

yeah

#

I think it optimized out panic.....

#

because it looks like it gets decently far in the boot process, then just dies and panic doesn't catch it

#

and I mean that's certainly possible, but very unlikely

#

hmmm, yeah I don't like that

(gdb) hb malloc
Function "malloc" not defined.
Make hw breakpoint pending on future shared library load? (y or [n])

#

huh

#

it still triple faults

#

actually, I wonder if the issue is that I'm trying to initialize a region of memory that's super small (500K)

#

I'll just add a gate that says if the regions is <8M, subtract it's size from the total system memory, and don't initialize it

#

actually wait, what am I doing, I can just use serial to debug it

ebon reef Jul 15, 2023, 10:57 PM

#

hexed citrus actually, I wonder if the issue is that I'm trying to initialize a region of mem...

hexed citrus Jul 15, 2023, 11:06 PM

#

alright, I've moved some of the info to my logging function

#

that way I can actually see what's going on over serial

#

these sure are some logs

#

bruh

#

why is malloc starting at NULL and working it's way up

#

#

(it's staying at the same address because that's my malloc tester doing stuff there

#

that's test #1: essentially void *a = malloc(64); free(a);

#

later on (in test #2), it's void *a = malloc(64); void *b = malloc(64); free(a);, and leaks 64 bytes (or in this case, 4K)

#

and that's where it crashes

#

it starts trampling over something that the system really doesn't like

#

ohhhh I think I know why

#

it seems like it only supports 1 region

#

huh

#

no

#

it very much works with multiple

#

ah, I found why

#

it's because it just isn't designed to store the base address of the region

#

it just assumes that everything starts at 0

hexed citrus Jul 16, 2023, 6:28 AM

#

ok this is going to be very jank

#

but in theory it now keeps track of the base address of each region

#

and record a bunch of info that should let us figure out what the base address of any bit in the bitmap

#

key phrase: in theory
entirely untested

#

aand page fault

#

nice

#

aha
the return statement is.... dereferencing NULL

if (bitmapOff > regions[i].bitmapEnd) {
    // it's the previous entry
    return regions[i - 1].startAddr;
}

...huh

#

oh my god is it that fucking NULL bug again I swear to god

#

I will literally delete GCC from my system if it's that again

#

hmm

// it's the previous entry
volatile memRegion_t reg = regions[i - 1];

maybe something really is wrong here

#

sigh.... welp time to add way too many checks to make sure that the most basic functions of the OS aren't falling apart

#

also totally fuck it up probably because it's 11:44PM and I fell asleep at like... 3AM last night, getting ~5h of sleep

#

this is going to end very well

#

.... what

#

literally how, it's basic math

#

oh

#

so the initialization is completely broken

#

YOO

#

ok this is like, not actually that exciting, it's still broken

#

but it's close

#

oh wait

#

no it's actually not

#

how is it getting multipled

#

for context, the base address of the large region is 0x962000

#

and it should be straying far from that, since it puts the bitmap there

#

but hey, it's.... kinda close

#

actually that address should still be within valid address space

#

¯_(ツ)_/¯

ebon reef Jul 16, 2023, 7:11 AM

#

Have u even figured out what address it's faulting on

hexed citrus Jul 16, 2023, 7:11 AM

#

huh, yeah the base address my function is returning is correct

ebon reef Jul 16, 2023, 7:11 AM

#

Or where the page fault is happening

hexed citrus Jul 16, 2023, 7:12 AM

#

ebon reef Have u even figured out what address it's faulting on

yeah something is dereferencing NULL

#

0x0E

#

not worried about that yet

ebon reef Jul 16, 2023, 7:12 AM

#

Not worried about what yet

hexed citrus Jul 16, 2023, 7:12 AM

#

I need to get the address that the PMM_Alloc function is returning to actually be valid first

#

somehow it's getting created incorrectly

#

ok ok a little info on the jankiness that my allocator pulls

#

when you init a region of memory, you pass in a pointer to the memory, and the size of the memory

ebon reef Jul 16, 2023, 7:13 AM

#

Why on earth is your pmm returning virtual addresses

#

It's a physical memory manager

hexed citrus Jul 16, 2023, 7:13 AM

#

it isn't

#

it's supposed to be returning physical addresses

#

that's the point

#

something is becoming very wrong

ebon reef Jul 16, 2023, 7:13 AM

#

Right and then what do you do with those physical addresses

hexed citrus Jul 16, 2023, 7:15 AM

#

I just do I/O on them, I haven't set up paging or anything yet
I just treat memory as separate chunks of contiguous memory that I can allocate blocks from
(these chunks come from the memory map, which we parse and then pass in the valid ones to the init function)

ebon reef Jul 16, 2023, 7:15 AM

#

Paging really isn't a "oh maybe I'll do it later" thing tbh

hexed citrus Jul 16, 2023, 7:16 AM

#

the issue is that this allocator is designed to be handling addresses that start at 0, not at any set value, which I'm trying to hack in

#

like, this is really horrendous, I fill in one of these for every region I allocate from

typedef struct {
    void *startAddr;
    void *endAddr;
    uint64_t bitmapEnd;
    uint64_t numBlks;
} memRegion_t;

3 of these variables essentially give the same info

#

my current issue: math
return (void *)(i * 4 * 8 + j + baseAddr(i, j));

#

oh wait

#

no that's perfectly fine I bet

#

the issue is this void *addr = (void *)((uint_fast64_t)frame * BLOCK_SIZE);

#

oh no wait and that fucks up the bitmap

#

AAA

#

ok so for the bitmap to work correctly, findFree really does need to return the low value

#

and then I just need to shoehorn in the base address just before we return

#

sounds like nothing could possibly break at all from that

#

(I can already tell that freeing is just not going to work until I find some way to reverse the base address as well)

#

and since the entire base address logic is based around 2 specific variables from the findFree function, and I need to get them out of the function easily, I'm just gonna dump them in a global variable before we leave, then use them in the caller

#

.... holy shit

#

I did it

#

it's allocating

#

granted all of the logging means that I'll be dead before it finishes the whopping 9999 iteration long test

#

but it's allocating

#

aand malloc test #2 just exploded

#

fun

#

see I can't help but notice that both frame and addr are... well wrong

#

they're both overwriting the bitmap

#

so that makes sense

#

the malloc test writes zeros to it

#

so the PMM just thinks that everything is free because we're trampling over the entire bitmap

#

or, more precisely, the first 64 bytes of it

#

repeatedly

#

ohhhh I never fixed PMM_DeInit

#

that's why

#

it tries to de-init the zone with the bitmap

#

and that doesn't really handle my base addr thing

#

¯_(ツ)_/¯

#

I'll just make it die if you call it, and go back to the old pointer hack method of just telling the pmm that the zone with the bitmap doesn't exist

#

i think it just works now

#

it doesn't look like it's overwriting the bitmap anymore

#

sigh but it didn't help

#

it still returns nonsense

#

it still just returns the base address + 4 every single time

#

technically it should be leaking memory every single time, since free doesn't actually work yet

#

bangs head on desk

#

I think I'm done with this nonsense for today
It's already almost 1am
I'll pick this up tmrw

hexed citrus Jul 16, 2023, 4:38 PM

#

2 bugs to fix today:
- find out what in the world the random characters are that are getting printed on the newlines
- find out why the allocator is dead

hexed citrus Jul 16, 2023, 6:45 PM

#

sigh I think i fixed part of it

#

and broke 10 more

hexed citrus Jul 16, 2023, 10:07 PM

#

ok

#

tons of work has been done

#

it now page faults by trying to set a bit in the bitmap

#

I am honestly so confused

hexed citrus Jul 16, 2023, 10:37 PM

#

idk but I think it's starting to go a little outside the range of the bitmap

hexed citrus Jul 16, 2023, 11:10 PM

#

aaa now I broke baseAddr

#

I just had it working......

hexed citrus Jul 16, 2023, 11:26 PM

#

well now I can catch the bug when it happens

#

no idea what to do about it though

#

the issue is that somehow it's being passed invalid values

#

well

#

actually no

#

it's valid

#

what

#

holy shit

#

it's running

#

WHAT

#

NO WAY

#

👀 it's doing the malloc test

#

holy shit

#

I think I did it

#

NO

#

wait

#

YES

#

THATS GOOD

#

some ancient error handling killed it

#

because the bit was being multiplied

#

so I set it to if it's > 4096, die

#

but now it triggered because it made it to 4096 allocs! 🥳

#

I'm gonna turn off all of the debug logging

#

and the strict error handling that restricts your memory

#

and re-enable the counter on the malloc tester

#

and watch in glory as it succeeds

#

actually wait

#

this would only end up using the final entry

#

I should really get free working

#

it breaks because it uses too much RAM i think

#

it worked when I only did the "Intentional leak" test

#

but when I added the "allocate then free" test, it died

#

uhh crap I think free works a little too well

#

also I can't comprehend where those stupid characters on the newlines are coming from but I am extremely angry at them

#

what

#

😭 it was working

#

now it's not

hexed citrus Jul 17, 2023, 12:22 AM

#

bruh I'm so confused

#

hold up

#

something's wrong here

#

actually, 2 things are

#

why is it leaking memory in malloc test 1, that's the one where it shouldn't
how does the pointer reset for malloc test 2, it shouldn't be able to do that
how is it not leaking memory for malloc test 2

hexed citrus Jul 17, 2023, 6:40 AM

#

I'm really just chipping away at this allocator aren't I

#

first I had to hack the initialization to get the address to be page aligned

#

now it's randomly skipping....

#

17551360 bytes

#

oh it literally says that, why did I do that math

hexed citrus Jul 17, 2023, 6:58 AM

#

oh man it could be worse

#

on my optiplex it page faults trying to initialize the bitmap

hexed citrus Jul 18, 2023, 6:57 AM

#

hm

#

I think i broke the allocator

#

by allocating tons of huge chunks of memory, it just falls apart

#

the entire system is barely still running

#

it can get through exactly 24 iterations

#

then it dies

#

suspiciously very close to half of the availible system memory thinkong

#

(each allocation is 10MB)

#

and then when i set the availible RAM it 1G, it dies at 49

#

iiiinteresting.....

#

I added a bunch of sanity checks to free

#

so I would imagine if it was failing outright I would know

#

the weird part is that it keeps going

#

it just absolutely slows to a crawl

#

and QEMU is absolutely hammering one of the cores

hexed citrus Jul 18, 2023, 7:30 AM

#

I'm having PIT troubles again

#

I'm always having PIT troubles...

#

I should really just abandon the PIT

#

I currently don't even try to initialize anything else, and I don't even check if the PIT is even there

#

it just yeets bytes into the I/O ports, and does no verification that it actually works

#

welp
looks like I'm implementing ACPI stuff

#

fun

hexed citrus Jul 18, 2023, 8:01 AM

#

cool

#

wait wtf
I swear this is QEMU, not bochs
unless the 1:13AM is actually starting to fuck up my brain and now I can't read the QEMU on the title bar of the window

#

I don't even have bochs on here

#

huh, well that's not good

buoyant gazelle Jul 18, 2023, 8:48 AM

#

the checksum? so not the "RSDT" string but the actual checksum field?

#

you cant trust that to be accurate

hexed citrus Jul 18, 2023, 5:34 PM

#

yup, the checksum field

#

idk it seems like it's wildly off

#

maybe I'll just print a warning about it and carry on

#

I guess some BIOS's just don't implement it

buoyant gazelle Jul 18, 2023, 6:03 PM

#

hexed citrus idk it seems like it's wildly off

2048 might be your mistake too tbh

#

but also like

#

dont bother

hexed citrus Jul 18, 2023, 6:19 PM

#

yeah I do check it, and then I just print if it's valid or not, I might later implement some logic where like, if something seems REALLY suspiciously wrong, AND the checksum is invalid, don't trust it
but idk, I'll first get it working, then worry about that

#

nice

#

ugh

#

I just went to go test this on real hw

#

but I forgot that the memory manager somehow page faults on my optiplex

#

like, it page faults INSTANTLY

#

the very first write in order to set up the bitmap

ebon reef Jul 18, 2023, 6:35 PM

#

good page fault or bad page fault

hexed citrus Jul 18, 2023, 6:35 PM

#

bad

#

according to the error code (0x02) it thinks that the page isn't present

torn spire Jul 18, 2023, 6:36 PM

#

then it isnt present

ebon reef Jul 18, 2023, 6:36 PM

#

dont take this the wrong way but i dont get how youve been working on debugging the same very small component for like a week now

hexed citrus Jul 18, 2023, 6:36 PM

#

ok but then why does the memory map say it is

ebon reef Jul 18, 2023, 6:36 PM

#

you may have some problem in your methodology somewhere

torn spire Jul 18, 2023, 6:36 PM

#

there are two possibilities

#

it isnt present
faulty cpu

#

i think the former is much more likely

hexed citrus Jul 18, 2023, 6:37 PM

#

oh I thought you meant faulty RAM for a second I was just gonna say "what I just bought this RAM like 2 months ago"

#

actually it is possible that this CPU is faulty

#

it's a Core 2 Duo E8400

torn spire Jul 18, 2023, 6:37 PM

#

why would a page not being present be due to faulty ram

ebon reef Jul 18, 2023, 6:37 PM

#

hexed citrus actually it is possible that this CPU is faulty

does it boot linux

#

does it boot windows

torn spire Jul 18, 2023, 6:37 PM

#

hexed citrus it's a Core 2 Duo E8400

how does that mean its faulty

ebon reef Jul 18, 2023, 6:38 PM

#

if it boots linux or it boots windows then its your fault

hexed citrus Jul 18, 2023, 6:38 PM

#

old, might be failing

#

idk

torn spire Jul 18, 2023, 6:38 PM

#

very unlikely

#

like so unlikely

hexed citrus Jul 18, 2023, 6:38 PM

#

I haven't checked actually

#

let me try booting into windows xp, one sec

#

but I doubt the CPU is faulty

#

more than just the memory management stuff would be going wrong

#

and probably more than just my OS would start to die (BIOS, bootloader, etc)

torn spire Jul 18, 2023, 6:40 PM

#

there is like a 0.00001% chance its not your fault

hexed citrus Jul 18, 2023, 6:40 PM

#

exactly

#

but the thing is that I don't know how it's my fault

ebon reef Jul 18, 2023, 6:40 PM

#

yeah thats

#

what a bug is

hexed citrus Jul 18, 2023, 6:40 PM

#

I don't even touch anything paging related

torn spire Jul 18, 2023, 6:41 PM

#

then you are accessing an invalid address

#

that wasnt mapped by your bootloader

hexed citrus Jul 18, 2023, 6:42 PM

#

the memory map provided by the bootloader lists that address until ~4G after it as being usable memory

ebon reef Jul 18, 2023, 6:44 PM

#

ok so if its limine then theres a small chance its a limine bug

#

what have you done to determine for sure

#

that youre accessing the address you think you are

#

because you might not be

#

assume nothing

hexed citrus Jul 18, 2023, 6:45 PM

#

- it's not a limine bug because I'm not using limine
- I am, I literally log the entire memory map, and I can see in the panic logs and the logs of the memory map, that CR2 is equal to the starting address of that region of the address space

ebon reef Jul 18, 2023, 6:46 PM

#

showe

hexed citrus Jul 18, 2023, 6:46 PM

#

one sec, it's hanging in the BIOS for some reason

warm osprey Jul 18, 2023, 6:46 PM

#

was this that one NT clone kernel?

hexed citrus Jul 18, 2023, 6:47 PM

#

nope

warm osprey Jul 18, 2023, 6:47 PM

#

my mental map of the projects of this server must be all wrong

topaz mango Jul 18, 2023, 6:48 PM

#

I remember this one as the one which was written with Bootboot and everyone was lapidating the guy for it

hexed citrus Jul 18, 2023, 6:48 PM

#

lol, sounds about right

torn spire Jul 18, 2023, 6:49 PM

#

no wonder it doesnt work

hexed citrus Jul 18, 2023, 6:50 PM

#

ebon reef showe

the logs, yes, it is in fact in the memory map
the panic screen, register of note: CR2 (also RAX, presumably that's where it dereferenced the address from)

topaz mango Jul 18, 2023, 6:51 PM

#

I may have asked this before but..

what's that weird blue line?

ebon reef Jul 18, 2023, 6:51 PM

#

possible bootboot bug

#

does bootboot guarantee that thats mapped?

hexed citrus Jul 18, 2023, 6:51 PM

#

that's it trying it's hardest to read an image from the initrd

hexed citrus Jul 18, 2023, 6:51 PM

#

ebon reef does bootboot guarantee that thats mapped?

90% sure, let me check the spec

#

yes, up to a maximum of 16GB

ebon reef Jul 18, 2023, 6:52 PM

#

bootboot gives you a lower half identity map? for shame

hexed citrus Jul 18, 2023, 6:54 PM

#

It is indeed possible that it's a bootboot bug

#

the x86_64 BIOS version isn't maintained

torn spire Jul 18, 2023, 6:55 PM

#

its really quite a simple fix

#

.!t limine

torn aspenBOT Jul 18, 2023, 6:55 PM

#

https://github.com/limine-bootloader/limine-barebones

GitHub

GitHub - limine-bootloader/limine-barebones: A simple Limine-compli...

A simple Limine-compliant kernel and build system example/template. - GitHub - limine-bootloader/limine-barebones: A simple Limine-compliant kernel and build system example/template.

topaz mango Jul 18, 2023, 6:55 PM

#

hexed citrus the x86_64 BIOS version isn't maintained

ah yes the most widely used platform isn't maintained

ebon reef Jul 18, 2023, 6:55 PM

#

hexed citrus \- it's not a limine bug because I'm not using limine \- I am, I literally log t...

also dont be getting uppity im not the guy whos been debugging the same 500 lines for the second week straight

hexed citrus Jul 18, 2023, 6:55 PM

#

yes but I'm not rewriting my kernel to add support for it, not yet

topaz mango Jul 18, 2023, 6:56 PM

#

topaz mango ah yes the most widely used platform isn't maintained

it's big brain time

torn spire Jul 18, 2023, 6:56 PM

#

hexed citrus yes but I'm not rewriting my kernel to add support for it, not yet

why not

#

this clearly doesnt work

hexed citrus Jul 18, 2023, 6:56 PM

#

topaz mango ah yes the most widely used platform isn't maintained

no, the x86_64-efi version is

ebon reef Jul 18, 2023, 6:56 PM

#

hexed citrus yes but I'm not rewriting my kernel to add support for it, not yet

if your kernel is trivial enough that switching the bootloader requires a rewrite then youre not losing much

torn spire Jul 18, 2023, 6:56 PM

#

and unless you want to go fix bootboot

topaz mango Jul 18, 2023, 6:56 PM

#

hexed citrus no, the x86_64-efi version is

ohh

torn spire Jul 18, 2023, 6:56 PM

#

ebon reef if your kernel is trivial enough that switching the bootloader requires a rewrit...

this is also a good point

hexed citrus Jul 18, 2023, 6:56 PM

#

well, not an entire rewrite

#

but I would need to rewrite tons of core stuff

#

I would need to gut the entire console driver for 1

topaz mango Jul 18, 2023, 6:57 PM

#

like what?

torn spire Jul 18, 2023, 6:57 PM

#

why would you need to do that

hexed citrus Jul 18, 2023, 6:57 PM

#

some memory stuff would have to go

#

also the SMP code

torn spire Jul 18, 2023, 6:57 PM

#

bootboot and limine's framebuffers cant be that different

#

its like 500 lines at most

#

for everything

hexed citrus Jul 18, 2023, 6:57 PM

#

oh, limine gives you (easy) direct framebuffer access, nvm, it wouldn't be that hard then

#

as long as it's RGBA

torn spire Jul 18, 2023, 6:58 PM

#

its whatever it tells you it is

hexed citrus Jul 18, 2023, 6:58 PM

#

yes but my kernel just assumes it's rgba

#

even bootboot tells you what it is

#

but it's most commonly rgba and I just never got around to abstracting it to use anything different

torn spire Jul 18, 2023, 6:58 PM

#

anyway that really doesnt sound like you need to change much at all

hexed citrus Jul 18, 2023, 6:59 PM

#

no, the rest of it is fine, since it's all abstracted around the console driver mostly, very little stuff directly accesses the framebuffer

torn spire Jul 18, 2023, 6:59 PM

#

then use limine

hexed citrus Jul 18, 2023, 6:59 PM

#

but yeah I get a lot of info from the bootboot struct

topaz mango Jul 18, 2023, 6:59 PM

#

btw "boot" means stupid in my language lol

#

so "bootboot" would mean "stupid stupid"

hexed citrus Jul 18, 2023, 6:59 PM

#

lol

#

oh yeah also the entire kernel.c, and makefile are just heavily modified code from the example C kernel, so I still need to keep the copyright banner

#

even though there's practically nothing left, so idk if it even still applies

ebon reef Jul 18, 2023, 7:01 PM

#

topaz mango btw "boot" means stupid in my language lol

what language is that

topaz mango Jul 18, 2023, 7:01 PM

#

Armenian

#

it also means thumb

torn spire Jul 18, 2023, 7:02 PM

#

stupid thumb

topaz mango Jul 18, 2023, 7:03 PM

#

it also means "obtuse" if you're talking about angles

hexed citrus Jul 18, 2023, 7:04 PM

#

so you're saying that "boot boot boot" could literally mean "stupid obtuse thumb"

ebon reef Jul 18, 2023, 7:04 PM

#

thats interesting since in english "obtuse" can mean stupid

topaz mango Jul 18, 2023, 7:04 PM

#

hexed citrus so you're saying that "boot boot boot" could literally mean "stupid obtuse thumb...

well.. yeah. but does that even make sense?

torn spire Jul 18, 2023, 7:05 PM

#

it grammatically makes sense

#

but the words together dont really make sense

ebon reef Jul 18, 2023, 7:05 PM

#

torn spire it grammatically makes sense

in the literal word for word english translation it does

hexed citrus Jul 18, 2023, 7:05 PM

#

partially, kind of odd, but not complete nonsense imo

torn spire Jul 18, 2023, 7:06 PM

#

ebon reef in the literal word for word english translation it does

i mean the fact that you are calling a thumb "stupid"

#

i guess there are some cases it could make sense

topaz mango Jul 18, 2023, 7:06 PM

#

well I mean it only makes sense if you cut through your thumb and make an obtuse angle with it

#

don't laugh but in Low German the word "butt" means "blunt, clumsy"

#

so it's an Indo-European thing ig

#

we went from discussion of a bootloader to linguistics

hexed citrus Jul 18, 2023, 7:10 PM

#

only 3.7KB of errors once I changed the linker script

#

this should be good

topaz mango Jul 18, 2023, 7:15 PM

#

so basically around 3789 characters?

hexed citrus Jul 18, 2023, 7:15 PM

#

3781, but now it doesn't even compile, because I've started doing stuff with the code itself

#

so far just commenting out stuff that would be unnecessary for a simple build

topaz mango Jul 18, 2023, 7:16 PM

#

understandable

#

good luck and keep us updated

hexed citrus Jul 18, 2023, 7:16 PM

#

yup

#

oh no

#

it's the memory

hexed citrus Jul 18, 2023, 7:39 PM

#

well I'll be damned

#

I've managed to get it to build

#

while replacing (some, but not all) of the existing bootboot functionality with limine

torn spire Jul 18, 2023, 7:57 PM

#

why would you want to do that

hexed citrus Jul 18, 2023, 7:57 PM

#

?

#

to get at least some of it working

#

I need it working before I can experiment with getting the rest working

#

currently I'm trying to fix the ISO creation script

#

i've somehow caused it to go up in flames

torn spire Jul 18, 2023, 8:01 PM

#

let me rephrase

#

how would you use limine and bootboot together

#

do you know what a bootloader is?

hexed citrus Jul 18, 2023, 8:01 PM

#

I wouldn't

#

how would I

torn spire Jul 18, 2023, 8:01 PM

#

but you just said

hexed citrus Jul 18, 2023, 8:01 PM

#

I'm migrating from bootboot to limine

torn spire Jul 18, 2023, 8:01 PM

#

you replaced part of it

#

as if that was like

#

your goal

hexed citrus Jul 18, 2023, 8:02 PM

#

part of the code

#

part of the kernel code that uses stuff that bootboot provided

#

not part of bootboot

#

lol

torn spire Jul 18, 2023, 8:02 PM

#

you didnt really make that clear

#

"bootboot functionality" i would take as being using bootboots protocol etc

hexed citrus Jul 18, 2023, 8:03 PM

#

ah, my bad

#

yeah, i meant as in part of the kernel code that uses the bootboot struct that it provides

#

heyyy it works!

#

now I just need to make a config file and it should work

#

that... doesn't quite look right

#

this is gonna be fun

#

no terminal output

#

it randomly resets

#

but if I cli; hlt as soon as the kernel start, it properly hangs

#

- the kernel loads ✅
- the kernel starts executing properly ✅
- literally anything else ❓

brave maple Jul 18, 2023, 8:25 PM

#

-no-shutdown -no-reboot, look at registers

hexed citrus Jul 18, 2023, 8:28 PM

#

now it doesn't even print anything 😔
hang on, let me try to figure out roughly where it's dying by just dumping a while (true) {} somewhere

#

aha

#

it dies extremely early on, trying to initialize SSE

brave maple Jul 18, 2023, 8:29 PM

#

For what do you need SSE in your kernel?

hexed citrus Jul 18, 2023, 8:30 PM

#

I don't, I just make use of it for performance, because it's a 64-bit kernel, and any AMD64 compatable CPU will have at least SSE 1 & 2

brave maple Jul 18, 2023, 8:31 PM

#

hexed citrus now it doesn't even print anything 😔 hang on, let me try to figure out roughly ...

Look up the RIP and use addr2line

hexed citrus Jul 18, 2023, 8:32 PM

#

no like, QEMU doesn't tell me when it resets, it prints stuff about CPU reset on initial bootup, but it doesn't print anything after that

#

I suppose I could just use gdbstub though

brave maple Jul 18, 2023, 8:33 PM

#

hexed citrus that... doesn't quite look right

Your qemu keeps resetting right?

hexed citrus Jul 18, 2023, 8:33 PM

#

yup

brave maple Jul 18, 2023, 8:33 PM

#

Include -no-shutdown and -no-reboot in your qemu flags

hexed citrus Jul 18, 2023, 8:33 PM

#

I do

#

now it doesn't

#

but like I said, it doesn't print info about what happened to the CPU

#

so I can't lookup RIP

#

that's why I'm setting up gdbstub

brave maple Jul 18, 2023, 8:34 PM

#

qemu ?commandline? => info registers

hexed citrus Jul 18, 2023, 8:34 PM

#

ah, the qemu monitor yeah

brave maple Jul 18, 2023, 8:34 PM

#

yup

hexed citrus Jul 18, 2023, 8:34 PM

#

too late ig

#

Thread 1 received signal SIGQUIT, Quit.
0xffffffff8000007a in .NoSSE3 ()

#

I wonder what's going on there

#

that's just checking if it supports any of the additional ones

brave maple Jul 18, 2023, 8:35 PM

#

I‘d really suggest not using SSE in the kernel, and just not do any floating point operations

hexed citrus Jul 18, 2023, 8:35 PM

#

it doesn't actually use anything more than SSE2

hexed citrus Jul 18, 2023, 8:36 PM

#

brave maple I‘d really suggest not using SSE in the kernel, and just not do any floating poi...

why?

#

it doesn't hurt anything, and since it's already a 64-bit only kernel, there are 0 cases where it's not supported

brave maple Jul 18, 2023, 8:36 PM

#

Its up to you of course, though you‘d always have to save sse states too in context switches

brave maple Jul 18, 2023, 8:38 PM

#

hexed citrus I wonder what's going on there

set a breakpoint to your kernel entry then, single step through your code if you want to find out

hexed citrus Jul 18, 2023, 8:39 PM

#

yeah, this kernel is nowhere near close to being able to run anything in userspace, so that doesn't matter in the slightest yet

#

if I ever get to that point, and it becomes a problem, then I can just disable it

#

(and make minor modifications to remove floating point stuff)

brave maple Jul 18, 2023, 8:40 PM

#

you can just avoid using floats, what do you need them for anyway?

hexed citrus Jul 18, 2023, 8:40 PM

#

¯_(ツ)_/¯

#

idk

#

I'm not going to go check rn

#

but again, I mainly just enable it so that GCC can optimize the code with it

torn spire Jul 18, 2023, 8:41 PM

#

hexed citrus it doesn't actually use anything more than SSE2

why are you using sse

hexed citrus Jul 18, 2023, 8:42 PM

#

again, I can't see how that's a bad thing, a little extra overhead for context switches sure, but that's not going to even matter for an extremely long time yet

torn spire Jul 18, 2023, 8:43 PM

#

"it doesnt matter yet" is a bad way to write code

brave maple Jul 18, 2023, 8:43 PM

#

it seems to matter now, as your SSE code is broken anyway seemingly

hexed citrus Jul 18, 2023, 8:44 PM

#

it isn't
I think limine just enabled it already, since I've seen this same behavior when running under SMP systems, and not properly redirecting the other cores elsewhere (causing it to init SSE twice)

torn spire Jul 18, 2023, 8:45 PM

#

i dont think limine will enable sse

brave maple Jul 18, 2023, 8:45 PM

#

limine doesn‘t afaik

torn spire Jul 18, 2023, 8:46 PM

#

but anyway i would just -mno-sse(1,2...)

hexed citrus Jul 18, 2023, 8:46 PM

#

huh, idk then

brave maple Jul 18, 2023, 8:46 PM

#

try removing sse, maybe something else is really broken

hexed citrus Jul 18, 2023, 8:46 PM

#

no, I'm keeping SSE, I'm going to get it working, I don't get why yall are so against it, it's fine

torn spire Jul 18, 2023, 8:47 PM

#

because every single time you go into the kernel you would need to save another 16 or whatever registers

#

and restore them of course

hexed citrus Jul 18, 2023, 8:47 PM

#

yes, but I never even leave the kernel yet

#

nor am I anywhere close to that

torn spire Jul 18, 2023, 8:48 PM

#

like i said, "it doesnt matter yet" isnt a good way to go about things

hexed citrus Jul 18, 2023, 8:48 PM

#

again, when I get to that point, if I see that the performance hit of context switches is genuinely not worth it, then I'll disable it

brave maple Jul 18, 2023, 8:48 PM

#

torn spire because every single time you go into the kernel you would need to save another ...

And between kernel „tasks“ too, not only to some user programs but the entire kernel requires saving those registers

torn spire Jul 18, 2023, 8:48 PM

#

yep any context switch

#

which would be happening many many times per second

#

and you are effectively doubling the delay of each one by using sse in the kernel

hexed citrus Jul 18, 2023, 8:49 PM

#

yeah, I literally don't do any context switching at all yet, and once again, I won't for a while

torn spire Jul 18, 2023, 8:49 PM

#

or maybe even more than doubling

brave maple Jul 18, 2023, 8:49 PM

#

hexed citrus again, when I get to that point, if I see that the performance hit of context sw...

You don‘t get performance really

#

Right now atleast

torn spire Jul 18, 2023, 8:49 PM

#

hexed citrus yeah, I literally don't do any context switching at all yet, and once again, I w...

saying that is a bit like saying "ill just use the weakest possible material for the walls of my house since its cheaper, ill worry about the roof crushing them under pressure later"

#

just because you arent doing something now, doesnt mean you cant think about the effects that thing may have

hexed citrus Jul 18, 2023, 8:50 PM

#

no, it's more like "I'll put up these thin wooden beams as the walls, because there is no roof to crush them, and there won't be for a long time"

torn spire Jul 18, 2023, 8:50 PM

#

yeah that sounds like a bad idea to me

brave maple Jul 18, 2023, 8:51 PM

#

brave maple You don‘t get performance really

I don‘t think you‘re doing heavy math, making use of SIMD or general FP operations

hexed citrus Jul 18, 2023, 8:51 PM

#

I actually do a decent amount with floats, not a ton but some
and I've seen in the disassembly before that GCC does a decent amount of optimizations with them

#

again

brave maple Jul 18, 2023, 8:52 PM

#

Again, why floats in your kernel?

hexed citrus Jul 18, 2023, 8:52 PM

#

if that becomes a problem, then I'll disable it

brave maple Jul 18, 2023, 8:52 PM

#

Where do you use them?

hexed citrus Jul 18, 2023, 8:52 PM

#

idk, I know that I do in some places, mainly stuff with logging

torn spire Jul 18, 2023, 8:52 PM

#

logging what

brave maple Jul 18, 2023, 8:52 PM

#

timestamps?

#

Just use two ints and you get the same result

hexed citrus Jul 18, 2023, 8:53 PM

#

i don't know... I haven't worked on this project for many months, and I just came back to it at most a week ago
I don't remember everything about it

#

I just know that I DO use floats

#

and that the performance increase from SSE WAS noticeable on my Core 2 Duo Optiplex from 2007 (my main testing machine)

brave maple Jul 18, 2023, 8:55 PM

#

do you use kvm or tcg?

hexed citrus Jul 18, 2023, 8:55 PM

#

I use QEMU/KVM for quick tests, but for real hardware, I use that optiplex that I just mentioned above

brave maple Jul 18, 2023, 8:56 PM

#

Ah I see

hexed citrus Jul 18, 2023, 8:57 PM

#

oh, it looks like limine actually does enable SSE

#

but the next thing that my code does is to setup serial I/O

#

and that breaks because conventional memory isn't mapped when it tries to access the BIOS Data Area

#

so it page faults

brave maple Jul 18, 2023, 9:01 PM

#

hexed citrus and that breaks because conventional memory isn't mapped when it tries to access...

HHDM exists in limine, though what do you mean with serial i/o?

hexed citrus Jul 18, 2023, 9:01 PM

#

like it tries to initialize the serial port

brave maple Jul 18, 2023, 9:01 PM

#

Thats portio

#

not memory

hexed citrus Jul 18, 2023, 9:02 PM

#

yes

#

I know

#

but I try to access the BDA

#

that way I can get the I/O port from the BIOS

#

because some map it to something weird

torn spire Jul 18, 2023, 9:02 PM

#

are you being booted by bios?

#

or uefi

hexed citrus Jul 18, 2023, 9:03 PM

#

as a fallback if it gives us 0, it just tries 0x3F8

hexed citrus Jul 18, 2023, 9:03 PM

#

torn spire are you being booted by bios?

BIOS

torn spire Jul 18, 2023, 9:03 PM

#

are you sure

hexed citrus Jul 18, 2023, 9:03 PM

#

yes

#

I am absolutely sure

#

QEMU loads up SeaBIOS

#

but yeah I don't do anything limine specific to access it if that's the issue

#

I just try to read from low memory

brave maple Jul 18, 2023, 9:04 PM

#

hexed citrus as a fallback if it gives us 0, it just tries 0x3F8

Then just use the fallback for now, until you sort out your paging

hexed citrus Jul 18, 2023, 9:04 PM

#

I don't even use paging

#

I just give out physical addresses

brave maple Jul 18, 2023, 9:04 PM

#

… if you want to access memory it has to be mapped

hexed citrus Jul 18, 2023, 9:05 PM

#

yes

#

I'm aware

brave maple Jul 18, 2023, 9:05 PM

#

So?

hexed citrus Jul 18, 2023, 9:05 PM

#

in bootboot everything was already mapped

#

I need to manually map everything in Limine?

brave maple Jul 18, 2023, 9:05 PM

#

Limine has a HHDM (not 100% sure on low mem ident mappings)

torn spire Jul 18, 2023, 9:05 PM

#

limine identity maps all memory

#

and also all memory to addr + hhdm

#

and your kernel

brave maple Jul 18, 2023, 9:06 PM

#

Yup

hexed citrus Jul 18, 2023, 9:06 PM

#

hang on, let me go look up how that works, because I have no idea what hhdm even is

brave maple Jul 18, 2023, 9:06 PM

#

Higher half data mapping

torn spire Jul 18, 2023, 9:06 PM

#

direct map*

brave maple Jul 18, 2023, 9:06 PM

#

oh right

hexed citrus Jul 18, 2023, 9:07 PM

#

oh so everything is mapped to the higher half of the address space

brave maple Jul 18, 2023, 9:07 PM

#

Yes

hexed citrus Jul 18, 2023, 9:07 PM

#

sigh that's gonna require a lot of rewriting

torn spire Jul 18, 2023, 9:07 PM

#

every physaddr is mapped to itself + hhdm_offset

hexed citrus Jul 18, 2023, 9:08 PM

#

is there any hack I can do to just make limine map everything at offset 0, or am I actually going to need to go rewrite anything that uses low memory

brave maple Jul 18, 2023, 9:08 PM

#

Hhdm is the first 4gb of memory identity mapped and possibly any other memory map entries above those 4gb mapped too

torn spire Jul 18, 2023, 9:08 PM

#

hexed citrus is there any hack I can do to just make limine map everything at offset 0, or am...

you want to do that anyway

#

or well

#

you will want to in the future but i guess you dont really care about the future

brave maple Jul 18, 2023, 9:10 PM

#

If you want to have some userspace stuff running in lower memory you can just keep everything in your hhdm and access that

hexed citrus Jul 18, 2023, 9:10 PM

#

I care about the future if it lines up with what I'm doing right now, or if what I'm doing now will make the future absurdly harder

#

that's pretty much it

torn spire Jul 18, 2023, 9:10 PM

#

brave maple If you want to have some userspace stuff running in lower memory you can just ke...

this is the ideal method

hexed citrus Jul 18, 2023, 9:11 PM

#

crap

#

I mean yeah I know that userspace stuff goes in lower memory

#

but I'm talking about really low memory

#

like the 640K of conventional memory

#

(where the BDA lives)

brave maple Jul 18, 2023, 9:11 PM

#

limine also maps from 0x1000 onward (virtually)

torn spire Jul 18, 2023, 9:11 PM

#

its good practice to separate them exactly in half

hexed citrus Jul 18, 2023, 9:11 PM

#

that's what I need access to right now

torn spire Jul 18, 2023, 9:12 PM

#

use hhdm for that

hexed citrus Jul 18, 2023, 9:12 PM

#

sigh welp time to go rewrite anything that uses it then.....

torn spire Jul 18, 2023, 9:13 PM

#

make sure you use the hhdm offset request

brave maple Jul 18, 2023, 9:13 PM

#

HHDM address is not fixed, maybe keep that in mind

hexed citrus Jul 18, 2023, 9:13 PM

#

yeah I know

#

I'm requesting it, then setting the bda pointer address based on it

#

aand now it jumps to null somewhere

#

great

#

after a small amount of testing in GDB, yeah, that should work

(gdb) p/x *(hhdm.response->offset + 0x400)
$8 = 0x3f8

#

yup, and the bda looks valid

#

ok cool so it makes it through initThings

#

now I just need to make TTY_Init() actually set up a TTY

hexed citrus Jul 18, 2023, 9:36 PM

#

yoo I can make a line

#

yup, here's a case where I do floating point math

#

calculating the width and height o the terminal

#

TTY_Width     = ((fb->width / font->width) * 0.885);
TTY_Height    = ((fb->height / font->height) * 0.99);

#

and it dies

#

because SSE isn't initialized

#

yup, no other reason that this would have died 0xffffffff80006e93 <+99>: pxor %xmm0,%xmm0

#

yoo SERIAL TEST!!!!!!!

#

serial works

#

and I've gotten SSE working too

#

alright so serial works

#

but my crappy framebuffer console doesn't

#

and the limine terminal stuff is deprecated

#

what to do.....

hexed citrus Jul 18, 2023, 10:18 PM

#

as far as I see it, I have 3 options here:

#

- just use limine terminal and disable the deprecation warnings (will probably end up stuck on an old version of limine then once it does finally get removed)
- fix my broken framebuffer terminal (it's slow, and would make booting take way longer)
- have no early boot console / only serial (panic woul suck balls since it just prints the message down serial, no debugging info)

violet hearth Jul 18, 2023, 10:21 PM

#

flanterm

hexed citrus Jul 18, 2023, 10:21 PM

#

doesn't work for early boot

#

no allocator

#

that was my plan all along for after the allocator starts

#

but currently it doesn't even get that far

violet hearth Jul 18, 2023, 10:22 PM

#

bump allocator

hexed citrus Jul 18, 2023, 10:23 PM

#

wdym, like change my existing allocator to a bump allocator? I don't see how that would help, since it still isn't initialized until later in the boot process

hexed citrus Jul 18, 2023, 10:38 PM

#

yeah I can't get this one working

#

it's totally dead

hexed citrus Jul 18, 2023, 11:04 PM

#

bruh not even flanterm works what

#

yeah it just dies

#

I think I'm just gonna need to use limine terminal

#

I honestly don't know how else I could do this

brave maple Jul 19, 2023, 5:00 AM

#

hexed citrus wdym, like change my existing allocator to a bump allocator? I don't see how th...

Flanterm includes one

hexed citrus Jul 19, 2023, 5:16 AM

#

yeah I probably don't want 2 different allocators

#

unless I can deactivate the bump allocator as soon as I've verified that the current one works

#

it just seems like it would cause a ton of issues

brave maple Jul 19, 2023, 11:39 AM

#

hexed citrus unless I can deactivate the bump allocator as soon as I've verified that the cur...

You can do that

brave maple Jul 19, 2023, 11:40 AM

#

hexed citrus it just seems like it would cause a ton of issues

if i remember correctly the bump allocator just makes use of a compile time arena

#

just a char array

hexed citrus Jul 24, 2023, 11:26 PM

#

after an extremely long time of debugging, I've finally done it

#

Techflash OS finally boots enough on Limine to the point where I can see it die

#

#

only took completely rewriting the entire console driver, and having my friend send a perl script to convert a 16-bit VGA font into my custom format (C code)

#

oh, and fixing a weird bug in my PIC initialization routines

#

it was reading the address 0x04 rather than the value $0x04

#

but now, oddly enough, it dies in the ISRHandler

#

because reading the code at a proper higher-half address causes what I can only assume is a page fault (exception handlers aren't loaded yet (?))

#

wait no, they are

// Initialize some exception handlers
initExceptions();
    
// Initialize the physical memory manager
PMM_Init();

#

but yeah, this is what I got when I tried to check wtf was going on with GDB

Thread 1 received signal SIGQUIT, Quit.
0xffffffff80005259 in ISRHandler (regs=<unavailable>)
    at hardware/CPU/interrupts/ISR.c:27
27      void ISRHandler(registers_t* regs) {
(gdb) disas
Dump of assembler code for function ISRHandler:
   0xffffffff80005251 <+0>:
Cannot access memory at address 0xffffffff80005251

#

which is just.... weird

#

I also don't believe that it's possible the Limine isn't mapping the entire kernel or some BS like that, since it's only 239K

hexed citrus Jul 25, 2023, 3:24 AM

#

sigh

#

complicating this EVEN MORE, when I go to load it now on real hardware, Limine just doesn't load

worldly drift Jul 25, 2023, 3:57 AM

#

explain?

#

what makes you think it "doesn't load"?

hexed citrus Jul 25, 2023, 3:57 AM

#

sure, gimme a sec, recording a video

worldly drift Jul 25, 2023, 3:57 AM

#

okay

hexed citrus Jul 25, 2023, 3:58 AM

#

worldly drift what makes you think it "doesn't load"?

as in the keyboard stops functioning, and the BIOS stays on screen forever, and the only thing I do is turn the PC off

worldly drift Jul 25, 2023, 3:58 AM

#

well that's odd

#

but it doesn't necessarily seem like a Limine issue, but we'll see

hexed citrus Jul 25, 2023, 4:03 AM

#

alright, video recorded, compressing it so that I can actually send it now

#

btw I did this by dding an ISO to a USB HDD, I do have some spare CDs to test it like that too, but it would be a pain, because I would need to hijack SATA power from something else if I wanted to hook up an optical drive to this thing

#

worldly drift Jul 25, 2023, 4:10 AM

#

are you sure that enter key even works

hexed citrus Jul 25, 2023, 4:10 AM

#

yes

worldly drift Jul 25, 2023, 4:10 AM

#

it doesn't seem to

hexed citrus Jul 25, 2023, 4:10 AM

#

the HDD light flashes every time I do

worldly drift Jul 25, 2023, 4:10 AM

#

hm

#

is this BIOS?

hexed citrus Jul 25, 2023, 4:11 AM

#

yup

worldly drift Jul 25, 2023, 4:11 AM

#

can you boot GRUB?

#

like this?

hexed citrus Jul 25, 2023, 4:11 AM

#

it could boot grub when I was using it in the exact same setup before

#

I bet if I use an old version that uses GRUB, it would still work

#

it's exactly how i used to boot my os

worldly drift Jul 25, 2023, 4:13 AM

#

well that's certainly odd

#

first thing it makes me thing you didn't properly install Limine on the drive

#

how did you install it?

hexed citrus Jul 25, 2023, 4:13 AM

#

it works fine in QEMU when loading the ISO

#

and I wrote it to the drive with sudo dd if=bin/TFOS_ISO.iso of=/dev/sdb bs=512 status=progress

worldly drift Jul 25, 2023, 4:14 AM

#

well that's not really what i have asked

#

if i am asking questions it is for a specific reason

#

you see, you could be loading the QEMU ISO using -cdrom which would go through the eltorito boot path which is different compared from the hdd boot path which the usb drive would take

#

if you would please answer the questions instead of trying jumping ahead that'd be nice for everyone :)

hexed citrus Jul 25, 2023, 4:15 AM

#

ah, sorry

hexed citrus Jul 25, 2023, 4:17 AM

#

hexed citrus and I wrote it to the drive with `sudo dd if=bin/TFOS_ISO.iso of=/dev/sdb bs=512...

but yeah, thats how ^

worldly drift Jul 25, 2023, 4:18 AM

#

but that is still not the question i asked

#

let me rephrase

hexed citrus Jul 25, 2023, 4:18 AM

#

?

worldly drift Jul 25, 2023, 4:18 AM

#

how do you put Limine on the ISO

#

procedure

hexed citrus Jul 25, 2023, 4:18 AM

#

ohhh

#

sorry, misunderstood, let me go get the script I use

#

this is the portion which downloads the Limine files, and creates the ISO image

if ! [ -f limine/limine-bios.sys ]; then
    wget https://github.com/limine-bootloader/limine/raw/v5.x-branch-binary/limine-bios.sys -O limine/limine-bios.sys
fi
if ! [ -f limine/limine-uefi-cd.bin ]; then
    wget https://github.com/limine-bootloader/limine/raw/v5.x-branch-binary/limine-uefi-cd.bin -O limine/limine-uefi-cd.bin
fi
if ! [ -f limine/limine-bios-cd.bin ]; then
    wget https://github.com/limine-bootloader/limine/raw/v5.x-branch-binary/limine-bios-cd.bin -O limine/limine-bios-cd.bin
fi

cp -r limine/ isodir/
xorriso -as mkisofs -b limine/limine-bios-cd.bin --no-emul-boot --boot-load-size 4 --boot-info-table --efi-boot limine/limine-uefi-cd.bin --efi-boot-part --efi-boot-image --protective-msdos-label isodir -o bin/TFOS_ISO.iso

worldly drift Jul 25, 2023, 4:20 AM

#

you're not running limine bios-install <image>

#

you're welcome

hexed citrus Jul 25, 2023, 4:21 AM

#

ohh, thanks, I'll add that

hexed citrus Jul 25, 2023, 5:42 AM

#

well, now it boots on real hardware, but the entire reason that I was trying to run it on real hardware (intentionally slow down the TTY so I could see why the scrolling totally broke everything) didn't work. My new implementation is actually moderately good, so it's too fast for me to see what's going on 😔

#

I guess I could manually step through it with gdb

worldly drift Jul 25, 2023, 6:03 AM

#

or you can add explicit delays

hexed citrus Jul 25, 2023, 6:06 AM

#

it's so early in the boot process (it has just barely initialized the PIC), so it has no timer

torn spire Jul 25, 2023, 6:55 AM

#

hexed citrus it's so early in the boot process (it has just barely initialized the PIC), so i...

then move it later

hexed citrus Jul 25, 2023, 6:56 AM

#

I can't just move the TTY scrolling routines later.....

#

but yeah it looks like it's just entirely broken

#

yeah that.... doesn't quite look right

#

might just revert to the old code and see if i can fix it

#

whatever

#

I'm getting a huge headache

#

I'm just gonna go to bed

#

will keeping messing with it tomorrow

#

but I honestly have no idea what's wrong

torn spire Jul 25, 2023, 7:24 AM

#

hexed citrus I can't just move the TTY scrolling routines later.....

why not? just initialize the timer then the terminal

hexed citrus Jul 25, 2023, 7:25 AM

#

Because this is the very early boot terminal

torn spire Jul 25, 2023, 7:25 AM

#

okay?

hexed citrus Jul 25, 2023, 7:25 AM

#

Specifically for, we'll, when nothing is initialized yet

torn spire Jul 25, 2023, 7:25 AM

#

whats the point of that

#

i dont init the terminal until very late(after apic, idt, gdt, tss, pmm, vmm, kernel heap, hpet, apic timer and probably a few others)

hexed citrus Jul 25, 2023, 7:26 AM

#

Because it literally never even gets to initlizing the memory map yet

#

It dies somehow because the code isn't mapped

#

But now is too late

#

I'm going to bed

grim cedar Jul 25, 2023, 3:20 PM

#

worldly drift you're not running `limine bios-install <image>`

see? it's too difficult and complicated, bzt really has a point

worldly drift Jul 25, 2023, 3:21 PM

#

oh yeah

#

if you cannot run it without reading the documentation your software sucks

#

my bad

#

time to delete limine-bootloader/limine in shame

brave maple Jul 25, 2023, 3:44 PM

#

hexed citrus it's so early in the boot process (it has just barely initialized the PIC), so i...

just do nothing 1000000000 times

#

No need for a timer to just delay for some time

hexed citrus Jul 25, 2023, 4:38 PM

#

brave maple just do nothing 1000000000 times

that's what I ended up doing lol

#

well

#

before reverting the code

hexed citrus Jul 25, 2023, 4:39 PM

#

hexed citrus

it's how I got ^^^

hexed citrus Jul 25, 2023, 7:47 PM

#

well... it's not "good", but it's at least better than I thought

#

ah, that's why

#

it has no ANSI

#

?

#

maybe

#

actually idk

#

I might just need to move the ANSI and cursor handling into the buffer wrapper rather then

hexed citrus Jul 25, 2023, 11:08 PM

#

yooo

#

somehow the [ on the left isn't showing up though

hexed citrus Jul 27, 2023, 3:17 AM

#

I'm debating just restarting this

#

I'm also considering doing it in rust this time, I've heard tons of good stuff about it, and seen other people using it in their projects

#

never actually tried it

#

but I've heard that it's blazingly fast, and very safe

loud nimbus Jul 27, 2023, 3:20 AM

#

hexed citrus I'm also considering doing it in rust this time, I've heard tons of good stuff a...

I would not learn a new language doing osdev

hexed citrus Jul 27, 2023, 3:20 AM

#

idk

#

not sure on it yet

#

I'm for sure restarting the project

#

not sure on if I want to do it in rust or just stick with C

loud nimbus Jul 27, 2023, 3:21 AM

#

nothing wrong with restarting, I think its a good idea from time to time. As long as you make use of the previous experience

hexed citrus Jul 27, 2023, 3:22 AM

#

yup, while in the process of making tfos (this iteration, #3), I learned a lot about my personal style of coding, and gained a decent amount of extra experience with C in general

#

not letting it all go to waste

#

I'm definitely archiving this branch so I can reflect on it

#

and I'm obv keeping the really basic stuff, unrelated to the kernel, like the build system and config files

#

this isn't going to be a completely new project

#

just wiping the slate clean for a better foundation

#

and I'm gonna try to get less ahead of myself this time

#

like trying to make progress bar on the boot screen before even getting interrupts, or like trying to make a crappy strcmp based shell built into the kernel

#

although one thing that I might actually keep is the console driver, after a few minor tweaks

#

it's actually pretty robust

#

it needs a bit of work, and some bug tests for sure

#

actually.

#

now I'm not really sure

#

ok here's my plan
- definetly save all of this code, as horrible as some of it is, a lot of it is actually pretty useful stuff, and the bad stuff can serve as a good example of what NOT to do

#

- start learning basic rust, I don't want to get too deep into it yet, since osdev on it is going to be pretty different, since there's no standard library

#

- start writing the kernel again in rust once I've gotten a good grasp on the fundamentals of rust, and how it can mix with existing C/asm code

ebon reef Jul 27, 2023, 3:35 AM

#

hexed citrus not sure on if I want to do it in rust or just stick with C

the language wasn't the problem

#

rust will ultimately f rust rate u

#

more than continuing/rewriting in C would have

hexed citrus Jul 27, 2023, 3:36 AM

#

yeah, of course, there's nothing wrong with C

#

I love C

#

but it sure has it's issues

#

nothing's stopping you from doing really dumb stuff

#

(which is ultimately why my OS sucks)

ebon reef Jul 27, 2023, 3:36 AM

#

if I can write a nice kernel in dragonfruit anyone can write a nice kernel in C

hexed citrus Jul 27, 2023, 3:36 AM

#

not a direct issue of C

#

rather an issue of me doing dumb things and nothing's stopping me

#

rust could help that

ebon reef Jul 27, 2023, 3:37 AM

#

Rust doesn't stop you from dumbly structuring it or dumbly logic bugging it

hexed citrus Jul 27, 2023, 3:37 AM

#

no, of course not

#

that's the job of the programmer to catch, and in some cases, static analyzers

#

but I rarely make massive logic bugs like that, and when I do, I usually catch them

#

am I saying that never happens? no, of course

#

but rust could prevent the really stupid stuff

#

idk

#

I'm gonna try out rust a bit, if i find that it really doesn't help, then I'll give up

#

yeah I'm not really a fan of this tbh
I think I'll just use C

#

but i would preferably like to set up some kind of testing to ensure that I'm not doing really dumb stuff

#

a dream that I always had was to be able to have something that sits on the other end of the QEMU serial port, and gives the OS things to test
the OS would spit back out the raw results (not just a yes or no, to prevent issues with the OS fucking it up)
and it would give it a list of just random junk to test, like hammer the allocator and make sure it's returning valid addresses, or abuse printf and see if I can get it to fail, etc

#

obv that won't work for a very long time

#

but some kind of way to test it for simple issues, from outside of the OS itself would be really nice

hexed citrus Jul 27, 2023, 5:12 AM

#

ok well what started out as making a new kernel actually turned into making a new build system

#

as it turns out, good old
./clean.sh; ./all.sh --no-pull; ./usb.sh /dev/sdd just wasn't gonna cut it

#

so I'm now cramming all of this into a makefile

torn spire Jul 27, 2023, 7:24 AM

#

hexed citrus but i would preferably like to set up some kind of testing to ensure that I'm no...

then write tests

distant fox Jul 27, 2023, 1:31 PM

#

hexed citrus so I'm now cramming all of this into a makefile

Just use cargo

hexed citrus Jul 28, 2023, 3:57 AM

#

welp, first steps ig
the makefile is producing a kernel (currently just halts, and it doesn't even get put in the right place for limine to see it)
and it can make an ISO with limine, and do shit with mcopy in order to put all of limine-uefi-cd.bin's files directly on the ISO filesystem to make xorriso happy

hexed citrus Jul 28, 2023, 11:30 PM

#

first stages of kernel rewrite are beginning

#

things are much more organized this time

ebon reef Jul 28, 2023, 11:40 PM

#

write design docs before working on major components

#

why?

#

its cooler
it helps u think things through

#

although i dont follow my own advice and i only did that with my IPC system

#

but i regret that somewhat

hexed citrus Jul 28, 2023, 11:45 PM

#

I do that to a limited degree
before I actually start writing code, I'll think of "ok, what exactly am I trying to accomplish with this specific block of code"
I agree thinking things through is good 👍... to a certain extent
I think overly planning stuff out, especially simple stuff isn't necessarily a good idea, it often leads to over engineering what should really just be a simple function
Now that part doesn't always apply, when doing really complicated stuff, then a full step by step plan is a great idea, e.g. memory manager, VFS, etc
for a very beginnings of the boot process (enabling serial for logging, then enabling SSE), like what I'm working on right now, you don't really need much of a plan for that, just a rough outline of what you're trying to accomplish, not a full plan of exactly how you're planning to accomplish them

torn spire Jul 28, 2023, 11:46 PM

#

memory manager doesnt need a big design

#

apart from maybe how the kernel provides memory to a process

hexed citrus Jul 28, 2023, 11:47 PM

#

eh, kind of on the edge, depending on how you design it, it could be complicated, or it could not be

#

but it's one case where a plan does indeed help

torn spire Jul 28, 2023, 11:48 PM

#

i dont really see where you have anywhere to design anything

#

pmm, vmm, kernel heap

#

all freelist or allocator of your choice

ebon reef Jul 28, 2023, 11:50 PM

#

torn spire memory manager doesnt need a big design

i beg to differ

hexed citrus Jul 29, 2023, 3:14 AM

#

well I finally got serial working, and discovered 2 bugs:

I forgot to add the success case for enabling SSE, so it always fails
I forgot that it uses KASLR, so it dies trying to load the address of the "failed" string

but yeah now it does this
No SSE Support!! Hanging...

brave maple Jul 29, 2023, 7:04 AM

#

torn spire pmm, vmm, kernel heap

VMM alone does need some thought

torn spire Jul 29, 2023, 7:07 AM

#

brave maple VMM alone does need some thought

just pick a good algorithm and implement it

ebon reef Jul 29, 2023, 7:14 AM

#

torn spire just pick a good algorithm and implement it

have u done that

torn spire Jul 29, 2023, 7:25 AM

#

yes

ebon reef Jul 29, 2023, 7:25 AM

#

do u have it up somewhere

torn spire Jul 29, 2023, 7:26 AM

#

are you going to try to discredit my statement because of bad code or something

#

but sure

#

let me find the file

ebon reef Jul 29, 2023, 7:26 AM

#

no im going to judge whether you even know what a vmm is

torn spire Jul 29, 2023, 7:26 AM

#

okay

#

https://github.com/solar-mist/ralos/blob/old/kernel/src/mm/vmm.cpp

#

its a bit messy

ebon reef Jul 29, 2023, 7:33 AM

#

ya so

#

u think its easy because youve never done a nontrivial one

#

pretty much it

torn spire Jul 29, 2023, 7:34 AM

#

what is there that i havent done that is nontrivial

ebon reef Jul 29, 2023, 8:00 AM

#

file mapping, shared memory, page cache, copy-on-write, page swapping

#

in no particular order

#

these are "simple" vmm things

distant fox Jul 29, 2023, 12:22 PM

#

A good vmm is arguably the hardest part of a kernel to write

loud nimbus Jul 29, 2023, 12:29 PM

#

not as hard as good docs 😔

distant fox Jul 29, 2023, 12:33 PM

#

True

hexed citrus Jul 29, 2023, 6:05 PM

#

currently fixing up all of the asm so that it can be position independant

hexed citrus Jul 29, 2023, 6:21 PM

#

and just noticed that my serial code doesn't work

#

fun

#

serial!

#

now I need to plan out some stuff that i should do next, I'm thinking the big ones:
- PMM
- Terminal
- Parse ACPI tables
- SMP

hexed citrus Jul 29, 2023, 6:50 PM

#

I already had put the SMP request in there for some reason, so I figured why not just print the info

#

also, my fairly bad plan for handling SMP:
- the secondary CPUs wait, halted, until waken up by an interrupt
- when awakened, they'll check smpFuncAddrs[myCoreNumber] and see if they have anything to do
- if yes, they go do it, NULL out the address, and go back to sleep. if they don't have anything to do, then they go back to sleep instantly

#

I'll still need to look into if any of that is even possible though

#

never really handled SMP yet

ebon reef Jul 29, 2023, 6:57 PM

#

You're supposed to schedule threads on the cores

brave maple Jul 29, 2023, 6:57 PM

#

hexed citrus never really handled SMP yet

I let the scheduler schedule tasks after initializing the kernel, so exactly what you do on your main core you do on the other cores too

hexed citrus Jul 29, 2023, 6:59 PM

#

yeah I've never really handled scheduling either, my kernel has always just been a giant block of initializing stuff, and then spinlock forever (unless you count the shitty shell I made where the user could... I don't even remember anymore, I think they could reboot?)

#

and yeah that was terrible, so I killed it

#

I think I'm gonna go do some research

hexed citrus Jul 29, 2023, 7:43 PM

#

alright, I think I've decided how I want to set it up

ebon reef Jul 29, 2023, 7:46 PM

#

hexed citrus yeah I've never really handled scheduling either, my kernel has always just been...

Not what spinlock means

hexed citrus Jul 29, 2023, 7:46 PM

#

while (true) {}... unless I'm mistaken this is a spinlock, just with no way to.. well.. unlock it

#

hence, "forever"

cinder hamlet Jul 29, 2023, 7:52 PM

#

you cant call it a spinlock because there is no lock its just an infinite loop

ebon reef Jul 29, 2023, 7:52 PM

#

hexed citrus `while (true) {}`... unless I'm mistaken this is a spinlock, just with no way to...

It's a spinlock if you're an idiot

hexed citrus Jul 29, 2023, 7:53 PM

#

yeah, I see that now that I actually think about it

hexed citrus Jul 29, 2023, 8:01 PM

#

hexed citrus alright, I think I've decided how I want to set it up

but yeah, my plan is this
the console driver will have a configurable "write" function, which tells it how it's supposed to actually write characters, like to the framebuffer, VGA text mode, serial, port E9, etc
when the kernel initializes, it will have the following logic:

- Was serial tested to work?
    - YES: Set the write function to log to serial
    - NO: Set it to a stub that writes everything to a buffer in RAM, if the system comes up successfully, dump the buffer to the framebuffer
- Call `ARCH_Init();`, a wrapper that calls the appropriate function for the platform (e.g. `x86_64_Init();`, `i386_Init()`, `AArch64_Init()`, etc)
    - Since x86_64 is the only platform supported currently, so nothing else really makes a difference
    - Init GDT, IDT, Excpetion handlers
    - Init anything else that belongs here that I'm not thinking of atm
- `PMM_Init()`, which will use the memory map from Limine, but it converts
   it to a custom format first, in such a way that it's expandable to other initial memory map formats
- Init flanterm with the newly acquired memory
- Set the console write function to flanterm
- more goes on after here ofc, but this is all that I'm planning for now

hexed citrus Jul 29, 2023, 8:17 PM

#

this might not look any different from the last picture, because it isn't, but now it's going through my configurable TTY_Write wrapper

hexed citrus Jul 29, 2023, 10:25 PM

#

wtf, now randomly limine just stopped giving an SMP response

hexed citrus Jul 29, 2023, 10:45 PM

#

???? now it only gives a response if flags is 0

#

ig no X2APIC for me then.... weird.....

#

it's so weird, this works:

volatile struct limine_smp_request smpRequest = {
    .id = LIMINE_SMP_REQUEST,
    .revision = 0,
    .flags = 0
};

and I get this back

SMP Response Info:
  - Response revision: 0
  - Number of CPUs: 2
  - Bootstrap Processor Local APIC ID: 0
  - X2APIC: Disabled

but when I try to enable the X2APIC with .flags = 1, what i get back is a NULL pointer:

if (smpRequest.response == NULL) {
    puts("Limine didn't give us a valid SMP response...\r\n");
}

output:

Limine didn't give us a valid SMP response...

hexed citrus Jul 30, 2023, 12:54 AM

#

welp, never got X2APIC working, but I ported over my old interrupts code, cleaning it up a bit where I could, and now it works

#

(not messing with the GDT is probably gonna cause issues in the future, but it's useful for early boot, in case I ever need to call back into Limine code)

distant fox Jul 30, 2023, 1:01 AM

#

hexed citrus welp, never got X2APIC working, but I ported over my old interrupts code, cleani...

What's the point of the first message lol

hexed citrus Jul 30, 2023, 1:02 AM

#

to ensure that the function was actually getting called mainly

#

doesn't really serve much of a purpose now

worldly drift Jul 30, 2023, 6:34 AM

#

hexed citrus it's so weird, this works: ```c volatile struct limine_smp_request smpRequest = ...

this shouldn't happen

#

i will take a look at it

hexed citrus Jul 30, 2023, 6:35 AM

#

ah, alright

worldly drift Jul 30, 2023, 6:37 AM

#

neigh

#

works on my machine

hexed citrus Jul 30, 2023, 6:37 AM

#

huh

#

ok, I can confirm that in Bochs, emulating a machine with no X2APIC support, it sets everything correctly, and says that the X2APIC is disabled

#

let me test in QEMU again (where I had issues before)

#

and there it's NULL

#

in case you want to test it quickly, my exact qemu command line is this, in the mean time, I'll see if I can get a minimal reproducible example on my machine
qemu-system-x86_64 -no-shutdown -no-reboot -m 1024M -smp 2 -cpu core2duo --enable-kvm -cdrom bin/TFOS_ISO.iso -display gtk -s -d cpu_reset,int,pcall,unimp $(EXTRAARGS) -serial stdio

#

here, this is pretty much the minimum to reproduce the issue
to build, make clean all. It will auto download all of the files it needs from the limine github repo
to test in QEMU, where it dies, make run after building
to test with bochs, where it works, ./bochs.sh after building, after it gets to the black screen, you can kill it and check log/COM1 for the serial output
when it works, it'll print the info, 2 cores, etc
when it doesn't work (limine gives NULL), it'll print that it didn't give a valid response

📎 limine-bug-minimal.tar

hexed citrus Jul 30, 2023, 7:21 AM

#

hm, also of note, after adjusting the QEMU command -cpu max rather than core2duo, it enables the X2APIC properly..... weird

#

so the bug is basically, under QEMU, if the CPU doesn't support X2APIC (or at least, is a core 2 duo, I didn't test anything else), rather than just saying that the X2APIC wasn't enabled in the flags, it just doesn't give a response at all

worldly drift Jul 30, 2023, 8:28 AM

#

you're not supposed to use anything other than -cpu host with -enable-kvm

#

it's a qemu bug or general misfeature to allow it

brave maple Jul 30, 2023, 10:07 AM

#

hexed citrus welp, never got X2APIC working, but I ported over my old interrupts code, cleani...

You probably want to replace the GDT though

hexed citrus Jul 30, 2023, 4:43 PM

#

worldly drift you're not supposed to use anything other than `-cpu host` with `-enable-kvm`

ah, I'll look into it more on other hardware then, see if it's just a weird QEMU issue causing that

hexed citrus Jul 30, 2023, 4:43 PM

#

brave maple You probably want to replace the GDT though

Also yeah, I realized that now, nothing is working

#

I thought I could get away with it, because Limine's is pretty sane, but I guess not ¯_(ツ)_/¯

#

ooh fun, I love it when my existing code, that I intentionally went for because it just works, doesn't just work

#

ok, stepping through it with GDB, my GDT and IDT are loaded

#

then why does it explode

#

in x86_Exceptions_Init it seems.... huh

#

???? it calls an int3

#

I know i didn't intentionally put a breakpoint there or anything

#

void x86_64_Init() {
    x86_GDT_Init();
    x86_IDT_Init();
    x86_Exceptions_Init();
}

Dump of assembler code for function x86_64_Init:
   0xffffffff80001587 <+0>:     push   %rbp
   0xffffffff80001588 <+1>:     mov    %rsp,%rbp
   0xffffffff8000158b <+4>:     call   0xffffffff80001be1 <x86_GDT_Init>
   0xffffffff80001590 <+9>:     call   0xffffffff80001dbe <x86_IDT_Init>
   0xffffffff80001595 <+14>:    call   0xffffffff80001868 <x86_Exceptions_Init>
=> 0xffffffff8000159a <+19>:    int3
   0xffffffff8000159b <+20>:    pop    %rbp
   0xffffffff8000159c <+21>:    ret

right before it cleans up the stack and gets out of there too, weird....

#

I wonder if it's actually there in the binary or if gdb is just being weird

#

no, it's just a nop

#

??

#

weird....

worldly drift Jul 30, 2023, 10:32 PM

#

check your compiler warnings and fix them

hexed citrus Jul 31, 2023, 4:59 PM

#

oh, using the system's GCC? Yeah that's known to have some issue, I don't test it very often, I usually just use my cross compiler

worldly drift Jul 31, 2023, 5:39 PM

#

i mean like

#

build with -Wall -Wextra and fix the warnings

hexed citrus Jul 31, 2023, 5:39 PM

#

I do

#

I enable way more warning than that

#

but there are no warnings when I use the cross compiler

worldly drift Jul 31, 2023, 5:40 PM

#

well you could still be triggering UB hence the int3

hexed citrus Jul 31, 2023, 5:40 PM

#

yeah I checked, in the binary it was just a nop

#

it was something that gdb did

#

for some reason....

brave maple Jul 31, 2023, 5:56 PM

#

hexed citrus it was something that gdb did

gdb doesn't do stuff though

#

atleast if you dont tell it to

hexed citrus Aug 2, 2023, 2:16 AM

#

picked up work on this again a bit

#

going back to school soon, so I'm gonna try to get as much done while I still can

gilded stag Aug 2, 2023, 11:27 AM

#

WHAT???

grim cedar Aug 2, 2023, 11:30 AM

#

no

#

also, necropost :^)

hexed citrus Aug 2, 2023, 3:20 PM

#

lol, yeah that was a while ago, basically I had just discovered the __builtin_*x* functions and didn't check that they were actually what I thought they were, and got so confused nothing worked

hexed citrus Aug 3, 2023, 5:34 PM

#

oh... I just realized why SSE and floating points weren't working

#

I check for SSE, but I never actually enable it....

hexed citrus Aug 3, 2023, 5:54 PM

#

sigh now my panic code is all wrong

olive inlet Aug 3, 2023, 7:42 PM

#

im sorry but how exactly do you mess up a print and a 1: hlt; jmp 1b?

#

im not judging, no no, i would never do that... just curious :^) ive done worse things

hexed citrus Aug 3, 2023, 7:43 PM

#

the printing part

olive inlet Aug 3, 2023, 7:43 PM

#

ah fair

hexed citrus Aug 3, 2023, 7:43 PM

#

I would need to look into it more

#

I got sidetracked with other things

#

I think my interrupt handler isn't proplerly pushing all of the registers on to the stack

#

which is weird because it worked before

#

weirdly enough, it works until it prints the error code

#

and then everything after that just craps itself

#

either completely invalid, or the wrong register

#

which makes 0 sense

hexed citrus Aug 3, 2023, 10:18 PM

#

yeah, this is the issue, it prints everything up until the error code correctly, then it somehow does:
- err is nothing from the QEMU log
- CR2 might be correct, but it's just 0, so no real way to tell
- RIP no idea, according to the backtrace, I was never at 0x00
- RBP is correct
- RSP is too generic to tell what it's actually pulling from
- CS is RIP
- SS is RSP
- flags is CS

#

looking at my struct, it looks like something is off by 1 variable

#

everything has just been shifted left by 1

cinder hamlet Aug 3, 2023, 10:25 PM

#

error code doesnt get pushed at all if there is none
make sure you are handling that

hexed citrus Aug 3, 2023, 10:26 PM

#

yup, I do

cinder hamlet Aug 3, 2023, 10:30 PM

#

well you clearly arent handling it correctly

hexed citrus Aug 3, 2023, 10:30 PM

#

yes, I know

#

the weird thing is that it worked before

#

I have no idea what I did that broke it

#

literally the only differences are renaming stuff

#

wtf

hexed citrus Aug 3, 2023, 11:11 PM

#

cinder hamlet error code doesnt get pushed at all if there is none make sure you are handling ...

wtf.... by not handling it, it works
I must be going insane

#

oh great, now sprintf is #GPing at the very start, according to addrline, this is to blame (something in the setup ig?)
int sprintf(char* s, const char* format, ...) {

#

hang on, I'll disassemble it and see what the exact instruction is

#

ah, the first possible SSE instruction.... great
I love it when I enable SSE and it still doesn't work.....

#

I swear i'm so confused by this

#

I've never been able to get SSE working after switching to Limine

#

let me go back and check, but I don't remember seeing anything about SSE when looking at Limine docs

#

yeah, the only mention I can see of SSE in Limine is that there's no need to save SSE registers when using limine_terminal, so I doubt Limine is the issue... maybe I'm just going insane, I'll test it on a real PC

#

That can't be the issue

#

since sprintf works other times

#

wait no

#

it has a conditional branch just before this

#

so then, yeah SSE must just not be working, I can't think of any other reason this would cause a #GP
movaps %xmm0,-0x80(%rbp)

#

actually, I might have just found the issue, and it seems like it's actually an issue with GCC

#

When the source or destination operand is a memory operand, the operand must be aligned on a 16-byte (128-bit version), 32-byte (VEX.256 encoded version) or 64-byte (EVEX.512 encoded version) boundary or a general-protection exception (#GP) will be generated.

#

and %rbp was not 16-byte aligned 0xFFFF80003FF6EED8, causing the offset to not be 16-byte aligned, causing it to #GP

#

probably just using a broken version of GCC, let me just go rebuild my toolchain

hexed citrus Aug 4, 2023, 12:15 AM

#

hmmm, nope

#

weird...

distant fox Aug 4, 2023, 12:29 AM

#

hexed citrus so then, yeah SSE must just not be working, I can't think of any other reason th...

Why are you using sse in the kernel

#

Compile with -mno-sse -mno-sse2 etc

hexed citrus Aug 4, 2023, 12:34 AM

#

not very helpful.... I've always used SSE and it's always worked

distant fox Aug 4, 2023, 12:35 AM

#

It shouldn't have

#

You absolutely do not want sse in the kernel

#

You'd need to manage the kernel state as well as thread states

#

Which leads to slowdowns

#

For what? a bit more performance

#

Even then if you're using sprintf before enabling sse then that won't work

#

So just compile with the proper arguments (-mgeneral-regs-only work too)

hexed citrus Aug 4, 2023, 12:37 AM

#

I don't, I enable SSE before even calling into C code

distant fox Aug 4, 2023, 12:38 AM

#

Any reason why you're using sse

#

Are you even using it explicitly?

hexed citrus Aug 4, 2023, 12:39 AM

#

mainly performance, because I don't do any context switches yet, so it just makes testing faster

distant fox Aug 4, 2023, 12:39 AM

#

doesn't matter much

#

Just compile with the proper flags and be done with it

worldly drift Aug 4, 2023, 2:45 AM

#

i can confirm that Limine has nothing to do with your SSE not working, btw

#

it cannot have anything to do with it, there isn't like

#

a way to permanently disable SSE or something

hexed citrus Aug 4, 2023, 6:40 AM

#

yeah, it's just weird that it worked under bootboot, but after switching to Limine it's the biggest issue i've had that just manifests in the most random places

#

what's even weirder, is that the code itself is fine
(since my code is written using a kind of libc, that's what made this so easy)
I ported the PMM memory map printing code, and the logging code, as well as the entire printf implementation into it's own program
I compiled it with the same flags that I did for the kernel (-march=core2, which enables SSE), and it works almost fine, printing the size of the range is kinda messed up, but it mostly works, and doesn't crash

#

yup, I can execute that instruction just fine on linux

#

so something is horribly wrong

loud nimbus Aug 4, 2023, 8:02 AM

#

I mean yeah you'd hope it works under Linux

#

I'll jump on the "why sse in the kernel" band wagon

#

But also there's a very specific set of steps for enabling sse, they're outlined in volume 3 of the sdm

#

Somewhere around chapter 15 iirc, not at a computer so i can't say for Sue

hexed citrus Aug 4, 2023, 1:13 PM

#

Yeah, just read through that section, and I was doing everything almost correct, I switched setting CR0 & CR4 around to match, however it still doesn't work....

#

I honestly wonder if GCC is just generating some funky code, since I remember reading that, should the operands be memory, it needs to be 16-byte aligned, however, when that code getting called, %rbp is not aligned

loud nimbus Aug 4, 2023, 1:20 PM

#

hexed citrus I honestly wonder if GCC is just generating some funky code, since I remember re...

honestly I find it unlikely that GCC of all things is the issue. I would double check your sse init and your reasons for supporting it in the kernel.

hexed citrus Aug 4, 2023, 1:22 PM

#

again, I've enabled SSE before, and even if I just grab an exact copy of my old code, that I KNOW worked under bootboot, it just doesn't work....

hexed citrus Aug 4, 2023, 1:25 PM

#

loud nimbus honestly I find it unlikely that GCC of all things is the issue. I would double ...

also, it is entirely possible, my cross compiler is based off the latest GCC sources from git, because I never implemented a way to go find the latest stable tarball into the script, it just git clones the latest source code, so it's possible that something is bugged
between a whole month though? (07/16 was the old version, last one was compiled just yesterday, 08/03), with that idk if it's actually a GCC issue

#

but yeah, my old kernel could handle SSE just fine, but immediately after porting to Liminem, that was the first issue I had, that SSE was just.... not getting enabled somehow

loud nimbus Aug 4, 2023, 1:39 PM

#

so I'm writing from the perspecive of someone fairly familiar with the limine boot protocol. It makes no guartenees about SSE, so this sounds like you're relying on some accidental behaviour of bootboot and carrying that over to your limine implementation.

hexed citrus Aug 4, 2023, 1:40 PM

#

yes, that's... the entire point is that it makes no guarantees about SSE, and that's fine
bootboot didn't either
I tried to manually enable it, exactly like I do now, when I was using bootboot

#

but it actually worked there, and I have no idea why it isn't here

grim cedar Aug 4, 2023, 1:51 PM

#

maybe you need to debug it :^)

violet hearth Aug 4, 2023, 1:54 PM

#

@hexed citrus UEFI runs with SSE on by default, so while bootboot doesn't enable it, it's already on (and same for qemu -kernel iirc)

#

sounds like your SSE init has always been broken

hexed citrus Aug 4, 2023, 1:55 PM

#

no, because I remeber needing it, otherwise I would get #UD

#

I still have the code, I can go check

loud nimbus Aug 4, 2023, 1:57 PM

#

again I would ask you to ask whether SSE in the kernel is worth it

hexed citrus Aug 4, 2023, 2:00 PM

#

yeah, I still see no reason why not to
as I've mentioned multiples, if, like yall said, (if) once I ever get to the point of doing context switching, if it becomes a problem to need to save the additional registers, then I'll just disable it
until then, I'm going to continue trying to get what should be a very basic feature working

distant fox Aug 4, 2023, 5:04 PM

#

Lots of reasons why not to do it

brave maple Aug 4, 2023, 6:22 PM

#

hexed citrus yeah, I still see no reason why not to as I've mentioned multiples, if, like yal...

You‘re somehow stuck on that very basic feature since a week or two

fading karma Aug 4, 2023, 8:03 PM

#

hexed citrus yeah, I still see no reason why not to as I've mentioned multiples, if, like yal...

did you set MXCSR the way the compiler expects it to be?

hexed citrus Aug 4, 2023, 8:05 PM

#

if it expects it to be set up in the same way that then intel manual says, then yes
give me a sec and I'll check, juggling 3 things at once here

fading karma Aug 4, 2023, 8:05 PM

#

it expects it to be set up the way the sysv ABI says

#

hexed citrus Aug 4, 2023, 8:51 PM

#

alright, i found a table online of the bitmask values for those bits, and if I'm not mistaken, that should end up as 0x1F80, but after moving that into memory, and running ldmxcsr on it, SSE still doesn't work
I'll go check out that relavent section of the SysV ABI and see if I'm missing something

hexed citrus Aug 4, 2023, 10:40 PM

#

also, for all of you saying to just disable SSE, for the heck of it, I tried it:

stdio/printf.c:342:15: error: SSE register return with SSE disabled
  342 | static double applyScaling(double num, scalingFactor_t normalization) {
      |               ^~~~~~~~~~~~

doesn't even work

fading karma Aug 4, 2023, 10:40 PM

#

no they told you not to use SSE in the kernel

#

which means don’t use floats

hexed citrus Aug 4, 2023, 10:42 PM

#

yeah, and that just doesn't work, I use floats in at least a few places
welp, back to trying to get SSE to work once again

cinder hamlet Aug 4, 2023, 10:49 PM

#

get sse to work so that you have to remove it 5 seconds later when you need to do context switching

distant fox Aug 4, 2023, 10:50 PM

#

hexed citrus yeah, and that just doesn't work, I use floats in at least a few places welp, ba...

Don't use floats

cinder hamlet Aug 4, 2023, 10:51 PM

#

also i think you will need to save sse registers in interrupts too if you enable it in kernel

hexed citrus Aug 4, 2023, 10:51 PM

#

cinder hamlet get sse to work so that you have to remove it 5 seconds later when you need to d...

if you replace "seconds" with "months" there, maybe
I'm going back to school in 2 more days, progress is going to grind to a halt

distant fox Aug 4, 2023, 10:53 PM

#

Why are you so stubborn?

#

Stop using floats

hexed citrus Aug 4, 2023, 10:53 PM

#

distant fox Don't use floats

if only I could still print numbers with decimal point in them, and without entirely rewriting printf (where that error came from), I would do that
but there's just no way that I can think of, without making any time that I need to do anything with decimal numbers really paniful

distant fox Aug 4, 2023, 10:55 PM

#

And why do you need to print numbers with decimals?

cinder hamlet Aug 4, 2023, 10:55 PM

#

why would you ever need to do anything with decimal numbers in kernel

distant fox Aug 4, 2023, 10:55 PM

#

And you can just use the remainder

cinder hamlet Aug 4, 2023, 11:01 PM

#

hexed citrus if only I could still print numbers with decimal point in them, and without enti...

also i dont think you need to rewrite the printf you are using, it has some #define settings to turn off all the float stuff

hexed citrus Aug 4, 2023, 11:06 PM

#

I would need to look into it, but I think that then, I would no longer be able to print floats, which like I said is fine, as long as I can mimic the functionality (to a certain extent at least), and it doesn't make the code super ugly by doing it
Now I haven't looked into using the remainder to determine the decimal place, and I have no idea if there is a more optimized way to do it, but what I imagine is that I would need to loop, doing division, until it can calculate every digit that it needs, and then pack each digit into an int

cinder hamlet Aug 4, 2023, 11:07 PM

#

you dont need to make it so complicated

#Techflash OS