sure
#Zinnia
somber solar
cyan nexus
near the data prob
hoary cave
i store my data in my struct page
somber solar
doesn't that waste a lot of memory though?
cyan nexus
8 bytes
somber solar
but I don't really need metadata for a pmm, do I
somber solar
well this got a lot easier
hoary cave
https://github.com/48cf/osdev/blob/main/kernel/src/mm.c
relevant functions:
order_from_sizesize_from_orderadd_to_free_listpop_from_free_listfree_regionmm_initmm_alloc_pagemm_free_page
hoary cave
<https://github.com/48cf/osdev/blob/main/kernel/src/mm.c>
relevant functions:
-...
though my implementation relies on being able to store the next/order/max_order in the struct page responsible for the page it's tracking
lofty copper
but I don't really need metadata for a pmm, do I
You probably want to store metadata for shared memory anyway, so you can just union it with that
near tartan
oh nah
hoary cave
like bro wtf are all these people doing
somber solar
lmao
near tartan
trolled
cyan nexus
XD
near tartan
mimalloc using malloc underneath
hoary cave
well i mean it's quite normal under non-kernel circumstances
hoary cave
for performance?
hoary cave
if you know your constraints you may be able to allocate memory in a better way
and buddy may as well be a good candidate for something, idk
lofty copper
why make an allocator
How else would you do it?
lofty copper
like you have a maximum order or smth
If you're implementing zones, you can't know how many there will be
hoary cave
memory regions probably
hoary cave
well u know that ahead of time, kinda, no?
lofty copper
Idk
hoary cave
you know how many you want to support
just not how many you will have
you might not have memory >4G
idk how numa works
lofty copper
Yeah
near tartan
does numa matter when starting out? or this that some high tier optimization?
lofty copper
I have <1MB, <4GB and >4GB atm
hoary cave
server systems mostly i guess
accessing memory that belongs to another processor is slow as balls apparently
so you want to allocate physical memory with respect to that
near tartan
accessing memory that belongs to another processor is slow as balls apparently
makes sense
lofty copper
server systems mostly i guess
hoary cave
that's why i said mostly
but native numa is seen on systems with multiple cpu sockets
lofty copper
Do new consumer CPUs use them?
lofty copper
Now that both AMD and Intel stuff is using chiplets
hoary cave
they all share one memory bus
lofty copper
Yeah
hoary cave
that's not numa
near tartan
is it normal to feel like a brainlet
somber solar
yes
near tartan
idk but every time i have to think about mem management my brain shuts off
somber solar
me when uvm
hybrid island
idk but every time i have to think about mem management my brain shuts off
memory management is a whole thing, isn't it?
Certainly got me a few times in BadgerOS while initially setting up vmem, though my kryptonite is more like schedulers
cyan nexus
idk but every time i have to think about mem management my brain shuts off
same with threads XD
grave peak
<https://github.com/48cf/osdev/blob/main/kernel/src/mm.c>
relevant functions:
-...
When thread
And you say you don't know stuff damn
near tartan
real
hoary cave
When thread
there already was a thread
but i kind of abandoned it
just because i can write a buddy allocator doesn't mean i can write a kernel 😭
i still don't know how to write a decent scheduler
hoary cave
like i know how to implement context switching between kernel threads
that's piss easy
but idk how to tie it to a scheduler
preemption vs explicit thread switch is something idk stuff about
preemption is hard lol
near tartan
is it really?
near tartan
storm bobcat
preemption vs explicit thread switch is something idk stuff about
preemption is either done by a timer or by the thread itself when it e.g blocks
hoary cave
ok but how does a timer do it?
hoary cave
nah just me being a literal idiot
near tartan
allocators will stay the bane of my existence forever man
im pretty sure i did everything correctly, but i somehow manage to have more pages than memory and a buddy with an order of 0
note to self: check page and buddy orders again, maybe calculation is off
if i can't get this working today I'll just use a bitmap for now...
hoary cave
im pretty sure i did everything correctly, but i somehow manage to have more pag...
order 0 is just a 4k page
or should be, anyway
near tartan
hm
it trips the assertion " buddy.order == page.order-1"
i beg you to take a look at some point today so i can finally move on from this shit. i quadruple checked all functions already and it still doesn't work.
idle flower
Had to skip a bit of conversation; At least for me I figured out how to do a lockless buddy with 24B/page if you squeeze it overhead and O(1) fit everything except alloc/dealloc
And there is one more place where you need continuous memory, when you aggregate towards larger pages
idle flower
if i can't get this working today I'll just use a bitmap for now...
I could give you my buddy once I actually wrote it
lofty copper
i can't do either
My scheduler is also total trash, I don't really get how quantum and time slices and related are supposed to work
hybrid island
My scheduler is also total trash, I don't really get how quantum and time slices...
The concept of a quantum in this context is basically the "smallest increment possible", though time quantum is often said when time slice is actually meant. Then, a time slice is merely how long the scheduler gives a thread before preempting it, assuming it does not yield before then.
lofty copper
But what happens to io bound threads?
Like do you just give them a full time slice?
hybrid island
But what happens to io bound threads?
An I/O bound thread will likely block on said I/O, thereby implicitly yielding and resuming only when the I/O is ready.
hybrid island
Like do you just give them a full time slice?
Usually these threads don't get scheduled again until they can be unblocked; until it no longer needs to wait for the current I/O operation.
lofty copper
I get that
But do you place them at the end of the ready queue and give them a full time slice? In which case they get to run for more time than the rest of the threads
vast lotus
I keep track of how much CPU time the thread has used and it automatically yields if that exceeds its time slice
It could yield a thousand times for IO, the timeslice clock will keep ticking
hybrid island
This is a question of scheduling policy. In BadgerOS, a thread that is unblocked will be placed at the front of the queue, getting a full time slice immediately. Not ideal for perfect fairness, but great for response time.
If you don't want to have this issue then you can place the thread at the end of the queue, a blocking action behaving as though the thread had yielded the remainder of its time slice because the other threads in the queue will run before the thread that just unblocked.
lofty copper
That's kinda what I do at the moment
hybrid island
Important note: My beginning of queue means thread that will start next, end of queue means thread that will start last.
marble salmon
why "only slightly"?
vast lotus
well it's still not particularly complex
marble salmon
It's linear in the number of pages
Vs. logarithmic
so in that sense it's exponentially worse
vast lotus
Oh I wasn't talking about computational complexity, just implementation complexity
distant cypress
maaaaaaaan
if page is an order-N page that you are freeing, and page_buddy(page, N) is free but of a different order than page, you cannot merge them.
remember that if you coalesce a page with its buddy, to remove whichever page has the higher address, and is thus misaligned for an order-N+1 page, and keep the other (potentially coalescing with an order-N+2, order-N+3, ... page)
you can only merge two pages if they are buddies and of the same order
so idk what the "expected buddy for order 2 to have order 1, but it was 0" thing is supposed to mean.
near tartan
i looked at iretq's code and it had that assert
distant cypress
can you link iretq's code?
distant cypress
nvm. found it
thanks anyways
I'm just going to assume that order_from_size, ´size_from_order, etc. are correct
L243-L254 hurt my eyes
why not just
struct page *page = pop_from_free_list(order);
page->current_order = desired_order;
for (; order != desired_order; ) {
struct page *buddy = buddy_of (page, --order);
buddy->current_order = order;
add_to_free_list (buddy);
}
yeah so I assume this kernel panic originates from your translation of L248 of iretq's code
in which case, iretq's invariant that order_of(page_buddy(page, order_of(page)) == order_of(page)-1 for any free page with order > 0 is probably broken somewhere
Page::pop_from_free_list() should probably clear page.on_free_list
near tartan
@vast lotus how do you get away with not using the idt_frame_t during a context switch?
how are rax, rdx, etc. saved?
vast lotus
rax, rdx, etc. are caller-saved registers
so whatever called x86_64_switch_thread is responsible for saving them
that effectively means they're already somewhere on the stack
near tartan
oh so you just save the kernel stack + callee saved regs?
vast lotus
yep
near tartan
ahh
vast lotus
the compiler will take care of the rest at the callsite
near tartan
oh damn so i would need sysv C abi, right?
vast lotus
any specified abi would work (you'd just need to save a different set of registers) but yeah
near tartan
well, rust abi isn't exactly stable :D
so like, if you can just use the stack, why save any registers at all if they're already on the stack?
vast lotus
the callee-saved registers aren't guaranteed to be on the stack, since they're callee-saved
vast lotus
any specified abi would work (you'd just need to save a different set of registe...
I think the rust syntax would be something like this? as long as the compiler knows what your ASM expects you're fine, even if the code calling this function is a different callconv ```rust
#[naked]
extern "sysv" fn switch_thread(from: *mut usize, to: usize) {
// ...
}
i can do extern C
near tartan
the callee-saved registers aren't guaranteed to be on the stack, since they're c...
they're not? aren't they always part of idt_frame_t?
like, when a preempt ipi happens, does that not always give you the regs
vast lotus
those are the registers of the interrupted context, the kernel code leading up to the context switch might have (read: probably will have) used those registers for other purposes
remember, this part of the scheduler should not rely on interrupts being a thing at all
near tartan
yea i'm having trouble getting that in my head
vast lotus
pretend you're writing a cooperative scheduler - the only difference is sometimes you're calling yield from an interrupt
near tartan
tbh i'm still failing to see what exactly the call stack is and where the registers of the preempted task are saved
would really appreciate a call stack example
assming an interrupt
vast lotus
user_func_a
user_func_b
<INTERRUPT> <-- idt_frame_t contains the values the registers had here
x86_64_idt_thunks[X86_64_IDT_LAPIC_TIMER]
x86_64_idt_entry
x86_64_idt_dispatch
x86_64_handle_timer
preempt_unlock
do_yield <-- this was queued by one of the events x86_64_handle_timer ran
arch_switch_thread
x86_64_switch_thread
arch_switch_thread saves rax, rcx, rdx, rsi, rdi, r8, r9, r10, r11 to the stack (or discards them if they aren't used after that point)
x86_64_switch_thread saves rbx, rbp, r12, r13, r14, r15 to the stack, then saves rsp to the thread structure
when this thread is switched back to, x86_64_switch_thread restores rsp from the thread structure and restores rbx, rbp, r12, r13, r14, r15 from the loaded rsp, then returns to arch_switch_thread, which restores the registers it saved
then all the functions on the call stack return until eventually x86_64_idt_entry restores the interrupted code's registers and does an iretq
vast lotus
arch_switch_thread saves rax, rcx, rdx, rsi, rdi, r8, r9, r10, r11 to the stack ...
https://github.com/proxima-os/hydrogen/blob/rewrite2/arch%2Fx86_64%2Fkernel%2Fproc%2Fsched.c#L12-L20 this is all that's explicitly saved and restored to/from the stack on context switches, the rest is taken care of by the compiler at the callsite
(the rip there is saved by the call instruction that led to x86_64_switch_thread)
new threads are initialized by building this structure on a fresh stack with rip pointing to a small thunk that moves some of the callee saved registers in that struct into the argument registers before calling into c code
near tartan
hm, so would i have to make sure all functions called up until that point are extern "C"?
or just arch_switch
vast lotus
no, you just have to make sure the actual assembly function that switches threads is extern "C"
near tartan
oh
vast lotus
in this case x86_64_switch_thread
near tartan
that makes sense to be extern C
vast lotus
arch_switch_thread would be extern "Rust", the compiler will take care of any callconv differences
near tartan
oh wait i think i get it?
so the way i understand it now, assuming 2 threads for example:
- t1 is preempted
- idt handler pushes all regs to the kernel stack
- do_yield is called to switch to t2
- arch prepares the switch by changing e.g. fsbase and gsbase
- callee saved regs are saved to t1 task
- callee saved regs from t2 are loaded
- rsp is switched to t2's kernel stack
- code returns until it's back at the idt handler (?)
- t2 idt_frame is popped back
- iretq
vast lotus
rsp is a callee saved reg so that being an explicit step is a bit redundant but that's correct yes
vast lotus
in my case I also arrange for x86_64_switch_thread to return the location it saved the previous task's rsp to which lets me get a pointer to the thread that was switched away from, I use this for e.g. putting exiting tasks on the reaper queue
vast lotus
how do i build a stack for a new task?
you determine what stack values x86_64_switch_thread expects and initialize them in such a way that your thread init function gets called
in my case: x86_64_switch_thread expects this struct. rbx is initialized to the function that should be called in the new thread, r12 to the argument that function wants, and rip to x86_64_thread_entry. x86_64_thead_entry is a small ASM thunk that sets up the call convention and calls x86_64_init_thead(old_rsp_location, rbx, r12). at this point we're in C code again
there's some initial processing done in that init_thread call and then the function initially passed through (rbx,r12) is called
vast lotus
near tartan
❤️
vast lotus
oh and CONTAINER(a, b, c) is just a helper macro that returns a pointer to the a structure whose member b is at c
vast lotus
I'm not familiar enough with the linux src to say but judging by the name, probably
vast lotus
there's no reason it's specially r12
near tartan
is it just arbitrarily used by your code for kernel threads?
vast lotus
it just has to be one of the registers x86_64_switch_thread restores
near tartan
got it
vast lotus
the small ASM stub moves it into the right register for it to be picked up as an argument to the thread's first c function
near tartan
i see
vast lotus
it's essentially a miniature callconv
marble salmon
the disadvantage of not saving all regs is that you can't always inspect userspace state when you're in the kernel
so you need to unwind to the kernel entry when that becomes necessary
vast lotus
You're still saving all regs on interrupt entry, so you have the full user state
You're just not saving all kernel registers, but there's never really a reason to read those
marble salmon
Ah i misunderstood what you were proposing
I thought you just saved caller saved regs on kernel entry
vast lotus
Nah I save all regs on interrupts, we were only talking about context switches though
near tartan
@vast lotus as long as the last function i call before the switch has C ABI, this should work, right?
vast lotus
the only function whose abi declaration matters is the actual assembly function doing the switch
by adding extern "C" to that function, you're saying, among other things, "this function expects that the caller has saved the registers that the C abi (aka System V ABI) says should be saved by the caller"
the compiler will take care of satisfying that requirement
near tartan
yeah
lofty copper
It also doesn't work on mine
lofty copper
it just shuts down
lofty copper
maybe display out is broken?
I'm connecting to a 4K monitor
I can try disabling ACPI
Like DTB only mode
Though idk, Ubuntu did work in ACPI mode somehow
It doesn't work in ACPI mode either 
near tartan
time to buy orion o6
lofty copper
I guess I can just flash different firmware..?
Also Ubuntu does work
Poorly
So like mainline linux needs to get their shit together idk
I can also try limine with linux protocol instead of efistub
lofty copper
idk, I'm using whatever the installers ship with
mainline ubuntu is working ok actually
near tartan
shrug
lofty copper
I ran speedtest-cli and it gave very slow ethernet speeds
but I guess it was just a weird server
(I need to figure out how to compile the limine template)
(I failed successfully the last time I tried it (last weekend))
Ubuntu's kernel is not that old?
spark surge
neat
ionic lava
@near tartan If you are going to test what I asked to, then use this build, pls. It turned, that I checked for the strict WC match for memory attribute, but what if it's more bits over there? So I changed it to print about any range, having WC bit in the Attribute field. It's weird, but seems like the only machine this test has been conducted on with WC typed framebuffer was that Lenovo Snapdragon laptop. Others, both x64 and arm64 just don't. So, if you'll do the test, pls, take a pic.
near tartan
okay
lofty copper
<@173337436989030401> If you are going to test what I asked to, then use this bu...
@ionic lava
grave peak
Looks like it hung?
ionic lava
it just halts CPU in the end of what it has reached to, Thanks, @near tartan ^_^
grave peak
Also didn't it try to print something at the end
ionic lava
whoopsie. anyway, thanks to microkernel. 😄
now it's a sh1t ton of WC regions. 😄
that doubling in the end is a bug of printing function. it didn't print after that.
lofty copper
now it's a sh1t ton of WC regions. 😄
what does this mean? (I'm planning to support ARM eventually and want to know)
ionic lava
the 1st number is an address (system), the 2nd is number of pages, the 3rd is the attribute. 0x0E means, it's ored WC, WT, WB, those 2 with additional 8 in the topmost bite are RUNTIME. It's just a jump into kernel test. it's running now in the kernel address space with all system control registers set up. Because of too many entries, the top of the print got scrolled up and we didn't see, what EL edk is set to run. But now, when I got why WC attribute didn't appear, this huge 455 list might be excluded.
here, I removed that excessive list, so we can see the EL edk runs in. It's important for collecting knowledge. If you can test it yet once, please test. @lofty copper
ionic lava
Nice, thank you, so it's EL2. On qualcomm it's EL1.
near tartan
yea you're not getting el2 on qcom
ionic lava
lol, so, edk just marks all pages as WC. 😄 nice, very helpful.
marble salmon
Qualcomm hypervisor my beloved
near tartan
lol
on x1e, not even linux is allowed el2
because some signed blob bullshit from microsoft
somber solar
@near tartan didn't you say something about using eir as a prekernel for menix :^)
near tartan
i said i could do a prekernel like eir
somber solar
ah
ionic lava
@lofty copper is this OPi 5 (Plus)? Well, now I have tests on all feasible arm real hardware except rpi, but I have no idea if edk works there. both uboot (its UEFI) and edk have shown as pretty normal things to deal with. at least so far. Thank to all, now to the further work.
lofty copper
<@350748255929499649> is this OPi 5 (Plus)? Well, now I have tests on all feasib...
Yes
I hed RPi4, but it was sent to Russia lol
lofty copper
Nice, thank you, so it's EL2. On qualcomm it's EL1.
So like you can run KVM and stuff?
ionic lava
yes.
near tartan
relevant code:
[stack frame setup](https://github.com/proxima-os/hydrogen/blob/r...
i found that linux does the same
pretty cool tbh
near tartan
@vast lotus why are you putting these functions in their own section?
grave peak
--gc-sections
do u have a warning enabled where it tells u u have sections that arent explicitly specified in the linker script?
i think thats better than blindly discarding stuff
or actually maybe gc-sections is smarter
but the warning is still good
hybrid island
do u have a warning enabled where it tells u u have sections that arent explicit...
That's not how gc-sections works
It deletes sections that have no references at all
hybrid island
All you have to do is instead of .text also include .text.* in your linker script.
near tartan
i had this anyways
grave peak
u kinda had to do this for bss already and maybe other sections
at least c++ sometimes generates .bss.idk
near tartan
yea
grave peak
rust idk tho
near tartan
better safe than sorry
grave peak
yeah it wont hurt
near tartan
i actually do a funny in my script
i don't know why but, if i want dynsyms to be loaded i have to put them in the text segment
or else i'll get linker errors
grave peak
lol
near tartan
how
hybrid island
Which segment it's in shouldn't matter
hybrid island
Well it's telling you right there
hybrid island
myes
hybrid island
Aren't linker scripts just fun?
hybrid island
String'd
near tartan
ok
hoary cave
near tartan
okay i'm getting a fucking page fault before it prints ANYTHING to serial
it might actually be stack
idle flower
give it 64MB stacks in Debug build and print the current stack usage at certain points
near tartan
MEGABYTES???
near tartan
yea no still crashes with 256 mb
hoary cave
trust bro
idle flower
aparently more than marvin anticipated
cyan nexus
framebuffer would use at least half of that
near tartan
it can't be stack
cyan nexus
lol
hybrid island
BadgerOS has been running on 16KiB stacks for its entire life now
No problems there
cyan nexus
oh wait
near tartan
this only happened after i changed the lds
cyan nexus
64 MB stack??
cyan nexus
ah ok XD
idle flower
arent they used by pic/pie? so that the programm might write to them after obtaining control?
near tartan
oh
near tartan
crashes even if i put it in data
yes
lets see what happens if i put it back to text
?????????????
it does not work
WHY DID IT BREAK SUDDENLY???
idle flower
the smell of ub
near tartan
okay i'm going to reset this fucking ld script
the only diff i have rn is that i do what @hoary cave said
near tartan
`*(.rodata .rodata.*)` pls
this
it's still broken??????????????
i fucking reverted
idle flower
i once had the problem with linker scripts that cargo didnt detect changes to them and the incremental builds where invalid because of that and each time i needed to touch the linker script i had to disable incremental builds
near tartan
no like i do a rebuild
i'm nuking the result every time
?????????????????????????????????????????
xbps is creating a dir with the name of the package when i install it
what the hell man
i'm going to explode
eternal wharf
are you sure you're not messing up the xbps args?
near tartan
i'm not calling xbps directly
jinx is
and this has worked before
it just now decided to break
oh bruh
@spark surge i'm sorry
i may have broken something
echoing sysroot yields this:
idk how
wait...
eternal wharf
near tartan
omfg
please kill me
idk how i didn't see it
60iq strikes again
this is exactly why i love osdev
it still doesn't work btw 
okay wait
hear me out... what if
this is a limine bug
i know, daring today
eternal wharf
cpu bug
hybrid island
OSDev is a good consistent way to make yourself feel like a troglodite
near tartan
but this page faults before my early init runs
i'm gonna make it print a letter to serial as the very first op
eternal wharf
OSDev is a good consistent way to make yourself feel like a troglodite
real
near tartan
if that doesn't work idk what will
hybrid island
Have you checked whether your kernel ELF file uses constructors?
near tartan
how can it
hybrid island
.init, ELF initialization functions (not called that; I forget the exact name) can be implicitly emitted by your compiler.
near tartan
i mean yes, i have an .init_array section, but rust doesn't have language-level constructors
i have .init too yes
hybrid island
This does seem like a case for running GDB on the very first instruction after bootloader handover.
eternal wharf
when did that happen
like half a month ago
somber solar
isn't arch supposed to be bleeding edge
near tartan
idk
hybrid island
yes
near tartan
they are also behind on xfce packages
somber solar
last time my friend checked, on llvm/clang too
hybrid island
isn't arch supposed to be bleeding edge
But in practice you still need packagers so the less important packages including this one won't be updated as quickly.
eternal wharf
i'm guessing qemu 10 used to be in testing for a while
somber solar
But in practice you still need packagers so the less important packages includin...
what about llvm
hybrid island
Skill issue
hybrid island
it crashes right at the start <:meme:575445094589661215>
That means Limine loaded your executable incorrectly
hybrid island
i put a breakpoint on _start, right
Use function, not line, breakpoint.
eternal wharf
single step through the instructions?
near tartan
i have
one sec
okay it actually gets somewhere
it doesn't die immediately
it sets the idt
then ?????
hybrid island
Does your code rely on the addresses of the sections as defined by symbols in your ldscript?
near tartan
not at this point
it should print at least the memory map before it even touches virtual memory
hybrid island
I suppose later for when you take over the page table.
near tartan
yes
hybrid island
How easy is it for me to download and debug this with you?
hybrid island
just takes a while to build llvm <:meme:575445094589661215>
right
somber solar
you need to build llvm to compile the kernel?
near tartan
yes
somber solar
why
hybrid island
I'm on a laptop right now so I'll wait with that one.
somber solar
wdym uacpi?
somber solar
ah L
near tartan
and i need host-llvm anyways for other builds
hybrid island
If I'm gonna compile LLVM, I'm gonna compile it on my desktop, which is actually capable of this. My laptop didn't have enough RAM last time I tried compiling LLVM.
somber solar
If I'm gonna compile LLVM, I'm gonna compile it on my desktop, which is actually...
how much does it have?
hybrid island
Oh it has 40G now. Must have been a long time ago then because that had like 16 at the time.
somber solar
I can compile llvm with 16 gigs
hybrid island
Still though, don't want to burn my battery in public transit for two hours only to get home before I compile Menix
somber solar
just don't give it more than 8 cores
spark surge
i was pung
hybrid island
so it can generate bindings
I still don't get why this needs custom-built LLVM btw. uACPI doesn't depend explicitly on stdlib after all.
hybrid island
it doesn't but debian ships outdated llvm
ah
It sounded like you were talking for the kernel.
near tartan
you could just compile the kernel on its own but then you'd have to wire it yourself
no
you do need libclang though
and clang
hybrid island
That means I already have what I need.
I already installed both a while back
Then the next question from me to you is: How compile Menix?
hybrid island
Yeah but just the kernel repo? Or do I need the bootstrap repo too?
near tartan
if you don't want to wire everything yourself, just clone the bootstrap repo (that will build llvm though)
hybrid island
I'll just comment out the LLVM builder lol
hybrid island
o
near tartan
and it prolly doesn't have an up to date llvm
hybrid island
just the kernel it is
hybrid island
Main branch correct? Because I get this error:
lofty copper
at least c++ sometimes generates .bss.idk
Wouldn't you want to sort those?
near tartan
you're probably on an outdated toolchain
hybrid island
you're probably on an outdated toolchain
I do no has nightly
grave peak
Wouldn't you want to sort those?
idk, why?
hybrid island
I just did cargo build since you said as much
lofty copper
idk, why?
I know that you have to do it with constructors
Maybe there's something similar
grave peak
maybe
near tartan
oh bruh
i think i have found the culprit?
.early_array is NOT a fan of being run
???????
what the hell?
this is obviously wrong
hybrid island
Not good
hybrid island
How do you define it?
hybrid island
Can you verify the array size with objdump -t?
near tartan
here
exactly 8 bytes large
which is that one function
hm
early_end is wrong
early_start is correct
only serial::init should be in there
hybrid island
Does the assembly look like what you'd expect? You'd expect to see two lea instructions, one for each end of the array.
near tartan
Does the assembly look like what you'd expect? You'd expect to see two `lea` ins...
pretty much
hybrid island
Since I haven't managed to build it yet, care to show me said asm?
near tartan
hybrid island
??? This assembly doesn't look correct to me
near tartan
how so
hybrid island
It's calling libm_math::exp2? Why? Where are the references to your LD symbols?
near tartan
oh yeah good point
hybrid island
call <sin+...> -> obviously incorrect
hybrid island
It's weird to me your GOT doesn't have a symbol. Could this be causing problems?
BTW I managed to build your kernel
near tartan
target/release
or target/debug
sorry wait
target/x86_64-kernel/debug
is what you want
menix.kso
hybrid island
Ok
I've built it and I see assembly but not how that assembly gets its hands on LD_EARLY_ARRAY
eternal wharf
seems legit
plt moment
near tartan
ye
eternal wharf
try linking with -z now?
near tartan
what it do
eternal wharf
turns off lazy binding
eternal wharf
should turn JUMP_SLOT relocs into GLOB_DAT
actually limine does not do either of those
eternal wharf
link with -Bsymbolic or build with rust's equivalent of -fno-semantic-interposition
eternal wharf
why is a plt needed at all
the plt and got is used to minimize the amount of relocations, instead of needing to relocate every call site you only relocate the got entry
near tartan
link with `-Bsymbolic` or build with rust's equivalent of `-fno-semantic-interpo...
same issue
grave peak
the plt and got is used to minimize the amount of relocations, instead of needin...
but it introduces a runtime cost no?
near tartan
minimal
eternal wharf
you can turn the plt off with -fno-plt to have the compiler inline the plt thunk into every call site
hybrid island
but it introduces a runtime cost no?
It theoretically does but modern CPUs don't care
eternal wharf
you can turn the plt off with `-fno-plt` to have the compiler inline the plt thu...
but that will still go through the GOT
hybrid island
They will predict the branch target address before fully decoding the instruction levels of don't care
eternal wharf
same issue
hmm, well basically what you want is for the main kernel executable to not have any JUMP_SLOT or GLOB_DAT relocations
only RELATIVE
since that's the only one limine handles
near tartan
well it would panic if it encountered those, right?
eternal wharf
no because undefined weak functions also cause JUMP_SLOT
it just writes 0 to the relocation target for it
hybrid island
-static-pie?
near tartan
just checked, i only have RELATIVE relocs
eternal wharf
`-static-pie`?
afaicu the idea is for the main kernel exec to export symbols?
hybrid island
afaicu the idea is for the main kernel exec to export symbols?
-static-pie won't work then
near tartan
yes
hybrid island
So you want to export symbols and basically interpret your own kernel as a dynamic library for the purpose of dynamic modules?
eternal wharf
actually my bad, limine does resolve JUMP_SLOT and GLOB_DAT fully
near tartan
So you want to export symbols and basically interpret your own kernel as a dynam...
pretty much
it allows me to treat modules as regular .so's
which have DT_NEEDED and so on
that integrates tightly into the cargo build system
eternal wharf
actually my bad, limine does resolve JUMP_SLOT and GLOB_DAT fully
if it's not weak, it looks for the target symbol and uses it's address
hybrid island
Does Limine dynamically link modules into the kernel this way?
eternal wharf
no limine does not do any dynamic linking
near tartan
limine just loads them as blobs
hybrid island
I see
eternal wharf
it only processes a small set of relocations on the main kernel executable
near tartan
the kernel has functions to relocate and resolve
still, i don't get why this is broken now
eternal wharf
send executable?
near tartan
eternal wharf
neat readelf can demangle rust names
eternal wharf
these relocations look strange, why are the addends negative ```
ffffffff80291950 0000000000000008 R_X86_64_RELATIVE -7fd51788
or is readelf trolling
eternal wharf
>>> hex((-0x7fd51788) & ((1<<64)-1))
'0xffffffff802ae878'
strange
eternal wharf
i call it like this
whats the order of operations with & and as here?
&(... as ...) would be right, (&...) as ... wouldn't? (unless const fn() is a function pointer already?)
eternal wharf
only serial::init should be in there
end points one past?
hybrid island
Is this with the original ldscript again?
near tartan
yep
eternal wharf
`&(... as ...)` would be right, `(&...) as ...` wouldn't? (unless `const fn()` i...
actually the debugger output suggests what's in parenthesis is right
near tartan
whats the order of operations with `&` and `as` here?
first & then as
eternal wharf
yeah
near tartan
otherwise it wouldn't even compile
near tartan
end points one past?
oh well i think the executable i sent you has 2 entries
serial init and a test function
which should be empty
while cur < end
eternal wharf
well yes but i mean end points exactly after all early_init entries
so it makes sense that in the debugger it points to an unrelated fn
near tartan
oh
right
but the address for end is somewhere where it shouldn't be
actually wtf
imagine some bullshit with callconv lol
i hate everything
i think i know what's actually going on
@eternal wharf lol
the debugger was wrong
i did a little UB moment
eternal wharf
near tartan
it tries to read the command line to determine if it should print to COM1
but that hasn't been set in early_init
mfw
or rather,
i shouldn't be calling early_init at that point
easy fix
@hybrid island @eternal wharf thanks for trying to help anyways
near tartan
is it normal for my tsc to overflow after like 5 seconds
i can't multiply by 1 sec and divide by frequency
this kind of wraps immediately
wtf should i do here
hybrid island
Actually yeah I see this happening if the TSC is like GHz kinda fast.
near tartan
yep, mine is 3.8GHz
hybrid island
In a 64-bit multiplication like this that could overflow quickly
hybrid island
divide+remainder by tsc freq, use remainder to calculate sub-second time
near tartan
how fast is the tsc going to wrap?
hybrid island
how fast is the tsc going to wrap?
Well at least 4 billion seconds by that method
Or 126 years
near tartan
good enough
hybrid island
I should do this in BadgerOS too tbh
But I've been spoiled with MHz-speed timers :P
hybrid island
I've been using the time CSR for RISC-V reasons
near tartan
makes sense
i wanna start porting to risc-v as soon as i have scheduling running
(or as soon as userspace works again lmao)
hybrid island
riscv is fun
near tartan
i know
hybrid island
is almost like your nickname contains riscv
grave peak
yes yes
near tartan
smh
near tartan
lets see if it still boots on real hw
hybrid island
very good
grave peak
letsgo
hybrid island
let's try fw13
Running hobby os on framework laptop
I'll get to that eventually too but first I need x86_64 support literally at all.
near tartan
3/3
hybrid island
very cool
near tartan
menix doesn't boot on my amd machine bc i don't have !x2apic support :(
eternal wharf
divide+remainder by tsc freq, use remainder to calculate sub-second time
can also do 128 bit multiplication + fixed point math to replace 128 bit division with a shift
hybrid island
can also do 128 bit multiplication + fixed point math to replace 128 bit divisio...
This is probably more efficient actually
Since the coefficient can be precalculated
near tartan
can also do 128 bit multiplication + fixed point math to replace 128 bit divisio...
what's the math for that?
eternal wharf
hybrid island
what's the math for that?
coeff = (1 << 64) / tsc_freq
Then when you want time, do (ticks * coeff) >> 64
All in 128-bit context of course
near tartan
i wonder how rust translates u128
hybrid island
i wonder how rust translates u128
Multiple 64-bit multiplies, it's not how rust translates it but how clang does so.
eternal wharf
x86 has native 128 bit unsigned mul
or rather
64x64 = 128
which is what the managarm code takes advantage of
near tartan
i should too
hybrid island
So does RISC-V. And FWIW ARM prolly also does that.
eternal wharf
the managarm timer code precomputes freq / 1e9 and 1e9 / freq as two fixed point fractions (with a 64-bit size)
with the point being positioned dynamically based on how many bits are actually needed
to avoid 128-bit division when computing the constants
eternal wharf
the managarm timer code precomputes `freq / 1e9` and `1e9 / freq` as two fixed p...
that was both ticks -> ns and ns -> ticks are just a mul and a shift
eternal wharf
lmao
eternal wharf
i need to be up at 6 today
not worth going to sleep at this point
near tartan
insomnia time
eternal wharf
hybrid island
Idk your tz but it's 00:27 here
hybrid island
Lmao
near tartan
uhhhhh
eternal wharf
('ate dst)
near tartan
gmt+1?
hybrid island
Oh you're German, we're basically neighbors
near tartan
yea
hate time zones
(also thanks windows for storing fucking timestamps in local time inside efi)
eternal wharf
does windows still program the rtc to local time instead of utc?
near tartan
it appears so
hybrid island
Yes it does
near tartan
whenever i switch back from windows my clock is off by an hour
eternal wharf
amazign
hybrid island
Just don't use windows
eternal wharf
yeah easy as
i nuked my windows partition recently and haven't been happier
free 80GB for my linux rootfs
near tartan
i have it on a different nvme now, but i have like 3 games I cant play on linux
cuz ac
somber solar
whenever i switch back from windows my clock is off by an hour
tell windows to use utc
reg add HKLM\SYSTEM\CurrentControlSet\Control\TimeZoneInformation /v RealTimeIsUniversal /t REG_DWORD /d 1
dense vine
i do but i dont need to compile it myself
near tartan
can't you get it pre-compiled for the container?
no
i use the host llvm to also build programs with clang
for user space
so i can just reuse that
the container comes with debian
so forget ever getting uo to date llvm
dense vine
cant you usre fedora or arch or something else for the container?
near tartan
cant you usre fedora or arch or something else for the container?
no
jinx requires debian
near tartan
to the rust nerds here: how the hell does one detect the current crate type?
something along the lines of #[cfg(crate_type = "dylib")]
dense vine
you cant do that
near tartan
cringe
dense vine
what you can do is make a custom cfg flag
dense vine
yeah
or -cfg SOMETHING in rustflags
println!("cargo:rustc-cfg=something"); // build.rs
#[cfg(something)]
and in build.rs you can just read cargo toml
to see what crate type it is
near tartan
how exactly am i supposed to map an ELF like this?
This shares flags with a virtual address that's less than a page big
lofty copper
Where did it come from?
near tartan
rust
it's a kernel module i made
do i just add an align(4k) everywhere
oh shit yeah that was it
we are so fucking back
grave peak
I mean the member of the struct
grave peak
It looks like an array
grave peak
What's its type?
near tartan
&[PciVariant]
grave peak
Interesting
grave peak
'Static means it must be a global?
grave peak
Btw how does it know the length of that array
near tartan
rust
eternal wharf
i'd imagine it just stores it
grave peak
So its like a slice
near tartan
it's exactly a slice
grave peak
Ah
near tartan
tbh that's terminology bingo, an array would be [PciVariant; 16]
i want to have a dynamic amount of possible variants, so just use a slice
grave peak
Is it possible to make an unbounded array that doesn't store the size?
near tartan
not safely
grave peak
I see
grave peak
Ig
near tartan
this pci variant matching is quite nice
i just have to implement the MMIO stuff
actually i think that's already working
grave peak
How easy is it to make an inline data structure similar to how container_of works with e.g. rb trees?
grave peak
Do u have something like that for lists for example?
near tartan
nope, but i will have to at some point for sure
there's intrusive_collections
which use traits to implement struct member access
grave peak
Interesting
near tartan
the loaded module references a symbol that should have been relocated
but this is obviously not mapped
man, issue after issue
grave peak
Limine bug?
near tartan
i hope not
but the module has no knowledge of this address
and the kernel doesn't either if it's been relocated
AHA
it doesn't relocate symbols
mfw
ok bruh this is bad
i don't think it's the bootloader's job tbh
but that means i can't use kaslr
near tartan
eh fair enough
idk how to enforce this though
i have to build as a dylib for this shit to work
but that makes me ET_DYN
near tartan
@grave peak lol the random page fault was a tlb flush problem
it works now even with kaslr
grave peak
nice
near tartan
tomorrow i will have time to finally go to user space
(and not do a lot because the vfs is like 3 lines atm)
near tartan
does anyone have a better idea for logos besides the gears
i might want something less boring
but i can't think of anything that's just a cat or seal
current logo for reference
ionic lava
current logo for reference
rusty and blazing fast. 
dense vine
maybe that pushes uacpi points even more
It does
Opt level 3 does most of the work but fat lto will get you at least 300k
dense vine
i might get back to my 5 mil
what cpu?
near tartan
intel ultra 265kf
dense vine
can you test my kernel?
near tartan
send iso + cmdline
dense vine
qemu-system-x86_64 \
-M q35 \
-cpu host \
-display none \
-debugcon stdio \
-bios OVMF_x86_64.fd \
-enable-kvm \
-cdrom chronos-x86_64.iso
-smp 4 -m 2G | grep 'avg'
its a bit slower without the OVMF
somber solar
-display none
dense vine
same points without it, the flag is just for convenience when testing
why are you stalking me
somber solar
@near tartan could you also test this
qemu-system-x86_64 -serial stdio -cpu host,migratable=off -M q35 -accel kvm -bios OVMF_x86_64.fd -cdrom image-x86_64.iso
run it a few times, I get vastly different scores on each boot
near tartan
ill be home in a few h
near tartan
run it a few times, I get vastly different scores on each boot
same
sometimes i get 2.7, sometimes 4.8
dense vine
same
idle flower
How do you guys create ISO files?
I have a tool that will hopefully work but it's not very automated
fiery marlin
I assume most people are using xorriso or however you spell it
little oasis
i just make disk images
Same here.... I'd rathere make a disk image, and go from there...
near tartan
i might just distribute compressed disk images
little oasis
That don't sound like a bad idea honestly
dense vine
limine template just straight up gives you a way to make an iso
you can take a look at it, just a makefile
near tartan
what if i don't want to
little oasis
limine template just straight up gives you a way to make an iso
I'd rather make my disk image.
dense vine
had to deal with this shit
my neck
little oasis
Mobo wouldn't detect m.2s
Oh God 😳
near tartan
dense vine
tuff
near tartan
cursed but it work
dense vine
what was wrong with it
wait do you have two seperate pcs connected to those two monitors
?
near tartan
wait do you have two seperate pcs connected to those two monitors
yes
near tartan
what was wrong with it
broken bios
near tartan
and?
near tartan
no
i need to see what my main pc is doing and what my server is doing
if my server can't boot i can't ssh into it
so i need to connect to it physically
dense vine
Ah makes sense