and considering its hard to reproduce
#Nyaux
surreal path
it could be related to some kind of race condition
this seems to be a one in a million issue
making it even harder to trigger
fuck
i cannot get it to trigger
guess we will have to live with this bug
pretty much undebuggable
without a way to actually trigger it
(at least for my case)
brisk zenith
i love heisenbugs
ebon needle
copypaste from me 
surreal path
no
ebon needle
is nyaux now have games ???
surreal path
is nyaux now have games ???
yea
ebon needle
waiting for nyaux youtube showcase day 1
surreal path
ill send in dms
elder shoal
Does that still not exist damn
kind root
yep
elder shoal
so uh mathew, when astral
I tried doing that yesterday but I ran into some fucked up mlibc build failure and gave up
I can try again today after I do eberything I need to do
surreal path
I tried doing that yesterday but I ran into some fucked up mlibc build failure a...
i can help with that
whats up
with the build?
elder shoal
fcntl.cpp doesnt build cuz its complaining about a type mismatch with struct file_handle
Or something like that
surreal path
fcntl.cpp doesnt build cuz its complaining about a type mismatch with struct fil...
ill look into it
just after i implement this syscall for doom
so it can actually get the fucking time
for CLOCK_REALTIME with clock_gettime do i HAVE to use the cmos or can i use something more accurate
like what would be BEST to use
brisk zenith
the best (imo) would be maintain a "real time when boot time was 0" value
which would be initialized from cmos (or limine's boot time request)
brisk zenith
that's how i do it
surreal path
may i see it?
brisk zenith
apparently i misremembered: my implementation uses a 128-bit timestamp instead of a 64-bit one and so i had to use a seqlock
https://github.com/proxima-os/hydrogen/blob/a5704b19bf2507ac70acdfc6e5b252444a25f040/kernel/util/time.c#L16 this is where it's initialized
surreal path
i dont think ive asked you but what is seqlock exactly? i know youve told me that atomics basically force the cpu to load things/store things a certain order which makes sense
but what does seqlock exactly MEAN
brisk zenith
seqlock is a way to have low-read-overhead shared variables larger than what the isa can handle atomically
quick quiver
a seqlock uses a sequence number to check if data was modified while you read it
brisk zenith
essentially on read you read a version number (which also encodes whether an update is currently in progress) before and after reading the data
if it was odd (update in progress) or not equal (updated in the middle of the read) you retry the read
on write you first store version+1 (to indicate the update being in progress), then do the writes, then store version+2 (to indicate the update is done)
note that you can't have concurrent writers with seqlocks, you need a separate lock for that
surreal path
ah okay so bascially ur waiting til whatever code is done with it so u can modify it?
brisk zenith
note that you can't have concurrent writers with seqlocks, you need a separate l...
(you can reuse the update-in-progress bit as a 1-bit spinlock)
brisk zenith
ah okay so bascially ur waiting til whatever code is done with it so u can modif...
yeah
surreal path
note that you can't have concurrent writers with seqlocks, you need a separate l...
they cant tho? cause they have to wait til the update is done before they can modify it right
brisk zenith
they cant tho? cause they have to wait til the update is done before they can mo...
that's one way to have a separate lock for it
brisk zenith
yeah
if the variable is rarely written this results in very low read-side overhead
and very high read-side scalability
surreal path
that's one way to have a separate lock for it
huh?
brisk zenith
huh?
by waiting until the update is done (=version is even), you've essentially turned the lowest bit of the version number into a spinlock
surreal path
but dont u already do that for a seqlock????
brisk zenith
on the read side, yes, but that's different since there you don't set the bit
on the write side you don't necessarily check if an update was in progress
thorn bramble
for readers the LSB is a indication that what they are reading is the up-to-date stuff
for writers it could be used as a spinlock
thorn bramble
for readers the LSB is a indication that what they are reading is the up-to-date...
basically if bit 0 is set then you just re-read until it's not set
surreal path
so anyone that wants to write to a seqlock can just do it unwillingly?
but anyone that wants to READ it
brisk zenith
read side: ```
do {
do {
version = atomically get version;
} while (version is odd);
do read;
memory fence to prevent reads from being reordered past the loop condition;
} while (version matches (atomically get version));
write side: ```
do {
version = atomically get version;
} while (version is odd or cmpxchg (version + 1) failed);
do write;
atomically set version to version + 2;
note that due to the cmpxchg on the write side, only one writer can be active at a time: the loop is essentially a spinlock acquire
thorn bramble
don't you have to set bit 0 to 1 first?
brisk zenith
whereas on the read side any number of readers can be active at a time as long as no writers are
brisk zenith
don't you have to set bit 0 to 1 first?
see or cmpxchg (version + 1) failed
brisk zenith
write side: ```
do {
version = atomically get version;
} while (version is o...
now when you know only one writer is going to be active at a time regardless, you don't need the cmpxchg and can instead use a store, which can be faster in some cases
surreal path
write side: ```
do {
version = atomically get version;
} while (version is o...
and how would a write that uses a seperate lock look like
just so i know i get it
thorn bramble
lock(&separate_lock);
atomic_add(&version, 1);
// do the update
atomic_add(&version, 1);
unlock(&separate_lock);
acquire lock;
version = atomically get version;
atomically set version to version + 1;
memory fence to prevent writes from being reordered before the atomic set;
do write;
atomically set version to version + 2;
release lock;
lol we both missed the release 
brisk zenith
note in my pseudocode atomic ops implicitly use acquire/release semantics
the memory fence is important on the read side because release loads don't exist and acquire loads don't prevent prior accesses from being reordered past them
the memory fence is important on the write side because acquire stores don't exist and release stores don't prevent future accesses from being reordered before them
surreal path
so what a memory fenece essentially does is force all prior accesses to be ordered??
if im reading this right?
brisk zenith
with explicit orderings: ```c
static size_t seq_version;
void seq_read_example(void) {
for (;;) {
size_t version = __atomic_load_n(&seq_version, __ATOMIC_ACQUIRE); // acquire prevents do_read() from being reordered before it
if (version & 1) continue;
do_read();
__atomic_thread_fence(__ATOMIC_ACQUIRE); // this turns do_read() into one giant acquire op, preventing the version load from being reordered before it
if (__atomic_load_n(&seq_version, __ATOMIC_RELAXED) == version) break; // relaxed is fine here, we don't care about stuff after this being reordered before it
}
}
void seq_write_example(void) {
size_t version = __atomic_load_n(&seq_version, __ATOMIC_RELAXED); // relaxed is fine here, we don't care about stuff before this being reordered after it
__atomic_store_n(&seq_version, version + 1, __ATOMIC_RELAXED); // ditto
__atomic_thread_fence(__ATOMIC_RELEASE); // this turns do_write() into one giant release op, preventing the version store from being reordered after
do_write();
__atomic_store_n(&seq_version, version + 2, __ATOMIC_RELEASE); // release prevents do_write() from being reordered after it
}
holy hell
brisk zenith
so what a memory fenece essentially does is force all prior accesses to be order...
think of atomics like a dependency tree
- an release store depends on all (both atomic and non-atomic) accesses in the thread that happened before it
- an acquire load depends on some prior release store, and transitively all (atomic and non-atomic) accesses in the releasing thread that happened before the release
- an acquire fence turns all prior loads into one giant acquire load
- a release fence turns all future stores into one giant release store
this implies:
- no access before a release store can be reordered after it
- no access after an acquire load can be reordered before it
- no access after an acquire fence can be reordered before it
- no access before a release fence can be reordered after it
brisk zenith
note that if you want to get language lawyery fences technically only operate on atomic ops, i.e. they can lift relaxed into acquire/release but not regular accesses, but on any reasonable implementation (including gcc and clang) they operate on regular accesses as well
surreal path
also monkuous congrats on mod!!!!
brisk zenith
if you want a full, language-lawyer-correct, seq lock implementation, do_read and do_write need to use relaxed atomic ops
brisk zenith
also monkuous congrats on mod!!!!
thanks
surreal path
🥳
surreal path
note that if you want to get language lawyery fences technically only operate on...
what is language lawyer correct huh?
brisk zenith
language-lawyer-correct means 100% correct according to the C Standard
brisk zenith
if you want a full, language-lawyer-correct, seq lock implementation, do_read an...
but that often imposes really inconvenient restrictions, this for example
surreal path
language-lawyer-correct means 100% correct according to the C Standard
idrc
brisk zenith
so most people only care about "this will be correct on any reasonable implementation"
surreal path
with explicit orderings: ```c
static size_t seq_version;
void seq_read_example(...
oh theres a problem with the atomic relaxed line because ur not checking it with the version variable we defined?
brisk zenith
oh oops
surreal path
lol
brisk zenith
fixed now
surreal path
write side: ```
do {
version = atomically get version;
} while (version is o...
what atomic operations (RELEASE, ACQUIRE, RELAX etc) would each be? for this version of this write
brisk zenith
size_t version = __atomic_load_n(&seq_version, __ATOMIC_RELAXED); // relaxed is fine here, we don't care about stuff before this being reordered after it
for (;;) {
if (version & 1) {
spin_loop_hint();
continue;
}
if (__atomic_compare_exchange_n(&seq_version, &version, version + 1, true, __ATOMIC_ACQUIRE, __ATOMIC_RELAXED)) break; // cmpxchg is (partially) a load, so it can be acquire - which means we do not need any fences to prevent do_write() from being reordered before it
}
do_write();
__atomic_store_n(&seq_version, version + 2, __ATOMIC_RELEASE); // release prevents do_write() from being reordered after it
the pause instruction on x86
brisk zenith
it's not necessary but it helps with hyper-threading performance, hypervisor performance, and (supposedly) power usage
brisk zenith
i guess it would be applicable there yeah
surreal path
epic
@brisk zenith this looks good?
i copied your comments too
so i would understand it reading it later in my code
brisk zenith
lgtm
surreal path
epic
surreal path
@brisk zenith question about your code for hydrogen what is boot_timestamp_seq and why is there uint64_t boot_timestamp_low;
uint64_t boot_timestamp_high;
and what unit is it in, ms? ns? fentoseconds?
brisk zenith
boot_timestamp_seq is the seqlock version field
boot_timestamp_low/boot_timestamp_high are together a signed 128-bit integer that is the number of nanoseconds since 1970-01-01T00:00Z for ns_since_boot=0
aka the current posix time is (boot_timestamp_low:boot_timestamp_high + ns_since_boot())/NS_PER_SEC
surreal path
and boot_timestamp?
brisk zenith
i don't think i have just boot_timestamp anywhere
brisk zenith
i guess i accidentally committed that somehow
brisk zenith
good choice
i'm currently dealing with trying to make an architecture-generic system that can handle both PAE and non-PAE 32-bit paging in one binary and wow this is incredibly annoying
especially without hhdm
brisk zenith
worst part (imo) is that cr3 still only accepts 32 bit addresses even with PAE so the assumption "pmap->root is the top-level page table" is no longer valid
(always allocating the top level page table in <4G memory is a Bad Idea imo, so instead i have one top-level page table for every cpu and copy the new top-level page table to it whenever i switch page tables)
(this isn't as bad as it sounds because with pae the top level page table is only 32 bytes, but it's really annoying to work into the arch generic stuff)
surreal path
that is so genuinely annoying
also monkuous maybe ur other systems faulting is related to that framebuffer issue nyaux still had this one their systems@flat nymph
flat nymph
?
kind root
(always allocating the top level page table in <4G memory is a Bad Idea imo, so ...
the top level "table" must be like 64-byte aligned right
flat nymph
yeah
kind root
and the values are cached right after its loaded
brisk zenith
the top level "table" must be like 64-byte aligned right
32-byte aligned yeah
kind root
so it can be reused
brisk zenith
and the values are cached right after its loaded
not on AMD!
flat nymph
I think I also had issues with framebuffer
flat nymph
On pmOS and someone's PC
kind root
not on AMD!
ah right
flat nymph
i'm currently dealing with trying to make an architecture-generic system that ca...
My kernel supports it
brisk zenith
this is the pae switch function i use
kind root
yeah u have to have a per cpu "magazine" to reload the cr3 every time lmao
flat nymph
My kernel supports it
But I have arch-specific paging functions
kind root
wtf were they smoking when designing 32-bit pae is beyond me
why the hell is cr3 still 32 bits also
flat nymph
But I have arch-specific paging functions
Not specifications of page table geometries per arch
kind root
why the hell are all bits reserved as well
flat nymph
why the hell is cr3 still 32 bits also
How would you load a 64 bit one?
brisk zenith
why the hell is cr3 still 32 bits also
exactly, this is the sole cause for the most invasive changes to my pmap code
flat nymph
You would need a separate register
brisk zenith
How would you load a 64 bit one?
similar to how you load msrs/xcrs
kind root
^
flat nymph
kind root
just make cr3 use two 32-bit regs
flat nymph
And they had those in Pentium
kind root
in both modes
brisk zenith
xcr is newer but msr is since pentium yeah
kind root
cr3h 
brisk zenith
IA32_PDPT
flat nymph
I think I also had issues with framebuffer
I mean RISC-V is also broken
The common denominator is flanterm
kind root
wait this is the nyaux thread
surreal path
yes lol
flat nymph
lounge 3
surreal path
real
elder shoal
loungeos
surreal path
!!!
kind root
uLounge
flat nymph
Though uACPI thread is still winning by the message count
kind root
quickly spam messages here
surreal path
lol
flat nymph
how many messages were related to owc?
supple robin
The common denominator is flanterm
kind root
me when ironclad crashes
flat nymph
I need to learn Ada
surreal path
just to verify this is correct on the kernel side right?
wait HUH????
hold on need to regen and rebuild mlibc
surreal path
just to verify this is correct on the kernel side right?
is this?
whta ze fuck
wait
supple robin
nyaux you really should stop stack tracing userland
surreal path
yea that fixed it
surreal path
nyaux you really should stop stack tracing userland
yea ill fix that
flat nymph
nyaux you really should stop stack tracing userland
just catch pagefaults
supple robin
well my stack trace checks if the return address is mapped
surreal path
just to verify this is correct on the kernel side right?
yea i have no idea because the mlibc typedefs for time_t are confusing me
vscode says its long but
i dont trust it
flat nymph
well my stack trace checks if the return address is mapped
All of my memory reads from user just let it pagefault and catch those
surreal path
also the fact that its int128_t is how i store it in my kernel and nanoseconds is a long???? how is that gonna work?? @brisk zenith
cause it would just get cutoff
right
tawdry mirage
mini-lspci is written in a way that allows adding different backends easily
flat nymph
also the fact that its int128_t is how i store it in my kernel and nanoseconds i...
It's u64 seconds and u64 nanoseconds
Probably
brisk zenith
also the fact that its int128_t is how i store it in my kernel and nanoseconds i...
nanoseconds is the number of nanoseconds to add to time (which is in seconds)
so you return *time = timestamp / NS_PER_SEC; *nanoseconds = timestamp % NS_PER_SEC; (plus all the stuff to handle negative timestamps correctly)
surreal path
number of nanoseconds to add to time??
brisk zenith
so you return `*time = timestamp / NS_PER_SEC; *nanoseconds = timestamp % NS_PER...
converting nanoseconds to timespec is this
surreal path
so like nanoseconds contains the second half of the timespec??
brisk zenith
yeah
surreal path
makes sense
surreal path
so you return `*time = timestamp / NS_PER_SEC; *nanoseconds = timestamp % NS_PER...
and how would i handle negative timestamps?
sorry im really bad at math
flat nymph
Signed seconds probably
surreal path
also should i have a kernel thread consistently update timestamp?
brisk zenith
// C compilers are allowed to (and do) use truncating division, where i.e. (-15 / 10, -15 % 10) = (-1, -5) instead of (-2, 5)
// This ensures that no matter whether the compiler uses truncating division the result is the latter.
__int128_t seconds = (timestamp - (NS_PER_SEC - 1)) / NS_PER_SEC;
*time = seconds;
*nanoseconds = timestamp - (seconds * NS_PER_SEC);
now i do not know how this math works but who cares !!!!
brisk zenith
also should i have a kernel thread consistently update timestamp?
the timestamp you store under the seqlock is the time at boot, you only have to update it if the time warps (i.e. NTP or similar)
you get the real timestamp using stored_timestamp + ns_since_boot()
surreal path
the `timestamp` you store under the seqlock is the time at boot, you only have t...
right that makes sense
brisk zenith
now i do not know how this math works but who cares !!!!
it's similar to how when you want to a rounding-up integer division you do (x + (y - 1)) / y
surreal path
oh i dont even know how THAT works, thats how bad i am at math
trust me
i dont even know how to add fractions
😭
brisk zenith
i don't know how to do that without a calculator either except in the most simple cases
well i know the procedure but not how to execute it
brisk zenith
get the denominators equal and add the numerators
but i do not know the algorithm for getting the denominators equal
surreal path
i dont know either
surreal path
the `timestamp` you store under the seqlock is the time at boot, you only have t...
static void read_shit(void *data, void *variable) {
*(__int128_t*)variable = info.timestamp;
}
struct __syscall_ret syscall_clockget(int clock, long *time, long *nanosecs) {
switch (clock) {
case CLOCK_REALTIME:
__int128_t timestamp = 0;
seq_read(&info.lock, read_shit, NULL, ×tamp);
timestamp = timestamp + GenericTimerGetns();
__int128_t seconds = (timestamp - (1000000000 - 1)) / 1000000000;
*time = seconds;
*nanosecs = timestamp - (seconds * 1000000000);
return (struct __syscall_ret){.ret = 0, .errno = 0};
break;
}
return (struct __syscall_ret){.ret = 0, .errno = ENOSYS};
}
should i add the timestamp with get_ns_since_boot?
brisk zenith
yeah
surreal path
alright
uhh
dunno if thats right
bru
ill regen + rebuild mlibc and see if that fixes it?
still doesnt work
umm
surreal path
```C
static void read_shit(void *data, void *variable) {
*(__int128_t*)variabl...
that is all my code for that
int sys_clock_get(int clock, time_t *secs, long *nanos) {
__syscall_ret ret = __syscall2(SYSCALL_CLOCKGET, (uint64_t)secs, (uint64_t)nanos);
if (ret.err != 0)
return ret.err;
return 0;
}
mlibc side
brisk zenith
```cpp
int sys_clock_get(int clock, time_t *secs, long *nanos) {
__syscall_...
you're not passing clock
surreal path
bru
💀 💀 💀
yea good catch
looks normal i think
still wrong
forgot to rebuild mlibc
💀
edgy pilot
bro stop time travelling
surreal path
its fucking animated
tho that doesnt look right
now it stopped being animated?
timer problems
:cccc
ok so ill explain whats happening
time is being inconsisent
sometimes it gives reasonably values
sometimes no
elder shoal
I hate when I accidentally time travel
surreal path
ok its not deadlocking
with the seqlock
ill try printing timestamp
this looks write yea?
wtf???
yea the math is wrong somehow
no i didnt add limine time
lets see now
its doing the same thing when seconds get to 5?
i store nanoseconds with a size_t
perhaps thats bad?
no still storing it with u128
still the same issue
@brisk zenith i believe the math u sent me is wrong (sorry for the ton of pings 😭 )
brisk zenith
i might've not been clear on this, the fancy math (as opposed to / and %) is only for when timestamp is negative
surreal path
store as u64
its store as u64
edgy pilot
c++lover grammar
surreal path
ok bro
surreal path
i might've not been clear on this, the fancy math (as opposed to / and %) is onl...
okay so i need to check when timestamp is negative somehow??
edgy pilot
bro
surreal path
wasup
brisk zenith
it's just timestamp < 0
edgy pilot
wdym somehow
edgy pilot
it's just `timestamp < 0`
surreal path
yea im idiot
MY BAD
HOLY SHIT
my brain is fried today
sorry i didnt read the question properly
i think my brain is dying
moment of truth
edgy pilot
moment of lies
surreal path
real
yea it didnt solve shit
fuck
am i supposed to set timestamp ?
its not doing the other code path
edgy pilot
you're like an astronaut who depressurised the airlock and then started to put on the suit
surreal path
real
__int128_t timestamp = 0;
seq_read(&info.lock, read_shit, NULL, ×tamp);
kprintf("TIMESTAMP: from boot: %lld\r\n", (long long)timestamp);
timestamp = timestamp + GenericTimerGetns();
kprintf("TIMESTAMP: added with getns: %lld\r\n", (long long)timestamp);
if (timestamp < 0) {
__int128_t seconds = (timestamp - (1000000000 - 1)) / 1000000000;
*time = seconds;
*nanosecs = timestamp - (seconds * 1000000000);
kprintf("TIMESTAMP: time is %ld, nanosecs is: %ld\r\n", *time, *nanosecs);
return (struct __syscall_ret){.ret = 0, .errno = 0};
} else {
*time = timestamp / 1000000000;
*nanosecs = timestamp % 1000000000;
kprintf("TIMESTAMP: different time is %ld, nanosecs is: %ld\r\n", *time, *nanosecs);
return (struct __syscall_ret){.ret = 0, .errno = 0};
}
what i do now
its always different time so timestamp never gets negative
surreal path
```C
static void read_shit(void *data, void *variable) {
*(__int128_t*)variabl...
^
yea i have no fucking clue
surreal path
i might've not been clear on this, the fancy math (as opposed to / and %) is onl...
now u see the thing here is that timestamp never goes negative
meaning that fancy math never gets executed
brisk zenith
yea that sounds about right
surreal path
yea that sounds about right
you mean the code or the behavior i outlined
brisk zenith
both
what does the full function look like? the weird time output might be because it's not intending to use CLOCK_REALTIME
surreal path
alright hold on
static void read_shit(void *data, void *variable) {
*(__int128_t*)variable = info.timestamp;
}
struct __syscall_ret syscall_clockget(int clock, long *time, long *nanosecs) {
switch (clock) {
case CLOCK_REALTIME:
__int128_t timestamp = 0;
seq_read(&info.lock, read_shit, NULL, ×tamp);
kprintf("TIMESTAMP: from boot: %lld\r\n", (long long)timestamp);
timestamp = timestamp + GenericTimerGetns();
kprintf("TIMESTAMP: added with getns: %lld\r\n", (long long)timestamp);
if (timestamp < 0) {
__int128_t seconds = (timestamp - (1000000000 - 1)) / 1000000000;
*time = seconds;
*nanosecs = timestamp - (seconds * 1000000000);
kprintf("TIMESTAMP: time is %ld, nanosecs is: %ld\r\n", *time, *nanosecs);
return (struct __syscall_ret){.ret = 0, .errno = 0};
} else {
*time = timestamp / 1000000000;
*nanosecs = timestamp % 1000000000;
kprintf("TIMESTAMP: different time is %ld, nanosecs is: %ld\r\n", *time, *nanosecs);
return (struct __syscall_ret){.ret = 0, .errno = 0};
}
break;
default:
break;
}
return (struct __syscall_ret){.ret = 0, .errno = ENOSYS};
}
full function
also how i set it at boot
void GenericTimerInit() {
if ((!arch_check_kvm_clock()) && !arch_check_can_pvclock()) {
Timer = static_cast<void *>(new HPET);
} else {
Timer = static_cast<void *>(new pvclock);
}
info.timestamp = limine_boot_time.response->boot_time;
}
as scheduling isnt active when timer is inited its fine to just
brisk zenith
yeah idk that all looks right
surreal path
do this without going through the lock
surreal path
yeah idk that all looks right
i pushed the code, maybe take a look if you'd like to
brisk zenith
i pushed the code, maybe take a look if you'd like to
(sorry for the late response) you don't seem to be using 128-bit arithmetic for kvmclock intermediates, this means it'll wrap every few seconds, perhaps that's what's causing it?
surreal path
(sorry for the late response) you don't seem to be using 128-bit arithmetic for ...
wdym?
brisk zenith
one of the intermediate values for kvmclock to nanosecond conversion increases extremely quickly
surreal path
rght
brisk zenith
if you use 64 bit arithmetic for the calculation it'll overflow within seconds
brisk zenith
i don't have it cloned locally and i'm about to go to sleep, but if you want to try it yourself it's this calculation https://github.com/proxima-os/hydrogen/blob/8491581f0f4e4b9ece7f620b6674e43565289cc5/arch/x86_64/private/kernel/x86_64/kvmclock.h#L46
hexed sluice
that's cool
surreal path
!!!
worldly condor
nice!
cinder plinth
Love the managarm nerding in the background
surreal path
(sorry for the late response) you don't seem to be using 128-bit arithmetic for ...
im noticing a difference between pvclock and hpet, where the intial boot animation for doom doesnt play on pvclock and it just looks more choppy on pvclock in general weirdly, i know it def has something to do with the timer but yea
rich trellis
🔥🔥🔥🔥🎉
daring juniper
Yaaaaaay! 🎉
mossy belfry
we got guis before disk drivers
molten grotto
surreal path
<:poggers:1337923490329792552>
epic ik
surreal path
now why is pvclock being weird with doom i have ABSOLUTELY no idea
surreal path
what are you want to develop now
input
surreal path
now why is pvclock being weird with doom i have ABSOLUTELY no idea
ive printed the values and they look very similar to the HPET so
no idea
could be the seqlock implementation?
no
seqlock is used regardless of timer type
edgy pilot
there's always the same answer to every problem imaginable
surreal path
which is
edgy pilot
skill issue
surreal path
crazy
edgy pilot
I know
surreal path
ive printed the values and they look very similar to the HPET so
now to be fair the pvclock implementation is very similar to monkuous's for getting the nanoseconds so idk exactly
surreal path
broken cpu
lmao
kind root
ai cpu that fixes bugs at runtime when
surreal path
real
maybe its because wsl is using hyperv clocksource
and that is probs using something else
plus its a nested vm
so idk
i would try this on native linux to verify this theory but
kind root
vms in wsl dont have a stable tsc iirc
surreal path
then thats probs it
brisk zenith
i would try this on native linux to verify this theory but
i'll try if you send an iso
surreal path
i'll try if you send an iso
ok hold on
iso is gonna still be big cause no initramfs compression
cinder plinth
what i did for clock gettime is just time_at_boot + rdtsc() / freq essentially
surreal path
speaking of which i should add that to nyaux
kind root
just compress the iso
surreal path
kk
cinder plinth
iso is gonna still be big cause no initramfs compression
limine can do it for you
surreal path
wait really?
cinder plinth
yeah
surreal path
whats the option
cinder plinth
idk youd have to ask mint
kind root
mint when people ping instead of reading docs
surreal path
real
cinder plinth
but I remember I had a conversation with mint and pitust about tinf and stuff (which is used by limine to uncompress)
brisk zenith
in the changelog it says this for limine 8.0.0 - Removed support for GZ-compressed files (and internal Limine protocol modules).
cinder plinth
ah
surreal path
in the changelog it says this for limine 8.0.0 ```
- Removed support for...
fuck
:c
surreal path
but I remember I had a conversation with mint and pitust about tinf and stuff (w...
maybe i should port tinf to nyaux then
kind root
did u compress it
surreal path
yea
kind root
how is it so big
cinder plinth
compress it again
kind root
wtf did u put ther elmao
surreal path
its just the iso
kind root
i mean on the iso
kind root
how did u make it bigger than ubuntu if u only support doom
brisk zenith
how is it so big? proxima's initramfs with bash+coreutils+gcc-libs is only 84.3 MiB uncompressed
surreal path
yea no idea
brisk zenith
and that's with debug info for everything
surreal path
i compile everything with debug info
idk could be my recipes
anyways
should be uploaded soon
kind root
its 400mb
btw i recommend speedcrunch for this sort of stuff
u just type number / gibi and it shows u
surreal path
never hard of that
kind root
works with hex etc
surreal path
heard*
kind root
very nice for converting shit like that
surreal path
sounds cool
brisk zenith
i just type whatever/1024/1024 in my app launcher and it shows the number of mebibytes
surreal path
alright epic
brisk zenith
can u have a compressed ustar tar btw?
no variant of tar has any special compression handling
surreal path
i thought tar.gz might be its own format
brisk zenith
tar.gz is just uncompressed tar piped through gz
tar.xz is just uncompressed tar piped through xz
etc
kind root
i just type `whatever/1024/1024` in my app launcher and it shows the number of m...
yeah that works, the cool thing about speedcrunch is it keeps the history and u can use ans to access the most recent one etc
cinder plinth
well it didnt suck like it worked
kind root
every self respecting osdever made at least two filesystems:
- a custom shitty ramfs
- a custom shitty fat clone
brisk zenith
i have made neither
kind root
L
brisk zenith
guess i should get on that
surreal path
didnt make fat clone or a shitty ramfs either tbf
cinder plinth
well it didnt suck like it worked
but atp the decoding wasnt worth it when you could just lz4 the archive and get similar speeds and better compression
kind root
didnt make fat clone or a shitty ramfs either tbf
do it 
cinder plinth
mine was essentially TAR with a weird custom compression algorithm
because I was trying to solve the problem of slow boot times
kind root
RLE
surreal path
slow boot times is due to the fuckin pmm in nyaux
cinder plinth
RLE <:trl:829316641191886888>
huh?
surreal path
it inserts a pmm node structure into every page via the hhdm
kind root
best compression algo
surreal path
i really should write a better pmm
kind root
yes
cinder plinth
best compression algo
yea it was basically that
surreal path
maybe get a few pages to throw the pmm structures in which would point to the pages and not be in the pages themselves
cinder plinth
yea it was basically that
each block of unique data had its own block
which could be referenced by the files
surreal path
maybe get a few pages to throw the pmm structures in which would point to the pa...
this is probs how the impl for most freelist allocators work
kind root
i mean deflate is like this as well its just two separate compression algos combined
cinder plinth
so that avoids data deduplication
kind root
do a buddy
surreal path
https://files.catbox.moe/q60gie.gz @brisk zenith
cinder plinth
it inserts a pmm node structure into every page via the hhdm
I would just do it on a per-region basis
surreal path
I would just do it on a per-region basis
?
surreal path
do a buddy
buddy sounds painful
kind root
nah its not painful
brisk zenith
buddy isn't as hard as it sounds, but imo there are very few reasons to prefer it over just a regular old free list
brisk zenith
faster to initialize a buddy in general
if you only have one page per free list entry sure but it's trivial to add a counter
on init, for every memory map entry one free list entry
cinder plinth
the fact that NT pretty much still uses a free list to this day tells me it's good enough
brisk zenith
on alloc, remove one from the entry
on free, add an entry with 1 page
cinder plinth
yea that's what I said
surreal path
also im gonna assume ill need continuous phys pages for shit like DMA soon
kind root
on alloc, remove one from the entry
on free, add an entry with 1 page
u mean like have contiguous freelist entries?
sounds like it would fragment instantly
brisk zenith
avoiding fragmentation is not an objective
cinder plinth
also im gonna assume ill need continuous phys pages for shit like DMA soon
no because good devices dont need it
surreal path
?
kind root
avoiding fragmentation is not an objective
ig
cinder plinth
sounds like it would fragment instantly
worst case, everything degrades to page-size at runtime which is equivalent to initializing all the pages
which doesnt do anything
kind root
if u attach to __alloc_pages with bpftrace u will see quite a lot of allocations bigger than order 0
cinder plinth
since it's still O(1)
kind root
usually in the network stack at least
they do use kvmalloc which can fallback to vmalloc usually
but still
brisk zenith
u mean like have contiguous freelist entries?
basically this ```c
page_t *allocate(void) {
page_t *page = free_pages;
size_t idx = --page->free.count;
if (idx == 0) free_pages = page->free.next;
return page + idx;
}
void free(page_t *page) {
page->free.count = 1;
page->free.next = free_pages;
free_pages = page;
}
void add_region(page_t *page, size_t count) {
page->free.count = count;
page->free.next = free_pages;
free_pages = page;
}
yeah
surreal path
no because good devices dont need it
im still confused what u mean by this lol
kind root
good devices support scatter-gather
cinder plinth
modern/good devices use scatter gather lists
surreal path
scatter gather?
kind root
old devices u can still DMA using IOMMU
surreal path
interesting
kind root
scatter gather?
a list of fragmented physical pages to use as one DMA region
or even sub pages
kind root
so it allows e.g. read(2) to read with one request even though the physical memory for page cache is fragmented
surreal path
old devices u can still DMA using IOMMU
to this day i still dont know how an iommu works lol
surreal path
so it allows e.g. read(2) to read with one request even though the physical memo...
thats cool
kind root
it just makes physical address u give to devices be virtual
surreal path
holy thats cool
kind root
that use a special translation table only usable from the device
surreal path
why is it supposidly tricky tho
brisk zenith
https://files.catbox.moe/q60gie.gz <@938223886439436320>
alright what am i testing again
kind root
why is it supposidly tricky tho
because there are IOMMU groups
surreal path
alright what am i testing again
doomgeneric using pvclock
see if its stable on native linux
kind root
so there are multiple devices using one iommu usually etc
surreal path
so there are multiple devices using one iommu usually etc
right makes sense
kind root
and u have to carefully program all memory that might be accessible by the device
so its not that trivial
easy to foget something
for example scratch memory u give to xhci
etc
cinder plinth
and it's cpu-specific
which sucks
brisk zenith
what's the iwad path
surreal path
what's the iwad path
/root/doom.wad
i should implement more userspace fs shit ngl, things like opendir or chdir
kind root
but ideally your dma allocator api allows to transparently map via iommu
i forget what its called on linux
brisk zenith
/root/doom.wad
how do i force it to use hpet? i don't have a point of reference
kind root
https://elixir.bootlin.com/linux/v6.15.1/source/drivers/nvme/host/pci.c#L767
heres nvme scatter gather list building code for example
surreal path
how do i force it to use hpet? i don't have a point of reference
ill have to compile a different iso for u
hold on
really should make nyaux kernel options an actual thing
brisk zenith
yeah on proxima it'd just be cmdline: x86.notsc x86.nokvmclock
surreal path
really unweidly to hard code shit like the init program
surreal path
yeah on proxima it'd just be `cmdline: x86.notsc x86.nokvmclock`
yea makes sense
uploading
@brisk zenith https://files.catbox.moe/6pcqog.gz non pvclock iso
brisk zenith
i'll have to compare it with hpet but this is what kvmclock looks like
seems fine to me
surreal path
looks like a wsl issue
kind root
no ps2 driver still damn
surreal path
also damn 5 servers 😭
surreal path
no ps2 driver still damn
thats next up
i want to make an input abstraction interface first
then get ps2
brisk zenith
yeah hpet looks exactly the same
surreal path
yep wsl issue then
surreal path
ofc u would say that
supple robin
xhci when
elder shoal
bro died with the ps2 driver
Im trying to revive but the haters are sabotaging me
brisk zenith
huh that was easier than i thought
surreal path
huh that was easier than i thought
what are u trying to do?
also why is doom on the top left for u?
why is it in the middle for me?
surreal path
Im trying to revive but the haters are sabotaging me
real
brisk zenith
what are u trying to do?
I was just curious how hard it is to port doom
turned out I already had basically everything necessary and only had to add a way for userspace to know where the framebuffer is
it's in the top left because the positioning is handled by DG_DrawFrame and my code is different from yours
surreal path
it's in the top left because the positioning is handled by DG_DrawFrame and my c...
hold on lemme check my dg draw frame
i stole it from astral
it seems like positioning is handled ??
so why is it in the middle then
hmmm
brisk zenith
moffset
surreal path
oh right
brisk zenith
for reference this is what mine looks like
as you can see it just starts from the top left, no offset applied
surreal path
right makes sense
and i see u directly right to it because u have the mmap file shit implemented
surreal path
huh? why does mmap have to go through /dev/mem???
brisk zenith
I should probably have a separate fb device, rn I just use /dev/mem and have the kernel create text files in /dev/fb/<n> that store the fb details
surreal path
right
brisk zenith
huh? why does mmap have to go through /dev/mem???
it doesn't, I just don't have any other devices implemented that map physical memory directly into userspace
surreal path
makes sense
brisk zenith
it'd be trivial to add another but I didn't feel like it
brisk zenith
the way I do it is I have it as a vfs op and the vfs op is a thin wrapper around vmm_map
surreal path
whats the shit u need to do in order to get that working
brisk zenith
depends on the type of file you're mmapping
for stuff like /dev/mem and /dev/fb which map physical memory directly into userspace all you need is vaddr allocation and paging
for mmapping regular files you need a page cache (and copy-on-write if you want to support private mappings)
brisk zenith
yeah a page cache is pretty much the only way to have correct semantics for mmapping regular files
surreal path
:c
brisk zenith
for map_private you could get away with reading from the file into a new anonymous page in the page fault handler, but for map_shared you need to be able to map the authoritative contents of a file directly into userspace, aka you need a page cache
surreal path
:cc
brisk zenith
it's not as hard as it sounds, especially if you only have tmpfs
the hard parts of a page cache are mostly related to page eviction, which on a tmpfs will never happen outside of truncate()
surreal path
and how exactly does mmaping a device working
like u just give userspace the virt addr to the phys address of the device?? or smt like that. sorry if im not understanding i dont rlly know how it works
also i have no idea how page caches work tbf, which is even more pain
surreal path
the hard parts of a page cache are mostly related to page eviction, which on a t...
page eviction?
wdym truncate
brisk zenith
truncate() is the vfs op for setting the size of a file
surreal path
right
brisk zenith
page eviction is when the system is low on memory and needs to kick a page out of the page cache to use as some other memory
surreal path
i see
brisk zenith
and how exactly does mmaping a device working
like u just give userspace the vi...
for devices mmap is pretty much entirely device dependent
surreal path
for devices mmap is pretty much entirely device dependent
imagine u had some memory that is the fb, how would mmaping that work exactly. is there a general concept around mmaping files in general?
brisk zenith
imagine u had some memory that is the fb, how would mmaping that work exactly. i...
the way I do it: /dev/mem's mmap function passes a memory object to vmm_map whose post_map callback (which is called as soon as the region is allocated and mapped but before the vmm is unlocked) uses my paging code to map the allocated area to physical memory
oh and my memory objects have two callbacks: get_page (used by file objects, called by the page fault handler and returns a page_t *) and post_map (called whenever the object is mapped somewhere)
the /dev/mem object doesn't have get_page, so if a page fault is taken in a /dev/mem region a SIGBUS is sent
post_map takes care of ensuring that there will never be a page fault taken in a /dev/mem region (well, actually, it uses this mechanism to deny userspace access to memory that's owned by the kernel, but conceptually speaking it immediately maps the region to physical memory, preventing page faults)
surreal path
the way I do it: /dev/mem's mmap function passes a memory object to vmm_map whos...
memory object??? why callback??? what is SIGBUS??? holy shit im so stupid
i lit cannot understand anything
brisk zenith
a memory object is, conceptually, just some object that can be mapped to memory
a callback is used because it must be done after the region is allocated but before the vmm is unlocked, otherwise a different thread might erroneously get an unhandled page fault
surreal path
a callback is used because it must be done after the region is allocated but bef...
what must be done?? the mapping??
surreal path
a memory object is, conceptually, just some object that can be mapped to memory
okay i guess that makes sense
im assuming in this case the file is what gets mapped
im probs misunderstanding tbf
brisk zenith
SIGBUS is just a signal that says "hey you tried to access memory that's backed by an object, but we encountered an error while accessing said object"
surreal path
right
brisk zenith
it's also what's sent if e.g. an I/O error is encountered while accessing an mmapped file
brisk zenith
what must be done?? the mapping??
mapping the physical memory (since this is /dev/mem)
surreal path
mapping the physical memory (since this is /dev/mem)
so dev mem is like
brisk zenith
memory objects for regular files don't need (or have) this callback because they can handle page faults gracefully
surreal path
a file that reparents physical memory
brisk zenith
a device file yeah
surreal path
yea okay, so bascially on mmap, all of physical memory gets mapped to some addr or ?
supple robin
well user space gives a hint of where it wants it
surreal path
right
supple robin
and mmap returns the actual mapped address
surreal path
right
brisk zenith
not all of physical memory, just [offset,offset+size) (offset and size are parameters to the mmap call)
supple robin
whatever you want them to be
brisk zenith
userspace can give a hint to the kernel for where it wants the mapping to be placed in virtual memory but unless MAP_FIXED is specified the kernel can choose a different one
if MAP_FIXED is specified it must place the mapping at the specified address, if that's not possible it must error and if there's already a mapping there said mapping must be removed
surreal path
right
brisk zenith
are memory objects their own structure??
that's how I do it yes
mossy belfry
have you reached osdev final boss
surreal path
that's how I do it yes
what exactly do they store
surreal path
have you reached osdev final boss
page cache is osdev final boss real
mossy belfry
gpu accelarated doom when
mossy belfry
u have ur priorities straight tho fr
brisk zenith
what exactly do they store
depends on the memory object, the object for /dev/mem doesn't store anything (it doesn't need state) but anonymous memory objects (which are the backing memory for tmpfs regular files) store a radix tree of pages
surreal path
radix tree
brisk zenith
page tables are radix trees if that helps conceptualize it
surreal path
ig
again is there a reason for memory objects in the first place exactly
why is it needed for mmaping files
brisk zenith
they're just a convenient primitive imo
you can implement mmap in other ways as well
surreal path
i mean, what im trying to ask is basically what is the technical reason that memory objects would be used for. other then storing the pages that some file mmap'd or smt (not talking abt /dev/mem)
these questions are rlly stupid sorry ik, im js trying to wrap my head around this
brisk zenith
I just like the separation of concerns, if you're going full POSIX-like there's no reason not to put these callbacks in the file description or the vnode or something like that
that's what linux does I think
surreal path
so theres no reason conceptually that memory objects is rlly needed for mmaping files in general
if im understanding this
brisk zenith
that's what linux does I think
have a post_map-like thing in the file description and a radix tree stored in the vnode
brisk zenith
so theres no reason conceptually that memory objects is rlly needed for mmaping ...
nope it's just my preferred implementation
surreal path
nope it's just my preferred implementation
right, i dont think ill implement memory objects
i mean this is my first implementation for mmaping files at all, i never got this far in kernel dev so i wanna make this shrimple ig
what ill do for /dev/fb<num> for mmaping it is quite simple, unless MAP_FIXED is said ill just pick some random ah vaddr. make the fbs phys addr map to said vaddr and return vaddr to said mmap vfs op
i think that is reasonable
as for mmaping normal files
that will probs be later on the timeline ™️ as i need page cache
surreal path
what ill do for /dev/fb<num> for mmaping it is quite simple, unless MAP_FIXED is...
im assuming this sounds fine and reasonable?
@brisk zenith ?
brisk zenith
as long as "random ah vaddr" isn't literally an RNG it should be fine yeah
surreal path
as long as "random ah vaddr" isn't literally an RNG it should be fine yeah
rng real
alright epic
thanks for explaining this stuff, you really have the patience of a saint
cinder plinth
it doesn't, I just don't have any other devices implemented that map physical me...
you should probs add a way to make that restricted tho
brisk zenith
it already is restricted
/dev/mem does not allow you to access ram owned by the kernel
only firmware ram and mmio
cinder plinth
ah yeah that makes sense
brisk zenith
well it does have an ioctl for allocating physical memory which gives you an fd through which you can access kernel owned ram, but that doesn't count
cinder plinth
what i had in mind for something similar but not really is I give a capability over the whole available physical memory range to init and it carves it up and gives it to the proper servers
so the only "unsafe" process would be init but even then it's just firmware and mmio
brisk zenith
that's what I'd do for a "true" microkernel
hydrogen isn't really a microkernel because other than device drivers everything that a monolithic kernel would have in kernel space is in kernel space
cinder plinth
ah I see
cinder plinth
what i had in mind for something similar but not really is I give a capability o...
I'm not really sure how to do that dynamically tho yet
oh well
I guess the driver could request a physical memory range to init
when registering
brisk zenith
so the only "unsafe" process would be init but even then it's just firmware and ...
also there's no real way around at least one unsafe process like this because acpi requires arbitrary physical memory and pio access
cinder plinth
yeah I know
I'm not sure how to handle that yet
ACPI will just be a special privileged process I guess
(no i will not put acpi in the kernel!)
brisk zenith
I'm not really sure how to do that dynamically tho yet
I'd just make it so any given handle can only access a single contiguous range of memory, and if a device has multiple ranges, the driver gets multiple handles
brisk zenith
ACPI will just be a special privileged process I guess
it doesn't really need special privileges? just make init give it the full unrestricted handle
cinder plinth
what I had in mind involved only one driver having access to a certain memory range
brisk zenith
mmm that doesn't really work because AML sometimes requests hardware memory that might already be claimed by other drivers
for example I believe qemu q35's AML accesses the hpet?
one of the 0xfeX00000 things anyway
I'd just give the acpi/pci process the unrestricted handle (that's necessary for aml anyway) and make it responsible for carving out restricted handles to give to drivers
cinder plinth
I'd just give the acpi/pci process the unrestricted handle (that's necessary for...
oh yeah I guess acpi/pci can do bus enumeration and driver loading
ebon needle
also im gonna assume ill need continuous phys pages for shit like DMA soon
when nyaux buddy allocator
broken depot
Wait Nyaux uses a freelist allocator?
freelists are (relatively) simple but also not super efficient
O(n) worst time
daring juniper
I was just wondering since I'm trying to do a freelist lol (and it kind of sucks)
broken depot
Buddy allocators are cool
Think of it as some extra alignment requirements (each block aligned to its size) and then instead of a list of free pages, it's more like multiple lists.
daring juniper
Was gonna do a buddy, but I think a freelist is slightly faster? (I dunno I could be wrong)
brisk zenith
O(n) worst time
no? they're O(1) in all but the rarest cases
the vast vast vast majority of allocations are 1 page
and for the vast vast vast majority of allocations freelist is the fastest possible allocator (disregarding allocator-independent optimizations like magazines)
daring juniper
ye I thought so
cinder plinth
no? they're O(1) in all but the rarest cases
I think he meant O(n) if you need to allocate contiguous pages
brisk zenith
exactly
cinder plinth
but thats not something you do with a freelist anyway
brisk zenith
those are the said rarest cases
cinder plinth
freelist is pretty much always the fastest
broken depot
No but you should be using buddy for physical memory because it makes doing stuff like superpages easier.
cinder plinth
eh
surreal path
tf is a super page
brisk zenith
super pages (also known as huge pages) are when you use direct mappings in the upper level of the page tables, i.e. 2M or 1G pages on x86
daring juniper
Ohh, okay that makes sense
broken depot
why did yall call it super pages
super -> greater than
broken depot
RISC-V has mega, giga, peta, etc. pages.
broken depot
XD
brisk zenith
RISC-V has mega, giga, peta, etc. pages.
actually it has kibi(1024)pages, and no kilo(1000)/mega(1000000)/mebi(1048576)/giga(1000000000)/gibi(1073741824)/etc pages
ebon needle
tf is a super page
what is tf
broken depot
Tempted to call them überpages just to fuck with anyone reading my srcs
broken depot
actually it has kibi(1024)pages, and no kilo(1000)/mega(1000000)/mebi(1048576)/g...
You'd think so but no, I just gave the official names
broken depot
It also doesn't have kilopages, it starts at mega
broken depot
they're actually officially called that?
yes
surreal path
made logging better
edgy pilot
text
surreal path
?
edgy pilot
isn't it?
surreal path
i mean yea
edgy pilot
text
surreal path
but i wanted to make logging look nicer
broken depot
made logging better
You mean the [time][level] message thing?
surreal path
yes
broken depot
very good
surreal path
not all kprintf instances are converted to the new function
ebon needle
made logging better
try to add DEBUG and INFO
surreal path
there is
ebon needle
make all messages to your log
surreal path
i will
ebon needle
k
broken depot
What do you plan to put in your userspace?
broken depot
How are you gonna do drm? That seems like quite the undertaking with anything but downright ancient cards
ebon needle
udrm
broken depot
If BadgerOS ever gets to DRM, remind me to make it modular enough so that it can be integrated into other peoples' kernels
Because DRM seems to be the big unsolved problem in hobby OSDev land and if I'm going to at least try, you guys might as well be able to use it too right?
broken depot
wait what
How did I not know this
Oh it actually has a multitude of gfx drivers too
surreal path
How are you gonna do drm? That seems like quite the undertaking with anything bu...
it is
but idc
mossy belfry
made logging better
why are ur functions so yappy
me personally
i only print real bad things to stdout
i am working on a logging interface now that i have a vfs working and ext2/fat/iso9660(RO)
elder shoal
If your os doesnt yap is it really an os
surreal path
real
maiden ridge
add gooning
edgy pilot
add stability
mossy belfry
add gooning
when will we get an os called goonix
it'll be like nyarch but all custom and not linux
surreal path
when loglevel
i have loglevel
surreal path
and command line
crazy
little tired for the past few days
tmrw we get to work tho
coral dove
buddy sounds painful
It’s not trust
daring juniper
If you don't mind me asking, how long did it take for the PMM to init before? (used Nyaux as a ref for reworking my PMM, but I feel like I've done something terribly wrong cuz it takes 16 seconds to init, without kvm. 4-ish with kvm)
kind root
Some people say nyaux pmm is still initializing to this day
surreal path
If you don't mind me asking, how long did it take for the PMM to init before? (u...
dw in the next commit u can reference it
surreal path
pmm boots instantly basically
elder shoal
my pmm takes infinity to initialise (I am booting up on a turing machine
surreal path
@brisk zenith i have a question, okay so i have some page structure that points to a page and a next variable
so my question is, how exactly do i make this fast because freeing would be extremely slow like if i added a bool to pnode that says its allocated and then took its phys, freeing would be hellla HELLLA slow, is there a better way to do this without having the pnode structure in the page itself. i know menix has something called a PAGE_DB? im confused as to what it is tbf
cinder plinth
If you don't mind me asking, how long did it take for the PMM to init before? (u...
a few seconds is an insanely long time
like unironically
cinder plinth
<@938223886439436320> i have a question, okay so i have some page structure that...
freeing is just putting it on the list
allocating is popping from the list
cinder plinth
the page db is likely what is typically called a PFN db
surreal path
i lose the ptr and the structure
cinder plinth
its for vmm stuff
surreal path
but i cannot really do this but
like if i just pop off the structure where does that structure go
void *pmm_alloc() {
if (head.next == NULL) {
panic("no memory");
return NULL;
}
panic("no");
pnode *him = (pnode *)head.next;
if ((uint64_t)him->ptr < hhdm_request.response->offset) {
sprintf("pmm(): addr is %p\r\n", him);
}
assert((uint64_t)him->ptr >= hhdm_request.response->offset);
head.next = (struct pnode *)him->next;
allocated.next =
memset(him->ptr + hhdm_request.response->offset, 0, 4096); // just in case :)
return (void *)him->ptr + hhdm_request.response->offset;
}
where does him go
that is what i mean
shut laurel
then you have some pfn db?
surreal path
wtf is a pfn db
surreal path
thats the whole point, that him isnt in the page itself
why else am i rewritting the pmm
surreal path
how is it not in the page
because its slow as shit on init
so im rewritting it to NOT be
cinder plinth
it should be per region
shut laurel
wtf is a pfn db
just a collection of struct page basically
surreal path
it should be per region
i dont do that
surreal path
i dont do that srry
surreal path
just a collection of struct page basically
how would u access it exactly
cinder plinth
old = region->ptr; region->ptr+whaever.. return old;
daring juniper
a few seconds is an insanely long time
I know 😭
shut laurel
how would u access it exactly
some do a virtually continuous mapping where you access it like pfndb_base[phys addr >> page shift], but that requires you to map stuff early
I just do it per region, at the start of every region I reserve some amount of space for every page in the region, then the page meta is stored there
surreal path
wha
cinder plinth
some do a virtually continuous mapping where you access it like pfndb_base[phys ...
virtually contiguous is kinda tricky a bit to setup too, but it's worth it
per region is a bit slower (you dont get constant access time) but it's much easier
shut laurel
virtually contiguous is kinda tricky a bit to setup too, but it's worth it
I was too lazy
surreal path
some do a virtually continuous mapping where you access it like pfndb_base[phys ...
how does phys addr >> page shift get u the right shit wtf
what does pfndb store
wdym by region in this context?
shut laurel
region=memory region/physically contiguous area of usable RAM
surreal path
mhm
okay so wdym by per region, just for context i want my freelist allocator not to care about mutiple pages in one struct
each struct points to the page
cinder plinth
why not
surreal path
and points to the next struct
shut laurel
how does phys addr >> page shift get u the right shit wtf
what does pfndb store
...
the phys_addr>>page_shift, so for example for 0x1000 you shift right by 12, then you get 1 as pfn. That would then access pfndb_base[1], pfndb_base could be some struct page*
daring juniper
I'll making my PMM use a per-region basis, it does seem much quicker than having 1 entry per page
For init anyhow
surreal path
the phys_addr>>page_shift, so for example for 0x1000 you shift right by 12, then...
uh huh
how do u calculate the page shift
and
wdym by struct page*, what is struct page storing here
cinder plinth
page shift is just 12
shut laurel
the page shift is just the 000 part of the architecture page size
shut laurel
yes
surreal path
makes sense
shut laurel
wdym by struct page*, what is struct page storing here
struct page is storing whatever page metadata you need. I store my freelist pointers in them, along some other stuff
surreal path
would it be okay if i just stored a ptr to the actual pnode struct thing
struct pnode {
void *ptr;
struct pnode *next;
}
this for context
shut laurel
what is ptr? or what is pnode
surreal path
also cant i just reverse some pages from the memmap to store said pfndb
surreal path
what is ptr? or what is pnode
ptr points to the physical page
next is the next in the freelist
it is the actual freelist structure itself
shut laurel
thats basically the struct page, i just call it that cause I followed linux name or something
surreal path
right
shut laurel
also cant i just reverse some pages from the memmap to store said pfndb
how do you mean?
surreal path
how do you mean?
that is how im doing that for the freelist itself, i find a memmap usable region that can store the freelist, store it in there and then increase its base and decrease its length by the amount i have "allocated" for the freelist
shut laurel
well its just a list then, but of free pages, so a freelist
cinder plinth
its basically a pfndb already
surreal path
^
shut laurel
well you could loop through all of them
shut laurel
its a physically continuous pfndb xd
shut laurel
thats awfully slow
if you never have to store additional stuff (why would you need to do that if that pnode is all you have) it shouldn’t be slow
shut laurel
like just put on the list, pop off the list
shut laurel
how so
surreal path
```C
void *pmm_alloc() {
if (head.next == NULL) {
panic("no memory");
...
^
where does him go
surreal path
yea then u need a pfndb
shut laurel
if you want to go with the region approach:
make some region struct storing base/length of region
reserve size of region + size of pnode*total pages in region
put all pnodes after the reserved pages onto the freelist
I‘d store the pfn in the pnode, not a hhdm pointer)
shut laurel
then you‘ll want to have some boot allocator that chips memory of the memmap or bump allocates or whatever
shut laurel
then you‘ll want to have some boot allocator that chips memory of the memmap or ...
you‘ll also want to have a way to map memory before your pmm is init‘d with this early allocator
surreal path
ill just do the same i did for the freelist
surreal path
you‘ll also want to have a way to map memory before your pmm is init‘d with this...
what why
i have the hhdm
shut laurel
if you want an actual pfndb
surreal path
define actual here