#OBOS (not vibecoded)

1 messages · Page 9 of 1

real pecan
#

yeah the first call passes a size of 0 so its a fast return path

flint idol
#

called GetMemoryMap

real pecan
#

yeah it is that bug im afraid

flint idol
#

returned (some value)

real pecan
#

try the second pc

flint idol
real pecan
flint idol
#

[hangs]

flint idol
real pecan
#

first call u do to figure out the size of the memory map

#

then u retrieve it

flint idol
#

inb4 same bug

real pecan
#

its prevalent on older hw

#

i have it only one my one laptop from 2010

flint idol
#

Oof

#

Same bug

real pecan
#

the 2013 one doesnt have it for example

#

are they both same manufacturer or something?

flint idol
#

Subject 1 is from acer

#

#2 is dell

real pecan
#

yeah so how do u do the GetMemoryMap dance again

#

do u call it once then manage internally or

flint idol
#

#3 is lenovo

real pecan
#

and #3 boots fine?

flint idol
#

And is the only that worked on uefi

real pecan
#

i see

#

how old is it?

flint idol
#

2013

real pecan
#

i guess its luck based which manufacturers bothered to backport fixes

flint idol
#

Anyway, until you can workaround that bug, I will have to boot from csm

real pecan
#

yeah

#

is that liek

#

too big of a problem, like slower or something?

flint idol
#

nah

#

I could wait

real pecan
#

ill think of the best way to work around this

#

oh lol i see what limine does

#

it reads the memory map, allocates literally all free regions only leaving 64mib to firmware, then stores the memory map internally

#

so all allocations are done internally without uefi allocators

#

idk if thats necessarily good lol

#

ill come up with a better way

flint idol
#

maybe allocate memory "normally" (i.e., without a special type) and register its type somewhere

real pecan
#

yeah im thinking of just storing the type metadata aside in a tiny array

flint idol
#

then, when making the map, you those memory blocks with special types and change it on the go

real pecan
#

since its page aligned allocations i can use the bottom 9 bits to encode the type

#

so like an array where each allocation is represented by an 8 byte entry

flint idol
#

*bottom 12

real pecan
#

yeah 12 even

real pecan
#

and i can only record like special types, e.g. KERNEL or MODULE, and assume LOADER_RECLAIM if no entry in the array

#

that should bring down the array size to very very small

flint idol
#

I assume you just never free memory

real pecan
#

i do free

#

but only loader reclaim stuff

#

nothing that belongs to the proto

#

i have cleanup handlers that actually release quite a lot of memory

#

alll caches and stuff

#

to make the memory map cleaner

#

I might give you a proof of concept tomorrow

flint idol
#

also

real pecan
#

That you can use until I make a proper release

flint idol
#

when will you slap a lock onto uacpi

#

to make it thread safe

real pecan
#

In the process atm

#

I have a huge table code refactor that makes it thread safe

#

Osi Interfaces are already thread safe

#

Its a gradual transition

#

Carefully inspect everything and either add atomics or mutexes and stuff

flint idol
#

well good thing that by the time I get to porting uACPI, that'll probably be done

real pecan
#

Lol

flint idol
#

it'll be done

real pecan
#

Lmao

#

I want to implement the ability to have insane log traces in uacpi

#

If desired

#

For tracking down bugs like oberrow hell

#

Like entry to every function, every return code

#

Arguments

flint idol
#

which no one but me will use because it seems as if I cannot write memory-safe code

real pecan
#

I might make it compile time

#

Because otherwise feels like too much overhead

#

Even just if statements for checking log verbosity

#

Or like the strings compiled into the program

real pecan
#

How far

flint idol
#

stuck at vmm

#

because school

#

is taking up my free time

real pecan
#

Unlucky

#

Are you making sure its stable by testing in userland halfmemeright

flint idol
#

too much work

real pecan
#

Bruh

flint idol
#

all I really have rn is a context struct

lean glen
flint idol
#

with an rb tree

real pecan
#

Let me port that to C

lean glen
#

make uacpi use C++

#

C++ wrapper when actually

real pecan
#

Is that really needed tho lol

lean glen
#

yes

#

always

real pecan
#

Damn

real pecan
flint idol
#

it's freebsd's KEKW

real pecan
#

Lol

flint idol
#

last time I was working on the vmm I left off at MmH_InitializeContext

#

also I have to code with a cast on

#

which is annoying af

#

because my arm is in a weird position to type

real pecan
#

Damn

#

When did you break it

lean glen
#

static or reinterpret

#

or const even?

flint idol
#

dangers of going outside

flint idol
real pecan
#

Or bit

real pecan
flint idol
#

I first had a sling

#

then I got a check up with the doctor today

lean glen
#

i dont think i could happily live with one arm

flint idol
#

turned out to be a fracture

lean glen
#

literally all my hobbies involve me using two arms

flint idol
#

basically what happened is was playing football (or soccer in north american english)

#

went to slide tackle

#

my arm was kind of extended

#

and then I landed on it

#

probably causing it to hyper-extend

real pecan
#

I had a pretty severe fracture for my left arm when I was a kid

lean glen
#

it probs broke because of your body's weight on it

flint idol
real pecan
#

Its not a pleasant experience

lean glen
#

never had a fracture

flint idol
#

but of course it's my non dominant arm

lean glen
#

the worst i got is open heart surgery trl

flint idol
#

so my teachers can still make me work

#

on paper

#

well only < than a month left for school

#

I'll be mostly fine

sharp pike
real pecan
#

I mean I guess that works, like I already do that for bios

#

I just dont like the 64mib firmware limit

sharp pike
#

and yeah it's not that slow, if it was slow i would do something else

real pecan
#

And also the fact that the map can change technically

#

E.g. a pci device gets hotplugged and u dont get the bars in your limine map

#

They won't be mapped

#

As a result

sharp pike
#

yeah that's taken into account

real pecan
#

How?

flint idol
#

magic

sharp pike
#

after boot services are exited

real pecan
#

What happensb

sharp pike
#

actually i am not sure

#

i cannot recall off the top of my head

real pecan
#

Or actually it will probably panic

#

Because your memory map key will be outdated

#

You won't be able to exit boot services?

#

Idk

sharp pike
#

nah

#

it uses the key it gets right before the exit boot services call

#

as per spec

real pecan
#

That makes sense since firmware allocates memory left and right as well

sharp pike
#

yeah

real pecan
#

So what happens at the end, does it re parse the entries?

sharp pike
#

yeah

real pecan
#

If not then you won't map those

sharp pike
#

where is the firmware supposed to allocate a PCI BAR? unless you mean in some reserved region

#

then i guess you just won't get it

#

but it's not like it matters

real pecan
#

Reserved yes

sharp pike
#

Limine doesn't really map or not reserved regions

real pecan
sharp pike
#

not anymore

real pecan
#

Oh ok

sharp pike
#

it's a deprecated thing

real pecan
#

Yeah thats kinda idk

sharp pike
#

it's good, you're supposed to do it manually anyways

real pecan
#

That means u cant technically access any devices before vmm init

sharp pike
#

you shouldn't gather any BAR info from the memory map

sharp pike
real pecan
#

Yeah but what if its above

sharp pike
#

remember that the first 4GiB are always unconditionally mapped

#

then you'll have to map it yourself

flint idol
sharp pike
#

also the framebuffer is always mapped because that's not a reserved type entry

real pecan
#

Anyway if the spec doesn't guarantee it it doesn't matter

#

From a usability standpoint I dont really like that

sharp pike
#

i mean even if it did guarantee it it wouldn't matter

#

legacy BIOS is all over the place when it comes to device to memory map mappings

#

so it is never something reliable to begin with

real pecan
#

Yeah

#

Nothing about this shit is exactly reliable

flint idol
#

save this convo so new osdevs who think that a bootloader is the way to go think otherwise

sharp pike
#

lol

real pecan
#

Yeah its like

#

Some firmware is just insane

#

Oberrow bios devs decided to just zero out everything other than RGB, who gives a shit about X amirit

flint idol
#

"if windows boots it, it's fine"

sharp pike
real pecan
#

Huge reserved regions and 1g page support typically go hand in hand but yeah

sharp pike
#

typically... except on qemu where sometimes you get those with 2MiB pages only

real pecan
#

On qemu there's a huge AMD hyper transport hole at the top

sharp pike
#

yeah

real pecan
#

Its kind of annoying to deal with

#

Its 128gb I think

sharp pike
#

yeah something like that

flint idol
#

because of the "AMD ATOMBIOS"

#

thing

#

or nvm

#

I think that computer might have an amd gpu

#

and I never knew it

real pecan
#

Yeah outsourced to India to get to the state where windows boots for $2 an hour

#

Is my bet

flint idol
#

why tf do these gpus have something similar to aml

real pecan
#

Oh yeah

#

Thats a thing

flint idol
real pecan
#

Its less insane than aml ofc

flint idol
#

kinda makes me want to write an interpreter for it

real pecan
flint idol
#

except my question is, where is the documentation

real pecan
#

Soo what sort of commands exist there

flint idol
#

wouldn't writing an interpreter for this be basically writing a gpu driver

real pecan
#

I dont see any

real pecan
vernal chasm
#

Second time only reopened the first one xD

flint idol
#

in the rom

#

now what said commands are?

#

I have no idea

real pecan
#

Same

flint idol
#

I'll look for amd documentation

#

if it exists

real pecan
#

I get how it works but I dont get which command is which table

flint idol
#

it seems like there is a command header

#

which contains a list of supported commands I think

#

or maybe it's a table of commands, indexed by the command's value

#

which then has some opcodes that you can run

#

to execute said command

real pecan
#

Yeah but what the hell is the value supposed to mean

flint idol
#

I made the readme

#

I put instructions for how to build

#

and fixed a bug with the build system

flint idol
#

I'm hoping that tomorrow I can get back to developing the vmm after I refresh my memory on the working-set

#

I think my 3rd kernel was cursed, but was really really good at hiding it

#

especially considering my spin lock was broken for nearly a year

vale nymph
#

how

flint idol
#

something to do with cmpxchg

#

I've found the commit I made to obos exactly one year ago

#

I used to use a very good build system

#

called compile.sh

#

I also found the acpi shutdown code from the wiki

#

in my project

#

oh my god

#

this code is 💩

thick jolt
#

hows it going chat

flint idol
#

anyway

#

finally, after a couple of weeks

#

I can work on the vmm

thick jolt
#

!!

flint idol
#

bruh

#

school made me forget

#

all I read

#

over those couple days

#

but I ain't re-reading that

#

so I'll just try and remember

#

I think it's good to make the working set a dually linked list

#

and put the dually linked list in a context struct

# 
typedef struct context
{
    struct process* owner;
    struct page_node_tree pageNodeTree;
    page_node_list workingSet;
} context;```
#

pageNodeTree is an rb-tree

#

uhh

#

now what

#

I'll make a function to initialize the initial kernel ctx

#

it'll populate the rb tree

#

then, with that code in place, I'll find some way to populate the working set

#

using this algorithm

Page replacement algorithm

In a computer operating system that uses paging for virtual memory management, page replacement algorithms decide which memory pages to page out, sometimes called swap out, or write to disk, when a page of memory needs to be allocated. Page replacement happens when a requested page is not in memory (page fault) and a free page cannot be used to ...

#

after reading the part of the working set paper concerning the "use bits"

#

it makes sense

lean glen
#

i would recommend using clock

flint idol
#

currently trying to find out how to count memory references

#

for a page

#

for the aging algorithm

#

I know it has something to do with page faults

#

and the present bit

#

I think

weary hound
#

on x86 you have the accessed bit you can scan to see if a page was accessed

#

and dirty to see if a page was written to

flint idol
#

I know that

#

now how might I use that in my "age" counter

weary hound
#

periodically scan accessed bits, if it's 1, you know a page was recently accessed, otherwise you know it was not touched since last time

#

and afterwards clear the bit back to 0

flint idol
#

so would I just like

#

put it in some timer irq

#

but that wouldn't work

#

because irql > dispatch means the vmm won't work properly

weary hound
#

you could just schedule some periodic work?

devout niche
#

i don't do page aging yet (only two level fifo, which is similar in characteristics to clock or second chance) but that's how i will do it

flint idol
#

ok time to rather:
design a timer interface (boring)
just hook up the vmm to the timer irq used by the scheduler (easier, but I probably shouldn't)

#

this will be boring, designing a timer interface 🫠

flint idol
#

for now it'll just be using one timer

#

for which the arch provides a driver for

#

and there will be timer objects

flint idol
# 
typedef struct timer {
    union {
        timer_tick deadline; // periodic=false
        timer_tick interval; // periodic=true
    } timing;
    bool periodic;
    timer_handler handler;
} timer;```
#

seems good enough

#

for the timer object

real pecan
#

i would go for an enum

#

because u already do have an enum just a very limited one

flint idol
real pecan
#

yeah

# 
enum timer_type {
    TIMER_TYPE_DEADLINE,
    TIMER_TYPE_INTERVAL,
}
flint idol
#

yeah good idea

# 
typedef enum {
    TIMER_MODE_DEADLINE,
    TIMER_MODE_INTERVAL,
} timer_mode;
typedef struct timer {
    union {
        timer_tick deadline;
        timer_tick interval;
    } timing;
    timer_mode mode;
    timer_handler handler;
} timer;```
real pecan
#

yeah

flint idol
#

I think the timer will run at 4000hz

#

Because that sounds fair, 1/4 of a ms

#

who could possibly need a smaller unit of mesurement than that?

#

I mean my kernel would shit itself with a higher frequency than that

ornate ginkgo
#

I mean why limit yourself

#

let the hardware decide the limitations of your timers

flint idol
#

and if the timer runs too fast for the cpu to catch up

#

the kernel shits itself

ornate ginkgo
#

ok.

flint idol
#

I could add support for that I guess

#

but then there's still a lot of timer IRQs

real pecan
#

im serious

#

sound callbacks are very timing sensitive

ornate ginkgo
#

check current time, handle any expired timers, check current time again, repeat

real pecan
#

if u dont fill the buffer in time u will hear garbage instaed of audio playing

ornate ginkgo
#

to some extent, eventually you'd want to exit out of the handler ofc

flint idol
#

I could just make some "timer ticks" global (atomic) variable that's incremented in the handler

#

and differ the work of checking for expired timers to another thread

#

I think I'll use the hpet for the timer on x86-64

#

and use the lapic for the scheduler timer only

devout niche
#

i just do in the region of 1/50th of a second

vernal chasm
flint idol
#

I'm done the bulk of the timer interface

#

i.e., the dispatcher thread and some utility functions (one of which needed fixed point arithmetic)

#

I need to implement the HPET with IRQ support

#

So I'll need to also implement the I/O APIC (also known as ctrl+c ctrl+v from the previous kernel)

flint idol
#

This rewrite, I might support multiple I/O APICs

#

what could possibly go wrong?

flint idol
#

I'm done the madt parsing code for the IOAPIC-related entries

flint idol
#

I'm done writing the IOAPIC code

#

I didn't even end up taking any code from my previous kernel

#

except for structs

#

so now, hpet code

flint idol
#

I was making an organization for obos on github, because why not you know (could be useful in the future)

#

but conviniently, someone already made one /s

#

that just so happens to be completely empty

#

no one even owns it

#

maybe just oberrow_os

#

but that's too long

#

I'll just rename OBOS

#

name suggestions are welcome

#

I'll name it after a snake

#

because why not\

#

the taipan snake

#

because

#

taipan snakes are very venomous

#

just like the code in OBOS

#

note: this is not final

#

anyway I'm going to sleep

unkempt pawn
vale nymph
#

ABOS (Allocator Bug OS)

flint idol
#

@real pecan Can you compile hyper_install-win64.exe as release

#

and not debug

#

so it can work on school laptops without vs

real pecan
#

damn it is compiled in debug?

#

yeah ofc

flint idol
#

well it says ucrtbased.dll doesn't was not found

#

on execution

#

and VCRUNTIME140D.dll was not found

#

so I'd assume it's compiled in debug

real pecan
#

Yeah

#

Thats definitely not intended

flint idol
#

I'd be able to build obos on a school laptop otherwise

#

(don't ask how I got the entire toolchain)

#

(and a command prompt)

#

well the "command prompt" is really just a dumb script

#

that feels like cmd

#

as for the toolchain

image.png
#

I just extracted some files

#

I don't regret making obos buildable on windows

real pecan
#

Lol

ashen goblet
flint idol
#

I told my friend about my name suggestions

#

and he said TaipanOS was a good name

compact idol
flint idol
#

No

#

Because it sounded cool

flint idol
#

Quote of OBOS

image.png
real pecan
#

Lets hope its not in the allocator

flint idol
#

nah it's going in the VMM

#

like every file

#

I have no idea what any of the functions in the VMM do anymore

#

well I know what they do

#

but how

#

hehehehhe

#

barely

flint idol
#

Kernel Panic on CPU 0 in thread -1, owned by process -1. Reason: OBOS_PANIC_FATAL_ERROR. Information on the crash is below:
IRQL on call of the dispatcher is less than the IRQL of the vector reported by the architecture ("Core_GetIrql() < OBOS_IRQ_VECTOR_ID_TO_IRQL(frame->vector)").

#

This is being caused by gs_base being zero

#

but I conditionally swapgs

# 
    test qword [rsp+0xB8], 0x3 ; User code
    je .no_swapgs1
    mov rax, [Arch_KernelCR3]
    mov cr3, rax
    swapgs
.no_swapgs1:```
#

and the same thing is duplicated for the other side of the common stub

#

at this point I want to reset this commit

#

and start it from the beginning

#

it has waaaaaaaaay too many stuff in it

vernal chasm
real pecan
empty kernel
flint idol
#

It has a lot of unrelated stuff

unkempt pawn
#

i feel its better to just rewrite a few files rather than the entire kernel, then you'd get it done lol

#

its like game engine development

#

but I guess this is a general program engine

flint idol
#

I made it access nullptr as a way to ASSERT the condition on swapgs

flint idol
#

well all CPUs also have their scheduler timer enabled

#

But it happens before any CPU gets the chance to get its timer IRQ called, as this panic is in the irq dispatcher

#

the per-thread gs_base is initialized

#

oops

#

I forgot to modify the offset of something that uses interrupt_frame in asm

#

after I changed it to include cr3

#

still doesn't fix anything

vernal chasm
#

Well I didn't specify a full kernel rewrite...

flint idol
#

I fixed the bug

# 
pop rax

test qword [rsp+0xB8], 0x3 ; User code
#

was the bug (rsp should've point to a full interrupt context, but it didn't as a result of the pop)

flint idol
flint idol
#

I was able to get the HPET working

#

In periodic mode with IRQs enabled (routed through the IOAPIC of course)

#

Now hopefully it doesn't take prioirity over the scheduler

#

maybe I should make timers run at a higher IRQL

flint idol
#

the timer can now only be called at IRQL 3

#

except now it hangs somewhere

flint idol
#

after fixing some priority stuff, I get this:

Kernel Panic on CPU 3 in thread 6, owned by process 1. Reason: OBOS_PANIC_ASSERTION_FAILED. Information on the crash is below:
Assertion failed in function Core_Yield. File: C:/Code/obos/src/oboskrnl/scheduler/schedule.c, line 227. !(oldIrql & ~0xf)```
#

an invalid irql

#

in the yield function (only ever happens in thread 6 after the yield)

# 
void Core_Yield()
{
    irql oldIrql = Core_RaiseIrqlNoThread(IRQL_MASKED);
    OBOS_ASSERT(!(oldIrql & ~0xf));
    if (getCurrentThread)
    {
        CoreS_SaveRegisterContextAndYield(&getCurrentThread->context);
        OBOS_ASSERT(!(oldIrql & ~0xf)); // <-- Here
        Core_LowerIrqlNoThread(oldIrql);
        return;
    }
    Core_Schedule();
    OBOS_ASSERT(!(oldIrql & ~0xf));
    Core_LowerIrqlNoThread(oldIrql);
}```
#

except there already is an assertion for that same condition after the irql raise

#

so the stack is corrupted somewhere it seems

flint idol
#

Well, removing the print statement for the timer irq made this appear less

#

which indicates that the timer irq is happening too fast

#

After exactly 10000 timer ticks, the kernel hangs while trying to acquire a spinlock

#

ok I think I fixed it

#

the function for readying threads always assumed they were blocked/unreadied

#

but the timer irq was readying an already-readied thread

#

causing goofy stuff to happen

flint idol
#

I was wondering why my code directory for obos was so large

#

then it turned out I had an 8gib qemu log

narrow temple
#

ik the message is very old but how did you implement the file path when handling an exception?

flint idol
narrow temple
flint idol
#

It's such an old version of the kernel

narrow temple
#

I could check the source code if so

flint idol
#

Well I think it used the __FILE__ macro

#

The newest version of the kernel just does a reg dump on exception

#

Along with some more info

narrow temple
flint idol
#

My scheduler is kind of bad regarding priorities. That's because the scheduler raises priorities a lot, which is good, except there are only 5 priorities, which means priorities are almost useless. So, my solution to this is to add 4 sub-priorities per prioritity

#

Basically, how this will work is:

# 
typedef enum
{
    THREAD_PRIORITY_INVALID = -1,
    THREAD_PRIORITY_IDLE = 0,
    THREAD_PRIORITY_LOW = 4,
    THREAD_PRIORITY_NORMAL = 8,
    THREAD_PRIORITY_HIGH = 12,
    THREAD_PRIORITY_URGENT = 16,
    THREAD_PRIORITY_MAX_VALUE = THREAD_PRIORITY_URGENT+4,
} thread_priority;```
#

and some other changes

#

This way, threads that are being starved won't have the same priority as ones that should be (a lot?) more important than them

#

I'll also make it so that threads' priorities can be raised, so long as they stay within their priority range

#

An example of my scheduler pretty much sucking is with the timer dispatcher thread

#

It's urgent, and is constantly being woken up

#

yet it barely runs

#

(it's overtaken by the idle thread)

#

TODO: Fix scheduler

real pecan
#

is obos stable now

flint idol
#

Hehe no

vale nymph
#

Stability is overrated, just reboot and try again

flint idol
#

It is on qemu

#

Just on test subject 1 (#2 is awaiting the bug fixes to hyper), it panics

#

but only prints:

STOP
Ke```
#

so I have pretty much no idea what went wrong

#

all I know is that it was somewhere while handling the HPET irq

flint idol
#

(somewhere)

lunar narwhal
#

how

flint idol
#

let me think

#

I have no idea how I thought of this

#

ok I think I fixed it

#

offending lines:

if (++list->noStarvationQuantum < Core_ThreadPriorityToQuantum[THREAD_PRIORITY_MAX_VALUE - priority])
        return;```
#

it was literally priority inversion lol

#

except wait

#

I interpreted what it meant too fast

#

except wait, now the problem is suddenly gone

flint idol
ashen goblet
#

When does a threads priority get raised?

flint idol
#

every priority list has it's own "quantum"

#

for each yield, the scheduler goes through the priority lists

#

and the scheduler will raise the priority for a couple threads in all priority lists where their quantum < Core_ThreadPriorityToQuantum[THREAD_PRIORITY_MAX_VALUE - priority]

#

hopefully that makes sense

flint idol
#

It seemed to have been the fact the scheduler was starting to look for threads at the head of a list and not the tail when raising priorities

#

well I think it wasn't a bug with the scheduler causing weird stuff to happen with that thread

#

and more so the fact that the thread was just blocked and never unblocked at one point

#

Anyway, time to test the timer interface.

real pecan
flint idol
#

no #2 is the one with the framebuffer issues

real pecan
#

Didn't I give u an installer where thats fixed

flint idol
real pecan
#

Im out of town for the week so no releases

flint idol
#

ok

flint idol
#

*the timing is messed up though

#

hmm now the scheduler magically removes the timer dispatch thread after a bit

#

(only when there is a timer object registered)

flint idol
#

but now, the timer ticks variable is incrementing very, very, slowly

#

(it's supposed to increment on each timer irq)

#

(the timer irq runs at 1mhz)

devout niche
#

that's extreme

#

i have a timer tick every ~1/50th of a second

flint idol
#

[ DEBUG ] HPET frequency: 100000000, configured HPET frequency: 100000000

#

says otherwise

#

oh wait

#

that's 100 mhz

#

the timer irq only does two things:
tells timer dispatcher to wake up
increment timer ticks variable

#

so having it run fast shouldn't be that big of a problem, right?

devout niche
flint idol
#

well it's running on qemu tcg

#

so idk how fast that is

#

but my cpu is reported to be 2ghz

devout niche
#

getting yourself interrupted 100 million times a second will dominate or exceed the execution time on any CPU today

flint idol
#

I mean the timer irq really just runs at the hpet frequency

devout niche
#

consider that means that if the whole process takes 100 cycles on your CPU that will be your CPU spending half its time servicing timer interrupts

flint idol
#

so I'll just change my frequency divisor to 1000

devout niche
#

this is still excessive

#

i've never seen a kernel exceed 1000hz clock tick

flint idol
#

1000 hz is one ms

#

so then how would I get smaller timing events

#

I assume I would just use some timer counter

#

and poll

devout niche
#

high res timers aren't too important generally, you can deal with them separately

flint idol
#

I'll just make my timer run at 1000 hz then

#

and worry about higher precision later

real pecan
#

1000 is still a lot

flint idol
#

all of my previous kernels' timers have ran at 1000 hz

devout niche
#

if you are keen to futureproof you could do dynamic ticks or something from the getgo

#

where you set a oneshot timer for the head of the timer queue's time of expiry, then when your timer interrupt happens you set up the next timer

steep ravine
#

thats quite smart

devout niche
#

the hpet, i don't know anything about it other than that it stands for high performance entertaiment timer or something like that, and rumours i've heard that it has slowness issues

steep ravine
#

i dont know why i never thought of that

vale nymph
devout niche
vale nymph
#

Tickless ftw

devout niche
#

by instead reading the current time when you need to

#

rdtsc, that's quite speedy, a good candidate for this

steep ravine
#

yeah i talked with nooby about getting rid of ticks in my kernel

#

and it basically involved using rdtsc to figure out the current time when needed

#

i like that idea but i cant seem to get around to implementing it

devout niche
vale nymph
weary hound
#

if you have invariant tsc you can just use that to figure out the current time

vale nymph
#

Yeah I will likely use that, then fall back to hpet and if all else fails I will leave it running on a timer to increment a variable every 100 ms or something

steep ravine
#

100ms ☠️

vale nymph
#

Who needs precision

steep ravine
#

right

flint idol
#

audio

steep ravine
#

ntp exists for a reason

vale nymph
# flint idol audio

I mean most of the things use the per cpu timers which have higher precision, this would be just for the timeekeper functions that handle dates and stuff

flint idol
barren merlin
#

you probably should commit a bit more often

flint idol
#

this was only one slight mistake where I got side tracked

#

well not slight

#

but most files changed only had one change,
#include <error.h>

flint idol
#

if anyone was wondering

#

for my own future reference: TODO: Fix timer irq on real hw

#

Anyway, time for the VMM

vernal chasm
flint idol
#

Yes, but then I needed to get some sort of timer interface

vernal chasm
#

And then it wasn't all about vmm kekw

flint idol
#

which needed:
IOAPIC code
Timer code

#

and then there were also the new vmm context stuff

vernal chasm
#

But the timer was for the page eviction?

flint idol
#

for stuff like that

vernal chasm
#

I mean you could've added that at a later time

flint idol
#

But since I figured I'd need a timer interface eventually, I might as well do it now

vernal chasm
flint idol
#

Instead of committing the changes I had for the VMM, I just continued on

#

causing this mess of a commit

vernal chasm
#

oof

#

But I know how that is

#

I do it sometimes too often

#

Or I just implement everything in one commit

#

xD

flint idol
#

sometimes I wonder, "skill issue or ADHD"

#

in case anyone was wondering the difference between adhd and laziness

#

it's that laziness is when you don't wanna do something

#

but adhd is when

vernal chasm
#

O_O

flint idol
#

noradrenaline and dopamine isn't released as much in the brain (or prefrontal cortex idk)

#

or is just broken

#

causing some random ahh proteins in the prefrontal cortex to go oof

#

and make you no focus

#

or something like that

flint idol
#

it isn't as sensitive

#

I think

#

If anyone studies neurobiology, an explanation would be nice

flint idol
#

oh and iirc from the article, the same thing happens when one is tired

vernal chasm
#

Yes, my focus vanishes, and then I fail to be able to visually focus

#

I turn into sid at that point

flint idol
#

I might get treatment for ADHD soon

#

see how it feels

#

because currently, my noradrenaline and dopamine are not noradrenalineing and dopamineing

vernal chasm
#

Damn

flint idol
#

sad am I right

#

anyway, back to osdev

#

ever look at code and wonder

#

how the fuck does this work

vernal chasm
#

old image, but yes...

20230303_234019.jpg
#

That's me when I've been awake for 30 hours

flint idol
#

why are your eyes misaligned

vernal chasm
vernal chasm
#

And the right one is just gone xD

flint idol
vernal chasm
#

wdym O_O

flint idol
#

idk

vernal chasm
flint idol
#

it's something

vernal chasm
#

I mean I barely ever go outside

#

No point to it

flint idol
#

oh btw I found your ip address

#

127.0.0.1

vernal chasm
#

Oh dang

#

It's actually 192.168.50.1 :>

flint idol
#

frick

vernal chasm
#

Yours is 10.0.0.43 :>

flint idol
#

anyway, enough joke ip doxxing before an actual dox happens

#

I feel like something's going to go wrong

vernal chasm
#

I live somewhere in Norway gl finding me 😉

flint idol
#

If I could find your full name

vernal chasm
#

Yeah probably

flint idol
#

I could use some norway db

#

to find you

vernal chasm
#

You aint gonna find my door tho xD

flint idol
#

Ok I have this so far:
'Markus - -'

vernal chasm
#

Oh wow, how could one find my first name I wonder kekw

flint idol
#

@real pecan what's marca's full name

vernal chasm
#

Holy moly :>

#

Anyways I can tell you really quickly there are like 5 people with that name 🙂

#

Maybe even more

ornate ginkgo
vernal chasm
#

Anyways idk why that java error log file was there...

#

Probably forgot to delete it before doing the full add . && git commit kekw

#

Well now it's privated 🙂

flint idol
#

it is with pleasure to announce that, OBOS is stable

#

i.e., it finishes booting without crashing

vernal chasm
#

PogW
Once in a lifetime event

flint idol
#

scroll up in hotgarbage @vernal chasm

vernal chasm
flint idol
#

until my last messages

vernal chasm
#

oof

#

I mean I doubt you found me tho 🙂

flint idol
#

I wanna confirm idk why

flint idol
vernal chasm
flint idol
#

REDACTED

#

is some guy's middle name

vernal chasm
#

Yeah naah

#

😉

flint idol
#

🤨

vernal chasm
#

what

flint idol
#

weird

#

in the address' street view

flint idol
#

probably not you though

vernal chasm
#

Dafuq you on about kekw

#

Also I think a good few people have the same hairstyle that I have xD

#

It's like the most generic hairstyle

flint idol
#

and according to redacted sources, you don't own a car

#

and the guy was getting in a car

vernal chasm
flint idol
vernal chasm
#

Exactly 🙂

#

But yes I don't own a car

#

But I probably will have to get a license soon™️

#

But idk

flint idol
#

@vernal chasm can you count

#

in thecherno

vernal chasm
#

O_O

#

1

flint idol
#

I made a github org for obos

#

profile picture recommendations welcome

#

all repos on my github profile are useless now

#

because all projects there are dead

#

well now there are none except a very old obos repo

#

fact: when I started osdev, I didn't know how version control worked

flint idol
#

or libc

#

or basically anything

viscid nova
#

finally

#

on God

flint idol
#

(I'm going to be honest, I kind of learnt to code a lot of basic stuff on the way)

#

it's like the solar eclipse a couple months ago

#

it was rare and everyone was hyped

#

except only lasted a couple hours

flint idol
#

(difficulty level: impossible)

viscid nova
#

/j

flint idol
#

stop being silly

#

it does nothing

#

(user-related)

viscid nova
#

I gotta actually plan what I'm gonna do with my kernel too

flint idol
#

the kernel log is just a bunch of initialization

#

until it's finished initializing all components

viscid nova
#

I gotta actually set real goals instead of just some ideas of goals

#

but schools been fumbling me over for the past week (and this one too)

#

hopefully this is the last week tho 🤞

flint idol
#

my kernel's goal is: don't crash and make something interesting

#

then worry about userspace

viscid nova
#

on the long run ofc

#

on the short run I need to learn more about osdeving in general

#

to know which steps I'll take exactly

flint idol
#

well it's actually kind of simple, if @thick jolt could almost get bash

#

then basically anyone can

viscid nova
#

LMFAO

#

real tho

#

thing is, idk if I want or not to port mlibc

flint idol
#

(no offence intended)

viscid nova
#

it seems like my kernel will triple in size (amount of shit I'll need to write)

#

if I do

flint idol
viscid nova
#

and following posix strictly is ehhhh

#

but I might though

flint idol
viscid nova
#

getting bash to run will be pretty epic

viscid nova
#

a huge part of it

flint idol
#

"Ah yes, I would like to implement some boring functions in some old ass standard please"

viscid nova
#

especially when I saw how unix sockets worked, it seems boring as shit

#

I feel like if I do implement mlibc it's gonna be more serious of a kernel in a way, but it will be "just another one"

#

that made a posix like kernel with limine and mlibc

#

idk tho, might change my mind later on

#

later on I mean by next week

flint idol
#

anyway gl

#

I feel like I'd gain some reputation for obos if I just stopped saying every little bug that pops up

#

and instead posted whenever I make new features or components of the kernel done

#

or have an idea related to a component

#

anyway, that's something for future me to deal with

#

(and by future me I mean me tomorrow)

viscid nova
#

seems like a kernel with good potential

flint idol
#

I think I waste time describing bugs too

viscid nova
flint idol
#

I probably could've spent a lot less time fixing that one allocator bug

#

if I hadn't shared every small detail

#

but it's too late now

viscid nova
#

at least it got fixed

flint idol
#

yeah

#

now it doesn't matter because it's rewrite 5

#

and not #4 where the bug was

viscid nova
flint idol
#

I have the 2nd most messages for a progress report thread on this server KEKW

flint idol
viscid nova
#

18k when meme

flint idol
#

Nyaux has ~13500

#

messages

viscid nova
#

he had lots of problems in his kernel iirc

#

everytime I accessed his thread it was a different problem atleast

#

so that might be why

flint idol
#

it was

#

possibly the least stable os here

#

sometimes I have the urge to do userspace dev

#

is there a helpline I can call

#

if it goes on

viscid nova
flint idol
#

I can't wait to implement uacpi and watch as the kernel burns to the ground

flint idol
viscid nova
#

just checked mlibc repo again and I remembered I'll have to re-do big part of my VFS 😭

flint idol
#

I'm implementing uACPI with the driver interface

viscid nova
#

and probably I'll re-do my scheduler, but that's not mlibc related, I just think having processes and threads is gonna be nicer than having tasks only

flint idol
#

so I don't have to worry about it

#

for uhhh

#

let me make an estimate for the vmm

#

anywhere from 2 weeks to 5 years

viscid nova
#

shi u don't have a vmm yet?

flint idol
#

no

#

if you knew what the working-set model was

#

you'd know why

#

(you probably just have a glorified page table manager)

viscid nova
#

but the biggest part of my VMM yeah it is the pt manager

flint idol
#

still managing page tables in a glorified way

viscid nova
#

I mean I don't need much else than mapping and unmapping

flint idol
#

this is a virtual memory manager

viscid nova
#

creating pms etc

viscid nova
flint idol
#

I'm implementing the working set model

viscid nova
#

you still do some page table magic cuz u map things

flint idol
flint idol
#

many good kernels do this

#

I'm implementing the algorithm called "aging" hopefully

#

you can find an explanation online

viscid nova
#

ah alr

#

Page replacement algorithm?

flint idol
#

indeed

#

the VMM will still have a "VMA" obvioiusly

#

now if you were wondering how I allocated memory before, I just had some simple allocator

viscid nova
#

wait so it's kinda like page cacheing?

#

at least they say that on wikipedia

#

havent looked at the os wiki yet

viscid nova
flint idol
#

the kernel figures out it's contents though

#

you can find a paper on it in the #virtual-memory pinned messages

viscid nova
#

when this week is over I'll need to take a look at a lot of virtual memory concepts again

#

like the tlb shootdown too

inland radish
ashen goblet
inland radish
#

Yeah

real pecan
inland radish
#

Sure

#

But I was talking about hobby kernels

devout niche
#

since a lot of kernels do still rely on ticking

inland radish
#

that can fire interrupts when a tick counter reaches 0

devout niche
#

so "tickless" is really taken to be a power-saving optimisation in the popular use

real pecan
#

Tickless just means optimized ticks yeah

#

It does tick but on demand and not continuously

#

And not on all CPUs

flint idol
#

so technically my kernel is tickless

#

because there is only a timer on one cpu

#

(not including lapic timer)

real pecan
#

If you do oneshot on demand ticks then yes

flint idol
#

and it doesn't tick itself, it uses the hpet counter

flint idol
vernal chasm
#

A proper tickless would've just been a non preemtive scheduler tho, wouldn't it?

#

As in coroutines...

#

Or fibers

flint idol
#

bruh I want to watch the euros games while doing osdev

inland radish
#

like 1 or 10 seconds

flint idol
#

but I don't have any subscriptions to anything where I can watch it

vernal chasm
flint idol
#

yields

#

idle thread

#

never yields again

inland radish
vernal chasm
#

They'll just swap thread like every cpu clock O_O

vernal chasm
flint idol
#

my kernel can do preemptive multitasking

#

for reasons I forget

inland radish
#

Tickless does not mean cooperative

#

It just means you aren't using a fixed tick rate to control timing

vernal chasm
#

Also tickless would imply you don't have ticks.
Random tick rate doesn't mean tickless xD

devout niche
vernal chasm
#

Hence why I said proper...

#

Since there you don't have ticks at all, it's just "oh the current thread doesn't want to work now, guess I'll switch to the next thread"

devout niche
vernal chasm
# devout niche you can still have timeslices on a kernel which has no tick-concept

You don't need to have a tick value that increases or something, just the point of something interrupting at some time could be thought of as a tick...
So setting a timer with a random time delta to reschedule after a different amount of time every time would still be a ticked system.
Or having it be an external interrupt that is signaled when you click a button or something would still be a ticked system...

#

At least that's how I see it

devout niche
#

it's just a timer irq

#

ticks means periodic and regular (and tickless confusingly includes this with the proviso that on special occasions the ticks can stop for a while)

vernal chasm
#

Tbf even what I said about a non preemtive scheduler could still be seen as a ticked system, since the death of the thread or the time it yields could be a tick...

devout niche
#

if everything is a tick then nothing is a tick

vernal chasm
#

A tick would be an event that happens at some time

devout niche
#

that's just a timer

#

ticks have to be regular and periodic

vernal chasm
#

What about ticks that people get?
They're not regular and periodic :>

flint idol
#

why is my thread being spammed with this

#

anyway, I'm using the freebsd rb tree impl for the first time

#

hopefully nothing goes wrong

#

no pf

#

we good

devout niche
#

queue.h is too

vernal chasm
#

Funnily enough cambridge's dictionary doesn't even have anything on a computer science "tick" word

devout niche
#

strictly speaking their tree is a weak AVL tree now

flint idol
# 
int cmp_page_node(page_node* right, page_node* left)
{
    return !(right == left);
}```
#

I'm trying to understand why I didn't use the address variable

#

but whatever

devout niche
flint idol
#

it returns one when the two things are the same

#

*different

#

and zero otherwise

#

or is it like memcmp

devout niche
#

tree.h requires to be able to establish ordering as well as mere nonequality

#

otherwise it won't know where the nodes should go

flint idol
#

If I can't have duplicate nodes, can I just assert if two nodes are the same

devout niche
#

it needs to be able to find identical nodes so you can look up by key

#

lookup is done by passing a variable of the same type as the node, with whatever the cmp function compares set to the value you hope to find a node matching

# 
static inline intptr_t
ubc_window_cmp(ubc_window_t *x, ubc_window_t *y)
{
    if (x->offset < y->offset)
        return -1;
    else if (x->offset > y->offset)
        return 1;
    else
        return 0;
}

this is how the cmp function should tend to look

flint idol
#

that's basically what I just changed it to

# 
int cmp_page_node(page_node* right, page_node* left)
{
    if (right->addr == left->addr)
        return 0;
    return right->addr < left->addr ? -1 : 1;
}```
inland radish
flint idol
#

After a little bug fix

#

My kernel is using 200kib more memory

#

because it has every (virtual) page on the system saved in an rb-tree

#

(including the hhdm)

#

now I got a slight problem on my hands

#

how tf do I link memory allocations through "OBOS_BasicMM" to the VMM

#

ok I got it

#

nvm

inland radish
#

is your rb-tree not an intrusive data structure

flint idol
#

it is

flint idol
#

I am now going to implement the thing that counts the age and the part that pages in/out shit

#

idk what it's called

#

atm, swap will just be a chunk of mem reserved

#

until I can get a file/partition

devout niche
weary hound
flint idol
#

that gives me an idea

flint idol
#

I should have some event at each thread's end of time slice

#

wait nvm

flint idol
#

but then I realized there is no time wasted except in the idle thread

flint idol
#

Guess who got his cast off

#

I did

#

Anyway

real pecan
flint idol
#

Maybe

real pecan
#

That's my guess

real pecan
#

I almost passed out when mine was removed

flint idol
#

Weird

#

Numb when I got it 9ff

#

*off

#

It smelt like shit

real pecan
#

You have to build your muscles back up

flint idol
#

Now it's stiff

real pecan
#

Yeah

flint idol
#

Doctor said I'll need to do physiotherapy

inland radish
#

How did you guys get in a cast?

real pecan
#

Your brain gets confused

#

When the hand suddenly gets very light

#

Its common

flint idol
#

Slide tackle

#

Land on arm by accident

#

Fracture and soft tissue injury

inland radish
#

Oh, damn

#

And you infy?

flint idol
#

You said plural