doesnt even fit in discord
#viperOS
icy comet
south yew
also oh my god that codegen is so bad
south yew
doesnt seem like properly regalloced code to me
you are just shuffling around registers for literally the whole function
aren't you
icy comet
what do you mean by regalloc
south yew
like allocating ir thingies to registers
icy comet
i mean
south yew
and spilling whatever you can't into stack slots
south yew
so not really i see
icy comet
.LFB0:
.cfi_startproc
pushq %rbp
.cfi_def_cfa_offset 16
.cfi_offset 6, -16
movq %rsp, %rbp
.cfi_def_cfa_register 6
movl %edi, -36(%rbp)
movl %esi, -40(%rbp)
movl %edx, -44(%rbp)
movq 40+req(%rip), %rax
movq %rax, -32(%rbp)
movq -32(%rbp), %rax
movq 16(%rax), %rax
movq (%rax), %rax
movq %rax, -24(%rbp)
movq -24(%rbp), %rax
movq (%rax), %rax
movq %rax, -16(%rbp)
movl -36(%rbp), %eax
sall $2, %eax
cltq
addq %rax, -16(%rbp)
movl -40(%rbp), %eax
movslq %eax, %rdx
movq -24(%rbp), %rax
movq 24(%rax), %rax
imulq %rdx, %rax
addq %rax, -16(%rbp)
movq -16(%rbp), %rax
movq %rax, -8(%rbp)
movq -8(%rbp), %rax
movl -44(%rbp), %edx
movl %edx, (%rax)
nop
popq %rbp
.cfi_def_cfa 7, 8
ret``` in my defense this is gcc's equivalent for `-O0`well yeah
movq 40+req(%rip), %rax
movq 16(%rax), %rax
movq (%rax), %rax
movslq %esi, %rsi
imulq 24(%rax), %rsi
sall $2, %edi
movslq %edi, %rdi
addq (%rax), %rsi
movl %edx, (%rsi,%rdi)
ret```i will attempt to add some optimizations then
at some point in the next few days
one pretty easy one i can see is where i do x86asm lea reg, [something] mov reg2, [reg]
can just shorten that to mov reg2, [something]
south yew
you just need an unbad ir/isel/regalloc layer
icy comet
in what sense
icy comet
surely you have dumps or something
ok heres the dump
for that func
i probably should try to assign registers to allocas
maybe i should allocate virtual registers initially
then assign those to physical ones/stack slots
south yew
for that func
okay i actually looked at it
icy comet
okay i actually looked at it
wdym
south yew
except more optimize needed
icy comet
but ill look into some better techniques also
south yew
i think youd get much more value out of trying to forward stores to loads
icy comet
i will look into that then
is that essentially if i have mov T1, 43 mov eax, T1 just eliminating T1 altogether
icy comet
wow its been a while since ive worked on this
currently im in the process of designing the driver system
icy comet
unfortunately all my work on the driver loading stuff are on my pc at home, and im only on my laptop for the next few days
so i will find something else to do
most likely vmm things
icy comet
ok i have a basic virtual mem allocator
nothing fancy like demand paging yet but ill do that soon(probably)
somber citrus
rip, is it usb c?
icy comet
unfortunately not, it's some random lenovo cable
icy comet
aha!
its fixed!
i hit the bottom part of it a few times and now it charges, though i think i broke the fan a bit
icy comet
we have a super basic scheduler now
just cooperative multitasking and kernel threads only
now the age old question
what to do next
i like to work on one thing just for a bit, then move over to something else, then go back to some other stuff, etc etc
i seem to work better when i do that, because im not fixated on the same things constantly
icy comet
ok i think ill be doing some vmm stuff for a while
did the bulk of the work on lazy mapping today, just need to.. map it
but ill be doing that tomorrow since its late and i had some more laptop charging issues
other things to do tomorrow will maybe be cow and whatever other virtual memory stuff i can think of
icy comet
i managed to save all of yesterdays work onto a separate branch before my laptop ran out of charge
somber citrus
haha nice
icy comet
i will be home some time this evening so i can actually get things done
though i will still need to fix my laptop
icy comet
hmm, im not sure where i want to put drivers in the filesystem for now
i guess i can just have / and /dev be tmpfs directories
then a device filesystem mounted to /dev
icy comet
node = fs::LookupPathName("/dev/sample");
node->write(nullptr, 0);``` devfs appears to be working 🥳lazy mapping is also done now
icy comet
got a better build system for the drivers now
should be easy to add more when i want to
im probably gonna work on the scheduler soon
icy comet
im considering putting all the kernel init stuff into a thread so i can have proper blocking instead of using spinlocks everywhere
icy comet
void Block()
{
auto current = threadList->current();
threadList->remove();
blockedThreads->push_back(current);
}
void Unblock(Thread* thread)
{
auto it = std::find(blockedThreads->begin(), blockedThreads->end(), thread);
if (it == blockedThreads->end()) return; // probably error here
threadList->push(*it);
blockedThreads->erase(it);
}``` perfectnice! somehow IF is getting cleared
no cli is being executed
nor am i physically changing rflags
icy comet
ok that is fixed
i think it was getting messed up when interrupts happened and then never restored(now i save it in the thread context struct)
the scheduling looks a bit "uneven"
A and B are printed by two different processes
somber citrus
the scheduling looks a bit "uneven"
Haha this is that meme from the osdev wiki
Is it the scheduling that's uneven or just the logging? If a thread is pre tempted while holding a lock for logging that might do it.
icy comet
no lock whatsoever
icy comet
Haha this is that meme from the osdev wiki
i think i know the one youre talking about lol
something semaphore iirc
somber citrus
Yeah that sounds familiar haha
icy comet
today will be more scheduler things
first of all, locking primitives
then i will probably work on a proper event system
most likely i will just have a generic Event class that stores a list of its waiting threads, then all the other events can derive from that
then you can emit each event and that unblocks all of the waiting threads
glad drum
sounds nice!
icy comet
ok i got none of that done because i had training and things to do beforehand
but thats fine
icy comet
now that i think about it, it would probably make more sense to have semaphores and suchlike use the event system
since they behave very similarly
the semaphore can create a new event and store all of the waiting threads in it as normal, but it could override the onEmit function to decrement the counter and only unblock one process
the event queue will obviously need some kind of locking as well
i guess i can just use a spinlock there? its unlikely that something will access it for a huge amount of time
hmm, another issue
events will need to be allocated when they go onto the event list
but if they are created in the scheduler, that creates an indirect dependency on the allocator
which isnt ideal
wicked rain
now that i think about it, it would probably make more sense to have semaphores ...
read mintia
it does this essentially
icy comet
thanks, ill have a look at this in a bit
icy comet
apart from the strange language this makes sense
i still need to think about how to resolve this dependency issue
i could defer the adding of the event to a kernel worker thread maybe?
though that would need an allocation of the DPC object as well
so the same issue is still there
wicked rain
what are you worried about having to allocate exactly?
icy comet
the event objects themselves
lets say we create some event
WaitForProcessEvent
that would need to be allocated
though i guess waitforprocess has nothing to do with the scheduler actually in terms of creating the event object
wicked rain
who says you have to allocate the event object then
icy comet
how else would it come into existence
wicked rain
embed it in the process or something
icy comet
that is actually a good point
i can do the same for the semaphores as well
just throw them into the semaphore object
wicked rain
thats basically what i was doing
i embedded a dispatcher object header in the semaphore
also "strange language"
i take offense to that.
icy comet
:D
wicked rain
basically
theyre also described in ke.doc in the NT workbook
which is written by dave cutler
icy comet
i unfortunately dont own it
icy comet
ok we have mutexes
void Mutex::lock()
{
bool expected = false;
if (!mLock.compare_exchange(expected, true))
{
mUnlockEvent.getWaitQueue().push_back(sched::Current());
sched::Block();
}
}
void Mutex::unlock()
{
if (!mUnlockEvent.getWaitQueue().head()) // no threads waiting on the lock
{
mLock.store(false);
}
else
{
mUnlockEvent.onEmit();
}
}``` i hope this is correctThread* head = mWaitQueue.head();
mWaitQueue.pop_front();
sched::Unblock(head);``` this is `onEmit`it seems to be working so im just gonna assume that meanas its fine until it comes back to bite me
ok of course this wont work
the wait queue operations are not atomic
and therefore preemption can occur inside of them
which can mess things up a bit
harsh ridge
and therefore preemption can occur inside of them
suppressing preemption is the typical solution
icy comet
yeah i guess that works
harsh ridge
make sure also that wake queues don't wake a thread before it's actually asleep
icy comet
pushcli at the start and popcli at the end
harsh ridge
wdym?
supposing someone puts themselves on a wait queue, it must be impossible for them to be woken until after context switch to a different thraed
icy comet
the Unblock just puts the thread to the end of the scheduler's thread queue
harsh ridge
otherwise the same thread might run on two cores at once
icy comet
ok somehow we are jumping to a random address
the callstack seems to be pretty garbled as well
icy comet
now there is a GPF on an iretq
ok yeah
its trying to load the 6568th GDT entry
somehow it works as intended in gdb
icy comet
i probably wont be working on this as much soon, i have a lot to study for
school year just started, and i have SATs and the TMUA to do
amongst all the other college essays and such
icy comet
i have no idea at all how, but the iret frame is getting completely trashed
icy comet
everything was fixed!
i wasnt disabling preemption around context switches
so weird stuff was happening
icy comet
im not sure where i actually want to store the process structures to prevent them just getting leaked
though i guess if the thread struct stores a pointer to its parent process then it wouldnt be leaked
icy comet
got various annoying boring bugs to fix so i will try to get those done today
icy comet
hmm, somehow both threads are waiting for the mutex to unlock
which obviously doesnt really work since theres nothing to unlock it if they both wait
icy comet
rewriting this yet again
just finished reimplementing the driver loader, it loads PIE executables to some random kernel address then gets their header (which is given a special segment in the elf file) and puts them in a special filesystem in the vfs which then calls back into the header to find the read, write functions etc
__attribute__((section(".driver_header"))) volatile DriverHeader header = {
.read = read,
.write = write,
.ioctl = ioctl,
.name = "fb"
};``` so a driver just needs to provide these and an entry pointand the rest is handled by the kernel
icy comet
working on an event bus now
the general idea is pretty simple, there are separate structs for each event type, eg file poll events etc, which can be waited upon and readied
events get allocated and are put in a big event bus object
which is scanned when things like writes are done
icy comet
added a couple syscalls
namely open, close, read, write, pipe, stat, fstat, exit, waitpid, getpid
and a primitive poll
and seek
icy comet
think ive got enough syscalls to start writing a couple userspace programs and seeing where it dies
although at some point i need to make the scheduler be not cooperative
since putting in calls to sched_yield() everywhere is a bit annoying
icy comet
set up most of the stuff needed to write the dynamic linker which should get done tomorrow
it finds the program interpreter from the interp segment then loads that alongside the actual executable, then jumps immediately to the program interpreter
need to add the auxvals as well though
and also load the PT_PHDR segment into the process' memory
but thats for later
somber citrus
cool, nice to see you back!
icy comet
necessary auxvals are done
currently the program interpreter just jumps to the exec but at least it works
now for the hard part
icy comet
super simple dynamic linker is functional
doesnt have any dlopen and such yet but it will resolve symbols at runtime which is good enough for now
it also doesnt use shared memory for libraries (since i dont have shared memory) but that will come later
icy comet
fixed a few bugs in the dynamic linker
turns out i was using the index into the rela section as an index into the symbol table
which is not how it works
but that is fixed now
icy comet
scheduler is preemptive now
fixed far too many stupid mistakes in the process but alas
think im gonna work on some userspace stuff now
icy comet
most basic ever shell is done
just takes your input, searches for it in /bin/ and runs the program if it exists
but nice to see something real instead of just hello world printed in more complicated ways
icy comet
somehow it only lets you run one program
if you try to run a second one it fucks up the stack of the calling process
icy comet
fixed a bunch of bugs, mostly dynamic linker stuff
#include <stdio.h>
int main()
{
puts("Hello, world!");
}
``` `hello` is looking much better nowjust finished file-backed mmap
read-only for now except for device files
but once munmap is added then it will write to the file on close
icy comet
finished a kernel worker thread that cleans up processes that have exited
before they just sat there eating up memory so its nice to have them actually freed properly
icy comet
working on reimplementing my old implementation of signals
shouldnt be too much work since all the design is done
just need to convert it to C
then i also want to make waitpid communicate a bit better with the calling process so it can detect signals and such
so then the shell can print "Segmentation fault" and such
icy comet
as expected signals werent too hard