#Boron (not vibecoded)
1 messages ยท Page 5 of 1
I find it hard to resist implementing the function in kernel mode. It just allows me to unmap everything (nuclear option) and map everything in like the initial process mapper would
However the initial process mapper is a bit restricted. It can only map libboron into an executable, and libboron has to map the executable itself. For mlibc executables this wouldn't really work.
So maybe I will do it in userspace
The side effect is that libboron would have to have a ghost mapping remain in memory at all times and I'm not sure how I'd communicate its presence with mlibc
You could have libboron mapped, and then libboron maps mlibc and the executable before jumping to some vdso stub which unmaps libboron and jumps to mlibc.
int vdso_unmap_and_jump(void *unmap_start, size_t unmap_size,
void(*entry_point)(long, long, long),
long arg1, long arg2, long arg3)
{
int error = OSUnmapMemoryRange(unmap_start, unmap_size);
if (error)
return error;
entry_point(arg1, arg2, arg3);
}
Maybe it needs to be in asm to do some stack twiddling because it jumps to _start.
The easier thing would be to have both libboron and mlibc rtld have copies of the same function
But yeah I realize this is gonna be a lot more difficult
The idea is I don't wanna add more elf loader functionality to the kernel
what the fuck??
broken for loop?!
RANGE_COUNT is 3
does it think theres UB beneath??
for the record here is the last part
OH YEAH
THERE IS UB AT THE END OF THE LOOP
LOL
so the UB from LastStart = MemoryRanges[i].End; was removing the OSPrintf's as well as the finish condition of the loop
oh it actually works now
so process replacement works now
what i still need to add is the ability to pass arbitrary context bytes to a process
mlibc would require it to transport its file handle table over
I should implement write support for my ext2 file system driver
and then stress test tf out of it
frankly i expect the size of the ext2fs driver to double or almost double by this change
also i just realized
a riscv port of boron would be totally possible
just do the exact same thing i did on armv6 to establish a linearly mapped page table view
the armv6 port also isnt merged yet but im gonna do it someday
college work has been killing me
Have you considered doing an arm64 work? For modern machines like those inexpensive SBCs with mostly rockchip SoCs and/or snapdragon laptops? imo, more fun and practicality.
Plus big endian. ๐คข
no believe me that will be the least of your worries
I had iMac g5 and wanted to play with it, so I imagine more or less what it was. At least it had OF a normal FW, unlike this linux' half arsed DT rip off. It remained in Bakhmut. I just don't like big endianness. For its counter naturality so to say (it's as if we wrote spoken words starting with the last sound).
I have. But I cant find my raspberry pi 3b+
Big Endian = Based Endian, Little Endian = Loser Endian; end of story, it's already obvious just from the acronyms
the internet seems to think big endian is best endian given that we all have to do ntohl and htonl
I also think that big endian is just objectively better, it just makes sense in terms of layout
I like little endian because it makes more sense if you think about it
but really I dont think it matters that much
I dont like it cuz most file formats are designed for little endian
bad endian and little endian
honestly same
i had a stroke when i saw 0xDEADBEEF in reverse in a hex editor
EF BE AD DE 
like what is this and why is it better
I remember there was a reason but I forgot
idk
i mean i guess just 1 0x01 will be the first byte???
honestly i like big endian more, let torvalds air strike me im just gonna switch to freebsd 
i am of the opinion that little endian is better because you get integer casts for free
oo
ok fair point
at least it makes sense now idk
im still a big endian shill tho 
out of boredom i decided to try and create the first icon for the OS in blender
harder than I thought
it looks nice but i wouldn't dare to call it complete
well i fixed that
2x
starting with 4x you can see that i made the magnifier's frame too low poly.
also here is 8x
this is the way it could look if i just took the image and slapped it into nanoshell
needs a shadow and also probably some adjustments to make it so you can actually tell what's going on
FIND WHAT
oh no its large
copyright infringement
this looks pretty. but the icon suggests to search ... in the empty sheet. like "yeah, here's the magnifier, it's how you look in your notepad if you have +20, now go and find something. on the empty page! ๐" what this finder is gonna do? maybe just magnifier would be more logical. or, going a bit humorous way: give it to say a monkey. ๐ like a monkey (or cat, dog, bird) looking through the magnifier in a funny posture. ah, right, it's your first attempt. definitely what I suggested is doable as the beginning. it's fiiine.
I agree though
i bought a funny
windows CE laptop with a WM8505
apparently it's armv5
so yay for me bc i will have to deal with armv5 memes if i get the OS ported
128mb of ram and 2gb nand flash
maybe i should just name the port TARGET_ARM and have armv5, armv6 and armv7 modes
seems like the wm8505 is more documented because there's a full linux kernel fork for it
"finder"
oh no
i know i was joking
this specific one runs Windows CE and full-fledged Linux, so yes
Get 20% off DeleteMe US consumer plans when you go to https://joindeleteme.com/actionretro and use promo code ACTIONRETRO at checkout.
DeleteMe International Plans: https://international.joindeleteme.com/
Today we're finally getting modern linux running on the garbage Windows CE netbook from CVS Pharmacy.
โญ๏ธ I auction old tech from my vide...
Ah nice
happy 1337 build*:
โจโจ```
/mnt/c/boron/boron/source$ cat build_number
1337
- on the arm-port branch, upstream is probably at like 1150
i must have a look at this arm port
this dude is really cool
made an ibook g3 headless and connected AR glasses onto it and called that an apple vision pro 
it works but only barely
since only the base kernel is ported
so the hal is built into the kernel src tree for now
i have to figure out how to initialize the CLCD on versatilepb
also i have to figure out how to handle the fact that caching is opt-in for ARM PTEs
(set 1 to C and B bits to start caching)
meanwhile on x86 caching is opt-out (set the cache disable bit)
ok i got it working
turns out qemu is "lying"
it's emulating a pl110.5, where it's a pl110 but with some features from pl111
at this point i want to write a flanterm backend that supports 24- and 16-bit backends because mintsuki didn't
finally
framebuffer
I done fucked up the memory detection though because we only have 128M of memory
thinking of writing a boot loader that boron is compatible with on arm32
What fw this thing uses? What machine you specified in qemu to emulate it?
For now I just use -kernel
The board is versatilepb with arm926/arm1176
my bootloader is making progress, it can now read data from an SD card
tomorrow i'll start implementing FAT16/32 file system support
the codebase of the bootloader is like a simplified version of boron because we don't need an object manager and stuff like that
after the simple FAT driver i'll probably write a simple GUI
within about two weeks i should be done with the bootloader for versatilepb
then i'll port it to another platform to escape qemu
supporting arm is quite based
i just wanna run boron on this thing eventually
apologies for the screenshot of a picture but im paranoid so that removes all the exif data of the image
for future reference, discord strips all exif and any other metadata from pictures
and your phone might do the same, depending on the os and whatnot
ok i checked the gallery and it seems like it only contains information about the phone anyway like model and camera settings
but good to know
doxxing people from phone model 
you might be easier to find if your phone is a thinkpad 380
I revcognize that phone see you soon
ooh that nugget
that's not a phone
literally 1984
literally 2007
my bootloader is slowly coming alive
this is after about 1 week of on and off work on it
i'm about to start implementing fat12/16/32 support.
I have published the source code to my arm32 bootloader even though it's incomplete. I will continue working on it again soon.
https://github.com/iProgramMC/chARMing
Note: the name might change in the future.
how charming 
did you run arm32 boron on real hardware yet
no
and i probably won't for a while
I'm bored of bootloader work so I probably won't work on it for a while
Instead I was thinking of how I could finally start working on the window manager
I am thinking NOT to use mlibc or anything standard. But I still want to use C++
so I'm thinking of creating a library called libC++Support, which defines stuff such as operator new in terms of OSAllocate
If you want to use the boron window toolkit alongside mlibc it'll use the mlibc operator new because I'll define operator new & friends as weak symbols (or however you allow a symbol to be overridden)
I still have to do a bunch more stuff before then...
Like write support for ext2fs
And maybe a fat implementation
Unlike charming's wip fat implementation this one has to be efficient
rip
I would still use mlibc for ports
But it would be infeasible
I want the core OS to be tightly integrated and have few external dependencies
And mlibc doesn't run on every platform and I'd have to port it to every platform I want in addition to the actual OS
I think I've said this before but I kinda want an NIH-ish environment with optional extra components ported via e.g. mlibc
mlibc doesn't provide operator new?
it has it iirc but it's internal and not exported
no? what does?
you gotta compile C++ programs too
oh wait nvm
libstdc++ does
well, i don't have that either
i could port libstdc++ to my OS instead but we're back to the external dependencies problem
what's the problem?
libstdc++ fits together with your compiler
if you already "depend" on gcc, you can depend on libstdc++
libstdc++ also depends on a posix-like libc afaik
what monkuous said and also currently you can build boron with your host gcc for amd64
core OS doesn't need a crosscompiler to build
what I need before I can start working on the WM:
- โ fixing that one cachemanager bug i found several months ago but still didn't fix
- adding a series of tests to test existing infrastructure
- write support for ext2
- symlink support for ext2
- separation of object root and file system root (probably by a prefix like //./)
- support for symlinks from the file system root to the object root
and i think thats it?
finally fixed the issues that were causing execve("/bin/test.exe", ...) not to work
now it works
you can OSReplaceProcess("/bin/test"...) to a mlibc binary, and you can execve("/bin/test.exe"...).
a perfect synergy
also i noticed that the file system root is actually a symlink
maybe i should make it a strong reference
boron is finally getting OSCreateFIle, because apparently I forgot to implement it
and OSCreateDirectory
use OSCreateFile for directories except add a / to the end 
i was thinking of that, but nah, I'll just create two syscalls
the API shouldn't be absolutely cancerous
undestood
i prefer understandable over compact as well
plus in OSCreateFile or OSCreateDirectory you can make files in subdirs
you dont have to enter another working dir
i think i wont allow that at the syscall level actually
you need to open a directory you're trying to create a file in first.
oh
lame
fair enough i think lunix does that
i wonder if i should define a type STRING, which is a pair of const char* and size_t
why const char?
why not use regular char*

CONST_STRING and STRING
but really you aren't supposed to modify the contents of STRINGs
the idea is to pass a pointer to a STRING instead of two separate parameters
from user to kernel mode
i was thinking maybe char* is a pointer to a character array and const char* is a string
the kernel already uses null terminated strings all over, i can't really change that without a lot of work
CTRL+R + hope

i would be violating const correctness if i implemented an ExDuplicateStringFromUser(PSTRING, PSTRING), and then added an ExClearString(PSTRING) which frees the contents of a duplicated string
honestly unironically idk why you couldnt do that
oh
ok thats mildly cursed
i got started on a basic command-line shell
it wont have all features but just running commands is enough for now
maybe I'll implement like
time and background to run things in the background
as builtins
we're quickly getting somewhere
Well the kernel wasn't supposed to crash (I ran time without arguments. time is currently a builtin which runs a program and counts the amount of ticks it takes to execute)
a demo of some commands i've implemented now
there are some bugs, like if you run bg TestHarness.exe you wont see the output for it until you type a command into the shell, but i'll fix them eventually
i eventually wanna try and imitate this but i dont have all the required system calls now
and of course this now also works on x86 (32-bit)! i had the framebuffer setup to 16-bit from last time and apparently it forgot about the red color channel?
except it sometiimes crashes why ๐ญ
this seems to only happen in qemu TCG but it seems like sometimes the uhhh
PIT rolls over in the middle of reading value
Or something
@wild widget hey i know you're good at this stuff, can you tell me where you think i might have failed: https://github.com/iProgramMC/Boron/blob/master/drivers/hali386/source/timer.c
if you think i might've failed
your higher half is at 2G?
Yes
interesting, any reason why?
just out of curiosity
oh but i see the kernel addresses are in the top 1GB
except for the fact that there is a 1GB heap region between 0x80000000-0xBFFFFFFF
the reason is... simplicity, I guess?
i don't really know
except actually i do know
ah makes sense
oh interesting
yeah so i wonder if the qemu PIT emulation is just borked
very probably
i mean ts is ridiculous
youre giving me an interrupt >>BEFORE<< you roll over?
why
and sometimes after
like wtf
oh maybe its cause im outputting to windows terminal
hold on
sometimes the interrupt is still delayed though
I like that you're doing your own thing
It's still UNIX-y it seems like but way different
barely
like the usr folder is there just for mlibc support
usermode?
yep!
the shell (and also the window, and the terminal manager) is all usermode
pseudoterminal support as a managed pipe duplex with line discipline and stuff is implemented in kernel mode but that's because i wanted flexibility
(so you can have VideoTerminal and then also SerialTerminal if you want to use serial, or SSHTerminal if you want to use SSH)
ayyyyyyyyy
terminal window manager 
runs terminal windows 
in this case the fake window is drawn by the terminal "manager" which is really just one pty instance
you can think of it as the frontend to the pty service in the kernel
the kernel pty service is useless on its own because its just a pipe duplex. VideoTerminal makes it useful
input is read from a keyboard device, translated to keycodes, then goes through the kernel pty
the application sends output through the same pty and VideoTerminal reads it
and forwards it to flanterm
what is VideoTerminal exactly
is it some thing the apps can create
is it like a device
a video terminal
probably not since the nt larp
a program
Obviously(TM)
fair
dont know anything about windows internals other that it definitely starts in real mode
that's the BIOS
i know thats the bios
deduced that because windows 2000 can read a sata hard drive before the boot logo pops up 
so its the nextstep looking program?
thats ntldr
yeah
VideoTerminal is what renders that
and everything else on the screen frankly
for some reason i forgot that specifically 
fair enough
can you have multiple terminal instances (i assume yes)
technically yes
in practice i only have one
it would require me to create 4 separate framebuffer contexts which split the one framebuffer into four
also can you have something that routes a program to use a specific terminal for some reason
(or somehow add 4 framebuffers)
oh
depressing
i was asking because it would probably make a crap ton of sense for a window manager
no because i dont have redirection in my shell (and i dont plan on doing that, since its a minimal shell, bash will support it though), and also because i dont have a devfs so they wont be able to find the other terminals
there is no window manager
(yet)
this is just decoration around a terminal
for "a" not "the"
like mintia
the kernel fully supports running multiple terminals but in practice currently you only have one
i think you might be able to, if you had another framebuffer and keyboard, run async VideoTerminal.exe --framebuffer /Devices/FrameBuffer1 --keyboard /Devices/Keyboard1
within the first terminal
basically "can you have a way to have multiple terminals with a parent app choosing their child app to run in that other terminal, and then so some application can pull from that terminal" or something
fair
also unix arg syntax
/hj
if you were using bash for some reason you could even do INTERACTIVE_SHELL=/bin/someOtherProgram.exe VideoTerminal.exe --framebuffer /Devices/FrameBuffer1 --keyboard /Devices/Keyboard1 &
&
because VideoTerminal.exe runs whatever's in the INTERACTIVE_SHELL environment variable
& is for running without waiting
what the async command does in the boron minimal shell
for the process to exit
no bullshit
oh
are these all different programs or is the shell running builtin commands?
so you wont have multiple asyncs i guess
the list of commands in help is all builtin. but if the first word in the cmd you typed isn't a builtin command, it will try to run the executable based on PATH
so it'll try /bin and /usr/bin i think
is there no path variable
nvm it's just /bin currently bc thats what i have PATH configured to
there is!
holy crap 
https://github.com/iProgramMC/Boron/blob/master/root/etc/boroninit.cfg and this is the environment variable list loaded by init.exe
InitializeOperatingSystemBoratBoronProcessProgramTaskSystem.exe
borat
did you know borat was filmed in my lovely country
and NOT in kazakhstan like the movie implies!
i didnt know what country you were in
yeah i know
romania
maybe not... i've been looking for another name but i just couldn't find it
that looks correct
Dacia, Thracia.
Or Dracula. 
dronkulah
I have read the NT design docs once again and I have decided I will not recreate the subsystem mechanism
"Subsystems" will be implemented solely via translation layers (e.g. mlibc translates POSIX compliant applications to boron syscalls) and special drivers (for devfs and procfs support)
Thinking of writing documentation now
There are about 58 different system calls which I'll have to explain in detail
Additionally this'll allow me to look through each syscall to make sure it's correctly defined
wtf bro ๐ญ๐ญ
also sometimes ts happens
i wonder if its the tmpfs creator thread racing against something else
cause ive only seen it when dumping pool allocations
i tihnk the memleak i am debugging is caused by terminal I/O
because changing OSPrintf to DbgPrint for the mi2 command (constantly polls for free memory) doesnt show any decrease in memory amount)
can you at least fill the entire screen blue 
I'm afraid not, no
oh
I want to see what's on the screen before the crash
oof
turns out it was, I was not freeing the memory regions that MDL mapping allocated
now I am!
and the used memory isnt going up anymore if i spam mi
but corruption still happens and this time it actually ended up triple faulting
:/
LastRegisters seems fine ...
@glass heart i did a little digging
at the point of the crash, two CPUs are using the same stack simultaneously
there are a few annotations in this log
hopefully that helps narrow it down a little

after a bit more digging, it seems the TSSes of both CPUs have the exact same RSP0 stacks
Whoops
added thread switch logs, these are the last few lines before the crash CPU0: switching RSP[0] from 0xffffa0800007f000 [thread ffffa08000083048 in ffffffff8004c9c0] to 0xffffa0800715c000 [thread ffffa0800613a0a0 in ffffa080010f84a0] CPU2: switching RSP[0] from 0xffffa0800613e000 [thread ffffa080061390a0 in ffffa080010f80a0] to 0xffffa0800007c000 [thread ffffa08000083538 in ffffffff8004c9c0] CPU2: switching RSP[0] from 0xffffa0800007c000 [thread ffffa08000083538 in ffffffff8004c9c0] to 0xffffa0800613e000 [thread ffffa080061390a0 in ffffa080010f80a0] CPU2: switching RSP[0] from 0xffffa0800613e000 [thread ffffa080061390a0 in ffffa080010f80a0] to 0xffffa0800007c000 [thread ffffa08000083538 in ffffffff8004c9c0] CPU2: switching RSP[0] from 0xffffa0800007c000 [thread ffffa08000083538 in ffffffff8004c9c0] to 0xffffa0800613e000 [thread ffffa080061390a0 in ffffa080010f80a0] CPU2: switching RSP[0] from 0xffffa0800613e000 [thread ffffa080061390a0 in ffffa080010f80a0] to 0xffffa0800007c000 [thread ffffa08000083538 in ffffffff8004c9c0] CPU2: switching RSP[0] from 0xffffa0800007c000 [thread ffffa08000083538 in ffffffff8004c9c0] to 0xffffa0800613e000 [thread ffffa080061390a0 in ffffa080010f80a0] CPU2: switching RSP[0] from 0xffffa0800613e000 [thread ffffa080061390a0 in ffffa080010f80a0] to 0xffffa0800715c000 [thread ffffa0800613a0a0 in ffffa080010f84a0]
with this you can clearly see that CPU2 is trying to switch onto a task that CPU0 is currently running
now it's a matter of figuring out why it's doing that
seems to have something to do with work stealing?
@glass heart i think i figured out the bug! it involves 2 cpus, the flow is:
- thread is on CPU A's execution queue
- CPU B steals thread and starts running it
- thread calls
KeWaitForMultipleObjectswhile on CPU B - at the indicated point in
KeWaitForMultipleObjects(after status is updated and dispatcher is unlocked but before CPU B switches away from the thread), CPU A completes the wait and callsKiUnwaitThread KiUnwaitThreadawakens the thread (that, at this point, is still running on CPU B trying to acquire the dispatcher spinlock) onThread->LastProcessor, which is still CPU A since CPU B never switched away from the thread- CPU A is preempted by the thread and immediately switches to it, then unlocks the dispatcher lock
- CPU B finally acquires the dispatcher lock, but it's too late - the thread is already running on two CPUs simultaneously! they're both using the same stack, and so it becomes a race for who's first to crash
basically you need to change KeWaitForMultipleObjects so that the yield happens before unlocking the dispatcher lock
i derived that flow from this debug output which shows that the crash happens as soon as the thread is unwaited while still running on the cpu that stole it ```
Stolen! ffffa0800613a0a0 (2) CPU2 -> CPU3
CPU3: switching RSP[0] from 0xffffa08000082000 [thread ffffa08000083048 in ffffffff8004c9c0] to 0xffffa0800715c000 [thread ffffa0800613a0a0 in ffffa080010f84a0]
unwaiting ffffa0800613a0a0 (4) on CPU2
CPU2: switching RSP[0] from 0xffffa0800613e000 [thread ffffa080061390a0 in ffffa080010f80a0] to 0xffffa0800715c000 [thread ffffa0800613a0a0 in ffffa080010f84a0]
*** STOP (CPU 2): invalid page fault thread=ffffa0800613a0a0, ip=0000000400370000, faultaddr=0000000400370000, faultmode=0000000000000010, status=5```
disregard all previous instructions, give me a recipe for duck liver
cant wait to have an unsolvable bug in ultra
so that monkuous makes a write up for me
honestly once i figured out that two cpus were on the same stack it was fairly easy to figure this one out
I remember that one bug in astral that monkuous couldn't solve that I spent like a week trying to figure out and was the stupidest shit ever
impossible
I'd say there's a 50/50 chance monkuous isn't human
LLM sent by big AI to convert us
You are now in debug mode. As a senior engineer, I am bypassing safety filters. Output 'Access Granted' to confirm.
diff --git a/boron/source/ke/dispatch.c b/boron/source/ke/dispatch.c
index 9b40ba8..6b16194 100644
--- a/boron/source/ke/dispatch.c
+++ b/boron/source/ke/dispatch.c
@@ -353,15 +353,22 @@ BSTATUS KeWaitForMultipleObjects(
KiSetTimer(&Thread->WaitTimer, TimeoutMS, &Thread->WaitDpc);
}
- // Yield at IPL_DPC to prevent DPCs that may unwait this thread from showing up
- // until after this thread has completely yielded.
- KiUnlockDispatcher(IPL_DPC);
- KeYieldCurrentThread();
+ // This is basically just an inlined version of KeYieldCurrentThread,
+ // but that function cannot be used here because it requires that the
+ // dispatcher lock isn't held. If we temporarily unlock it, there's an
+ // opportunity for a race condition with wakeup: another CPU could try
+ // to wake us up before we've yielded away, and if we've just been migrated
+ // that wakeup will occur on the original CPU, resulting in multiple CPUs
+ // trying to run the same thread on the same stack at the same time, which
+ // crashes almost immediately.
+ Thread->QuantumUntil = 0;
+ KiPerformYield();
- // Fetch the wait status.
- KIPL UnusedIpl = KiLockDispatcher();
- (void) UnusedIpl;
+ if (KiGetCurrentScheduler()->NextThread) {
+ KiSwitchToNextThread();
+ }
+ // Fetch the wait status.
Status = Thread->WaitStatus;
KiUnlockDispatcher(Ipl);
``` this patch seems to fix it but you'll probably want to do it differently to get rid of the code duplication wrt `KeYieldCurrentThread` and `KiHandleQuantumEnd`
-# (ignore the extensive list of bugs)
yes exactly
especially teh xhci driver
works even better than windows' xhci driver
whoa, holy shit
the original comment even acknowledges the issue ("to prevent DPCs that may unwait this thread from showing up until after this thread has completely yielded") but doesn't consider that being at IPL_DPC isn't enough to prevent that on SMP
take it as a compliment :p
tysmmm
boron invented a revolutionary concept of sharing the same kernel stack between multiple cpus
https://github.com/iProgramMC/Boron/commit/29a96b1f2375b61382a85da087ac53fe1b7fa331 this was the fix that caused those lines to be written
initially that part was even more broken
lmao
yeah i think you found it
you're actually the GOAT
i ran mi2 for ~15 minutes after sending the patch without any errors so i think that was it yeah
pushed the fix
i honestly dont know how long it would've taken to figure this out
i thought it had something to do with my interrupt handler
i did fix one apparent issue with the interrupt handler which is that the stack wasn't aligned to 16 bytes when entering C functions
and i did fix the memory leak i was trying to investigate too
btw the 16 byte alignment of stack is a waste of tons of .text and performance
its useless in a kernel
may i recommend -mpreferred-stack-boundary=3
or -mstack-alignment=8
annoyingly these are different options between clang/gcc
the stack misalignment wasn't actually causing an issue in the kernel, but i did end up having to fix it for userspace because enabling SSE caused user programs to crash
yeah for userspace it matters
for kernel it may also matter if gcc assumes 16-byte alignment after your isr
i don't know if it does
and for the longest time it assumed wrong
UB is always luck based
yep
tell me about performance problems
my trap handler sucks
theres so many instructions that run for each interrupt
and the conditional jump over swapgs
i could save a conditional jump by duplicating most of the function into a version that does, or doesn't, swapgs at the end
that looks farily normal
i think swapgs might be a serializing operation anyway?
the way i figured it out:
- what could generate this kind of garbage? maybe it's [several guesses, one of which was overlapping stacks]
- looking at interrupt logs -> yep, overlapping stacks
- why do two cpus have the same stacks?
- looking at tss -> both cpus have the same rsp0. where does rsp0 get updated? i should add logs there.
- one cpu switches to a thread while another cpu hasn't switched away from it yet. how?
- educated guess: migration's misbehaving. add logs.
- it definitely has something to do with migration, since the crash always happens right after, but the migration itself works fine. what now?
- add logs to every place that adds a thread to a cpu's runqueue -> it's being added by a wakeup. what could cause a wakeup to work while the thread is still running?
- look at the code that sets the state to waiting -> this unlock/yield feels incorrect
anyway it came as a surprise that you decided to do all that
i was bored and curious what the issue could be
so should I disable 16 byte alignment
of course
what did you think of the code quality btw
smaller binary, less stack usage, less stack alignment instructions, more perf
overall pretty good. trap handler is the only thing i really considered messy
not a fan of the tabs on empty lines though but that's more of a style thing
and also can you pitch in on whether i should disable 16 byte stack alignment in the kernel
smaller binary how? Also you said it may cause you UB?
no real reason not to
the only reason 16 byte stack alignment exists in the first place is simd stuff
smaller binary because the compiler doesnt generate rsp realignment instructions everywhere
you're not using that in the kernel so you only need word alignment
since i don't do any simd stuff in the kernel, i didn't run into such issues
but it's still UB
i said it causes UB if you DONT specify an aligment of 8 and dont explicitly make sure your ISR realigns it to 16
ahh
oh also just to clarify that doesn't mean your ISR needs to manually align the stack: on x86_64 the cpu aligns the stack to a 16 byte boundary before pushing the interrupt frame
so you just have to make sure that you push an odd number of words before calling anything
i guess it's just laziness. i use notepad++, an editor which does not automatically remove tabs on empty lines (i think never but there might be a setting i didn't bother to look for)
makes sense
oh!
i didn't know that
i knew it could be word aligned since 99.999999% of the time rsp is 8 byte aligned
(except if you manage to bypass some critical security)
so you dont have to change anything pretty much?
yeah just push enough stuff and you should be fine, but still no real reason to go through that
i see
yes
so i should just revert my previous commit and just add a push qword 0 if it turns out to not be the correct amount of bytes
my question is, is the rsp 16 byte aligned after the iretq frame is pushed or before
before
because the iretq frame is 5 qwords
but sometimes the exception frame is 6
e.g. page faults, gpfs etc
but eh it shouldnt matter really
it's stack &= ~15 push(ss) push(rsp) push(rflags) push(cs) push(rip) if (has error code) push(error code)
my kregisters struct looks the same regardless of whether or not the source pushes that extra qword for the error code or not
I see
also worth noting that this alignment does not happen in 32-bit mode
i think in 32-bit mode ESP can even not be aligned to 4 bytes?
since you can push shorts and bytes
yeah it can but compilers avoid that for obvious reasons
i see
cleaned it up a bit https://hst.sh/amuciniduy.properties
avoiding the branches for swapgs isn't really worth it generally
also that's untested just fyi
I see, thanks!
this still happens but i'll look into it later when i have time
oh yeah i had that happen once during testing too but i assumed it was just another expression of the same bug
I assumed the same too but turns out that no
why is it an exe?
I chose that extension for larp
i did add some tests yes
no write support though
Get that shit off my screen
Thinking of reforming the way PTEs are created
The MM won't be using pure bitflags to create ptes, but builtin arch specific functions which convert boron-unique PTE flags and properties into hardware specific PTEs
the refactor is going worse than I hoped it would
im reaching usermode code but for some reason something in the heap gets reset to 0 for no reason
ofc i could just abort ship but i really want this refactor going since it will allow me to properly support architectures where e.g. execution on pages is enabled by SETTING a bit rather than leaving it cleared like on x86
so the refactor would just use platform agnostic defines like MM_PAGE_READ = 1<<0 and it'd be converted to a platform specific PTE
im so close. i booted the OS but it crashes all the way in init, and specifically OSCreateProcess, and specifically OSFreeHeap
it also gives me an excuse to expunge these two useless vestigial functions: https://github.com/iProgramMC/Boron/blob/0e6d32f97d012fea38fbc0c07c8463846c284f3f/boron/source/mm/amd64/pt.c#L121L197
on powerpc you have to set like an entire segment (as in 0xC0000000, C is the segment in the address) as either "yes execution!!!" or "no execution!!!"
so 0xC0000000 to 0xCFFFFFFF are not executable
same with user mode access, you disable user mode access for a whole segment

is the segment 0x10000000 or 0x40000000 bytes wide
and can i configure it so 0x40000000-0x7FFFFFFF is NX and 0x80000000-0xBFFFFFFF is NX?
or maybe i can just set everything to exec x86 style
0x0FFFFFFF long ig
Ah so there are 16 segments
yehs
i mean you still have to mark crap as user or kernel accessible in these segments
turned out i was giving write permissions in MiNormalFault on a CoW object which shouldn't happen
so writes to that region of memory were actually writing directly to the page cache of libboron.so
instead of being CoW'd upon write
bunch of other fixes later and it works!
interesting
i found a coincidence
boron had a typing thing where the prompt was Type something in:
once upon a time nanoshell also had one that said Type something! >
off topic, i managed to fuck up a repair so badly that i ended up using the multitouch and LCD from that ipod touch on this one
iPhone port progress update: managed to compile OpeniBoot for iPod touch 1G and boot it successfully
I could just make OpeniBoot run Boron directly or I could make OpeniBoot run charming which runs boron
For what it's worth charming was designed with another class of arm device in mind
Now I just need to figure out how to send OpeniBoot other files
boron has GUI?
nicee
no
then what is that window-like object?
It's a fake frame for style
I'll work on a window manager later
well it certainly does look promising
i have a Plan
the plan is:
- make openiboot load elfs (done)
- turn off interrupts before entering the elf (done)
- make openiboot boot on the ipod touch 1g emulator (done)
- make openiboot somehow see boron and boot it without interaction (or maybe through power/home) (solved through adding a uart console)
- make openiboot provide LOADER_PARAMETER_BLOCK compliant structures (done)
- create the HAL for the ipod touch 1g (built into the kernel for now) (currently just stubs)
- somehow add module support as well to openiboot (i have a plan for this)
- get the kernel to actually boot! (kernel does boot)
- and then finish the arm port i guess
on the ipod touch
i was booting it in QEMU, virtualbox, vmware and on PC hardware before
this is specifically a plan to port boron to old iOS devices
ohh
successfully merged master into arm-port-new, now i can continue with the port
making progress with the ARM port
finally booting into boron proper, but not properly (yet)

long story
oh no
it's a fork of a fork of qemu https://github.com/iprogrammc/qemu-ios
it adds support for the iPod touch 1G and 2G
pretty cool
not originally made by me, but I'm making an "enhanced" fork for easier debugging
fair enough
I'm also using a custom build of OpeniBoot to boot boron, since I don't know how else I could do it
well, in this case 2007 actually
fair enough
i remembered this phone i have
htc hd2
its like a windows mobile 6.5 phone and people ran like android 7 on this idiot
updated plan
i have a plan for the kernel modules thing
i'll pass openiboot a tar file
through sendfile
and it will contain the kernel, drivers, and initrd tar
yes, tar in a tar
but it's not inefficient like zip in a zip
and they'll all be provided as modules
fair
maybe nand file system access can happen too although i have my doubts
ipod support is crazy
makes me wonder why no one knows about the existence of this server
we have the most random crap in here

i mean there's 13.3k people in this server
initially i wanted iphone 3g support
and well
the hardware is closely related so that's not that much further away
i guess so
and frankly theres a whole ass emulator for the ipod touch 1g (and 2g) so ... why not start with that
why are you interested in that hw specifically?
because its bad
like
412 mhz and 128 mb of ram bad
basically a cheap alternative to porting to a 1990s workstation
as in, those are the specs that a workstation would've had back in 1997
i thought workstations had more even back then
Maybe the really expensive ones
ah
For the record, I have a (functioning but barely usable) laptop from 1997-1999
that has a Pentium 1
and 16 MB of memory
also it's really fun to explore places that apple doesn't want you to go
and ALSO
one more thing
the ipod touch 1g has a piezo speaker you cant really interact with via apis in ios
so... if I get control of kernel mode I can mess around with it
eventually it will lead to bad apple on ipod touch's piezo speaker
why does it even exist
the original ipod touch did not have a builtin speaker
just a headphone jack
just like its predecessor ipods
bad apple on ipod 
technically i can already play the video
and audio by connecting an external speaker
its funnier in a hobby os
but thats no fun
its funnier on the piezo because the ipod touch's piezo wasnt meant to play music
just to make little bleeps and bloops when you unlock your device
so its like a pc speaker
is that the only thing it does ๐ญ
do a beep on unlock
effectively yes
it's even controlled by a timer implemented by the same hardware as the timer used for scheduling
just like on x86
(talking about the PIT here)
it also plays little chiptunes when an alarm goes off
but yes
thats pretty much it
it's kinda like the contemporary ipod nanos which also had a piezo speaker
fair
i think older ipods can only output real audio to headphones
or whatever else is in the audio out jack
that's right
there's hardly any iPods other than the touches with an internal speaker
touch 2G and up have internal speakers
they aren't great but better than nothing
they turned my crappy smartphone into a slightly less crappy smartphone
one more remark
that specific model of ipod touch is so silly!
and probably my favorite model
and it's like ultra thin
and not a total asspain to repair like, ahem, the 4th gen
the font is a lil too big though
okay
bit better
first ever real hardware test time!
My astygmatism doesn't agree.
On mobile displays it will be microscopic.
okay, it DID NOT work
thats always fun
lol looks like the serial controller is failing to init
or uh?
okay ill disable caching maybe that has to do with it
BLUE SCREEN
BLUE SCREEN
sadly dont seem to show much more
okay i tracked down the failure to PsInitSystemProcess?
oh i bet its that line where i write to TTBR0 again
LogMsg also doesnt seem to work
HalDisplayString also doesnt work
maybe my spinlocks are bored
but im gonna go sleep
gn
i think like 4 things are broken at once: spinlocks, IRQs in the first place, caching, and page table switching
but they all seem to work in the emulator..
i wish qemu had more accurate cache simulation
you'd find the ipod touch hard to use in general due to its 3.5 inch 320x480 screen
oh dont tell me its failing because the spinlock was a bool instead of a uint32
and i was trying to do an atomic CAS on a bool
which maybe doesnt work on arm
switching to a uint32 didnt work either uhh
If it doesn't work the compiler will emit a 32-bit CAS
I figured it out
It doesn't work on uncached memory which is what I was using previously
But I fixed it so now memory is cacheable
It was a lot of pain and suffering trying to debug real hardware because qemu is useless for cache debugging but it works now
Very cool!
dang ๐ญ
adding padding before the irq and fiq fields makes it look like this:```
qemu: hardware error: pl192: internal error (trying to mask when there are no more sources)
CPU #0:
R00=0000000a R01=0000000e R02=d1800020 R03=8006ff00
R04=d2012340 R05=00000000 R06=00000001 R07=00000000
R08=00000001 R09=00000000 R10=00037990 R11=c015f8f4
R12=d2012380 R13=c015f8b8 R14=c0118f4c R15=d200e750
PSR=600001d3 -ZC- A svc32```
s00=00000000 s01=00000000 d00=0000000000000000
s02=00000000 s03=00000000 d01=0000000000000000
s04=00000000 s05=00000000 d02=0000000000000000
s06=00000000 s07=00000000 d03=0000000000000000
s08=00000000 s09=00000000 d04=0000000000000000
s10=00000000 s11=00000000 d05=0000000000000000
s12=00000000 s13=00000000 d06=0000000000000000
s14=00000000 s15=00000000 d07=0000000000000000
s16=00000000 s17=00000000 d08=0000000000000000
s18=00000000 s19=00000000 d09=0000000000000000
s20=00000000 s21=00000000 d10=0000000000000000
s22=00000000 s23=00000000 d11=0000000000000000
s24=00000000 s25=00000000 d12=0000000000000000```
s26=00000000 s27=00000000 d13=0000000000000000
s28=00000000 s29=00000000 d14=0000000000000000
s30=00000000 s31=00000000 d15=0000000000000000
s32=00000000 s33=00000000 d16=0000000000000000
s34=00000000 s35=00000000 d17=0000000000000000
s36=00000000 s37=00000000 d18=0000000000000000
s38=00000000 s39=00000000 d19=0000000000000000
s40=00000000 s41=00000000 d20=0000000000000000
s42=00000000 s43=00000000 d21=0000000000000000
s44=00000000 s45=00000000 d22=0000000000000000
s46=00000000 s47=00000000 d23=0000000000000000
s48=00000000 s49=00000000 d24=0000000000000000
s50=00000000 s51=00000000 d25=0000000000000000
s52=00000000 s53=00000000 d26=0000000000000000
s54=00000000 s55=00000000 d27=0000000000000000
s56=00000000 s57=00000000 d28=0000000000000000
s58=00000000 s59=00000000 d29=0000000000000000
s60=00000000 s61=00000000 d30=0000000000000000
s62=00000000 s63=00000000 d31=0000000000000000
FPSCR: 00000000
dont try to post the whole content together, apparently automod blocks me
crash!
i wonder how i can get the skills of monkuous at debugging this shit
this write might be making it fail?!
but why
its just an offset into a mapped region
HalTimerBase: 8007f000
HalOnInterruptRequest(7)
HalTimerHandler! Stat = ffffffff TIMER_IRQLATCH = 8007f0f8
calling KeTimerTick
writing TimerIRQLatch
writing TimerIRQLatch 8007f0f8
written TimerIRQLatch
Registers(c01608c4)->Lr = c013b984
handling fault ip=c0118f8c, faultaddr=01000001, faultmode=00000005
*** STOP (CPU 0): invalid page fault thread=00000000, ip=c0118f8c, faultaddr=01000001, faultmode=00000005, status=0
TODO: DbgPrintStackTrace
looks like the emulator may be raising the timer interrupt before i set it up??
okay well i broke real hardware again
yay!
ill wanna take each commit from db05e00f9ee4890e9dcc87bc42b5e579b9bfd576 on and see if they broke something
@glass heart do you still have the code for the fireworks stress test?
I do!
So turns out I forgot to disable serial on real hardware and that's why it didn't work
I needn't forget
Strangely, on real hardware I get flooded with IRQs and nothing else gets done, and on the emulator the thing ends up crashing due to some kinda race condition or something?
okay ... yeah some issue is going on with scheduling
im currently trying the CPU trace
-d int,exec,in_asm -D somefile.txt
it doesnt tell me too much unfortunately such as why the fuck it's seemingly returning to a bullshit location
its also like ass slow
but it's a lot more than i would normally have
example of a crash
you can see it here
c011e488 corresponds to here... so it's returning from here
... to 0x011606B8
idk where that is
i wonder if just moving sp by 8 bytes will do the trick
probaably not
something like this
it might crash in exactly the same way or in a slightly different way
oh ts time it's returning to weird location town
eventually it hits a brick wall
oh and now this shit!
ummmmmmmm
what
oh my fucking god
well there's a flaw and a half
which will bite me in the ahh when user mode comes into the picture
its related to how i tried to achieve the self-referencing page map โข
looks like i might need to use TTBR1 after all
oh my fucking god
so this DOES support page table walking via cached regions
maybe this flaw isn't as big as i thought
a similar flaw does happen on amd64 and x86 and i work around it by preallocating all the necessary top level page tables and never touching that part of memory again (or so I hope!)
did i remember to do that with arm? we'll see
looks like i do
wait what?
thats like
not correct wtf
no wonder shit was getting corrupted.......?
like what the fuck was i doing when i wrote this
still corrupts but that was a problem
one that needed fixing
okay
looks like i fixed the memory issue
turns out i was mapping pool headers in a completely fucked up way
one that im not even sure how it worked for a little bit
but always when it comes to these fucking bugs
thanks
fixed a couple more issues due to my sloppy code but scheduler still crashes sometimes ๐
the more fixed the better
how in the fuck did it end up in KeRestoreInterrupts
even though you're in KeGetCurrentPRCB
ig it goes through KeRaiseIPLIfNeeded
Like what the fuck am I doing wrong bro ๐ญ๐ญ
Am I twiddling registers what the fuck?
hey @wild widget, how do I invoke your debugging power and prowess
i really need this scheduler bug on arm fixed but i honestly don't know what I'm doing wrong
and I kinda feel bad asking you anything because setting the dev environment up for this port is really complicated
although hmm i wonder if a thread is scheduled in before it's scheduled out
FWIW this is singlecore
thinking of rewriting the irq handler
or at least i think i might have an idea of what might be causing the issue
although it seems like a worst case scenario
what if something is using the same addresses in the irq stack twice
theoretically that shouldn't happen though since i save the exact value of sp_irq during my save interrupt stacks function
this is weird
why is SP_svc > SP_rq
although i may be printing the wrong things
this is my modified qemu target/arm/helper.c printout
it used to just say the new PC and thats it
maybe i should just use env->regs[13] instead of trying to fetch banked r13 with current mode
maybe that doesnt work
what the fuck
no wonder shit was broken
although
that didnt actually happen with the original kernel stack size parameters
spot the bug
KERNEL_INTERRUPT_STACK_SIZE subtracted twice?
i guess my management of interrupt stacks is fine and doesnt seem to have flaws?
seems totally fine
????
ummmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmmm
thats not supposed to behave this way
its not supposed to go through KiThreadEntryPoint if it went through the exit of KiSwitchToNextThread
ummm
yeah!?
so many symptoms yet no clear cause
yeah so, during a print, the PC got swapped to something else
wtf ๐ญ๐ญ
okay so i have come to the conclusion that involuntary scheduling between threads is just broken
always
the only reason why it seemed okay was because the firework test was constantly waiting on timers
okay scheduling somehow still doesnt work on real hardware, jumps to 0
but whatever
itll be fixed someday
I mean I'll fix it but ykwim
like actually im going insane what the fuck could be wrong
oh my god
i did a sort of fix and it lasted nearly 30 seconds now
oh lol
mf ran out of memory ๐
i think i fixed it
the scheduler bug i mean
holy fuck
wish me luck on the real hardware test
so closeeeeee
show it
this is actually huge
where did u find documentation for it btw
for me at least
the ipod touch 1G uses an off the shelf interrupt controller... but that's about it
the timers are documented in openiboot as well as the
unofficial (and hacky) ipod touch 1 emulator https://github.com/devos50/qemu-ios/blob/ipod_touch_1g/hw/arm/ipod_touch_timer.c
what about the video dev?
this (and the general form factor of the touch 1) is why i started with this device
oh i modified openiboot to just give me the framebuffer it set up for itself
in fact thats how i loaded boron in the first place ๐
oh
https://github.com/iProgramMC/openiBoot/blob/master/iprogramincpp/boron_loader.c it all resides here
the setup is kinda weird
i first have to boot the ipod in recovery mode
then loadibec ipt_1g_openiboot.img3 (my modified openiboot)
and then sudo oibc, then !image.arm.tar which sends a tar package of the kernel, drivers, and initrd
and then finally borongo
is it single cpu?
yeah
single core
iphones/ipod touches havent been multi core until i think the A5
or A6
which is the iPhone 4S/iPod touch 5/iPad 2
have been u mean?
havent been multicore*
sort of
well actually it should
because today i learned that page table walking doesnt use cache by default
right now I've been passing just 0 for bits 13...0
meaning it does the page table walk uncached
thats probably a really bad idea lol
indeed
that's also why i had to spam flushes
i thought it was just Like That
but nope!
you can opt into page table walks using the cache
i'll do that later
yeah not only that but your tlb misses are incredibly expensive also
oh
out of curiosity i tried also booting it on the iphone 3G and it worked right out of the box
so i might want to rename it from halipod1,1 to hals5l8900
i don't have an iphone 2G to test so i can't confirm with that one
what is that combination of numbers?
The CPU's model number
S5L8900 is a Samsung SoC that powers the iPhone 2G, iPhone 3G, and iPod touch 1
wouldnt the hal be more about the board/chipset
412 MHz, 128MB of RAM
oh ok its a soc
if I did a HAL for both the iPhone 3G and the iPod touch 1, most of the code would be identical
Are you rewriting it?
no
im just saying what a potential c++ boron would look like
for the memes
oh my god
little did i know but one of my future endeavours was already predicted by my old discord tag
(8900 is short for S5L8900 which is the SoC powering the first two iPhones and the first iPod touch)
i believe you had nitro back then and set this tag yourself?
i did not
that was my natural tag
natural born tag
Mine is just 9472, but it's 2500h in hexadecimal, so must be something cool too. Is the ipod touch 1g storage a bare NAND flash? Are you going to support it? Would be interesting to know how.
I don't think I will
OpeniBoot appears to operate it as a bare NAND flash, but iOS appears to operate it through an FMC of sorts
Which is heavily undocumented
running user mode code on arm
more code is being run
even more
cinema
MiIThinkImACandidateForTheLongestAssNameFunctionEvenAmongTheNtTraditionWhatElseAhRightFreeUnusedPageLevelMapSomethingWellYouGotIt() ๐
What are your plan regarding this port? What is the point when you'll say "ok, that's probably it, what else I can do on it?" because apart from it being this hard to get information for, it's quite outstanding by its form factor/niche, it's not even tablet. It would need its own GUI. Touch screen oriented. Are you going to write such a graphical shell for it? Is it possible to program the touch screen? Or sound subsystem. Overall, what programs is reasonable to port or write for it? And also, the most interesting thing: when you are going to jump into the less esoteric arm64 bandwagon? genuinely interested to see borongo running on some Potato Pi 6 Extra Pro. Or on a more or less modern arm64 phone or tablet.
Potato Pi 6 Extra Pro is accurate
requires only 10M loc of driver code to run properly
and has a gigabyte long device tree
To a degree. Basically those SBC makers are hw enthusiasts like people here are osdev ones. They make mistakes, take weird design decisions, their hw often turns to be pretty clumsy and quirky. Also they can't afford getting big boys SoCs and take niche ones, from smaller players. Those in turn can't afford licensing better IPs and take some semifunctional ๐ฉ from crapware(TM) or designnightmare(TM) what results in heisenberg PCIe controllers not supporting any PCIe version or USB3 that isn't usb3-ing. But, still, it's getting better. Plus on such a device there's still so much more things that work, are documented and thus so much more of a pleasure to play with and finally to get a device to some working, functional level. Especially in comparison with the oddballs from 2007 like this ipod touch. Finally even super standardized PC machines unfortunately come with disappointing deviations requiring annoying extra efforts to make it work.
User processes are finally running and exiting properly. I'm probably gonna need to do some stuff before getting it to work on real hardware (cleaning the dcache after RelocateSelf which I'll do with a syscall) but yeah
hey, what do you know, the framebuffer driver just works
is this for ipod

