anyway tomorrow i will continue on this
#OBOS (not vibecoded)
honest citrus
sterile sparrow
mine
blazing python
i got only 39
near linden
welp
flint idol
ok we ball
retransmission is implemented
and it works over "a network"
(my home wifi)
flint idol
it seems like SACK processing is kind of broken
and also it seems like packet loss is not dealt with kindly
and that kinda just continues on forever
each time sending more data
stark sigil
holy crap lois
flint idol
Force pushing
they wanted squashed and rebased commits
dense vigil
ah
leaden carbon
flint idol
and that kinda just continues on forever
FFUCK WAIT I ONOW WHT THIS HAPPENS
ILL EXPLAIN IN A SECOND
indeed it is a bug with retransmission
iirc
I forgot to set hdr.seq to the right value
while retransmitting segments
flint idol
i fixed the tcp retransmission bug
the bug was two things:
- hdr.seq was being set to snd.nxt, which is not what a retransmission should do (it should set the sequence to the sequence of the retransmitted segment, duh)
- and snd.nxt was being updated, which it should not do (because the connection state should not change except for the congestion window, possibly)
no way
i can download a file on obos from obos
just there is a "tiny" bug
the kernel will kinda just start sending these weird keep alives
after the FINs are exchanged
which i think is a bug with retransmission
inland radish
the kernel will kinda just start sending these weird keep alives
why did you decide to put your TCP stack in the kernel instead of userspace
flint idol
why did you decide to put your TCP stack in the kernel instead of userspace
isnt that the norm? also because sockets
and by that last thing, I mean because it makes it easier to implement networking sockets (i.e., AF_INET sockets)
flint idol
and by that last thing, I mean because it makes it easier to implement networkin...
(but perhaps this is debatable)
i dont know of a single non-microkernel kernel that puts the network stack in userspace
although that would be cool
anyway, if anyone wants to test my obos webserver, that is up
try downloading http://obos-dev.ddns.net:8080/largefile
the sha256sum of that file is: 6b2a6a0b2cf8bd1f8d2a1ef4daea91623a14b3b10752b5a083c2fc9b14bbf332
on a good (internet) connection, i get around 1mbps speeds
knotty totem
flint idol
was able to download 105.9k
can u retry now
knotty totem
just timed out
now works
actually
I tried again again lol
still stops at 105.9k
flint idol
just timed out
[ LOG ] (thread 15, process 2) syscall Sys_RecvFrom returned 0x20 (OBOS_STATUS_TIMED_OUT)
same here 
how weird it stops for me too
it worked like
right before i committed the code
:aaaaa:
flint idol
still stops at 105.9k
okok now it should work (i changed absolutely no code at all)
knotty totem
nope
flint idol
oiajdeolkjasjdas\
knotty totem
105.9k instantly, then nothing
flint idol
seems like ur network is the problem ngl
knotty totem
😂
knotty totem
currently downloading at a limited rate of 1k to see where it stops now
20k so far, going strong
knotty totem
i do wonder how many hops from ur network to my vps
16
knotty totem
20k so far, going strong
stopped at 64002
flint idol
who crashed
knotty totem
you
flint idol
it's not quite a crash, interestingly
the http server kind of a bit hangs sometimes
no clue why
if u want just do sudo ss -K dst 71.19.148.125 to kill off that connection
as it's probably in FIN-WAIT-2 if u ctrl+c'ed
well ig i have to fix my shit a bit more 
knotty totem
as it's probably in FIN-WAIT-2 if u ctrl+c'ed
yeah no it's you still
flint idol
i stopped the server
knotty totem
ah
flint idol
so it makes quite a bit of sense if it isn't responding 
ive seen that having this printf:
printf("ACK_LEFT %d ACK_RIGHT %d SND.NXT %d SND.UNA %d\n", ack_left - con->state.snd.iss, ack_right - con->state.snd.iss, con->state.snd.nxt - con->state.snd.iss, con->state.snd.una - con->state.snd.iss);
increased reliability
for some reason
all it is is a debug log i had when rewriting the transmission code
ah wait
i think it's my selective ACK code
which doesn't really seem to work right
flint idol
ah
i disabled selective ACK could u try now (should work now?)
selective ACKs seem to mess up my retransmission code
knotty totem
stopped at 97656
flint idol
bryufhsdfhsdihds
flint idol
can u physically move closer to my vps
knotty totem
after doing that ss command, it's 105.9k again
flint idol
it's in oklahoma
flint idol
first let me see how far i get locally
knotty totem
wait
no
im stupid
I copied the wrong link and it was trying https instead
okay I tried again and... nothing.
the vps is openbsd and idk what the equivalent to ss is there
flint idol
ss just kills active sockets
you dont technically have to do it but i do that just to not leave any open connections on my host
flint idol
told u it was ur network
knotty totem
what the actual fuck
that's so weird, usually my vps is the one that has issues connecting to things
I can only get about 100k/s though
flint idol
how many hops from your vps to mine out of curiosity
knotty totem
9
flint idol
interesting
anyway my problem seems to be that:
- my congestion control code is bad
- and my my retransmission code is unreliable
flint idol
`sha256sum` preferably
but the sum doesnt really matter
i want to see if it partially downloaded everything properly
knotty totem
bde1c55f4e890cb359d6bae674313b43ddfb442af6ddfbc94e91ec24590cf170
7110656 bytes
I think
flint idol
bde1c55f4e890cb359d6bae674313b43ddfb442af6ddfbc94e91ec24590cf170
7110656 bytes
yay the file integrity is good:
bde1c55f4e890cb359d6bae674313b43ddfb442af6ddfbc94e91ec24590cf170 largefile.part
my tcp code still has a lot of work to be done on it
but it:
a) works on my machine
b) works on other machines (sometimes)
now, @dense vigil was able to do a full download of the file
hm
also is the favorite part of the tcp stack for a trans person a retransmission timer?
(get it cuz like retransmission timer, like it's a joke hahaha)
((ill stop))
anyway, the webserver running on obos may or may not work depending on where u are
(maybe my tcp stack is racist?)
the fastest i can get download speeds are 3 megabytes a second
and that is on the vps itself
knotty totem
what is the localhost speeds?
flint idol
^
knotty totem
no I mean on obos
flint idol
let me try
knotty totem
you have a loopback interface right?
flint idol
yes
it downloads 32k then stalls
(im not kidding)
((my LO interface is terrible))
knotty totem
💀
flint idol
ik it's a problem with my LO interface because
an OBOS can communicate with another OBOS just fine
with tcp
but
when an OBOS tries to talk to itself
it gets shy or smth
and kind of just fails
i think that is because i dont know how queues were supposed to work or something idefk
flint idol
i have added initialization of the utmpx entries to obos
for now, the BOOT_TIME entry is initialized
so uptime works
uncut narwhal
uh it's consistent at least
well thats something at least
flint idol
in case anyone was wondering this is how the kernel currently looks like:
DOOM1.WAD looks like this:
this is what libstdc++-v3.so looks like:
i personally think it looks the coolest, although i do not like c++
stark sigil
its like that atari game which used its own source code as a forcefield effect
dense vigil
in case anyone was wondering this is how the kernel currently looks like:
Why does your kernel have so many 0s
uncut narwhal
this is what libstdc++-v3.so looks like:
heck c++ but this is the coolest screen nonsense for sure
torpid wigeon
Why does your kernel have so many 0s
inb4 messed up bss
💀
@flint idol send me your kernel binary
the gray part before the void looks like dynamic stuff
and after it looks like symtab
flint idol
<@841762704766140486> send me your kernel binary
k
torpid wigeon
5mb 💀
oh wait yeah
that's debug info lol
wait no
@flint idol broi
the zeroes are all .data
flint idol
ye
torpid wigeon
💀
flint idol
flint idol
this is driver info for in-kernel drivers
torpid wigeon
can you give me a link to where this is handled?
flint idol
because my vnodes have references to drivers, if i want sockets, for example, i need to make a mock driver header
torpid wigeon
i think you messed up a size or alignment
flint idol
i think you messed up a size or alignment
here's an example
https://github.com/OBOS-dev/obos/blob/master/src/oboskrnl/vfs/pipe.c
flint idol
it's a 912 byte structure
with like 256 bytes reserved
also 64 bytes for the driver name field
given this is the fifo driver, there are also a bunch of undefined ftable entries
there are also 256 bytes worth of ACPI PnP IDs
that are unused
(aka zero)
and an unused pciId field
this is mainly because all these fields are zero
trim bay
ffffffff800f53e0 g O .data 0000000000080000 OBOS_ArchSyscallTable
flint idol
hehe
trim bay
why is your syscall table 512 KiB
flint idol
trim bay
well you have almost a full megabyte of zeroes with OBOS_SyscallTable and OBOS_ArchSyscallTable combined
flint idol
oops
torpid wigeon
lmfao
flint idol
i shrunk the table to 512 entries
torpid wigeon
why isn't this in the bss tho
flint idol
#define SYSCALL_BEGIN (0)
#define SYSCALL_END (0x200)```
(from)
```c
#define SYSCALL_BEGIN (0)
#define SYSCALL_END (0x10000)```this isn't even used you can just make the whole table a constant and only have space for the entries you actually need
trim bay
why isn't this in the bss tho
because the syscalls that do exist are statically registered in the table
torpid wigeon
...
trim bay
so there's some non-zero entries at the beginning in both tables
trim bay
yeah
flint idol
that's so dumb
ask me from 1.325 years ago
torpid wigeon
huge waste of memory
trim bay
obos will NOT fit on a floppy disk
torpid wigeon
i recommend a hex editor btw
very nice to analyze binaries
ghidra/IDA as well
010editor is paid but it has this templating thing
trim bay
i just did readelf -WS to see what section was consuming all that memory, then objdump -t | sort to see what symbols were there and looked for large ones
flint idol
stripped oboskrnl binary minus very large syscall table
torpid wigeon
mfw -4MB
trim bay
010editor is paid but it has this templating thing
imhex has this as well and is free
torpid wigeon
flint idol
this also saves 2M of memory
*1M
torpid wigeon
lol your data section is only 18kb now
flint idol
obos does NOT have a lot of data
flint idol
this is how the kernel looks like now (the zeros at the bottom are not from the binary)
anyway i gtg now
flint idol
i might make an optimization to my tlb shootdowns
to set and reset cr3 to clear the tlb for very large TLB shootdowns
blazing python
to set and reset cr3 to clear the tlb for very large TLB shootdowns
cant invplg be enough ?
flint idol
idk
for like really large shootdowns maybe not
ive read that somewhere on this server I think
blazing python
for like really large shootdowns maybe not
is tlb shootdowns used for syncing unmap and etc. in smp
flint idol
yes
flint idol
[uACPI][INFO]: successfully loaded 1 AML blob, 1705 ops in 0ms (avg 4635948/s)```w/ lazy irql
i was able to get 4.6M ops/s
except for the fact that lazy irql is completely broken this is pretty cool
flint idol
```
[uACPI][INFO]: successfully loaded 1 AML blob, 1705 ops in 0ms (avg 4635948/...
and this would put me behind menix in that metric
on the leaderboard
dense vigil
I wonder if you could get on the leaderboards with a kernel that’s only goal is to run uACPI fast
dense vigil
True
flint idol
hold on might snatch this
flint idol
first boot of obos with upstream mlibc!!
-# (absolutely nothing has changed)
oh wait
i forgot to actually
install the updated package
oh fuck
i didnt think that would happen
i need to fix my xbps package generation
flint idol
so like
ig someone who used to own my vps' ip
had a domain
cuz I searched obos-dev out of boredom
and found m.shh.sh which is a domain that has my vps ip
flint idol
I need to fix a bug
on line 393 of mixer.c
in obos-aud
and i also want to fix smth to do with audio quality
I want to change the master volume to be applied on the final buffer
not on anything intermediate
I have an idea for my allocator
a majority of the allocations in my kernel are calloc
so what if I have two freelists
one for zeroed blocks (so basically newly mapped blocks)
and another for dirty blocks
The normal allocation functions will try to take dirty blocks first then fallback to zeroed blocks then fallback to mapping a new block of memory
and the calloc function will do the same but steps one and two reversed
and will also only memset memory if it was from the dirty blocks list
@dense vigil what's ur opinion on this
dense vigil
Why not keep track of dirty and clean pages
And then have a background thread to clean pages etc
And then you can just get fresh pages etc
And do that
flint idol
Why not keep track of dirty and clean pages
this isnt a page allocator
this is the heap
dense vigil
I mean I’m also talking for the page allocator
flint idol
the page allocator doesn't need to keep track of dirty and clean pages
dense vigil
But this is a good idea I approve
flint idol
and will also only memset memory if it was from the dirty blocks list
my idea behind this is so that calloc wont always have to do a memset
so basically just to remove a redundant memset
flint idol
And then have a background thread to clean pages etc
also ive heard that this is terrible for cache
the background zeroing thread
dense vigil
Huh neat
flint idol
idk maybe im wrong
distant zenith
idk maybe im wrong
You’re correct
flint idol
How do you know if it’s empty though
the idea is that newly mapped memory then that is guaranteed to be zero
with mmap
flint idol
Why not a full slab allocator though
idk exactly what a slab allocator looks like but from my understanding my allocator is pretty similar to one
except the block sizes go by powers of two
uncut narwhal
The normal allocation functions will try to take dirty blocks first then fallbac...
i do this too almost exactly
flint idol
interesting
uncut narwhal
a slab is powers of two, but its for objects smaller than a page rather than groups of pages
uncut narwhal
interesting
though i dont actually use this much because i dont have a tree-based paged pool yet, only my buddy allocator nonpaged pool where pages always stay resident
uncut narwhal
interesting
do you have a zeroing thread? thats part of my plan, to have a low priority thread that takes nonzero free pages and zeros them out
flint idol
no
flint idol
also ive heard that this is terrible for cache
also like this
uncut narwhal
so how do pages get added to the zero list?
main girder
also ive heard that this is terrible for cache
this is contested
main girder
You’re correct
this is contested
uncut narwhal
also id def be interested in hearing more about if a zeroing thread is bad for cache and what the arguments would be
on both sides
main girder
the windows kernel still uses a zeroing thread and they just use nontemporal writes to avoid polluting cache
flint idol
so how do pages get added to the zero list?
no such thing in my kernel
uncut narwhal
the fact that the windows kernel uses free and zero lists and a zeroing thread is actually where i got the idea to do that
uncut narwhal
no such thing in my kernel
oh it was an idea, not something you do yet, got it
main girder
the windows kernel team did their own benchmarks in response to the movement against zeroing threads in BSDland and received results that encouraged them to keep their own
but they use fancy non cache polluting writes
which the BSD people may not have bothered experimenting with
or didnt exist back then
to the same degree of usefulness they do now
uncut narwhal
yeah
uncut narwhal
and it isnt for pages but rather heap blocks
how does obos's heap work btw? ive got a buddy allocator for nonpaged pool rn but looking into examples of how to actually do it for the paged stuff and just other ideas in general
main girder
the common wisdom against a zero page thread comes from i think dfbsd who benchmarked it and decided it was actually now a pessimization on modern hardware as it was implemented (which hadnt changed much since the 90s) and they removed it
without further experimentation
uncut narwhal
(i also use zig and zig allocators prefer not doing calloc and instead defaults to filling stuff with an undefined poison value on debug modes and letting people initialize their own stuff with case-specific defaults so calloc is way less common)
flint idol
how does obos's heap work btw? ive got a buddy allocator for nonpaged pool rn bu...
it has one cache for each possible power of two allocation size from 16 bytes to 4GiB. each cache contains a freelist and a spinlock. the allocation function finds the appropriate cache for the size requested, locks the cache takes a node from the freelist and if it doesnt exist it allocates a new page and tries again, then it unlocks the cache and returns the allocated memory
the free function just adds the node to the freelist of the appropriate cache
uncut narwhal
it has one cache for each possible power of two allocation size from 16 bytes to...
ok so if your max was 2048 instead of 4gib youd have described a slab allocator directly btw, its just you keep going up to bigger sizes lol
flint idol
it has one cache for each possible power of two allocation size from 16 bytes to...
the new page is then divided into size chunks
and added to the cache
uncut narwhal
yeah thats what a slab allocator is, each page worth of items in the cache is called a slab in the allocator
flint idol
ic
if the kernel requests a page or more then it generally always falls back to mapping a new region
uncut narwhal
(and then in 06 some linux dev nerds looked at it and decided to go overboard and optimize it by cramming all the metadata for the free list into struct page with extra nested union hell, creating slub)
flint idol
if the kernel requests a page or more then it generally always falls back to map...
actually this is the case all the time i believe
uncut narwhal
(and then some nerd made slub again but readable and put it in proxima and it get 10x better perf than my version of the same allocator for some reason)
uncut narwhal
actually this is the case all the time i believe
ok yeah that makes sense. how does obos handle mappings? do you have an hhdm or are you using some tree structure or something?
flint idol
the vma does some cursed shit with my rb tree to find avaliable space in between regions
uncut narwhal
ah, rb tree nonsense, got it
teal canyon
which the BSD people may not have bothered experimenting with
That's interesting. Are there any benchmarks from Windows?
main girder
That's interesting. Are there any benchmarks from Windows?
im going by what a windows kernel guy said when we brought it up to him
he didnt show us the benchmarks but he said "we saw this and did our own benchmarks and couldnt replicate the bad results with our own page zeroer" or something like that
teal canyon
Can their data be trusted as relevant for non-Windows systems?
It seems that they experimented with this subsystem, and not always successfully: https://randomascii.wordpress.com/2022/07/11/slower-memory-zeroing-through-parallelism/ Seeing this, I believe that this is battle-hardened information.
PS: By the way, when I click on the X to remove the image coming from the link, does it disappear only for me or for everyone?
flint idol
PS: By the way, when I click on the X to remove the image coming from the link, ...
for everyone
main girder
It seems that they experimented with this subsystem, and not always successfully...
yeah ive read this
unfortunate
i think their decision to keep the page zeroing thread predates the introduction of that bug though
and i think the bug has been fixed
teal canyon
for everyone
Phew, thank you.
teal canyon
and i think the bug has been fixed
That's right.
hybrid kraken
and i think the bug has been fixed
They fixed it, and after that they probably squeezed the maximum out of parallel zeroing
And I don't know if it will work that well without such a system
Yeah, it's a whole subsystem in the memory manager with separate calibration during initialization, measurements and dynamic adaptations
flint idol
[uACPI][INFO]: successfully loaded 1 AML blob, 1702 ops in 0ms (avg 3507181/s)```i made the allocator optimizations i said yesterday
and i also made it allocate more pages at once
and got this much ops/s
so i think it's a worthwhile optimization
[uACPI][INFO]: successfully loaded 1 AML blob, 1702 ops in 0ms (avg 3823035/s)```and this already puts me ahead of Orange
but i would need more effort to get ahead of keyronex
dense vigil
and this already puts me ahead of Orange
Orange is at 4 million+
real pecan
its a duplicate score
flint idol
oh
i guess i need some more optimizations
clang is better at optimizing right 
[uACPI][INFO]: successfully loaded 1 AML blob, 1702 ops in 0ms (avg 5157075/s)```its old
flint idol
blazing python
p2w
flint idol
if i like redo my VMA i could get really high scores
but that is a whole lot of work icl
dense vigil
Bump allocate it all 
flint idol
ngl not even a bad idea
to only check in between regions after the bump allocator fails
hm bump allocation in vma doesn't help it
[uACPI][INFO]: successfully loaded 1 AML blob, 1702 ops in 0ms (avg 5438429/s)```
I love it when the TSC calibration code glitches[uACPI][INFO]: successfully loaded 1 AML blob, 1702 ops in 0ms (avg 3906779/s)```
was able to get this, seems legit enoughtry to do pvclock
blazing python
who's that
timer for kvm
flint idol
yes i searched it
flint idol
[uACPI][INFO]: successfully loaded 1 AML blob, 1702 ops in 0ms (avg 5378603/s)```im not sure if it's a timing glitch this time
i mean every other time ive gotten 3M+
but never >4M
so prob
😔
rn im optimizing hot paths in the kernel
like the irql interfaces
distant zenith
rn im optimizing hot paths in the kernel
Why not use performance counters or pebs in this case
Could implement a simple performance monitoring api this way
flint idol
Why not use performance counters or pebs in this case
?
flint idol
I am confused what this has to do with optimizing hot paths in the kernel
distant zenith
I mean, if you’re optimising hot paths in the kernel, you’d want to see where your bottlenecks are and what’s causing them
Hence why strategies like TMAM exist
flint idol
I mean, if you’re optimising hot paths in the kernel, you’d want to see where yo...
I have some benchmark thing that monkuous made with the gcc option
that i use for that
distant zenith
Fairs
flint idol
and i know this is a hot path
because it is the irql manipulation functions
every irq
every spinlcok
every normal lock
every event object
uses it
flint idol
every event object
if u r calling into any code there is a 99% chance that the irql is changed in some way im pretty sure
distant zenith
Nuttx have some pretty nice benchmarking utilities, osperf and cyclictest are really good (these aren’t nuttx specific)
flint idol
oki
distant zenith
Nuttx have some pretty nice benchmarking utilities, osperf and cyclictest are re...
Makes it a lot easier to see where too much time is being spent
Performance counters /pebs can narrow down the reasons quite a lot
flint idol
bump
this morning I worked on file truncation
in the VFS
as I did not have that
and it was messing up programs like nano
and it almost kinda works
except for some reason somewhere i forgot to unlock a mutex which makes no sense
oh wait i see it now
I also need to fix a bug that happens when compiling with clang on windows
or well
the m$ provided clang in vs2022
because im tryna boot obos
but when i built it
that build crashes in the driver elf loader (?)
which is weird
i would've worked on it after school
but i had to finish a safety assignment
for tech
or smth
(who even needed safety to begin with?)
flint idol
wtf
it was because
it expect Sys_GetCWD to zero out the buffer
or ig
it was just because i forgot to null-terminate the buffer
anyway nano now works properly
i just pushed all my commits
im gonna have to PR in my O_TRUNC thing
to managarm/mlibc
(soon)
empty kernel
I think I found a typo: https://github.com/OBOS-dev/obos/blob/master/dependencies/limine.cmake#L25. I think you forgot to put "Linux" after "on".
stark sigil
I think I found a typo: https://github.com/OBOS-dev/obos/blob/master/dependencie...
if you're turned off you cant compile obos though, this is correct
flint idol
I think I found a typo: https://github.com/OBOS-dev/obos/blob/master/dependencie...
oh right
noticed this the other day while tryna compile on windows
stark sigil
why didnt you fix it
stark sigil
was at school
stark sigil
forgot i made that mistake
dementia
flint idol
forgot i made that mistake
ig I forgot i noticed the mistake
dense vigil
ig I forgot i noticed the mistake
cheap is talk. patches you send
flint idol
I want to optimize obos' memory overhead tbh
it uses a shitton of memory to boot (20M of physical!!)
and thats a lot and I have no idea why
dense vigil
and thats a lot and I have no idea why
Because your using 20m of physical memory? Try not using it
flint idol
dense vigil
Nah but seriously
Can’t you like have your PMM log every allocated page
And then look at it through serial
And see what’s calling it and shit
flint idol
Can’t you like have your PMM log every allocated page
no?
its 20 fucking megabytes lmfao
dense vigil
It would be slow but i mean?
flint idol
it would be untrackable
dense vigil
You could have it tossed into a program to parse it and analyze it
Isn’t there a way to know what called a function for debugging?
flint idol
gcc has a profiler
that lets you do thag
but this is untraceable because of the sheer number of allocations
and because the majority of allocations through the pmm go through the vmm
flint idol
But then couldn’t the VMM log things?
the vmm reports 13M of memory used at boot
the rest is overhead that only God knows where it comes from
real pecan
Record every allocation using a macro and line/file intrinsics and put them in a hash table
Linux can do that in new versions btw
uncut narwhal
i actually just added counters to my allocator myself, its really useful to have
lean glen
Record every allocation using a macro and line/file intrinsics and put them in a...
why a hash table
why not just output them then filter them using a script or something
flint idol
why not just output them then filter them using a script or something
actually have done that before
it's a pretty cool script imo
(it's written in c++)
although
it serves the purpose of checking for leaks and memory bugs
rather than just tracking memory usage
real pecan
why a hash table
To be able to access the stats at runtime after boot, especially on real hw
lean glen
the key would be file:line?
real pecan
Yep
uncut narwhal
does obos do preemption of threads?
flint idol
idk man I feel like that is kind of necessary for the whole "userspace" thing
flint idol
does obos do preemption of threads?
elaborate a bit cuz i feel like there is more to this question than I can see
uncut narwhal
elaborate a bit cuz i feel like there is more to this question than I can see
specifically preempting a thread by enqueuing a thread of higher priority
flint idol
oh no not yet
uncut narwhal
thats the one im not sure how to do in my kernel rn lol
flint idol
I tried that when I say hyenasky's message in #schedulers
today
but alas, something hung
uncut narwhal
rip
flint idol
thats the one im not sure how to do in my kernel rn lol
I could try to get a working solution tomorrow
or you could read hyenasky's code (?)
uncut narwhal
I could try to get a working solution tomorrow
no pressure, just obos is an os i know works so while im ruminating on this i figured id check on if obos does the thing lol
uncut narwhal
tbh my main concern is the idea of doing thread context switching in an interrupt handler since there is no instant where im in between contexts
where do you signal eoi on the timer interrupt? i think thatll actually help answer my question indirectly now that im thinking more about what im actually trying to figure out lol
found it
thanks for all the inspiration mate
flint idol
where do you signal eoi on the timer interrupt? i think thatll actually help ans...
EOIs are signaled at the beginning of each interrupt
after irql is raised
flint idol
np
uncut narwhal
and i can keep using my batshit compiler abuse of a context switch mechanism lmao
id recommend it for obos but unfortunately c compilers dont like it
flint idol
could u send some code im kinda curious
uncut narwhal
it relies on a compiler always adhering exactly to a register constraint on inline asm
flint idol
quite interesting
uncut narwhal
which is called from https://codeberg.org/Khitiara/imaginarium/src/commit/9b1dade793b8c592beadb8514b9369e722c5ed61/src/krnl/thread/dispatch.zig#L108-L145
its normal looking context switch code. load the new rbp, rsp, rip from memory into gprs, save the old rbp, rsp, rip to memory, and then actually load the rbp, rsp, and rip. except instead of saving registers using instructions i just tell the compiler yeah im clobbering everything please take care of it for me
by re-returning my arg pointer through the inline asm constraints i can transfer a pointer to the new thread
from the old thread
which i can then containerof (zig calls it fieldparentptr instead) to get a pointer to a struct on the old thread's stack
and then use that to get the old thread pointer and call schedule
if you have questions feel free to come over to my thread and i can try to answer, dont want to clog up obos chat too much
someone in my thread said that this wont work on c because c compilers are allowed to just say fuck you and ignore the register constraint completely if they want
idk how true that is tho
inland radish
thats the one im not sure how to do in my kernel rn lol
when a thread is appended to the runqueue, if it's higher priority than the current thread, then force preemption
thats it
flint idol
by re-returning my arg pointer through the inline asm constraints i can transfer...
cool (I understood nothing)
flint idol
oh the burning urge
to do linux syscall compat
oh the stale branches for tcp congestion control
and xhci
yearning to be worked on
yet
i do nothing
what a sad reality, for these abandoned features
uncut narwhal
F
flint idol
anyway time to fix the ext driver
real pecan
to do linux syscall compat
do it
real pecan
send
real pecan
dont listen to haters
uncut narwhal
lmao
flint idol
it's 2:5 votes rn lmao
uncut narwhal
do listen to the haters (disclaimer: is one of the haters)
sterile sparrow
https://discord.com/channels/440442961147199490/1227738482152181761/147298197193...
be unique
flint idol
i already have a syscall api
im bored
dont wanna do xhci
dont wanna deal with tcp
sterile sparrow
stark sigil
flint idol
mhm one day ill have a working ext2 driver that doesn't corrupt the filesystem
until then
enjoy your non-persistent ramfs
like i want to do something
but all there is to do is booooooooooooooooooring
can some1 send patches?
say
where would someone start for linux compat
ig i need some file descriptor syscall won't i
then i would need to
rewrite every preexisting syscall to translate from obos_status to errno
and then rewrite every syscall to actually be linux compatible
that's a whole lot of work icl
main girder
you should have carefully designed the semantics of your native syscalls to not directly conflict with those of Linux and then do your Linux subsystem as a distinct set of wrappers around the native calls
flint idol
and then rewrite every syscall to actually be linux compatible
(well it's not a rewrite technically, but i would have to write a lot of syscall wrappers)
ah fuck it
why not
nvm
im just gonna do smth else
i could probably fix scheduler bugs
but the scheduler has no "bugs" (that i know of)
but it isnt the best
hm
acpi trolling me
uncut narwhal
welp lmao
stark sigil
big trollage
flint idol
cc @real pecan how come S3 returned?
hm it mightve just failed
I dont check for errors
real pecan
idk? print out the return value lol
flint idol
idk if its reproducible 😔
real pecan
maybe u had suspend disabled in qemu
sterile sparrow
lucky
flint idol
bump
sterile sparrow
bump
dense vigil
OBOS 🔛 🔝
stark sigil
Managerms 
flint idol
someone find a bug and ill fix it
even better: someone fix a bug and send patches
empty kernel
someone find a bug and ill fix it
how about the bash not building bug?
that's probably an annoying one though
flint idol
build system bug, I could probably fix it by making builds reproducible with a container
flint idol
deleting discord for fluxxer soon
age verification as we all know
I am on the osdev server on fluxxer as oberrow ofc
and I will be making an obos server for progress report until they get threads
stark sigil
I am on the osdev server on fluxxer as oberrow ofc
im not on the osdev server, the website refuses to feed me the mac os installer
😔
the web client doesnt work either
white mulch
im not on the osdev server, the website refuses to feed me the mac os installer
they have issues rn
stark sigil
obviously
white mulch
flint idol
OBOS fluxer server: https://fluxer.gg/vW82XQJ1
flint idol
@knotty totem i remember you were having problems with downloading files from a webserver running on obos from your home network? if I were to start a webserver would you be able to drop me a packet capture so I can examine why it breaks?
curl -O http://obos-dev.ddns.net:8080/largefile
and ping me when you can test it (if you want to of course)
knotty totem
<@1153169946050834534> i remember you were having problems with downloading file...
Not right now but yes
flint idol
ok thanks
knotty totem
`curl -O http://obos-dev.ddns.net:8080/largefile`
and ping me when you can test ...
alright I'm ready
just tried and didn't get any response at all, in ~5sec
flint idol
just tried and didn't get any response at all, in ~5sec
yeah because the server is down
if you want to retry try in 5 seconds
and dont forget the pcap
knotty totem
alright pcap is goin
:/ firefox isn't letting me upload files
wtf
give me a minute 😔
flint idol
give me a minute 😔
ok
knotty totem
dm'd with link to it since I still can't fucking figure out file uploads
weak kestrel
weak kestrel
even more based
uncut narwhal
nice
ornate ginkgo
very impressive, good work
now connect to obos from obos via a router running obos
vale nymph
U can also do it without nat for internetworking if u aint a coward 
Make sure to send me ur ip if u do
flint idol
Make sure to send me ur ip if u do
127.0.0.1
dense vigil
Make sure to send me ur ip if u do
10.66.66.2
uncut narwhal
whoops
stark sigil
death
flint idol
this seems to be the initrd driver promising it found a file that doesn't actually exist
stark sigil
bruhb
flint idol
this seems to be the initrd driver promising it found a file that doesn't actual...
i did this by making fake_root_hehe then doing ln -s ../usr usr in fake_root_hehe
then chroot'ing into it
stark sigil
psychotic
uncut narwhal
nice
flint idol
i was able to chroot into an ext2 disk
which is cool ig
havent done anything on this for a while
flint idol
i just need to implement NAT
yesh so can somebody send patches for NAT
ill work on the xhci driver if someone send patches
flint idol
i might implement a mountable devfs
i just made it so the permission namespace thing i did
works in chroots as well
so you can limit chroot permissions
(this is in a chroot)
of course, one can just chmod that file to reenable whatever was disabled
so as long as no one thinks of that you keep the /sys/perm namespace outside of the chroot you are fine
dense vigil
Why not do kernel level enforcement of this
Like some Linux security things they do does similar?
flint idol
the point is that the user can change the capability's permissions
for a chroot, I am going to add another syscall, Sys_ChangePermRoot, which allows the permission root to be outside of the chroot
dense vigil
Ahhh
torpid wigeon
when openrc port
flint idol
in 7 weeks and 3 days
what is ispeed and what is ospeed in termios
ok
found it out
dense vigil
What are they
flint idol
ispeed is input baud rate
ospeed is output baud rate
dense vigil
(x86 uart does not differentiate between the two)
I ❤️ x86 UART
uncut narwhal
one of these days I'm going to find a teletypewriter and start using it for osdev/with hobby oses
dense vigil
I need to make my driver a bit better by not hard coding things but it’s really nice
dense vigil
one of these days I'm going to find a teletypewriter and start using it for osde...
Would that work with any old plan serial interface
Or parallel
uncut narwhal
Would that work with any old plan serial interface
serial but they're usually 5 bit characters iirc so would need conversion
depends on the era of teletypewriter though
uncut narwhal
5 bit baudot code lol, named after the same guy as the baud rate
uncut narwhal
teletypes (btw that's where tty comes from, TeleTYpe) predate 7 bit ascii by like 80 years at least
flint idol
nvm
Do not be tempted to use a divisor of 0 to try to get an infinite baud rate, it won't work. Most serial controllers will generate a unspecified and unpredictable baud rate (and anyway infinite baud would mean infinite transmission errors as they are proportional.)
uncut narwhal
lol
also generally baud rate is one of those things that needs to be synced up between all the things on the line afaik
so itd need to be configurable for sure
dense vigil
do u know what happens if divisor is set to zero in x86 uart
Undefined
dense vigil
teletypes (btw that's where tty comes from, TeleTYpe) predate 7 bit ascii by lik...
:O
uncut narwhal
:O
from like 1870ish through 1930 there was one 5-bit code, then from 1930ish until ascii was invented there was a different incompatible standard 5 bit code, and then teletypes got like half a decade of running on ascii before electronic terminals completely replaced them
flint idol
so itd need to be configurable for sure
it definitely is, through termios
and through my weird ioctl
dense vigil
Isn’t termios through ioctl?
uncut narwhal
and then after a few decades remote terminals got replaced by just putting the terminal into the damn computer, and then people started making software terminals because we missed remote terminals so much lmao
dense vigil
from like 1870ish through 1930 there was one 5-bit code, then from 1930ish until...
I wonder why 5 bits? I know ascii is 7 bits (but it takes up the full byte) to make it easier for electronics to interface with it
I guess computers then were much different in terms of that?
uncut narwhal
I wonder why 5 bits? I know ascii is 7 bits (but it takes up the full byte) to m...
teletypes predate computers, originally itd be just two of them connected over the wire and anythign typed on one would appear on the other
dense vigil
Huh neat
flint idol
Isn’t termios through ioctl?
yes
but i also have a weird /dev/COM* ioctl that isnt defined anywhere else
dense vigil
I should implement them so I have source compatability with OBOS
flint idol
they are not used anywhere.
uncut narwhal
Huh neat
yeah its cool. they even invented cryptography machines for them in the 30s that operate by passing 5 bits in and getting a different 5 bits out
dense vigil
Like the enigma machine but automatic?
uncut narwhal
yep
flint idol
i got `/dev/ttyS0` to work
now i want to figure out a way to get some sort of login running on it
dense vigil
now i want to figure out a way to get some sort of `login` running on it
getty?
flint idol
wassat again
flint idol
nah im just gonna use my own login thing
flint idol
then what is it for
dense vigil
You use it to configure it and everything etc
And it spawns login on that interface
And handles the FD stuff iirc
uncut narwhal
https://tenor.com/view/themoreyouknow-more-know-gif-4483207
oh also the 5 bit code is fully stateful, if you send 0x1B you swap to figures mode and if you send 0x1F you swap to letters mode, and theres no upper or lower case letters
flint idol
meh
dense vigil
oh also the 5 bit code is fully stateful, if you send 0x1B you swap to figures m...
Is that even worth the saved bits
It’s 3 bits
uncut narwhal
Is that even worth the saved bits
why do you think ascii was invented lol, its because this 5 bit shit was both annoying and not good enough
also with teletypes being physical typewriters you get shit like the state of the art being 75 baud
which is not a lot of baud
dense vigil
why do you think ascii was invented lol, its because this 5 bit shit was both an...
True XD
But what were they smoking when they made the 5 bits shit
uncut narwhal
idk originally for baudot, the newer 5 bit code in the 30s was to try and keep compat with physical equipment and youd just need to swap around what character meant what signal - by then they had tape punchers and readers that all expected 5 bits
uncut narwhal
hell yeah
dense vigil
That’s also what getty handles iirc
flint idol
no but the login should not stop working
flint idol
it should just start a new login prompt
what it does not do is: start a new login prompt
it hangs or something
dense vigil
Yeah that’s what getty does :p it would spawn a new login prompt? Wouldn’t login replace its self with bash
flint idol
it should just start a new login prompt
as in: it is programmed to do this
[ DEBUG ] (thread 21, process 9) syscall Sys_WaitProcess returned 0x0 (OBOS_STATUS_SUCCESS)
[ DEBUG ] (thread 21, process 9) syscall Sys_HandleClose/SysS_GDBStubBindDevice(0x4000051, 0x0, 0x0, 0x0, 0x0)
[ DEBUG ] (thread 21, process 9) syscall Sys_HandleClose/SysS_GDBStubBindDevice returned 0x0 (OBOS_STATUS_SUCCESS)
[ DEBUG ] (thread 21, process 9) syscall Sys_FdIoctl(0x0, 0x2, 0x803da0, 0x3c, 0x0)
[ DEBUG ] (thread 21, process 9) syscall Sys_FdIoctl returned 0x0 (OBOS_STATUS_SUCCESS)
[ DEBUG ] (thread 21, process 9) syscall Sys_FdIoctl(0x0, 0x1, 0x803da0, 0x3c, 0x0)
[ DEBUG ] (thread 21, process 9) syscall Sys_FdIoctl returned 0x0 (OBOS_STATUS_SUCCESS)
[ DEBUG ] (thread 21, process 9) syscall Sys_FdWrite(0x2, 0x4023c0, 0x4, 0x802a40, 0x0)
[ DEBUG ] (thread 21, process 9) syscall Sys_FdWrite returned 0x0 (OBOS_STATUS_SUCCESS)
[ DEBUG ] (thread 21, process 9) syscall Sys_FdWrite(0x2, 0x401af4, 0x8, 0x802e10, 0x0)
[ DEBUG ] (thread 21, process 9) syscall Sys_FdWrite returned 0x0 (OBOS_STATUS_SUCCESS)
this is the syscall log of login
it writes and hangs
flint idol
oh yeah
a few weeks ago
I made a change to my allocator that I documented here
about how I changed it to have the two freelists
one for zero blocks and one for regular free blocks
I forgot to mention a reason of why this is theoretically quicker
flint idol
I forgot to mention a reason of why this is theoretically quicker
memzero pages in every single page that it zeroes
which is undesirable for already zero blocks
so not doing that can technically make it faster in the short term
dense vigil
so hows the weather
flint idol
dense vigil
yeah that one is unblocked lol
flint idol
dense vigil
you could use your desktop and then inject the cookies into OBOS 
dense vigil
can files download
flint idol
yes
dense vigil
oooo
tall sluice
In obos-strap when building bash what's the difference between these two?
[ "make", "install", "DESTDIR=${destdir}" ],
[ "make", "DESTDIR=${bin_package_prefix}", "install" ],
one puts it into the sysroot the other puts it into a different sysroot for the binary package
aka the xbps package
tall sluice
one puts it into the sysroot the other puts it into a different sysroot for the ...
wdym by "different sysroot for the binary package"?
flint idol
the binary package's files are collected in a different directory
so that they can be archived into the xbps file
check build bin pkg.c of obos strap if you want to see what I mean
tall sluice
check build bin pkg.c of obos strap if you want to see what I mean
Imma look
tall sluice
Okay where specifically?
Like
Does it just archive the package on the host for later use/sharing the pkg?
vale nymph
I have always wondered how easy it would be to get ebuilds to work for packaging and building the distro
flint idol
I have always wondered how easy it would be to get ebuilds to work for packaging...
idk
Anyway today I plan on testing obos on a school computer
in a vm today
then bug fixes then on real hw
rn I might review the tcp congestion control branch
flint idol
I got it to boot on the vm but the perf is dogwater
flint idol
looking for motivation to work on this
i need to take my mind off stuff
ill just
cherry pick a commit into master
and call it a day
uncut narwhal
lol
tall sluice
the binary package's files are collected in a different directory
Yo can I explain what u mean?
flint idol
sure
flint idol
no
torpid wigeon
are you in like 9th grade or something
flint idol
im 15 yeah
snow current
you're only 15???????
flint idol
yes
snow current
damn
torpid wigeon
lol
blazing python
im 15 yeah
i thought you are older than me
torpid wigeon
makes sense then
flint idol
i thought you are older than me
no
torpid wigeon
same
inland radish
Idk I was 15 and I wasn't doing stuff nearly that advanced
flint idol
nah technically im turning 15 this year so im 14
snow current
why does everyone thing oberrow is like 20 or something
short mortar
I thought you were like 18
inland radish
In fact after I turned 16 I started writing my first OS which was nanoshell
snow current
wtf
inland radish
Before I had some shittier OSes
flint idol
why does everyone thing oberrow is like 20 or something
because u dont see 15 year olds at this level
blazing python
even binder thought that oberrow is older than binder
snow current
even binder thought that oberrow is older than binder
idk about that one
uncut narwhal
every time i see OBOS i almost misread it as oboe
flint idol
I should learn to play the oboe
dense vigil
I should learn to play the oboe
Connect the oboe to your computer as a keyboard and mouse so you can program obos using an oboe
uncut narwhal
make midi driver for obos and hook an oboe to it
dense vigil
And program a music synthesizer that makes oboe noises
flint idol
truly spoken by people who have no idea how musical instruments work