#OBOS (not vibecoded)

1 messages · Page 42 of 1

dense vigil
#

later

flint idol
#

no such thing, i told myself that once upon a time and i rewrote because of it

dense vigil
#

nahhh

#

that will just be small refactor 4

#

after small refactor 3 for smp

#

which is after bash

flint idol
#

i think im gonna do x2apic support

dense vigil
#

I mean I did it because I chose to

#

I very well could of done it differently but I wanted to make it branchlesser

flint idol
#

for some stupid fucking reason

#

nproc doesn't work with x2apic enabled

#

what

#

nproc doesn't work anymore

#

how sad

#

anyway

flint idol
#

obos type download speeds on my host 🥀

#

obos now gets slightly farther on real hw

dense vigil
flint idol
#

I think my pc's cpu hates me

#

lack of support for new cpus is one reason why I want to rewrite the x86 backend

#

wish me luck

#

@dense vigil

dense vigil
#

Good luck

flint idol
#
    dev->base = map_registers(dev->pci_bar->bar->phys, dev->pci_bar->bar->size, false);
#

P,GFSDLP;KLSD,;L'

#

SKDJFOLP;DSKF;LSWED\

#

OMG

#

i map the bar cacheable

#

fsdk;lfsdmn;fswd

#

im gonna sprinkle in some mfence and wbinvd instructions

#

this was all to no avail

dense vigil
flint idol
#

yeah I marked in noncacheable

#

did nothing

#

im gonna do the same to the TRB rings

dense vigil
#

Lmao

weary hound
#

on x86 mmio is made UC via MTRRs

#

and dma should be cache coherent

#

maybe the compiler is reordering trb writes with the doorbell ring?

flint idol
#

that should theoretically be a memory barrier and also it should at least flush the cache if dumb shit happened somehow

#

of course i could also check to see if the writes during initialization are in order

#

at least get that out of the way

weary hound
#

what problem are you experiencing in the first place

flint idol
#

it works on qemu w/ xhci pass through, but not on that same hw

weary hound
#

is memory decode and busmastering enabled in the pci config register

flint idol
#

I enabled bus master and irq

#

but not memory decode

#

uh nvm I enabled memory decode

#
 dev->dev->resource_cmd_register->cmd_register |= 0x6; Drv_PCISetResource(dev->dev->resource_cmd_register);```
lean glen
#

I still don't like that naming scheme 💀

flint idol
#

but also

real pecan
#

Patches are cheap, send talks

flint idol
real pecan
#

Not bad

#

Just speed it up by a million to get zero

flint idol
#

lol

devout niche
#

am i right in thinking your packet send from tcp context is synchronous?

#

i had a brief click through the incoming packet handling code

flint idol
devout niche
#

and it seemed that sending ack for instance will ultimately wait synchronously in NetH_SendEthernetPacket

flint idol
#

oh thx for pointing that out actually

flint idol
devout niche
flint idol
#

oh how interesting

#

I am not sure if writes to the NIC ever actually block

devout niche
#

i do this entirely asynchronously (in fact right now i don't even do tx packet queueing at all) and it doesn't even matter if the packet doesn't get sent since the retransmit timer will handle it

flint idol
#

I gtg now

#

at mosque

flint idol
#

cuz rn it sends a bunch of ACKs for small amounts of data and im not sure if thats a bad thing

devout niche
#

if you do delayed acks eventually then that would call for something similar

#

but still probably reasonable to try to ack after every 2 full segments

flint idol
#

delayed ACKs improved consistency in download speeds

#

before speeds would fluncate a lot

#

now they are consistent

dense vigil
#

Btw @flint idol the intel simics logs whenever USB has a spec violation

#

And for other violations too

flint idol
#

shiii

#

(download link?)

dense vigil
#

Under personal NDA til I finish using it to implement feature

#

Because I wana be first

#

Il tell you though later

#

And it’s free

flint idol
#

just found it

dense vigil
#

Just try and use it

#

Ask @snow current How hard it was

snow current
#

it's fucking terrible

#

I rage quit tryna get it to work lmao 😭

flint idol
#

cuz if i downloaded 5933k of data in 28 seconds, then that should mean my download speed has an average of 212k/s

#

but curl says 52k

#

for avg download

#

it's probably because i dont implement any socket options

#

no it's because my timer source for userspace is bad

#

207G download speed, everyone switch to obos

dense vigil
flint idol
#

whats a real network never heard of it trl

dense vigil
#

This

flint idol
dense vigil
#

Amazing download speed

flint idol
flint idol
#

so obos boots on simics

#

how nice

#

however, it's quite sluggish

dense vigil
#

yeah its simics

#

and you need a special VM backend

#

thats not KVM for it

flint idol
#

yeah i installed vmxcon

#

but i dont seem to see any effect

#

oh wait its not laoded

snow current
#

I didn't bother

#

it's not slow enough for me to care

flint idol
#

damn

#

and for some reason simics doesn't like my dhcp requests

flint idol
#

im absorbing rfc9293 rn

#

and apparently my sequence numbers are not to spec

#

I just use an rng

#

but apparently it should use a monotonic timer combined with some sort of cryptographic hash that is computed with these parameters:

ISN = M + F(localip, localport, remoteip, remoteport, secretkey)```
flint idol
#

these diagrams are making me laugh idk why I might be going slightly insane

#

when i think of a RST in TCP I always think of it like one tcp peer telling its peer to go kill itself

flint idol
flint idol
#

also TODO: congestion control for TCP

flint idol
#

CHAT

#

i got my final marks back for hte semester

#

93 math 😔
95 music
85 "geography"
90 english

#

anyway

#

i think i know how im gonna celebrate this trl

#

[REDACTED]

snow current
dense vigil
#

But it’s a completely different command

flint idol
#

guys

#

i was diagnosed with silly window syndrome 😔 (elite ball knowledge required)

#

||The "Silly Window Syndrome" (SWS) is a stable pattern of small incremental window movements resulting in extremely poor TCP performance. Algorithms to avoid SWS are described below for both the sending side and the receiving side. RFC 1122 contains more detailed discussion of the SWS problem. Note that the Nagle algorithm and the sender SWS avoidance algorithm play complementary roles in improving performance. The Nagle algorithm discourages sending tiny segments when the data to be sent increases in small increments, while the SWS avoidance algorithm discourages small segments resulting from the right window edge advancing in small increments.||

#

A TCP implementation MUST include a SWS avoidance algorithm in the sender (MUST-38).
blah blah blah ill do whatever i want rfc9293

#

yeah i cant lie my tcp transmission code is deeply fucked

#

i bet RFC writers tell scary stories about my tcp implementation around a campfire at night

#

"congestion? whats a congestion, never heard of 'em?"

#

to debug my loopback device on obos

#

i am going to do smth insane

#

im gonna use wireshark's ability to use stdin for captures

#

then im gonna make obos

#

write a PCAP file as it sends packets on the LO device

#

of course where will it write this pcap file?

#

it will write it onto serial

#

and serial will be routed to wireshark

#

through the stdin thing it has

#

⁨⁨```c
static bool lo_sent_file_header = false;
if (!lo_sent_file_header)
{
struct {
uint32_t magic_number;
uint16_t minor;
uint16_t major;
uint32_t resv[2];
uint32_t snap_len;
uint16_t link_type;
uint16_t flags;
} OBOS_PACK hdr = {
.magic_number = 0xA1B2C3D4,
.major = 2,
.minor = 4,
.snap_len = 0x40000,
.link_type = 1,
.flags = 0x5000
};
write_to_serial(&hdr, sizeof(hdr));
lo_sent_file_header = true;
}

    struct {
        uint32_t tstamp_sec;
        uint32_t tstamp_usec;
        uint32_t captured_packet_length;
        uint32_t network_packet_length;
        char data[];
    } *record = ZeroAllocate(OBOS_KernelAllocator, 1, sizeof(*record) + pckt->size, nullptr);
    record->network_packet_length = pckt->size;
    record->captured_packet_length = record->network_packet_length;
    record->tstamp_sec = get_current_time();
    record->tstamp_usec = record->tstamp_sec * 1000000;
    memcpy(record->data, pckt->buffer, pckt->size);
    
    write_to_serial(record, sizeof(*record)+pckt->size);

    Free(OBOS_KernelAllocator, record, sizeof(*record)+pckt->size);```⁩⁩
#

nice it worked

flint idol
#

the pcap thing works

uncut narwhal
flint idol
#

⁨⁨```c
// Released into the public domain

    static bool lo_sent_file_header = false;
    if (!lo_sent_file_header)
    {
        struct {
            uint32_t magic_number;
            uint16_t minor;
            uint16_t major;
            uint32_t resv[2];
            uint32_t snap_len;
            uint16_t link_type;
            uint16_t flags;
        } __attribute__((packed)) hdr = {
            .magic_number = 0xA1B2C3D4,
            .major = 2,
            .minor = 4,
            .snap_len = 0x40000,
            .link_type = 1,
            .flags = 0x5000
        };
        write_to_serial(&hdr, sizeof(hdr));
        lo_sent_file_header = true;
    }

    struct {
        uint32_t tstamp_sec;
        uint32_t tstamp_usec;
        uint32_t captured_packet_length;
        uint32_t network_packet_length;
        char data[];
    } record = {};
    record.network_packet_length = pckt->size;
    record.captured_packet_length = record.network_packet_length;
    record.tstamp_sec = get_current_time();
    record.tstamp_usec = record.tstamp_sec * 1000000;
    
    write_to_serial(&record, sizeof(record));
    write_to_serial(pckt->buffer, pckt->size);```⁩⁩
flint idol
#

if you ever need to debug ur network stack on ur loopback device

uncut narwhal
#

thanks

flint idol
flint idol
flint idol
#

(it works only once)

flint idol
#

oh fun fact ig

#

obos' loopback device

#

is slower than receiving the data from a remote server

#

ig that's just my tcp implementation, maybe

flint idol
#

@dense vigil

#

i am hosting a public webserver on obos

#

pls no DDOS meme

inland radish
flint idol
#

it crashed meme

#

shit embed fail

#

for future reference:
sudo firewall-cmd --permanent --zone=public --add-forward-port=port=8080:proto=tcp:toaddr=192.168.1.2:toport=80

flint idol
#

weird crash

#

it page faults trying to execute a null fptr

#

but dispatch dpcs doesn't call any function ptrs

#

nor do any of the functions it calls

#

and it faults sometime while calling ⁨Core_SpinlockRelease(&target->dpc_queue_lock, oldIrql);

#

while this does call lower irql which

#

dispatches DPCs

#

it makes sure dpc->handler isnt null before calling it

#

so that can't be it

#

the only thing i can think of is a weak function called somewhere

flint idol
#

i decided to make a core dump of obos in qemu at the time of crash on the vps

#

and im gonna examine and see what exactly happened

flint idol
#

it is 952 pm

#

time to implement congestion control

#

time to read rfc5681

#

i feel so sophistocated when i read RFCs

#

:>

uncut narwhal
#

good luck

flint idol
#

thx

flint idol
#

im understanding it but this gonna be difficult to implement

#

might need a small rewrite of the tcp transmission code

#

rfc9293 my beloved

flint idol
#

before i do congestion control im gonna implement some mitigations for attacks described by RFC5961

blazing python
flint idol
#

website is literally down smh

blazing python
flint idol
#

that's the plan

#

hopefully my vps provider is chill with this trl

dense vigil
#

It would be limited to 100mbps

#

Which for OBOS

#

Is like uhhh

#

A neutrino bomb

flint idol
#

lol

#

i actually uh do not think i even pay enough for my vps to allow such bandwith

dense vigil
#

I would use an HTTP stress testing util

#

So I could tune it or some shits

#

It’s what I used to test my HTTP server impl in HRMR

#

(This was over WiFi ti my desktop iirc)

#

(And the HTTP part was using the thread pool and it kept up really well putting tasks on the queue and having the threads take them and deliver them to the epoll loop and shit)

inland radish
flint idol
#

tbf idr what im paying for

#

all i know is that im paying 10 bucks a month and ⁨⁨ssh⁩⁩ing into the vps is painful

#

yeah 750mbps out

dense vigil
#

I know some VPS providers that may be better

#

They give me 10gbps

#

3.50 a month

#

For 1gb ram

#

Though they always out of fucking stock

dense vigil
#

Lmao

flint idol
#

idk

dense vigil
#

RFC4

inland radish
# flint idol

also how is it possible to boot custom OSes on a VPS?

knotty totem
#

Depends on the provider

#

Many will let you mount your own ISO

flint idol
#

but i just used a vm on the vps to host an http server on obos

knotty totem
#

They enable nested virtualization on yours? Or are you running it on qemu tcg

flint idol
#

qemu tcg

knotty totem
#

Ah

flint idol
#

once I fix my tcp transmission code and I get congestion control

#

and fix some instability in my tcp thing

#

I will be bringing the obos webserver back up

#

and hopefully I can uncover some bugs/vulnerabilities with obos

#

I also might add some watchdog timer to obos so that I dont need to manually reboot that server when it crashes

#

would be nmi powered, if after like a minute the userspace doesnt write a byte to the device, it will reset the kernel with a triple fault

dense vigil
#

Why not have it set the IDT to 0 or something

#

More reliable

#

At triple faulting*

inland radish
#

set CR3 to 0

dense vigil
#

Something could be there

#

Making the IDT limit 0

#

Will certainly always crash

flint idol
#
#ifdef __x86_64__
    UACPI_ARCH_DISABLE_INTERRUPTS();
    // Try triple faulting.
    struct {
        uint16_t limit;
        uint64_t base;
    } OBOS_PACK gdtr = {.limit=0x18-1};
    asm volatile ("lgdt (%0); mov $0x8, %%ax; mov %%ax, %%ss; push $0;" : :"r"(&gdtr) :"memory");
    // We should not be here anymore.
    OBOS_UNREACHABLE;
#endif```
#

this is what I do if acpi reboot fails

flint idol
flint idol
flint idol
#

i fixed some bugs in my tcp transmission code

#

now it actually respects the MSS

#

and it also now sends packets properly

#

and retransmits packets properly

#

i think it retransmits packets properly

#

what kind of shit is obos smoking 💀

flint idol
#

The obos test webserver should be back up

#

if anyone wants to try and crash it

wet smelt
#

my requests are timing out 😔

flint idol
#

⁨```
Flags [S.], seq 13167384, ack 1832486055, win 65535, options [mss 1460,[bad opt]]

#

idk who's sending this bad packet

#

now while obos is probably handling it right

#

idk why it hung

#

it should work again

#

i will continue working on my kernel in the meantime

real pecan
#

RCE on oberrow computer because of crappy obos net stack trl

flint idol
#

hold on that server will be down for a bit

#

i need to make sure that no packets are retransmitted if the connection is reset

blazing python
flint idol
#

did u try it

blazing python
#

yes

flint idol
#

dna internet?

#

im tcpdumping as well to check out

#

what causes the kernel to die

blazing python
#

im using vpn

flint idol
#

ur vpn put you in finland i think

#

hm wait

#

i think someone was somehow trying to get obos to send a challenge ACK

#

as they established a connection, only up to SYN-RECEIVED

#

then tried to send another SYN

wet smelt
#

idk im just trying to connect to it on my browser

#

and i am in finland

#

so probably me

flint idol
#

your thing is sending obos a SYN

#

obos sends a SYN, ACK back

#

then your thing never ACKs it

#

and instead sends another SYN

#

which according to RFC5961 is a type of attack or smth

wet smelt
#

the another syn could be me refereshing the page

flint idol
#

nah it would use a different port number, most likely

wet smelt
#

yeah right

flint idol
#

and here it's on the same port

#

ah nvm i think i see a problem

#

yeah that's it

#

i misread my packet dump

#

obos isn't seeing the ACKs that are sent back

#

and it's also not seeing the RSTs that are sent when your thing gives up

#

which means

#

it's a bug with my e1000 drive

#

r

#

that's causing it to drop packets

#

while indeed this is a bizzaire bug

#

i think i have found the cause

flint idol
#

anyway that server should be back up

#

someone just downloaded ⁨/home/obos/.bashrc

#

how exciting

#

this is quite exciting

#

wow obos is getting crawled by web spiders

#

and fun fact

#

as soon as you send a link in discord

#

it crawls it

#

this was right after i sent a link to this on discord

trim bay
#

given the timing that's probably discord's embed generator

flint idol
#

true

#

and some1 just made it stop working 😔

trim bay
#

that's me

#

i tried to download /usr/bin/xterm

#

just via firefox, nothing special

flint idol
#

ah

trim bay
#

it got through the first 0.7MiB, said the download failed, when retrying got to 1.1MiB, failed again and now it's refusing connections

flint idol
#

well you just found a bug so i will be generating a core dump and rebooting

#

my new favourite qemu monitor command is ⁨dump-guest-memory

#

it gneerates a coredump file using the guest memory

#

anyway that server is back up

flint idol
wet smelt
flint idol
#

someone managed to find a bug on obos

#

they were using a vpn

flint idol
sterile sparrow
#

first time i've seen it up lol

#

uh oh i think i killed it

flint idol
#

it's still alive

sterile sparrow
#

oh yeah

flint idol
#

transmission is kinda broken so there is a non-zero chance it wont work

sterile sparrow
#

ah

flint idol
#

that'll be fixed once i do congestion control

sterile sparrow
#

that was so close lol

flint idol
#

at least it didn't crash

sterile sparrow
#

true lol

#

not loading anymore for me :(

uncut narwhal
flint idol
#

it died for me as well 😔

#

time to reboot

blazing python
flint idol
#

it definitely would make sense if this were a congestion problem

uncut narwhal
#

one of these days ill get networking up and can have my os download things from yours lol

flint idol
#

as im a lot closer physically to that server than you are

flint idol
#

i once ⁨ssh⁩ed into astral from obos

#

it was an astronomical moment, truly

sterile sparrow
#

o it completed

flint idol
#

lets go

sterile sparrow
uncut narwhal
#

if youve got a link you want me to try downloading from windows i can do that btw, dunno if anyone else ehre is using windows's network stack

flint idol
flint idol
#

try downloading the ahci driver

#

ah fucking hell

#

the kernel hung

#

wait a sec

uncut narwhal
#

right as i start it

flint idol
#

it's back up now

uncut narwhal
#

got to 118.8k and then stopped

#

no idea if thats a crash or if its just slow tho

flint idol
#

dang

uncut narwhal
#

still sitting at that number

real pecan
#

the entire web should move to obos ngl

uncut narwhal
#

lmao

sterile sparrow
#

yes

flint idol
flint idol
blazing python
#

obos pterodactyl when

flint idol
#

obos can't even talk to another obos

uncut narwhal
#

attempt two it stopped at 118.0k instead of 118.8k

#

still stopped though

flint idol
#

the download worked on obos

#

smh

#

obos' TCP stack is better than NT's confirmed

white mulch
#

I need to try downloading it on crescent troll

flint idol
#

do it

flint idol
uncut narwhal
flint idol
#

damn

#

obos' network stack caused obos' network stack to crash

#

i would send a screenshot but some tcpdump logs got caught in the crash dump

uncut narwhal
#

rip

flint idol
#

but anyway

#

that server is up after the crash

#

i should probably have this on a reboot timer

#

or add some sort of watchdog device

#

software watchdog through an NMI that happens every like minute idk

#

im gonna implement tcp congestion algorithms now

#

@devout niche what tcp congestion algorithms does fadanoid use, out of curiosity?

devout niche
flint idol
#

i see

#

there is no way lol

#

some malicious site tried to POST something lol

#

⁨```
14:38:35.633057 IP srv-91-224-92-115.serveroffer.net.49316 > 192.168.1.2.http: Flags [P.], seq 1:71, ack 1, win 64240, length 70: HTTP: GET /cgi-bin/luci/;stok=/locale HTTP/1.1

#

smth else did this

#

⁨```
14:38:54.423113 IP 192.168.1.2.http > 193.142.147.209.45378: Flags [F.], seq 1, ack 1, win 65535, length 0

#

and this is teh ip of another malicious server that tried probing at obos

sterile sparrow
#

lol

flint idol
#

i should make obos just shit out utter bs on port 80

#

to confuse web crawlers

flint idol
#

which is a bug on some TP-link router

flint idol
torpid wigeon
#

TIL you're canadian

snow current
#

oh ew

flint idol
#

i indeed do live in canada

#

my parents are from syria

flint idol
snow current
#

speak funny

flint idol
#

are u american or smth

snow current
#

fuck nah

flint idol
#

what are u

white mulch
flint idol
#

lol

real pecan
#

what is darkhttpd

flint idol
#

idk stardust told me i should port a few months ago

#

i think cuz it was easy to port

#

it's written in pure C

flint idol
#

did it download the file successfully or?

white mulch
#

I think yes, I just tried to print the data

flint idol
#

i see

flint idol
#

of the file

#

and i can compare it against the actual file

#

ok that server is down for a bit

#

im fixing a bug where retransmissions of FINs would happen when they should not

flint idol
#

anyway that server is back up

#

since i am procrastinating TCP congestion control im gonna port git

dense vigil
#

well its more of a full server framework but would be neat to see lmfao

flint idol
#

epoll is scary

dense vigil
#

i mean how bad can it be

flint idol
#

i dont want to find out

dense vigil
flint idol
#

yeah git does not work at all

#

it cant even initialize a repo

#

however, it was insanely easy to compile

#

currently actively implementing congestion control

flint idol
#

im implementing the tahoe algorithm

#

with slow start and congestion avoidance

#

im not really understanding when and how the cwnd and ssthresh values should be updated

#

bah why does congestion have to exist why cant packets just magically make their way to the destination instantly

flint idol
#

uh I need to change how packet enqueuing works

#

as the sequence numbers are completely messed up sometimes

#

which is why more often than not people would get incomplete downloads

#

the bug is currently that snd.nxt is not moved when it would go past the send window edge

#

which is correct behavior

#

but the push data to remote function

#

doesnt account for that

#

nor does the retransmission timer

#

what should happen is that assignment of the sequence number should happen when it actually transmits the segment

real pecan
#

How does cogestion work

flint idol
#

congestion control is tcp's best attempt at not overloading a network

#

how this is done varies by algorithm

#

but you generally have one variable in the tcp state that is called the congestion window

flint idol
#

how the congestion window is managed differs by what algorithm is used

#

but in the one I am doing, the initial congestion window is initialized through the value of MSS

#

and it also has another variable called ssthresh

#

which is the slow start threshold

#

the ss threshold starts at an infinite value

#

and is modified at a few points

#

which i forgor

real pecan
#

Interesting

flint idol
#

as im new to how this works

flint idol
#

for a description of some algorithms

flint idol
#

this algorithm says congestion happens whenever the tcp has to do a retransmission

flint idol
#

and the slow start threshold is set to congestion window / 2

flint idol
#

the congestion window increases whenever an ACK is sent by the peer

#

and by what degree it increases depends on whether the tcp is currently using slow start (i.e., congestion window < slow start threshold), or congestion avoidance

dense vigil
#

iirc the slow start window is normaly 14kb? and thats why your site is good to be under that for most cases

sterile sparrow
#

this sounds way more fun than debugging xorg race conditions for 3 weeks

flint idol
#

otherwise it is increased by this formula: ⁨⁨mss * mss/cwnd⁩⁩

dense vigil
flint idol
#

it is

#

a lot funner

dense vigil
snow current
#

I love race conditions

#

they are really fun to debug

flint idol
# dense vigil <:true:1320478073358913629>

in the slow start algorithm it starts based off this:
⁨⁨```c
if (con->state.snd.smss <= 1095)
con->state.ciw = 4 * con->state.snd.smss;
else if (con->state.snd.smss <= 2190)
con->state.ciw = 3 * con->state.snd.smss;
else
con->state.ciw = 2 * con->state.snd.smss;

dense vigil
#

just set race_condition=false

#

your done

flint idol
dense vigil
snow current
#

turning off multithreading

dense vigil
#

it can still happen on one core

snow current
#

well

#

that's easy to debug tho

dense vigil
#

thats why you set race_condition=false

#

its the magic bullet

#

just like how c++ is the magic bullet for shotgunning your leg off

flint idol
#

so the congestion window starts at 4380 bytes

flint idol
#

weirdly

#

there are attacker IPs that send a SYN to obos

#

and obos SYNs back

#

but they just RST back

#

idk if this is a port scan

#

or if they're tryna abuse a CVE

#

it's quite weird

flint idol
#

wasnt congestion control or anything

#

my tx code had a bug i hadnt noticed yet

#

the obos server now has a build with experimental congestion control btw

uncut narwhal
#

nice

flint idol
uncut narwhal
flint idol
#

obos, sees reset
must mean i need to retransmit SYN,ACK

#

hold on ill bring it back up

#

its back up

uncut narwhal
#

nw lol, just figured id try it

#

ok this time it went so fast i didnt even get to see output from invoke-webrequest

#

and worked

flint idol
#

damn

#

u sure it actually made a request?

#

im looking at the kernel logs and it hasnt logged anything to do with congestion control

flint idol
uncut narwhal
#

same result, ima see if invoke-webrequest has some sort of verbose option to get speeds showing

flint idol
flint idol
uncut narwhal
#

🎉

flint idol
#

for some reason, this build seems to be less stable than the last one

#

as the kernel/web server stops responding to GET requests

#

after enough requests

flint idol
#

it turns out when i was implementing some attack mitigations in RFC5961, I accidentally made the RST handling code drop the reset if the segment was acceptable

#

instead of if it is unacceptable

#

unfortunately, my congestion control algorithm seems to fold really easily

flint idol
#

im gonna commit what I have so far

#

and go to sleep

#

gn yall

flint idol
#

my computer sci class has a lot of pcs im gonna see if they will let me test obos on them

sterile sparrow
#

lol i'll be taking inspiration from that :^)

flint idol
#

didnt ask yet

flint idol
#

today im going to be fixing bugs causing the http server to break

flint idol
dense vigil
#

Then do your homework on OBOS

#

Problem solved

sterile sparrow
flint idol
#

but homework nonetheless

sterile sparrow
#

in my schools religion class the teacher plays movies while we just use our phones

#

it's fucking pointless

flint idol
#

lol

sterile sparrow
#

not to mention they're the same movies always

flint idol
#

damn my vfs syscall source file is 3249 lines

#

why do i have so many vfs syscalls

flint idol
#

wtf is this crash

#

the faulting instruction is a rep movsb

#

the faulting instruction is nowhere near rdi or rsi

dense vigil
flint idol
#

0x210292 is rflags

#

let me check my bitfield

#

it's unset

dense vigil
#

btw quick question do you remember the CPUID thing for 1gb and 2mb pages

#

wtf is this doing

flint idol
#

2mb are always supported

flint idol
dense vigil
#

test on kvm

flint idol
#

no such fault ever happens

flint idol
#

anyway

#

s

snow current
#

leaf=0x80000001, subleaf=0, edx, bit 26

dense vigil
#

ty

snow current
#

iirc

flint idol
# flint idol s

i still have many bugs to fix with tcp transmission on obos

snow current
#

yeah

dense vigil
#

it is i saw on sandpile

flint idol
#

never knew checksum checks actually failed over networks

#

i thought that was a myth made up by big RFC to sell more checksum functions

dense vigil
#

talk is big dont send patches

flint idol
sterile sparrow
#

my school blocks some github pages ips but not all for some reason

#

but hey they don't block my ip so

flint idol
#

when u built obos

#

i was able to reproduce it

#

I reproduced it when trying to boot obos on a school pc in the comp sci room

#

had compilers pre-installed on it it was quite easy

#

and vs studio 2022

#

and obos already builds on windows anyway

#

then when I booted it on virtual box (also pre-installed)

#

I reproded the crash

dense vigil
flint idol
#

the reason I couldnt reproduce it is because it didnt happen with ld, only with lld

#

the driver interface was trying to access the driver header's function table before applying the relocations

flint idol
dense vigil
flint idol
dense vigil
#

Wait why wouldn’t RIP be going to zero?

flint idol
#

why would it?

#

it wasnt calling any function in the ftable

#

it was just checking if one was null

#
diff --git a/src/oboskrnl/driver_interface/loader.c b/src/oboskrnl/driver_interface/loader.c
index 0e71a53..bea4473 100644
--- a/src/oboskrnl/driver_interface/loader.c
+++ b/src/oboskrnl/driver_interface/loader.c
@@ -163,12 +163,6 @@ OBOS_NO_UBSAN driver_id *Drv_LoadDriver(const void* file_, size_t szFile, obos_s
             *status = st;
         return nullptr;
     }
-    if (!header_.ftable.driver_cleanup_callback)
-    {
-        OBOS_Error("%s: Refusing to load a driver without a cleanup callback.\n", __func__);
-        if (status) *status = OBOS_STATUS_INVALID_HEADER;
-        return nullptr;
-    }
     for (driver_node* curr = Drv_LoadedDrivers.head; curr; )
     {
         if (strncmp(header_.driverName, curr->data->header.driverName, 64))
@@ -397,7 +391,10 @@ obos_status Drv_UnrefDriver(driver_id* driver)
             driver->main_thread->free(driver->main_thread);
         driver->main_thread = 0;
     }
-    driver->header.ftable.driver_cleanup_callback();
+    if (driver->header.ftable.driver_cleanup_callback)
+        driver->header.ftable.driver_cleanup_callback();
+    else
+        OBOS_Error("%s: memory leak! driver_cleanup_callback is nullptr!\n", __func__);
     for (driver_node* node = driver->dependencies.head; node; )
     {
         // node->data->refCnt--;
#

this is the diff

#

tcp is driving me a bit insane
(have to get this off my chest im telling you today?)
((elite ball knowledge required to understand that trl))

dense vigil
flint idol
#

yeah

dense vigil
#

I’m on mobile and can’t read diffs

flint idol
#

yes

flint idol
flint idol
#

ohoshahsbsheus

#

I get it now

#

i get it now

#

ill be redesigning my retransmission queue tomorrow

#

to be a buffer

#

and when data is ACKed or the retransmission timer is hit, it will try and send more data

#

more details tomorrow

#

after I play terraria

flint idol
#

anyway the reason I am doing this

#

is because sequence numbers become weird and out of order because I managed to get the kernel to send them wrong

#

so how this new transmission queue will work is there will be infinite space like the rx buffer

#

and the semantics will be the same

#

but the consumer is the network stack

dense vigil
flint idol
#

true

#

I should probably also limit the rx buffer

#

but problem for later

dense vigil
#

Just limit it to like 16k max sized packets worth of space

flint idol
flint idol
#

in TCP

#

it should probably have SND.WND*2 size

dense vigil
flint idol
#

in tcp

#

as the goal with transmission is to send as much as the network and receiver can handle to maximize throughput

dense vigil
#

I’m not saying 16kb I’m saying enough space to fit 16 thousand packets of the max TCP packet size

#

Or is that like nothing

flint idol
#

"max packet size" can mean a lot

#

it can mean the MSS (max segment size) sent by the peer

#

it could also mean 64K because that is generally the ip max packet size

dense vigil
flint idol
#

1460*

#

1460*16000 is a lot of bytes, realistically if i were to limit this buffer it would be double the amount the peer can handle (so send window * 2 bytes, which is likely around 128K bytes)

dense vigil
#

Still less than infinite so my point stands trl

#

But yeah that sounds about right

#

Could even make it a 2mb page worth of buffer size?

#

And then have some TLB bonuses

flint idol
#

and hopefully this will allow for larger files to be sent as right now it is completely broken

#

and this buffer wont actually be the exact same as the rx buffer

#

it might have some more internal read pointers

#

which i will describe later

dense vigil
#

Nice nice

dense vigil
flint idol
#

i might do it

dense vigil
#

Then I think I will too

#

Since my buddy allocator will do 2mb to 4kb pages hmmCat

flint idol
#

I found out how to bypass my school's firewall

#

if i use curl --resolve

#

it completely bypasses it

#

only works on my site tho

dense vigil
flint idol
#

https I have found the firewall returns invalid certificate

flint idol
# flint idol if i use curl --resolve

I figure this is because they examine the http payload for the Host: section and also any other hints of the host (dns queries as well as hints in the SSL connection i think)

#

but --resolve makes it oblivious to the fact that im trying to communicate with an IP that it blocks

#

because all the http headers and everything contain the unrestricted domain on it

flint idol
#

I should rewrite obos as a microkernel

#

all the cool kids are doing one

#

jk

#

maybe one day ill write a microkernel

flint idol
#

cc @dense vigil u were wondering about this

dense vigil
#

Fuck yah

#

Is it working

flint idol
#

what the tcp thing

#

no i have barely been home all dau

#

day*

#

didnt check yet

#

im gonna work on fixing obos real hw bugs and xhci soon

#

so I can actually have smth to show

#

when I try testing obos

#

I actually have found that obos doesn't boot on a 10th gen intel core cpu

#

in vbox

flint idol
flint idol
#

(who would've guessed)

uncut narwhal
flint idol
#

hey if anyone wants ill also test their kernels if they send me an image

flint idol
dense vigil
#

And generate an image

flint idol
#

sure

dense vigil
#

I cannot wait til I get exit() working tomorrow LETSFUCKINGGOOOOOO

flint idol
#

😔

uncut narwhal
#

mine doesnt have any way to get info out of it unless youve got outb 0xE9 atm or id send it

dense vigil
#

I wouldn’t mind moving to Canada lowkey

flint idol
#

canadas amazing

dense vigil
uncut narwhal
dense vigil
#

You can make the serial port a kernel cmdline argument

flint idol
#

detect it with acpi

dense vigil
#

And probing is a form of detection

dense vigil
#

Imo

lean glen
#

Or get me accepted into uoft or uwaterloo

uncut narwhal
flint idol
#

but I think u could get in

#

toronto is so meh

#

tbh

lean glen
#

uoft is excellent

flint idol
#

im talking about the city

uncut narwhal
#

technically my bootloader does have display stuff so i could send an image but youd just stop getting feedback after exit boot services and my kernel ends by shutting down with uefi runtime services so itd probably just display stuff for a bit and then shut off lmao

lean glen
#

oh yeah probably

flint idol
#

its like rip off Canadian new york

lean glen
#

probably will have to end up with McGill

lean glen
flint idol
lean glen
#

That is, if McGill accepts me

#

I'm not sure if I wanna do CS or CE yet, and CE is more competitive

#

I'm not even sure if I wanna go to McGill, kinda dont wanna move cities but otoh it's a world renowned uni with access to a world renowned CS lab

flint idol
#

I see

lean glen
#

👁️ 🦭

#

(there is no sea emoji)

#

Do you know where you wanna go

flint idol
#

im not sure yet, but I am definitely aiming for uwaterloo or for smth good like that

lean glen
#

CS?

flint idol
#

yeah

lean glen
#

do you think you have the grades?

flint idol
#

im not old enough to know yet

lean glen
#

if not then CE is less competitive apparently

flint idol
lean glen
#

I thought we were the same age

flint idol
#

nah

#

im younger

flint idol
#

but im definitely going to lock in because I want to get somewhere good

#

in the future

lean glen
#

huh, I was sure you were like 17-18

flint idol
#

nah

#

I always left my age vague

#

cuz when I joined this server I was underage for discord lmao

lean glen
dense vigil
#

im 17 and i feel wofully under done here yaaar

flint idol
dense vigil
#

i have that tism too

#

you out tismed me

flint idol
dense vigil
#

(i am diagnosed with the AAA package)

flint idol
#

autism, adhd, anxiety?

dense vigil
#

yup

flint idol
#

dang

dense vigil
#

i hate it

lean glen
#

The most impressive young guy here I can think of is andy

flint idol
lean glen
#

but I haven't really seen him active for like 2 years

#

He's my age, but aero was pretty damn good when he was 14

flint idol
#

hm

#

I recognize this name idk why

#

is he the rust guy

lean glen
#

Yeah

#

aero blew up a bit on GitHub too

flint idol
#

interesting

#

seems like I have some catching up to do trl

lean glen
#

it's not really a competition

#

We're all doing our own stuff

lean glen
#

nah it's normal to feel this way

#

But keep in mind people here are not representative of most developers

#

There's a heavy selection bias, and there's very smart people that will make you feel dumb even though you're still over average

#

You just gotta think about it optimistically and learn from them

dense vigil
#

i wonder where id actualy rank on some hypothetical global leaderboard

dense vigil
#

but then again i look at #osdev-misc-0

#

and realize

#

maybe im doing doing sooo bad

lean glen
#

Well if you successfully enabled paging you're probably top 1% of this server statistically lol

dense vigil
#

i have userspace workin and doom ported with possibly the most janky mlibc port ever

snow current
lean glen
snow current
#

oh yeah

dense vigil
#

What

#

There is nothing wrong with that

snow current
dense vigil
#

Ikr!

#

I’m adding in exit() tomorrow tho lmao

#

A proper one

snow current
#

sure sure

uncut narwhal
#

hell i wrote my own bootloader for my project and still have never actually manually enabled paging lmao

#

thanks uefi

dense vigil
#

Heck most people can’t even get a GDT

flint idol
#

if u have done smp u have successfully enabled paging

uncut narwhal
vale nymph
flint idol
#

doesnt*

#

and isnt that only usable with 32 bit addresses

#

i mean x2apic tho...

#

well gg u got me there

vale nymph
#

unreal mode

#

u can just do mov [ds:ebx], XXX after the 386 nobody will stop you cro

uncut narwhal
flint idol
#

unless of course it is long mode smp

snow current
#

or you are in long mode at all

flint idol
flint idol
#

not me, thats for sure

trim bay
flint idol
#

someone should make an SMP capable real mode os

flint idol
#

also a todo for my init program is to initialize the utmpx entries

uncut narwhal
flint idol
#

letsgo
my mlibc sysdep PR got approved

uncut narwhal
#

yo nice

#

also, i do know a project that has no paging/virtual memory at all but i dont know offhand if it does smp, dont think so

#

it has multitasking i know that much

flint idol
#

damn

dense vigil
#

il do it

dense vigil
#

@flint idol so how’s that network stack coming along

flint idol
dense vigil
#

well the fixing of TCP?

flint idol
#

no progress since I described my idea of the buffer instead of queue

#

for tx

#

progress will continue sometime in the next three days

vale nymph
#

@flint idol what webserver did u use

flint idol
#

darkhttpd

flint idol
vale nymph
flint idol
#

um what happened on the 25th of january lol

dense vigil
flint idol
#

bruh

#

why is this legal

#

for fucks sake ive had ai scrapers on the goddamn obos web server

#

as in

#

the web server RUNNING ON OBOS

#

fsdfnsdo

dense vigil
#

But they do something called

#

“Lobbying”

flint idol
#

lobbying

#

yeah

#

this and the epstein shit bro like this world is so fucked

dense vigil
#

#introduce-yourself message

#

OBOS is for the new world

flint idol
#

lmao

#

yes it is

#

obos for the new world

#

FTNW

flint idol
flint idol
#

i like this a lot more

#

then my old queue

dense vigil
flint idol
#

one

#

tx_buffer.lock

#

is how many this implementation

#

more locks than your entire kernel

dense vigil
#

I’m going to add spinlock.h

#

Just you see

#

@flint idol hows this

flint idol
#

cring

#

atomic flag better trl

#

also feel like a clanker made this idk why

#

not architecture independant either

dense vigil
#

i stole it from reddit

flint idol
#

doesn't turn off IRQs

#

either

flint idol
#

3/10

dense vigil
#

why would it need to disable IRQs

flint idol
#

deadlocks?

dense vigil
flint idol
#

the whole point of 'em are that they're interrupt safe

dense vigil
#

ok what would you prefer i make trl

#

ive never used atomics

flint idol
#

what

dense vigil
#

i mean like

#

what is the atomic flag

flint idol
#

atomic_flag

dense vigil
#

isnt that a C++ thing?

flint idol
#

no

dense vigil
#

oh right i have no stdatomic PSP1G_pspTrollar

flint idol
#

was added in C11

flint idol
#

ur using clang how do u even mess that up??

blazing python
dense vigil
blazing python
#

at least because you can just add std::atomic<sometype> and not care about it

dense vigil
#

thats part of the limine C template

#

so people using the template afaik would have no stdatomic.h

flint idol
#

unless u pass -nostdinc u will have it

dense vigil
#

well well well trl

#

how tf did that even get in there