#OBOS (not vibecoded)

1 messages · Page 41 of 1

flint idol
#

my memory manager already has a shitton of overhead

dense vigil
#

Well it would be more of a tracking of the MMAPd region its self describing the status of the pages in there

#

And that’s owned by the processes

#

And then the kernel has a reclaim list that points to all the different MMAPd files and the procs that have them mapped

flint idol
#

I would have to keep a list of virtual addresses and contexts that have a struct page mapped

dense vigil
#

So the kernel goes to the list and then tells all the processes on one of the MMAPd files to unmap it

dense vigil
flint idol
#

the kernel would go through the list and remap the pages as unpresent

flint idol
dense vigil
flint idol
#

I could keep a list of page ranges that have the file mapped in the vnode

#

the page range already contains information about what range of the file is mapped

#

so I would do some simple arithmetic and iteration over that list to invalidate mapped pages

dense vigil
#

And a page range is part of the VMM right

#

Like when you MMAP 4MB or some crap

#

It keeps track of that’s

#

Right

flint idol
dense vigil
#

I think this shall work

flint idol
#

well no I go through the standby list

#

#1141057599584878645 message

flint idol
#

the pmm goes through the standby list to reclaim memory when it needs to do an allocation on OOM

dense vigil
#

What is the standby list?

dense vigil
#

So a list of things on standby to be reclaimed?

flint idol
#

list of things that can be reclaimed

#

reclamation in obos only happens when the pmm freelists are completely exhausted

dense vigil
#

Ahhh

#

So that would refer to such vnodes then? So that you can go through and invalidate the things

flint idol
#

yeah

#

specifically the struct page refers to a vnode

#

and also contains the file offset

#

that it refers to in the pagecache

dense vigil
#

Basically all the info on how to properly invalidate it in the page cache and unmap it from everything etc

dense vigil
#

Yes the struct page for the standby list.

flint idol
#

the standby list is simply a list of non dirty, reclaimable (i.e., pages written back to disk or swap) pages

dense vigil
#

Or wait what’s a struct page exactly

#

A PFNdb entry?

flint idol
dense vigil
#

Ahhh so that would have pointers to the vnodes

flint idol
#

yes

dense vigil
#

And then when you need to reclaim a page that has pointers to all the data to unmap and then invalidate the cache entry and then have it freed

#

And it could also see if it’s dirty and have that sent back to disk etc

flint idol
#

the modified page list has dirty pages

dense vigil
#

Ahhh

#

And when you MMAP a file and read from it it’s okay

#

But then if you write it’s marked dirty

#

Etc etc?

flint idol
dense vigil
#

I presume if the standby list is empty you can tell the driver to flush the dirty pages to disk?

flint idol
#

there is a thread called the page writer which is responsible for taking pages on the modified page list and writing them back to disk, and throwing them onto standby

#

(i say writing them back to disk but technically it can be written back anywhere)

dense vigil
flint idol
#

and the dirty and standby page lists include both anonymous pages and file pages

#

anonymous pages are written back to swap while file pages use the backing driver to writeback the pages

#

(i say backing driver instead of fs driver because any block device can also be a file page)

dense vigil
#

Okay but I think I get the gist of this

#

Thank you for information

#

I hope I don’t forget it 💀

flint idol
#

btw i would also ask fadanoid about his MM design his is probably cooler than mine

dense vigil
#

Thank you cool USB man

flint idol
#

np

dense vigil
#

Hopefully I can get off my ass today and write some code 💀

#

I’ve mostly been absorbing knowledge

flint idol
#

I promised my friends I would start a terraria server in like 20 minutes

#

so I might not get much done

#

anyway I will now be implementing TCP SACK

dense vigil
#

I was uhhh also wanting to play terraria 💀

flint idol
#

idk if i need to change transmission logic yet

dense vigil
#

Why do you need SACK?

#

also don’t make a joke about this don’t make a joke about it being called SACK

flint idol
dense vigil
#

Btw for my ram block device what’s your opinion on using a radix tree for it hmmCat

flint idol
flint idol
dense vigil
#

For an in memory one

#

Similar to page table

flint idol
#

seems fun

dense vigil
#

Yeah shouldn’t be too bad

flint idol
#

i would personally have done a linear buffer but like that seems a lot more memory-efficient

dense vigil
# dense vigil Like when you MMAP 4MB or some crap

Wait how does this know what parts it can free from because it may not be a fully backed backing. Oh wait that’s why you use the PFNdb and have it separate and this is just so you can know where to look and unmap crap

dense vigil
#

So if you never touch a block

#

It’s just never allocated

#

I’m gonna limit it to 256mb and do two layers have 256

#

Just because easy

flint idol
#

oki

dense vigil
#

And 99% of use cases for such a device won’t need more

flint idol
#

there's some dumbass in #osdev-misc-1 lmao u should check it out

#

making unintellectual arguments about how "linux bad windows good" 🥀

dense vigil
#

🥀

#

Be there in a sec

flint idol
#

like which parts can be reclaimed? which parts can be unmapped?

dense vigil
flint idol
#

it checks the pfndb

dense vigil
#

Yeah

#

I just wanted to make sure that’s the reasoning for the PFNDB

#

For that

#

And the mapping is just so you know what to mark as non available and where etc

#

For the references

#

Right?

flint idol
#

y

#

ea

flint idol
#

Since the data receiver may later discard data reported in a SACK option, the sender MUST NOT discard data before it is acknowledged by the Acknowledgment Number field in the TCP header.

#

interesting

dense vigil
#

Why not just assume a perfect world where no packets get lost

#

It’s the users problem

flint idol
#

UDP has literally not changed since august 1980

dense vigil
#

XD

#

Dose your thing handle UDP well

flint idol
#

yeah

#

literally the easiest shit ever

#

reception: just throw all received packets onto a per-port data queue

dense vigil
#

Who needs TCP then PSP1G_pspTrollar

flint idol
#

transmission: formulate the 4-field header and send it to the destination through the IP stack

flint idol
#

minecraft

#

ftp

#

ssh

#

etc.

flint idol
#

lol

dense vigil
#

I mean witeguard is UDP right

#

And implements TCP ontop

#

Or some shit

flint idol
#

sending a udp packet is literally just:

udp_header hdr = {
  .src_port=host_to_be16(src),
  .dest_port=host_to_be16(dest),
  .length=host_to_be16(length),
  .chksum=0 // checksum is not even fuckin required
};```
flint idol
#

it probably sends IP packets on top of a UDP socket

dense vigil
flint idol
#

not for udp

#

although the ip header does have a checksum that is required to exist and be checked

dense vigil
flint idol
#

reason why: i couldnt figure out the algorithm properly 💀

dense vigil
#

Use the check summmm

#

You have to sum the check

flint idol
#

talk is cheap send patches

flint idol
dense vigil
flint idol
#

mkay im waiting for the patches to come in

dense vigil
#

This isn’t so bad

#

Tbh

flint idol
#

tbh i was just too lazy

#

btw obos does have a ones complement function u can use

dense vigil
#

It’s just append some data and sum some bytes right

flint idol
#

something like that

dense vigil
#

Well prepend

flint idol
#

no append

dense vigil
#

It says prefixed

#

But it dosnt matter

flint idol
#

yeah the pseudo-header is prepended

dense vigil
#

Because it’s just summing it all

flint idol
dense vigil
#

yeah

#

It’s pseudo header header data padded data to make the length divisible by 2?

flint idol
#

btw u can copy tcp_chksum

dense vigil
#

And then sum all the bytes with ones complement

flint idol
#

you just need to generate the psuedo-header

#
static uint16_t tcp_chksum(const void *seg1, size_t sz_seg1, const void* seg2, size_t sz_seg2)
dense vigil
#

It’s legit the same PSP1G_pspTrollar

#

Lmfao

flint idol
#

(i did this to avoid having to do any data copying in the tcp stack)

dense vigil
#

So
Copy that function

#

Then generate the pseudo header

flint idol
dense vigil
#

Checksum that a little

#

And then the rest of the function

dense vigil
flint idol
#

i was joking

#

but if you want to send patches sure

dense vigil
#

I mean I want to 🙏

flint idol
#

(dont send actual patches make a PR im not tryna review raw patch files)

dense vigil
#

Shit uhhh

#

Does the master repo build eighty

flint idol
#

yeah just make the pseudo header, copy paste tcp_chksum, set the checksum in the header, profit

dense vigil
#

Right

#

So I can do this

flint idol
#

build eighty?

#

what

dense vigil
#

Built right

#

Autocrat

#

Autocorrect

#

😭

flint idol
#

yeah the master branch builds... why wouldnt it

#

anyway obos now sends SACK PERM in the SYN packets

#

now i need to actually implement selective ACK

#

first i need to parse the response SYN or the response ACK to see if the peer sent SACK PERM back

dense vigil
#

Last time it didn’t and crashed on me when I tried to run it

flint idol
#

well it built

#

but it didnt run

#

there is a dfference

dense vigil
#

Does it run

flint idol
#

runs for me

dense vigil
flint idol
#

gh actions build also runs

dense vigil
#

Got it

flint idol
#

no

flint idol
#

seg1 is intended for the pseudo-header

dense vigil
#

It takes two segments?

#

Ahhhh

flint idol
#

seg2 is for the data

dense vigil
#

Smart

flint idol
#

and then they are checksummed as if they were one big block

dense vigil
flint idol
#

so that it doesnt have to do any data moving

blazing python
#

obos xhci bug fix when

flint idol
flint idol
blazing python
dense vigil
flint idol
dense vigil
#

I’m gonna make an edit for that

#

To be a USB plug

#

Brb

flint idol
#

(at least, TRBs work)

blazing python
#

try to do polling later

#

instead of interrupts

flint idol
#

i tried to poll

#

nothing ever showed up

#

it's a problem with the enqueue itself

#

either that, or the doorbell() function is broken

flint idol
#

but that was also to no avail

dense vigil
flint idol
#

lmao

#

beautiful

#

still some instability

#

but that seems unrelated to SACK itself

flint idol
#

The SACK option SHOULD be filled out by repeating the most recently reported SACK blocks (based on first SACK blocks in previous SACK options) that are not subsets of a SACK block already included in the SACK option being constructed. This assures that in normal operation, any segment remaining part of a non-contiguous block of data held by the data receiver is reported in at least three successive SACK options, even for large-window TCP implementations [RFC1323]). After the first SACK block, the following SACK blocks in the SACK option may be listed in arbitrary order.

#

no clue what that is supposed to mean

#

anyway, SACK does increase TCP performance a ton

#

my only problem is that wireshark is reporting a lot of duplicate ACKs

#

(sample)

#

i see why, because hdr.ack is the same

#

but idk if that's just wireshark unrightfully complaining about my selective ACKs or if it's an actual problem

devout niche
flint idol
#

as in the host os

#

and it downloads the file instantly

devout niche
#

try with a large file, i wonder how it compares to keyronex at present

#

i have yet to add header prediction which will hopefully improve mine

flint idol
#

avg. download is 300k

#

(it's really slow cuz obos is dogpoop)

dense vigil
#

And if it can even do that

flint idol
#

but i havent checked

flint idol
dense vigil
flint idol
flint idol
#

well it's owrking

#

working*

#

but le transfer speeds are le shit because obos is le shit™

dense vigil
#

whats teh speed

flint idol
#

5k/s

#

💀

dense vigil
weary hound
#

download it over an usb 3 nic :^)

flint idol
#

um a part of the reason why everything is so slow is probably because there are so many retransmissions

#

literally for like every packet transmitted

#

and pretty much every ACK in wireshark is marked as duplicate

#

like i genuinely do not know if it is TCP or the nic driver at this point

#

and i have like no way of knowing

#

i think i see a problem

#

it ACKs a packet

#

and fails to advance rcv.nxt

#

and then later sends a selective ACK without having advanced it yet

#

and that happens for pretty much every other packet

#

why how interesting

flint idol
#

bruh why is the nic pushing packets with length 4 to the rx queue

#

and with EOP set

dense vigil
#

maybe dont tell the nic to push packets with a length of 4 to the rx queue

flint idol
#

good idea thanks

#

i shouldve thought of that

flint idol
dense vigil
#

yeah every nic has a setting to push packets with a length of 4 to the rx queue randomly

flint idol
#

"this clown inserts himself into every thread"

#

lmaoooo

dense vigil
flint idol
#

maybe it's because when im not in my progress report thread

flint idol
#

i found the rxctl bit to disable that

dense vigil
flint idol
#

no im too naive or smth

dense vigil
#

stick up ass

flint idol
#

ohhh

dense vigil
#

ok im gonna go eat food

#

then work on my ext2 crap

#

i may actualy make a fuckin standalone version

#

so i can have a test suite on it?

#

wait sorry wc

flint idol
#

which obos sometimes gets

#

and there are very little retransmissions

#

and no SACK packets sent

#

because i presume the kernel gets everything in order

#

wtf

#

why does every other packet in the nic queue

#

have a length of 4 bytes

dense vigil
#

tf

flint idol
#

no

#

it's a 4-byte runt

#

that has no meaning

dense vigil
#

so how do you detect the size

flint idol
#

an ethernet header has 14 bytes btw

flint idol
dense vigil
flint idol
#

the frame size

dense vigil
#

what if you just tossed every other packet trl

#

or tossed 4 byte ones

flint idol
#

the kernel discards runts

#

ethernet says that packets have to be 64 bytes or larger, otherwise it is a runt

#

anyway i think the problem is elsewhere anyway

flint idol
#

shouldnt do any harm on the server, but the i might abruptly end the connection if i see there is a bug or obos is doing sm shit wrong

dense vigil
#

idc as long as its not breakin shit on my end

flint idol
#

it should not™

dense vigil
#

im expecting to get hits for downloads n crap

#

just dont dos me tro

#

and if you do

#

you have to send patches

#

to uhh make my kernel better

flint idol
#

as pretty much everything will reset connections after enough lost ACKs

dense vigil
#

we will see

flint idol
dense vigil
#

dont make me make an ethernet version of the meme

flint idol
#

lol

#
e1000e_rx_descr Next RX descriptor: ring #0, PA: 0x443d1340, length: 16
e1000e_rx_desc_buff_write buffer #0, addr: 0x4439c000, offset: 0, from: 0x7fff48a38660, length: 60
e1000e_rx_metadata_status_flags status_flags is 0x1
e1000e_rx_descr Next RX descriptor: ring #0, PA: 0x443d1350, length: 16
e1000e_rx_desc_buff_write buffer #0, addr: 0x4439b000, offset: 0, from: 0x5603560d4304, length: 4```
#

qemu is trolling me rn

#

sometimes i feel like rewriting obos since i have gathered le many knowledge over this past year and a half

#

and i want to write it better

#

but i feel like im in too deep

#

😔

dense vigil
#

Maybe make an OBOS3

flint idol
#

we already at obos 5 KEKW

dense vigil
#

Oh I know

flint idol
#

NOBOS: new OBOS trl

dense vigil
#

But it’s still called “OBOS”

#

Rights

flint idol
#

yes it is still called OBOS

dense vigil
#

Call it “OBOS3”

#

To confuse people

flint idol
#

lol

dense vigil
#

And then act like OBOS2 is a cognito hazard

flint idol
#

OBOS3

dense vigil
#

That erased its self

flint idol
dense vigil
#

OBOS2?

#

What could that be

#

There is no OBOS2

flint idol
#

it's in a repo somewhere on gh

#

find it

#

OBOS

dense vigil
#

What if you just called it somthing else

#

“something else”

#

With the miss spelling

flint idol
dense vigil
#

The somthing else kernel

flint idol
#

what if i call it managram

#

(not managarm)

#

it will be a non-pragmatic, synchronous monolithic kernel

flint idol
#

idk

dense vigil
#

His name is managram

#

That’s the joke

flint idol
#

oh

#

omg hahahahashdasghdhahahahahahahhahazsdycasg dahahhahahahaha

dense vigil
#

I thought managarm was actually managram and named after him

#

So this would be perfect

flint idol
#

lol

dense vigil
#

😭

flint idol
#

ill name it fadanoid, after chuck fadanoid

dense vigil
#

What if you call it evalynOS

#

After me

#

Because I’m so good and cool

flint idol
#

without the uppercase E for trademark reasons trl

dense vigil
#

Actually

#

I trademark both

flint idol
#

anyway im not rewriting

flint idol
dense vigil
#

I use capital and lowercase

#

Mixed in places

flint idol
#

what about evaLynOs

dense vigil
#

Damn you got me there

flint idol
#

named after my two imaginary exes (ive been single all my life), eva and lyn

dense vigil
#

Btw how does your tmpFS work

#

And handle MMAP

flint idol
#

(undertale reference did you catch it?)

dense vigil
#

Because aparently my idea is dumb and stupid with ext2 and a ram block device to make things simpler

dense vigil
dense vigil
dense vigil
flint idol
#

well this fills me with determination to work on tcp trl

dense vigil
#

And I got shat on for wanting VFS in kernel and filesystem drivers in user

#

I just wana do somthing different and cool

flint idol
dense vigil
flint idol
#

lol

dense vigil
#

Dosnt even matter

#

It’s an init rd

#

It’s going to get fucking nuked

#

After root is mounted

#

And it has a really small size

#

So slongesab

#

Idk why my idea was said to just be bad

flint idol
#

do it to prove them wrong (or right)

dense vigil
#

They also said it wouldn’t fit enough the normal VFS system?

#

Which no it would very well would

#

Idk

#

I’m just gonna do it and worst case redo it

#

Worst case I have a mostly working ext2 driver I can polish for something else

flint idol
#

im gonna sing in here in a bit wait

#

fuck i forgot the lyrics

#

greduoijtgfhrewsuoighnrews

#

ive no clue what's causing the tcp to discard me data

#

i added logs to the TCP stack

#

and it reports nothing or very little packets dropped

dense vigil
#

You believe in me

#

Which means I can do it

#

I will prove them wrong (or right and eat my own words)

flint idol
#

I SHIVERED BENEATH YOU
ALL WRAPPED UP IN EMBERS
IT WAS A NIGHT
TO REMEMMMBER

#

THEN IO WALKED AWAY

dense vigil
#

But either way somthing good will come of it

flint idol
#

YOU ASKED ME TO STAY

#

NOW UR THINKIN OF WHAT I COULDA BEEN

#

BUT YOUVE BECOME SOMEONE IVE SEEN BEFORE

#

I SWORE... ID SEEN YOU BEFORE..?

#

anyway uh

#

"a night to remember - beabadoobee&laufey"

#

it sounds a lot better when ur not reading lyrics from a discord screen

#

i think

#

idk my taste in music is shi

dense vigil
#

We should sing bohemian rhapsody

#

But I wana say thank you for being a good from oberrow

#

And believing in me and my ideas

flint idol
dense vigil
flint idol
#

nop

#

NOP-0x90

#

im an x86 cpui fsdofkdspolfkdslp;

#

i might be a bit tired or smtghrsweuij

#

smth

#

ofkdsoklfm

#

ads

#

fsdfsd

#

ffsgsdf

dense vigil
flint idol
#

what is happening in the lyrics

#

why are we sayin bismillah

dense vigil
dense vigil
flint idol
#

fuck

#

u got me there

flint idol
dense vigil
flint idol
#

just transliterated

dense vigil
#

@flint idol ya know what you need

#

a break from USB and TCP/IP

#

lets play terraria

flint idol
#

that's a good idea but it's a bit late rn

dense vigil
#

maybe tomorrow?

#

i was just gonna play an hour or some then sleep

flint idol
#

tomorrow im free yeah

#

im free until monday

#

done all my exams

dense vigil
#

oh nice nice

flint idol
#

and there is no chance im going outside in this -20 celsius weather

dense vigil
#

yeah lets play tomorrow and take a break from osdev both trl

flint idol
flint idol
dense vigil
#

oh shi terraria added non binary voices and its a slime gloop XD

#

based

flint idol
#

damn

flint idol
#

i barely have played this game

dense vigil
#

i havent played in ages so

#

we just playing normal mode

flint idol
#

guess we're the same then

dense vigil
#

i may make an expert world just for one boss to get the shield

#

because its easy enough

flint idol
#

i c

dense vigil
#

the eye of cathulu in expert mode drops a really sick item

flint idol
#

minecraft would also be nice, but alas, multiplayer is hard to do on java

dense vigil
#

and diff is tied to world not charater

#

so ezpz

dense vigil
#

if you wanted to

flint idol
#

oo

dense vigil
#

but im kinda bored of it

#

well its also my git server

flint idol
#

fair

#

lol

#

lies

dense vigil
#

its not up rn

#

lmao

flint idol
#

smh

dense vigil
#

i direct you to here though

#

hosted on the same box

#

that has all my old playthroughs

flint idol
#

interesting

dense vigil
#

all ran on that server

flint idol
#

oh nvm it's back up

dense vigil
#

no static ip

flint idol
#

no it's a static ip

dense vigil
flint idol
#

i couldn't be damned to pay for a domain

dense vigil
#

i have email server tho

#

so i have domain

#

shi whens your birthday

flint idol
#

also obos also happens to be the name of a big norweigian houes development company

#

june 8

dense vigil
#

il buy you a domain if i can

flint idol
#

nah it's good lol

flint idol
dense vigil
#

exists

flint idol
#

btw for legal purposes, obos' name is not obos, and it is oberrow os

flint idol
#

idk

dense vigil
#

what about this

#

when i get some spare cash

#

il buy it for ya

#

and even give hosting

#

with SSL and everything

flint idol
#

nono it's good

#

my vps is only 10 bucks a month and i only host files on it

#

but it's a really low-effort website

dense vigil
#

i mean its nothing to me to get for you

flint idol
#

i insist, it is fine

dense vigil
#

okay ;3

#

lmk if there is anything else you need

#

also why no SSL on there slongesab

#

lets encrypt is free

flint idol
#

hm

dense vigil
#

and its like 2 comnmands

#

install certbot

#

certbot install or whatever

#

but yeah @flint idol do the SSL pls

flint idol
#

yeah

#

The requested apache plugin does not appear to be installed

#

yeah what's the goddamn plugin

#

apache documentation is shit

dense vigil
#

also you should setup copyparty for your file server trl and also use a markdown redner framework for the main site

#

is realy easy

dense vigil
#

makin my world

flint idol
#

oh maybe the reason tcp is so slow is because of the lack of congestion control in obos

#

*a reason

flint idol
#

wait

#

i havent considered

#

that obos is sending ACKs

#

but the nic driver fails to send them

#

and silently drops them

#

also something i will note is that obos sends a lot of detached ACKs

#

so if it received like 10 packets for example

#

it will ACK each of those 10 packets' with one packet each

#

but what it should technically do is after getting 10 packets, it should ACK the 10th packet only

#

why every ~56 packets there is retransmisison

#

that has to be a coincidence

#

nvm

dense vigil
flint idol
#

7*8=56

dense vigil
flint idol
#

with a simple optimization i was able to improve speed on external hosts

#

i made the nic driver insert packets into the network stack

#

instead of having the network stack have to wait for packets from the nic

#

but one thing i am noticing is that download speeds are inconsistent

#

throughout the connection

#

and it kinda looks like sin(x)=y

#

the fluncations

#

it goes from fast to slow, then to fast, repeat

#

which sounds like a problem with window management (?)

#

i need a solution for all these short ACK bursts

#

i have just observed that download speeds are faster using the qemu user backend

#

probably because it's linux doing all the heavy lifting

dense vigil
#

i wonder what happens on a physical nic?

flint idol
#

i could try on a test subject...

dense vigil
#

btw my file server caps at 100mbps btw

flint idol
#

obos wasn't making it that far anyway

dense vigil
#

i can spin one up that has up to 10gbps (still 1gbps realistic due to stuff)

flint idol
dense vigil
flint idol
#

i am getting kb speeds on obos

dense vigil
#

but like for later if you need

flint idol
#

i dont even think i have fast enough wifi

dense vigil
#

do a speed test

flint idol
#

~500mbps

#

download

#

also the wifi in my room is quite shit

dense vigil
#

yeah you may hit the cap on that server

flint idol
#

for the real hardware test

#

i will only be doing it against my pc

#

through a direct ethernet connection

#

because only god knows what the nic driver is about to do

dense vigil
#

imagine it works perfectly

flint idol
#

oh fuh no the iso is like a gb with all the packages i was testing installed\

dense vigil
#

first try

flint idol
#

that's not gonna load until february

dense vigil
#

huh

flint idol
#

i might also eventually test this on my laptop

#

which has the original NIC that obos has ever had a driver for

flint idol
#

this laptop is running on hopes and dreams

dense vigil
#

god undertale and deltarune are so good

#

obos deltarune port when

flint idol
#

this laptop's ethernet port is extremely damaged

dense vigil
#

make a wifi driver

flint idol
#

if only i had like a usb driver and a usb nic

#

i could use

dense vigil
#

usb you say

#

XHCI you say

flint idol
#

clear

#

oops

#

this isnt my terminal

dense vigil
#

it is mine now

flint idol
#

i swear this ethernet port is in a quantum superposition

#

IS THE INTERFACE DOWN OR UP AHHHH

dense vigil
#

sudo claude "make xhci driver pls thank u"

flint idol
#

lol

#

TCP over USB

dense vigil
#

TCP over display port

flint idol
#

yeah this laptop is dead 💀

dense vigil
#

rip laptop

flint idol
#

i have another test subject but it needs an external display

dense vigil
#

use a tv

flint idol
dense vigil
flint idol
knotty totem
dense vigil
#

you need to go back and date alphys and goto the true lab

knotty totem
#

and actually it's ethernet over usb

flint idol
#

ok

#

i do also have another test subject but it's in the basement (scary)

#

evalyn can u send me a display port cable trl

#

my mailing address is 4 york st toronto, ontario trl

dense vigil
#

also btw if you play terraria alone before going to hard mode make sure to block off the evil biomes because that will make them spresd like c razy and fuck up your works tro

dense vigil
#

i only have two at all

flint idol
#

uh we should probably go to sleep our speech is becoming non-coherent

dense vigil
flint idol
#

lol

dense vigil
#

thats why i have 3 decent IPS 1080p moitors for my setup

flint idol
#

if i unplug my pc from my monitor i could use that display port cable

dense vigil
#

only 75 and 60hz tho

flint idol
dense vigil
#

but we must test

flint idol
#

ill implement the x2apic and test it on my pc trl

dense vigil
#

x2 apic is easy

#

btw

flint idol
#

yeah i just havent gotten to it yet

#

theoretically, obos is compatible with the RTL8125

dense vigil
#

does that work with 09:00.0 Network controller: Realtek Semiconductor Co., Ltd. RTL8852BE PCIe 802.11ax Wireless Network Controller

#

wait no

#

thats my wifi

#

0a:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8211/8411 PCI Express Gigabit Ethernet Controller (rev 15)

#

this one lmfao

flint idol
#

what's the device id + class code + subclass + prog if

dense vigil
#

how do i check

flint idol
#

lspci -v

#

er no

#

i forgor

#

u need to go through sysfs

#
cat /sys/bus/pci/devices/location here/device 
#

obos only tries to initialize these device ids, so ig it's incompatible

static uint16_t device_ids[] = {
    0x8161,
    0x8168,
    0x8169,
    0x8136,
};```
#

and out of the 4, it's only been tested on the RTL 8169

#

if i could i would support all realtek cards that fall under the 'r8169' linux driver

#

but i am too lazy and there is no way to test it and no reward

dense vigil
flint idol
#

it is supported then

dense vigil
#

sick

flint idol
#

as that is my laptop's NIC which is what i used to test the obos network stack before i added the e1000 driver

dense vigil
#

btw fuck whoever made IOMMU group 15 on my desktop

#

it has BOTH my nvmes and my network card and wifi card and the sata ports and the XHCI controller

flint idol
#

lmao

dense vigil
#

this is also partly why i want an HEDT board so the IOMMU groups are good 🙏

flint idol
#

well the driver definitely initializes the device

#

when i do passthrough

#

but that's just about it

flint idol
#

i think something i could do to optimize tcp performance is to decrease copying

#

and allocation

#

currently Net_TCPPushReceivedData copies the received data into the intermediate buffer

#

and then if it sends an ACK it copies all the data in the intermediate buffer into a dynamically reallocated buffer for the tcp socket implementation to read

#

which might not be ideal

#

as allocation is quite inefficient (if you ask me)

#

also the window seems to become full quite often

#

the rx window

dense vigil
#

how large is that buffer for the nic?

flint idol
#

nic has absolutely nothin to do with rx window

dense vigil
#

ahh

flint idol
#

rx window being full means the peer thinks that it cant send anymore data

dense vigil
#

ahhh

flint idol
#

and it seems like that's happening as obos isn't sending ACKs fast enough

dense vigil
#

also i finished up on this one trl

flint idol
#

loll

dense vigil
flint idol
#

the extra ACKs is because I dont really know how to delay ACKs properly so i just send ACKs as soon as possible

#

and they aren't extra ACKs, per se

#

they are just ACKs that could be put into one ACK packet

dense vigil
#

Ahh

#

What about the 4 byte packets

dense vigil
#

So you get a packet and add it to the queue or check if it hasn’t been flushed in long enough

flint idol
flint idol
#

but im not quite a fan of that solution

dense vigil
flint idol
#

i could probably have an ACK timer

#

but i tried that and it was slower

dense vigil
#

What about an ACK timer and an ACK limit

flint idol
#

because that makes no sense

dense vigil
#

So you send a packet after every X packets or on a timer

dense vigil
flint idol
#

i "fixed" the closed windows

flint idol
flint idol
#

now

#

another probable problem is that the kernel is just slow

#

or scheduling policies are not being fair to the network stack

dense vigil
flint idol
flint idol
#

ill have the nic have a queue of connections that need an ACK sent on them

#

and after the network stack is done processing currently available packets

#

it will go through that queue and ACK everything

dense vigil
flint idol
#

nah i got a better idea

#

im gonna go microoptimize stuff

dense vigil
#

how micro we talkin?

flint idol
#

very

dense vigil
#

uOPS level?

flint idol
#

maybe

dense vigil
#

you will need this then

flint idol
#

im about to remove essentially every sanity check in critical paths

#

checks like if (!list) return OBOS_STATUS_INVALID_ARGUMENT;

#

or smth

dense vigil
#

what happens when thoes fail?

flint idol
#

a page fault

dense vigil
#

Can you prevent them elsewhere?

flint idol
#

maybe idk

dense vigil
#

Also how does your copy to userspace look like trl

#

I heard you manually checked all the page tables right

flint idol
flint idol
dense vigil
flint idol
#

user memcpy is not used for syscalls like read

dense vigil
flint idol
#
    status = OBOS_STATUS_SUCCESS;
    void* kbuf = Mm_MapViewOfUserMemory(CoreS_GetCPULocalPtr()->currentContext, (void*)buf, nullptr, nBytes, 0, true, &status);
    if (obos_is_error(status))
        return status;

    size_t nRead_ = 0;
    status = Vfs_FdRead(fd->un.fd, kbuf, nBytes, &nRead_);

    if (nRead)
        memcpy_k_to_usr(nRead, &nRead_, sizeof(size_t));

    if (obos_is_error(status))
    {
        Mm_VirtualMemoryFree(&Mm_KernelContext, kbuf, nBytes);
        return status;
    }

    Mm_VirtualMemoryFree(&Mm_KernelContext, kbuf, nBytes);```
#

it maps a "view" of the user buffer

#

then it does the read

#

and then it returns

#

and it's as shrimple as that™

dense vigil
#

This seems slow PSP1G_pspTrollar

flint idol
#

who knows

#

(it definitely is)

dense vigil
#

Why don’t you try my fancy cool method

#

That’s faster

#

And maybe help the TCP stack

flint idol
dense vigil
#

Huh?

flint idol
#

kernel page table isolation?

#

user page tables do not contain any kernel page tables

dense vigil
#

How does that work

flint idol
#

except for the bare minimum for a context switch into kernel mode

#

context switch meaning IRQ or syscall

dense vigil
#

True

flint idol
#

e.g., the isr stubs or syscall trap handler are mapped

#

but not driver code

#

or anything else

#
        memzero(Arch_MapToHHDM(cached_root), OBOS_PAGE_SIZE);
        // Map the ISR handlers.
        map_range(cached_root, (uintptr_t)&Arch_StartISRHandlersText, (uintptr_t)&Arch_EndISRHandlersText, BIT(0));
        // Map Arch_KernelCR3
        map_range(cached_root, (uintptr_t)&Arch_KernelCR3, (uintptr_t)(&Arch_KernelCR3 + 1), BIT(0) | BIT_TYPE(63, UL));
        // Map CoreS_SwitchToThreadContext
        map_range(cached_root, (uintptr_t)&CoreS_SwitchToThreadContext, (uintptr_t)&CoreS_SwitchToThreadContextEnd, BIT(0));
        // Map kernel stacks.
        for (size_t i = 0; i < Core_CpuCount; i++)
        {
            cpu_local* const cpu = Core_CpuInfo + i;
            map_range(cached_root, (uintptr_t)cpu->arch_specific.ist_stack, (uintptr_t)cpu->arch_specific.ist_stack + 0x20000, 1|BIT_TYPE(63, UL)|2);
        }
        // Map cpu local structs.
        map_range(cached_root, (uintptr_t)Core_CpuInfo, (uintptr_t)(Core_CpuInfo + Core_CpuCount), 1|BIT_TYPE(63, UL)|2);
        // Map IDT
        extern struct idtEntry g_idtEntries[256];
        map_range(cached_root, (uintptr_t)&g_idtEntries, ((uintptr_t)&g_idtEntries) + 0x1000, 1|BIT_TYPE(63, UL)|2);
        extern uintptr_t Arch_IRQHandlers[256];
        map_range(cached_root, (uintptr_t)&Arch_IRQHandlers, ((uintptr_t)&Arch_IRQHandlers) + sizeof(Arch_IRQHandlers), 1|BIT_TYPE(63, UL)|2);
        // Map syscall trap handler.
        extern char Arch_SyscallTrapHandlerEnd;
        extern char Arch_SyscallTrapHandler;
        map_range(cached_root, (uintptr_t)&Arch_SyscallTrapHandler, (uintptr_t)&Arch_SyscallTrapHandlerEnd, 1|2);```
#

this is the only kernel memory mapped in user mode

#

this does make any syscall or IRQ extremely slow

#

because it has to change cr3 twice

#

im gonna open the SDM and read on PCIDs

dense vigil
#

i may not be smart

#

but this sounds really stupid

flint idol
#

same here

#

it's not stupid with PCIDs

#

otherwise it is quite stupid

dense vigil
#

how do PCIDs work

flint idol
#

like very stupid

dense vigil
flint idol
#

blah blah

#

x86 sdm 3a 5.10.1

#

read up

dense vigil
#

nah id win

#

also how much secuirity does this give?

#

at all

flint idol
#

it's a spectre/meltdown mitigation

dense vigil
#

ahh true

flint idol
#

also it's the only possible thing on some architectures (i.e., m68k)

dense vigil
#

linux does it?

flint idol
#

yes

dense vigil
#

damn, this sounds like it will be a PITA to implement trl

flint idol
#

urp and srp

#

urp is the page table it uses while in user mode

#

and srp is the one it uses in kernel mode

#

making it theoretically only possible for the kernel to look at user memory through a view (which i already do)

#

unless of course srp and urp are always the same

dense vigil
#

though who cares about spectre/meltdown tro

flint idol
#

idk security nerds

#

i have no idea why i decided to do KPTI

dense vigil
#

and without PCIDs trl

flint idol
#

i did it without PCIDs because i had no idea what those were/didn't give enough of a fuck

#

just committed everything trl

flint idol
#

after i merge the xhci-driver branch obos is probably gonna reach like 75k loc

#

how exciting

#

man the burning urge to rewrite major parts of the kernel

#

it's starting to get to me

dense vigil
flint idol
#

OBOS6

dense vigil
#

OBOS69

flint idol
#

if i were to rewrite one day i would have the exact same syscall api+abi

#

because i am too lazy to port mlibc again

dense vigil
#

do binary compat

flint idol
#

yeah

#

that's what i would do

dense vigil
flint idol
#

usermode binary compat

#

and maybe even driver binary compat

dense vigil
#

do a micro kernel

#

call it OBOSmini

flint idol
#

miniOBOS

dense vigil
#

microminiOBOS

flint idol
#

then i am probably writing a really slow microkernel

dense vigil
#

just make it fast lol

#

just set fast=true

flint idol
#

ok if i were to rewrite the vfs that would cost me 10k loc

#

mm: 4k loc

#

driver interface: 2.3k loc

#

scheduler and synchronization primitives: ~3k loc

#

x86_64 functions: 9k loc

#

network stack: 5k loc

#

allocator: 500 loc (the allocator is fast enough so im not too worried about it)

#

irq+timer interface: 1k loc

#

drivers: 12k loc

#

and that is everything important

#

the shittiest code quality in all of these is probably the code under arch/x86_64

#

t'was written very lazily

#

it would also be arguably the easiest to rewrite

dense vigil
#

maybe if you make a new kernel dont go for a m68k port too?

flint idol
#

probably

dense vigil
#

my plan was to do a seperate 32bit kernel forked off my main one

flint idol
#

the m68k port is very unmaintained

#

idk if it even builds rn

dense vigil
#

my kernels internal name is gonna be called kernel

#

and the 32bit one woll be kernel32

flint idol
#

ic

dense vigil
#

as seperate will mean they can be better optimized for the bit shite

flint idol
#

ic

dense vigil
#

like HHDM things?

#

i use that heavily

flint idol
#

u should not be using HHDM enough that it is impossible to stop using it

dense vigil
#

maube

flint idol
#

it's your choice how i decide to waste the next 4 days

dense vigil
#

let me see how bad it is first

flint idol
#

kk

#

i say it's quite bad

#

not for any reason i just have a hunch

flint idol
#

huh what's wrong with my isr stub code?

dense vigil
#

it just looks messy

#

imo

dense vigil
#

okay but look at mine

#

and look my into my eyes

#

and say mine is more messy

#

🙏

flint idol
dense vigil
#

i dont

flint idol
#

yeah

#

but i do

#

and you dont swapgs

#

and you dont have to switch cr3

dense vigil
#

i do that elsewhere

flint idol
#

i do it all in asm because why would i make the C code do that

#

and add a layer of indirection

dense vigil
dense vigil
#

you mean the switch case

#

and use a table?

#

i have plans to do that

flint idol
#

no i mean the whole dispatch_interupt

#

yeah

#

until you do

#

speak not about "messy code"

dense vigil
#

talk cheap send patches

flint idol
#

no u

#

u called my code messy

dense vigil
#

true

flint idol
#

and compared it to your code

#

when your code does less

dense vigil
#

okay give me a list of stuff mine is missing

#

and il implement it

flint idol
dense vigil
#

okay il use a table

#

anything else boss

flint idol
#

swapgs in asm

dense vigil
#

alright

flint idol
#

🥀 is your kernel not architecture-separated

dense vigil
#

its somewhat 🥀

#

but i need to work on that