#OBOS (not vibecoded)
1 messages · Page 34 of 1
it faults on mulss
mulss %xmm4, %xmm0```
; in intel asm syntax
mulss xmm0, xmm4```
movss xmm0, [rax+8]```
that was the instruction before it
it loads 0x3f4ccccd
into the first dword of xmm0
then zeroes the rest
I got a disassembly around the faulting instruction
you have to set up one of the sse registers like sysv abi expects
and also set up the fpu registers like it expects while youre at it
just one or all of them?
time to go to the sysv abi docs
so uhh this
// all sse exceptions masked
// initialise the x87 FPU state as it would be after the FNINIT instruction
#define CTX_XINIT(x, u) {\
(x)->mxcsr = 0x1f80; \
*((uint16_t *)&(x)->fx[0]) = 0x37f; \
*((uint16_t *)&(x)->fx[2]) = 0; \
(x)->fx[4] = 0; \
}
yeah that fixed it
which makes sense, since I was getting a precision error
in mxcsr
at the time of fault
and those should've been masked
obos is somehow almost at 41k loc
I need to implement file creation in my vfs
the fat driver is capable of doing so
but the vfs isn't
well I'll be damned
(it's faulting accessing *tail of a linked list)
this happens when I load my ahci driver
and it happens specifically while probing for partitions
fixed
forgot to zero something somewhere

[ ERROR ] FAT: Error following cluster chain: Cluster is over disk boundaries. Aborting.```
bruhhhh
yeah might be a bug with my FAT12 fat traversal code (I couldn't be bothered to debug that atm)
somewhere, there have been big regressions with the fat driver
I cannot even simply read a file anymore
womp
Obos is a bootloader
I don't get it
Reading a sector to phys 0
lol
this is the 15th time I've had to debug the populate_physical_regions function in my ahci driver
fixed
(probably)
so there has been this weird bug where after a few commands are run
bash segfaults a bunch of times
then the kernel either triple faults or just idles
and I have no fucking idea why
I am unsure if it happens with builtin commands as well
as in, if this still happens if I only use builtin commands
it does not
which means it is probably a bug with fork
I think you have a bug in kernel, I hope this is helpful 
oh thanks, I thought it was a cpu bug
[ LOG ] User thread 7 SIGSEGV (rip ffff90000058bc00, cr2 ffff90000058bc00)```

in a different crash, I get a GPF
ret ; faulting instruction```
stack
my kernel rn:

how do you even corrupt stack like that?
(said me after recently fixing a bug where I was memcpying all over it)
I have a feeling that this is a bug with copy on write
nvm
oh no the bug is still there
fr
anyway, I think I have deduced the bug to the kernel thinking that some page_ranges shouldn't be marked as CoW
even though they should be
and the reason it thinks that is because the page_range is already mapped as asymmetric CoW, as opposed to symmetric CoW which fork requires

I am moving CoW to be on the physical page layer
instead of having it in the page_range
where do you even have it now?
.
I handle CoW by looking at struct page
I am now in the process of making it use struct page
before it did something else which wasn't very nicely working with fork
like I have a flag that says that a page is anonymous (which is for pages not backed by a file/cloned from a file), so then when the process tries to read it, if refcount > 1, it is copied
I examined the faulting instruction, it was .byte 0x1f
or smth
wait it may actually be a SIMD meme

nvm

I decided to examine it at the elf's load-time
it's a bug with mapping files
it is not a bug with mapping files
I have no idea where this bug is
it's stack corruption
in the user program
how do you get to a point where userspace stack corruption causes the kernel to tripple fault? 
that is gold
idk
but now it was replaced with anotehr bug
*another
mapped 517000-917000 with protection flags 0x0, map flags 0x24
[ LOG ] User thread 7 SIGSEGV (rip 1d1be, cr2 916f78)```
how delightful
the map flags are GUARD_PAGE | NON_PAGED
so basically a stack
Core_GetCurrentThread()->context.stackBase = Mm_VirtualMemoryAlloc(ctx, nullptr, 4*1024*1024, 0, VMA_FLAGS_GUARD_PAGE|VMA_FLAGS_NON_PAGED, nullptr, nullptr);
when I omit VMA_FLAGS_NON_PAGED, it gives me SIGILL
fixed
I was forgetting to map the stack as user
so now, I get SIGILL once again
I was investigating a suspicious page fault at stack_base-0x2000
n'wut ze actuel fock
obos interpreter??
oACPI
uOBOS
dynamic stacks?
no
I think LAI has trolled me once with that 
My stacks would grow by 1 page, and it was allocating giant structs/arrays on stack
common LAI L
my solution was to give it 2GB of stack 
the location of the crash is at mlibc/options/rtld/generic/main.cpp:391
I decided to revert my changes to the VMM
and I will be redoing them
I decided I wanted to make my shutdown command a bit more flexible
only to discover that getchar() mysterious breaks on obos' mlibc
[ DEBUG ] (thread 8) syscall Sys_FdRead(0x0, 0xa7c008, 0x1000, 0x916bc0, 0x0)```
and perhaps that's why, it's make a 0x1000 byte read on stdin
which on obos would wait until it gets 0x1000 bytes on the serial port (aka stdin)
then return
static char obos_getchar()
{
char ch = 0;
syscall4(Sys_FdRead, 0, &ch, 1, NULL);
return ch;
}
this is my workaround
thats the wrong behavior though
in cooked mode its supposed to wait for the user to press enter and then expose the whole line for reading at once
I don't have TTYs yet
in raw mode its supposed to block until any data is available, and return up to N bytes from the buffer
stdin is just an open FD to /dev/COM1
I plan on changing that in the near future
I just need TTYs
yeah but on unix-likes serial ports behave like ttys
you can set /dev/ttyS0 to stdin/stdout and it should work more or less™️
we have a problem with wake from suspend
at least when done from usermode
nvm, was just a one-time thing
anyway, I am going to make the on_wake and on_suspend functions for the uart driver
done
and they do indeed work
although, the ahci driver is quite ded
u dont reinit it after resume?
I never bothered writing the necessary code in the ahci driver
to reinit it
I wonder if I could just recall into the ahci driver's entry point in it's on_wake callback
well that loses all in-flight transfers across suspend right? or do you wait for them to complete first
since I never bothered to do anything in the ahci driver in on_suspend and on_wake, all transfers would be discarded
I think it would be a good idea to wait for them to complete
in on_suspend
so what happens if one thread is doing a read syscall while u suspend at the same time
well reading from an ahci drive (directly or indirectly), causes the caller thread to block until the transfer finishes
I guess it depends on which fd was read
but at the start, the suspend syscall stops all other CPUs
until wake
this is why windows drivers have a power notification thing iirc
what does that do?
lets your drivers know that the power state is changing?
so do stuff according to that
C kinda requires it (?) (not even POSIX)
this is probably the best way
yeah, it also removes the need for logic to restart the transfers on wake from suspend
you can also just not suspend if there are physical device IOs in progress
thats probably a good cue that its not a good time to suspend
to begin with
I think this is a microkernel issue, but I don't know what to do with stopping disk drivers, while other processes might need to be swapping in code responsible for sleeping
I think a long time ago, I was using dd of=/dev/sdb if=some_large.iso, and linux allowed me to suspend while it was happening
maybe I am misremembering
yeah i guess its different if its a i JUST WANNA SUSPEND RIGHT NOWWW!!!!! thing
rather than an automatic thing
youd want a way to do that
i mean, doing dd doesnt mean there are infinite inflight requests
it just stops accepting new ones and blocks
and the driver flushes existing stuff
I'm going to test that rn
is obos's io system synchronous
just frick you man
obos is still the only hobby os to support suspend
what happened to crescent?
when you do async a good tip is that "blocking the IO" like here maps cleanly to "pending the IO"
"pend" seems to be the preferred verb to refer to enqueuing an IO for later as opposed to immediately completing it
in async io systems
which may later result in the thread blocking if it requested synchronous service but not necessarily
I ran sudo dd of=/dev/sdb if=/dev/zero oflag=sync
then suspended, linux seemed to not mind, and just resumed the operation after wake from suspend
dont think its complete resume support like obos
ah
technically, I don't have complete resume support, because on 100% of real hardware
the framebuffer dies
because bah gpu drivers
but also oberrow i went from a sync io system to async so its not a fatal condition lol
we need uDRM 
fr
it appears like ahci wants more than just PCI restored
Nobody supports S0 idle though 
to restore the HBA
to the AHCI spec I go...
yeah its pretty cool
you basically resume with literally everything reset but RAM
at a real mode vector
you might even bump into a seabios bug
can suspend be used to enter weird CPU modes? 
where they used ECAM to restore ECAM space
so nothing actually happened when they tried restoring the ECAM space
oh thats a really ancient thing to be able to do
could it be a reason they're removing it on newer hw?
nah its becase of resume latency
it just takes a while to re-init everything
s0 idle is way faster
with suspend to idle u resume where u left off
no reset vector or anything
no apparently on modern laptops theres hardware assisted s0 idle
it's more involved
well the AHCI spec mentions stuff about D states
for the controller
nothing about acpi sleep states (doubt I'll find anything about that tbh)
maybe I should just follow the instructions for putting the HBA into D3
and out of D3
why the hell do they have some controller specific methods for doing that
why not just an aml method
well considering its a pci device
but arent they identical
aka pci d state and acpi d state
at least its not Debug =
ADBG is what its normally called for when they're using their custom logging infra
least retarded bios patching
why does it just say "Arg0"
they patch the logging calls from production firmware
but they patch it in a dumb way
oh god
RetarBIOS
yeah because it was stripped
its what implements logging logic
e.g. handrolled aml serial driver i posted earlier
makes sense
what sort of stuff does it have
Makes sense
this is what the AHCI spec says on D states
Then it just means this device applies to all things behind the port
Ah ok
I have been reading linux source to figure out what I must do
on wake I must:
- go into "AHCI mode" (set HBA.GHC.AE)
- reset the controller (set HBA.GHC.HR)
- go into "AHCI mode" (set HBA.GHC.AE) again
- initialize the controller
- enable transactions
- profit
on suspend I must:
- stop further transactions from happening
- wait for any pending transactions to complete
- disable HBA interrupts (HBA.GHC.IE=0)
- profit
Looks simple enough
in the sense that I get no IRQs
and idek if any commands get submitted
I'll be tracing ahci in qemu to see what's wrong
ahci_mem_read_32 ahci(0x5639f6cb2350): mem read @ 0x0: 0xc0141f05
ahci_mem_read ahci(0x5639f6cb2350): read4 @ 0x0: 0x00000000c0141f05
ahci_mem_read_32_host ahci(0x5639f6cb2350): mem read [reg:GHC] @ 0x4: 0x80000002
ahci_mem_read_32 ahci(0x5639f6cb2350): mem read @ 0x4: 0x80000002
ahci_mem_read ahci(0x5639f6cb2350): read4 @ 0x4: 0x0000000080000002
ahci_mem_write ahci(0x5639f6cb2350): write4 @ 0x4: 0x0000000080000000
ahci_check_irq ahci(0x5639f6cb2350): check irq 0x00000000 --> 0x00000000
ahci_irq_lower ahci(0x5639f6cb2350): lower irq
ahci_mem_write_host ahci(0x5639f6cb2350) write4 [reg:GHC] @ 0x4: 0x0000000080000000```
this is the last thing I get before suspend
aka clearing HBA.GHC.IE
that's a diff of after wake
I seem to have a problem with receiving IRQs after wake
as the read of mnt/test.txt does indeed complete
ahci_start_dma ahci(0x56456259b2a0)[1]: start dma
ahci_populate_sglist ahci(0x56456259b2a0)[1]
ahci_dma_prepare_buf ahci(0x56456259b2a0)[1]: prepare buf limit=4096 prepared=4096
ahci_cmd_done ahci(0x56456259b2a0)[1]: cmd done
ahci_trigger_irq ahci(0x56456259b2a0)[1]: trigger irq +DHRS (0x00000001); irqstat: 0x00000000 --> 0x00000001; effective: 0x00000000
ahci_check_irq ahci(0x56456259b2a0): check irq 0x00000000 --> 0x00000000
ahci_irq_lower ahci(0x56456259b2a0): lower irq
this is the last thing I saw in the log
ahci_start_dma ahci(0x56456259b2a0)[1]: start dma
ahci_populate_sglist ahci(0x56456259b2a0)[1]
ahci_dma_prepare_buf ahci(0x56456259b2a0)[1]: prepare buf limit=4096 prepared=4096
ahci_cmd_done ahci(0x56456259b2a0)[1]: cmd done
ahci_trigger_irq ahci(0x56456259b2a0)[1]: trigger irq +DHRS (0x00000001); irqstat: 0x00000000 --> 0x00000001; effective: 0x00000001
ahci_check_irq ahci(0x56456259b2a0): check irq 0x00000000 --> 0x00000002
ahci_irq_raise ahci(0x56456259b2a0): raise irq```
this is the last read before suspend
ig it thinks the irq is already up?
if (s->control_regs.irqstatus &&
(s->control_regs.ghc & HOST_CTL_IRQ_EN)) {
trace_ahci_irq_raise(s);
qemu_irq_raise(s->irq);
} else {
trace_ahci_irq_lower(s);
qemu_irq_lower(s->irq);
}```
perhaps it thinks I disabled that IRQ?
considering check irq 0x00000000 --> 0x00000000
maybe its not enabled because the bios didn't initialize ahci after wake? 
at that point, HBA.GHC.IE is set
well either way, I reset the HBA
so it wouldn't matter
what the BIOS does
and this method of initialization also works in general
attach gdb to qemu
fair
oh wait I fixed it
while (hPort->tfd & 0x80 && hPort->tfd & 0x8 && CoreS_GetTimerTick() < deadline)
OBOSS_SpinlockHint();
if ((hPort->tfd & 0x88) != 0)
continue;
StartCommandEngine(hPort);
+ hPort->is = 0xffffffff;
+ hPort->ie = 0xffffffff;
+ hPort->serr = 0xffffffff;```
u probably want to do that before u start it
before the command engine is started?
yes
y
well yes
but I just woke from suspend
why the hell should I care about IRQs
from the AHCI controller
maybe it would generate some sort of an error
like spinup would fail or something
and u would just not notice that
not like I handle that in the first place
still
Nice
it's actually pretty cool to have suspend
just make the kernel do nothing for x time
then wake it
i also do that every night for 8-10 hours
hibernation is scarier bc iirc you need to write all of physical memory to disk
yeah
hibernation is boot from scratch basically
windows does save some userspace state i think
but not enough to resume everything ofc
e.g. your random hello world cli will not resume
tbh I have no idea what I'd need to save for hibernate
I know I would need to save some physical memory
but the question is what
im not even sure, for a basic os theres nothing to save and its not much different from s5
maybe dump every working set on disk or whatever
wrote a small command to dump memory usage statistics
[ DEBUG ] (thread 8) syscall Sys_FdOpenAt(0xc, 0xffffffffffffff9c, 0xbfe000, 0x9, 0x0)
[ DEBUG ] (thread 8) syscall Sys_FdOpenAt returned 0x0000000000000002
[ DEBUG ] (thread 8) syscall Sys_FdAlloc(0x0, 0x0, 0x0, 0x0, 0x0)
[ DEBUG ] (thread 8) syscall Sys_FdAlloc returned 0x000000000000000d
[ DEBUG ] (thread 8) syscall Sys_FdOpenAt(0xd, 0xffffffffffffff9c, 0xbfe000, 0x1, 0x0)
[ DEBUG ] (thread 8) syscall Sys_FdOpenAt returned 0x0000000000000002
wat da shit
(0xffffffffffffff9c should not be passed to this syscall)
[ DEBUG ] (thread 8) syscall Sys_OpenDir(0x410a3833, 0x916dcc, 0x0, 0x0, 0x0)
[ DEBUG ] (thread 8) syscall Sys_OpenDir returned 0x000000000200000c```
that's the value passed as dirfd
this is the only open dir syscall of the entire process
I think my dirent code is a bit broken
"touchate"
"ttyeout"
"whoamisum"
teeepme
its normal
yeah I saw that in the host's /usr/bin
its the same as test
you use it in shell scripts
when you do something like ```sh
if [ a = 1 ]
I see
LS_COLORS='rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.7z=01;31:*.ace=01;31:*.alz=01;31:*.apk=01;31:*.arc=01;31:*.arj=01;31:*.bz=01;31:*.bz2=01;31:*.cab=01;31:*.cpio=01;31:*.crate=01;31:*.deb=01;31:*.drpm=01;31:*.dwm=01;31:*.dz=01;31:*.ear=01;31:*.egg=01;31:*.esd=01;31:*.gz=01;31:*.jar=01;31:*.lha=01;31:*.lrz=01;31:*.lz=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.lzo=01;31:*.pyz=01;31:*.rar=01;31:*.rpm=01;31:*.rz=01;31:*.sar=01;31:*.swm=01;31:*.t7z=01;31:*.tar=01;31:*.taz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tgz=01;31:*.tlz=01;31:*.txz=01;31:*.tz=01;31:*.tzo=01;31:*.tzst=01;31:*.udeb=01;31:*.war=01;31:*.whl=01;31:*.wim=01;31:*.xz=01;31:*.z=01;31:*.zip=01;31:*.zoo=01;31:*.zst=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:*~=00;90:*#=00;90:*.bak=00;90:*.crdownload=00;90:*.dpkg-dist=00;90:*.dpkg-new=00;90:*.dpkg-old=00;90:*.dpkg-tmp=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:*.swp=00;90:*.tmp=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:';```
trying to put that into bash
crashes the kernel 
with some fuckery
I now have colorful ls
[ LIBC ] mlibc: charset::is_alnum() is not implemented for the full Unicode charset3;
[ LIBC ] In function mbrlen, file ../../repos/mlibc/options/ansi/generic/wchar.cpp:58
__ensure(!"decode_wtranscode() errors are not handled") failed37
[ LIBC ] mlibc panicked! exiting program...
I have encountered my first mlibc unimplemented
also why doesn't ls -l work on obos
@vale nymph ||I like pinging mathewnd when something fnuy happens in obos' userspace||
anyway, I am going to port gnu hello
definitely the hardest port I have ever done
lmao
obos stablest kernel here 200%
[ DEBUG ] (thread 8) syscall Sys_ExecVE(0x3a7000, 0x20947, 0xbfeff0, 0xe7fe00, 0x0)
[ DEBUG ] (thread 8) syscall Sys_ExecVE returned 0x000000000000001f
0x1f is OBOS_STATUS_PAGE_FAULT
it returns that status here
wtf
[ LIBC ] In function wprintf, file ../../repos/mlibc/options/ansi/generic/stdio.cpp:904
__ensure(Functionality is not implemented) failed
[ DEBUG ] (thread 9) syscall Sys_LibCLog returned 0x0000000000000000
[ DEBUG ] (thread 9) syscall Sys_LibCLog(0x4109bd70, 0x0, 0x0, 0x0, 0x0)
[ LIBC ] mlibc panicked! exiting program...```
why can't mlibc run gnu hello
I just fixed a bug where ls --help would crash the kernel
these bugs seem to be because of problems with physical pages being freed while they're in use
aka I'm not adding one to the refcount of some page somewhere
or I'm double unreferencing the refcount of some page somewhere
I'm porting diffutils rn
it's funny how I can use the same 3 patches for like 5 different gnu stuff
diff: standard file descriptors: No such file or directory (ENOENT)```

[ DEBUG ] (thread 9) syscall Sys_FdOpen(0xd, 0x42abcc, 0x0, 0x0, 0x0)
[ DEBUG ] (thread 9) syscall Sys_FdOpen returned 0x000000000000000b
0xb is OBOS_STATUS_NOT_FOUND
as for why it says standard file descriptors, idk
this also happens when I just do diff --help
ah I think it's trying to open /dev/stdout
and stuff like that
nay, it is trying to open /dev/full and /dev/null
I added /dev/null, /dev/full, and /dev/zero to obos
diff -u doesn't work because of missing /etc/localtime
color=always worked
obos memory usage stats
to boot into a bare bash shell
for the funnies, I will suspend obos, then suspend my computer
then when I wake up I will wake my computer from suspend, then wake obos from suspend
absolutely remarkable progress btw
Next thing I want to do is unstub my pipe sysdeps
And fix some bugs with pipes in the kernel
After that I guess I will be implementing file creation in the vfs
No stat? :nobitches:
I have stat
Your stat is broken or.smth
I remember having that on astral and it being due to stat
Maybe fstat is at issue
I'll debug it after school
I'll check that out, thanks
I don't even have sys_fstat
Unless fstat is done through sys_stat
It is
I need a syscall log to debug further
Currently compiling gcc on my laptop
this is so slow
finished building gcc
turns out I needed a newer version of meson
than debian bookworm's version
it's on 1.0.1
but meson setup --wipe's behaviour was different then
causing mlibc to error out on build
Out of all of the stat syscalls, all of them had FSFDT_PATH and succeeded
Which is weird
Funky
Indeed
So it stats a few files
All stat syscalls succeed
Wait
Error opening file window to etc/localtime
Putting a fake localtime works
After, I will probably take a detour from userspace stuff
And implement an NIC driver
And if I am brave enough, a network stack
Either qemu virt net, or an actual ethernet card
For the latter option, I was looking at the NE2000 because it sounds simple (enough)
Or a "Realtek Semiconductor Co., Ltd. RTL810xE PCI Express Fast Ethernet Controller"
Which is my laptop's NIC
I found a programming guide for the RTL8100
Idk if that is the same
I found a datasheet for the card, but no specification
The kernel module in use is r8169
Found the driver in Linux source
So I guess that's my reference
If I choose that NIC
Bruh the driver is 5k lines
If one copies enums from Linux, could they be in trouble license-wise
And implementing this seems to also get me a lot more NIC support
And the osdev wiki also has a page for it
Implementing an NIC driver will be interesting
Linked here is a specification
If I am not brave enough, I will be using lwIP probably
testing obos on real hw
and hyper is taking an awfully long time to load the initrd
nvm I am dumb
frick
obos crashes
bruh what the hell
this even happens on master
in qemu
this didn't happen 1 day ago
this didn't even happen 1 hour ago
wth
how does changing the contents of the initrd
cause a crash in the memory manager
during early boot
bruh
the physical address is zero
added more logs
if I add more logs
the behaviour changes
found out the problem
it's allegedly trying to allocate a huge page
on an unaligned boundary
fixed
it was unzeroed memory
wrong channel
OH MY BAD
the driver can now fetch the MAC address
I supose the next thing to do is "Setting up the Rx Descriptors"
done
I should use the r8169 as a timekeeper in obos 
I tried registering the IRQ
and alas, ubsan reported a type mismatch
perhaps I should've tested my MSI-X code
yoyo
I received an IRQ from the RTL8169
I specifically got a "Link change" irq
although that appears to be the only IRQ
I receive
Idk lol
I will continue working on the driver in the near future
I want to be able to receive messages
Btw this means basically that I plugged something in or plugged something out
I just committed some of the code I wrote yesterday
just IRQ stuff and NIC initialization
receiving and sending messages is what I'll do next
*packets
From what I know about networking, you send packets to MAC addresses?
And then you have a bunch of protocols that allow you to get an IP Address
ARP and DHCP I think? Idk
(these are just notes to myself, I will be researching this soon)
I think all NIC drivers will require either a new callback in the driver interface, or an ioctl, to open a MAC address to receive/send messages
open mac address?
Like
I have callbacks to write/read data
That take in an abstract handle
And that function just takes a mac address and returns an abstract handle for that MAC
address
what would that handle be used for?
.
why not just pass it directly to the read/write data fn?
I have a reason, but it's hard to explain
I'm also not at home rn
obos_status(*read_sync)(dev_desc desc, void* buf, size_t blkCount, size_t blkOffset, size_t* nBlkRead);
obos_status(*write_sync)(dev_desc desc, const void* buf, size_t blkCount, size_t blkOffset, size_t* nBlkWritten);```
I basically intend to use those functions of the driver header to communicate with the NIC
dev_desc is the abstract handle I was talking about
ah so its basically a generic read/write interface
Yea
no suspend support?
Not yet lol
It's pretty simple though
The spec tells me what I need to do
When I move the device into D3
And out of D3 back to D0
Which iirc are the only states it supports
is there some secret stuff u must do?
Nah
but
I would be able to configure the magic packet or whatever
To get the NIC to wake me from suspend
Which is pretty cool
Which I need to configure through the NIC's registers
Everything I'm talking about rn is from here
then hope that the phy state doesn't get reset when you resume from suspend lol
Chapter 9.7 is power management
if the phy state is reset then you need to do annoying revision specific configuration of the phy registers afaik (but idk if it does on suspend, maybe it doesn't)
and at least on my laptop when pxe booted its already set up enough by the firmware so you don't need to do it to start with (I am not completely sure if that's the case when not pxe booting, I don't remember if I have tried)
yep, there are Ethernet frames which have source & dest MACs and the payload itself, and in the payload usually is an IP packet which has its own structure or an ARP packet
ARP is broadcasted whenever the host doesn't know the MAC address of the destination node which it wants to send an IP packet to
DHCP allocates and reserves IP addresses in a network and other stuff
I mean a DHCP server does that
I see, thanks
this is OSI level 2-3
also DHCP is used for PXE boot
the client sends a request to the DHCP server asking for details about the TFTP server to boot from I think, like its IP address
that happens after the DHCP server replies with a free IP address that the client itself can use
Marker
yeah idk why but I am only getting link status IRQs
half of one
if not less
can't send/recv frames yet
Elixir Cross Referencer - source code of Linux v6.13-rc3: drivers/net/ethernet/realtek/r8169_main.c
found linux's init sequence
for the r8169
crazy
still is to me
tryna figure out why I don't get IRQs when a frame is sent
Elixir Cross Referencer - source code of Linux v6.13-rc3: drivers/net/ethernet/realtek/r8169_main.c
marker
https://elixir.bootlin.com/linux/v6.13-rc3/source/drivers/net/ethernet/realtek/r8169_main.c#L4579
Elixir Cross Referencer - source code of Linux v6.13-rc3: drivers/net/ethernet/realtek/r8169_main.c
your tx/rx reg offsets are the wrong way around, that's probably why. also I am not completely sure about this but I am pretty sure you should write the high part of the address registers first before low (idr why exactly but I remember having issues with the addresses getting messed up)
qwinci you're a lifesaver
I did rewrite my init code btw
to be more like linux's init sequence
So yeah, I can receive IRQs for incoming packets
Next is receiving the packets and throwing them into buffers
Then sending packets
I think I will have the r8169_device have a linked list of buffers for received data from different mac addresses
Each buffer will be ref counted
And each buffer will actually be a linked list of packets
Actually
Each buffer will look like:
struct r8169_buffer {
void* buf;
size_t capacity;
r8169_mac_handle_list opened_handles;
};```
lets gooo
howd u print it like that
// hexdump clone lol
OBOS_NO_KASAN void hexdump(void* _buff, size_t nBytes, const size_t width)
{
bool printCh = false;
uint8_t* buff = (uint8_t*)_buff;
printf(" Address: ");
for(uint8_t i = 0; i < ((uint8_t)width) + 1; i++)
printf("%02x ", i);
printf("\n%016lx: ", buff);
for (size_t i = 0, chI = 0; i < nBytes; i++, chI++)
{
if (printCh)
{
char ch = buff[i];
switch (ch)
{
case '\n':
case '\t':
case '\r':
case '\b':
case '\a':
case '\0':
{
ch = '.';
break;
}
default:
break;
}
printf("%c", ch);
}
else
printf("%02x ", buff[i]);
if (chI == (size_t)(width + (!(i < (width + 1)) || printCh)))
{
chI = 0;
if (!printCh)
i -= (width + 1);
else
printf(" |\n%016lx: ", &buff[i + 1]);
printCh = !printCh;
if (printCh)
printf("\t| ");
}
}
printf("\n");
}```
I no have phd
well i wouldnt have thought of this code
I thought of this code about a year ago maybe
physical address of the NIC
physical address of the nic?
yes
in what type of addressing space
so you have ip address
mhm
right
MAC address is on a more physical scale
found this in the first search result
kk
tomorrow, god willing, I will start writing transmitting code
for the r8169 driver
after that I probably will implement suspend capabilities
let sgooo
am I right to assume that implementing UDP isn't that hard
yeah its like 200 lines depending on the impl ofc
After that, I might start making a network stack
Or porting one
you can also make netlog which might be useful to debug things like suspend if you have no serial or graphical output (assuming that the network card resume works ofc)
I have one in my kernel (though its over tcp) but I can't say I really ever used it outside of testing that it works
but at least its there, it might come in useful if I get back to doing gpu stuff
though a gdb server would likely be more useful for that but its more work with proper stepping/interruption and stuff
I already have a gdbstub in my kernel
Although it stopped compiling a while ago
So I need to fix the compiler errors
have you also implemented interruption support in it?
In my gdbstub?
yes
Kinda, it needs a hook in the driver
To detect when gdb sends an interrupt signal
Gtg now
nice
Yeah udp is super simple
yeah
Yeah I expected UDP to be simple when I looked at the format in wireshark
for printCh i would instead use isprint from ctype
my one is better
uses more colors and more utf8
That's cool
yes
currently implementing tx support in my r8169 driver
internally, it will be done by generating a frame, adding it to tx_frames, which schedules the frame to be sent in the near future
with the near future probably being when the tx_frames list exceeds a certain threshold
or a manual call to r8169_queue_transmition
which takes all frames in that list, then puts them in the Tx_Set of the driver
or TxH_Set for high-priority frames
CHAT
r8169_frame frame = {};
r8169_frame_generate(&Devices[0], &frame, "Hello from obos' NIC driver!!\n", 31, FRAME_PURPOSE_TX);
r8169_buffer_add_frame(&Devices[0].tx_buffer, &frame);
r8169_queue_transmition(&Devices[0], true);```
so obos' r8169 driver can now transmit data over the network
or well, send frames on the network
what I am going to do next:
- implement the required callbacks in the driver
- start a network stack?
also, every NIC driver will require an ioctl called GET_MAC_ADDRESS
which does one thing: returns the mac address of an NIC controller by the driver
I have implemented all callbacks for the driver
I need to now:
- handle TX_OK interrupts
- actually register the network interfaces in the kernel
I did that
now, I'll implement on_suspend and on_wake callbacks
void on_wake()
{
for (size_t i = 0; i < nDevices; i++)
{
r8169_reset(&Devices[i]);
Devices[i].suspended = false;
}
}
void on_suspend()
{
for (size_t i = 0; i < nDevices; i++)
Devices[i].suspended = true;
}```
theroetically, it is as simple as that
I already handle suspended being true in r8169_reset
or maybe it's even simpler
Elixir Cross Referencer - source code of Linux v6.13-rc3: drivers/net/ethernet/realtek/r8169_main.c
I'll see, as there's only one way to find out
I am unable to send a message on ethernet after wake from suspend
fd nic = {};
Vfs_FdOpen(&nic, "/dev/r8169-eth0", FD_OFLAGS_READ|FD_OFLAGS_WRITE);
Vfs_FdWrite(&nic, "before suspend", 15, 0);
OBOS_Suspend();
Vfs_FdWrite(&nic, "after suspend", 14, 0);```
the 2nd write is reached
but I never get anything on wireshark
it probably has something to do with phy or whatever
the bug actually turned out to be because the NIC driver doesn't seem like it wants to send the 2nd frame
when I removed the 1st write to the NIC, I got the packet after suspend
it could also be the host is dropping the 2nd packet
obos making more progress then nyaux fr

I think I fixed the bug
but wireshark doesn't show me the 2nd packet
so it's rather:
- the bug isn't fixed
- my pc drops the 2nd packet
tomorrow I will make review all this code
make sure I didn't do anything dumb
then start a network stack probably
Implements an R8169 driver for obos. Currently supports transmission and reception.
Does Linux even drop packets it doesn't recognize as valid
It didn't drop my first packet
But the 2nd packet is never seen in wireshark, despite me getting a TX_OK irq
did you look at the host traffic or did you make qemu generate the dump?
Qemu doesn't have an r8169 device
So I am exclusively testing this on real gw
*hw
ah
there is rtl8139 in qemu which is basically the same (though it uses a legacy pci irq and you need to setup some extra bits)
I can look into that later
It's interesting that I get TX OK for both packets
But I only see the first
in wireshark
I guess I will have to wait until I write code to actually send valid packets
To see if the host is dropping my packets
Also is there a way to get my pc to sort of act as a router on the ethernet adapter
To make it so I can test my network stack on real hw without connecting obos' unstable code to my main network
yes you can enable packet forwarding + iptables or whatever
yes
ip addr add dev <interface> <ip>/24
then sysctl -w net.ipv4.ip_forward=1
then ```
cat >rules.nft <<EOF
flush ruleset
table ip force-masquarade {
chain masquerade-config {
type nat hook postrouting priority srcnat; policy accept;
iifname "<interface>" masquerade
}
}
EOF
sudo nft -f rules.nft
alternatively, depending on how expensive your router is, you might get away with just applying 802.1q tags onto the incoming traffic on linux, then relaying them to the proper router, and then instructing it to give it its own subnet
one day I'll learn iptables..
or (again, if your router is fancy enough) tell your router to consider the port you use for obos to be on its own subnet
learn nftables (the modern replacement) instead 
i was just typing nft on my Ubuntu terminal to see what it is lol
are they easier?
to understand
So this causes it to forward packets on the ethernet adapter to my router?
imo yes
that tells it to NAT the packets
if you just want to blindly forward ethernet, you can just do ip link set dev <obos interface> master <main interface>
I see
I wonder if I can crash any devices by sending an extremely bogus packet on the network from obos
Like if I just send a bunch of 0xff on the network
xD
likely no, the router should usually discard invalid packets afaik (and even if they would reach other devices they shouldn't die because of it unless there is some kind of bug)
https://lwn.net/Articles/928581/
At one point, Linux would send some broken packets that took down all of the Sun machines on the network.
💀
Both the broken packets and the crashing because of the broken packets
all internet will break once nyaux gets onto it
Networking is pretty exciting
Yes
What NICs does astral support?
Virtio net but adding a new one is trivial
Ok
real
fixed
can u put the number 47
in every source file
somewhere

anyways
@flint idol whats next for obos
Network stack
after that?
ah okioki
merged the driver
https://github.com/OBOS-dev/obos/commit/8939da42a79c675b2e5e533a8065211273d31744
just found this commit lol
- xor rax, rax
+ xor eax, eax```
- xor rax, rax
+ xor eax, eax
+ ; 42
...
+ 48,000 additions, 200 deletions
add 42
?
42
Go away

b-b-b--but
anyways
since u have a nic driver
whats next for the network stack now
impling tcp? or smt
idk how networking works
@flint idol ?
ah i see okay
okay i seee
then UDP is probably on top of those
tcp when 
TCP is also on top of those layers
or maybe it goes
UDP->IP->NIC
ARP->IP->NIC
idk, haven't read the specs in detail
I am writing down structs for IPv4
thanks
np
I'm reading a spec from 1981 lmao
lmao
it is updated by RFC1349
a quick glance tells me that has to do with ICMP
nvm
https://datatracker.ietf.org/doc/html/rfc792
ICMP is in RFC792
why is udp easy as shit
fd nic = {};
Vfs_FdOpen(&nic, "/dev/r8169-eth0", FD_OFLAGS_READ|FD_OFLAGS_WRITE);
udp_header* udp_hdr = nullptr;
OBOS_ENSURE(obos_is_success(Net_FormatUDPPacket(&udp_hdr, "test1", 5, 0x8000, 0x8000)));
ip_header* ip_hdr = nullptr;
ip_addr src = { .comp1=192,.comp2=168,.comp3=2,.comp4=255 };
ip_addr dest = { .addr=0xffffffff };
OBOS_ENSURE(obos_is_success(Net_FormatIPv4Packet(&ip_hdr, udp_hdr, sizeof(*udp_hdr)+5, IPv4_PRECEDENCE_ROUTINE, &src, &dest, 60, 0x11, 0, false)));
ethernet2_header* eth_hdr = nullptr;
mac_address destm = { 0x98,0xee,0xcb,0x39,0xd9,0x37 };
mac_address srcm = {};
Vfs_FdIoctl(&nic, IOCTL_ETHERNET_INTERFACE_MAC_REQUEST, &srcm);
size_t nToSend = 0;
OBOS_ENSURE(obos_is_success(Net_FormatEthernet2Packet(ð_hdr, ip_hdr, be16_to_host(ip_hdr->packet_length), &destm, &srcm, ETHERNET2_TYPE_IPv4, &nToSend)));
OBOS_Debug("%d\n", nToSend);
OBOS_ENSURE(obos_is_success(Vfs_FdWrite(&nic, eth_hdr, nToSend, 0)));```
the bug isn't fixed
was the answer to that question
for when I wake up:
- fix bug with r8169 driver
- implement reception of packets in the network stack
- implement ARP probably
- implement some other shit idk
Oh I should revive the m68k port of obos soon
dhcp
Ok
hey @vale nymph you don't mind if I take this do you 
https://github.com/Mathewnd/Astral/blob/8c42826cf1933dbb90c8cd6b1ad58b197227f6c3/kernel-src/io/net/ipv4.c#L86-L103
bruh 
I passed the wrong size
(whoops)
thanks mathewnd 
0000 48 65 6c 6c 6f 20 66 72 6f 6d 20 4f 42 4f 53 27 Hello from OBOS'
0010 20 6e 65 74 77 6f 72 6b 20 73 74 61 63 6b 21 network stack!
anyway there's a UDP packet being snet
but now with a proper checksum value (oops)
anyway gtg
nice
the bug has to do with any Tx descriptor after the first
the NIC kind of just.. ignores them
lesgooo
congrats
obos on google soon
i meant obos with a web browser loading a google web page
but ykyk
bru
maybe I should poll the bits in TxPoll
before sending a packet
wait nvm
I should be setting the NIC_OWN bit
after everything
https://elixir.bootlin.com/linux/v6.13-rc3/source/drivers/net/ethernet/realtek/r8169_main.c#L646
just found a problem in a comment
Elixir Cross Referencer - source code of Linux v6.13-rc3: drivers/net/ethernet/realtek/r8169_main.c
in linux
I fixed the bug
I just discovered a bug in OBOS
specifically regarding uacpi_kernel_schedule_work
it does an allocation
right
actually nvm
it's a trivial bug to fix
it was using the paged pool allocator
in an interrupt context
which caused a page fault
at an IRQL > DISPATCH
crashing the kernel
why is @thick jolt invading my PC 
if (memcmp(buff, "OBOS_Shutdown", 13))
OBOS_Shutdown();```
to facillitate debugging, I added this to my NIC driver
and wrote a dumb C program that sends raw bytes onto an interface
specifically in the receive IRQ
I want to make my network stack asynchronous
but I don't feel like changing my IO system right now
so what I'll do is add a required function for NIC drivers that registers a callback which is called when there is data ready
obos_status(*set_data_ready_cb)(void* vn, void(*cb)(void* userdata, void* vn), void* userdata);
lmao
I just love it when it says I have two users signed in on my PC when only one is signed in
yeah that's me
void data_ready(void* userdata, void* vn_, size_t bytes_ready)
{
OBOS_UNUSED(userdata);
const char* interface = nullptr;
vnode* vn = vn_;
vn->un.device->driver->header.ftable.query_user_readable_name(vn->desc, &interface);
OBOS_Debug("%ld bytes ready on interface %s\n", bytes_ready, interface);
void* buffer = OBOS_KernelAllocator->Allocate(OBOS_KernelAllocator, bytes_ready, nullptr);
size_t read = 0, bytes_left = bytes_ready;
size_t i = 0;
while (bytes_left)
{
vn->un.device->driver->header.ftable.read_sync(vn->desc, (void*)((uintptr_t)buffer+(bytes_ready-bytes_left)), bytes_left, 0, &read);
OBOS_Debug("Packet %lu\n", i);
hexdump((void*)((uintptr_t)buffer+(bytes_ready-bytes_left)), read, 15);
bytes_left -= read;
i++;
}
}```
I made this test data ready callback
anyway
for when I wake up probably:
- IP reception
- UDP reception
- ICMP probably
when I'm done that:
- ARP
- DHCP
I just get some stupid blockchain projects, some revolt client, mc anticheat, some microsoft library for jwt tokens, some cancer rna analyzing thing and if I don't append github then I get bicycles and all kinds of stores that sell them
with crescent os its within the first page at least but it still has bicycle related links and some weird thing too
for me, it's the third result
obos can now respond to ARP requests
funny how I can see OBOS leaking memory
using arping
the ping started at ~6 ms
it rose to 16 ms after ~400 pings
it's angy at you because it know that obos will overtake it in 2025
oh
