#EvalynOS

1 messages Β· Page 32 of 1

clear bison
#

I'm afraid no FPU is gonna be horrible

#

Or maybe not

hazy saddle
#

i mean? the kernel dosnt need FPU

clear bison
#

It can do integer ops pretty fast

hazy saddle
#

an X server is about the most heavy thing id ever wana attempt

viral bison
#

THANK GOD

hazy saddle
#

the horrible part may be emulating the fpu

viral bison
hazy saddle
#

not how that works

#

it triggers an exception

viral bison
#

ah, f

clear bison
viral bison
#

this was the older one

#

you cannot even search

hazy saddle
#

The MC68LC060 is a derivative of the MC68060. The MC68LC060 has the same execution unit and MMU as the MC68060, but has no FPU. The MC68LC060 is 100% pin compatible with the MC68060. Disregard all information concerning the FPU when reading this manual. The following difference exists between the MC68LC060 and the MC68060

  • The MC68LC060 does not contain an FPU. When floating-point instructions are encountered, a floating-point disabled exception is taken.
  • Bits 31–16 of the processor configuration register contain 0000010000110001, identifying the device as an MC68LC/EC060.
viral bison
#

wish me luck πŸ’€

hazy saddle
#

ah they do provide the code needed for this though aparently

viral bison
#

@hazy saddle can't you find this package?

hazy saddle
#

it stopped being made in 98 πŸ’€

viral bison
#

fuuuuu

#

anyway, time to go to sleep, tmr i'll make the footprint πŸ’€

hazy saddle
#

wtf is this

#

the repo is an utter mess

#

fpsp.s dosnt even exist

#

wtf

#

πŸ’€

#

is the 68060 even worth it over the 68040 atp??

#

@urban mulch

#

look at ts

#

😭

tawny wren
#

m68k port?

hazy saddle
#

But the FPU emulation code is apparently annoying as fuck to get

#

Since I'm not paying 300bucks for an m68k

tawny wren
hazy saddle
#

.

#

honestly

#

could it be cheaper to just FPGA the m68k seperate from the sotubridge fpga πŸ₯€

tawny wren
hazy saddle
#

this also exists

#

😭

#

aparently

#

and this

#

I do wonder if there is anything still fucking weird / old that's 32bit with an MMU/fpu

#

that could be workable for this

#

And not x86

clear bison
#

πŸ‘€

hazy saddle
#

Not to mention licencing

clear bison
#

There's no licencing problems if you change the code just enough and nobody else knows trl

hazy saddle
#

😭

clear bison
#

I think fpsp.asm is just like a weird decompiled/obfuscated version of 68060_lib/fpuemu/*

#

As that seems much more readable

#

Omg i found the counterpart of mpreg, fpreg

hazy saddle
hazy saddle
#

I wonder how hard porting this shit is

#

If we were to do the FPU less 060

#

Actually

#

Genuinely

#

Binder may have a good option on this

#

@spark fern since we are potentially building a m68k based machine should we go with the 68040 with MMU and FPU of go with the 68060 with the MMU but no FPU and use the software emulation libraries for it.
The goal is to hopefully port my kernel and run an X server or atleast bash and some basic tools.
The SBC would be using an FPGA to emulate alot of the non CPU core things like to give it 512mb of ddr2 along with UARTS display etc etc

clear bison
#

I think binder got stuck on the 6502 era of commodore

#

He didn't get to touch the Amiga

#

So no m68k knowledge obviously

hazy saddle
#

Or atleast talked about it somewhat

hazy saddle
#

also this is cursed and bad practise af

#

but iirc since i mamage the HHDM very cleanly

#

eg only ram is mapped with no overage

#

i could use the HHDM for MMIO

#

only issue is MMIO via HHDM would need a seperate helper due to LA64 using DMW's w/ the UC HHDM at a seperate offset

hazy saddle
#

i could get the PLEN and reserve all of that

#

but thats wasteful

#

i could also just have a seperate allocator for managing MMIO regions in the kernel that gets a large virtual allocation and bump allocates it and refills when needed

#

since virtual memory (on 64bit arches) is free, and saving some memory on vmem segments could be good

magic saffron
#

I know you're interested in the hardware stuff, but qemu has an m68k system to get you started. I believe it's system emulations are reasonably accurate to the real things too.

hazy saddle
magic saffron
#

fair enough

hazy saddle
#

I'm probably gonna try and get scheduling to work tomorrow

#

If I feel good enough

#

I barely could play any games today 😭

#

I am making some progress on the iotel stuff thankfully

magic saffron
#

you could be the fourth person in this server (afaik, probably not realistically) to port their kernel to m68k πŸ˜›

hazy saddle
# hazy saddle I am making some progress on the iotel stuff thankfully

Hello fedi, I haven't posted here in a while but i have something interesting.
I have been investigating something that is utterly confusing and I need others to help find samples of this happening.

A certain set of intel CPUs apparently report the CPUID vendor string as GenuineIotel with the n in intel having a singular bit flipped making it an o.

This is listed on Wikipedia leading to one singular report of this and when contacting that person directly on bsky they were unable to provide more information on this. Along with me attempting to send a patch on the LKML to handle this and also maybe see if other have more info on this.

The LKML thread is a little inactive with me waiting on further instructions on how to handle it they way they wanted (halting/panicking along with a flag to bypass)

I have found a bunch of sparse reports over online separate from the one mainly cited but they are incomplete.

I need help from other nerds online to try and see if they have proce…

#

Sent this to a bunch of friends

#

If it gets enough traction I can hopefully get some test subjects

magic saffron
#

good luck, it's an interesting thread to follow

hazy saddle
#

Probably the worst part of the m68k thing on real hardware will be larping as a 90s PC bios

#

😭

#

Since il have to make firmware

hazy saddle
magic saffron
#

yeah

hazy saddle
#

Ty ;3

#

Hopefully I get a reply and I can make a proper v2 patch

magic saffron
#

give it AML

hazy saddle
magic saffron
#

make it boot limine kernels natively

hazy saddle
#

But I did want to go on a somewhat UEFIish direction

#

The main crazy part will be it having a bunch of modern peripherals

#

Like a 16550 UART

#

SD cards

#

Etc

magic saffron
#

sounds like a quirky machine

hazy saddle
#

Yup

#

Will be quite funny

#

But it will also try and be somewhat simple for the kernel to use

#

Also 512mb of ddr2 UltraSPARC

#

An FPGA will be the south bridge and handle alot of this stuff

magic saffron
hazy saddle
#

i was thinking of just passing a dtbβ„’

magic saffron
#

yeah that also works

hazy saddle
#

Because this would keep the m68k it's self under 100 bucks to buy

magic saffron
#

I think summoning bindage was a big move meme

hazy saddle
#

And the other parts would maybe be 300 in total

#

Meaning it could be somewhat reasonable for people to built

#

😭

#

verses 250 bucks for the cpu

magic saffron
hazy saddle
#

I mean 512m is about the limit for a direct map

magic saffron
#

or if some of it is being dedicated as vram I guess that makes sense

hazy saddle
#

And I actually will try and avoid a direct map

#

Il use recursive page tables for this arches paging routines

#

And other uses of HHDM will get split into using temp mappings on 32bit arches

magic saffron
hazy saddle
#

Ofc

#

It is many people working on it

#

But il definitely be putting out all the Firmware files etc

#

Oh and the protocol will load the kernel into the upper half etc with paging ofc

#

Il probably make it give page tables that

  • map you in the upper 256m
  • gives 512m of direct map
  • has a recursive set of page tables to use aswell
#

Which should fit in the upper 1g?

hazy saddle
#

And to try and make it run as much software as possible and be usable

#

Eg hopefully an X server and a light DE as the final goal

#

it would also be funny to have a custom IRQ controller on here but also have a standard PIT timer that can do the CPU interrupt timer and the pc speaker trl

#

goal is to also make a cursed board in some ways

magic saffron
magic saffron
hazy saddle
#

With the FPU falling behind still

magic saffron
#

hmm I see

hazy saddle
#

The firmware I also want to have a menu for configuring boot and other things because it could be cool to get Linux working + a dual boot

#

Since the firmware would handle ext2 natively

hazy saddle
#

Which is probably useful to do tbh on such old of a CPU

hazy saddle
#

If anyone wishes to test if they have the ioteld cpu they can simply run this command on Linux
awk -F': *' '/^vendor_id/ {print $2}' /proc/cpuinfo | sort | uniq -c | awk '{print $2 ": " $1}'

viral bison
#

bc they have different pinouts

spark fern
#

my first 486 did all soft float and was quite usable

spark fern
#

is this a full super scalar with multiple pipelines?

#

is the fpga going to run so fast that it really doesn't matter?

viral bison
#

yes

hazy saddle
hazy saddle
spark fern
#

reference... I ran linux + xwindows in a 25mhz 486 with no fpu

hazy saddle
spark fern
#

I wouldn't imagine they would be hard to find

hazy saddle
#

it was mentioned in the manual, was annoying to find anythibg beyond that

spark fern
#

could probably also rip them out of linux/bsd

hazy saddle
#

true, hopefuly the licencing on thoes is fine

#

eded up missing some things when doing la64 exception names abd decancering them

viral bison
#

thankfully this pinout doesn't see so bad

hazy saddle
#

some of the names πŸ₯€

viral bison
#

@spark fern do you think this is mirrored?

#

the pinout i mean

spark fern
#

I wouldn't be surprised if the LLM couldn't whip up vaguely decent soft float routines

hazy saddle
#

thank you for the help btw

hazy saddle
spark fern
#

pure C soft float

viral bison
#

bruh it's flipped

#

why

#

WHY do they do that

spark fern
#

maybe so you can 'look' at the pint while it's in the socket?

viral bison
#

normally in datasheets they show from the top only

#

it makes it less confusing

spark fern
#

I haven't looked at hardware specs much

viral bison
#

oh god i hate how they placed the data pins

#

thanfully we are using an fpga

#

so we can mix and match the data lines

spark fern
#

have you picked the fpga yet?

viral bison
#

yes

#

this one

#

comes with DDR3 / LPDDR3 / LPDDR2 driver block too

#

and LVDS

#

and SPI

spark fern
#

not familiar with that line

viral bison
#

chose it bc of price, amount of cells, features and gpios

spark fern
#

31k LE

viral bison
#

ye

#

it was the highest in this price range

spark fern
#

you writing your own HDL impl?

viral bison
#

we have to, lol

spark fern
#

why?

viral bison
#

well, i don't existing hdl code exists for this

spark fern
#

can at least start with the tg68k

#

or wf68k30

viral bison
#

?

spark fern
#

there was talk about an port of apollo...

viral bison
#

we using a real cpu

#

not fpga that acts as an m68k

spark fern
#

OH

#

kk, that is different

hazy saddle
spark fern
viral bison
#

footprint done

#

gg

opaque marten
#

no.

hazy saddle
opaque marten
wanton grove
#

yes

#

i386

#

!

clear bison
#

πŸ’€ πŸ’€ πŸ’€

hazy saddle
viral bison
#

oh god so many unknown pins

#

@hazy saddle what do we use the UPA pins for?

hazy saddle
#

Idk 😭

#

What do they do

viral bison
#

User-defined signals, controlled by the corresponding user attribute bits from the
address translation entry.

hazy saddle
#

I guess extra things for page tables?

viral bison
#

i'm still reading the docs, there are 20 pins i have never heard of πŸ’€

#

i'm gonna ask clanker

hazy saddle
#

These could be used to overlay MMIO ontop of normal ram right?

#

And translate differently on the FPGA

viral bison
#

prob

hazy saddle
#

Eg get more than 4gb of physical address space

#

Probably should wire them into the FPGA for later

viral bison
#

i'll wire them in anyway

#

we have more than enough pins

hazy saddle
#

Could also be used to map a page as returning hard wired zeros?

#

Or other useful things

viral bison
#

πŸ’€

#

why does this even exist

#

oh, prob bc you can reset from sw?

hazy saddle
#

What's the max amount of ram we could attach to the FPGA?

hazy saddle
viral bison
hazy saddle
#

This is for the 68060 one right

viral bison
#

8Gbit

viral bison
#

so 1Gbyte of ram

#

we are limited to only 512MB with ddr2

#

but 1GB with ddr3 / lpddr3

#

and i think the best method would be to just solder the ddr3 IC

hazy saddle
#

Well we are using ddr2 anyways

#

Since ddr3 is too hard to wire right

#

And with only 512mb of ddr2,

#

We won't need UPAs for that specifically

viral bison
#

ddr3 and ddr2 are very similar

#
  • if i just solder the ddr3 chip it should be easier
#

oh wait

#

we cannot even use the full sodimm module πŸ’€

#

we can only use a quarter

#

sticks use a 64 bit bus

#

we only have 16 bits

hazy saddle
#

Then we need to make our own custom sockets

#

And custom sticks

viral bison
#

we could only use 2 sticks

#

or well

#

ICs srry

#

and they are 30 bucks each (if we don't steal them from sticks)

hazy saddle
#

Can't we fit some ICs on a tiny board with a connector

viral bison
#

i mean, we can just connect the whole sodimm and only use a quarter XD

hazy saddle
#

But that would what a 2gb sodimm?

viral bison
#

we can use any size i think

#

if we go like that

#

tho we would only be able to use /4 of it

hazy saddle
#

But then we can only use a quarter?

viral bison
#

ye

hazy saddle
#

We would want 512m ram

#

We would also loose bandwith I presume

viral bison
#

not really

hazy saddle
#

We can just use a stick like this?

viral bison
#

ye

#

even ddr3l is possible tbh

#

it's only 200 mhz faster

hazy saddle
#

I'd ask Emma 😭

viral bison
#

@urban mulch

#

lol

#

tho we could reach 2GB of ram if we spend 60 bucks for the single ICs

#

or steal some from sticks

#

and it should be safer than using sodimm

hazy saddle
#

We don't want 2gb ram though

#

Just 512m

viral bison
#

why not use the full 2gb tho?

#

if we can get it

hazy saddle
#

You said only a quarter could be used

#

Also thoes extra PTE pins may not work nice with the cache

viral bison
#

extra PTE pins?

#

oh the UPAs?

hazy saddle
#

Yes

viral bison
#

x2 of these and we get 512MB

#

also, we can upgrade these later bc most ram ICs are pin to pin compatible

hazy saddle
viral bison
hazy saddle
#

How many pins do these need

viral bison
hazy saddle
viral bison
#

VDDs and VSSs can be grouped

#

NCs mean no connected

hazy saddle
#

Use that connector and form factor

#

But for socket able memory

viral bison
#

prob yes

#

but it is more costly

#

like prob around 30 bucks for the extra pieces and pcb

hazy saddle
#

It's both extra cursed and extra cool because it makes upgrading easier

#

And you can fit some extra things on the modules

#

To identity how much ram it has

#

The firmware could poke a register on the FPGA to check the ram size based on the modules reporting

#

Make it a little more PC like

viral bison
#

if you like the extra price sure

hazy saddle
#

Idk

viral bison
#

@hazy saddle for interrupts, we only have this in the cpu

#

should we get a PIC if it can be connected? trl

viral bison
#

true trl

#

fuck it, every pin is going to that fpga

hazy saddle
#

I wanted to actually make a custom really simple IRQ controller

#

But use the PIT timer

viral bison
#

tho, would we have enougn LEs to actually make all that hardware, lol

sterile plank
#

whatchu guys doing

viral bison
hazy saddle
viral bison
sterile plank
#

which is..

viral bison
viral bison
sterile plank
sterile plank
#

good luck

viral bison
sterile plank
viral bison
#

also @hazy saddle we gonna get usbc power delivery too XD

#

bc it might be the only way we can get that much power via usb

#

i could add a battery module too if you want trl

hazy saddle
#

Maybe for a v2

#

V1 should try and be cheap

viral bison
#

batteries are expensive tho

hazy saddle
#

Let's leave that for making it a handheld πŸ’€

viral bison
#

XD

viral bison
#

we making it way more cursed

hazy saddle
#

I need to work on actual kernel shit anyways now

#

Tomorrow if I don't feel poopy 😭

#

Maybe we could move the talk about this bullshit elsewhere

viral bison
hazy saddle
#

It's not project specific though

viral bison
#

it kinda is tho

hazy saddle
#

Atleast at this point anyways

#

We can just use osdev3

hazy saddle
#

so i cant do sched rn

#

well

#

i can make the code

#

and maybe test it ig

#

but it wont be able to preempt

hazy saddle
#

Me when loongarch64 isn't bullshit and has a CPUID to tell you if the timer is constant and to tell you the frequency of it and setting up the timer to give interrupts isn't annoying shit

#

I mean it can still be non constant ig πŸ’€

#

But I think I could just panic on that case?

#

Since there wouldn't be any other way to reliably get time

#

For riscv I'm gonna need more dtb shit

#

😭

#

I'm also going to have to modify CPUlocal to store a self pointer (to the fixed region)

#

That way you can lock a lock etc

hazy saddle
#

Okay yeah I'm going to need to add helpers to get pointers that go into CPU local to handle spinlocks there

hazy saddle
#

lmao

#

another common gcc L

#

😭

#

this only compiles via clang

#

Oh yeah I'm also gonna skip having it in the fixed CPU local

#

Interrupts are disabled here anyways

#

And for getting the current thread generically il just implement an assembly function anyways

#

I only really need the fixed CPU local for things touched by assembly that also has no temps

#

Which is only the current thread pointer

#

And not the spinlock

hazy saddle
#

Il still need the pointer helpers though etc

hazy saddle
#

Ended up reworking alot more of the CPU local

#

But scheduling works

#

And it should just work with preemptionβ„’

#

Oh also it works for all the architectures

#

@tender mantle

#

If you want to steal the scheduling code too

#

You can

#

For your riscv port

hazy saddle
# hazy saddle

Also scheduling is init properly before any memory management because of hyena sky

#

So it will have a thread marked active etc

hazy saddle
#

Honestly I still have more refactors I wana do on this

#

Because of some half stupid decisions

#

I'm probably gonna push this

#

Do some GDT/IDT refactors

#

And some other various things

wraith star
#

gdt refactor ok?

hazy saddle
#

So the GDT stuff can working On APs

hazy saddle
#

Main thing is having architecture spesisifc fixed CPU local is kind of stupid

#

And at this point only serves to save 16 bytes on x86

#

At the cost of some BULLSHIT with the abstractions

#

Still partly more expensive for things outside of the fixed region

#

But that's fine for shit like the GDT

#

Or like things that aren't too much needed to be ultra fast

#

if I properly make the fixed region generic other shit would be better

signal plaza
hazy saddle
#

Iirc @urban vale advocates for this alot. And I think I did it right?

signal plaza
#

I'm starting to get back into osdev again so I'll certainly reference the code trl

#

Although I hope to get through task switching without needing to steal anything...

hazy saddle
#

It's really simple

#

😭

#

I would just give it to you

signal plaza
#

Lol I'm pretty sure that the reason Serpyntyne wouldn't task switch was something dumb

#

Ideally it'll disappear when I write task switching code again

hazy saddle
#
; rdi = pointer to store old stack pointer at
; rsi = new stack pointer
arch_thread_switch:
    push rbx
    push rbp
    push r12
    push r13
    push r14
    push r15

    ; swap stacks
    mov [rdi], rsp
    mov rsp, rsi

    ; unlock the scheduler spinlock
    mov dword [gs:rel fixed_cpu_local + 24], 0

    ; enable interrupts
    sti

    pop r15
    pop r14
    pop r13
    pop r12
    pop rbp
    pop rbx
    ret
#

plus this to setup a fresh stack to be able to get switched to

wide mica
#

big scheduler lock πŸ’€

hazy saddle
#

and its RR

wide mica
#

why do you have it

#

why not a thread lock

#

actually

#

why does the queue lock protect the thread state

hazy saddle
#

The amount of time the thing is locked is so tiny it dosnt really matter?

wide mica
#

I'm not sure this is right

signal plaza
hazy saddle
#

And it was working

#

And it's what solved several race conditions

signal plaza
#

I suspect I was setting them to have the same stack or something

hazy saddle
#

Wtf

hazy saddle
wide mica
#

typically to protect a thread switching out from its own stack u have a per thread lock

#

that you hold when you try to run into

#

sec

#
CPU0: thread goes to sleep
CPU0: queue lock is held
CPU0: switch():
CPU0: do whatever with regs

CPU1: thread wakes back up on another CPU
CPU1: acquire queue lock
CPU1: switch():
CPU1: do whatever with regs
CPU0: queue lock released
OOPS! CPU0 and CPU1 race on thread state
#

u cant do this unless you pin a thread to a CPU I believe

hazy saddle
#

I'm think sure that lock was mostly to prevent it interrupting it's self which was something that was helping? Also since the queue is per CPU aswell this shouldn't happen?

#

When would you want to lock and unlock the thread specifically in this sequence though

#

On a switch to and then unlock after you swap stacks?

wide mica
#

you take a thread lock when switching to it, you release it when switching from it

wraith star
#

^

#

you release the lock after you have switched stacks

hazy saddle
#

Ah so unlock prev lock next

#

im gonna take a nap then change that

wraith star
#

hmm, actually

#

does a thread lock even make sense

#

i think it will deadlock

wide mica
#

why

hazy saddle
#

yeah i was thinking of that too

wraith star
#

isnt it better to lock the runqueue

#

if you want to lock, lock the information regarding blocking

wide mica
#

can you explain the deadlock

hazy saddle
wide mica
#

I have per-thread locks and all the blocking paths dont require some kind of dispatcher/runqueue lock

#

because I do the windows 7 thing where you have transition states for a thread

wraith star
#

let me check what i have done and think a little bit

#

because i genuinely forgot

#

hmm, i use two locks: one lock for the runqueue, and another lock for the task state/blocking information (aka, which wait queue i am on)

#

when scheduling, lock the runqueue, pick next task, switch to it, unlock runqueue
the task lock is taken when changing the state (waking up, blocking), and when it is taken, shortly the runqueue lock is taken too for enqueuing or dequeuing

#

what do u do @wide mica ?

wide mica
wraith star
#

how do you get the next thread?

wide mica
wraith star
#

okay, but i suppose those threads are in a list or smth

wide mica
#

yeah, the clock() function is a DPC enqueued on timeslice timer expiry which sets the cpu.next_thread field, then after DPC dispatch I check whether or not cpu.next_thread is null, if it's not then I switch to it

#

when selecting a thread to run I hold both the current thread's lock and the queue lock

#

the next thread thing is very clever, I stole it from MINTIA which stole it from NT

hazy saddle
#

I mean mine should work since no two things should have access to the thread struct.
It will only be in one CPU run que at a time and it won't be eligible for changing CPUs if it's in a running state. And the lock is only released when it's all proper?

wraith star
#

i just dont understand why you take task locks for scheduling

#

unless your kernel is capable of letting tasks deque other tasks which are possibly in execution

wide mica
#

also the thread priority is updated under its lock

#

so i have to take it

hazy saddle
#

Oh yeah I do need a lock

#

I'm just too tired to know the proper spots to take and release them without deadlocking bullshit

wraith star
#

why does recalculating the thread priority require you to lock the task?

hazy saddle
#

If the previous thread happens to get taken by another core

#

Another core could be modifying it's state switching away from it

wraith star
#

i guess it depends on where you recalculate the priority

wide mica
hazy saddle
#

I need to lock the threads when I start to do the things

#

Then unlock after

urban vale
#

(On a queue, attached to the NextThread link, not on a queue, etc)

wide mica
urban vale
wide mica
#

sec

wraith star
#

well, ig what locks you have to take or not depends on your kernel

#

in mine i dont have a use yet for a task lock to be taken during scheduling, however, there is a lock that is taken when waiting on a waitqueue or waking those tasks

wide mica
#

importantly this is because prb can be remote

#

ig it would be unlikely but it is racy

urban vale
#

The ready queue lock is held

#

of the processor

#

So there shouldn't be a race there

wide mica
#

but you modify CurrentThread without holding the queue lock

urban vale
#

Where

wide mica
#

KepSwitchThread is called without the queue held no?

#

here

#

you'd have to move the current thread update under the queue lock ig or just not load remote CPU current threads

#

what i ended up doing is only relying on the stashed atomic thread priority

#

actually the coolest way to fix this would be to make the thread zone type-stable memory

#

it's still racy but worst case is that it's stale

urban vale
#

The right solution i think is to use the stashed current thread priority & interactivity thing

#

Not sure why I wasn't

wide mica
#

yeah

#

I believe making thread structures type stable is useful for other stuff though, like checking if a mutex owner is running without racing on freed memory

urban vale
#

Relying on type stability only sounds good to me for stuff that is permanent anyway like pfndb

#

Otherwise I'd like to keep everything fluid

wide mica
#

I agree that it doesn't sound good but I've seen a bunch of kernels do it at least for thread structures it seems

urban vale
#

Which ones

wide mica
#

Probably assuming that it won't grow unbounded without running out of memory from other stuff first

#

I've seen it in freebsd at least, which is where I learned the trick

#

xnu also does it, i think that is what they call sequestering

urban vale
#

I mean I'm pretty sure they do eventually free the physical memory after some epoch has passed or something

wide mica
#

Not from what I've read

#

Unless you mean XNU then maybe

urban vale
#

They both do SMR pretty sure

wide mica
#

yea but that is unrelated

#

that's for lock-free data structures

#

I dont understand what it's all doing but it doesnt seem to free here

#

this is freebsd

urban vale
wide mica
#

They don't read the thread structs in a read critical section through SMR

#

And the thread zones are not marked SMR anyway

humble scroll
wide mica
#

is it similar to type-stable memory?

urban vale
#

@wide mica I'm going to fix that race by grabbing ready queue lock while setting CurrentThread probably

wide mica
#

both approaches work

#

lock might be better for accuracy tho

urban vale
#

wel there's a problem, it matters whether the current thread is the idle thread at thread insertion time

#

because any thread should preempt the idle thread, even another thread at idle priority which normally can't preempt anything

wide mica
#

mhm you're right

urban vale
#

I need to redesign that code a little

#

That's a coexisting related problem

wide mica
#

for other cpus

dense hare
#

hi

#

Are there any Repo link?

dense hare
#

Interresting. Which Features does it have?

hazy saddle
#

It's pretty basic rn due to the rewrite

dense hare
#

I mean i am not against viewing the code but i think it should have been written in README file

#

like any normal Project

hazy saddle
dense hare
#

Cause i am not sure i will understand any of this

hazy saddle
#

It's just a basic overview and some instructions

dense hare
#

(WARNING: Because not an OS Developer)

hazy saddle
#

@wraith star I wonder what I should be doing to fix the lock issues

urban vale
hazy saddle
#

I am still like partly unable to understand this after my nap wahhgone

#

And my brain loves to be slow somtimes

wide mica
#

i sent a freebsd patch for this

viral bison
#

my threads get reassigned to another cpu when they get switched, i need to fix it too, lol

#

i'll add a balancer thread

urban vale
wide mica
#

unironically i will

wraith star
hazy saddle
viral bison
wraith star
#

sometimes it also comes down to feel meme

viral bison
wide mica
#

what a horrible idea

#

you learn nothing and you dont understand your own code anymore

viral bison
#

i don't mean to make them make code

#

just point you in the right location to look

#

and then you fix it yourself

hazy saddle
#

i know where it is

tender mantle
#

you should beable to solve this problem without any ai at all

hazy saddle
#

the scheduler

#

i know what files this deals with

viral bison
#

oh scheduler πŸ’€

#

bruuhh, i hate real hw xhci

urban vale
#

I got bored of copy pasting mintia2 code at Claude and asking it what it thinks because it just kept going "This is a very well engineered piece of code, part of a sophisticated operating system kernel, that accounts for every case. One thing to flag: the comment above line 21 may not be completely up to date. On second thought, it is."

wide mica
#

does your company pay claude

hazy saddle
#

half the time i deal with a clanker the thing is so incompetent i have to do it myself

urban vale
#

why not post the gif that family guy is parodying

wraith star
#

i find the petter griffin one funnier

wide mica
#

ai is only good at basic refactors and to help explain stuff imo

gentle quest
wide mica
#

maybe if u use fable or something

viral bison
#

nah, even 5.5

#

it helped me to find a deadlock that was preventing my os from booting in real hw

prime marten
#

do you use ai like claude for bug fixing or idea generation?

wraith star
#

i gotta say that claude is pretty good at creating websites with all the best and whistles

prime marten
#

how does that work, you send your project in some format that llm can understand?

wide mica
#

i gave it a bunch of cool designs

urban vale
viral bison
#

true

wide mica
prime marten
#

is it like 1 agent per file

wide mica
#

no the agent reads files one by one

viral bison
#

or uses grep

#

@hazy saddle Checking for libarchive >= 3.3.3 via pkg-config ... libarchive.pc file not found, exiting.

gentle quest
wide mica
#

the only thing I'll praise LLMs for is learning tbh

#

it's been very useful to me

urban vale
#

They know very little about my interests

wide mica
#

i meant for school too

#

like to generate quizzes before the exam, clarify things, etc

viral bison
#

and find resources online

craggy narwhal
viral bison
#

Evalyn pls add a list of packages that needs to be installed in the os

viral bison
#

it will fault if you don't have libssl or libarchive installed

hazy saddle
#

You need it's deps

#

And it says that in the readme

#

It links to jinx

#

And that lists shit

viral bison
#

it doesn't say those libs in there πŸ’€

hazy saddle
#

What are you even doing that's faulting

viral bison
#

it was missing libarchive and libssl

prime marten
viral bison
#

I: Configuring ifupdown...
I: Configuring libc-bin...
I: Configuring ca-certificates...
I: Base system installed successfully.
make: *** [Makefile 17 bootstrap] Error 1

#

is this correct?

hazy saddle
#

What command started that

viral bison
#

make bootstrap

hazy saddle
#

Idfk then

#

Works on my machine

#

Jinx uses containers

viral bison
#

and make run does fail

#

bc it cannot find limine.h

hazy saddle
#

Yeah because bootstrap failed

#

πŸ₯€

#

And you need to run get deps

#

Which that runs

viral bison
#

it doesn't even tell me why it failed πŸ’€

hazy saddle
#

And the bootstrap gets other things needed to make isos

#

Idfk how to help

viral bison
#

f

hazy saddle
#

that way the queue when unlocked can never contain threads in progress/running?

haughty acorn
viral bison
#

same

hazy saddle
#

it would also make sense since for mutexes they shouldnt be in the run queue

haughty acorn
#

a thread can only be in the run queue if its in the ready state

#

in my kernel

hazy saddle
#

issue would be moving the current thread there, and knowing to not move it back when putting in a mutex then rescheduling?

wraith star
#

i keep the task in the queue because it eliminates complexity

#

if you are the highest priority process, next_task() will be yourself

haughty acorn
#

i do a slightly weird thing and smuggle the old thread pointer through to the new thread and then the post-context switch part of yield/threadstart uses that pointer and some flags smuggled through to update the old thread's state after it stops being current so current doesnt become invalid

#

my yield also has a force flag and if thats false it does a peek first, and if the peek is lower priority it just aborts the yield. if force is true and youre still highest prio itll get requeued by the idle thread

hazy saddle
#

i mean. thread needs to block -> puts that in the state -> but in mutex qeue -> calls reshcedule -> it ends up in both the runqeue and the mutex qeue

wraith star
#

well, when you put in the mutex queue, dequeue yourself

haughty acorn
#

yeah just dont put it in the runqueue?

wraith star
#

and also disable irqs/put yourself to dispatch ipl

hazy saddle
#

when you switch away from the task

wraith star
#

have a task status

#

TASK_BLOCKED or whatever

haughty acorn
#

you have a flag for what the new task state is yeah

#

and only push back on if new state is ready

wraith star
#

imo, the scheduler should not care about task statuses in other places other than the after switch call

haughty acorn
#

putting on the wait queue would set that to blocked so the status isnt ready and thus it doesnt get put back on the runqueue

hazy saddle
#

So I should have task running and task blocked and task ready?

wide mica
#

i have zombie and terminated too

clear bison
#

Q eysknso oz l,o9$8 zozl

wide mica
#

and selected

#

when it'll run next

haughty acorn
#

(theres a nuance to this that @wide mica can probably find the video about where you need a pending block status to work around race conditions if the wait is aborted)

wraith star
#

maybe collapse task running with task ready if you don't really care about what tasks are executing in a given moment

clear bison
#

Tf did i type

haughty acorn
haughty acorn
#

yeah

#

but yeah the bottom line is it wont be on both wait and run queues because youll know from its status that it isnt ready to run and cant go onto the run queue

wraith star
#

and overall, the scheduler itself shouldn't be doing enqueues and dequeues to any sort of runqueues

#

have the task blocking code do the enqueues and dequeues, have the task creating code do the enqueues

haughty acorn
#

apart from the post-yield callback anyway

#

in case of manual yield or preempt

wraith star
#

yeah

haughty acorn
#

a task goes onto a run queue when its wait finishes, it is created, or its switched away from without blocking. it goes onto a wait queue when it starts waiting. and it goes onto a cleanup queue when terminated depending on how exactly you do that

#

i personally call it a ready queue instead of a run queue because its a queue of ready threads but naming is a personal decision so you do you

hazy saddle
#

so i should modify this to take in the thread_t*s them self, then call a function on the new stack that will manage the run queue based on thread states and then return

#

and that should fix this??

wraith star
#

the preempt needed probably looks a bit sus

hazy saddle
#

I also keep redoing my CPU local partly because it's kinda borked for some shit 😭

haughty acorn
#

(there is more complexity later about priority inversion and temporary boosting and other such fun details but what atie and i describe is a perfectly functional priority-round-robin scheduler)

wraith star
#

yeah

hazy saddle
# haughty acorn yes exactly

The main issue now is that the run queues can be inconsistent with the previous thread inside of it and causing races

hazy saddle
#

This should also work with priorities anyways right?,

haughty acorn
#

yeah it works with prio

#

it just doesnt handle starvation

haughty acorn
#

or whatever its called when a low prio thing cant function because a high prio thing exists

hazy saddle
haughty acorn
#

yeah thats what i meant. this handles priorities but not that

hazy saddle
#

Oh I'm just trying to handle the race condition rn 😭

haughty acorn
#

you can release the queue lock in the post-yield

#

(oh and you also call the post-yield function as part of threadstart for this to work, the idea is that the beginning of threadstart looks like the post-swap part of context switch)

hazy saddle
#

Current thread also still exists in the queue

#

Along with the previous one etc

haughty acorn
#

oh thats the issue

#

only ready-to-run threads should be in the queue

hazy saddle
#

yeah

haughty acorn
#

current should be a separate var

hazy saddle
#

it is mostly just not removed from the list

haughty acorn
#

that way when you push you can detect priority preempt conditions, you dont have to worry about whether to remove from list because it already was when it started running, etc

hazy saddle
#

yeah, after swapping stacks prev gets pushed if needed and the new current is popped

haughty acorn
#

well youd pop new current before swapping stacks

#

so you have it and know what stack to swap to

#

the prev thread gets left in limbo as a local smuggled through the stack swap in a register

wide mica
#

you have to have some kind of mechanism for the thread to stop swapping its stack

haughty acorn
#

the post-yield does a cmpxchg expecting wait_pending to wait_committed and if the old was instead ready then it queues it back up

wide mica
#

that's not what I'm referring to

#

i believe

haughty acorn
#

then what are you referring to?

#

because thats the only case i can think of where a thread would be woken when not already post-stack-swap

wraith star
#

in mine, waits and wakes are mutually exclusive by a spinlock in the task struct, and when waiting, the lock is held until after the context switch

haughty acorn
#

i tried that and got deadlocked lmao

wraith star
#

how?

haughty acorn
#

i think my code was just shit

wide mica
#

and people keep roasting me

#

😭

wraith star
#

where

#

lmao

#

now?

#

or before

wide mica
#

no

#

i said you have a per thread lock u hold until the switch is done

hazy saddle
wide mica
#

just thread->lock

haughty acorn
# wraith star how?

oh i remember now my thing deadlocked because of a nested lock caused by trying to wake it from an interrupt handler running on the same core

wide mica
#

you could have an atomic bool as well ig

hazy saddle
#

eh i still think there are other issues with thr run qeeue i need to fix

haughty acorn
wraith star
#

what is the IRQ were talking about

haughty acorn
#

nvme command queue

#

the apic just completely forgot the interrupt instead of waiting when i disabled interrupts, even when i used TPR instead of cli/sti

wraith star
#

uh

haughty acorn
#

i still dont know why it happened and its 80% of why my motivation to keep working on my kernel is at a low

wraith star
#

that's not good

haughty acorn
#

correct!

wide mica
#

This is why you have DPCs

wraith star
#

are you doing eoi somewhere where you shouldn't?

haughty acorn
#

my entire sched and dpc and irq handling needs help badly

haughty acorn
#

as i say i still dont know what i fucked up and ended up just taking time off and now im in side-project-land with zaml instead of trying to debug this hell

wraith star
#

for edge interrupts, the eoi has to be at the beginning

urban vale
wraith star
#

otherwise you miss new irqs

haughty acorn
wraith star
#

msis are always edge

haughty acorn
#

ok that is likely at least part of the issue

wide mica
#

his coolness has no limits

haughty acorn
#

at least its one msi per queue so repeat vs delay/combined nvme interrupts are idempotent

#

but yeah when i make more progress on or get bored of zaml ill go back to my thing and fix this shit lmao

hazy saddle
#

btw this means i dont need a lock on the thread qeueue since the threads being swapped can never be grabbed?

haughty acorn
#

the swapping ones cant but multiple can be queued at once possibly

#

(if say a semaphore or something like that gets a bunch released at once)

wide mica
hazy saddle
#

i mean only have the lock when its activly being modified

#

but that would fuck with alot of crap

#

since it would unlock and regrab right after anyways

hazy saddle
#

Okay now I just wana cleanup my CPU local things

#

Because the previous stuff worked fine

#

But the schedular changes some things up

#

And it's a bit messy rn

#

Then blocking would just a preempt disable, set to blocked, add current thread to queue enable preemption then reschedule

#

@wide mica @wraith star @haughty acorn thanks for the help

wraith star
#

if you wake up from irq context, be careful

#

np

hazy saddle
wraith star
#

disable preempt won't be enoug

hazy saddle
#

Ig then mutexes need to disable IRQs when operating on them

wide mica
urban vale
#

Yeah this is precisely what DPCs are for

wide mica
#

you typically NEVER wanna take a lock from an irq context unless it's some pathological case where you have no choice

hazy saddle
#

I guess il learn how to do thoes once I do mutexes

#

Since I have other crap to do

hazy saddle
#

Like making the CPU local layout non arch specific and paging reworks and some userspace things and timers

#

I'm probably gonna make structs for each subsystem to put into the CPU local thing, this will also make accessing parts of it much faster since it's always a fixed offset for the core kernel things

haughty acorn
#

theyre p easy

hazy saddle
#

Then drivers that need extra CPUlocal can do that seperate with the slower helpers

#

Actually I can solve that by just adding an extra data pointer to the timing system for the drivers that activate to have extra data for them

#

Holy fuck I over engineered the CPU local badly

#

😭

haughty acorn
#

(DPCs can be cpu-local or agnostic depending how you do them, if you schedule one from a thread you just run it immediately if you schedule one from an irq you run it as soon as you finish irq handling and turn irqs back on)

#

(the only DPC quirk is scheduling one from a dpc or onto a specific cpu from a different cpu but those arent hard to handle)

wide mica
#

guys APC or DPC which is the best
APC could be a strong contender since like every kernel has them pretty much (maybe not linux idk) + they're slick, stealthy and cool
but DPCs are pretty cool too and more useful i feel like, its like a good friend you can always rely on

haughty acorn
#

both tbh

#

they are differentβ„’

#

and both are useful

wraith star
#

imagine having dpcs

wide mica
#

noo this is vibes debate

#

obviously they're different meme

wraith star
#

and imagine having apcs

haughty acorn
#

i like dpc vibes better because theyre easier to implement

hazy saddle
#

my cpu local code also was broken on la64 bwcause half of it ended up with riscv assembly syntax and i never noticed it

#

😭

haughty acorn
#

F

hazy saddle
#

Welp time to fix ts

#

And have a partly working scheduler

#

Well it won't be fully working until SMP on all architectures and I handle shoot downs and other bullshit

#

And thread migrations

hazy saddle
#

?????????

#

lmfao

#

it was LTO tossing out my cpu local

#

😭

#

I wonder if that was also causing other bullshit

hazy saddle
#

I'm gonna have to clean this up tomorrow

#

And then implement the timer subsystem

viral bison
#

symbol + footprint done

hazy saddle
#

tis should be correct

#

im just finnalizing my scheduling sutff and cpu local reworks

#

it goes schedule() -> arch_thread_switch() -> schedule_finalize() -> arch_thread_switch()

#

and the arch_thread_switch does the final unlock and then restores things

#

i keep the lock held the entire time even if i dont FULLY need it

#

since locking unlocking and relocking and then unlocking would be dumb :p

#

and doing the unlock in assembly generates less instrictions than needing to use CPU_LOCAL_GET_SCHED_LOCK_PTR() etc

magic saffron
hazy saddle
magic saffron
hazy saddle
#

I think I solved thoes kinds of race conditions by making sure the thread is only in the scheduler queue if it's eligible to be scheduled

magic saffron
#

if the thread isnt locked during context switch it might commit to a wait: but before it can complete it's context switch another cpu can satisfy the wait and wake the thread. The waking cpu places it in a run queue somewhere or it may preempt on the waking cpu and begin running immediately.

hazy saddle
#

Ah this shouldn't be able to happen since the thread is owned by one thing at once here, eg it won't be in the schedule queue and the wait queue at the same time

magic saffron
#

right but what if its in the wait queue and currently executing (has committed to a wait but not yet switched to the next thread).

hazy saddle
#

the current thread would be refered to by the current thread in cpu local, if its status is blocked it wont get added to the main qeueue, and grabbing a mutex would disable preemption to not race on thoes operations

magic saffron
#

yeah you wouldnt have this issue on UP, its an issue where one cpu wakes a thread was preparing to sleep on another.

hazy saddle
#

so it should be fineβ„’

#

it will prolly be a bit til i implement shootdowns and other things anyways like that

#

since im gonns wait for riscv and la64 smp for that

young hamlet
#

And another core steals the task calling schedule / the task calling schedule gets push migrated

hazy saddle
#

schedule() runs with IRQs disabled?

#

thats an irq lock

young hamlet
#

Is it only called with interrupts/preemption disabled

hazy saddle
#

also getting the lock is a single instruction

#

or at the very least should be impodent to not cause issues

#

since it either just gets tp and does a single add on riscs

#

or first loads the self pointer on x86

young hamlet
# hazy saddle also getting the lock is a single instruction

(Assuming your macro CPU_LOCAL_GET_SCHED_LOCK_PTR gets a per runqueue lock rather than one which is global) In between the time you read what lock to grab from it, and grabbing the lock itself, if preemption occurs, this may be bad if your task is migrated, as when it wakes up it’ll grab the wrong lock (as then the queue which is read would be different)

Ofc, this isn’t an issue if schedule is called in a context where the task cannot be involuntarily preempted

hazy saddle
#

i think il fix this by having irq/non irq locks

#

then just disable irqs seperatly

#

Thanks for pointing that out

wide mica
#

Otherwise there's a race between reading the cpu and acquiring the lock

hazy saddle
#

spinlock_lock also is not not an irq lock and irq_spinlock_lock is

wide mica
#

It is very rare that you need to disable interrupts

#

also same thing with finalize here

hazy saddle
hazy saddle
#

so i can just disable the preemption

wide mica
#

yeah

#

Unless you take locks from the irq for some reason

#

In that case, don't

hazy saddle
#

i only take the scheduler lock so

wide mica
#

wdym

hazy saddle
#

the one for the thread qeueue?

wide mica
#

Yeah but don't take it in an irq

hazy saddle
#

but what if im trying to preempt from an IRQ?

wide mica
#

DPC

hazy saddle
#

i really still dont understand what the diffrence between running in a DPC is vs running in an IRQ context

wide mica
#

DPC runs with interrupts enabled and preemption disabled

hazy saddle
#

my IRQs also run with them enabled too?

#

im like 99% sure this would still fuck up badly :p

wide mica
#

Obviously this is still wrong

#

Same bug as before

hazy saddle
#

yeahb but its worse now because idk how the fuck its meant to be fixed without disabling irqs :p

wide mica
#

You have to disable irqs for a very short time when disabling preemption

#

But the point is to not disable them across lock holds

hazy saddle
#

that wasnt even what had made me worried about this

#

prev->preempt_disable_counter -= 1;
idk if there is a place for this that wont race?

wide mica
#

If preemption is disabled then that shouldn't race no?

hazy saddle
#

Well when the new active thread is set preemption would then get enabled?

#

unless i disable preemption on both threads??

wide mica
#

Why is the counter per thread

young hamlet
#

Per thread preemption counts are kinda based

wide mica
#

Why

young hamlet
# wide mica Why

Can allow ur task to get scheduled out if it’s preemption is disabled without side effects

wide mica
#

True

young hamlet
#

But how useful this is, I don’t know

wide mica
#

Yeah it's not really useful

#

Sounds like it'd kill latency

#

Or rather

#

Make it higher

hazy saddle
young hamlet
#

Afaik Linux and Xen both have it in gs (on x86 platforms)

wide mica
#

virgin preempt counter vs chad irql/ipl

wraith star
#

in linux the preempt counter is in current

young hamlet
wraith star
#

in x86

#

idk other arches

#

oh wait nvm it's a per core

#

in other arches it's in current

hazy saddle
#

So what should I be doing

wraith star
#

switch to a per core counter

#

you already have percpus set up in place, and in x86 you can do inc gs:whatever

hazy saddle
wraith star
#

in those other arches it is not possible to increase or decrease cpu locals, so the counter is in the task struct

hazy saddle
#

I do have plenty of room on the CPU locals on other arches anyways

#

So it's fineβ„’

#

And I can just do a quick enable / disable irq

wraith star
#

nah

wide mica
#

Temporarily

wraith star
#

eh

wide mica
#

That is what I do

hazy saddle
#

I mean

#

On other arches it's cheaper than x86 iirc

#

Because no flags memes

hazy saddle
#

Just on the RISCs instead of x86

wraith star
#

the value of current is always going to be stable in a task

#

(obviously)

#

so you can put the preempt counter in the task struct

#

and you know that you can only be interrupted in instruction boundaries, so you have either increased/decreased or nothing

hazy saddle
#

so in theory would this work?

wide mica
#

Anyway I think if you wanna do irqls you have to make it per cpu

hazy saddle
#

What if you did both

wraith star
hazy saddle
# hazy saddle so in theory would this work?

Getting the current thread is always stable and one instruction so that won't race.
And then it can switch safely the tasks etc. then when it's finalizing it disables it and then it can race

#

I think I'd need to disable IRQs there?

#

Unless I preempt disable on next and then switch and then decrement both?