#uDRM - a portable Direct Rendering Manager
1 messages · Page 5 of 1
then we can still use map_device_memory and it would return not supported and the driver would then fallback to single page allocations and scatter gather
there is "can" and "can do well"
i mean i hardcode an 8 entry queue atm lol
well if your kernel cant provide adequate resources/features and the driver has to use a suboptimal method thats the kernels fault
no, it's the driver's fault for needing a feature that is very hard to implement efficiently
it also has difficulties with fragmentation
again: windows doesn't have it, so your os does not need it either
thats usually device memory
not host memory
but hmm
virtio_shm?
okay this sucks
making very large allocations is annoying for many designs
my allocator at least can deal with it, but its just bump allocating out of the memory map
and i wouldnt do that in a performance-targetting design
just use scatter gather where ever its possible and if its not and you really need a big blob of memory then request a bigger block of memory from the kernel ig
then if it doesn't support that it just doesn't use the driver for that card lol
before that ask the kernel if iommu is supported
gpus have page tables for a reason, though
someone with more knowledge than me needs to decide this
then the kernel needs a way to signal capabilities to drivers
i wouldn't use large physical pages most of the time anyways
ifdefs :^)
#if defined UAPI_KERNEL_HAS_FAST_LARGE_PHYS_ALLOC
either also a iommu ifdef or both are combined and are abstracted behind a single function
iommus are not global
for example you may have an iommu for the normal gpu, but not the integrated gpu
does such a plattform exists?
m1
as this exact scenario soundy incredibly stupid design wise
i.e. yes
its not
iommus are expensive
and gotta go fast
the m1 has unified memory
and the GPU has its own page tables anyway
like for userspace mapping or something
then ifdefs per driver?
its a runtime value
sometimes you just
dont have an iommu
so its a runtime per-device thing
? iommu is hardware stuff afaik it doesnt just disappear on reboot
yes, but it does change between systems
also it could if the user turns it off in the bios
also, GPUs are runtime hotplug which is fun
so yeah uh im glad im not the one designing this!
also qcom phones from the looks of it
reasonable
so yeah its not that uncommon

drm will have a hotplug event
whatever linux drm can support
is it hard to support?
Also, there are PCs without that (?)
Like my kernel supports fast contiguous allocations of arbitrary sizes, and many people also use buddys and stuff like that
(1) i think it's just a normal(ish) driver, but it messes with things that all other drivers need
(2) yes
I don't think I even own a PC with an iommu
Afaik all x86_64 cpus that support virtualization have one
3/3 of my computers with virtualization do not have an IOMMU
Are they older?
Iommu is part of the virtualization extension but I don't know which gen
And maybe they only gave Server chips one
Maybe the mb doesn't support it
The table signature?
Wdym
How do you know the acpi table for the iommu is invalid?
Linux says so
It has 0 as the physical address
In the table
Intel
#osdev-misc-0 message
Wikipedia lists the Core2 Quad Q9400 as one of the first supporting cpus (2007/08)
Maybe it's disabled in uefi?
you need iommus for thunderbolt
or secure thunderbolt anyway
so i think its gonna be a thing in all new stuff
are there any systems with thunderbolt without an iommu?
i wonder if a thunderbolt device can do DMA without being limited by an iommu, if yes it could be used for cheats and probably a lot more stuff
if you dont have an iommu yes
otherwise lolnope
well yes I'm asking whether there are systems without an iommu for thunderbolt, I'd assume most do
because letting something do DMA on a USB port is probably a security hole so no sane design should let that happen?
never underestimate the insanity of firmware devs/plattform designers
firewire:
Most likely it's disabled in bios, try searching the bios settings (it can be well hidden sometimes)
Anything Skylake and newer usually comes with vtd enabled by default
so apparently core i3s dont have a iommu so its possible for the user to combine a i3 with a thunderbolt mainboard
Insanity
So from our discussions I think we need three different phys mem allocation paths:
-iommu mapping
-large continuous allocation
-"give me a list of pages"
The lowest one would be guaranteed to be supported, large allocations support with a define and iommu mapping with a define(to tell the driver that at least the kernel supports it) and fails with not_supported
Why do you even care if iommu is supported tho? The driver never really needs to touch physical memory directly so having every allocation just give a device address and virtual address is enough no?
Kaby lake i3 and upwards have iommu, but yeah in older architectures i3s didn't really have iommu
a iommu makes it very easy for the kernel and driver, the driver needs physical addresses for scatter gather lists, but tbh the iommu alloc and large alloc could be unified into one alloc function
a iommu means that no matter how fragmented your physical memory is you can always allocate memory for devices as long as you have sufficient total memory
I mean, technically yes but practically most allocations you need aren't that large
I agree that an API that makes it easy to have contig in virt and non-contig in device address can be nice
Linux basically just has dma_alloc_coherent, there are functions to map virtual memory to the device but they are highly discouraged from the ease of misuse
"large" means everything that is larger than the base pagesize and not a higher pagesize
Yeah that's pretty rare
Hardware vendors know that you need to keep minimal down to a page or two so they design the hardware with that in mind
Xhci segmented queues exists for a reason
yeah most devices work with <= 4kb contiguous physical mem allocations and don't really benefit from larger ones that much
so we want to artifically limit uAPI to those devices?
you could make larger queues with it, but nics for example usually cap at 512-1024 entries in a queue which even with 16 bytes per descriptor is only 8-16k which is really not that much
I say that no driver wants to have random defines around memory management just to know if it could happen to have a large allocation, or if it thinks it can benefit it could very much try to make the allocation and fallback
I think only one API is actually needed which is an alloc_sg, if the kernel can return it as a single contig device range so be it, otherwise it will segment it
only the scatter gather alloc would be gurranted to be supported by a kernel the other one is optional and signaled by define iommu_kernel_support and large_alloc_support ored together
But why do you need any other one
And again the driver doesn't care for either
It could always just try and fail
You will have an if either way
And having random ifdefs around memory management is just error prone
and for iommu it has to fallback anyway as its a runtime thing
And map_sg always succeeds, you can have a flag telling it it must be contig in device memory for the case you do care (which tbf there are, but the driver shouldn't spam it around if it supports both)
Like device address == physical address without iommu, but the driver literally doesn't care
For the driver it could be wherever
it would also have to fallback if memory is too fragmented
so we could make it one single function that would return an array of {virt_addr,device_address,size}
with iommu or large alloc support it would be a single entry
It can always be contig in virt memory tho
Unless we want to support mmu-less drivers lel
there is also a virtual map function already
couldn't it be like host phys addr + device phys addr + size?
Why do we need a virt map tho, in what case does a driver needs to map arbitrary physical address (ignoring uacpi completely)
good point i forgot that
or actually wait no that wouldn't work as the current map function just maps contiguous physical memory to some kernel-decided virt addr
ig returning an array of device addr + size and then a virt addr separately would work? though the driver doesn't always need the virt addr so ig there could be a flag for that
in what scenario does the driver not need access to memory mapped for a device?
we could add that as a flag parameter
though I think those specifically also need to be saved/restored on suspend/wake so you'd need the virt addr to be able to do that but there might be cases when you don't need to do that
Pass null as the parameter to the out virt address
And it returns a handle you use to free
that would make the function less readable
How so
Like you want to return a handle anyways, how else would you return the virt addr
how would you differentiate a handle from a valid virt address
They are two different things
A handle is something pointer sized and opaque to the driver, only used for things operating on the mapping (in practice free)
Virt is the virtual address you are given back to access the memory
And we could have it optional
Like it's an out parameter
void**
you currently want to pass both trou one parameter
No?
if i understand correctly
uapi_handle_t thing(void** out_virt, ...)
nevermind i understod you wrong
Technically we might want to return a status instead, but personally I think for memory allocation a null handle is enough to signify the allocation failed
we should signal why it failed, if we add allocation constrains like force continuos or <4GB
so that the driver may handle it or just aborts
so uapi_mem_status thing(virtaddrhint, size, flags, handle)
Now that I think of it, does 2 even need a define? This is a bit of a corner case, but large contiguous allocations can fail as well, so in that case or if the kernel doesn't implement them it can just return an error
with the now unified alloc function i dont think so, maybe as a hint for the driver
I think there should be another one: large virtually contiguous allocation. This would primarily be of use to the driver itself, for example to setup some sort of double-buffering in software.
It's debatable whether this would have any use, though. After all, it should be userspace that renders to the framebuffer, not the kernel
allocation does not necessarily mean mapping
virtually contiguous is not an allocator property, but a mapping thing
udrm disappeared from active threads 
How's uDRM doing? 
Does this already support modesetting?
no lol
tired, mentally drained atm
expect actual progress start of feb
i get 2 weeks off then
I'm asking because I'm in a mood for S3 sleep and it would be nice if it could do that
ok np
okay time to make a shkwve_modeset crate so i can beat you out 
go ahead 
marv needs that S5 sleep before he can work on uDRM 
FR
first i need to find my other laptop so i can do real hardware debugging
because if you only do VMs is it really modesetting
or in other words, my work laptop fucking sucks
so i'm building myself a work pc
work pc
9600X
Use bochs vbr
Vbr
Vbe
Nah I'll just fix ns16550 then
And I have 2 PCs with COM ports
So I can test on physical hardware
For playing Minecraft 
troll
if you need help or just example driver source code for virtio gpu I have fully implemented one on my kernel, it supports 2D+3D with both control and cursor queues implemented: https://github.com/WivOS/WivOS
vouch
Do you actually have mesa ported and it works with 3d accel with your driver?
The 3D backend is custom, but should be really easy to port for mesa, as the GPU shader language is the same as mesa's intermediate language
What can it do?
Currently has a half working OpenGL API, so it can load shaders and render some index based models
You implemented a virtio GPU based opengl backend from scratch?
Yeah
3k 4k if I remember it correctly
Where's the code for it?

Also lol found an old school os still with lai
Was it on pause for a while?
Yeah, I didn't have much time
Work consumes a lot of hours, but I'm having more time recently
That's pretty cool because Leo wasn't even able to get it to work with mesa giving him a ready to go command buffer
🤣🤣
What can the os do atm?
Currently it can fully boot into a custom compositor based environment, it supports xhci mouse and keyboard, the compositor is really basic and doesn't have too much for now.
Nvme, etc
Yeah

Are you gonna make a progress thread?
At least it tries to be
There is already one xD
Custom libc too?
mlibc
Ah
The last progress I made was to implement the cursor thing on virtio
looking through the code rn
you're relying a lot on HHDM
(not inherently a bad thing, but might cause issues)
What's the problem with that?
You mean using that on drivers?
It's not hard to port that to other types of mapping on the drivers
cool
what range do you map?
lower 4G of phyiscal or the entire phys space?
size yea
lower 4G if I remember correctly
above 4g too?
If I remember correctly
Yeah above
i'm just mentioning this because it happened to me lol
That part of the os is very old, so I just say probably because I remember traces of it 
Yeah, if you map non existent memory that will crash
Anyways, what's the problem with HHDM?
nothing, just uDRM won't be able to use it at all
Why?
because uDRM also needs to work in user space
couldnt you make map return hhdm addresses?
ah
ah yeah
just hhdm map into userspace 4head

🤣🤣
Even Nintendo switch os has HHDM xD
kernel only tho
well, my kernel does as well
and the userspace gpu driver is behind an SMMU
unless you search for "bypass the smmu" in the datasheet because there is a bit to bypass the smmu from the gpu side
How's virtio going btw
.
i can get it working today or tomorrow
I'll be in VC at about 18:30 cet
When uDRM
soon™️ is also gta vi
i still need to find a way to test using an existing kernel
my own kernel is too bad
managarm is too phd for me
linux
We got gta6 before udrm
Do you need mesa?
yes
literally only does uDRM and nothing else
in userspace or patch it into linux?
or a kernel module?
userspace would be fucking based lol
yea userspace would be based af
i mean udrm is meant to run on microkernels
do userspace then
hm
it's easier for debugging too
and you can launch it on your host system provided it has a gpu that uDRM supports :3
:3
without rebooting possibly
don't forget --depth 1
and do you need the kernel for doing it in userspace? does linux expose the needed stuff to userspace?
uhhhhhh
e.g. can you map a device's pci configuration space to a process?
iirc for Linux userspace drivers you need a small kernel module that does hard irqs
that's finee
but all other stuff can be done in userspace
yep
iirc read yes, but idk about mapping
you can read pci cfg space through sysfs
udrm through sysfs is diabolical
i have not worked with linux userspace stuff before
time to learn then!
don't call the platform just linux imo
call it linux userspace or something
you might have a udrm in kernel space later
yes thats what vfio is about
i literally told you this before already btw
i have memory loss
i can tell
lol
vfio is the stuff you do for passthrough
so you pretend udrm is a """virtual machine"""
so that you can steal the gpu for testing stuff
and that works for modeset at least
or it doesn't, I haven't got it to work for 11/12gen intel igpus
the south bridge registers that are used for stuff like display detection just don't properly work
That's news to me
I've never tried anything behind coffee lake tho
yeah, I also tried it with the igd stuff but no dice (for an example SHOTPLUG_CTL + SDEIIR just returns all zeros regardless if there is something connected or not iirc)
yeah idk, I did have to add the display class other pci match to the vfio code but ig there might be something else too
it is supposed to work yeah, I tested it on bare metal boot and it worked just fine
Idk how sriov relates to all of it too
Intel changed around how all of the virtualization modes work with tiger lake iirc
gpu moment
It's almost like gpus are somewhat small computers of their own

especially when you start to consider all the render stuff too
EUs executing their own code and then communicating with the shared functions using message queues (well its not really exposed like that, its just a send() instruction to send a message to a specific target function)
(on intel)
Re-asking here, what's the status of uAPI?
How is uDRM going? 
i should have something working in the next few weeks
i have little to do until march
cool
im using linux as a test kernel since that's the easiest for me to work with and is guaranteed to have everything i need
okay linux is saying hello to udrm
@maiden birch this might be too late but uapi is really not a convenient name
got any better ideas?
right now seems like the perfect time to change it since nothing uses it
because of include/uapi in the linux sources?
μapi
include/µapi
correct
alright. a new name is needed
uniform driver interface?
some cursed wiki.osdev.org thing?
(although it sucks terribly)
I think I read about something like that
unified udi
my opinion is that we need no "uAPI"/common driver interface. the reason for this is that the kernel APIs required by eg. uDRM and uACPI differ substantially, to the point that the cost of maintaining a separate "uAPI" exceeds the benefit
uHDA would also use it
write the code for obos
but you are right
and force everyone to port obos interfaces
do you have msix
@maiden birch wdyt
im not against it
one thing that I would like to add is that we should strive towards keeping APIs similar in semantics. for example, if I have implemented uacpi_kernel_map and want to integrate uDRM, ideally I should just be able to copy-paste the implementation of uacpi_kernel_map into udrm_kernel_map and it should work
(this might be a bad example, I guess, because GPU drivers might want specific caching modes for certain things, and that cannot be done by uacpi_kernel_map as of today)
µuudi
Just ultraAPI?
imagine ultraAPI in linux
quite understandably
(except ACPICA now)
i never saw a greater design by committee in my life
(there are 18 operating systems using it already?)
ultra advanced performance unified driver interface
UAPUDI
and i don't mind designs by committee but UDI is insane and the osdev wiki community sought to make it moreso by making it bytecode
huh?=
whattheactualfuck
let me show you a bit of what a udi driver looks like
Let's not do that.
I actually wanted to implement udi once
thankfully people here stopped me
you can design something really well
and you can implement it really well
those are two separate things.
and I believe we should strive for the latter
iirc Torvalds said something along the lines of "Linux is good engineering, not good design."
and I agree with that to some extent
static udi_pio_trans_t cmos_trans_read[] = {
/* R0 <- SCRATCH_ADDR {offset into scratch of address} */
{ UDI_PIO_LOAD_IMM+UDI_PIO_R0, UDI_PIO_2BYTE, SCRATCH_ADDR },
/* R1 <- address */
{ UDI_PIO_LOAD+UDI_PIO_SCRATCH+UDI_PIO_R0,
UDI_PIO_1BYTE, UDI_PIO_R1 },
/* R0 <- SCRATCH_COUNT {offset into scratch of count} */
{ UDI_PIO_LOAD_IMM+UDI_PIO_R0, UDI_PIO_2BYTE, SCRATCH_COUNT },
/* R2 <- count */
{ UDI_PIO_LOAD+UDI_PIO_SCRATCH+UDI_PIO_R0,
UDI_PIO_1BYTE, UDI_PIO_R2 },
/* R0 <- 0 {current buffer offset} */
{ UDI_PIO_LOAD_IMM+UDI_PIO_R0, UDI_PIO_2BYTE, 0 },
/* begin main loop */
{ UDI_PIO_LABEL, 0, 1 },
/* output address from R1 to address register */
{ UDI_PIO_OUT+UDI_PIO_DIRECT+UDI_PIO_R1,
UDI_PIO_1BYTE, CMOS_ADDR },
/* input value from data register into next buffer location */
{ UDI_PIO_IN+UDI_PIO_BUF+UDI_PIO_R0,
UDI_PIO_1BYTE, CMOS_DATA },
/* decrement count (in R2) */
{ UDI_PIO_ADD_IMM+UDI_PIO_R2, UDI_PIO_1BYTE, (udi_ubit8_t)-1 },
/* if count is zero, we're done */
{ UDI_PIO_CSKIP+UDI_PIO_R2, UDI_PIO_1BYTE, UDI_PIO_NZ },
{ UDI_PIO_END_IMM, UDI_PIO_2BYTE, 0 },
/* increment address and buffer offset */
{ UDI_PIO_ADD_IMM+UDI_PIO_R1, UDI_PIO_1BYTE, 1 },
{ UDI_PIO_ADD_IMM+UDI_PIO_R0, UDI_PIO_1BYTE, 1 },
/* go back to main loop */
{ UDI_PIO_BRANCH, 0, 1 }
};


ACPI is (often) bytecode, and UDI is apparently also (sometimes) bytecode... lui (lightweight UDI interpreter) or uUDI when?

Perhaps, although unique kapi per each project does sound annoying
what's worse udi (from the relatively superficial inspection i made of it) seems extremely static
things must be defined in advance in properties textfiles and it looks to be an interface fundamentally incompatible with treating modern day needs like multi-queue devices with core local queues + MSIs
ULTRA UNIFIED DRIVER INTERFACE
IT'S BECOMING REAL
everything is being unified we're approaching singularity
all osdev projects will merge
... and that is a bad thing
https://gist.github.com/netbsduser/5ce709f8a1079199be133435332af52b these 5500 lines of code are an UDI driver for a very simple NIC from the early 90s
and some, i assume, of UDI are good ideas
if well over 90% of your code is just dealing with an interface instead of doing the work that it should be doing, that interface (UDI) is probably bad
or rather
it is a hint that a better interface is probably possible
This looks like parody
No way this is real
no. obviously fadanoid just wrote 5496 lines of actual code to make a joke
surprising that they didn't at least invent some assembler for it so you don't have to fuss around with the arrays of structs by hand
https://gist.github.com/netbsduser/75e10b3309cfd71b79443bfc0b39dd00 here is a simpler driver for some scsi controller
"""""simpler"""""
lmao
I don't think it's that bad
definitely fewer lines
believe it or not there is allegedly a full USB stack that was implemented in this

at least full by the standards of the late 90s
unfortunately it's sco so it's proprietary
where do u get these
and does anyone actually use them
go figure
i have a copy of the UDI Implementor's Licensed Reference Implementation sitting on my hard drive from some time ago, probably 2023 when i last looked at udi
i dont like this
Probably not a single soul does
there are some people keen advocates of it on osdev forum, and even for making it even more complicated: https://forum.osdev.org/viewtopic.php?p=179282#p179282
amusing remark from there:
The complexity of ACPI makes people want to use ACPICA. The complexity of UDI should make people want to use Project UDI.
lmao
https://github.com/thepowersgang/libudi-rs/blob/master/samples/net_rtl8139/src/pio_ops.rs it looks like someone did later for their rusticated UDI
jfc
// - Reset Sequence
// > Power on
LOAD_IMM.B R0, 0x00;
OUT.B Regs::Config1 as _, R0;
// > Reset and wait for the reset bit to clear
LOAD_IMM.B R0, 0x10;
OUT.B Regs::Cmd as _ ,R0;
LABEL 1;
IN.B R0, Regs::Cmd as _;
AND_IMM.B R0, 0x10;
CSKIP.B R0 Z;
BRANCH 1;
// > Reset complete
// > Disable interrupts
LOAD_IMM.S R0, 0x00;
OUT.S Regs::Imr as _, R0;
aml if it was even worse
Why cant this sequence be represented as a series of kapi calls
they call it "The PIO subsystem"
uapi_load_imm(UAPI_R0, 0); 
well judging by the example drivers posted earlier that's a failure
they still need C sources
The c language is like fully arch independent lmfao
on a binary level
and if you need C sources why have bytecode
^
I don't understand this whatsoever as well
the official reason, if you are interested, is because they thought in the future drivers would be run mostly on special coprocessors, and only those coprocessors would have access to the register spaces of devices
...why did they think that?
What the hell
that's fucking stupid
so udi drivers require all such accesses to happen by instructions to the udi pio subsystem, so they can add support for these coprocessors in the future
they didn't think of hardware vendors not giving a singular shit lmfao
Was it designed in the 70s
i mean, look at the risc-v hardware people
it's literally a "everyone does what they feel like" situation
I've never seen driver coprocessors
How old is udi
the committee sat from 1995 and completed the specification in 2001
tldr unified interfaces are bad a lot of the time
https://web.archive.org/web/20040207162052fw_/http://www.projectudi.org/Docs/txt/UDI_advantages_and_opportunities.txt here is a whitepaper they released
UDI does IOPs, obviously you like it
i wonder if it's feasible to make a linux abstraction layer for the drivers
freebsd does that sometimes
uDRM - a portable Direct Rendering Manager
How's uDRM doing? 
still doing research on xe modeset
X E
intel X E.
What will be the oldest Nvidia card supported? I have a few quadro fx 570s lying around
Riva TNT2 
rtx 2000
I'm currently only going to do nvidia gpud with GSP
iirc 1600 series has GSP too
How hard would it be to write a driver for such an old card if one would have full documentation?
i think they mean quadro fx 750
holy shit that's ancient
idk, probably best to look at nouveau to figure out what it takes
I have a PC with AMD iGPU which has documentation for it...
Idk the one that was before GCN
for amd i only have Polaris, Vega and RDNA3 cards
But those don't have documentation
the code is open
that's my documentation
also, i have my sources
i also have access to the pci spec 
legally
What an osdev way to flex about money :^)
"I have an apple II" (I don't)
Share immediately 
I do
you wish

I have it somehow if you really want it
Any progress?
the project has been cancelled
I saw an unread udrm thread and I got excited for a moment smh my head
lil might be seeing some work soon™️
yep
no
no
Good luck
even luck won't help him
sudo rm /boot/init*
Didn't work
Try rebooting then, must be busted state 
jokes on you my initrd is called rammydisky.img
oh why :(
lil
big
if true

Would anyone be able to give me the gh link lol
I won't bother scrolling up this conversation on phone
it's dead
too much work
also yeah using lil + a small drm handler isn't very hard
doesnt lil only provide driver for intel gpu
at the moment
but it's designed to support anything
does it have like 3d acceleration?
no
ig its time to buy a intel igpu
or a dgpu

intel arcs are pretty good
do you mean porting nvidia-open? i dont really know a lot about these stuff