Use Reshade for motion vectors

[silver pollen]

Using reshade's motion vectors to enhance frame gen quality (by making a reshade version of LSFG or somehow hooking to the reshade motion vectors)

[toxic void]

I don't think it will happen, but if it were to, I think it would be likely that the dev would build his own hooks from scratch, and there would have to be a list of games that it just doesn't work on which i think is being avoided because lossless is supposed to be universal.

[silver pollen]

Interesting, although it could still be universal, only using the motion vector when the game supports it, and reverting to the base LSFG model when not. I cant remember any game ive tried where reshade + plugins is incompatible, only maybe online games with anti-cheat, which not many people use Lossless on due to input lag (and 60 to 120 is stable enough to not need motion vectors). But it could maybe require dev of 2 models (one with Motion vecs and the other one without) which could be too tedious to realistically do..

[coral crescent]

you forgot that motion estimation also takes performance so unless ths makes lsfg way better, this would just be idling here

the minor updates since lsfg 3.0 have been for quality/performance

[silver pollen]

Maybe, but from my personal experience, using motion vectors from ReShade with RTGI never gave me a big performance hit. Maybe the model could take a long time to estimate it. In that case yep it could be an issue

[foggy remnant]

Yeah reshade motion vectors only seem to cut off a max of 30 fps in my experience

And that’s at very extreme settings

More realistic hit would be 5-10

[misty swan]

Perhaps a universal solution would be automatic vector detection, interpreted by movement in the game, plus an automatic crosshair detection system so the relative movement is also detected.

Like instead of tailoring to specific games, it detects the engine the game runs on.

That reduces the complexity of implementation, right?

[languid steppe]

it would 100% be a great idea, just have it be able to be turned on and off so LS is still universal

[silver pollen]

Yoo why did this come back from the dead, it wouldn't really need to detect the game engine as reshade doesn't need this (only the graphical api)

[silver pollen]

LS needs to stay "outside the game" not adding any hooks to the actual engine, it would more be like LS detecting that reshade is present and using its motion vectors (perhaps with a LS reshade module that exports this info to LS) so the actual LS can stay completely without game interaction

[brazen valley]

I suppose this would primarily tackle artifacting but not latency right ?

If it still sits outside the game

[silver pollen]

It would definitely remove a lot of artifacting, not sure about latency though, not sure if reshade could add a lot of latency

[brazen valley]

If you could also inject the whole LS algorithm into games that could reduce latency as it would sit before the frame buffer like in game FG

[silver pollen]

Yeah but it could maybe a bit break the "universal" philosophy, and could potentially get you banned in online games

[silver pollen]

But toggleable injection in some games could be pretty good (fallout)

[misty swan]

Detection is not the same as hooking. It's -looking-

Just trying to suggest ways to make frame generation smarter, especially for people who can't afford the best GPU

[brazen valley]

There’s that other software which injects FG / scaling if I’m not mistaken

Optiscaler

But I think it relies on the game having an upscaling / FG option in first place

[unkempt mason]

besides needing dlss or sometimes fsr the biggest limit of optiscaler's fg is that it has to be running in dx12

any dx12 game with dlss or fsr is pretty much granted to have fg already

since LS doesn't use motion vectors in general the first biggest challenge is to tell LS how to use MVs

and after building that entire system the whole question of where to get MVs is a much easier task comparatively

reshade motion vector shaders don't even need to be injected into the game to work since they only need the screenspace and no depth info

that's how all optical motion vectors work besides engine MVs which actually do need integration access

[misty swan]

I imagine many people would want to contribute to this.

I also, need to start researching how to live without DirectX, because I'm going with Linux.

30 seconds later...

Well, hello Vulkan.

[undone peak]

LSFG's optical flow is working the very same way ReShade MVs do

it's just that it prioritizes performance while keeping the quality solid enough

if it was like e.g. marty mcfly's newton optical flow, it would tank the performance noticably

[misty swan]

Please explain the emphasis on staying "outside the game". I'm still learning

[hot jolt]

It’s just a general philosophy it follows to maintain maximum compatibility

[unkempt mason]

in general that means not needing to collect any data that requires game access

like accessing the game's geometry or depth buffer requires ingame access

but something that's just on your screen like the colors aka screenspace is what LS uses

and that wouldn't need to go inside the game to get

[unkempt mason]

can i ask something

[unkempt mason]

is this even true?

i thought LS was just input frame --> black box --> output

and there were no motion vector function

[hot jolt]

LS does use motion data it’s just that it estimates it itself

[unkempt mason]

because i vaguely remember ths saying something like "everything is handled by the model"

[hot jolt]

Then it uses said data to warp pixels

[unkempt mason]

and the only "debug view" i've ever seen was the UI one

[hot jolt]

Actually there was an internal debug view of it at some point

Lemme see if I can find a screenshot or video

[unkempt mason]

that's really cool so it basically has it's own ai optical flow already

[hot jolt]

Found it

https://media.discordapp.net/attachments/1236717847011393546/1383823564255527115/2025-06-15_18-00-02.mp4?ex=696e471f&is=696cf59f&hm=2f713f8936354d99770c1a5299e12404bc32f8b984d18eb38dd6373d70f59932&

This was during the development of lsfg 3.1 if I remember correctly

[unkempt mason]

yeah i was surprised how new the date says it was

[hot jolt]

Actually the option is probably still exist it’s just THS disables a few config file options for public builds

[unkempt mason]

i don't know what to think of the quality of the MVs lol

it could just be the dev view is wack

[hot jolt]

No it’s not that

That video specifically was just a showcase of LSFGs current unideal pattern/stair handling

[unkempt mason]

i see yeah

far away and close up both read the same color

i just didn't know why left and right had different colors

oh that's probably the very thing that the flow scale tweaks

anyway thanks for showing me that

[silver pollen]

If i remember properly reshade with addons support accesses a bit more than just screenspace to get much proper motion vector

[charred sphinx]

is this relo's vid?

[hot jolt]

Yes