LS2 spatial upscaling

[steel wave]

Consider this being a "question" rather than a "suggestion".

I've been toying around with LS1 recently, and I'm glad to say I'm impressed. It's more efficient than FSR, captures more fine details than FSR, goes easier on the UIs than FSR, and overall is simply better than FSR despite theoretically being the same thing - a spatial upscaling algorithm.

Looking at how FSR works (coz it's open-sourced), I think LS1 is some sort of an enhanced fork that replaces FSR's custom Lanczos-based scaler for a custom Catmull-based one (its edge smoothing is nowhere near as aggressive as the FSR implementation). It also replaces FSR's contrast-based sharpening with something that seems to be luma-based (hence the fine detail preservation).

Is there any room for further improvements like LSFG? Temporal upsampling tech is definitely going to be a standard in consumer-grade 3D rendering for a while, but I think there's a lot to be done in terms of unaliased image upscaling (as FSR and LS1 seem to favor aliased images by design, especially via TAA) and performance (since the current performance mode seems to run the whole thing on a less precise grid). Any thoughts?

[umbral tendon]

LS can't access to jitter/ even jitter the camera in the game.

if you want temporal AA in any game then the best bet is to use ReShade

& if u want updates on the improvements on upscaling in ReShade then follow this guy's shader updates

#804451693853016085 message

[flint lichen]

i assume ths will go back and continue to work on LS1 after LSFG is in a good place, but i couldn’t tell you for sure, only way to find out is to wait and see, honestly

[upbeat nacelle]

Since optical motion vectors exist and color, luminance, and contrast also exist for edge detection realistically the biggest difference in the missing data from lack of game integration would be easily differentiating UI menu elements by ignoring them with depth and more accurate motion detection from engine motion

So I’d imagine an fsr 2 that has trouble with UI

Lot more potential though since machine learning is an option

[steel wave]

Actually, I've been revisiting some Playstation Nixxes ports recently

Mainly Insomniac games

While they have FSR 3 too, they also have something called IGTI (Insomniac Games Temporal Injection) which is basically an in-house upsampling tech that was designed with the OG PS4 in mind

Since it was designed before real-time upsampling as we know it today, it works in a quite sophisticated manner:

1. The game draws an antialiased LQ frame
2. The game draws a new LQ frame, but moves it horizontally and vertically one pixel per axis
3. The hardware renders next frames until the image hits every corner of the display.
4. From now on, the game uses the last frames to reconstruct detail on-the-go.
5. Now, the game draws LQ frames that look like high-res HQ frames.

Rinse and repeat

It's basically a very early FSR 2 that uses pixel extrapolation instead of motion vectors (just like LSFG is a spatial-based answer to FSR FG)

tbf I think such upscaler is perfectly achievable within LS and its post-process nature, as it would require nothing but an anti-aliased image

It also has more in common with checkerboard rendering rather than temporal upsampling as we know it

Plus it's designed around low resolutions like 540p

(at least if we're talking about image being reconstructed into 1080p)

[umbral tendon]

the moves horizontally & verticl one pixel per axis is called jittering, LS doesn't have access to the game's camera and does not have history of the games frames to even do any kind of TAA.

[umbral tendon]

...

[open umbra]

oh by the way LS1 is not a fork of FSR it uses ML like lsfg

[steel wave]

Yeah, but it was used to tweak the UI detection threshold and scaling parameters

Having ML to do its thing during the development cycle and having ML to do its thing in real time are two very different things in the long run

AFAIK same thing applies to LSFG

ML was used to tweak UI detection and extrapolation maths to reduce smudging and artifacts we see in e.g. AFMF

[open umbra]

i'm talking about ls1 and it is fully ML based

not just for parameters

[umbral tendon]

LS is a neural network based spatial scaler

it's basically like Anime4K but with a much less processing cost

[steel wave]

So... it works like Google's RAISR? 🤔

It's a ML-based spatial scaler too

And it works in real-time too

tbf I was sure LS1 was just an enhanced FSR1, but since I'm using it on FHD display I couldn't see it shine in "QHD -> 4K" use cases

I heard it does wonders on higher resolutions