UI scaling blur

[torn trail]

@sand gazelle A thread for you

[sand gazelle]

I've been trying to solve the problems of my game icons (in particular RecipeSlot icons) looking blurry at UI scale 125, at which icons are 40px wide:

At present icons have linear-minification, linear-magnification and linear-mip-level; these flags result in a D3D11_FILTER_MIN_MAG_MIP_LINEAR SamplingMode which sounds like trilinear filtering for both shrinking and expanding.

My understanding of trilinear filtering is that it uses linear interpolation to remove the jump that's visible when you slowly zoom in or out of a texture and switch from one bilinear-filtered mipmap to the next. I don't see why this is desirable for static images like icons, though this opinion is non-expert as as I haven't done any DirectX/3D programming.

That said, I believe trilinear filtering is the cause of my blurry icons. To remove my GPU from the equation I have reimplemented trilinear filtering using Python PIL and the results look similar.

Here is a manual trilinear-filtered 40px image (a blend of 0.75 * bilinear-filtered 32px icon + 0.25 * bilinear-filtered 64px icon).

And here is a manual bilinear downscale of the 64px image.

The trilinear one looks blurry while the bilinear one doesn't.

Some other people have the blurry icon issue but many do not. I don't know why this is but can make some guesses:

• If you have a higher UI scale then you have more detail to begin with and you get closer to the highest detail (64px) mipmap, so the blurriness effect causes less damage.
• If you have UI scale 100 or 200 then the icon size is the same as a mipmap and there is no interpolation anyway.
• If your GPU renders the game at a higher resolution than your monitor then (guess) maybe you go straight to a resolution above effective UI scale 200 and only have the highest-detail mipmap - hence you have bilinear filtering not trilinear filtering and don't have the problem to begin with.
• Maybe a straight line interpolation between (e.g.) a 32px mipmap and a 64px mipmap with a straight line isn't the best way to smooth out mipmap transitions - so perhaps more recent GPUs / GPU drivers implement D3D11_FILTER_MIN_MAG_MIP_LINEAR by keeping the line continuous but jumping to the higher-resolution mipmap far more quickly (so 2x linear + 1x nonlinear instead of trilinear). This is 100% speculation.

I found a post of someone complaining of new blurriness back in 2019 https://forums.factorio.com/viewtopic.php?t=67563; as a result of this posila added the legacy-gpu-no-mipmaps flag. Enabling this does solve the blurriness problem for me but extremely crudely - by reducing the number of mipmaps to 1, trilinear filtering becomes impossible. As a result there's a notable degradation in visual fidelity when zoomed out. IMO this approach is very much throwing the baby out with the bathwater: I believe my problem isn't the presence of mipmaps, it's interpolating between them.

The easiest way to fix the issue (for me at least) is to remove linear_mip_level from SpriteOption::Icon.flags.bits. This turns out to be a 1-bit change to factorio.exe. The fix is similar to what another user suggested on the 2019 forum thread.

I think the result looks better on my GTX 1070 than on a more modern GPU of someone who had much sharper icons to begin with. With UI scale 125, LHS is @stuck raptor 's RTX and RHS is my modified factorio.exe. In particular see the differences in the tank, arithmetic combinator and passive provider chest.

And here are pictures of a fully zoomed-out assembler making stack inserters (original trilinear / original with legacy-gpu-no-mipmaps / new modified). The original one looks blurry, the legacy-gpu-no-mipmaps one is missing lots of detail and the new one looks the best to me.

[torn trail]

If this is a suggestion or request, you should probably put it on the forums, not here. Little chance of developers seeing it here

[sand gazelle]

If I were the only person who played the game then I'd ask for an extra settings flag to disable linear_mip_level in icons; I'll leave it to the devs to judge what best to do, if anything.

Another thing: when I first posted about the blurriness issue, JG made the sensible suggestion of writing a mod to provide 40px mipmaps myself (meaning icons shouldn't get filtered) but it didn't work out. Here's the 40px icon I tried to use:

and here is how it appeared in game, between unmodified storage / active provider chests. If you look carefully you can see the red chest looks a bit smaller and blurrier - and I have traced this down to an actual bug.

What's happening is that for UI scale 125 and a 40px icon, we have sprite->originalWidth = (short) 40 and sprite->scale = (float) 0.8 (which is 1/1.25). GuiGraphics::drawSpriteToRectangle contains inlined copies of Sprite::getFinalWidth which computes ceil((double) sprite->originalWidth * sprite->scale). Inside this, the mulsd instruction multiplies a double (40) by a float (0.8ish) to give the result 32 + 2**{-28}, which ceil rounds up to 33. This results in an incorrect xscale and the icon gets rendered as 38x38 instead of 40x40. This is the cause of the excessive blurriness. I have verified that by adding a bit of tolerance (say 1e-6), the game does render the 40px icon at the correct size. I don't know whether this affects anything in normal gameplay, but it does mean that JG's suggested fix didn't work.

There are some less important loose threads I encountered while debugging this issue, for instance the atlas flags don't always appear consistent with how sprites are rendered. Specifically: the atlas containing the icons has the not-compressed flag; I have looked inside a saved atlas-cache.dat and can see the uncompressed icon data there. However I believe that the icons have sprite->flags.compressedSpriteFormat == CustomBC3_YCoCg_A and so are rendered compressed; I have also verified that with a scale 1 rendering the pixels differ slightly from the true values, which is what you expect from GPU compression. So I can't see the rationale for storing the uncompressed data instead of just the CustomBC3_YCoCg_A mipmaps.

[sand gazelle]

i raised it last week but it sounded like a 'working as intended' - i'm trying to find other users who have blurriness issues (there are some) to see if the fix works for them

[sand gazelle]

the fix also works a charm on two potatoey laptops i have (integrated graphics etc)