#weird lighting on signs

1 messages · Page 1 of 1 (latest)

twin nest
#

library level, Steam/Proton

summer anchor
twin nest
#

Desktop Linux/proton steam, no special commands, AMD Radeon RX 5500M; I'll try to reproduce on windows too

summer anchor
safe pebble
#

does this only happen with russian text?
can't reproduce this with german or english on my laptop - AMD Radeon 760M (radeonsi phoenix
LLVM 19.1.7 DRM 3.61 6.14.5-arch1-1)

#

oops, turns out I can reproduce it - when running the game in Proton instead of using the native port

#

no idea why that should make a difference, but at least here it does.
@twin nest is it the same on your machine? does this look fine when using my native linux port?

spring lodge
#

This is where having a frame capture would be helpful because could see exactly what’s wrong in the light buffer

#

My guess is what’s happening in the lit font rendering is writing a 0 to the light buffer render target which then the post process effect reads and it overbrightens it significantly

safe pebble
#

sucks that it only happens in proton, not sure how to use apitrace there

#

(for some reason it doesn't even run in regular wine)

twin nest
twin nest
spring lodge
#

Yeah the light correction effect is only applied when zoom inspecting

#

So if the buffer has bad values in it, that wouldn’t impact the rendering when not inspecting

twin nest
#

there's also this screenshot (the note in Stargazer lies on the edge of a shadow)

250501_13-21-29.png
spring lodge
#

Hmm interesting

safe pebble
safe pebble
#

I got an apitrace for running the windows version in wine where this bug happens

#

unfortunately it's 630MB 🙃

#

and I can reproduce it with llvmpipe

#

there are so many gl calls each frame -_-

thorn canyon
safe pebble
#

thanks, good to know :)
got it to work with wine (and skindeep.exe I compiled myself) now. I'm getting close to the problem but.. it's so many fucking gl calls, it's unbelievable: >12000 in a single frame

thorn canyon
#

Stab in the dark but I recently added a workaround for z-fighting in the remastered Tomb Raiders force_gl_depth_component_type_int=true

safe pebble
#

nah, it's something about a texture that is first rendered into and later used as a mask

#

in the "good" trace (from native skindeep) the mask masks the whole postit except the letter, in the bad trrace (from wine) it masks only the letters.. or sth like that. currently trying to find out where/how that mask texture is created

#

because the whole postit is supposed to be highlighted => as bright as the area around the letters in the screenshot above

#

good mask

image.png
#

bad mask (in GL_TEXTURE2)

image.png
#

@spring lodge how is that brightness mask texture created? and what kind of opengl call is used to tell it to render into that texture? that texture has id 30 in my trace, but glBindTexture(..., 30) is only called when this texture is used, apparently? (I assume it gets updated each frame?)

#

I wish I actually knew opengl :-p

#

hmm, apparently glFramebufferTexture2D() and it looks like that is only done once and afterwards they don't need to be "synced"

thorn canyon
#

If its hard to look through the gl calls you could try simply dumping the output of both traces into a text file run via apitrace replay --verbose name.trace &> dump.txt then try using diff or meld to spot any differences

spring lodge
#

The font render should use basic surf

#

Just look at the shaders that write to output=2

safe pebble
#

I think it's not the font rendering that's going wrong, but the rendering of what's supposed to become the black quad

#

the font is probably rendered on top later and that should be fine

safe pebble
#

hrm, this makes no sense, too much noise..
@twin nest do you think you can create a minimal testmap that reproduces the issue? ideally spawning the player so she immediately looks at that thing?

safe pebble
spring lodge
safe pebble
#

I don't think it's the font rendering

#

if the font stuff from the bad mask was rendered on top of a black quad it would be fine

#

this is probably exactly what's supposed to happen: write the dark quad and then the font on top, the letters themselves grey or whatever and the background of the polys containing them black

safe pebble
#

for reference, the finished frame in broken vs good

image.png image.png
spring lodge
#

It almost seems like the alpha isn’t working

#

Can you try force setting the glimp.brokenAMD variable to true?

twin nest
twin nest
#

oh and also i do see this bug on Linux port and English locale, definitely

#

also you can open the memory palace - on the same map, the note in MP is overlit

safe pebble
#

thank you very much, @twin nest !

#

with your testmap it also happens with the native version 🙃

safe pebble
#

but only on my laptop with AMD GPU and with llvmpipe, not on my desktop with nvidia, so I can compare on my desktop with nvidia vs llvmpipe.
and using qapitrace with the trace from your map is so fucking fast, it's unbelievable. compared to my other trace from sh_library01 it's a difference like night and day

#

on the downside, the bug is different now:

image.png
#

above on the good screenshot the whole note was as bright as the area around the letters - now it's the opposite, the area around the letters is brighter on the bad screenshot

safe pebble
#

but the difference is again in that brightness texture:

image.png
safe pebble
#

This is GL_COLOR_ATTACHMENT2 - on the left when running with nvidias driver, on the right when running with llvmpipe (radeonsi appaered to look the same).
The upper surfaces are from gl call 70891, the lower ones from gl call 70892 (in my sd_native2.trace that I'll try to upload somewhere soon).
So apparently the mesa driver renders something into that framebuffer (lightOutput or interactionLightBlend.frag ) here while nvidia doesn't (and nvidia looks more correct)

image.png
safe pebble
#

oh damn, of course the gl call numbers are different now that I trimmed the trace -_-

#

the interesting gl call (that adds the glitchy black blocks where the letters are) is 51347 in the trimmed trace

#

and the drawcall that uses texture id 30 aka GL_COLOR_ATTACHMENT2 to make the rendered note look glitchy is 52203

#

At this point I'm out of ideas, but maybe @spring lodge or @thorn canyon have an idea WTF is going wrong there?

safe pebble
#

hmm it kinda seems like on nvidia lightOutput = light; in interaction.frag has no effect (at least visible in game, haven't traced that yet), because changing it to lightoutput = vec4(1,0,0,1); or vec4(0,0,0,0) or vec4(1,1,1,1) doesn't make a difference

safe pebble
#

this is potentially interesting: this is from a new trace, where I modified the shaders (I wish apitrace let me do this in the tool like renderdoc does).
interactionLightBlend.frag uses lightOutput = vec4(0,0,1,1); (=> sets lightOutput to blue)
interaction.frag uses lightOutput = vec4(1,0,0,1); (=> sets lightOutput to red)
On this screenshot above is GL_TEXTURE2 (that brightness texture) in the drawcall that makes the area round the letters look weird (on mesa drivers).
on the left is how it looks when making qapitrace use llvmpipe, on the right is how it looks with the nvidia driver

image.png
#

the color on the right is #ff00ff, i.e. vec4(1,0,1,1)
so in the nvidia driver the results of interaction.frag and interactionLightBlend.frag are blended, on mesa they aren't.
(also note that this buffer is GL_RGB16F, i.e. without an alpha chan)

#

the shader used in that drawcall does this, btw:

// Get luminance at this texel
float luminance = texture(uTexture2, temp1.xy).x; // this is the texture from the screenshot

and then does calculations with it. this means that it only uses the red channel, even though this texture is RGB and, as far as I can tell, gets meaningful data assigned to .x and .y (aka .g and .b) as well (maybe an average of RGB should be used?).
so with the left texture (where the letter-blocks are plain blue, #0000ff), luminace will be 0, while on the right one (where they are #ff00ff) it'll be 1

spring lodge
#

oh wait is this the classic Nvidia is happy with uninit components but other drivers aren't

#

in that light texture only the red component is sampled from, see brightness.frag line 32

#

originally that RGB16F texture was like a R16 (normalized int texture with one component), but not every driver supported normalized 16 bit so I switched to RGB16F

safe pebble
#

must have something to do with blending

#

at the draw call that adds the blue/pink rectangles to the red background (see my last screenshot), the blend state is this:

image.png
#

the drawcall uses interactionLightBlend.frag, but I modified the line setting lightOutput to:
lightOutput = vec4(0,0,1,1);

#

and nvidia adds the colors (adds blue to red to create that pink), while Mesa replaces red with blue (in the affected areas)

spring lodge
#

out of curiosity what happens if you do like

safe pebble
#

is that GL_BLEND state (supposed to be) used for all fragment shader output, or just for the first or something (the proper color output)?

#

and assuming it's supposed to be used for all, is this behavior correct given the blend state (I haven't looked up what it means exactly yet)

spring lodge
# 
vec4 lightAlpha = light * alpha.w;
lightOutput = vec4(lightAlpha.w, lightAlpha.w, lightAlpha.w, 1.0f);
#

mainly I'm curious what's in that 4th component

safe pebble
#

in which shader?

spring lodge
#

because if it's different b/w the two drivers that would explain it

#

in interactionLightBlend.frag

#

that's what the draw call is using right?

safe pebble
#

yes

spring lodge
#

oh wait I think I understand now

#

or wait hm

#

I wonder if this fixes it

# 
lightOutput = light * alpha.w;
lightOutput.w = alpha.w
safe pebble
#

nope

#

would be weird as the buffer behind lightOutput doesn't have a .w/.a chan

#

(the other suggestions didn't help either)

spring lodge
#

well it would use it for alpha blending still even if the destination doesn't have an A

safe pebble
#

one potential source of weirdness certainly is that GL_BLEND_SRC is GL_SRC_ALPHA, even though the source doesn't really have an alpha chan

spring lodge
#

but yeah it seems like the blending is applying to both location 0/1 on nvidia but only location 0 on the mesa/amd one?

safe pebble
#

maybe blending then assumes alpha = 1 in one driver and alpha = 0 in the other?

#

my brain is mush at this point, but with the blend state above, is it correct that dst_R = (src_R / src_A) + dst_R, where src is what is currently computed in the shader and dst is what is already in the buffer?

spring lodge
#

does changing it to RGBA16F fix it?

safe pebble
#

(in that case, with src_A = 1 it'd be src_R + dst_R which is what we're seeing on nvidia, and with src_a = 0 we're in a world of shit because we're dividing by 0)

#

no, doesn't fix it

#

not even when explicitly setting lightOutput.a = 1;

spring lodge
#

I found a post from 2005 which says alpha blending on floating point output buffers only works on NV hardware lmao

#

There must be an extension

#

I wonder if that’s what it is though

safe pebble
#

one would hope that this has changed by now

spring lodge
#

It certainly explains the result we’re seeing though

safe pebble
spring lodge
#

It does work on the PC AMD driver though

#

Is it possible whatever the extension is just isn’t implemented in Mesa?

safe pebble
#

qglTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA8, glConfig.vidWidth, glConfig.vidHeight, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL); doesn't help either

#

here's the trace I used for the screenshots above, with the modified shaders that write just red or blue

📎 sd_bla_min.trace.zip
#

(search for "float")

#

it mentions float textures several times without any note of extensions, apparently the only specialty is that (unlike with normalized int textures) the blending result is not clamped to [0, 1]

spring lodge
safe pebble
#

interaction blend

spring lodge
#

I think try an rgba texture set it to 1 in regular interaction but don’t change it in blend?

#

Because if you set it to 1 in blend then it would overwrite all the transparent pixels which is the bad result anyway, like that would break it on Nvidia also

safe pebble
#

no difference

spring lodge
#

Also I think you have src and dst backwards maybe? Isn’t dst is what’s in the buffer right now and src is what you’re writing,

#

Dst alpha is set to 1

#

In the blend state

safe pebble
#

"where src is what is currently computed in the shader and dst is what is already in the buffer?"

#

idea: you could try to fuck up the shaders so it looks the same kind of broken on nvidia (or AMD windows or whatever)

#

maybe we can deduce what's going wrong from there

spring lodge
#

Wouldn’t setting the interaction blend alpha to 1 just break it on Nvidia though?

#

Like I’d expect you get the same result then

safe pebble
#

I'll try

#

no

spring lodge
#

Huh

safe pebble
#

gotta take a break now, otherwise my brain will run out of my ears

spring lodge
#

Haha sounds good, thanks for all the research!

#

I have to go to jury duty but I will look at it later tonight hopefully

safe pebble
#

just got one small idea: where is that blend func set? a few calls before glDrawElements() that adds the blue/pink crap is a glBlendFunc() call

#

I could try if using (GL_ONE, GL_ONE) makes a difference

#

(no worries if you answer that later, have fun at jury duty, I just wanted to get the idea out of my head)

#

glBlendFunci() (from GL4.0) might be relevant, and maybe also glEnablei(GL_BLEND, idx)

spring lodge
#

My understanding is glBlendFunc is supposed to set it for all targets bound

safe pebble
#

maybe it doesn't in Mesa?
@thorn canyon any idea?

safe pebble
#

ok, found where that blending is set, RB_GLSL_CreateDrawInteractions() in draw_glsl.cpp, GL_State(GLS_SRCBLEND_SRC_ALPHA | GLS_DSTBLEND_ONE | GLS_DEPTHMASK | backEnd.depthFunc);
however, GLS_SRCBLEND_ONE doesn't help

#

calling

qglEnablei(GL_BLEND, 1);
qglEnablei(GL_BLEND, 2);
qglBlendFunci(1, GL_ONE, GL_ONE);
qglBlendFunci(2, GL_ONE, GL_ONE);

after the GL_State() call doesn't help either (in the red/blue testcase). I might really be out of ideas now

spring lodge
#

If you change interaction blend to just write vec4(0,0,0,0) to light output, what happens?

safe pebble
#

based on the original shaders or my modifications that write red/blue?

spring lodge
#

Instead of writing the blue in blend just write all 0s

safe pebble
#

if interaction.frag writes 0,0,0,0 and interactionLightBlend writes what it's supposed to write, it looks ok in both cases
but I think you meant the other way around

#

with interaction writing red and interactionLightBlend writing 0,0,0,0 it looks broken on mesa and ok-ish on nvidia, like before

#

same when interaction writes light (like originally) and interactionLightBlend writing 0,0,0,0

spring lodge
#

Ok, it seems like just in general the alpha blending is not working on mesa

safe pebble
#

why alpha?

#

even with glBlendFunc(GL_ONE, GL_ONE) it behaved like this

#

blending in general seems broken for that brightness texture thing

#

with or without alpha

spring lodge
#

Oh, yeah I meant just blending in general

#

Does Mesa have a software render mode?

safe pebble
#

yes, llvmpipe, that's what I'm using most of the time

#

(only occasionally I verify on my laptop on its AMD iGPU)

spring lodge
#

Ah ok

safe pebble
#

you could compile llvmpipe for windows to see if it behaves the same there - you can actually build an opengl32.dll from it

spring lodge
#

In the original apitrace, are the glBlendFunc etc calls definitely are buffers are set to color attachment 0 and 2?

safe pebble
#

and the screenshot above from the glblend state was from that glDrawElements at the bottom, which is what added the blue/pink crap

safe pebble
#

but it's also clear from the code that blending is enabled: the only place using interactionLightBlend is RB_GLSL_CreateDrawInteractions() and right before that call there's GL_State(GLS_SRCBLEND_SRC_ALPHA | GLS_DSTBLEND_ONE | GLS_DEPTHMASK | backEnd.depthFunc);
(ok, not sure where the glEnable(GL_BLEND) is, but why would you configure src and dst blend mode if you wouldn't expect blending to be enabled)

spring lodge
#

My theory is maybe this is the issue

#

Because there is an integer render target, even though it’s not active, it thinks it needs to do per-color buffer blend state but it might be that the state is only being set for color buffer 0

#

which should be like the “global” state since we don’t set per target

#

I have no idea if it’s true but haha that’s my theory

#

Wouldn’t be surprised if it’s related to integer render targets somehow though since the other bug was also

#

Which makes me figure the path is not used that often

safe pebble
#

could be

#

however, I added these calls in RB_GLSL_CreateDrawInteractions(), without effect:

        GL_State(GLS_SRCBLEND_SRC_ALPHA | GLS_DSTBLEND_ONE | GLS_DEPTHMASK | backEnd.depthFunc);
        // the next 4 are new
        qglEnablei(GL_BLEND, 1);
        qglEnablei(GL_BLEND, 2);
        qglBlendFunci(1, GL_ONE, GL_ONE);
        qglBlendFunci(2, GL_ONE, GL_ONE);

        R_GLSL_SetActiveProgram("interactionLightBlend");
#

so even if the blend state was only set for color buffer 0, it should now be set for the other ones as well

#

(I wasn't sure if I needed to use index 1 because it's for layout(location = 1) out vec4 lightOutput; or 2 because it's GL_COLOR_ATTACHMENT2, so I set both)

safe pebble
#

would it be much work to turn that integer color attachment into normalized integer or something?

#

(what is it good for anyway?)

#

if I use qglTexImage2D(GL_TEXTURE_2D, 0, GL_R8, glConfig.vidWidth, glConfig.vidHeight, 0, GL_RED, GL_UNSIGNED_BYTE, NULL); for customMaskTexture, the testlevel looks like it should, so your hunch about this being related to the integer texture (AGAIN) might've been correct

#

(unless something else broke by changing the custom mask texture type which just accidentally "fixes" the notes issue)

thorn canyon
#

I was concerned there were more integer render target issues when looking through the code when fixing the other bug. @safe pebble are you able to delete the if statement @spring lodge points out above and see if that fixes the problem?

spring lodge
safe pebble
#

why does the mask require an integer texture anyway?

safe pebble
#

those logic ops have been around since opengl 1.0, but (not-normalized) integer textures are a lot more recent, AFAIK, so the logic ops shouldn't need them?

safe pebble
spring lodge
#

So like you mean have the shader convert from the normalized float to the uint representation and then do the mask?

safe pebble
#

yeah, maybe (would just have to multiply with 255 and cast to int, right?) - where is that mask used in the shaders?

spring lodge
#

Brightness.frag and Outline.frag

safe pebble
#

how do you get those different integer values into the texture/buffer?

#

ok, set with customMaskPass.frag

#

@thorn canyon do you know of those logic ops, when used with regular normalized integer textures like GL_R8, operate on the integer values?
So when storing 8.0/255.0 in the shader, it's treated as 8u by the logical op?

#

if so, in theory it should work to make that texture GL_R8 and

  • in customMaskPass.frag use resultCustom.x = float(uCustomPostValue) * (1.0/255.0); (or similar, whatever makes the glsl compiler happy)
  • in brightness.frag use uint inspectMask = uint(texture(uTexture1, temp1.xy).r * 255.0) & POST_INSPECT_LUMINANCE;
  • in outline.frag use int mCenter = int( uint(texture(uTexture1, uvCenter).r * 255.0) & outlineMasks ); (and similar for top/right/topright)
thorn canyon
#

Do you have the api trace for download somewhere?

safe pebble
thorn canyon
#

Honestly I can't answer your question. I don't normially work in this part of the driver, not my strong area

safe pebble
#

ok, no problem :)

#

I'll try to find an old tutorial for GL logical ops then

thorn canyon
#

Think I maybe found the issue just need a second to understand the code I was blindly editing 😛

safe pebble
#

\o/

spring lodge
safe pebble
#

hmm yeah, probably, I guess I should take screenshots of the working state first for comparison

#

Beware that OpenGL specs also specify glLogicOp() logical operations being disabled with sRGB framebuffer target.
not relevant here, I think, but good to know
(from https://unlimited3d.wordpress.com/2020/01/08/srgb-color-space-in-opengl/)

Unlimited 3D
Kirill Gavrilov Tartynskih
sRGB color space in OpenGL

RGB (red+green+blue) is the most commonly used color model, and computer graphics is not an exception. Mixture of red, green and blue (also called primaries for RGB) with different proportions allo…

thorn canyon
#

Ok, I think I understand what is happening. There is some blending code that is skipped if an integer buffer is detected. The code is to naive is take the number of color attachments and uses that count to check against the integer buffer mask so it thinks the second color attachment is the integer buffer since its attached at color attachment 1

safe pebble
#

sounds very plausible

#

is using integer color attachments so unusual or why do we run into so many bugs there?
(I mean usually I find one mesa bug every few years or so :D)

#

and it's highly unusual that AMDs windows driver works better than radeonsi, never happened before

thorn canyon
#

well its probably just a corner case that nobody normally hits when using them

#

This should solve the issue

#

I'll be adding piglit tests for these issues so any regression in future should be caught

safe pebble
#

yes, this works, thanks a lot for fixing it! :)

spring lodge
thorn canyon
#

No problem. All your help tracking it down made it much easier 🙂

spring lodge
#

definitely, thanks a bunch, I woudl not have been able to figure this out as quickly since i don't have an easy way to debug on SteamDeck haha

safe pebble
#

but those have been supported since GL3 or so? they should've been used more at some point in the last 15 years or so?

thorn canyon
#

Is this that last of the linux/mesa specific issues?

safe pebble
#

if this also fixes the one that looked the same but only happened in wine/proton, then I think so

spring lodge
#

how long does it usually take for Mesa driver changes to be pushed up to the Steam Deck drivers?

spring lodge
spring lodge
safe pebble
#

probably pretty few since vulkan came out, but the old ones are still played and some source ports (and TDM) do advanced shit

#

reminds me I wanted to check if TDM has the same line issue, I think someone mentioned that

spring lodge
#

well and also Steam Deck greatly increased the number of people gaming on linux

safe pebble
#

didn't some tomb raider game use opengl?

#

in its linux port

#

I vaguely remember something like that..
but then of course again, someone has to report and investigate such glitches

spring lodge
#

"The port is using Vulkan on Linux" for Rise of the Tomb Raider

safe pebble
#

and nowadays when native linux ports don't work people just use proton and the directx renderer 🤡

spring lodge
#

right

safe pebble
#

Tomb Raider 2013 used OpenGL

spring lodge
#

ah ok

#

but that was 12 years ago maybe it worked then haha

thorn canyon
safe pebble
#

do they stick to mesa releases or do they push git snapshots or custom patched versions?

safe pebble
thorn canyon
#

I'm not entirely sure. I know that a while ago it was getting very stable and bugs were fixed but not landing for a long time. Someone got assigned to making sure we update more regularly but not sure if they just stick to official bug releases

#

*getting very stale

#

Even if they do sitck to mesa releases the bug releases are dropped fairly often. For example the first fix for the begin calls went out in a release today

safe pebble
#

funny, now 25.0.x is more up-to-date than 25.1.x

safe pebble
#

I wonder if as a workaround it would've helped to make the integer buffer the last color attachment

#

(so when iterating the active ones ctx->DrawBuffer->_IntegerBuffers & (1 << i) wouldn't match)

#

yep, seems like it does \o/

safe pebble
#

verified on my laptop with radeonsi

#

might even work around similar bugs we just haven't noticed yet

twin nest
#

huh. I think it also improved the lighting on the vent locks

#

if you press Shift, zoom into the locked vents and wiggle your camera around, unpatched versions have this weird light spots

safe pebble
#

huh. I wonder if that is another mesa bug or the same one as this

#

(probably someone who understands the mesa source should grep for DrawBuffer->_IntegerBuffers and check all occurrences - good news is, it's only 10, or 14 if you grep for only _IntegerBuffers)

spring lodge
spring lodge
# twin nest huh. I think it also improved the lighting on the vent locks

There were some other issues on Proton/Steam Deck with the light brightness effect which makes sense given the bug

If you had a volumetric light it probably alpha blend and instead would make sense that it’d over-bright it. So there was a bug on Steam Deck where some of the notes were too bright

Would make sense in the vent cause there’s like dust and stuff which also is interaction blend and that could cause problems as well

twin nest
#

not in the vent, on the vent 🙂 #1344023574004105226 message

safe pebble
#

@thorn canyon I wonder if it would be easier to set up _IntegerBuffers in the way most of the checks erroneously expect it to be than to fix all the checks

thorn canyon
safe pebble
#

and probably the other ones are similarly bad documented

#

GLbitfield _IntegerBuffers; /**< Which color buffers are integer valued */
I mean that kinda explains what it's for on a very high level, but the meaning of the bits is also kinda important?!

thorn canyon