Hmm does the Stencil State look good for | Official Unity Discord | Page 1

vapid charm Oct 19, 2022, 1:13 PM

#

testing a thread in here instead

#

I haven't used renderdoc that much yet.., trying to find the pipeline state...

shadow igloo Oct 19, 2022, 1:14 PM

#

vapid charm Oct 19, 2022, 1:14 PM

#

ah yes of course.., I just made a new capture.., I'll check once I find my way to the correct screen from unity (the editor has so many screens that renderdoc catches!)

#

is this what you're thinking about?

#

oh what.., greater or equal..

shadow igloo Oct 19, 2022, 1:17 PM

#

🤔

vapid charm Oct 19, 2022, 1:18 PM

#

oh no hang on.., that's just the depth. Thought for a second it was the stencil comparison function

#

I must admit the read mask is something I don't grasp. I can't see it affecting anything though

shadow igloo Oct 19, 2022, 1:21 PM

#

So it's passing the depth and stencil test (according to the overlays), but it's not writing any value?

vapid charm Oct 19, 2022, 1:22 PM

#

it's only writing when the value was 0 (as opposed to where it's already written 255)

#

Here the green is my stencil overlay test. White = 255 and grey = 12 and black = 0

#

this is the result

shadow igloo Oct 19, 2022, 1:24 PM

#

I think the issue is that the WriteMask, which is set to 12 there, is making it only write to those bits and so it won't write 0 to the other bits.

#

And it looks like nothing is changed because all the bits are set to 1 when it's set to 255

vapid charm Oct 19, 2022, 1:25 PM

#

ooh

#

aah.., I should think of the bits themselves then probably?

shadow igloo Oct 19, 2022, 1:26 PM

#

So the correct stencil state should be WriteMask set to FF, ReadMask doesn't matter because the comparison func is set to always, and Ref should be set to 0C/12

vapid charm Oct 19, 2022, 1:26 PM

#

I guess I shall test writing 0 instead of 255 for that part and see where it fails then

#

oh wait.., just reading what you typed

#

I don't get your suggestion. If writemask is set to FF then it'll be 255 right? I want to set it to 12 in the case above.., which it does for the pixels that already are 0.., but I'll try setting the objects that currently are 0 to 255 and see what happens

shadow igloo Oct 19, 2022, 1:28 PM

#

The WriteMask determines what bits the GPU is allowed to write to in this draw call. The way it's set up now, it's trying to write the ref value, which is set to FF (255), but the WriteMask is only allowing it to write to the bits that make up 12, so it ends up writing 12

#

Except it can't write 0 to the other bits, so it's more like an OR operation

vapid charm Oct 19, 2022, 1:29 PM

#

ooh., that makes much more sense!!

I thought write mask was what I wanted to write.., not that ref did that

#

so grateful for your time!

shadow igloo Oct 19, 2022, 1:30 PM

#

No problem

vapid charm Oct 19, 2022, 1:30 PM

#

yeah now it's more or less flawless.., haha. Hate it when it's almost correct and it causes the weirdest bugs

#

thanks again!

#

hit me up if you ever are looking for tech art jobs in sweden

#Hmm does the Stencil State look good for