[URP] Stencil Portals

[brave beacon]

Hello,

I am working on a project where I intend to make use of Stencil-based Portals. I am currently using Unity 6.1 URP.

I do not wish to use a Render Texture for my approach, as the Stencil method has many advantages (namely performance).

My set-up currently uses two cameras - the main Base camera and the portal Overlay camera, while the Overlay is part of Base's stack.

I already have the movement and direction of the camera set-up correctly, and I only need to find a way to correctly apply the Overlay camera's output to the desired portal surface.

I am looking for either of these two solutions:
A) Have the Overlay camera capture the entire screen and using the stencil buffer render only where the stencil buffer is equal to desired value (meaning, the portal surface has a shader that sets stencil value to desired value)
B) Use the camera's output and use it as an input to a shader (applied on the portal surface) and scale it to the surface's UV (this works if using Asymmetrical projection), but without rendering to a texture (RenderTexture; if that is even possible).

I already have the method A) sort of working, however one major thing troubles me - for unknown reasons, the Overlay camera doesn't seem to write to the depth buffer at all, so geometry visible behind the portal is displaying in the portal as well.

For the method B), I am unsure whether it is possible to use the camera output in the shader without having to use a RenderTexture as intermediate step, which (at least to my understanding) could mean hurting performance.

Do you have any suggestions how to make this work in Unity or alternative approaches to this problem?
I'd be very thankful for any suggestions.

[daring edge]

so geometry visible behind the portal is displaying in the portal as well

This is because the camera is behind the portal. You need to set the camera's clipping plane to the portal plane, which is an oblique projection matrix. Should be able to find tutorials explaining that setup, e.g. https://danielilett.com/2019-12-18-tut4-3-matrix-matching/

[tame wyvern]

I highly recommend watching the rendering section of this talk by Portal 2 developers. You can hear straight from the horse's mouth all the problems they needed to solve to make stencil portals work, including oblique matrices. They even coined a term for this problem, Banana Juice.

https://youtu.be/riijspB9DIQ?si=P_PeVD-BDufpbuH3&t=276

[daring edge]

Also to clarify. Afaik overlay camera's don't really "capture the entire screen", they just render on top of the last camera in the stack. But that should be fine as long as you render the portal/stencil mask before that camera renders, and can override the stencil test during rendering the overlay camera.

Though that setup probably limits it to only rendering one portal unless you add additional overlay cameras.
iirc in the past I've instead used a normal camera for rendering portals, moved it into place and called UniversalRenderPipeline.RenderSingleCamera, that way it should be possible to repeat that with different stencil values. (Though I think that function is now obsolete so you'll have to find a suitable replacement)

[brave beacon]

Thanks for the answers everyone!

In my case, this was really caused by the Depth buffer not clearing properly for the portal surface (not the 'banana juice' as the Portal devs put it), but it seems I can clear the depth buffer either with a Render Pass or a Shader and so far it works.

[tame wyvern]

For performance, you should also override the cullingMatrix of the portal camera to constrain the draw calls to things visible through the portal. That is also mentioned in the talk at some point.

[brave beacon]

Interesting, thanks for the info, I'll look into the replacement for the function!

One things I've been really wondering about is if it's possible to skip the RenderTexture step and apply the camera's output directly to a surface.

I kind of want to try a slightly different approach using Asymmetrical projection, where the camera's frustrum basically matches the (square) portal surface exactly. I got it working great with a RenderTexture, though ideally I'd want to avoid rendering first to a texture and then apply it, if I could - if there's any meaningful performance difference.

In theory, the Asymmetrical projection should render only what the camera will ever see (I can provide a screenshot in a sec) and it has the advantage of making the near-z culling incredibly simple (just by taking the dot of player's distance from the portal and the portal's forward vector)

This is the asymmetrical projection I mean.

Here's an article that talks about it as well: https://jsantell.com/portals-with-asymmetric-projection/

In short, the camera is rotated perpendicular to the portal surface, while it copies the relative position of the player exactly. Then by just manipulating the projection matrix, it matches the corners of the portal surface and thus it draws only what you would actually see if looking through the portal.

Works great (the portal effect is the same), but I noticed that when using a RenderTexture that would then be applied to the portal surface, the resolution of the portal suffered up-close

Now I am not certain, but I think if I could make the portal surface use a material with a shader that takes the output of the camera directly, I could achieve better resolution.

[tame wyvern]

Are you considering this as a potential performance improvement over stencil portals to solve the same problem of just rendering portals? Or is this solution giving you something that stencil portals can't?

[brave beacon]

Well, it's primarily a convenience (apart from testing a different approach), since if I could configure the camera input used per-material, it could make it easily scalable as well. It's just the question of whether it is needed to use RenderTextures or I can skip them directly for a performance benefit (if any?).

[tame wyvern]

It's not possible to feed the output of a camera into a material without the use of a render texture. Cameras don't exist in graphics APIs. Rendering is just setting a target render texture, setting view+projection matrix and then submitting draw calls to draw pixels on the currently assigned render texture.

If you want something to appear on screen without a second RenderTexture, you have to be rendering directly to the screen texture. So all you can change after that is the view projection matrices, rasterization state (like depth, stencil and blending settings), and what draw calls you submit. Unity's cameras do the work submitting the draw calls for you, based on the view and projection matrices you configure.

The stencil approach is close to a perfect solution for portal rendering, but I agree, it's not very convenient to configure out of the box. If you do all the setup by hand, it can be a mess. I recommend you set up some editor tooling to make this process simpler.

You could technically project meshes onto the portal surface, to flatten the 3D meshes onto the flat portal surface. But depth will be impossible unless you have some gap between distant objects.

[brave beacon]

Thank you very much, that makes it much clearer for me.

I think I will have the stencil approach ready for rendering special cases (In my game, I have one special re-usable Portal apart from others, so it won't be that problematic to have a specific set-up like a unique Renderer for it), and I will try to investigate the asymmetrical projection further for other use-cases.
It might be possible to scale the resolution for the asymmetrical approach dynamically to save up performance, but if I am confined to a RenderTexture with it, it might be preferable to choose a different option.