Getting an Outline Render Feature to only outline the outermost edges

[rich rampart]

For a while I have used this plugin for outlines (similar to the outlines you get when selecting an object in the inspector), but it requires the Compatibility Mode render options. I have a different, separate issue that requires the rendergraph to fix, but it means I can no longer do this outline feature. I've switched over to this plugin but I'm having a problem with it. This version currently highlights all edges of the object with the outline (first picture). The old outline system would only outline the outermost edge of the object (Second picture).

I've got the full source of both render features, with the new non-working one being done in Shader Graph, while the older one is a .shader file. I don't know enough about shader code or graph to know what to look for to port this functionality to the new system. Does anyone know what sort of things should I be looking for that might cause these internal lines inside of the shader?

Barring that, anyone know of a good URP Outline Render Feature that doesn't require the Compatibility Mode that can do what I'm looking for? I don't particularly care how it looks, this is purely functional rather than aesthetic, I just need to outline arbitrary objects.

[cerulean cloud]

This is screenspace? Wonder how that even works. I could only imagine it's looking for normals that point outwards from each other.

[rich rampart]

I have no idea, I just use it. How would I be able to tell?

[cerulean cloud]

Well, what you usually have to work with using screenspace is that depth, normals, and/or color

I assume it is screenspace though because it is a render feature?

[fresh gazelle]

I know of 3 techniques for outlines:

• render objects in own pass, edge detect, outline
• inverse hull
• normal/depth buffer detection

[rich rampart]

Do you know which ones would produce an outcome like the second picture? Just along the outside edge of the mesh?

[cerulean cloud]

I guess the question is are you using a outline material on the object, or are you just enabling this render feature.

[fresh gazelle]

both images look weird to be tbh, the outlines dont seem to match the object?

[rich rampart]

I've got the render feature enabled, and when I want the object to be outlined, I change its layer to one the feature applies to

[rich rampart]

The object is a little weird, the top bit is a separate object, and it's under the terrain a little

The "roof" is a separate mesh since it can be disabled independently of the walls to view from above

[cerulean cloud]

Oh, huh. Maybe it is local if you are doing layer selection

[fresh gazelle]

Yea if that is rendered in its own pass to produce the outline then depth is not kept which explains this somewhat

[rich rampart]

I can show the feature settings if it helps:

This one's the old one that uses the compatibility mode, but looks how I want

And this is the new one that is highlighting corners and other edges instead of just being an outline

[cerulean cloud]

yeah it's an override

meaning it's doing a secondary pass for the outlines local to the object

so you have vertex information here

[fresh gazelle]

I think for the best results it would require:

• pass to render objects to produce depth mask
• use depth copy to make outline (edge detect) but skip if mask depth is less

[cerulean cloud]

Ok, I think what it's doing is it is actually screenspace, but it uses the viewspacenormals rendering as a mask

instead of using the depth/normals from the viewport texture

which is odd, I would just think that doing it local here would work fine too

but the thing with screenspace outlines is that you're less likely to run into problems that inverted hull presents and that's probably the idea here

Anyway, it probably does Roberts cross alg to find those edges, but the question is what is the operation to prevent that specific edge from rendering

Which I would think is some normal comparison of sorts

[fresh gazelle]

I love outlines so simple 💀

I used to just do inverse hull on mobile

[cerulean cloud]

Inverse would be a problem for this exactly scenarion as those walls would have some badly extruding outlines

[rich rampart]

There is a "roberts cross multiplier" on the new one, although I don't notice it doing anything

I have no idea what a Robert's Cross is though

[cerulean cloud]

https://ameye.dev/notes/rendering-outlines/#detection-of-discontinuity
Some ways to go about it there for screen space

https://ameye.dev/notes/edge-detection-outlines/
which there is a basic example of it in HLSL here

[rich rampart]

The "Blurred Buffer" effect looks the closest to the one I had that worked, so maybe I just need to find a way to do that?

[fresh gazelle]

Good detection + masking and depth rejection are the main parts in my opinion

[rich rampart]

Jump Flood also looks promising

[cerulean cloud]

Jump Flood is usually what you want if you want real thick outlines

similar to how text and signed distance fields work with it all

usually you see it in photoshop when you use the stride/outline filters

[rich rampart]

I think I'll try to figure out how to do this "Blurred Buffer" in Shader Graph, since it looks like this new method uses a material to apply the outline. It'll take me a while, I'm not very good at this

[cerulean cloud]

I don't actually like full screen stuff because I do find it hard to get it correct, but there are cases like these sharp walls where it does make sense

Locally, instead of inverted hull you'd actually have custom vertex channels to define how a lot of the outlines are to be blended, which you'd see in games like Guilty Gear and more stylized games.

[rich rampart]

I just need something to replicate the "selected item highlight" like the editor has

[cerulean cloud]

Looking at it again, I think it could just be stencil sillouhette outlines

yeah the blurred buffer you were talking about

or if you go up a bit on that page you'd see Silhouette Buffer

Usually you just scale the mesh up and render the same mesh behind it for a real easy outline which works in some cases, but of course you don't get those inner outlines, which you don't want anyway

but in your example, it's not being scaled(?), but being rendered in front with some edge adjustments

[rich rampart]

I don't know if that will work because the mesh's origin isn't necessarily in the center. If I scale it up it's not going to be growing in all directions

[cerulean cloud]

yeah, it's basically the sillouhete idea, but you're rendering it before/after the post-processing pass

[rich rampart]

So, the explanation makes sense, I now just need to figure out how to translate those steps into Shader Graph, or find a way to write a shader that this thing'll accept.

[cerulean cloud]

but the question on why is the outlines not around the wall but inside of it

because I'm not entirely sure if there's edge detection here, or there's some way you're making the mask by cookie cutting it from the original rendering

because you can probably just use stencils and be done with it honestly

[rich rampart]

It's around the wall. There's actually two meshes for that wall, the actual wall and then a separate "roof" object. Here it is without the roof object, if it makes more sense:

[cerulean cloud]

like for example, render the wall twice, but use a unlit material for the second wall and scale it up by 1.1

then position the actual wall in front of it. If that works then all you need to do is figure out the stencil ordering

the logic of the stencil would be
If wall pixels -> don't render wall outline pixel, otherwise render outline pixel

somewhere after transparents

But there is a few problems with scaling some meshes. It's not always a good uniform scaling unfortunately, and does depend if the pivot is centered correctly

[rich rampart]

The pivot for these objects is roughly a thousand units away from any faces

They're all runtime-generated meshes from point cloud data and the origin of all of them is 0,0,0 since all the vertex data I have is in world space for the source program

[fresh gazelle]

Inverse hull works well here because you "scale out" using vert normals

which may be applicable in this case you mention

[cerulean cloud]

It could probably work, but I think you couldn't extrude them out too much if you do want thicker outlines

[rich rampart]

Okay, I have some things to try but no more time today to try them, so I'll give these a go and see how far I can manage to get tomorrow.

There will likely be more questions, but I at least have enough to do some research with

[fresh gazelle]

This is why its a poor choice and producing a mask and using edge detection is just better

[cerulean cloud]

but for this example I think it should probably be fine

My favorites are usually just throwing on stencil outlines and calling it a day lol

[fresh gazelle]

A pass to make a mask provides greater flexibility (especially if it draws depth too)

[cerulean cloud]

I could never get inverted hull to look nice, nor any outlines to look nice on the closer rim of my models. Like those real stylized looking games will do a lot of custom vertex data to specify where those outlines should be.

[fresh gazelle]

Yea customising them is hard. I remember someone who did a pass with colours to control the effect

[rich rampart]

Apparently, using a stencil buffer in Shader Graph is just literally impossible since it requires multiple passes and apparently Shader Graph can't do that.

So now I'm trying to just update the old render feature to the new API

After looking at the documentation for how to do so I have now been radicalized and will be firebombing Unity HQ

The official migration guide just says "If this is a third party asset, just ask the developer to update it! 🙂"

[rich rampart]

Literally the first fucking step of the "How to make a render feature compatible with the render graph":

"Let's take a look at the render graph viewer and see what passes it does so we know what functions to write!"
I FUCKING CAN'T. ITS NOT COMPATIBLE WITH THE RENDER GRAPH. THAT'S WHY I'M HERE.

[fresh gazelle]

I'm not sure this is true. Stencil operation use does not need multiple passes by default but some techniques may

I forget though if shader graph shaders even expose stencil settings

[fresh gazelle]

Can you clarify what you wanted to try?

[rich rampart]

At least it's what this random user on the forums thinks, and the hundreds of unanswered posts asking similar things seem to agree:
https://discussions.unity.com/t/shadergraph-how-to-make-outline-effect/710983

[rich rampart]

I am trying to figure out how to just get the old one to work without compatibility mode.

[rich rampart]

Both the original and my in-progress slapdash fix (there should be two pastes here):
https://paste.myst.rs/xxs9pea0

I've managed to get as far as "Make a custom class you can pass with all your data wrapped up, then move your old execute stuff to something you pass in SetRenderFunc"

Some questions remain:

• What do I replace RenderingData with?
• I still don't know how to deal with _renderer.cameraColorTargetHandle
• No clue how to deal with ExecuteCommandBuffer stuff. It seems to just not exist in the RasterGraphContext object
• I don't actually know what those readonly lists are doing and I just have to hope that passing them to the PassData object does what it's supposed to do

I am severely out of my depth here.

[hearty iris]

The ContextContainer frameData contains various data classes which would probably act as the replacement for RenderingData.
e.g.

UniversalRenderingData universalRenderingData = frameData.Get<UniversalRenderingData>();
UniversalCameraData cameraData = frameData.Get<UniversalCameraData>();
UniversalLightData lightData = frameData.Get<UniversalLightData>();
UniversalResourceData resourceData = frameData.Get<UniversalResourceData>();

(might be more, idk)
resourceData contains most render targets so you can access the cameraColor and depthTexture targets using that

Also in ExecuteRender the RasterGraphContext param contains a .cmd (which gets executed for you), use that instead of CommandBufferPool.Get

[rich rampart]

Am I on the right track of just packing a bunch of crap into PassData before calling the builder? So I can just set variables in it from RecordRenderGraph so that I can use those data sources in the custom Execute function, right?

[rich rampart]

And this means I don't need any ExecuteCommandBuffer anywhere, right? I just build up context.cmd and it fires it for me?

[hearty iris]

Hmm kinda. If it's needed by the custom execute then yes it needs to be passed through that PassData. But you should be able to use CreateRendererList outside that and just pass the reference to that RendererList. Also to declare the list with the rendergraph builder. A bit like my example here : https://www.cyanilux.com/tutorials/custom-renderer-features/?tab=Unity6#drawrenderers

[rich rampart]

Oh my god this is already infinitely more useful than any piece of official Unity Documentation I could find on this hellish API 🙏

[rich rampart]

I'm having a little bit of trouble identifying the "payload" from the boilerplate. Everything here seems to have an analogue to the render pass except the Settings object. Does that mean the entirety of the actual functionality of the renderer is in here:

if (settings.overrideMaterial != null) {
    drawSettings.overrideMaterial = settings.overrideMaterial;
    drawSettings.overrideMaterialPassIndex = settings.overrideMaterialPassIndex;
}

And the rest of it is all just passing options?

This bit also looks a little suspect, is this something else specific to this particular render feature?

// Set Render Target
  builder.SetRenderAttachment(resourceData.activeColorTexture, 0);
builder.SetRenderAttachmentDepth(resourceData.activeDepthTexture);

[hearty iris]

The Settings object is just what I use for exposing fields on the RendererFeature, so the user would assign a material there, similar to the RenderObjects feature overrides.

SetRenderAttachment/SetRenderAttachmentDepth configure the target textures the objects get rendered to, so would be important for any feature.
(Except some that manually use SetRenderTarget via an UnsafePass instead of RasterPass. The built-in AddBlitPass and AddCopyPass use that iirc)

[hearty iris]

Though it seems in the OutlineRenderer class they use SetRenderTarget so you may also need to use an UnsafePass rather than Raster one. Or convert more of their code than just the render pass.

[rich rampart]

Yeah, I am running into that I think. Is it basically the same, just substituting AddUnsafePass instead of AddRenderPass?

[hearty iris]

I think so yeah

[rich rampart]

Okay, the OutlineRenderer takes in a CommandBuffer so I need to rewrite some stuff to use UnsafeCommandBuffer, which looks like it has trouble with GetTemporaryRT

The renderer in a pastebin:
https://paste.myst.rs/68p78n20

And some different parameters to SetGlobalTexture apparently

[hearty iris]

I'd probably swap the GetTemporaryRTs out for creating textures through the rendergraph API. Maybe something like

var desc = renderGraph.GetTextureDesc(resourceData.cameraColor);
desc.colorFormat = RtFormat;
desc.name = "_MaskTexId";
resources.MaskTexId = renderGraph.CreateTexture(desc);
desc.name = "_TempTexId";
resources.TempTexId = renderGraph.CreateTexture(desc);

In RecordRenderGraph. Then builder.UseTexture() on them and pass through PassData

But I think there's also CommandBufferHelpers.GetNativeCommandBuffer(context.cmd); to convert it to a CommandBuffer type.

[rich rampart]

What are RtFormat and resources in this snippet?

[rich rampart]

That would probably be easier if that works, let me try that

[hearty iris]

Stuff in OutlineRenderer. RtFormat is a field defined on line 58. resources is a param in the constructor which is where GetTemporaryRT is currently used

[rich rampart]

Ah, I was in the wrong file, since I was trying to add it in the OutlinePass's RecordRenderGraph

Okay, it is no longer yelling. I think I'm almost there, I just need to know where to put all this stuff. This is what's left in what used to be the Execute function. The CreateRendererList stuff was moved to
InitRendererLists, but since that's inside of a using for OutlineRenderer, I'd need this to be in ExecuteDrawPass since I need the command from the context for the OutlineRenderer constructor. Where would this go so it has access to all the data it needs?

if (passData.outlineLayerMask != 0)
{
    var filteringSettings = new FilteringSettings(RenderQueueRange.all, passData.outlineLayerMask, passData.outlineRenderingLayerMask);
    var renderStateBlock = new RenderStateBlock(RenderStateMask.Nothing);
    var depthTexture = new RenderTargetIdentifier("_CameraDepthTexture");

    if (outlineSettings.IsAlphaTestingEnabled())
    {
        passData.drawingSettings.overrideMaterialPassIndex = OutlineResources.RenderShaderAlphaTestPassId;
        context.cmd.SetGlobalFloat(outlineResources.AlphaCutoffId, outlineSettings.OutlineAlphaCutoff);
    }
    else
    {
        passData.drawingSettings.overrideMaterialPassIndex = OutlineResources.RenderShaderDefaultPassId;
    }

                
    using (var renderer = new OutlineRenderer(CommandBufferHelpers.GetNativeCommandBuffer(context.cmd), outlineResources, passData.renderer.cameraColorTargetHandle, depthTexture, camData.cameraTargetDescriptor))
    {
        renderer.RenderObjectClear(outlineSettings.OutlineRenderMode);
        RendererListParams rParams = new RendererListParams(renderingData.cullResults, passData.drawingSettings, filteringSettings);
        RendererList rList = context.CreateRendererList(ref rParams);
        context.DrawRenderers(renderingData.cullResults, passData.drawingSettings, ref filteringSettings, ref renderStateBlock);
        renderer.RenderOutline(outlineSettings);
    }
}
            
if (passData.outlineLayers)
{
    var depthTexture = new RenderTargetIdentifier("_CameraDepthTexture");

    using (var renderer = new OutlineRenderer(CommandBufferHelpers.GetNativeCommandBuffer(context.cmd), outlineResources, passData.renderer.cameraColorTargetHandle, depthTexture, camData.cameraTargetDescriptor))
    {
        passData.renderObjects.Clear();
        passData.outlineLayers.GetRenderObjects(passData.renderObjects);
        renderer.Render(passData.renderObjects);
    }
}

Actually, in the original original, it looked like this:

context.DrawRenderers(renderingData.cullResults, ref drawingSettings, ref filteringSettings, ref renderStateBlock);

[hearty iris]

Should be able to call InitRendererLists from RecordRenderGraph since it is only doing CreateRendererList not drawing yet

[rich rampart]

Yeah, I think I confused myself with one of my half-implemented fixes. Right now I'm just not sure where to actually do the rendering. The part where it creates an instance of OutlineRenderer and calls RenderOutline on it. Since it uses the command buffer, it needs to be in ExecuteDrawPass, right? Do I just take this chunk and replace the DrawRenderers line with the DrawRendererList line?

using (var renderer = new OutlineRenderer(CommandBufferHelpers.GetNativeCommandBuffer(context.cmd), outlineResources, passData.renderer.cameraColorTargetHandle, depthTexture, camData.cameraTargetDescriptor))
{
    renderer.RenderObjectClear(outlineSettings.OutlineRenderMode);
    context.DrawRenderers(renderingData.cullResults, passData.drawingSettings, ref filteringSettings, ref renderStateBlock);
    renderer.RenderOutline(outlineSettings);
}

[hearty iris]

Yeah I'd try that

[rich rampart]

Okay, almost there. No more compile errors, I think I have most things resolved, I just have one last deprecated thing: data.renderer.cameraColorTargetHandle

data.renderer being an instance of ScriptableRenderer passed to it in Setup. The compiler doesn't say what to replace this with, just to use compatibility mode if I want to use it, which is the entire thing I'm trying to avoid. I need to pass a RenderTargetIdentifier there but I have no idea what that means or how to get the one I want

[hearty iris]

Replacement would probably be resourceData.cameraColor

[rich rampart]

No more errors, no more warnings but... no more outline
I put something on one of the layers and I'm not getting an outline. No errors or warnings on changing the layer either, it just doesn't do anything.

https://paste.myst.rs/5aa2q3ru

[hearty iris]

I would add builder.AllowPassCulling(false); to make sure rendergraph isn't trying to optimise the pass away since you aren't using SetRenderAttachment

Can also use the Render Graph Viewer window to check the pass is running

[rich rampart]

Oh my god it's working

It's lighter than it used to be but that's a fuckin outline!

I cannot stress enough how helpful you've been, I actually kinda have somewhat of an idea how a render pass works now

Even if I don't know all the magic incatations to make it not just delete your render pass for kicks