How can I use shader graph to create a shader with highlighted corners and crevasses like this?

[blissful crag]

Hello, I am not very experienced at using the shader graph and need some help.

I intend to use this as the base shader for my URP ground and wall materials. I'm sure the crevasses can be handled through lots of AO, but I'm not sure how to approach the brightened corners.

I want it to be procedural instead of texture based so that I can make the levels any shape needed, although they will likely be straight or simple angles similar to what is in the screenshot.

Thanks in advance!

[lyric schooner]

it seems like the "forward" edges are lightened and the "backward" edges are darkened

so maybe part of the effect is an edge shader

[knotty nebula]

I'm pretty certain there's no way in the fragment shader of the object itself to know how close you are from a corner (unless you have somehow baked the information into the mesh or use a texture). You can certainly apply that effect in post processing, but I don't know if that's what you want. Since you can do that in blender shaders, is there a reason not to bake the highlights in blender into a texture? I'm pretty confident post processing is the only other viable way to achieve that in unity. I'm thinking there could maybe be some way to achieve that by doing a depth pre-pass and sampling the depths near the mesh similar to post processing but at that point you might as well do it via actual PP shader

[blissful crag]

Hmm in that case, do you have any tips for how I could create this as a PP shader and only have it apply to objects that I would designate as part of the level geometry? I can't bake it into the texture itself because the levels will change shape frequently and I would rather have a more procedural system. I might make a level creator later so it would need to work as players modify the terrain constantly

[knotty nebula]

PP always applies to the whole screen. One way to only apply to set of objects is to render those objects with separate camera and apply post processing to that camera only

[blissful crag]

Hmm this is starting to get complicated for just wanting some nice looking edges on my level geo lol

[knotty nebula]

That is often the case with very specific visual effects. Although it sounds easy, it is in fact not due to how fragment shaders work. Fragment and vertex shaders generally only have access to information about the triangle it is rendering but nothing about surrounding geometry. That is unfortunate but makes it possible to render objects with extreme parallelism which makes it very fast. Working around that limitation is not an easy problem to solve.

I don't know how blender does that. I think the source is available somewhere, could be worth a look. I assume it is some sort of PP effect

[lyric schooner]

maybe you could use normal data to render

like change in normals

[knotty nebula]

in a PP shader sure

[blissful crag]

So in a global pp shader there is no way to have it only apply to some objects? Nothing like checking the layer or material of geo on the screen?

I think having them render on a different camera is probably fine actually, just another thing to figure out

Honestly it's really surprising for me to find out how hard this apparently is. I can imagine numerous uses for edge detection like this in shaders. I would have imagined a more streamlined solution exists lol

Someone on my reddit thread for this found something https://github.com/federicocasares/cavifree

Might be usable but I haven't checked it fully yet until I get back to my pc

[knotty nebula]

Not really. a PP shader does not have access to the layer of the pixels by default. I think you could in the shaders of the objects mark the pixels to the stencil buffer and then only apply the post processing effect to pixels with certain stencil value but you might as well do it with the two camera setup

[knotty nebula]

Looks nice, that indeed uses a PP shader to achieve that. No need to write the shader yourself if one already exists of course (make sure to check the license)

[blissful crag]

Yeah it saves me having to figure out convolution lol

[knotty nebula]

That's just the way GPUs and grpahics libraries work. PP is a great tool achieving something like that which in the fragment shader is not possible due to technical limitations. Unity could of course add their own PP shader graph node that does some sort of edge detection. I thought I saw something like that mentioned in the roadmap somewhere but I cannot find it right now, don't know if it has landed yet or never will.

[blissful crag]

This shader does seem a little expensive so I wonder if the code can be modified to somehow bake data into the geo which can be used for a normal material shader and then simply run that once every time geo changes