Dynamic volumetric smoke/fog shader

[manic estuary]

Ok so an effect like smoke rising from the ground.

Unlike a local volumetric fog volume, I don't want a 3d texture to just scroll here. I want it animated.

Kinda like standard 2d flipbook smoke but in 3 and volumetric. What should I look into here?

Alembic? With a special fog shader? Maybe that works? Maybe 3d noise or texture withba volumetric fog shader and distort and manipulate it in some way?

Not sure where to begin here. Resources for this are pretty much non existent.

[paper sorrel]

So you're talking of HDRP, right ?
What about the volumetric fog particle output from vfx graph ? 🙂
https://docs.unity3d.com/Packages/com.unity.visualeffectgraph@17.0/manual/Context-OutputParticleHDRPVolumetricFog.html

[manic estuary]

Ooooh. This is why I ask you guys things. You know a lot of obscure shit (would you call this obscure?)

This definitely does sound like it would help me achieve what I want

So I can output a larger amount of small particles without my computer dying, as opposed to lit particles?

Cause the issue with lit particles, other than pc dying, was that the lighting looked kinda flat and shit cause they ARE flat

But if I can use more smaller particles, it should look better

[paper sorrel]

They can't be too small either, as they still need to write in the volumetric fog buffer. But at least you can give it a try.

If lit particles are to expensive, there's also the 6-way lighting particles : https://unity.com/blog/engine-platform/realistic-smoke-with-6-way-lighting-in-vfx-graph

[manic estuary]

Maybe there is something I'm not understanding here, but won't they have the same problem as lit particles.
I.e. they are literally square cutouts and contain no geometry or information to indicate how they should be lit, other than just uniformly as a square would

Does this solve anything there, other than performance?

I know of 6 way lighting but I'm not sure it would be good for my game

In fact, it probably wouldn't

3d textures still seem like the best way to render this but I would need to manipulate them

I doubt adding nornal maps to 2d textures here would help much

[gleaming goblet]

Wouldn't be "good"?

[paper sorrel]

6Way lighting is a bit more complex than simply slapping a normal map on a 2D quad.
Each of the "lightmap" contains de light influence for the X/Y/Z axes, one for negative and one for positive axes.

[gleaming goblet]

Likely you wouldn't use "square" textures even if the particles are square, a smoke texture helps and a normal map will too
HDRP's diffusion profiles can be used to spread the light more evenly so it's not lit quite like a quad

I think the advantage of 6-way lighting over normal maps is that it skips the need for both normal maps and diffusion profile / SSS while allowing you to define how light would pass "through" it

[manic estuary]

I wasn't talking about 6 way here, I was talking about genrral lit particles or the fog particles

[manic estuary]

Ye but I guess the disadvantage is it's not "fully accurate" and not very modifiable if I'm using flipbooks

I want to make it as good as possible

I am open to trying multiple approaches tho

And will probably end up doing so anyway

I know you knkw my game doesn't use static lighting

[gleaming goblet]

Afaik they're the most accurate option available to you
More work to create maybe but just as "modifiable" as anything else if you're in charge of that process

Remy's blog link shows them being used with realtime lighting

3D textures alone won't help with lighting, but they're the type of asset to use if you want to mask volumetric fog in particular ways
https://docs.unity3d.com/Packages/com.unity.render-pipelines.high-definition@17.0/manual/create-a-fog-volume-shader.html

[manic estuary]

I'm not expressing myself clearly. It's for silt for my game.

Since it's pretty much the only effect I'll have (other than air bubbles) and since it's omnipresent, I want to make it look as good as possible.

I would try to make it find intersections and bend around the hand for example and such. Have some kind of environment interactions you could normally achieve with a shader

Hence, flipbook textures not being verz moldable. I mean, it can be done technically but it's not very viable I don't think

[gleaming goblet]

Volumetric fog is generally way less 'accurate' when it comes to detail than particles / 6-way lighting, but its advantage is that it's more 'accurate' for filling 3D space

Of course you can use them together, and might want to as silt has many different grain sizes to it

Might want to use billboard particles with ordinary normal maps on top of those for more solid pieces
A lot of options with a lot of overlap

[gleaming goblet]

What do you mean by that exactly?

[manic estuary]

I didn't know it was WAY less accurate, but it's less accurate in terms of direction

Also

Flipbooks are still 2d...what if I swim by a silt cloud?

I guess I could combine but I'll probably end up testing all possible variations

[manic estuary]

I mean I guess I could twist and distort flipbook uvs but I can't imagine it looking that good

Tbh as far as 6 way accuracy goes, at least it's always gonna be lit from frontal angles...

[gleaming goblet]

Why would you distort it?

Maybe we're operating on different assumptions for the use case, as well as imagining different meaning for "accuracy"

[manic estuary]

For example, if I run my hand through silt, I want the silt to distort and wrap around the hand and follow it

I was imagining some kind of volume shader and detecting intersections and distorting uvs around intersections and whatnot

ofc this is all just high level talk, I would need to see it in action to see if it works.

I would just prefer if I had a good base to start from

[gleaming goblet]

I have never seen an effect that works quite like that
It could be interesting but probably far from realistic
Rather those are usually done with forces applied to particles, which VFX graph is great at

Or rarely a 2D fluid simulation translated into 3D fog

https://shahriyarshahrabi.medium.com/interactive-volumetric-fog-with-fluid-dynamics-and-arbitrary-boundaries-f82fdee86397
This technique, although not the most ideal example of it

Usually too performance costly, inflexible and hard to implement to see this in games much

[manic estuary]

Yeah I know it's unusual

Probably forces applied to particles moght look good

With some stretching probably as I think that would really sell the effect

[gleaming goblet]

Also, if the only light source is always pointing forward from the player's perspective, 6-way lighting won't really do anything

[manic estuary]

Rather than just sending particles every which way

[manic estuary]

Well, not always straight on but I thought I could adjust the angles of capture?

[manic estuary]

Yeah, a fluid simulation would be cool so it could hit and expand the cave walls and all but probably too costly

[gleaming goblet]

The 6-way lighting texture is not something you really "adjust", the maps are there so the material can respond to any direction of incoming light

[manic estuary]

I thought I could bake the angles I want so I thought it would be even more accurate as I have more references from the front

But apparently not

[gleaming goblet]

Not quite
Similarly you can't make a normal map more "accurate" by pointing its vectors towards the most likely direction of incoming light either

[manic estuary]

True

[gleaming goblet]

It's a very interesting idea but kind of shocking to consider it as an option over ordinary simulated billboard particles, especially if your style is realistic

It sounds rather like you aren't really familiar with what the VFX graph can do

[manic estuary]

Actually no, I'm a lot more familiar with the particle system

but I don't see how that would help tbh.

I'm still just manipulating the position of billboards

[gleaming goblet]

Mostly the same stuff but you aren't limited to the PS modules and you get 1000 times the number of particles

[manic estuary]

hmm I think I understand what you're getting at

[gleaming goblet]

So making effects that look volumetric is much easier due to the sheer density of individually simulated particles

[manic estuary]

rather than making some flipbook billboards etc. you'd instead simulate literal fog particles

or at least small flipbooks or something

the smaller I go with this, the more I am sure I could do it with vfx graph, yes

[gleaming goblet]

VFX graph has plenty of demos and examples which could be helpful here

[manic estuary]

but vfx graph isn't really made to interact with the environment, is it?

[gleaming goblet]

It probably has more methods of interaction than the particle system
But the hard part is that you can't really have interaction against an arbitrary number of objects
PS can just get any nearby colliders or force fields, but whatever VFX graph particles interact with must exist within the graph
Though those objects can be passed into it with a script

[manic estuary]

well yeah

I want collision with outside objects

as I thought, nothing will be simple lol

can I add a collider with a script onto summoned particles and direct forces for a part of them or something?

[gleaming goblet]

If you beforehand which objects the vfx graph needs to know about, that's pretty easy

With arbitrary objects you probably need to maintain an array of some kind

[manic estuary]

so what if that object was a giant chuck of terrain with a mesh collider?

it seems like I will just need to try every method known to man and see how it looks

[gleaming goblet]

Could make an SDF of the terrain, and check if you can balance the resolution of the SDF with performance
Could make SDFs into chunks and swap them by proximity
Collide against depth buffer might even be good enough for

[manic estuary]

What do you think of this?

https://assetstore.unity.com/packages/tools/physics/fluxy-2-5d-fluid-simulator-203795

I know, I know, it's an asset fom the store

refer to the second video

[gleaming goblet]

Looks interesting and promising
But since the volumetric effect is fake, it may be easy to expose its limitations if the player can look into it very closely

[manic estuary]

on the upside, the 3d is fake which is the only way to have it perform well probably

but on the downside, the 3d is fake, yea

might look into it more

[gleaming goblet]

There is no one great option

Tech art is about compromises and knowing how to present the tech you use in the best possible light

[manic estuary]

lol

thanks, HR

but yeah

I will think on this

doing some other stuff right now

good thing is I can make my own shader for this, which might help SOMEHOW

[manic estuary]

I had a chat with the fluxy asset creator and basically, it couldn't work for my use case.

He suggested vfx graph with a very high number of small particles and some bigger ones to account for "volume", which is exactly what you suggested @gleaming goblet

[gleaming goblet]

I guessed as much, but specific reason why not?

[manic estuary]

Because the effect that I needed (the moving smoke that follows an object) is billboarded so I can't really use a volumetric shader without it looking stupid

The other thing with the volumetric shader works statically in a container as a plane (oriented in world space)

It's not meant to move or have shapes other than the bounding box

[gleaming goblet]

Moving smoke in this case is different from the disturbed silt?

[manic estuary]

I did wonder whether I could do some alpha shenanigans to hide the rest of it

[manic estuary]

No?

There was a different effect in the first video where the smoke appeared out of nowhere and followed a mesh (i.e. hand)

The volumetric effect just fills a predetermined bounding box

[gleaming goblet]

I would imagine that's useful for ground silt

But maybe it differs from your vision

[manic estuary]

What do you mean?

Am I crazy?

[gleaming goblet]

Any of us could be or we just have different ideas in our heads of what the result should or would look like
I think I've seen both the kind of underwater ground that billows up into a cloud when disturbed, as well as ground that has a constant foggy layer of silt kept up by the water flow that swirls around when disturbed

[manic estuary]

I'm talking about the first thing

The billowing silt when you disturb it

I will eventually need the second one too, but that's a different matter and easier to achieve anyway

[gleaming goblet]

This whole thread could've gone a lot more efficiently if you had demonstrated more precisely what you're working with and what you exactly want

Like 6-way lighting for example is a great technique for exactly what you ask
But entirely superfluous if you have just one forward-facing light source, as I gather you do?

[manic estuary]

I did say like smoke rising from the ground, in my defense

Maybe I should have specified that the silt was also supposed to rise

[manic estuary]

Yeah, I only have a forward facing light source

[gleaming goblet]

The inaccuracy of word based communication is that there are multiple different but valid interpretations, usually in ways you don't realise to expect

[manic estuary]

Alright

But in any case, six way lighting doesn't solve the problem of dynamic interactions with the environment

I really wanted something like a fluid simulation

[gleaming goblet]

That was regarding the issue of "flat lighting" which shouldn't really be an issue?

That's another thing that'd have been helpful to demonstrate
If you're seeing that as a problem, showing how it is a problem could reveal that you're using the particles in some unintended way

Anyway individual particles are pretty great at fluidlike effects, which you can see in many VFX Graph demos

Particle System only has noise as a force, but in VFX graph you can make that work relative to particle velocity and individual size too, to get more realistic settling and vortex behavior

[manic estuary]

But afaik, it's good for unbound fluid like behaviour, no?

I.e. non interactive with the environment

would probably be very expensive even if I could make it interact with everything

[gleaming goblet]

Like I mentioned VFX Graph also can interact with colliders and forces but technically those need to be defined inside the Graph so it's more complicated and optimization is up to you if you need it to interact with non-predefined objects

Particle System colliders are expensive also so I'm not sure either is really "better" at it, PS is just easier

[manic estuary]

no way I would ever do this with the particle system

to get any kind of good look, I'd need a lot of particles

they would need to be lit

everything would explode

ps is very good at what it does, and what it does is NOT this

[gleaming goblet]

Sure
Look at VFX Graph examples for effects like that and give it a try
When you encounter specific problems show them here

[manic estuary]

also, potentially moronic, but would you spend any time considering that fluxy asset with the pre-defined bounding box but somehow masking out the alpha so that it's only around the hands or whatever?

[manic estuary]

yeah I guess I should really look into it more

used vfx graph only superficially, I'm not familiar with its power

[gleaming goblet]

Kinda looks like it supports something like that already?

In addition to that the video has a "negative space" trail with the 2.5 volumetric effect, as well as arbitrary color swirls
Seems like it would also supports a "positive space" trail with volumetrics, but you could confirm that with the creator

[manic estuary]

I think that's what he was talking about when he said it was billboarded

i.e. it could not be made volumetric as it's not oriented in world space

would look stupid in any case when turning

[gleaming goblet]

Turning?

[manic estuary]

the camera

[gleaming goblet]

As you see in the trailer, the fog itself is actually volumetric and looks fine with the camera going right up to it

But the simulation is 2D

Meaning the fog/dust can form a spiral only horizontally, and for rendering is "stretched" vertically to give it volumetric thickness

[manic estuary]

yes but you are talking about the bounded fog?

wait no

are you talking about the letters at the end of the second video?

[gleaming goblet]

Yes
There you can spot how the volumetric effect is executed

[manic estuary]

yeah, I hadn't considered that

I think he was specifically saying the billboard stuff is for the moving offset thing

tbf that's an effect that already pre-exists too

it's pretty much the same as the ground thing

[manic estuary]

@gleaming goblet sanity check: right now I'm thinking a shitload of output point particles with some kind of fluid flow and probably some bigger volumetric fog ones on top.

Also, baked sdfs and all

what do you think

[gleaming goblet]

It sounds allright to me but there's a lot of wiggle room within those parameters
Showing the spots where you want them and how they would be lit would help, as well as some placeholders or mockups how they should look if possible

Earlier I wished to see a demonstration of what issue of flat lighting you experienced with billboards but we never got it

[manic estuary]

alright I'll try to create something just as a mockup

[manic estuary]

I didn't use normal maps so you can imagine it in your head

but regardless, they were very taxing on performance and now I want them to interact with hands and stuff so it's probably irrelevant at this point

but I'm sure I have videos somewhere I can pull up anyway

[gleaming goblet]

I can imagine multiple possible scenarios
You really need to start showing things

[manic estuary]

alright

I Will find a vid

https://streamable.com/hddy77

this is the best example I could find

[gleaming goblet]

It reveals a lot, more than you might expect
Which technique is the plume we see there?

[manic estuary]

it's on me if I'm too dumb to realize that lol

it's just a lit particle system flipbook

[gleaming goblet]

Just one particle?

[manic estuary]

probably not

I tried those variations tho

with more particles, less particles

even tried one, yes

couldn't tell you how many in that vid but I'd guess several

[gleaming goblet]

It looks pretty good already
With the right kind of flipbook animation, soft / depth fading and a few particles per impact you can do a lot

Responding to touch in a flowy kind of way is harder
Anything that doesn't use fluid simulation is bound to look not very realistic so I'd make a compromise there and maybe not try
The flowy feel of flipbooks gives a convincing feeling already

An imperfect collision detection might make it worse

[manic estuary]

hmmm...do you think sdf collision is imprecise?

[manic estuary]

I already use my soft particle flipbook shader here I think

so basically that's the exact old system I was using

you're saying it's pretty much as good as it's gonna get?

when I turn it to unlit, there's no problems

lit sometimes cuts my fps in half, especially if I get close

[gleaming goblet]

It's as "precise" as needed but it won't work like fluid
Moving an object in a fluid through particles causes vortices in its wake which accelerate the particles into consistent flow patterns
Collision cannot account for that because the moving fluid would be separate from the object's collision

[manic estuary]

you are right

it will never look fluid like, it's just a basic "go bounce in the reflection vector direction" bounce

[gleaming goblet]

VFX Graph can do vector fields which can make convincing vortices
My idea was maybe lerping this vector field's influence on particle velocity by contact or distance to the moving object
But then the accelerated particles would flow past resting particles which intuitively doesn't make sense

I'm not sure if particles can affect each other, but I'd expect not without prohibitive cost

[manic estuary]

you are right again

so, in order to solve the resting particles problem, could I just accelerate the next ones in line?

Problem is, I don't understand quite how that works in real life physics

If you were to ask me to accurately describe how a fluid flows when disturbed by an object, I would just know to say it wraps around it

btw that was my original idea after the flipbook particle stuff. To make some kind of shader that distorts the volume/whatever it ends up being around where it intersects object but you kept saying it would look very unrealistic

I guess implying that's not how it works

[gleaming goblet]

Effectively wraps around it, but because of momentum and internal friction it also keeps going and forms those spiral patterns as it goes which carry the particles around
The problem really is that for us only the particles exist for simulation
Or vertices or other discrete points of data

So the physical medium is missing entirely
Fluid simulations do solve this, like the 2.5 one you looked at, but at a steep price which is why it was calculated in 2D

[manic estuary]

I liked that one a lot

But like, I don't see how I would make it work if I can go past the silt

Right now I'm trying something with the vfx graph particles, but as you said, it won't look like smoke/silt, rather than grains of sand flying for some reason

Maybe I could at least add some warping to the old flipbook particle solution or something just to elevate it a bit

Cause I don't think the vfx graph particles will look visually better than that, yes?

That was just for the physical stuff

But it won't look better visually

[gleaming goblet]

Fluid simulations typically let you introduce forces into the flow which would help
But a bigger problem might be that bcause it's a planar simulation it might really not work with your non-planar cave surface
That's also a hint that's revealed by visual demonstration that's obvious to you but unknown to us

[gleaming goblet]

I've seen very few cases where you can distort vertices or UVs in case of particles in a way that doesn't give away the billboard illusion so I would discount it as an option myself, but you may have an idea which I don't

[manic estuary]

I'm talking about a vertical effect here. I had a chat with the creator, and asked if I could maybe make it work by adding an invisible hand object driving the fog up and just alpha masking it around the hand if you know what I mean

but he just pointed that it's 2d and what happens when I dive past it and look at it

and I had no retort lol

[manic estuary]

I am sure I Don't have an idea that you don't

[gleaming goblet]

My concern with the vertical effect also

[manic estuary]

why tho? I could look at a solution that will dynamically create colliders in a radius or some predefined shape if that's what you're worried about

the cave wall would essentially be something that would collide and affect the flow

[gleaming goblet]

I mean camera getting up close to it and rotating

[manic estuary]

well yea

that's the only problem I see with it

it solves all other problems

probably would be performant too

more than lit particles

but it's a giant problem...

also a problem with the lit flipbook particles, no? Cause I can billboard that effect too

it will be the same thing, no?

I'm referring to the other offset effect which billboards by default, I could make a shader to make that volumetric...maybe I could even rotate the simulation, dunno if that's possible

I'm just thinking out loud. If I billboard this effect, will it look worse than billboarded particles?

[gleaming goblet]

Not that much
When they are set to fade by distance to camera and there's a handful with different depths, it's usually very convincing in my view

The 2.5D fog is a different thing because it's rendered as volumetric, so in a lot of situations rotating it is not necessary
But it's not a type of effect I've dealt with so I'm really not sure how it'd practically behave if rotated

The simulation probably isn't meant to be rotated, but I assume it doesn't really care
I believe it only checks if effectors intersect the simulation plane

I think the bottom line is that to get something really great you would have to get creative or technical, or both
Which is difficult if you don't already have knowledge and confidence to implement the particular kind of effect you want
So realistically I'd recommend settling for "adequate" rather than putting a lot of effort into something that might not even turn out well

[manic estuary]

I don't have confidence issues lol

just technical limitations

[manic estuary]

yeah but I figure the following:

my only environment decoration is literally rocks so I need to make them look good. My only light is my flashlight so I need to make it look good

my only vfx or almost only is the silt

basically I have to give all of these my all in order to compensate for the lack of variety

not that hard to make everything look good when you have a bunch of 3d models, lights, when you can bake them etc

I would rather spend a lot of time on something that might not turn out well

if I end up having to revert to basic flipbook particles, I will admit defeat

but I'll carry on with that, reluctantly

[gleaming goblet]

By confidence I mean lack of uncertainty, that you'd know what and how you need to make without having to learn and experiment

Simple effect can look much better than a complex one that doesn't turn out well

[manic estuary]

hmm I wish my standards were lower