particle system jobs

[livid aspen]

no you shouldn't have to do anything else. just schedule your job in that callback. what isnt getting called? the callback?

[glossy zephyr]

i think the callback isnt getting called, as when i chuck a debug log into it, it never runs

this is what i interpreted as "using the callback" incase my code is weird lol: ```cs
void OnParticleUpdateJobScheduled()
{
RenderParticlesBurst();
Debug.Log("testing?");

    job.Schedule(FluidRenderer).Complete();

    job.PColours.Dispose();
}

nothing in there runs ever

[livid aspen]

it will only happen when the particle system is playing and contains particles

what is RenderParticlesBurst() ?

[glossy zephyr]

oh, that could be an issue... RenderParticlesBurst() is what adds particles to the particle system.... thanks so much, ill quickly change up the way my code is and report back!

[livid aspen]

yeah. try to only schedule the job in that callback, nothing else if possible. and dispose your PColours at the end of the job code, either in the job, if it s single job (not a for-job), or with a second job chained to the burst job, if you're using a for-job for the particle stuff. if that makes sense?

[glossy zephyr]

dont really understand what a for job is, my job contains a for loop though if that is what you mean?

this is my job i came up with lol: ```cs
struct UpdateParticlesJob : IJobParticleSystem
{
[ReadOnly]
[NativeDisableUnsafePtrRestriction]
unsafe public Vector4* PosData;
[ReadOnly]
public float RenderRadius;
[ReadOnly]
public NativeArray<Color32> PColours;
[ReadOnly]
public int Slots;

    public void Execute(ParticleSystemJobData particles)
    {
        var PosX = particles.positions.x;
        var PosY = particles.positions.y;
        var PosZ = particles.positions.z;

        var Colours = particles.startColors;
        var SizeX = particles.sizes.x;
        var SizeY = particles.sizes.y;
        var SizeZ = particles.sizes.z;

        unsafe
        {
            for (int i = 0; i < Slots; i++)
            {
                PosX[i] = PosData[i].x;
                PosY[i] = PosData[i].y;
                PosZ[i] = PosData[i].z;
                SizeX[i] = RenderRadius;
                //SizeY[i] = RenderRadius;
                //SizeZ[i] = RenderRadius;
                Colours[i] = PColours[i];
            }
        }
    }
}

it isnt very nice having an unsafe pointer, but i dont really have a choice, c style arrays should still work well enough i hope

hmm ye i cant work out where to put the dispose thing, it seems wherever i put it, it still claims i never disposed of it and that im leaking memory

[livid aspen]

a "job" is a lump of code that can run on one your other cpu cores, so it can run while Unity's main thread is doing other things (like running some of your other script code)

[glossy zephyr]

ok!

[livid aspen]

if you allocate PColours in your MonoBehahiour Update, you can dispose it as the last line in this job.

[glossy zephyr]

im allocating it (i think?) in my fixed update

would that work?

[livid aspen]

also, if you add [BurstCompile] to the job (above the first "struct" line) you'll turn on Burst and get much better perf

[glossy zephyr]

oooh thanks!

[livid aspen]

FixedUpdate can run more than once per frame. so no, that won't work. you probably want to try and do your "set up" work in Update(), then run the particle job as you are now

[glossy zephyr]

ill try and do the allocation in update, the rest of the things dont need alocation so i should be able to do them in the fixed update?

[livid aspen]

yeah it could work. FixedUpdate just means it will be in sync with the physics. so you know if you need it in FixedUpdate better than i do 🙂

[glossy zephyr]

ye im doing fluid physics, some variables only can be read for the middle third of the physic update, due to the weird way ive set this up

hmm now apparently im not disposing of a native collection, is that the same as a native array?

[livid aspen]

yes its the same

[glossy zephyr]

ok!

[livid aspen]

if you're doing fluid physics, maybe this will help you learn the whole system: https://forum.unity.com/threads/particle-system-c-job-system-support.529284/#post-4733855 there is an example hre that does "particle self collision". so it chains some jobs together to do things like "find neighbour particles" and "apply collisions between neighbours". in theoy it shouldnt be too hard to replace the last bit with the pressure/density/fluid solver steps, depending what kind of fluid sim you are doing.

[glossy zephyr]

i think im disposing of PColours correctly, and i cant see any other native arrays, unless SizeX counts as one?

[glossy zephyr]

ooh thanks!

[livid aspen]

the only downside is the demo just uses Update(). so it might need some larger rework to deal with the fixed timesteps you need.

[glossy zephyr]

currently im just trying to get nvidia flex working. although it will be useful in future lol

[livid aspen]

if you go to the Jobs menu at the top bar of unity, there is an option to turn on leak stack traces. it will tell you which nativearray is leaking.

[glossy zephyr]

oh thanks!

[glossy zephyr]

where exactly?

i cant seem to find it

[livid aspen]

ah its changed

go into Safety Checks

maybe it's in there...

oh sorry i see you already did

[glossy zephyr]

lol its ok

i remember full stack traces existing the past aswell, weird

[livid aspen]

hmm it seems like it's on already. so i would expect the error to tell you which container leaked

yeah... i cant keep up 🙂

[glossy zephyr]

definitely aint on lol

[livid aspen]

hmm google says you may need the Entities package installed (which is annoying!) have you got that one?

[glossy zephyr]

ah i dont have that currently installed on this project, ill do that quickly, does installing the hybrid renderer still install everything required for dots like it used to?

ah yes it does it appears, good

[glossy zephyr]

ok found it, turns out that i have not disposed of pcolours apparently

weird

i think im disposing of them right. I'm meant to dispose it at the end of Execute() right?

[livid aspen]

well, maybe. it depends when/where you allocate it. and what allocator param you use (use TempJob)

[glossy zephyr]

im allocating it in update now, and ill try using TempJob thanks!

nope still leaking

weird, i have no clue

ok i dont really know what i did, but from messing around i managed to go from 10 fps to 80 fps, but im still leaking memory lol, oh also i forgot to mention it earlier, but ever since using the job it refuses to render

oh

oh

i worked it out....

[glossy zephyr]

remember this issue? Well my particle system never plays in order to save performance, so turns out i have just not been running the job ever, hence why it wasnt running... hence why it wasnt disposing of the native array....

is there any way around this?

[livid aspen]

only allocate the array when the system is playing?

[glossy zephyr]

well that is the thing, the system never plays

ever

to save performance

i figured if i was gonna set the particle state every time that it made sense to never bother playing it lol

[livid aspen]

so it spawns particles and lives in a paused state?

[glossy zephyr]

yes, according to testing back when i was using a regular for loop it appears to be faster that way lol

[livid aspen]

ah ok. hm yes i suppose the job wont work if it's paused. i suppose we should have an option for you to choose that behaviour. but we dont.. so i guess you'll have to play it, if you want to carry on with this experiment 🙂

just make sure start lifetime is set to infinity, so the system doesnt start killing your particles 🙂

[glossy zephyr]

ok, ill try playing it, any specific setting that lower the cost of playing it?

[glossy zephyr]

ooh clever, although i set the system's particles every physic frame, so i doubt it will be that important

guessing duration infinity, or should i turn looping on instead?

ah you cannot set it to infinity

guessing then i would set duration to any number, and looping to true then?

hmm i get this error, i assume it comes from PColours, guessing i can chuck the attribute it mentions, onto PColours in the job struct thing?

yup it fixed it, and now it is complaing about an array (does not say where sadly) apparently with index 0 being outside of range of length 1, which sounds a bit weird lol

ooh i wonder if i remove the burst compile tag it might tell me which array?

oh no, i accidentally created 4 race conditions, that is what is causing the issue, whoops lol

ok bigger issue, apparently i cant set particle positions

i have chucked so many if statements everywhere that the only posibility remaining is that when i try grab the ParticleSystem's particle's positions it fails

yep im certain

it has 0 length

despite there being 1000 if not more particles in the system

weird

hmm, ill eat dinner and think about the problem more soon, thanks for all the help so far lol!

[livid aspen]

i meant - set particle Start Lifetime to infinity, so whatever you spawn, doesnt die after N seconds.

you can type infinity into the Start Lifetime field.

[glossy zephyr]

dont worry i did lol

[glossy zephyr]

@livid aspen ok ive fixed it ish, my code works now, there are no race conditions, the only thing left to do is to work out how on earth i dispose of PColours, which despite my best efforts i cannot get to work

oh also it refuses to render

at this point im tempted to see if i can get rendering working without PColours and then add PColours back in later to deal with

[glossy zephyr]

actually i just thought of something

why am i trying to allocate and dispose every single time? Can't i just allocate once on start, and then dispose once on disable?

ok turns out im an idiot and i dont know how to add particles to the particle system

ive be using SetParticles, which doesnt actually add particles to the particle system, for some reason?

ok nevermind, it randomly started working, weird lol

still has the weird issue of not rendering if i cant see the origin

[glossy zephyr]

from googling it appears this is a very common issue, that no one really knows how to solve

[glossy zephyr]

still got no solution, nothing seems to want to work, weird

[glossy zephyr]

you got any ideas?

[livid aspen]

No, sorry, I can't recall exactly how this works, but my best recollection is that for a system with no particles the culling uses the game object transform location to decide if its visible, but once it has particles, the bounding box should be made from those. but that doesn't line up with what you are describing.

[glossy zephyr]

ye it is odd, Thanks for all the help!

[glossy zephyr]

i worked out what the issue is, and it is because im creating the particles before i set their position, this is because this is required for dots, but this sadly causes the bounds to throw a hissy fit and claim them to be 0 0 0, the only solutions i can see would be either to disable frustrum culling, or allow dots to somehow travel back in time, the 1st option seems more possible

[glossy zephyr]

theoretically i could also try manually set the bounds to be really large (i could also try manually calculating the correct bound spots, but sorting an array of 70 thousand particles is not fun, so ill just set the the bounds to some random large number and call it good enough)

[livid aspen]

how do you create the particles and set their positions? what exact API do you call to do each step?

new unity does have a Render.bounds that you can set yourself. Think it's 2022.2 or newer, though. but really it sounds like maybe a bug, so knowing the API you use may give me a clue.

[glossy zephyr]

im using nvidia flex for getting the positions, which means im essentially just loopin through a c style array (pointer madness) and setting the positions that way, im assuming due to the particles starting with a position of 0 0 0 before i set them using nvidia flex is causing the issue, it also seems to fix itself if i set the particles default positions to have 2 particles at opposite corners very far away

it doesnt matter much now, it works good enough, thanks for all the help, all i need to do now is figure out someway to apply a bilateral blur to the particles using a post proccessing shader somehow lol

[livid aspen]

yeah, it sounds like if they all spawn at 0,0,0 it will confuse the system to think it's got a zero-sized box at the origin. newer unity versions let you set renderer.bounds, so you could also use that to set a custom bounding box, if you know how big your system will be.

[glossy zephyr]

dont think there is an easy way to tell how big it is, atleat not without sorting all the particles into biggest to smallest, so ill probably just leave it as a little option as to where the first 2 particles positions are set on that split second, so it doesnt cull weirdly, and it appears this solution works well, and the little bit of lag due to not culling isnt that bad compared to the amount of lag i had with an unbursted for loop before i started doing this dots stuff, so thanks a ton for everything!