#Open Particle System
1 messages · Page 2 of 1
since if the project is using prebuilt installer, that prebuilt installer would need to be rebuilt if you want a clean build
and that would require cleaning the cache in the engine folder (engine/build) and the install folder itself
but if you're building a project using the engine as source code (not using install version / prebuilt sdk) you'd only need to clean the project/build folder
since it builds the engine into the project
its flexible / with tradeoffs
I still think there's a bug
it should never be printing out a tofu
System: Enum Value 'CPU' must be a member of enum type .
had the same issue in our fork
so wierd that I can't repro when I just clean build
maybe something w/ installer build?
I am suspecting it's the installer build. It didn't deploy the base class wizard, so it may be lagging behind in certain motions.
but why would it be a UUID mismatch
but yes that is another issue im noticing
@opal breach i was looking into the todo list of yours for the OpenParticleSystem under LONGER TERM: rework virtual const CameraTransform& GetParticleEditorCameraTransform() = 0; to remove the mutex requirement.
in OpenParticleViewportRenderer.cpp::GetParticleEditorCameraTransform i do not see anything async stuff and we do use the TransformBus to get the transform. So wouldn't it be just to remove the mutex stuff and thats it?
Maybe? I'd want to make absolutely sure. The particules update via a job system which is threaded
Also the only place where this is called is in ParticleSystem::Tick. which comes from ParticleFeatureProcessor::Simulate.
yes, the Simulate is executed by AZ::JobFfunction
so its called by a jobmanager
So yeah, possibly a cross thread call, which is not ok
or multi-threaded call, ie, more than 1 thread at a time
but we just read, you think this could cause a big issue, just wonder.
well, the problem is that its an ebus
ebus is not thread safe to dispatch across multiple threads without a mutex
theres like 3 levels of ebus usage
no mutex for connecting/disconnecting, no mutex for looping over listeners and dispatching calls
-- this is the default. You will smash the stack or other stuff if multiple threads either dispatch or connect/disconnect at the same time
lockless dispatch
-- if enabled, allows lockless dispatch and works across threads. However, connecting or disconnecting during dispatch will smash stack or corrupt memory. Good for things that connect and disconnect at known sync points where bus it not used
full mutex
-- same mutex is used for connect/disconnect as well as dispatch
its often the case that an ebus should not have been used in places where it is
especially, in a job system
so i run and put a breakpoint, and as mentioned it comes from AZ::JobFunction, so then we would have to leave the mutex in there, but how would we rework this?
we would find a way to not use an ebus or to cache the value somewhere in the job or at a known sync point that happens during job prep on the main thread like when it creates the job
so before the jobs start (assuming that is in the main thread) store the transform and pass to jobs?
possibly, that would be one option, basically
another would be to use older tech like, not ebus, to store the transform and lock it using something like a read write lock
theres a number of ways
like ebus shines in only 3 cases
- single threded (so no need for mutex)
- multi threaded with lock + rare calls. So for example, calling it a few times per frame.
- multi threaded with lockless dispatch, and rare, controlled connect/disconnect sites on 1 trhread. So like, connecting and disconnecting to it happens in some known, non-threaded location like on the main thread, and during that period, no dispatch occurs.
in all other cases, classic things like callbacks, singletons, pre-job data fills, etc, are better
- So in our case, its not single threaded
- It is multithreaded but not few times (it depends more on the number of particles and thats why we shouldn't use the multi threaded one?
so at the end we don't want to use the ebus multi-thread version of performance?
Yeah, mainly just need more investigation. It might not even be causing perf issues in which case we dont really care
But I always deeply suspect mutex use inside a job
a job is supposed to be a tiny (microsecond / nanosecond) duration job that causes not pauses or locks
having a mutex used inside one tends to cause the threads involved to linearize
making it pointless to multithread
Wow, over 1k posts, lol
We gotta come up with a better name than Open Particle System.
nickL, is there a distinct aspect of how it is implemented in Od that we could draw out as a super power and put that in the name?
Open is redundant, it's in an open engine...
It's got a builder/processor, it has its own UI/interface, so it's not just a system, it deals with specific types of assets
We can still keep ops as the tag line since it's used so much in the code...
but the whole creation and generator frameworks needs a more marketable name that shows more pizzaz than a, "Well, we couldn't think of a better name, so we ended up with this generic, not quite accurate name..."
Particle Genie Gem?
That just randomly pop'd for me, lol...
We got Lanscape Canvas Gem
I like how that uses canvas in the name
It's been discussed how to do the editor, fully integrate into existing editor vs make it its own separate editor
a particle system is really only basically the particle generator code and we have so much more than that....
Hmm, we also have the BlueJay Design System so we don't get too far off of the uniform look of the UI and the standard UX workflows...
I'll have to get my hands on the system and start using it to QA the UI/UX
and others should chime in
From the top of the post:
"Replace default Qt controls with O3DE Controls (From Qt Control Gallery) where appropriate to skin the viewport consistently
Investigate ComboBoxWidget.cpp - this should not be necessary to customize so much with proper reflection.
Make it so that the graphics view for the particle systems editor doesn't stack everything on top of each other initially.(Why is it a graphics view instead of property view, anyway? do we plan to connect nodes together in the future?"
PCaM Gem - Particle Creation and Management Gem
Also, I think we will be having more than one particle system gem in the mid and long runs...so a name should leave open room for others
That fits the modular ethos better also
That's why I was hinting at something more descriptive about the architecture
I'll take this to another channel so we can leave this one to development
its in the upcoming release, and already in development
its probably gonna have issues so its labled as (preview)
thank you
I think the main point of preview is to put it in peoples hands so they can figure out what it needs / is missing / is broken
I still am completely unable to open a project with OpenParticle enabled. This is from near-latest dev branch.
- cmake version 3.29.0-rc2
This is on a brand new project, only Open Particle enabled beyond the standard.
I am building from an "Install" build target prebuilt engine.
I wonder why I can't repro this...
"This class type SimuCore::ParticleCore::ParticleSystem::PreWarm could not be registered with duplicated Uuid: {7C91C584-EE7D-4646-80A2-9EAFAE63DE5B}"
Your branch is on development right?
I'll see if I get the same assert. Might want to make sure that uuid appears nowhere else
The only place I find {7C91C584-EE7D-4646-80A2-9EAFAE63DE5B} is in the engine files for PreWarm.
Oh, though as I look at it, it seems to have a hardcoded directory to my build server. C:/GS_Sys/GitLab_Builds/pJK2BkzC5/0/engines/o3de/Code/Framework/AzCore
It seems to have a hardcoded directory to the runner. But then I deploy it and it's DEFINITELY not the same place.
This is my path: D:\GS_Sys\Engines\GS_Play_Engine\Code\Framework\AzCore
WEll, since you built the installer in a different machine, it will have burned that machines paths into the debug info of the file.
when the file is compiled and its generating those pdb files and other debug information, is when file and line info is baked into the binary, so I don't think thats an issue since you're using an installer version
Ah boo.
Could it be that somehow having source code engine registered, while having a different install engine, both with OpenParticleSystem could cause a conflict somehow?
Otherwise, I just have a bunch of random projects that tried using OpenParticle.
Could I manually change the UUID in the engine code and see if that somehow works?
It seems:
The log confirms the Builder DLL is being loaded twice - two different memory addresses for OpenParticleSystem.Builder.Gem:
00007FFB42062852 (OpenParticleSystem.Builder.Gem) : DestroyModuleClass
00007FFB41EC1CF4 (OpenParticleSystem.Builder.Gem) : DestroyModuleClass
This causes every reflected type (ParticleConfig, PreWarm, Config, etc.) to register twice, crashing the AssetBuilder. Let me find why it's loaded twice.
There it is. The AssetBuilder loads both:
OpenParticleSystem.dll (the runtime module)
OpenParticleSystem.Builder.Gem.dll (the builder module)
And looking at the code:
Runtime module (OpenParticleSystemModule.cpp) registers:
ParticleComponent::CreateDescriptor()
Builder module (BuilderModule.cpp) ALSO registers:
ParticleComponent::CreateDescriptor()
EditorParticleComponent::CreateDescriptor()
When the AssetBuilder loads both DLLs, ParticleComponent gets registered twice, and its Reflect() call registers ParticleConfig (UUID {4B8EE7E5-...}), PreWarm, Config, etc. twice — exactly matching every error in your log.
The Fix
The BuilderModule should not re-register components that already come from the runtime module. Edit BuilderModule.cpp:23-27:
BuilderModule()
{
m_descriptors.push_back(BuilderComponent::CreateDescriptor());
// REMOVE these two - they're already loaded via OpenParticleSystem.dll (runtime module)
// m_descriptors.push_back(ParticleComponent::CreateDescriptor());
// m_descriptors.push_back(EditorParticleComponent::CreateDescriptor());
}
The BuilderComponent itself calls ParticleConfig::Reflect() etc. in its own Reflect() method — but those are typically guarded by the serialize context checking for duplicate registrations. The problem is CreateDescriptor() which registers the component descriptor with the application's component registry, and that has no duplicate guard.
NICELY spotted! I'll get on this when I can
which is very soon
since its probably broken in stabilization
no idea why it doees not crash me tbh
I assume fixing this fixes the crash and its not just something else
I think it's that the install package seems to build the runtime and headless, and universal and yadda, all together. It may be that them all building made the overlap?
I also am using profile.
But yeah, glad there was something measurable.
Excited to give the particles a try, ahaha.
The builder really shouldn't be loading both dlls, thats odd
OPS has a dedicated module for builder
it might be because of the reuse of the module target as an include?
ok hangon the same dll is being loaded twice by what I understand up top?
I'm not sure how to repro this
it would imply that the same dll appears twice in the "list of dlls to load"
ok yeah, I found and can repro it
its because of the convoluted, and wrong, dependency tree inside the cmakelist
this causes the builder gem dll to "depend on" the runtime gem dll, which is not correct
which causes the code in o3de which generates the list of dlls to load (as in, projectname.assetbuilder.setreg in the registry folder) to be like this:
"OpenParticleSystem": {
"Targets": {
"OpenParticleSystem": {
"Modules":["OpenParticleSystem.dll"]
},
"OpenParticleSystem.Builder": {
"Modules":["OpenParticleSystem.Builder.Gem.dll"]
},
"OpenParticleSystem.Builders": {
}
}
},
I've straightened out the dependency tree
after my change:
"OpenParticleSystem": {
"Targets": {
"OpenParticleSystem.Builder": {
"Modules":["OpenParticleSystem.Builder.Gem.dll"]
},
"OpenParticleSystem.Builders": {
}
}
},
after my change:
"OpenParticleSystem": {
"Targets": {
"OpenParticleSystem.Builder": {
"Modules":["OpenParticleSystem.Builder.Gem.dll"]
},
"OpenParticleSystem.Builders": {
}
}
},
https://github.com/o3de/o3de/pull/19665
I've targetted it to stab but you could cherrypick it if you want
Pushing it through my build pipeline now, I'll report on the results.
Sigh, I had conflicted files, took the new ones, and now my build cant find recast: FindRecastNavigation.cmake
I'm too tired to comprehend how to correct this.
What does this PR do?
The include directories for RecastNavigation are all capital letters - for example, if you actually go to https://github.com/recastnavigation/recastnavigation/tree/main/Detour...
maybe?
linux only issue tho
🤞
CMake Error at cmake/LYWrappers.cmake:629 (find_package):
No "FindRecastNavigation.cmake" found in CMAKE_MODULE_PATH.
Call Stack (most recent call first):
cmake/LYWrappers.cmake:266 (ly_parse_third_party_dependencies)
Gems/RecastNavigation/Code/CMakeLists.txt:23 (ly_add_target)
CMake Warning (dev) at cmake/LYWrappers.cmake:629 (find_package):
FindRecastNavigation.cmake must either be part of this project itself, in
this case adjust CMAKE_MODULE_PATH so that it points to the correct
location inside its source tree.
Or it must be installed by a package which has already been found via
find_package(). In this case make sure that package has indeed been found
and adjust CMAKE_MODULE_PATH to contain the location where that package has
installed FindRecastNavigation.cmake. This must be a location provided by
that package. This error in general means that the buildsystem of this
project is relying on a Find-module without ensuring that it is actually
available.
Call Stack (most recent call first):
cmake/LYWrappers.cmake:266 (ly_parse_third_party_dependencies)
Gems/RecastNavigation/Code/CMakeLists.txt:23 (ly_add_target)
This warning is for project developers. Use -Wno-dev to suppress it.
is the build you're running from an sdk install build, or source code
that file should be just in Gems/RecastNavigation/3rdParty
and the code to add it to CMAKE_MODULE_PATH should be in Gems/RecastNavigation/CMAkeLIsts.txt
in case it helps, we have a gem called RecastNavigation which is a wrapper around a 3rd party library called RecastNavigation.
So our gem (Gems/RecastNavigation) has a folder called 3rdParty, and inside taht folder, is a FindRecastNavigation.cmake file.
That file clones the 3rd party from its official repo in upstream github, and adds it to the build.
It also tells the cmake install system to copy install/FindRecastNavigation.cmake into the installer if you build the installer.
FWIW, cmake has basiclaly a 3 directory system
Source files ---> build folder ---> Copies go Install Directory from build
ly_add_target etc ly_install, install, etc
so all the ly_install and other commands are just telling it to take stuff from the build tree and put it in the installer folder when you make an installer
The root cmakelists for recast is just:
o3de_gem_setup("RecastNavigation")
add_subdirectory(Code)
uh
the find files are all there.
are you based on the development tree?
this is what development looks like
https://github.com/o3de/o3de/blob/development/Gems/RecastNavigation/CMakeLists.txt
it absolutely has that line there
I believe so. The last commit was atmosero doing unlit something.
list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/3rdParty)
maybe you got a bad merge somehow?
I'll have to see what I can do.
becuase your worktree doesn't look like development
development has this
o3de_gem_setup("RecastNavigation")
# when compiling from source, use the 3rdParty folder for CMake module "find" files.
list(APPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_SOURCE_DIR}/3rdParty)
add_subdirectory(Code)
that list append does the magic
you could always diff it with develpment, ie, git diff upstream/development
or even git diff upstream/development -- Gems/ to select a subtree for diff
but it sounds like your files don't match devleopment files
Yeah, my main, and the dev branch does have it. I guess one of my cuatom edits has the recast data flagged as more modern than that dev stuff.
I don't know why, as nothing I did was around recast..
Seems it picked up:
(26.05) Fix monolithic release installer builds (#19608)
When I merged the particle fix branch.
Reverted, cherry picked, and the file is correct now. I'll see how it goes.
Asset Processor didn't crash.
Particles loaded successfully.
I'm in!
Yay!
I am excited to actually get working with it. This is the first time I've ever gotten in. 👌
any feedback is welcome, especially bugs
Just wanted to confirm that I encountered the same error yesterday on an install build in the development source dir.
yeah, it still hasn't been merged
https://github.com/o3de/o3de/pull/19665 has yet to be reviewed by any reviewer and thus hasn't made it into stabilization
and thus not into dev
This is for peeps building, I assume the dev downloadable builds from the link on o3debinaries.org has a working OPS, correct?
I'm looking to start testing the dev builds this weekend
no later than next weekend
The above fix (19665) makes it work. So as soon as there is a build available with that fix in, it should work
I also have a PR to merge it in to development (https://github.com/o3de/o3de/pull/19697)
Good news! I just found my contemporaneous notes from the origional OPS demo session we had. Woot woot, good stuff! I just scanned it and there's some excellent mentions, especially by Sid such as offloading compute shaders outside of O3DE then bringing the results back in is possible if desired. It has been coded in already.
If we get out user docs first that would be good. Then it would be great to get some dev docs also.
Once nickL gets his docs done, I'll rally the testers to add any additional details found in testing.
Yeah, I got about half done today, actaully, at least the first draft for PR
Mainly its just documenting all the modules
particles consist of modules, basicsally. Like a module which picks what it looks like, another which picks how it moves/accelerates, another that picks how it animates its UVs, etc, and you can turn them on and off, to save processing. Like if you don't need them to change color over time you can set their initial color and then leave the "color over time" module disabled. Theres quite a few modules though
xlnt
The other thing in my notes I want to test out is the presenter said that generators have the ability to communicate with each other. I was working on that for physics objects but paused the work. So I have some interesting ideas on how to use that for "cascading" effects.
There's an event system, but as far as I can tell, its forwithin the same effect.
a particle effect can have multiple emitters in it
and emitters can start idle, waiting for an event from another one (in the same "effect"). The event can be sent by any emitter in the effect on various conditions like particle death. They come with a payload of position, rotation, velocity, and the emitter being triggered can "inherit" those properties
But yeah, still documenting
some of it involves reading the code and trying to figure out what the setting does
That's an excellent feature for a particle system.
I might want to see if eventually we can extend that to use the ebus instead, or an event system that non-particles can connect to
but right now its tied to the effect itself
and its basically an event callback tight loop
which might be intentional for performance
yup @ all the above
like I can see it being useful for a few events like "the entire effect is finished" but not great for a notify each time an individual particle dies
yup, you wouldn't use it a lot on an individual particle basis. You would tho use it for groups and their interaction with deflectors for example. So it's not just about generators, it's about deflectors and flow direction due to forces
Whitebox tool and ghost collision objects are really going to become important.
But we can focus on that later...better to get these initial parts done, then have workflows and use cases give us reasons to do things and test against.
Again, this is one of the reasons I ❤️ this engine and it being "incomplete" or "unfinished" as some say, because that's not a liability, that's a super power for us as a community to do what we want.
I'm really looking forward to making this an integral part of the virtual production tooling, it has potential to be best of class because of this "opportunity"
#1497128488535199784 should prove quite helpful for OPS. Gradients/Alpha over time are a common usage for particles. Being able to visualize and edit them easily is definitely a big win.
I opened the Particle Editor in dev branch on Ubuntu and got a lot of errors in the console
I created a new particle
then clicked View -> MyParticle
and then did it again
and then O3DE just crashed
make absolutely sure your cache is not carried over from a previous version of the engine that did not have the latest particle
its always possible we need more ubuntu or vulkan testing
I'm also not sure that the absolute minimal template has enough of atom included to actually function, the "default" template is good but the minimal is like "I want to rip out almost all of the engine, maybe replace all the passes and my own renderer stuff"
does it work with a project made from the "Default" template?
we can also see about improving the minimal template if we need to. it turns a lot of vital things off - maybe it shouldn't
I don't know, will try. I am still learning this engine
Cache has to be cleared the message you are seeing is from the lowend and mainpiple mssa
I wish I knew the renderer a bit more, maybe there is a version number we can change somewhere to cause it to rebuild all shader data on a new engine version?
like, totally invalidate the cache
there probably is, but I'm not sure where to bump it and when
I had to do a lookup on the pipeline and find the mssa as mssa is not supported in lowend pipeline it's a thing no game engine supports
I then just hardcoded it to 1 or 4
unclear why their editor would be using lowend tho
by default, I mean
if all they did is create project, build, open editor
Cache issues is a huge problem it effects so much of the code
Any huge code changes i always delete the cache project folder
Don't know what is going on, but just tried mainpipeline, then opened the particle editor and O3DE engine just froze
Maybe something on my side
I will look at it more when I get home the only changes I made was vetex buffers and lowend pipeline crashes and made sure it worked in windows and Vulcan etc..
Okay, i will also try with the default template
Multisample count is mssa that comes from the mainpipeline
Shouldn't be using lowend unless he has it selected in the configuration or from the commandline
Is anybody targeting Android? I've got an android dev lurking from my group that's ready to be onboarded.
He's not gfx savvy yet, but is eager to check it out. He'll also be jumping in on the virtual production project
So there's so cross cutting interests
@boreal bobcat was able to produce the crash seems like the widget is messing up when clicking things in the view widget. Its returning a null or garbage pointer on the window name for some reason.
ya i have the fix for the view stuff
i think my changes when he accepts the PR should fix all that as i dont get any of that
How are you getting it to freeze?
I run my project via terminal
Then opened the Material Editor and it freezes
And why does the OpenParticle Editor load some assets?
mostly loading the lighting assets and the material assets to the AP. the material editor is kind of buggy for creating particles (material editor doesn't truely fully support particles). Its a topic that needs to be discussed.
I havnt worked in linux in a while mostly been testing things in windows (dx12,vulkan) - (lowendpipeline, mainpipline)
Do you have freezes too, or this is only on my machine?
The Material Canvas freezes also
for particles? or in general?
In general
What is your pc specs? Would also have to ask some people that use linux if they are getting freezes.
The main editor works fine and rendering stuff too
The second Material Canvas editor is not responsive, its just frozen
I can even switch between two editors and continue working in the main editor
Anything in the user/log file that is showing funny stuff?
Okay, now the O3DE engine works without above issues
I found the reason of all freezes in the Material Editor and Material Canvas
Thanks to the following command
${O3DE_HOME}/scripts/o3de.sh disable-gem --gem-name OpenParticleSystem --project-name ${PROJECT_NAME}
@errant condor I could try to fix those issues in Linux, do you need some help?
i think the only way to test them would to probly download my o3de version compile it build a project and test it. waiting for the PR's to go to dev before it can be tested.
only bug i have right now is if i go to New - MyParticle
View - > Emitter (doesnt re-dock to its orignal location)
(everything else works fine)
My PR hasnt gone to dev due to some Particle rendering changes to the atom as it was needed. Currently the paricles render 5 vertex buffers on a mesh imagine you have a mesh with alot of vertex's thats alot of gpu buffer calling over and over on 5 only.
I changed it to use a structured buffer and wrote a shader code to it. Now have full control over the vertex buffer to do many things with it.
I also have a cool particle change that no other game engine has that will make particles cool to use.
@opal breach all the bugs are fixed should i just add it to the PR from my other previous one or open a new PR?
The only current issue is design pattern
When you startup particle editor it creates a emitter tab.
File -> New Particle - > MyParticle
The paritcle widget creates MyParticle tab (also in that tab is default Emitter tab)
Should we just remove the Emitter tab when we create or open a particle that already exists?
What should the Emitter (default tab) be used for?
I can set it up so that New or Old particles that the Emitter tab is removed from the View widget. Only when there is no file particle at all i can show the emitter default tab again.
Good questions, the best way to go about a project like this is to seperate things if possible.
If you have a critical bug fix for crashes, its always best to keep those as their own thing whenever possible, since those could be included as emergency patches into environments where people don't want any other changes yet
Alright was able to fix it all seems to be running well and as intented. Submit the New Pr with the changes. Only changed 2 files so easy fixes and easy to follow along just QT edits.
thank you. I hope you understand the need for making small incremental changes
like for example if we decide "this is crucial" and want to include it in say, 26.05.1 point release (we don't know if we'll do one yet) its really hard to accept a giant change that has all sorts of ui changes in it, compared to a targetted bug fix that changes only the crash and nothing else
There are some cases where docking are a issue and I've worked with qt since 4.0 days and to much of a rabbit hole to fix them. It even exist in the editor itself.