#vkResetCommandPool is extremely slow (for me) in multi-threaded context

9 messages · Page 1 of 1 (latest)

wanton fossil
#

hello people o/
I have some problems with my command buffer pools and I am quite honnestly not sure why that is 😅
I'm trying to multi-thread my render graph and at the begining of each frames, I do something like this:

    {
        LNE_PROFILE_SCOPE("ResetThreadContexts");
        uint32_t count{};
        for (auto& threadContext : frameContext.ThreadContexts)
        {
            count++;
            {
                LNE_PROFILE_SCOPE("ResetCommandPool");
                device.resetCommandPool(threadContext.CommandPool);
            }
            threadContext.IsPrimaryCommandBufferUsed = false;
            threadContext.CurrentSecondaryIndex = 0;
            threadContext.SecondaryCommandBuffers.clear();
        }
        LNE_TRACE(std::format("Num resets: {}", count));
    }
    device.resetFences(frameContext.WaitFence);

in single threaded context, I use only one command buffer from 1 command pool in my thread contexts since every commands are dispatched on the render thread. To multi-thread the graphics commands, I create a secondary command buffers with these flags:
beginInfo.flags = vk::CommandBufferUsageFlagBits::eRenderPassContinue | vk::CommandBufferUsageFlagBits::eOneTimeSubmit;
The problem that I have is that I followed the advice of this documentation sample: https://docs.vulkan.org/samples/latest/samples/performance/command_buffer_usage/README.html
I go from 15us for the reset of my main command pool single threaded to a whopping 600us AT LEAST FOR EACH COMMAND POOLS!!!! for the reset of each command pools that were used to create secondary buffers (perf image attached).
at the beginning, I was creating these command pools with the VK_COMMAND_POOL_CREATE_RESET_COMMAND_BUFFER_BIT and thought that was the problem since it wasn't recommended in the documentation sample but it didn't change anything without the flag.
Any ideas?

#

also note that their stats were taken with the same amount of commands were dispatched in the frame in single threaded-context vs multi-threaded context, my frame is rendered as well as in single threaded context, and I don't have any validation layers error whatsoever

wanton fossil
#

vkResetCommandPool is extremely slow (for me) in multi-threaded context

old parcel
#

did you remember that resetting pools does not destroy allocated command buffers?

wanton fossil
#

... Boy oh boy... I'm not sure since I don't have access to my pc right now but I think that's the problem indeed... Instead of using my free secondary command buffer list, I'm probably re-allocating them.
When I have access to my code I'll check it out

#

Thanks a lot for this clue

old parcel
#

so the more you allocate the more it has to reset so over time that'll just grow to infinity 😂

wanton fossil
rough canyon
#

Infinity is not limit