#Need help with InferenceEngine and YOLO

1 messages · Page 1 of 1 (latest)

golden frigate
#

Hello,

is there someone knowledgable in InferenceEngine here? I am trying to run the YOLO model to detect and classify objects. This technically already works. However, I am feeding the camera background of my device into the input tensor and basically everytime I run worker.ScheduleIterable(inputTensor); there seems to be a little hiccup.

I have already tried passing the camera's texture received by

Graphics.Blit(null, renderTexture, arCameraBackground.material);

to my input tensor using a compute shader as to not move the camera feed to the CPU once, no luck.

I've tried running the inference over multiple frames, no luck.
If anyone has an idea on what could fix that hiccup I would be very thankful 🙏🏻

broken cave
#

Start from profiling the hiccup.

golden frigate
#

I managed to find the problem. It seems like Unity's Functional.NMS runs entirely on the CPU ever since at least 2024 which caused the hiccup. Looks like I'll have to run my own using compute shaders. They promised back then that they'd fix it but apparently it is not of high priority, which is unfortunate because their own sample includes code using it.

gritty spade
#

for NMS

#

or at least it should be

#

what are you trying to do?

#

sentis is not a very mature platform

#

you kind of want to author your thing in torch, to have the exact output that you need, and then compile your own onnx engine and run that in unity and rely as little on their own stuff as possible