daring tinsel
Hi,
We are having hard time implementing the Camera Kit in our iOS app that uses the camera to track users' body. We tried to add container to apply snap lens via child view but it did not work.
Our vision is to keep our app functioning that uses the camera to track the users, then add filters on top of that. But it seems it's a either/or kind of thing?
Any help will be appreciated.
Thanks in advance
neat crystal
Hi @daring tinsel, what is the exact issue you are experiencing?
daring tinsel
Hi <@810247455945261057>, what is the exact issue you are experiencing?
Sorry i just saw your response. Is there an email to send you the details? Or do you prefer here?
neat crystal
Hi @daring tinsel, here is fine
daring tinsel
Ok @neat crystal , I will send it here.
We have a fitness app on iOS that uses the front-facing camera to track body movements and award points based on performance. To make the experience more engaging, we wanted to add face and body filters through Camera Kit.
However, we’ve noticed that when we run our motion-tracking model and Camera Kit simultaneously, our tracking accuracy drops significantly. Without Camera Kit, a well-performed session yields around 92–98% accuracy. But when we enable Camera Kit lenses, accuracy falls to about 30–40% under the same conditions. It seems like both SDKs are competing for the front camera, resulting in an “either/or” situation.
Our question: Is there a recommended way to integrate Camera Kit so that we can use its lenses without interfering with our existing tracking model that also relies on the front camera?
Thank you for your guidance!
neat crystal
Could you check what is the video frame resolution in pixels when you use your SDK vs CK + your SDK? I wonder if it's video quality or not enough processor power to handle both
You can also measure and compare "processed frames per second" for both cases
daring tinsel
I'm not technical but I will ask my developer to check that. Is there a specific way to check these numbers?
neat crystal
there is no special API for that unfortunately, would require some extra coding
daring tinsel
Ok will do and get back to you.
daring tinsel
We checked the resolution and found that Snap has high resolution and our camera has input variant resolution.
We changed the action move camera session to HD resolution or higher and the results improved by 10-20 % accuracy but still not reached the standard accuracy of what achieved without lens.
In terms of how we added the camera kit and Implementation:
custom view in view controller, set the custom view class as preview ( class use to set in view. provided by snap kit lens controller as per source ) .
Overlayed on top of view controller to track face filter .
Behind the scene , self.view for action classification.
Both have one camera session which start together when the screen is loaded .
Remaining functions and delegates are as per the ios sample app - source code .
@neat crystal are we implementing this the right way? Is there anything else you recommend checking/fixing to improve the accuracy?