#Sample Apps Bug: Invisible perspective camera objects in photoCapture

16 messages · Page 1 of 1 (latest)

still night
#

There's a bug for a while now where there's no visibility in the photo capture event in the sample apps. When can we espect this bug to be resolved? I rather not put resources in fixing this by extracting a frame from the videoEvent (which works fine).

zenith shard
#

Hi @still night ,

Thanks for reporting this issue. Could you provide us with more details about it? What sample app are you using? Could you share a screenshot comparing the photo output to the video output? Is it reproducible with any demo lens? If so, which one? Otherwise, would it be possible to share a sample code of a lens for which we can reproduce the issue?

still night
#

I think @golden temple found it in a sample apps outside of the Mirror one. So if you call the capturePhotoEvent any lens with for example 3D head objects will have those be invisible. Orthographic objects will be visible like screen images. Using the captureVideoEvent will display fine. Just use the mirror sample app in the docs and load a lens with a head objects and click the capture button. It will show the bug in the preview.

#

@zenith shard

zenith shard
#

Okay, great! I was able to reproduce it using the Mirrors sample above for a lens with a head object, as you described.

still night
#

Yes! That's good!

#

I did send this to @scenic crest a while back but that ticket hasn't been solved yet

#

Would be great to have this resolved as we're building a big camerakit app for mirrors here and would love to just use the photoCaptureEvent

#

Thanks for looking into it @zenith shard 🙂

zenith shard
#

Thanks for the details. We're going to start working on this issue soon. I'll keep you posted!

golden temple
#

@zenith shard
Our group is in the same boat. If you need more info: ours was built from the regular (not alternative and not mirror) ios mobile sample modified with a mirror AV input that we got from snap team some while ago. The bug is likely related to that mirrorAVinput file which is also in the mirror sample.
Hope you can help resolve this!

zenith shard
#

Hi all,
I have an update on this issue. It appears that when running Camera Kit based apps on macOS, AVCapturePhotoOutput doesn't correctly output lens features like head objects. However, there's a workaround: by setting AVCapturePhotoOutput to nil when creating PhotoCaptureOutput , we force PhotoCaptureOutput to use a pixel buffer from the video stream directly to capture photos, which works as intended!

    /// Configures the photo output to be ready to capture a new photo.
    fileprivate func configurePhotoCapture() {
        let avPhotoCaptureOutput: AVCapturePhotoOutput?
        
        if #available(iOS 14.0, *) {
            
            if ProcessInfo().isiOSAppOnMac {
                // `AVCapturePhotoOutput` is not fully supported on Mac for lens rendering. Therefore, we set it to `nil`, which forces `PhotoCaptureOutput` to use a pixel buffer from the video stream as the source for captured photos.
                avPhotoCaptureOutput = nil
            } else {
                let cpo = AVCapturePhotoOutput()
                if captureSession.canAddOutput(cpo) {
                    captureSession.addOutput(cpo)
                }
                avPhotoCaptureOutput = cpo
            }
        } else {
            avPhotoCaptureOutput = nil
        }
        
        photoCaptureOutput = PhotoCaptureOutput(capturePhotoOutput: avPhotoCaptureOutput)
        if let photoCaptureOutput = photoCaptureOutput {
            cameraKit.add(output: photoCaptureOutput)
        }
    }
#

cc @golden temple @still night

still night
#

Amazing thanks so much

#

And thank you again for the speedy insight, this really helps