VTube studio face tracking droping from 60 to 30 fps when minimizing window or alt- tabbing. | VTube Studio Headquarters | Page 1

merry gale Jan 20, 2024, 9:29 AM

#

Face tracking for any webcam drops from 55+ to 30+ as soon as the Vtube Studio window is out of focus, frames are not capped in nvidia profiles, happens both with a logitech c920 and my phone, a samsung a22 trough the iriun webcam app and pc software while connected by a usb cable. Happens with just VTube Studio running so it's not other software sucking up resources. Running windows 10 on an i5 12400f with 16gb of ddr4 ram at 3.200 mhz and a gtx 1660 6gb super nvidia video card. Tried every single thing from the teoubleshooting guide to no avail. As soon as the vtube studio window is minimized or becomes out of focus the framerate from face tracking drops to half and it gets fixed momentarily if I turn the webcam source off and on again. Let VTube studio run for an hour without minimizing it and it's steady at 55+ fps, minimize it or touch another window and its down to 30+ till the webcam source is restarted. Running vtube studio as admin without steam, with steams overlays disabled, gpu priority on, only spout is on as output and the fps is not capped when in background in nvidia profiles. I'v honestly no idea what's doing this.

merry gale Jan 20, 2024, 1:24 PM

#

I'v isolated the problem to OBS, apparently it was using the pre multiplied alpha option in composite mode in the spout2 source that caused it. Default composite mode runs fine at 55+ fps stable. I'll tag the thread as solved after running some tests and making sure that was it.

heady dawn Jan 20, 2024, 1:26 PM

#

merry gale I'v isolated the problem to OBS, apparently it was using the pre multiplied alph...

I have a similar issue, sorry, but would you be so kind as to comment on which option you activated or deactivated to solve the problem?

merry gale Jan 20, 2024, 1:29 PM

#

what are you using to capture vtube studio in obs? tehere's virtual camera, spout2 and I think ndi, I'm using spout 2, in the source options, the lil cog icon theres an option that says composite mode, I switched to default and it seems to run fine at 55+ frames now

#

nope, I'm back to 30 fps

heady dawn Jan 20, 2024, 1:32 PM

#

Thank you very much for responding so quickly. In my case, I had it configured as you mentioned, but I believe my issue may be caused by something else

merry gale Jan 20, 2024, 1:33 PM

#

try setting composite to legacy

#

seems to be working so i'm crossing my fingers

#

remember to turn the cam off and back on again to get back to 60 fps

#

haha, don't worry I'm working on this as we speak, honestly didn't think I'd find a solution

#

nope, back to 30 fps... I'll post back if I do find a solution but at least I think it's most likely OBS not vtube studio itself

heady dawn Jan 20, 2024, 1:37 PM

#

Thank you very much for your help. I hope to find a solution. Until a few days ago, it was working perfectly. I wonder what could be causing it...

merry gale Jan 20, 2024, 1:58 PM

#

are you using a webcam? I'm capturing the face tracking trough my phone trough an app called iriun webcam

#

it's not OBS, back to square 1... even with obs closed the framerate gets cut to half in VTube Studio

heady dawn Jan 20, 2024, 2:12 PM

#

Yes, I'm using a webcam, and in VTuberStudio, I perform facial tracking using NVIDIA Broadcast. To be more specific, what I'm experiencing is small interruptions in the character's movement, similar to the facial tracking being briefly cut off for about 1 second.

merry gale Jan 20, 2024, 2:12 PM

#

I think this could have been caused by a windows update and the devs haven't noticed the bug yet, there was one more person with this issue in the support thread but he never got it solved

#

oh but your issue is totally different then, I'm not having lag, I'm having the fps of the face tracking drop from 60 to 30 for some weird reason I haven't been able to pinpoint

heady dawn Jan 20, 2024, 2:25 PM

#

Sorry. I thought it would be something similar; I assumed that my lag issues could be caused by a loss of frames per second.

merry gale Jan 20, 2024, 2:51 PM

#

So this is what I'v found out, running ANY software that uses hardware acceleration makes VTube Studio's face tracking FPS drop from 55+ to 30+, turning GPU priority on or off does nothing.

#

Turning the camera source off and on again fixes the issue but as soon as you alt tab or select another software that is hardware accelerated, the issue arises again. OBS is hardware accelerated by default so I have no idea on how to solve this

sweet stirrup Jan 20, 2024, 7:54 PM

#

This is somewhat easily explained

#

don't turn on hardware acceleration if you have the monitor plugged into the iGPU on the computer

merry gale Jan 20, 2024, 7:54 PM

#

I have an i5 12400F it has no igpu

sweet stirrup Jan 20, 2024, 7:55 PM

#

When people run OBS or VTS or anything else in admin mode, the admin mode app has gpu priority

#

If you only have a dGPU, then turn the hardware acceleration off on sources playing video.

#

I guess this is a bit hard to explain, because some cameras are actually sending h264 not NV12/YUV2 to OBS

merry gale Jan 20, 2024, 7:57 PM

#

tried it already, turnod off hardware accel on chrome steam and discord

#

tried both xrgb and yuv2 with the same result

sweet stirrup Jan 20, 2024, 7:57 PM

#

Hmm

merry gale Jan 20, 2024, 7:58 PM

#

whatever it is, it's evading me,I'm an advanced user

sweet stirrup Jan 20, 2024, 7:58 PM

#

Well the logic does follow, what GPU do you have?

#

a 1660 should be sufficient

#

If you're using the regular OpenSeeFace webcam mode, it shouldn't be invoking hardware acceleration, as it only runs on the CPU.

merry gale Jan 20, 2024, 8:00 PM

#

that would be tracking mode 5, right?

sweet stirrup Jan 20, 2024, 8:01 PM

#

yes. What resolution do you have it set to?

merry gale Jan 20, 2024, 8:01 PM

#

the fps remains at 50+ fps even mwhen I minimize it for a while, tought it was obs since the scene compositon is hardware accelerated by defult but even with obs off the framerate of the tracking drops to 30+

#

1280x900 ,seems to be the crisper mode my phone will put out working as a webcam

#

tried 640x480 and it's still the same

sweet stirrup Jan 20, 2024, 8:02 PM

#

yeah I was about to ask that

#

Do you get the same problem with mediapipe?

#

media pipe only uses CPU on windows

merry gale Jan 20, 2024, 8:05 PM

#

haven't tried that, I will, but it's pretty laggy and buggy for what I need vtube studio to do

sweet stirrup Jan 20, 2024, 8:05 PM

#

So for reference, mediapipe and openseeface don't use any resolution greater than 480p

#

so you can save some processing by using 480p

merry gale Jan 20, 2024, 8:06 PM

#

that would be 640x480?

sweet stirrup Jan 20, 2024, 8:06 PM

#

But if this issue seems to be connected to hardware yeah

#

the thing that comes to mind is drivers, but people routinely complain about the logitech camera being an issue with OSF

merry gale Jan 20, 2024, 8:07 PM

#

I'm not using a logitech camera

#

a samsung s22 cellphones trough irium webcam via a usb cable

#

starting to believe that's the issue

sweet stirrup Jan 20, 2024, 8:08 PM

#

AFAIK, the nvidia card should have 2 hardware decoders minimum so if the camera is sending h264 that would be invoked, but if it's sending YUV/NV12/XRGB then it's possible the hardware acceleration is being invoked by the driver.

#

It's a bit difficult to determine what exactly any specific webcam software does since we can't see into it.

merry gale Jan 20, 2024, 8:09 PM

#

I have 2 options xrgb and yuv2

#

I understand

#

so I should be able to have full hardware acceleration as long as I'm using 1 decoder on vtube studio and 1 for the videogame I'm streaming that should be fine ,right?, but obs also uses hardware accel on everything from scene composition to sources

sweet stirrup Jan 20, 2024, 8:11 PM

#

right, there's a fixed amount of video decoder bandwidth so that can be like 2x1080p or 4x480p or some combination in between

#

If you have a bunch of chromium webviews open, that will consume hardware acceleration (think "video ads")

#

usually it's not as big of deal when it's decoding since most CPU's are fast enough to do it in software

merry gale Jan 20, 2024, 8:12 PM

#

yeah I have about 3 dozen tabs open X_X

sweet stirrup Jan 20, 2024, 8:13 PM

#

but because you have very little control over what invokes the hardware decoder, that cut in half of the frame rate tells me that something else is using the hardware decoder at the same time.

merry gale Jan 20, 2024, 8:13 PM

#

makes sense

#

I'll try closing all the tabs see if it does anything

sweet stirrup Jan 20, 2024, 8:14 PM

#

So usually what I do with OBS is turn off hardware accelerator on webviews and on the web browser being captured if you're doing that.

#

I set aside OperaGX for that

#

Chrome itself behaves kinda strange so I usually just close chrome entirely when streaming, otherwise ads tend to flash through the screen

merry gale Jan 20, 2024, 8:16 PM

#

I'm using it as a triple translator and for youtube videos for the hanging out part of the stream

#

when it comes to just playing video off youtube is firefox much better ? what do you use?

sweet stirrup Jan 20, 2024, 8:17 PM

#

I play the video in operaGX with the hardware acceleration turned off

merry gale Jan 20, 2024, 8:17 PM

#

I can keep 1 chrome window up for the translator/subtitler and just do media surfing somewhere else

sweet stirrup Jan 20, 2024, 8:18 PM

#

The translation/captions, if you are using what I think you're using are actually being run off google cloud services by the chrome browser.

merry gale Jan 21, 2024, 11:20 PM

#

thanks for all the help Kisai, I closed all chrome tabs, made aure chrome wasnt running in the background, and closed every peogram that could use hardware accel and I still get the same result, I dont think it's vtube studio, tried 4 digferent versions of the iriun webcam driver and the last version doesnt even go beyond 30+ fps, pretty sure its the buggy drivers and not the vtube studio software, the translator does run off google cloud services, I'm triying out opera gx as my default browser and I like it

shy zodiac Jan 24, 2024, 5:51 AM

#

I have the same issues except it's been happening for months now, only on certain games. The model will stutter (fps drops) but the game is fine. I have 3070 TI, 32GB ram, 12th gen i5 12600k. Notice it again because it's happening with PalWorld even though I turned all settings to low

#VTube studio face tracking droping from 60 to 30 fps when minimizing window or alt- tabbing.