Flickering in Video on Remotion Lambda | Remotion | Page 1

rich vale Dec 14, 2023, 12:47 AM

#

The rendering part of the flickering (if this helps)

<>
      <Video
        ref={videoRef}
        src={videoSrc}
        width={width}
        height={height}
        style={{
          display: "none",
        }}
        crossOrigin="anonymous" // Added crossOrigin attribute
        onError={handleError} // Added error handling
      />
      {videoData && videoTexture ? (
        <ThreeCanvas
          orthographic={false}
          width={width}
          height={height}
          style={{
            background: "linear-gradient(to bottom, #2c003e, #590080)",
          }}
          camera={{ fov: 75, position: [0, 0, 500] }}
        >
          <ambientLight intensity={0.15} />
          <pointLight args={[undefined, 0.4]} position={[0, 0, 0]} />
          <mesh position={[imageXPosition, 0, 0]} rotation={[0, 0.01, 0]}>
            <planeGeometry args={[videoData.width, videoData.height]} />
            <meshBasicMaterial map={videoTexture} toneMapped={false} />
          </mesh>
        </ThreeCanvas>
      ) : null}
    </>

tepid warren Dec 14, 2023, 2:07 AM

#

Hey Jacky, try using <OffthreadVideo /> instead of <Video />

rich vale Dec 14, 2023, 2:43 AM

#

Hmm. but I'll need ThreeCanvas - how would I use that with OffthreadVideo?

tepid warren Dec 14, 2023, 2:53 AM

#

oh i see. You could try re-encode the videoSrc so that the frame rate matches that of the composition. Here is the ffmpeg command to change the frame rate to 30 fps

ffmpeg -i input.mp4 -r 30 output.mp4

rich vale Dec 14, 2023, 2:55 AM

#

Yep! I tried that. Unfortunately I still ended up with a flickering video.

tepid warren Dec 14, 2023, 2:56 AM

#

🤷 dang, I wonder what the root issue is

#

i'm guessing this issue goes away when you render locally, without lambda

rich vale Dec 14, 2023, 2:56 AM

#

Yeah- I could get it rendering well with just studio. Very weird

#

I'll make it minimally reproducible.

tepid warren Dec 14, 2023, 2:57 AM

#

are you rendering on lambda via the CLI or via the API?

rich vale Dec 14, 2023, 2:58 AM

#

APi

tepid warren Dec 14, 2023, 2:59 AM

#

try setting framesPerLambda to the total number of frames of your video. You will lose the benefits of lambda distributing the process, so this is just for troubleshooting

rich vale Dec 14, 2023, 9:54 AM

#

I've been fiddling around with framesPerLambda - i tried a low number and the total number of frames. It's bizarre - the bottom video is flickering but the top one is not. The top one is the <Video> tag and the bottom one is the ThreeCanvas tag

rich vale Dec 14, 2023, 11:54 AM

#

Ah - I've pinpointed this bug to useVideoTexture - gonna enjoy studying these Remotion internals haha

rich vale Dec 15, 2023, 1:44 AM

#

Update - I couldn't figure out what was wrong. Instead, I tried to deploy the threejs + Remotion on AWS lambda to try get that working. Unfortunately I couldn't get that working.

An error occurred:
 Error  Stopping Lambda function because error occurred while rendering chunk 0:
   Error: Error creating WebGL context.
       at new WebGLRenderer (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:27249:12)
       at createRendererInstance (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:52142:69)
       at Object.configure (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:52252:18)
       at https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:53141:20
       at Rj (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:59398:332)
       at lk (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:59415:147)
       at jk (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:59414:446)
       at jk (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:59414:343)
       at jk (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:59414:343)
       at ik (https://remotionlambda-useast1-lmmx7y5hrr.s3.us-east-1.amazonaws.com/sites/react-three-template/bundle.js:59413:431)

at node_modules/three/build/three.module.js:26800
26798 │                                 } else {
26799 │ 
26800 │                                         throw new Error( 'Error creating WebGL context.' );

#

If anyone has ideas on how to resolve this particular issue -lmk! 🙂

tepid warren Dec 15, 2023, 1:53 AM

#

lambda doesn't have a GPU 🙂 here is an option if you want to go down that path (luckily myself and @lilac nebula put this together in the last few months!)
https://www.remotion.dev/docs/miscellaneous/cloud-gpu

Using the GPU in the cloud | Remotion | Make videos programmaticall...

Here is an example workflow for how to use the GPU in the cloud to render your videos using EC2 instances.

rich vale Dec 15, 2023, 2:48 AM

#

Oh crud - didn't realize that needed GPU. Explains a lot! thanks

distant kestrel Dec 15, 2023, 12:40 PM

#

hey @rich vale!
sorry you're having trouble - it seems like this issue would appear if remotion started rendering if all delayRender() handles are resolved and videoData && videoTexture is false.

useVideoTexture() is wrapped in delayRender(), so that should be fine.

so I suppose it has something to do with videoData. but I don't know where it is coming from. could you elaborate on that?

rich vale Dec 15, 2023, 12:45 PM

#

Yep! videoData is just the getMetadata - I'll attach the code below.

#

const VideoPan: React.FC<{ textItem: TextItem }> = ({ textItem }) => {
  const { width, height } = useVideoConfig();
  const frame = useCurrentFrame();
  const videoRef = useRef<HTMLVideoElement | null>(null);
  const cameraRef = useRef<THREE.PerspectiveCamera | null>(null);
  const videoSrc =
    textItem.videoSrc ||
    "https://launchrender-audio.s3.amazonaws.com/fps_60a.mp4";

  const videoTexture = useVideoTexture(videoRef);

  const [videoData, setVideoData] = useState<VideoMetadata | null>(null);
  // Add a delay render to ensure it doesn't flicker
  const { waitUntilDone } = prefetch(videoSrc);
  // lol idk
  waitUntilDone();

  useEffect(() => {
    return () => {
      if (videoRef.current) {
        videoRef.current.src =
          textItem.videoSrc ||
          "https://launchrender-audio.s3.amazonaws.com/fps_60a.mp4";
      }
    };
  }, [videoSrc]);

  useEffect(() => {
    return () => {
      // Dispose of the video texture
      if (videoTexture) {
        videoTexture.dispose();
      }
    };
  }, []);

  useEffect(() => {
    prefetch(videoSrc);
    if (videoRef.current) {
      videoRef.current.crossOrigin = "anonymous";
    }
  }, []);

  useEffect(() => {
    if (cameraRef.current) {
      console.error("Camera reference is not available");
      // Adjust these values to control the speed and direction of the pan
      const panSpeed = 0.1;
      cameraRef.current.position.x = -width / 2 + frame * panSpeed;
    }
  }, [frame, width]);

  useEffect(() => {
    getVideoMetadata(videoSrc)
      .then((data) => {
        setVideoData(data);
      })
      .catch((err) => console.log(err));
  }, [videoSrc]);

  const handleError = (error: any) => {
    console.log("ERROR:============= ");
    console.error("Video playback error:", error);
    console.error("Video playback native event:", error.nativeEvent);
  };

  const imageXPosition = interpolate(frame, [0, textItem.duration], [0, -500]);

tepid warren Dec 15, 2023, 12:47 PM

#

(you can write tsx after the 3 opening backticks and Discord will do the syntax highlighting for you)

rich vale Dec 15, 2023, 12:54 PM

#

@tepid warren Legend

distant kestrel Dec 15, 2023, 4:27 PM

#

@rich vale I see! the problem is simply that you need to wrap getVideoMetadata in delayRender(): https://www.remotion.dev/docs/delay-render

delayRender() and continueRender() | Remotion | Make videos program...

By calling delayRender(), you are signaling that a frame should not be immediately rendered and instead should wait on an asynchronous task to complete.

rich vale Dec 16, 2023, 1:21 AM

#

Oh wait - so just confirming - I need to delay the render until getVideoMetadata has played out?

#

I'll give it a try!

distant kestrel Dec 16, 2023, 2:20 PM

#

@rich vale yep!

rich vale Dec 16, 2023, 4:42 PM

#

Damn it it still flickers. I'll make a reproducible example now

rich vale Dec 16, 2023, 5:03 PM

#

Actually - I'm wondering if the flickering might be related to not having a GPU? I'm somehow rendering ThreeCanvas fine without it

distant kestrel Dec 18, 2023, 2:36 PM

#

@rich vale wanna post the current implmenentation you have after you wrapped it in delayRender()? I will check again if there is still something wrong.

you can also post a full repro of course.

I doubt it has to do with the GPU because it works in the end! you are just missing the first frame

rich vale Dec 19, 2023, 12:43 AM

#

Hey @distant kestrel I found the bug and managed to fix it on my end! It was to do with how I was loading it 🙂

#

Had to hire someone to quickly look into it for me haha

#

(I was losing my mind)

distant kestrel Dec 19, 2023, 10:08 AM

#

hah, cool you found someone! great news then 🔥

flat citrus Dec 29, 2023, 5:22 AM

#

@rich vale You can give a shot at cloud rendering service that I am building - https://recloud-orcin.vercel.app/ instead of managing it yourself

rich vale Jan 3, 2024, 1:59 AM

#

@flat citrus G pitch - I think your websites down though

flat citrus Jan 3, 2024, 3:16 AM

#

https://recloud-orcin.vercel.app/login

#

It works here

hard wolf Feb 6, 2024, 5:37 AM

#

tepid warren lambda doesn't have a GPU 🙂 here is an option if you want to go down that path ...

Hi Matt, i am getting below error while installing nvidia driver
2024-02-06 05:35:00 (63.2 MB/s) - ‘NVIDIA-Linux-driver.run’ saved [340891704/340891704]

Verifying archive integrity... OK
Uncompressing NVIDIA Accelerated Graphics Driver for Linux-x86_64 535.104.12.................................................................................................................................................................................................

WARNING: One or more modprobe configuration files to disable Nouveau are already present at: /usr/lib/modprobe.d/nvidia-installer-disable-nouveau.conf, /etc/modprobe.d/nvidia-installer-disable-nouveau.conf. Please be sure you have rebooted
your system since these files were written. If you have rebooted, then Nouveau may be enabled for other reasons, such as being included in the system initial ramdisk or in your X configuration file. Please consult the NVIDIA driver
README and your Linux distribution's documentation for details on how to correctly disable the Nouveau kernel driver.

ERROR: An error occurred while performing the step: "Building kernel modules". See /var/log/nvidia-installer.log for details.

ERROR: An error occurred while performing the step: "Checking to see whether the nvidia kernel module was successfully built". See /var/log/nvidia-installer.log for details.

ERROR: The nvidia kernel module was not created.

Can you please help on this

hard wolf Feb 6, 2024, 5:59 AM

#

WARNING: You do not appear to have an NVIDIA GPU supported by the 535.104.12 NVIDIA Linux graphics driver installed in this system. For further details, please see the appendix SUPPORTED NVIDIA GRAPHICS CHIPS in the README available on the
Linux driver download page at www.nvidia.com.

WARNING: One or more modprobe configuration files to disable Nouveau are already present at: /usr/lib/modprobe.d/nvidia-installer-disable-nouveau.conf, /etc/modprobe.d/nvidia-installer-disable-nouveau.conf. Please be sure you have rebooted
your system since these files were written. If you have rebooted, then Nouveau may be enabled for other reasons, such as being included in the system initial ramdisk or in your X configuration file. Please consult the NVIDIA driver
README and your Linux distribution's documentation for details on how to correctly disable the Nouveau kernel driver.

#

Hi @tepid warren can you help me to get resolve this issue

distant kestrel Feb 7, 2024, 1:37 PM

#

hard wolf WARNING: You do not appear to have an NVIDIA GPU supported by the 535.104.12 NVI...

did you choose the exact same AMI as described in the docs? I tested this not too long ago and it was still working!

hard wolf Feb 7, 2024, 8:08 PM

#

distant kestrel did you choose the exact same AMI as described in the docs? I tested this not to...

Yes I used the same AMI but on us-east-2 region @distant kestrel

distant kestrel Feb 8, 2024, 9:26 AM

#

hard wolf Yes I used the same AMI but on us-east-2 region <@701030996430159872>

can you try again, on us-east-1, trying to follow exactly the instructions on the website and see if it works?

if you confirm that is also broken on the second try, we'll file an issue and try to fix it

#Flickering in Video on Remotion Lambda