Chapter 237: Using a video as a texture in Three.js

Core Idea

If you want to embed a video as a texture in Three. js, you have multiple options.

Key Concepts

• Using @remotion/media<AvailableFrom v="4.0.387" />
• Notes
• Examples
• Using <OffthreadVideo>
• Using <Html5Video>
• See also

Code Examples

const videoSrc = 'https://remotion.media/video.mp4';

const videoWidth = 1920;
const videoHeight = 1080;
const aspectRatio = videoWidth / videoHeight;
const scale = 3;
const planeHeight = scale;
const planeWidth = aspectRatio * scale;

const Inner: React.FC = () => {
  const [canvasStuff] = useState(() => {
    const canvas = new OffscreenCanvas(videoWidth, videoHeight);
    const context = canvas.getContext('2d')!;
    const texture = new CanvasTexture(canvas);
    return {canvas, context, texture};
  });

  const {invalidate, advance} = useThree();
  const {isRendering} = useRemotionEnvironment();

  const onVideoFrame = useCallback(
    (frame: CanvasImageSource) => {
      canvasStuff.context.drawImage(frame, 0, 0, videoWidth, videoHeight);
      canvasStuff.texture.needsUpdate = true;
      if (isRendering) {
        // ThreeCanvas's ManualFrameRenderer already calls advance() in a
        // useEffect on frame change, but video frame extraction is async
        // (BroadcastChannel round-trip) and resolves after that useEffect.
        // So by the time onVideoFrame fires, the scene was already rendered
        // with the stale texture. We need a second advance() here to
        // re-render the scene now that the texture is actually updated.
        advance(performance.now());
      } else {
        // During preview with the default frameloop='always', the texture
    
// ...(truncated)

• What it demonstrates: Usage pattern for Using a video as a texture in Three.js from the official docs.

Key Takeaways

1. Understand using @remotion/media<availablefrom v="4.0.387" /> when working with Using a video as a texture in Three.js.
2. Understand notes when working with Using a video as a texture in Three.js.
3. Understand examples when working with Using a video as a texture in Three.js.
4. Understand using <offthreadvideo> when working with Using a video as a texture in Three.js.
5. Understand using <html5video> when working with Using a video as a texture in Three.js.

Connects To

• Animatedimage: related page in the Assets & Media section.
• Delaying: related page in the Assets & Media section.
• Exporting: related page in the Assets & Media section.