Chapter 858: VR — Look to Select (Manual)

Core Idea

For controller-less VR (like Google Cardboard-style headsets), the standard interaction is "look to select": raycast from the screen center (where the camera itself is aimed) and require the user to hold their gaze on an object for a short duration, visualized with a filling gauge, before it counts as selected.

Key Concepts

• Gaze-based picking: since there's no controller, the pick ray always comes from the center of the view — pass normalized coordinates (0, 0) to the same Raycaster-based PickHelper used for mouse picking.
• Dwell time, not instant selection: requiring the user to hold their gaze for a sustained duration avoids accidental selection just from looking around.
• A sliding 2-color texture as a progress gauge: a small DataTexture with two colors, applied to a shape (e.g. TorusGeometry) with NearestFilter and ClampToEdge wrapping, can visually represent "progress" by animating the texture's offset over time (via MathUtils.mapLinear) so the boundary between the two colors sweeps across the shape.
• Screen-fixed cursor: the gauge/cursor mesh is added as a child of the camera (with the camera itself added to the scene) so it always appears in a fixed spot in the user's view regardless of head orientation.

Code Examples

class PickHelper {
  constructor() {
    this.raycaster = new THREE.Raycaster();
  }
  pick(normalizedPosition, scene, camera, time) {
    this.raycaster.setFromCamera(normalizedPosition, camera);
    const hits = this.raycaster.intersectObjects(scene.children);
    // ...flash/track the first hit, compare to previous frame's hit to build dwell time...
  }
}

// gaze-based picking always uses the screen center:
pickHelper.pick({ x: 0, y: 0 }, scene, camera, time);

• What it demonstrates: reusing a mouse-picking PickHelper for gaze-based VR selection simply by always passing the center of the screen as the pick position.

Key Takeaways

1. Gaze-based ("look to select") picking is just raycasting from the screen center — no controller input needed.
2. Requiring sustained dwell time (not an instant hit) prevents accidental selection and gives the user a chance to change their mind.
3. A two-color DataTexture with an animated offset is a lightweight way to render a progress gauge without a custom shader.
4. Parenting a cursor/gauge mesh to the camera (with the camera added to the scene) keeps it fixed in the user's field of view.

Connects To

• Picking (manual): the raycasting PickHelper pattern this lesson adapts for gaze-based VR selection.
• OrthographicCamera / DataTexture / TorusGeometry: the specific classes used to build the on-screen gauge.