Chapter 837: Picking (Manual)

Core Idea

Two common ways to figure out which object a user clicked: CPU-side raycasting (simple, but can't see through per-pixel texture transparency) and GPU-based picking (draws each object with a unique id color offscreen and reads back one pixel — handles transparency correctly at the cost of drawing every object twice).

Key Concepts

• Raycasting (Raycaster): casts a ray from the camera through the mouse position and tests it against scene geometry; bounding-sphere/box checks first avoid testing every triangle of every object.
• Raycasting's transparency blind spot: JavaScript can't inspect a material's texture alpha to know if a "hit" point is actually a transparent pixel, so raycasting can't correctly pick through cutout/alpha-tested geometry.
• GPU picking: render a second, offscreen scene where each object is drawn in a flat color that encodes its id, then read the single pixel under the cursor to look up which object was picked.
• PerspectiveCamera.setViewOffset: lets you render only the tiny sub-rectangle of the frame corresponding to the cursor, so the offscreen id-pass can be limited to effectively one pixel instead of a full-resolution render.
• Encoding id as emissive color: one implementation abuses MeshPhongMaterial by setting emissive to the id color and color/specular to black with blending: NoBlending, so the rendered pixel is exactly the id color where the texture's alpha passes alphaTest — a custom shader would be the more correct, cheaper solution.
• Cost trade-off: GPU picking solves the transparency problem (and avoids raycasting's per-triangle cost scaling with scene complexity) but requires drawing the scene a second time every pick.

Code Examples

class PickHelper {
  constructor() {
    this.raycaster = new THREE.Raycaster();
    this.pickedObject = null;
  }
  pick(normalizedPosition, scene, camera) {
    this.raycaster.setFromCamera(normalizedPosition, camera);
    const intersectedObjects = this.raycaster.intersectObjects(scene.children);
    if (intersectedObjects.length) {
      this.pickedObject = intersectedObjects[0].object;
      this.pickedObject.material.emissive.setHex(0xffff00);
    }
  }
}

• What it demonstrates: the core raycasting-based picking pattern — cast a ray from the normalized mouse position and highlight the first intersected object.

Key Takeaways

1. Raycasting (Raycaster) is the simplest picking approach but can't distinguish a transparent texture pixel from an opaque one.
2. GPU picking (render ids to an offscreen buffer, read back one pixel) correctly handles alpha-tested/transparent geometry, at the cost of a second draw pass.
3. setViewOffset on the camera can restrict that second pass to roughly the one pixel under the cursor, limiting the extra cost.
4. A production-quality GPU-picking id pass is better done with a small custom shader than by repurposing MeshPhongMaterial's lighting-affected properties.

Connects To

• PerspectiveCamera: provides setViewOffset, used to limit the GPU-picking render.
• MeshPhongMaterial: repurposed (imperfectly) as the id-encoding material in the GPU-picking example.