Software Pipeline: Canny, Hough, Quads, Warp and Compose
Setup and Calibration
Real-Time Projection Mapping onto a Moving Cube
Computer Vision | Augmented Reality | Homography & Calibration
This project implemented a real-time projection mapping system that dynamically overlays images onto the faces of a moving physical cube using only classical vision methods. I led the technical development pipeline, from physical setup and camera-projector calibration to vision processing, homography estimation, and transformation logic.
The system detects visible cube faces using Canny edge detection, Hough line transforms, and combinatorial quad validation. It computes per-face homographies to warp images onto each visible face, which are then composited and projected using a calibrated transformation into projector space.
Contributions
System Setup: Designed the full experimental setup, including projector-camera alignment, cube motion constraints, and background control for robust edge detection.
Camera–Projector Calibration: Developed a calibration pipeline using a Charuco board, calculating camera intrinsics (K, distortion) and camera→projector homography (𝐻_cp) using OpenCV and NumPy.
Quad Detection Pipeline: Implemented a Canny + Probabilistic Hough + DBSCAN-based pipeline for line segment filtering and quad hypothesis generation, followed by convexity and area checks.
Geometric Transformation: Wrote and tested homography estimation and application for each detected face, handling point ordering, occlusion, and masking.
Projection Pipeline: Built the full transformation stack: image → detected quad → combined frame → projector space warp (via 𝐻_cp), displayed fullscreen at 30 Hz.
Optimization & Debugging: Refactored quad combinatorics for speed, filtered false positives, and introduced visualization tools for each transformation stage.
Results & Limitations
-
Achieved accurate dynamic projection onto 1–3 cube faces under controlled lighting
-
Latency bottlenecks (~1s/frame) due to combinatorial quad checks and warping; proposed GPU offloading and Kalman filtering for smoothing
-
Calibration accuracy was strong but degraded under lens distortion or angle misalignment