Vision System
Modular YOLO-based 3D vision pipeline for NVIDIA Jetson edge deployment
Overview
Modular YOLO-based 3D vision pipeline built for NVIDIA Jetson edge deployment. CUDA preprocessing handles raw sensor input; TensorRT-optimized inference delivers real-time throughput on hardware with no discrete GPU. Used on the 2023 and 2024 FRC robots for dynamic game-element tracking in 3D space.
Built the full pipeline: CUDA preprocessing, TensorRT model conversion and inference, 3D position estimation from detection bounding boxes and depth, and all deployment tooling including CMake builds, Docker images, and GitHub Actions CI. Integrated and validated on the robot coprocessor for both the 2023 and 2024 seasons.
Architecture
Raw frames enter a CUDA preprocessing stage (resize, normalize, format conversion) before hitting the TensorRT inference engine. Detections are unprojected to 3D using the camera intrinsics and a depth estimate, producing world-frame target positions the drive controller can consume directly. The modular design lets the detector model be swapped without changing the rest of the pipeline.