This project implements a low-latency, high-speed stereo optical tracking system capable of calculating the 6-DOF (Degrees of Freedom) position and rotation of infrared LED constellations in 3D space.

Description

Standard optical tracking solutions often face trade-offs between cost, latency, and sample rate. While the Nintendo Wii Remote (Wiimote) contains a capable infrared camera (Pixart), its native Bluetooth connection induces significant latency and limits the sample rate to approximately 100Hz.

By bypassing the Wiimote’s Bluetooth stack and interfacing directly with the sensors, this system eliminates wireless latency and unlocks the sensor’s full potential.

Key Features

• High-Speed Tracking: Achieves a native sample rate of 200Hz by communicating directly with the sensors via I2C (double the standard Wiimote rate).
• Scalable Constellations: Natively tracks 4-LED constellations, with added support for temporal multiplexing to track up to 8 LEDs.
• Stereo Vision: Utilizes a dual-sensor setup (synchronized via cable) to provide depth perception and robust 3D localization.
• Direct USB Communication: A master microcontroller handles data acquisition and transmits it to the PC via USB, ensuring minimal delay.

Technology Stack

• Hardware: Teensy 2.0 Microcontrollers based on Atmel 32u4, Pixart Infrared Sensors (extracted from Wiimotes).
• Firmware/Communication: Custom C++ implementation handling I2C sensor bridging and USB Serial communication.
• Calibration & Mathematics: The stereo vision algorithms and camera calibration are based on reference research (Newman & Hay, 2008), utilizing MATLAB (Camera Calibration Toolbox) to determine intrinsic and extrinsic parameters.
• Software:
  • WiiSpaceDLL: Implemented a core C++ library managing the tracking logic and integration with the compiled MATLAB runtime (libcalib).
  • WiiView: Implemented a visualization client built with Qt and OGRE 3D to render the tracking data in real-time.
  • VRPN Server: Integrated Virtual Reality Peripheral Network support for broadcasting tracking data to VR applications.

Images

 

Video