The Goal
To modernize the control and perception workflow for a Stäubli TX2-60L industrial robot. The objective was to create a safe, high-fidelity Sim-to-Real environment that bridges legacy control systems with next-generation, GPU-accelerated computer vision (ROS2).
The How
- Simulation: Utilized NVIDIA Isaac Sim to create a photorealistic twin of the robot and environment for risk-free testing.
- Hybrid Architecture: Engineered a bridge between ROS1 (for robust hardware control via MoveIt and VAL3 drivers) and ROS2 (for modern perception).
- Zero-Copy Performance: Implemented the ROS NITROS bridge to transfer high-bandwidth sensor data between ROS1 and ROS2 with zero-copy overhead, enabling the use of YOLOv8 and DetectNet for real-time object detection.
- DevOps: Fully containerized the complex environment using Docker and tmuxp, ensuring reproducible deployments across GPU workstations.
The Results
Digital twin of the Stäubli TX2-60L in NVIDIA Isaac Sim
Movement of the Stäubli TX2-60L in real life
- Achieved seamless motion planning and execution on both the virtual simulation and physical hardware, validating the Sim-to-Real pipeline.
- Successfully integrated a real-time object detection loop that influences robot behavior without adding latency to the control system.
- Created a portable, modular development environment that isolates dependencies for both the simulation and the hardware drivers.