ROS2 Mobile Manipulator

A 4-DOF autonomous robot integrated with SLAM, Nav2, and Multimodal Human-Robot Interaction.

This project features a ROS2-based autonomous mobile manipulator capable of environment mapping, autonomous navigation, and vision-guided grasping. It supports multimodal interaction via voice and hand gestures.

GitHub Repository

🤖 Full System Demonstration

Main Demo
Full Autonomous Mission: Navigating to a target area and performing precision grasping using Image-Based Visual Servoing (IBVS).

🗺️ Navigation & Mapping

SLAM
2D SLAM and environment mapping in Gazebo/RViz.
Navigation
Autonomous point-to-point navigation using Nav2 stack.

🗣️ Multimodal Interaction

Voice Control
Voice-activated retrieval task using offline Vosk API.
Gesture Control
Touchless control and task triggering via MediaPipe hand gestures.

Key Technical Implementation

  • Hardware Integration: URDF modeling in SolidWorks and Gazebo physics simulation.
  • Sensor Fusion: Deployed EKF to fuse IMU and wheel odometry, effectively eliminating rotational drift.
  • Vision Pipeline: HSV color segmentation and dual-loop PID control for pixel-perfect object alignment.
  • Multithreading: Decoupled I/O operations (audio/vision) from the high-frequency control loop to ensure real-time response.