Gesture-Controlled Tello Drone- Rapolas Kairys

Gesture Controlled Tello Drone Project

Goal of the Project

The main objective of this project is to control a Tello drone using gesture recognition from a laptop camera. Users can raise their arms in specific poses to make the drone move left, right, up or toggle flight (take off/land) by holding an “UP” pose for four seconds. This project aims to:

• Provide a way to interact with and control a small drone using hand gestures.
• Explore computer vision and machine learning for gesture recognition.
• Demonstrate real-time controls using multithreading to have a smooth video feed.

Description

The system uses:

• MediaPipe Pose Estimation to detect body landmarks (shoulders, elbows).
• OpenCV for video processing.
• DJITelloPy library to send commands to the Tello drone over Wi-Fi, handling takeoff, landing, and movement.
• Multithreading to ensure that drone commands (which can block) do not freeze the camera feed or the user interface.
• Custom gesture logic to determine gestures (LEFT arm up, RIGHT arm up, both arms up, or none).

When the user performs a gesture in front of the laptop camera, the system detects it and translates it into a drone command:

• Holding both arms raised for 4 seconds toggles flight (either takeoff or land).
• Once in the air, raising both arms makes the drone go up.
• Raising only the left or right arm makes the drone move left or right.
• Doing neither results in a hover command.
• The drone moves 30cm for all commands, which can be changed in drone_controller.py.
• The command has to be held for 1.5 seconds to take effect. This helps prevent accidental commands and ensure safety.
• While the program is running and a drone is connected pressing "t" makes the drone take off/land. Pressing "q" shuts down the program.