GESTRO: Skeleton-Based Dynamic Gesture Recognition for Robot Teleoperation
A new dataset of 11 dynamic gestures in 3,300 samples plus an attention-based LSTM model for real-time teleoperation of mobile robots.
Abstract
Lately, improving the efficiency of human-robot interaction (HRI) has gained significant importance, particularly in industrial settings, where the robot’s ability to understand and act on human gestures has become essential for a smooth collaboration. This paper contributes to the challenge of real-time gesture recognition for mobile robot teleoperation.
It introduces a new tailored dataset GESTRO with 11 dynamic gestures captured in 3,300 samples in total, dedicated for intuitively controlling mobile autonomous systems. Based on the proposed dataset, this paper introduces an LSTM (long short-term memory) model with attention mechanism, achieving an accuracy of 0.98. An extensive evaluation examines the data quality and the importance of dynamic sequence for accurate gesture detection. Finally, the proposed model is implemented on a mobile robotics system, demonstrating its usability in a realistic human-robot interaction setting.
The proposed dataset along with the presented models are made publicly available at github.com/thohemp/gestro.
Details
| Authors | Thorsten Hempel, Magnus Jung, Arman Ahmed Khan, Ayoub Al-Hamadi |
| Published in | Journal of Intelligent & Robotic Systems (2026) 112:51 (Springer) |
| Published | 16 April 2026 |
| Keywords | gesture recognition, human-robot interaction, keypoint detection, dynamic gestures, robot teleoperation |
| Code & data | github.com/thohemp/gestro |
| DOI | 10.1007/s10846-026-02395-9 |