A Touchless Vehicle Control System Using Deep Learning-Based Hand Gesture Recognition: System Design and Real-Time Implementation

The vehicle control system allows drivers to control essential vehicle features such as turning signals, windshield wipers, headlights, etc., by means of pre-defined hand gestures no physical contact is required with any vehicle controls. A Google MediaPipe-based hand area extractor was used to determine hand areas from video images, and a custom-trained YOLOv11 object detection model was used to classify detected hand areas into one of several hand gestures. In addition to the MediaPipe-based hand area extractor and the YOLOv11 object detection model, two additional components were included in order to provide reliability to both the gesture recognition and mapping to vehicle action. These are: a) Temporal Buffering Mechanism (TBM), which provides increased predictive capability for gesture recognition to minimize false positives due to gesture noise or mis-recognized gestures caused by variability in human motion  b) Adaptive Confidence Filtering Strategy (ACFS), which filters out weakly recognized gestures to ensure reliable mapping to vehicle action. Results of real time testing demonstrated that the proposed system effectively identifies and executes gesture commands mapped to vehicle action with acceptable latency. The proposed architecture addresses several significant limitations associated with: a) Driver Distraction  b) Accessibility  c) Smart Mobility and offers a scalable and practical approach to future generation intelligent vehicle control interfaces.