Real-Time Hand Gesture Recognition Using YOLOv11 for Intelligent Vehicle Control

The hand gesture recognition system presented here utilizes a YOLOv11 object detection model, which was specifically trained to recognize six pre-defined hand gestures as well as classify them based on how many fingers are opened (from zero to five). The hand gesture recognition system will map each hand gesture to one or multiple commands that can be used to interact with the vehicle without touching it. A total of about 7,500 images were created from this hand gesture data set and were then used to train and test the YOLOv11 model using a variety of lighting and backgrounds. The model was trained for 50 iterations, the mAP@0.5 (mean average precision at 0.5 IoU) of the model was 0.91, and the model's overall precision, recall, and F1 score were 86%, 75%, and 78% respectively. To further reduce the false positives caused by unstable gesture detection due to varying light sources and other environmental factors, we also added class-wise confidence threshold optimizations and implemented temporal buffer processing to provide additional stability to the gesture recognition process. The results show that the YOLOv11 object detection model works very effectively for real time hand gesture recognition when compared to previous versions of the YOLO architecture. As such, our proposed method could serve as a base for developing more advanced gesture-based HMI (human machine interface) systems for future ITS (Intelligent Transportation Systems).