Behavior Recognition for ASD Children through Audio & Video for Psychological Intervention using AI

Autism Spectrum Disorder (ASD) is a neurological condition that impacts an individual's cognitive, emotional, physical, and social well-being. This research focus on a multimodal method approach that utilizes both video and audio data. By integrating analyses of facial expressions and speech-related emotional indicators, this approach seeks to enhance the accuracy and reliability of autism diagnostics. Traditional methods, often limited to observational techniques and behavioral assessments, may not fully capture the subtle nuances of autism spectrum disorders (ASD). However, by analyzing synchronized video and audio data, it becomes possible to detect intricate patterns and variations in facial and vocal expressions that are characteristic of ASD. This multimodal system not only provides a richer dataset for analysis but also enables a more comprehensive understanding of the emotional and communicative cues associated with autism. Recognizing the gestures of autistic children is crucial for preventing meltdowns and self-harm. We introduced a method to identify gestures by detecting poses through a person pose estimation technique. The features extracted from the pose estimation are then used to develop a gesture classification model using supervised learning algorithms. Our proposed model achieved the highest accuracy with the Random Forest technique, exhibiting evaluation metrics of 83% precision and 71% recall.