Facial Emotion Recognition at Micro Level Using ReliefF and Facial Action Unit

Facial micro-emotion are small and quick facial expressions that show what a person is really feeling, even if they try to hide it. Analysing such emotions proves challenging since it happens within a fraction of a second, due to minor changes in facial muscles. This paper describes an implementation for detecting micro-emotions from video or webcam stream. Initially, the videos have been converted into frames, and a sliding window will convert those frames in proper and fixed segments. Face will be identified using framework MediaPipe FaceMesh. It helps to determine 468 landmarks on face. Using face width approach the faces in the frames has been normalized to fixed size images.  The relevant features /landmarks that are important for facial expression recognition has been extracted using ReliefF, to reduce the complexity of the algorithm.  It works well with the noisy image, different skin tones and lesser data set. A Facial Action Unit (AU) framework has been engaged for detecting facial movements. Further those AU s have been mapped to micro level emotions and trained by convolution neural network.  The experimentation has been performed on - CREMA D (Crowd Sourced Emotional Multimodal Actors Dataset), consisting of 7,442 original video clips. The results are promising and are able to detect the micro level emotions in noisy environment