Abstract

Effective Human Computer Intelligent Interaction (HCII) requires the information about the user’s identity, state and intent which can be extracted from images, so that computers can then react accordingly, e.g. systems behaving according to the emotional state of the person. The most expressive way humans display emotions is through facial expressions. Here, we propose an efficient method for emotion recognition from facial expressions in static color images containing the frontal view of the human face. Our goal is to categorize the facial expression in the given image into six basic emotional states – Happy, Sad, Anger, Fear, Disgust and Surprise. Our method consists of three steps, namely face detection and localization, facial feature extraction and emotion recognition. First, face detection is performed using a novel skin-color based segmentation and connected component analysis which is then followed by the exact face localization by using a knowledge based approach. Next, the extraction of facial features such as the eye and the mouth is performed by employing an iterative search algorithm, on the edge information of the localized face region in gray scale. Finally, emotion recognition is performed by giving the extracted eye and mouth blocks as inputs to a feed-forward neural network trained by back-propagation.