A real time facial emotion recognition using depth sensor and interfacing with Second Life based Virtual 3D avatar

Abstract

Facial emotions are the nonverbal form of communication and we find it very useful when we have no words to express our feelings then we use the gestures to express the same. To detect and track emotions dynamically, we have to use Kinect Sensor since it has camera to produce 3D depth maps. The Facial emotions are detected in real-time, by applying the mesh over the tracked face and thereby identifying the desired points for extracting the features. Facial Action Code Systems (FACS) and Facial Animated Parameters (FAP) are the places of interest for depicting the emotions. Second Life is the 3D Virtual world where people are able to create a digital character called 'Avatar' and thereby interacting with the people in the virtual world. The Avatar, which shows some gestures, needs some emotions in a particular scenario. Using the real-time facial emotion recognition based on Kinect depth sensor, avatar emotions are also generated in the 3D Virtual world using Second Life. This idea can be further extended to serve as a communication link so that speech and hearing impaired people can express their emotions. � 2014 IEEE.