Our voices carry our feelings: you can often hear anger, joy or sadness before you understand the words. This study tests how well a deep learning model can recognise emotions from speech.
The problem
Identifying emotional content in speech is valuable for mental health support, customer service and human–computer interaction, but it is challenging because emotions are expressed differently by different people.
What the research did
We combined two kinds of acoustic features, 13 Linear Predictive Coefficients and Mel Frequency Cepstral Coefficients, which capture how the voice sounds, and fed them into a two-dimensional convolutional neural network. We tested it on two well-known datasets, TESS and RAVDESS.
What we found
The model recognised speech emotions with an average accuracy of 99% on TESS and 73% on RAVDESS for speaker-dependent speech. The gap shows how much results depend on the dataset.
Who benefits
- Mental health and wellbeing apps, as one possible signal among many
- Call centres improving customer experience
- Voice assistants that respond more naturally
Limits and what comes next
Acted emotional speech in datasets differs from real conversation, and emotion recognition can be unreliable across languages and cultures. It should never be used to make decisions about people without their consent.
The team
This research was led by Somnath Bera and colleagues at Vidyasagar University, India, with me and Bachchu Paul.
Frequently asked questions
Can AI detect emotions from voice?
It can recognise emotions in recorded speech with varying accuracy: 99% on TESS and 73% on RAVDESS in this study.
What are MFCCs?
Mel Frequency Cepstral Coefficients, features that describe the sound of speech in a way similar to human hearing.
Why do results differ between datasets?
Datasets differ in speakers, recording conditions and how emotions are performed.