Reading Emotions in the Human Voice with AI

By Nuruzzaman Faruqui

·

Our voices carry our feelings: you can often hear anger, joy or sadness before you understand the words. This study tests how well a deep learning model can recognise emotions from speech.

99%accuracy on the TESS dataset
73%accuracy on the RAVDESS dataset
13LPC features combined with MFCCs

The problem

Identifying emotional content in speech is valuable for mental health support, customer service and human–computer interaction, but it is challenging because emotions are expressed differently by different people.

What the research did

We combined two kinds of acoustic features, 13 Linear Predictive Coefficients and Mel Frequency Cepstral Coefficients, which capture how the voice sounds, and fed them into a two-dimensional convolutional neural network. We tested it on two well-known datasets, TESS and RAVDESS.

What we found

The model recognised speech emotions with an average accuracy of 99% on TESS and 73% on RAVDESS for speaker-dependent speech. The gap shows how much results depend on the dataset.

Who benefits

  • Mental health and wellbeing apps, as one possible signal among many
  • Call centres improving customer experience
  • Voice assistants that respond more naturally

Limits and what comes next

Acted emotional speech in datasets differs from real conversation, and emotion recognition can be unreliable across languages and cultures. It should never be used to make decisions about people without their consent.

The team

This research was led by Somnath Bera and colleagues at Vidyasagar University, India, with me and Bachchu Paul.

Frequently asked questions

Can AI detect emotions from voice?

It can recognise emotions in recorded speech with varying accuracy: 99% on TESS and 73% on RAVDESS in this study.

What are MFCCs?

Mel Frequency Cepstral Coefficients, features that describe the sound of speech in a way similar to human hearing.

Why do results differ between datasets?

Datasets differ in speakers, recording conditions and how emotions are performed.

About the author

Nuruzzaman Faruqui is an award-winning AI educator and researcher who has taught artificial intelligence and machine learning at university level since 2018. A two-time winner of the Excellence in Teaching Award (2023, 2024) and winner of the Best Researcher Award (2024) at Daffodil International University, he is now a doctoral researcher in AI at Monash University, Australia. He has published 53 papers and six books, and writes these articles to make AI research easy for everyone to understand. His teaching · About Zaman · Get in touch

Keep reading