Teaching AI to Recognise Human Actions in Sports Videos

By Nuruzzaman Faruqui

·

Recognising what people are doing in a video, whether running, kicking or diving, is easy for us and hard for computers. This research tracks key points on the human body and uses them to classify actions in sports footage.

88.50%accuracy on UCF Sports
89.45%mean recognition rate on YouTube Action
2benchmark datasets

The problem

Understanding human interaction and behaviour in images and video requires many features: body shape, the position of body parts, and the environment. Tracking key body parts reliably under challenging conditions is difficult.

What the research did

We first estimated human pose using body key points and 2D skeleton features. We then used a technique called t-DSNE to remove redundant features, and fed the optimised features into a recognition engine based on a k-ary tree hashing algorithm.

What we found

The system recognised actions with 88.50% accuracy on the UCF Sports Action dataset and an 89.45% mean recognition rate on the YouTube Action database, with better body-part tracking and classification than comparable methods.

Who benefits

  • Sports analytics and coaching
  • Video search and indexing
  • Healthcare and rehabilitation, where movement analysis matters

Limits and what comes next

Results are from benchmark datasets, which are cleaner than real-world footage. Any use of action recognition on people should respect privacy.

The team

This research was led by Sandeep Trivedi with Nikhil Patel (University of Dubuque), me and Sheikh Badar ud din Tahir (Capital University of Science and Technology, Pakistan).

Frequently asked questions

What is human action recognition?

AI that identifies what a person is doing in images or video.

How does this method work?

It tracks key points on the body, removes redundant features and classifies the action with a k-ary tree hashing algorithm.

How accurate is it?

88.5% on UCF Sports and an 89.45% mean recognition rate on YouTube Action.

About the author

Nuruzzaman Faruqui is an award-winning AI educator and researcher who has taught artificial intelligence and machine learning at university level since 2018. A two-time winner of the Excellence in Teaching Award (2023, 2024) and winner of the Best Researcher Award (2024) at Daffodil International University, he is now a doctoral researcher in AI at Monash University, Australia. He has published 53 papers and six books, and writes these articles to make AI research easy for everyone to understand. His teaching · About Zaman · Get in touch

Keep reading