Recognising what people are doing in a video, whether running, kicking or diving, is easy for us and hard for computers. This research tracks key points on the human body and uses them to classify actions in sports footage.
The problem
Understanding human interaction and behaviour in images and video requires many features: body shape, the position of body parts, and the environment. Tracking key body parts reliably under challenging conditions is difficult.
What the research did
We first estimated human pose using body key points and 2D skeleton features. We then used a technique called t-DSNE to remove redundant features, and fed the optimised features into a recognition engine based on a k-ary tree hashing algorithm.
What we found
The system recognised actions with 88.50% accuracy on the UCF Sports Action dataset and an 89.45% mean recognition rate on the YouTube Action database, with better body-part tracking and classification than comparable methods.
Who benefits
- Sports analytics and coaching
- Video search and indexing
- Healthcare and rehabilitation, where movement analysis matters
Limits and what comes next
Results are from benchmark datasets, which are cleaner than real-world footage. Any use of action recognition on people should respect privacy.
The team
This research was led by Sandeep Trivedi with Nikhil Patel (University of Dubuque), me and Sheikh Badar ud din Tahir (Capital University of Science and Technology, Pakistan).
Frequently asked questions
What is human action recognition?
AI that identifies what a person is doing in images or video.
How does this method work?
It tracks key points on the body, removes redundant features and classifies the action with a k-ary tree hashing algorithm.
How accurate is it?
88.5% on UCF Sports and an 89.45% mean recognition rate on YouTube Action.