Researchers from the International Institute of Information Technology in Hyderabad, India, and Xerox Corp. have come up with a novel way to help computers understand human movement on video by studying cricket matches.

Teaching computers to interpret human movements on video is extremely difficult, because whatever system is used needs to understand which frames represent movement in the video. In addition, they need to work out exactly who is moving, and apply what the researchers call a “semantic label” to those movements.

The authors, from the Institute’s Centre for Visual Information Technology, noted in a research paper that each of the above tasks is achievable on its own, but doing them simultaneously is altogether more challenging.

That’s why the sport of cricket is so useful in teaching computer vision systems. Indians are absolutely fanatical about their cricket, and each game is widely documented with both video and text commentary easily available. The text commentaries, from the website ESPNCricInfo.com, details each player by name and gives a vivid description of their movements. The video actually captures their movements, and the researchers can use the text commentary to help computers interpret what’s going on.

Using the two datasets, the researchers were able to match specific frames of video with the portion of commentary describing what a player was doing in a particular moment during the match.

The researchers are still working on perfecting their system, but they believe it could well have much wider applications for use in video recognition software.

Image credit: Sticksports.com