You've been framed - keyframe technology

25 October 2013

Keyframe technology in action
Keyframe technology in action

Important visual sequences in news, security or personal camera footage will be faster to identify thanks to a keyframe technique developed by University of Sydney researchers.

Researcher of the technique, Dr Zhiyong Wang, senior lecturer at the School of Information Technologies says a surge in video recording devices such as digital cameras, mobile phones or tablets has led to an increase in consumer demand for effective and efficient methods of managing, storing or retrieving video data.

Dr Wang says the new technique will assist companies or individuals to quickly access the most interesting visual information in a video such as abnormal activity or behaviour dramatic facial expressions, or identifying the most important frames in lengthy video footage can be time consuming. PhD candidate, Ron Genliang Guan, who worked with Dr Zhiyong Wang to design the algorithms behind keyframe selection technology says the technique has the potential to revolutionise editing and viewing of video footage such as YouTube clips or security camera footage.

The pair who started working on the technique two years ago say:

"The goal of keyframe selection is to best represent a video with a minimal number of frames. Structurally a video is composed of a number of stories; each story is essentially a sequence of images which can have thousands of frames."

"The algorithm we have developed allows you to identify and extract important images by utilising the key points within a video.

It forms a global pool of unique key points for the entire length of the footage. These frames that sufficiently cover the key points are selected as keyframes for the footage," they state.

"The identified keyframes can serve as the table of contents or trailer of a video, like a headline of a news article," suggests Dr Wang.

The team applied their algorithm to numerous types of footage ranging from as personal videos, to news videos and sports videos.

Fast forwarding does not accommodate the content variation of a video claims Guan.

"Most existing keyframe selection methods focus on characterising the visual content of individual frames. As a result, important frames can slip past the user or viewer of the footage.

"We took a different approach and focused on the local details which create the visual content. The approach is intuitive, as the selected keyframes represent the prominent visual appearances and variations within a shot.

"For the users this means a reduction in the volume of data associated with a sequence, making the editing and viewing process significantly faster. It also assists in decreasing the data that needs to be stored, downloaded or transmitted, "says Guan.

Using the concept Guan has recently developed an app known as Visummly to help mobile phone users select the important frames from a video captured on their handsets.

For a demonstration of keyframe extraction click on the link below.

Follow University of Sydney Media on Twitter

Media Contact: Victoria Hollick, 02 9351 2579, 0401 711 361,