Research on Multimedia

Supervisors: Dr. Zhiyong Wang and Prof. David Feng

Multimedia information (i.e. image/video/audio) has greatly enriched our daily life while the increasing computing power, storage capacity, and bandwidth capacity make more and more multimedia applications feasible. For instance, it has been much more convenient to take high quality photos, add special effects, and share them with friends and family instantly. Meanwhile it has been realized that multimedia information possesses more challenges than conventional information in terms of acquisition, processing, management, and delivery. Innovative multimedia technologies, which play a crucial role in a wide range of application domains such as publishing industry, media industry, entertainment industry, scientific research, and museum, are highly demanded to further facilitate multimedia services and to more efficiently utilize multimedia information generated from diverse domains. Students involved in the projects will have the opportunity access the world class and industry level facilities of the Multimedia Studio, and will be able to develop the necessary skills for the next boom of job market.

1. Title: Automatic Image Content Annotation

Digital imagery has significantly expanded its horizon in many domains such as the personal domain and the scientific domain, which demands effective and efficient management of digital images in terms of their true contents. It is difficult to obtain the description of image content manually as a picture is worth a thousand words. It is even impossible for large scale repositories. Automatic image content annotation is to generate objective and consistent description of image content while achieving significant time and cost efficiency. Such a technique will facilitate visual data management such as next generation database in various domains including personal digital assets, entertainment, and scientific research as well as search engine. Students involved in this project will be trained to enhance their knowledge base while investigating novel image analysis techniques and machine learning techniques, and programming skills.

2. Title: Automatic Video Content Annotation

Similar with image content annotation, Automatic Video Content Annotation dealing with more content rich video data is to achieve meaningful video content representation by exploring video shots, video objects and their attributes, events, and high level concepts such as scenes and stories. This project consists of many tasks such as video content analysis and machine learning. Students can investigate specific tasks to participate in this exciting project so as to enhance their programming skills, problem solving skills, and knowledge base in the state-of-art video content analysis techniques including object segmentation, object tracking, event detection, and scene inference, and machine learning techniques.

3. Title: Web Image Annotation

The Web has been a wealthy information repository for many tasks such as knowledge discovery. This project is to leverage this advantage for image annotation, since images are often presented to better express ideas and closely related the web page content. Rather than empirically extract surrounding text to represent image contents, this project is to investigate novel techniques to acquire a web-scale repository of web documents, explore the correlation between image content and web page content and discover knowledge based on such correlation of the large scale web image repository. Students will enrich their knowledge on the Web by working on building the repository of web documents, machine learning, and image processing knowledge, and gain comprehensive understanding of web search in this project.

4. Title: Semantic Multimedia Information Retrieval

Most content-based multimedia information retrieval systems rely on similarity measurement (e.g. Euclidean distance) based on low level features. Recently, in multimedia information research area, focus has significantly shift towards extracting multimedia semantics. Therefore, new approaches to multimedia information retrieval are required to meet such new developments. This project is to develop multimedia information retrieval approaches by considering the semantic representation of multimedia data. Students will be trained to investigate approaches of knowledge representation, association discovery, and knowledge integration.

5. Title: Digital TV News Content Management

In general, TV news programs are organized into pre-defined categories such as domestic, international, sports, and finance. However, such categorizes are empirical and does not support efficient information searching. Most of time, one event is closely related to others. For example, a piece of news on a war may be influential to financial markets. This project is to investigate novel techniques to discover news categories in a finer granularity and to enable more intelligent information search. Students will gain thorough understanding of multimedia content analysis techniques.

6. Title: Multimedia Information Retrieval in Peer-to-Peer Networks

Peer-to-Peer computing has played a significant role in information sharing and many peer-to-peer networks have been developed such as BitTorrent. However, search functionality provided by these networks are based on meta data of the files being shared, which seriously constrained the sharing process since metadata does not provide accurate and complete description of the files, particularly those of rich content such as image and videos. This project is to investigate novel techniques to integrate peer-to-peer techniques with multimedia information retrieval for better file sharing. Students will gain thorough understanding of peer-to-peer techniques and multimedia information retrieval techniques.

7. Title: Plant Image Management

Obtaining sufficient knowledge of plants is crucial for the sustainable development for our earth, such as environment protection. The visual appearance of plants plays an important role in managing plant information and experienced botanists can identify plant species through visual hints of leaves such as shape contour and vein texture. This project is to improve plant image management through innovative visual information processing and machine learning techniques so as to accumulate and even explore knowledge for intelligent decision makings. Students will gain comprehensive knowledge of image processing, machine learning, computer graphics as well as interesting botanical knowledge.

8. Title: Human Motion Analysis, Modeling, Animation, and Synthesis

People are the focus in most activities; hence investigating human motion has been driven by a wide range of applications such as visual surveillance, 3D animation, advanced Human Computer Interaction, sports, and medical diagnosis and treatment. This project is to address a number of challenge issues in this area, including tracking, behaviour classification, modeling, animation, and sysnthesis. Students will gain comprehensive knowledge in computer vision, 3D modeling, and machine learning.