Honours Projects 2009
Projects supervised by Ronnie Taib (NICTA) and Vera Chung
Exploring 3D Haptic and Speech Interaction
The Phantom [1] is a desktop haptic device providing precise 3D computer input with high-fidelity force feedback. The research theme of this project is to determine situations and applications where the use of speech and the Phantom are relevant. For example, a user may want to describe or interact with 3D objects, but use natural language instead of learn a complex set of shortcuts.
The student will carry out a literature review in the field, while getting familiar with the hardware and software environment. This will lead to determine suitable application criteria, and the definition of a demonstrator application, in part A. The second phase of the project (thesis part B) will be devoted to the design and implementation of the application. A requirement will be set on instrumenting the code (i.e. providing logs and/or real-time probes) so that user input can be analysed in great detail. Finally, a user experiment will be designed and conducted to test the relevance of the application and unveil strengths and weaknesses.
Research Environment
The selected student will work with a team of senior researchers in the field of human-computer interaction, in the newly built NICTA premises at the Australian Technology Park.
Novelty and Contribution
Understanding how people interact using multiple modalities such as speech and gesture is still an open research question. This project will help fill the gap in specific domains requiring precise 3D positioning such as medical applications.
Expected Outcomes
As described in the abstract, the student will deliver a literature review, followed bu the implementation of the demonstrator application.
This project will provide an excellent basis for prospective PhD students or students intending to work in the Human Computer Interaction industry (e.g. gaming) but will require very strong organisational and programming skills (Java/C++).
Reference Materials
Persona-based automated output generation
Human-human interaction mainly relies on speech and gesture, but while the nature of such interaction has been the object of much research, some of its fine-grained characteristics remain unexplored. Following the works of Oviatt et al. [1], this project intends to analyse the effect of synchronisation of various modal channels on users.
Part A will consist of a review of prior art in the domain. Based on them, an application will be defined, in order to assess the effect of synchronisation of speech and gesture on users, along with appropriate assessment metrics. The student will implement that application and test it during a user experiment in Part B.
The student will design and implement evaluation scenarios involving “cool” devices and toys such as Wiimote, wall-sized screens, speech recognisers and more.
Research Environment
The selected student will work with a team of senior researchers in the field of human-computer interaction, in the newly built NICTA premises at the Australian Technology Park.
Novelty and Contribution
The study of multimodal user interaction, e.g. using gesture and speech has many research implications in fields ranging from telecommunications, gaming to psychology and human mind models. This project addresses the automation of multimodal annotations and analysis in a bid to explore modality synchronisation in more details.
Expected Outcomes
As described in the abstract, a review of prior art, followed by the design and implementation of a comprehensive user experiment will be delivered by the student. Very strong implementation skills (Java) are required, and some knowledge of the Java Media Framework would be a plus.
This project will provide an excellent basis for students interested in a PhD in multimodal interfaces or Human-Computer Interaction. However, strong time management and implementation skills will be required.
Reference Materials
- Oviatt, S., A. DeAngeli, et al. (1997). Integration and Synchronization of Input Modes During Multimodal Human-Computer Interaction. SIGCHI conference on Human factors in computing systems, Atlanta, Georgia, United States.