Computer-human adapted interaction

Our expert: Professor Judy Kay

The Cruiser Framework is efficient, extensible and reusable software for developing tabletop applications. It is designed to operate independently of the hardware platform. Supported hardware currently includes Mimio, SMART Board, Hitachi StarBoard, DiamondTouch, iNexio, CycloTouch, PQLabs. Cruiser also supports the TUIO protocol and can work with any hardware device that supports mouse emulation (no driver required). Easy creation of plug-ins can enhance or create new functionality and this flexibility enables creation of rich applications that reuse existing components. Cruiser is written in C++ and uses OpenGL for a highly responsive display.

Our expert: Professor Judy Kay

There is little published research on providing an emotionally engaging and effective human-computer interaction that uses a high-fidelity digital representation of an actual human face. This area is primed to expand rapidly in coming years and could change the way we interact with computers. Our cross-disciplinary focus explores the theoretical foundations of creating interfaces based on realistic avatars. Collaboration with key industry groups has led to the development of the Meet Mike Avatar, a virtual reality experience that uses the unreal engine in concert with special HTC VIVE custom hardware. Created from extensive face scanning, with further contributions from research teams such as Disney Zurich Research, the eye-scanning data achieves real-time rendered digital face representation. 

Our experts: Associate Professor Bob Kummerfeld, Professor Judy Kay

Virtual reality (VR) exergames are a fun way to exercise with potential user benefits. We define a VR exergame user model (VRex) that represents a user’s exertion, as well as their goals and preferences for exercise and games. VRex can generate data about users’ actual and perceived exertion and their satisfaction with each game, demonstrating the diversity of the user models. It has the potential to serve as a foundation for game personalisation and open model interfaces.

Our experts: Professor Judy Kay, Dr Martin Tomitsch

Our media wall enables content exploration on large public information displays. Users can interact with the displays via natural hand gestures to navigate through important announcements, find map information in their current location and receive navigational instructions.

Our experts: Associate Professor Kalina Yacef, Professor Judy Kay

This project concerns the intersection of three fields: tabletops in education from a computer supported collaborative learning (CSCL) perspective; educational data mining and analytics (EDM); and research on interactive tabletop applications from a HCI perspective. This intersection raises a challenging question: how can interaction data be exploited automatically to inform teachers and enhance their awareness of students' collaborative activity using interactive tabletops? Our main objective is to provide a novel approach to the design, implementation and evaluation of the conceptual and technological infrastructure that captures student’s activity at interactive tabletops. We analyse this data through interaction data analytics techniques. 

Our expert: Professor Judy Kay

We aim to gain understanding of how to harness the power of the lockscreen, viewing it as a new form of ambient display. As part of an interdisciplinary project advancing human-computer interaction and public health, we created FIT: a lockscreen app designed to help people maintain awareness of an important goal and to track progress on it. For example, we have tested FIT in terms of a person’s vegetable intake, where the app provides an ultra-lite journaling approach, unlike conventional food journaling. FIT is quick, easy and pleasing to use and helps people stay aware of their nutrition goals.

Our expert: Professor Judy Kay

Collection of detailed personal-activity data from commercial devices such as Fitbit or smartphones may provide insights to help achieve long-term health goals. A key challenge for making use of physical activity data is that people do not wear or carry tracking devices all day, every day, resulting in incomplete data. We aim to address the challenges in harnessing this incomplete data by taking account of people’s adherence, a measure that describes the completeness of activity tracker data. We explore several operational definitions to show that the adherence definition makes a large difference to the interpretation of such data.

Our experts: Professor Margaret Allman-Farinelli, Professor Judy Kay, Associate Professor Irena Koprinska, Associate Professor Kalina Yacef

Industry partner: Cancer Council NSW

This research explores new interfaces that harness mobile apps and wearable cameras to advance our understanding of nutrition in young adults. Our technology aims to make it possible to capture a new level of rich data about food intake, including the outlets where young adults buy food, when they buy and eat it and with whom, and the detailed nutritional composition and overall contribution to diet of these foods. The knowledge gained will form the evidence-base for formulation of policy and electronic and mobile-health promotion to reverse weight-gain problems in young adults in Australia.

Our experts: Professor Judy Kay

Rekindle is a self-led, web-based intervention that provides psychosexual support to cancer survivors with sexual dysfunction after treatment. Research has found that a majority of cancer survivors have at least one unmet psychosexual need and expressed the desire for information regarding these needs. By providing adequate information to patients, there are better medical outcomes, a decrease in distress and improved overall quality of life.

Our experts: Associate Professor Bob Kummerfeld, Professor Judy Kay

SAL is a simple, situated ambient logger device designed to help people achieve long-term goals for good health and wellbeing. It enables people to adopt better habits, such as taking breaks from computer use, putting sunscreen on each morning and eating the recommended daily serves of vegetables. It contains a minimalist, simple and ambient interface and supports very simple logging of actions for each goal. It displays the progress for the day, serving as a reminder for the user about their goals. It fits into their environment, drawing on their peripheral awareness of the information it displays.

Our expert: Associate Professor Kalina Yacef

The iEngage project leverages new technologies to help promote healthier behaviours in children with regard to physical activity and nutrition. Its digital platform provides children with information, education and skills to achieve their physical activity and nutrition goals. The platform also connects with activity trackers to provide continuous feedback and summarise the daily activity on a dashboard. The richness of data collected can help various stakeholders create an unprecedented landscape of health knowledge and actual behaviours; understand/monitor the impact of program; and provide further personalisation.

Our expert: Professor Judy Kay

Virtual reality (VR) games provide a good way for people to get exercise and can help them achieve recommended levels of activity. Accordingly, it is a new tool in the arsenal we need to help people become more active. Our foundation research was the first to measure the exertion people gain when playing commercial VR games. We have also analysed how this relates to their perceived exertion. We have created a user model that represents key aspects of the person’s response to games and their fitness, based on data about heart-rate, rest and exercise recovery-rate. This user model can drive several forms of personalisation and tailor the VR exercise experience to the individual. 

Our experts: Professor Judy Kay, Associate Professor Bob Kummerfeld

We are working with the EdX platform to explore new ways to support learning, especially lifelong and personalised learning. This work links with our body of work on learner modelling, open learner models and educational data mining.

Our experts: Professor Judy Kay, Associate Professor Kalina Yacef

Our aim is to design, implement and evaluate a conceptual and technological solution that captures students’ movement using wearable devices, This will help dance teachers and students evaluate dance skills acquisition using automated, personalised feedback.

Our expert: Associate Professor Kalina Yacef

Lack of social relationships is a contributing factor for attrition in massive open online courses (MOOCs). This project aims to alleviate this phenomenon by enhancing MOOCs with a peer-recommender system to encourage student-to-student interactions. It explores various recommendation strategies and different ways to integrate the peer-recommender system into the MOOC learning process.

Our experts: Professor Judy Kay, Associate Professor Bob Kummerfeld

A university degree is bound by many internal and external syllabus requirements from many different sources. The core and elective subjects that form part of a degree must satisfy all of these requirement sets, but the semantic mapping and relationships are missing in current systems. We intend to apply open learner modelling techniques and Web 2.0 technologies to develop a curriculum mapping system that addresses these problems and allows for flexible user introspection of complex curriculum information.

Our experts: Associate Professor James Curran, Professor Alan Fekete, Associate Professor Kalina Yacef, Professor Judy Kay, Associate Professor Bob Kummerfeld

Our research in pervasive computing and personalisation has many applications, especially in computer science education. Our researchers take a scholarly approach to teaching, and important innovations include: problem-based learning and new approaches to supporting learning; groupwork skills; undergraduate research training; a series of computer science summer schools for high school students; and creation of a mentor program.

Our experts: Associate Professor Kalina Yacef, Professor Judy Kay, Associate Professor Irena Koprinska

Web-based educational systems collect a tremendous amount of electronic data, ranging from simple histories of students’ interactions to detailed traces about their reasoning. However, less attention has been given to extracting academically-useful information from it. Data mining encompasses a range of techniques and algorithms for discovering interesting patterns hidden in large data sets. In this context, our goals include creation of new data mining methods suited for turning learners’ performance data into information of relevance to educators and researchers, as well as exploitation of the patterns found to improve adaptation of teaching systems.

Our experts: Associate Professor Kalina Yacef, Professor Judy Kay, Associate Professor Irena Koprinska

Reciprocal recommender involves people as both subjects and objects of the recommendation process. Some examples of domains where such recommenders are applied are: job recommendation; people recommendation on social networks; mentor-mentee matching; and online dating. Our research focuses on data mining and personalisation techniques to find people’s preferences and ultimately to satisfy both users in a reciprocal match.

Our experts: Associate Professor Bob Kummerfeld, Professor Judy Kay

Pervasive and ubiquitous computing offers significant potential for improving our lives. A key feature is that it collects and moves huge amounts of personal data. This immense, fast-growing collection offers great potential, including augmented cognition and life-long memories, providing people with ready access to pertinent information about themselves. Life-long learning is another potential benefit: that we can learn precisely when and where we need to. While these benefits are significant, they are accompanied by the issue of how to safeguard individuals’ privacy. There is an essential trade-off between the benefits of capturing and using personal data and the risk to privacy. This project aims to create the technology to allow people to control their personal information and its use in pervasive computing environments.

Our experts: Associate Professor Bob Kummerfeld, Professor Judy Kay

The Internet of Things (IoT) consists of a large number of interconnected, low-cost devices and the framework for managing them. While this provides the means for rich and ubiquitous personalised interaction, a key gap is the lack of support for user modelling to harness and manage personal data gathered. This project fills this gap by creating the IoTum user modelling framework for Internet of Things applications. Our design goals are to make it easy for IoT application developers to use and to tackle the difficulty of building ubiquitous systems.

Our experts: Associate Professor Bob Kummerfeld, Professor Judy Kay

Personis supports an accretion/resolution approach to reasoning about people, places and devices. Its design supports user control of both the information held about them and the way that it is used.

• Professor Judy Kay, Director
• Professor Alan Fekete
• Professor Bob Kummerfield
• Professor David Lowe
• Associate Professor James Curran
• Associate Professor Irena Koprinska
• Associate Professor Kalina Yacef
• Dr Josiah Poon
• Dr Anusha Withana