Affective Learning Technologies

What are we doing?

The design of collaborative learning activities has not been informed by the underlying affective dynamics learners go through. Although emotions have generally been recognized as having an impact in learning, particularly during collaborative activities, researchers have found them hard to research and model. Recent advances in biomedical engineering, neuroscience and data mining have increased researchers attention to this issue. We are at a point where significant accuracy in recognizing basic emotional states is feasible through a number of approaches. The identification of affective and mental states provides a magnifying glass into the processes involved in collaborative learning activities.

Why?

New data collection and processing techniques to be developed in this project will allow researchers to record the interactions (speech and text) and physiological signals of learners engaged in a collaborative activity (both online and face to face). As part of the development work sensors that record physiological signals while students collaborate (in a laboratory scenario) will be integrated with software that processes these signals, merges it with behavioural information (i.e. what learners write) and identifies the affective states that individuals go through while they collaborate. This will constitute the first research environment to study the affective and physiological aspects of collaborative learning.

Selected publications

  • Calvo, R. A., & D'Mello, S. K. (2011). New Perspectives on Affect and Learning Technologies. New York: Springer.
  • Calvo, Rafael A., & DMello, S. (2010). Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications. IEEE Transactions on Affective Computing, 1(1), 18-37. Published by the IEEE Computer Society.
  • Check our publications section for updates

Who

  • Rafael A. Calvo
  • Payam Aghaei Pour
  • Omar Alzoubi
  • M. Sazzad Hussein
  • Hamed Monkaresi
  • Narguess Nourbakhsh

Siento System

Siento is a platform that we are developing for recording multimodal affective data in different experimental situations. We describe Siento, a system that has been used in a number of studies collecting data from physiological sensors, webcams and screen-cams that record the screen with the user interaction. The platform allows for dimensional or categorical models of emotions, self-reported vs. third party reporting. This type of platforms can improve the repeatability of experiments. Studies can vary in their underlying emotion model, their annotation process and other particulars of each experiment. Some experiments are implemented in controlled environments using stimuli and measuring responses. Others aim to study interactions (i.e. with computers) in naturalistic scenarios. Different modalities can be evaluated in different scenarios, each with different requirements.

The platform is also used for data acquisition, feature extraction and data analysis applying machine learning techniques. Affective computing tools have generally been designed ad hoc for each experiment or application domain. They have often been built using Matlab like the Augsburg biosignal toolbox (AuBT) that focuses on physiological signal processing. Other Matlab tools such PRTools provide extensive pattern recognition techniques. Matlab can also be used to design user interfaces including self-reporting tools, video processing and more. Finally, all the tools can be integrated to form the Siento platform. Siento will be freely available for research use. See video below for a demonstration.

WebEmo

A software framework for remote and automatic evaluation of web usability by capturing users’ web interaction data besides other modalities such as video or physiological signals. The framework was designed so new modalities (e.g. new physiological or other sensors and external sources of information) can be simply integrated to the system through the input interface. The framework is also capable of collecting users’ opinion through different psychometric assessments. Affective computing methods have been integrated to the framework for automatic measurement of user’s affective states.