Use this URL to cite or link to this record in EThOS:
Title: Affective and cognitive state modelling within human-computer interaction
Author: Samara, Anas
ISNI:       0000 0004 8503 9384
Awarding Body: Ulster University
Current Institution: Ulster University
Date of Award: 2018
Availability of Full Text:
Access from EThOS:
Access from Institution:
There is an opportunity and necessity to enhance computer systems with automated intelligence in order to permit natural and reliable interaction similar to human-human interaction. The recent availability of unobtrusive input modalities, such as eye trackers and web cameras, has enabled the viable real-time detection of user's emotions and mental states. However, a key challenge is utilising such modalities to enable a computer to actively interact with users based on their emotions and mental states. Consequently, the aim of this PhD is to develop a user model that will be utilised to infer a user's emotional and cognitive state, which may be subsequently exploited to adapt the user experience in order to maximise the performance of the system and guarantee task completion. An underlying framework for adaptive Human-Computer Interaction has been developed that comprises a Perception Component responsible for capturing and modelling affective and cognitive states, and an Adaptation Component, which drives the appropriate adaptation to the User Interface or User Experience. The research presented within this thesis primarily contributes to the Perception Component by probing the use of facial expressions as an input modality for computer systems. Additionally, eye-gaze tracking data has been investigated for assessing and modelling cognitive workload. This work presented herein involves data collection from unobtrusive input modalities, as well as the development of machine learning algorithms to build user models from the collected data. Subsequently, user affective states and cognitive load have been modelled and investigated within various computer-based tasks. Moreover, the credibility of using facial expressions for modelling affective states and pupil size for modelling cognitive load has been explored and discussed. Intrinsically, pupil size variation can be used to model cognitive load during interaction with user interfaces, while facial expressions do not reflect the actual feelings of the user in that context.
Supervisor: Bond, Raymond ; Galway, Leo ; Wang, Hui Sponsor: Ulster University
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID:  DOI: Not available
Keywords: Affective Computing ; Facial Expression Analysis ; Human Computer Interaction ; Cognitive Load ; Eye Tracker