Use this URL to cite or link to this record in EThOS:
Title: Modelling usability inspection to understand evaluator judgement and performance
Author: Woolrych , Alan
ISNI:       0000 0004 2730 3458
Awarding Body: University of Sunderland
Current Institution: University of Sunderland
Date of Award: 2012
Availability of Full Text:
Access from EThOS:
This thesis presents a model of evaluator behaviour in usability evaluations and describes an evaluator centred approach to analytical usability evaluation assessment. Usability evaluations involving multiple evaluators often produce inconsistent results, even when using the same usability evaluation method. This would suggest that individual evaluator judgement is influenced by resources other than those provided by the usability evaluation method. This research sought to discover factors external to usability evaluation methods that influence evaluators in their decision making during usability evaluations, and thus, explaining such inconsistencies between evaluators. The research involved conducting two large analytical evaluations using multiple evaluators and subjecting the results to falsification testing to validate evaluation predictions. Extended Structured Problem Report Formats were specifically designed for usability problem reporting to enable the confident coding of all possible prediction types. The problem reports required explanation for decisions made in problem discovery and analysis thus aiding the identification of factors influencing evaluator judgement. The results show evaluation methods provide little support in usability problem discovery and analysis; hence evaluators rely on a variety of individual knowledge resources such as technical, design, domain and user knowledge for example. Such knowledge resources were not common to all evaluators. The absence, or inappropriate perception of different knowledge resources resulted in missed or inappropriately analysed candidate usability problems. The results provide a model of evaluator behaviour that explains why some evaluators discover usability problems that others fail to discover, and how evaluators make appropriate and inappropriate decisions about candidate usability problems that result in true and false positive, and true and false negative predictions.
Supervisor: Not available Sponsor: Not available
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID:  DOI: Not available