Use this URL to cite or link to this record in EThOS: http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.707306
Title: Enhancing questionnaire design through participant engagement to improve the outputs of evaluation
Author: Hume, Colette
ISNI:       0000 0004 6061 4522
Awarding Body: University of Sunderland
Current Institution: University of Sunderland
Date of Award: 2017
Availability of Full Text:
Access through EThOS:
Access through Institution:
Abstract:
Questionnaires are habitual choices for many user experience evaluators, providing a well-recognised and accepted, fast and cost effective method of collecting and analysing data. However, despite frequent and widespread use in evaluation, reliance on questionnaires can be problematic. Satisficing, acquiescence bias and straight lining are common response biases associated with questionnaires, typically resulting in suboptimal responses and provision of poor quality data. These problems can relate to a lack of engagement with evaluation tasks, yet there is a lack of previous research that has attempted to alleviate these limitations by making questionnaires more fun or enjoyable to enhance participant engagement. This research seeks to address whether ‘user evaluation questionnaires can be designed to be engaging to improve optimal responding. The aim of this research is to investigate if response quality can be improved through enhancing questionnaire design both to reduce common response biases and to maintain participant engagement. The evaluation context for this study was provided by MIXER, an interactive, narrative-based application for intercultural sensitivity learning, used and evaluated by 9-11 year old children in the classroom context. A series of Participatory Design studies with children investigated engagement and optimal responding with questionnaires. These initial studies informed the design of a series of questionnaires created in the form of three workbooks that were used to evaluate MIXER with over 400 children. 3 A mixed methods approach was used to evaluate the questionnaires. Results demonstrate that by making questionnaire completion more enjoyable data quality is improved. Response biases are reduced, quantitative data are more complete and qualitative responses are more verbose and meaningful compared to standard questionnaires. Further, children reported that completing the questionnaires was a fun and enjoyable activity that they would wish to repeat in the future. As a discipline in its own right, evaluation is under-investigated. Similarly user evaluation is not evaluated with a lack of papers considering this issue in this millennium. Thus, this research provides a significant contribution to the field of evaluation, highlighting that the outputs of user evaluation with questionnaires are improved when participant engagement informs questionnaire design. The result is a more positive evaluation experience for participants and in return a higher standard of data provision for evaluators and R&D teams.
Supervisor: Not available Sponsor: Not available
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID: uk.bl.ethos.707306  DOI: Not available
Keywords: Information Systems
Share: