Use this URL to cite or link to this record in EThOS:
Title: What do students want? : developing and validating a scale to measure student expectations of learning analytics
Author: Whitelock-Wainwright, Alexander
ISNI:       0000 0004 7656 8575
Awarding Body: University of Liverpool
Current Institution: University of Liverpool
Date of Award: 2019
Availability of Full Text:
Access from EThOS:
Access from Institution:
Higher education institutions are becoming increasingly interested in implementing learning analytics services. Reasons that are driving these intention to implement learning analytics services cover the desire to improve retention rates, learning performance, and satisfaction, to name a few. Despite these motivations, the implementation of learning analytics services remains at a nominal level, which can be attributed to the challenges that such adoptions introduce. One of these challenges refers to students having not been equally engaged in the implementation process. An example of this has been the development of learning analytics policies, which have been solely created on the basis of input from institutional managers and researchers, not students. Failing to gauge and understand what students expect from learning analytics is likely to result in a service that students are not satisfied as it does not align with their expectations. This thesis forms part of an overall multinational project known as SHEILA (Supporting Higher Education to Integrate Learning Analytics) aimed at creating a framework to address such challenges as improving student engagement in policy decision making. The main contribution of this work is the creation of a psychometrically sound instrument that provides higher education institutions with the means of measuring students' expectations (predicted and ideal) of learning analytics services (the Student Expectations of Learning Analytics Questionnaire; SELAQ). Chapter 2 presents the development of the SELAQ, which was based on the theoretical framework of expectations. The items included in the SELAQ were generated on the basis of a set of themes identified following an extensive review of the learning analytics literature. This process led to the generation of 79 items, these were then subject to peer review, which reduced the total number to 37 items. Three studies were then conducted in UK (United Kingdom) Higher Education Insitutions (pilot study, n = 191; study two, n = 674; study three, n = 191), which reduced the items from 37 to 19 (pilot study) and then from 19 to 12 (study two). In the pilot study and study two, exploratory factor analysis was used to reduce the number of items and also led to the identification of a two factor structure (Ethical and Privacy Expectations and Service Expectations). The validity of this two factor structure was supported using confirmatory factor analysis in study three. Chapter 3 presents the steps taken to increase the use of SELAQ by translating it for use in Estonia, the Netherlands, and Spain. Following the translation of the instrument for each locale, data was collected from Higher Education Institutions in each country (Estonia, n = 161; the Netherlands, n = 1247; Spain, n = 543). The collected data in each country was subject to factor analysis (confirmatory factor analysis and exploratory structural equation modelling) to evaluate the validity of the originally proposed two factor structure (Ethical and Privacy Expectations and Service Expectations) in Chapter 2. Findings showed the Dutch and Spanish versions of the SELAQ to be valid, whilst problems were encountered with the Estonian version. Chapter 4 utilises the data collected in Chapter 2 and Chapter 3 (Dutch student sample, n = 1247; English student sample, n = 191; Spanish student sample, n = 543) to determine whether the ideal and predicted scales are invariant. Utilising factor analysis techniques, specifically multi-group confirmatory factor analysis and alignment, it was found that the SELAQ scales were invariant. Moreover, the Dutch student sample was found to have high Ethical and Privacy Expectations, but low Service Expectations. The English student sample had high Service Expectations, whilst their Ethical and Privacy Expectations were low for the ideal expectation scale and comparable to the Dutch sample on the predicted expectation scale. As for the Spanish student sample, they had low Ethical and Privacy Expectations; however, their Service Expectations were high on the ideal expectation scale and low on the predicted expectation scale. Chapter 5 re-uses the data collected in Chapter 3, specifically the Dutch student sample (n = 1240; 7 respondents were dropped due to missing data), to explore whether student expectations of learning analytics are homogenous. Data from both SELAQ scales (ideal and predicted expectations) was subject to latent class analysis. For the ideal expectation scale, three groups were identified: Inflated Ideal Expectation group, High Ideal Expectation group, and Low Ideal Service Expectation group. Whereas, for the predicted expectation scale, four groups were identified: Inflated Predicted Expectation group, High Predicted Expectation group, Indifferent Predicted Expectation group, and Low Predicted Service Expectation group. Chapter 6 uses data collected from an additional sample of Irish students (n = 237) to determine whether the Big Five dimensions are personality are associated with student expectations of learning analytics. Using exploratory structural equation modelling, it was found that extraversion and neuroticism were positively related to students' Service Expectations. No personality dimension was found to be associated with Ethical and Privacy Expectations. The findings of this thesis are important for the future implementation of learning analytics services and for addressing the challenge of insufficient stakeholder engagement (Tsai, Moreno-Marcos, Tammets, Kollom, & Gašević, 2018). For one, the thesis provides a much needed framework to understand what students expect from learning analytics services, but also an instrument that can be used in multiple contexts. Furthermore, the work shows that student expectations are not homogenous and that they can be associated with specific background variables (e.g., age and personality). As for the wider implications of this work, it is clear that students should be engaged in any form of learning analytics service implementation as they are shown to have strong expectations. As for policy makers, the work shows that an accessible policy is required that addresses data security and consent, which is based upon students have stronger expectations towards these elements than service features. Finally, for Higher Education Institutions, the work shows that any learning analytics service implementation needs to be user-centred. Based on the responses to the SELAQ from students, it is clear that student agency should be upheld. This means that services should provide information that facilitates self-regulated learning and also enable students to make self-informed decisions using their data.
Supervisor: Gašević, Dragan ; Bennett, Kate ; Tejeiro, Ricardo Sponsor: Not available
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral