Use this URL to cite or link to this record in EThOS:
Title: The validity, reliability and responsiveness of procedure based assessment in simulated vascular procedures
Author: Green, Lucy Anne
ISNI:       0000 0004 5354 4693
Awarding Body: University of Hull and the University of York
Current Institution: University of Hull
Date of Award: 2014
Availability of Full Text:
Access from EThOS:
Access from Institution:
Introduction: Procedure based assessment (PBA) has been shown to be valid and reliable in the workplace however whether this translates to the simulation setting has not yet been demonstrated. Therefore the aim of this thesis is to demonstrate that PBA is a valid, reliable and responsive assessment tool in simulated vascular procedures. Methods: Three experiments based on simulated vascular operations were designed to explore the validity, reliability and responsiveness of PBA utilising 3 commonly performed vascular procedures. The global and task specific checklist (GTSC) and global summary score (GSS) of a modified PBA were analysed separately. Validity was determined by correlating performance with prior operative experience (number of operations previously observed and performed) and stage in surgical training. Reliability and responsiveness was determined by use of multiple raters and assessing change in performance over time. Results: The modified PBA was found to be a valid assessment method based on number of operations previously performed (r=0.446 p=0.029 for the GTSC and r=0.553 p= 0.005 for the GSS) but not for operations previously observed. Only the PBA GTSC was valid for stage of surgical training (r= 0.588 p=0.002). The modified PBA demonstrated good inter-rater reliability (r= 0.665 p= 0.005 for the GTSC and r= 0.843 p> 0.001 for the GSS) during simulated vascular procedures. Intra-rater reliability was not demonstrated. The PBA GSS was found to be responsive to improved performance (WSR p< 0.001) but the PBA GTSC was not (WSR p = 0.104). Conclusion: The modified PBA is a valid assessment of surgical skill when correlated with previous operative performance. Observation alone appears to contribute little to assessment outcomes. Performance is index specific and not fully dependant on training level. PBA has only partial reliability in simulated vascular procedure due to the lack of intra-rater reliability. PBA was responsive to practice effect suggesting it could be useful to monitor trainee performance in simulation. PBA potentially has a role in simulation assessment but did not demonstrate sufficient reliability for high stakes examination.
Supervisor: Chetter, Ian Sponsor: Not available
Qualification Name: Thesis (M.D.) Qualification Level: Doctoral
EThOS ID:  DOI: Not available
Keywords: Medicine