Use this URL to cite or link to this record in EThOS: http://ethos.bl.uk/OrderDetails.do?uin=uk.bl.ethos.578648
Title: Modelling and evaluation of paired-comparison experiments
Author: Li, Yuan
Awarding Body: University of Leeds
Current Institution: University of Leeds
Date of Award: 2011
Availability of Full Text:
Access from EThOS:
Abstract:
Paired-comparison is a popular method for deriving scale values; scale values are numbers that represent observers' psychophysical responses to sets of physical stimuli. The method requires that each observer is presented with pairs of stimuli and is asked which of the pair is greater in terms of the psychophysical property being investigated (for example, which of the pair is lighter). However, it is time consuming (especially when the number of stimuli n is large) since there are n(n-l )/2 possible paired comparisons and all of these must be considered. It is possible to carry out a so-called incomplete paired-comparison experiment where only a proportion p (0 < p < 1) of the pairs are considered. This thesis primarily addresses questions about the design of incomplete paired-comparison experiments. For instance, what is the smallest value of p and how few observers are required that still allows reliable estimates of the scale values? Monte-Carlo computational simulations were carried out with an ideal observer model assigned with bias. Data were analyzed based on Morrissey's least-squares solution. This evaluation indicated that satisfactory results can be obtained with as few as 30% (in the case where each observer compared the same pairs) or ...... 10% (in the case where each observer compared different pairs) of paired comparisons. However, the actual proportion of paired comparisons depends upon k (the number of observers) and n (the number of stimuli). A table was produced that indicated the value of p required (for various values of n and k) required to give a certain level of performance (this was somewhat arbitrarily defined as r2 = 0.95; where r is the expected Pearson product- moment correlation coefficient between the estimated scale values and their true values). A psychophysical experiment was conducted employing both the paired-comparison method and the categorical judgement method to estimate scale values. Results from the paired-comparison experiment were consistent with those predicted from the Monte-Carlo computational simulations. The paired-comparison experiment was analysed for various values of p and its performance compared with results from the categorical judgement method where n = 10. For the paired-comparison method where p = 1 (where all of the pairs are considered) the estimated scale values were more accurate than those from the categorical judgement experiment; however, as p reduces, the accuracy of the scale values from the paired-comparison method also reduced. The point where the two techniques gave broadly similar performance was at p = 0.2 (where each observer compared different sets of pairs) or p = 0.4 (where all observers compared the same set of pairs),
Supervisor: Not available Sponsor: Not available
Qualification Name: Thesis (Ph.D.) Qualification Level: Doctoral
EThOS ID: uk.bl.ethos.578648  DOI: Not available
Share: