An Enquiry Into the Method of Paired Comparison: Reliability, Scaling, and Thurstone’s Law of Comparative Judgment

The method of paired comparisons is used to measure individuals’ preference orderings of items presented to them as discrete binary choices. This paper reviews the theory and application of the paired comparison method, describes a new computer program available for eliciting the choices, and presents an analysis of methods for scaling paired choice data to estimate an interval scale measure of preference. A new procedure for isolating an individual’s inconsistent choices is described. Using data from five empirical studies, the reliability of respondents’ paired choices is assessed using measures of internal reliability, choice consistency, and test-retest reliability.