Item response theory

Nick Shryane, ISC.

Why do we believe that questionnaires tell us something meaningful about the inner life of respondents, their knowledge, attitudes and values? What justification do we have for using their responses to claim, for example, that one individual has a higher level of ‘civic duty’ or ‘optimism’ than another?

Quantitative social research often does not engage with these issues, and takes questionnaire (and other) data at ‘face value’, ignoring potential problems of reliability and validity.

Item Response Theory (IRT) comprises a theory of measurement and a family of statistical models that aim to provide justification and evidence for the reliability and validity of multivariate data.

The talk introduced the fundamental concepts and models of IRT, and provided a starting point for researchers wanting to use IRT in their work.

Figure 1. Hypothetical Item Characteristic Curves (ICCs)

Figure 1. Hypothetical Item Characteristic Curves (ICCs), showing the relationship between the level of an unobserved trait and the probability of endorsing a binary questionnaire item, for three items. ICCs are a useful graphical way of understanding the relationships between item responses and participants’ underlying characteristics, or traits.

Further resources

  • The Basics of Item Response Theory - very old (1985), very good, very free book on IRT by Frank Baker
  • Shryane, N. M., Corcoran, R., Rowse, et. al. (2008). Deception and false belief in paranoia: modelling theory of mind stories. Cognitive Neuropsychiatry, 13(1), 8-32 - A paper of mine that applies IRT to test the measurement validity of a ‘Theory of Mind’ questionnaire. (Theory of Mind is the ability to infer what another person is thinking or feeling, and is often impaired in some illnesses, e.g. schizophrenia.)

PDF slides

Download PDF slides of the presentation 'What is item response theory?'