Diagnostic classification
models

A brief introduction

W. Jake Thompson, Ph.D.

Conceptual foundations

• Traditional assessments and psychometric models measure an overall skill or ability
• Assume a continuous latent trait

Traditional methods

• The output is a weak ordering of albums due to error in estimates
  • Confident Taylor Swift (debut) is the worst
  • Not confident on ordering toward the middle of the distribution
• Limited in the types of questions that can be answered.
  • Why is Taylor Swift (debut) so low?
  • What aspects do each album demonstrate proficiency or competency of?
  • How much skill is “enough” to be competent?

Music example

• Rather than measuring overall musical knowledge, we can break music down into set of skills or attributes
  • Songwriting
  • Production
  • Vocals

• Attributes are categorical, often dichotomous (e.g., proficient vs. non-proficient)

Diagnostic classification models

• DCMs place individuals into groups according to proficiency of multiple attributes

	songwriting	production	vocals

Answering more questions

• Why is Taylor Swift (debut) so low?
  • Subpar songwriting, production, and vocals
• What aspects are albums competent/proficient in?
  • DCMs provide classifications directly

Diagnostic psychometrics

• Designed to be multidimensional
• No continuum of student achievement
• Categorical constructs
  • Usually binary (e.g., master/nonmaster, proficient/not proficient)
• Several different names in the literature
  • Diagnostic classification models (DCMs)
  • Cognitive diagnostic models (CDMs)
  • Skills assessment models
  • Latent response models
  • Restricted latent class models

Benefits of DCMs

• Fine-grained, multidimensional results
• Incorporates complex item structures
• High reliability with fewer items

Results from DCM-based assessments

	songwriting	production	vocals