TITLE

Rarely selected distractors in high stakes medical multiple-choice examinations and their recognition by item authors: a simulation and survey

AUTHOR(S)
Rogausch, Anja; Hofer, Rainer; Krebs, René
PUB. DATE
January 2010
SOURCE
BMC Medical Education;2010, Vol. 10, p85
SOURCE TYPE
Academic Journal
DOC. TYPE
Article
ABSTRACT
Background: Many medical exams use 5 options for multiple choice questions (MCQs), although the literature suggests that 3 options are optimal. Previous studies on this topic have often been based on non-medical examinations, so we sought to analyse rarely selected, 'non-functional' distractors (NF-D) in high stakes medical examinations, and their detection by item authors as well as psychometric changes resulting from a reduction in the number of options. Methods: Based on Swiss Federal MCQ examinations from 2005-2007, the frequency of NF-D (selected by <1% or <5% of the candidates) was calculated. Distractors that were chosen the least or second least were identified and candidates who chose them were allocated to the remaining options using two extreme assumptions about their hypothetical behaviour: In case rarely selected distractors were eliminated, candidates could randomly choose another option - or purposively choose the correct answer, from which they had originally been distracted. In a second step, 37 experts were asked to mark the least plausible options. The consequences of a reduction from 4 to 3 or 2 distractors - based on item statistics or on the experts' ratings - with respect to difficulty, discrimination and reliability were modelled. Results: About 70% of the 5-option-items had at least 1 NF-D selected by <1% of the candidates (97% for NF-Ds selected by <5%). Only a reduction to 2 distractors and assuming that candidates would switch to the correct answer in the absence of a 'non-functional' distractor led to relevant differences in reliability and difficulty (and to a lesser degree discrimination). The experts' ratings resulted in slightly greater changes compared to the statistical approach. Conclusions: Based on item statistics and/or an expert panel's recommendation, the choice of a varying number of 3-4 (or partly 2) plausible distractors could be performed without marked deteriorations in psychometric characteristics.
ACCESSION #
57221932

 

Related Articles

  • Pick-N multiple choice-exams: a comparison of scoring algorithms. Bauer, Daniel; Holzer, Matthias; Kopp, Veronika; Fischer, Martin R. // Advances in Health Sciences Education;May2011, Vol. 16 Issue 2, p211 

    To compare different scoring algorithms for Pick-N multiple correct answer multiple-choice (MC) exams regarding test reliability, student performance, total item discrimination and item difficulty. Data from six 3rd year medical students' end of term exams in internal medicine from 2005 to 2008...

  • Assessing declarative and procedural knowledge using multiple-choice questions. Abu-Zaid, Ahmed; Khan, Tehreem A. // Medical Education Online;2013, Vol. 18, p1 

    A letter to the editor is presented which discusses the use of multiple choice questions for the assessment of medical students.

  • The Effect of Answer Switching on Multiple-choice Test Performance. Mousavi, Seyed Abdollah; mousavi, seyed jaber; esbuei, robabeh mirzaei // Journal of Mazandaran University of Medical Sciences (JMUMS);12/21/2014, Vol. 24 Issue 117, p214 

    Background and purpose: In recent decades, several studies have been performed to assess the impact of answer switching on multiple choice questions final score. But still there is a general belief that changing the initial response could decrease the test scores. The aim of this study was to...

  • Is a picture worth a thousand words: an analysis of the difficulty and discrimination parameters of illustrated vs. text-alone vignettes in histology multiple choice questions. Holland, Jane; O'Sullivan, Robin; Arnett, Richard // BMC Medical Education;10/26/2015, Vol. 15, p1 

    Background: Advances in cognitive load theory have led to greater understanding of how we process verbal and visual material during learning, but the evidence base with regard to the use of images within written assessments is still sparse. This study examines whether the inclusion of images...

  • INTELLIGENT QUESTIONNAIRE DESIGN FOR EFFECTIVE PARTICIPANT EVALUATIONS. ELIAS, LISA // Training & Development (1839-8561);Apr2015, Vol. 42 Issue 2, Special section p8 

    The article offers suggestions for designing questionnaire forms to evaluate participants and also learning and development programs. Topics discussed include steps to be followed for designing questionnaire includes objectives of conducting the survey, questions based on the objectives, and...

  • INTELLIGENT QUESTIONNAIRE DESIGN FOR EFFECTIVE PARTICIPANT EVALUATIONS. ELIAS, LISA // Training & Development (1839-8561);Feb2015, Vol. 42 Issue 1, p8 

    The article offers suggestions for designing questionnaire forms to evaluate participants and also learning and development programs. Topics discussed include steps to be followed for designing questionnaire includes objectives of conducting the survey, questions based on the objectives, and...

  • Detecting cheating in written medical examinations by statistical analysis of similarity of answers: pilot study. McManus, I. C.; Lissauer, Tom; Williams, S. E. // BMJ: British Medical Journal (International Edition);5/7/2005, Vol. 330 Issue 7499, p1064 

    Objective To assess whether a computer program using a variant of Angoff's method can detect anomalous behaviour indicative of cheating in multiple choice medical examinations. Design Statistical analysis of 11 examinations held by the Royal College of Paediatrics and Child Health. Setting UK...

  • What the educators are saying.  // BMJ: British Medical Journal (International Edition);7/9/2005, Vol. 331 Issue 7508, p93 

    Presents excerpts from several medical education journals as of July 9, 2005. Impact of confidence on the outcome of evaluations of a clinician's skills; Results of research on answering strategies for answering multiple choice questions; Report that a physician's risk of a car accident...

  • The Effect of Schema-Vs-Translation-Based Instruction on Persian Medical Students' Learning of General English. Khodadady, Ebrahim; Elahi, Majid // English Language Teaching;Jan2012, Vol. 5 Issue 1, p146 

    This study explored the effect of employing two language teaching approaches, i.e., schema-based instruction (SBI) and translation-based instruction (TBI) on the structure and vocabulary knowledge as well as reading comprehension ability of sixty undergraduate students studying general English...

Share

Read the Article

Courtesy of THE LIBRARY OF VIRGINIA

Sorry, but this item is not currently available from your library.

Try another library?
Sign out of this library

Other Topics