Statistical Analysis of Multiple Choice Testing

The multiple-choice question exam is a very popular method of evaluation used by educators everywhere. The Air Command and Staff College Distance Learning Department uses multiple-choice exams for testing non-residence students. ACSC currently uses the Test Analysis and Development (TAD) software program's two quantitative measurements, Ease Index and Differentiation Index, to flag possible problem questions for qualitative review. They also use student feedback to flag questions for review. ACSC uses the Maxwell Academic Instructor School's Test Item Analysis Handout to examine qualitatively the flagged questions to determine which need revision. The purpose of this paper is to determine if the ACSC Distance Learning Department is doing a good job at test evaluation and whether there are better ways to determine the quality, effectiveness and fairness of multiple-choice questions.