Browse By:

Sunday January 20, 2019 Login |Register

A Project of

sponsored by

Guidelines for Multiple Testing in Impact Evaluations of Educational Interventions.

Bookmark and Share Report Misuse or Glitches

Publication Date: May 2008

Publisher(s): Mathematica Policy Research, Inc.

Author(s): Peter Z. Schochet

Funder(s): Institute of Education Sciences (U.S.)

Funder(s): Institute of Education Sciences (U.S.)

Topic: Education (Statistics, research, and research methods and financing)

Keywords: Statistical Power; Educational Interventions; Multiple Testing

Type: Report


Statistical procedures that correct for multiple testing typically result in hypothesis tests with reduced statistical power because adjustment methods reduce the likelihood of identifying real differences between contrasted groups. There is disagreement among researchers about the use of multiple testing procedures and the appropriate trade-off between type I error and statistical power (type II error). These guidelines were developed to handle multiple testing in education research. In addition, the report provides details on the nature of the multiple testing problem and the statistical solutions that have been proposed; the creation of composite outcomes measures; and the Bayesian hypothesis testing approach.