Cluster-Randomized Trial for a
Classroom Learning Program

RESEARCH QUESTION

What the Trial Set Out to Test

This cluster randomized trial example follows an educational psychology research team that randomizes whole classes to a reading comprehension program or to usual teaching. The team wants to know whether end-of-term reading comprehension differs between the two groups after accounting for where students started. Classes, not students, are allocated, because the program is delivered to a whole class by its teacher.

Students taught together by one teacher tend to have outcomes more alike than students in different classes, and the number of classes, not the number of students, limits the precision. The plan states the primary analysis, the assumed intraclass correlation and the handling of students who leave before the end of term before any outcome is examined.

STUDY DESIGN AND DATA

Thirty Classes, About 750 Students and Two Time Points

A two-arm trial in which 30 classes are randomized one to one, stratified by grade, with about 25 students per class and a standardized reading comprehension score at the start and end of term. Each record holds the class, the arm, the baseline and end-of-term scores, age, sex and a language-background indicator. All students enrolled in a class at randomization form the analysis population, identified from school rosters before the program starts, so that who is counted does not depend on knowing the class's arm.

METHOD AND ASSUMPTIONS

Why the Class Is the Unit, and What Must Hold

Analyzing students as independent when classes were randomized treats 750 students as 750 independent pieces of information, which overstates precision and gives intervals that are too narrow. With about 25 students per class and an assumed intraclass correlation of 0.10, the design effect is about 3.4, so the 750 students carry roughly the information of 220 independent students. The plan states that correlation and varies it, to show the precision that 30 classes can deliver across plausible values. The primary analysis is a two-level mixed model for the end-of-term score with a random intercept for class, the baseline score as a covariate and the stratification variable, which adjusts for starting differences and improves precision.

With only 30 classes, the usual large-sample approximations are too optimistic, so the plan uses a small-sample degrees-of-freedom adjustment and reports the intraclass correlation from the model with its interval. The effect is defined at the student level, and with unequal class sizes a student-weighted and a class-weighted effect can differ, so the plan names the first as primary and reports the second as a check. Students who leave before the end of term are analyzed in their original class when an end-of-term score exists. For those without one, the primary analysis assumes the scores are missing at random given the baseline score and class, and a tipping-point analysis shows how different they would have to be to change the conclusion.

OUTPUT AND INTERPRETATION

How the Results Are Read

The output has a flow diagram of classes and students from enrollment through allocation to analysis, the intraclass correlation with its confidence interval, the adjusted difference in end-of-term reading comprehension with its 95% confidence interval, and a standardized effect size whose scaling is stated. Before any outcome is seen, the team names the gain in reading comprehension it would regard as worth a term of class time, and the interval is judged against that gain.

Sensitivity analyses cover the covariance and covariate choices, the small-sample adjustment, the class-weighted estimate and the tipping-point result. The interpretation is limited to a program delivered to whole classes by their teachers, in these schools. The write-up notes that with 30 classes the interval is wide, and that an interval that includes zero is not evidence of equivalence. Secondary outcomes and subgroups are labeled exploratory.

REPORTING

What the Researcher Received

A written report structured to the CONSORT extension for cluster randomized trials, with a flow diagram that shows classes and students and statistical wording for the methods, matched to the output, for the team to check and incorporate. The trial design, the educational interpretation and the conclusions remain with the research team.