Examining heterogeneity of education intervention effects using quantile mixed models: a re-analysis of a cluster-randomized controlled trial of a fluency-based mathematics intervention

Thompson, P ORCID: 0000-0001-9940-6913, Owen, K and Hastings, RP (2023) Examining heterogeneity of education intervention effects using quantile mixed models: a re-analysis of a cluster-randomized controlled trial of a fluency-based mathematics intervention. International Journal of Research & Method in Education, 47 (1). pp. 49-64. ISSN 1743-727X

Preview

Text Examining heterogeneity of education intervention effects using quantile mixed models a reanalysis of a cluster randomized controlled trial of a flu.pdf - Published Version Available under License Creative Commons Attribution. Download (2MB) | Preview

Abstract

Traditionally, cluster randomized controlled trials are analyzed with the average intervention effect of interest. However, in populations that contain higher degrees of heterogeneity or variation may differ across different values of a covariate, which may not be optimal. Within education and social science contexts, exploring the variation in magnitude of treatment effect at different points in the population can indicate where the intervention is most effective rather than assuming an average effect.

Data from [Owen, K.L., et al., 2021. Implementation support improves outcomes of a fluency-based mathematics strategy: A cluster-randomized controlled trial. Journal of research on educational effectiveness, 14 (3), 523–542.] were reanalyzed using three modeling approaches: conditional mean-modeling reporting the average treatment effect using linear mixed models, and two quantile regression-based methods. Quantile regressions report the quantile treatment effects at different percentiles: 10th, 25th, 50th, 75th and 90th. For the Quantile approaches, a significant intervention effect in the median to upper quantiles was found and linear quantile mixed model showed improved fit over the other approaches.

An improved picture of intervention effects may be apparent using quantile regression methods when analyzing cluster randomized trials that have heterogeneous error variance. In particular, the linear quantile mixed model shows improved model fit allowing a multilevel framework to include random effects. There is considerable scope to extend this framework to incorporate more complex RCT designs.

Item Type:	Article
Uncontrolled Keywords:	13 Education; Education; 39 Education
Subjects:	L Education > L Education (General)
Divisions:	Education
Publisher:	Taylor & Francis
Date of acceptance:	1 April 2023
Date of first compliant Open Access:	21 May 2026
Date Deposited:	21 May 2026 10:58
Last Modified:	21 May 2026 10:58
DOI or ID number:	10.1080/1743727x.2023.2215699
URI:	https://researchonline.ljmu.ac.uk/id/eprint/28601

View Item