Online Tools

This tool provides estimates of design parameters (ICCs and R² values) useful when designing cluster-randomized experiments in schools. Data are available for studies to be conducted in Kindergarten through grade 12 with academic achievement as an outcome.

Educational experiments often involve the assignment of aggregate units such as schools or school districts (statistical clusters) to treatments. Experiments that do so are called cluster-randomized experiments. The sensitivity (statistical power, precision of treatment effect estimates, and minimum detectable effect size) of cluster randomized experiments depends on statistical significance level, sample size, and effect size, but also on the variance decomposition among levels of aggregation (as indicated by intraclass correlation or ICC values at each level of aggregation) and the effectiveness of any covariates used to explain variation at different levels of aggregation (as indicated by intraclass correlation or ICC values at each level of aggregation) and the effectiveness of any covariates used to explain variation at different levels of aggregation (as indicated by R2 values at each level of aggregation). We call the ICC and R2 values design parameters because the values of these parameters are necessary to design a cluster randomized experiment that has adequate sensitivity.

USE THE ONLINE INTRACLASS CORRELATION DATABASE TOOL

This tool helps researchers designing evaluations in schools to develop a recruitment plan that will allow results of the study to generalize to a well-defined policy relevant population. Researchers can also use the tool to report how well results from their study in schools might generalize to national and state populations of schools.

Educational experiments (and other evaluations) are typically conducted in samples of convenience, making it difficult to know where results could generalize and where they could not. This tool builds implements recent methods for stratified sampling, providing researchers with lists of schools to recruit for their study. Researchers interested in using the tool should sign up for an access code, which will be emailed to them typically in under an hour.

Get started using the Generalizer tool

When meta-analyses include multiple effect sizes from the same study, methods need to take the correlation between these effect sizes into account in the analyses. Robust Variance Estimation (RVE) provides an approach that is valid even when information on the correlation is unknown.

The following software are available to conduct these analyses across different platforms:

• robuMeta: available as an R package or a Stata module
• clubSandwich: R package
• reg sandwich: Stata module