Effect size in education research and practice

A guide for classroom teachers and teacher aides

Effect size – the impact that a particular strategy, technique, circumstance or issue has on student learning.

Effect size (in terms of teaching strategies) refers to the impact that a particular strategy, technique, circumstance or issue has on student learning (or other factor under measurement). When something has an impact on learning, the effect can be measured and expressed as a number. Simplistically speaking, this allows 2 separate effects to be compared in a way that makes sense to the average person with no statistical training. The measurement number is usually between 0 and 1 but it can also be as high as 2 or even be negative. A common rule of thumb is that an effect of 0.2 or higher is small (but still an effect nonetheless), 0.5 or higher is a medium effect and 0.8 and higher a very strong effect.ⁱ

When something has an impact on learning, the effect can be measured and expressed as a number.

It should be noted that due to the way effect size is calculated, comparing 2 effects in terms of the actual measurable improvements in the classroom is not as simple as comparing the 2 numbers. For example, an effect of 0.4 is not double the effect of 0.2 (in percentage terms, the actual difference is about 5% when calculated by the common language effects method, or CLE). There are also issues with comparing effect sizes that are calculated based on very different studies and contexts (for example, comparing the effect of a 20-year-old study of 5 participants from New Zealand with a recent study of 100,000 schools in the US is not a fair or valid comparison).

Another point to note is that a low effect size does not mean that a strategy is worthless or any less effective than a strategy with a higher effect size. For example, individualised instruction has a relatively low effect size of 0.29 in the table below, but that does not mean schools should simply abolish individualised instruction. What effect size does tell us is that strategies and influences with a known higher effect size should be regularly considered for implementation (when appropriate and relevant). It is very likely that these identified strategies will have a positive impact on learning. Effect size is a powerful tool for understanding what strategies are likely to have the biggest impact on the learning process and student achievement.

John Hattie is by far the most prominent and well-known academic in terms of effect-size work. Hattie collated hundreds of studies and has formulated a list of over 250 effect sizes. The table below shows a snapshot of some of the influences collected and synthesised by Hattie and how they relate to the strategies and skills found in this book. The selected influences are in descending order in terms of their effect size. While there are criticisms of Hattie’s work, this list provides a simple reference to remind us of what high-quality teaching looks like.ⁱⁱ

Foot notes:

1. Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Earlbaum Associates.
2. See either: Hattie, J. (2009). Visible Learning. New York: Routledge, Taylor & Francis Group, or the website: visible-learning.org for the full list. Also note other well-known sources including the What Works Clearinghouse found at https://ies.ed.gov/ncee/wwc/.