# Repeated Measures ANOVA using Python and R (with examples)

## Repeated measures ANOVA (Within-subjects ANOVA)

- Repeated measures ANOVA is used when the responses from the same subjects (experimental units) are measured repeatedly (more than two times) over a period of time or under different experimental conditions. For example, the number of leaves on plants measured every week for a month under certain treatment conditions.
- In repeated measure ANOVA, each subject serves as its own control, and each subject experiences all the levels of the treatments or time.
- The independent ANOVA is not appropriate for repeated measurements on the same subjects as the data violates the assumption of independence i.e. it does not interrogate within-subjects observation dependencies.
- Repeated measure ANOVA is an extension to the Paired
*t*-test (dependent*t*-test) and provides similar results as of Paired*t*-test when there are two time points or treatments. - Repeated measure ANOVA is mostly used in longitudinal study where subject responses are analyzed over a period of time

### Assumptions of repeated measures ANOVA

- The responses from subjects (dependent variable) should be continuous
- The responses between subjects should be independent
- The subjects are randomly selected from the population
- The subject’s responses (dependent variable) should be normally distributed for each level of time or treatment (within-subjects factor)
__Assumption of sphericity__: the variances of differences in responses between any two levels of the independent variable (within-subjects factor) should be equal (*Mauchly’s test of sphericity*). This assumption also known as homogeneity-of-variance-of-differences assumption.- There should be no outliers

### Repeated measures ANOVA Hypotheses

*Null hypothesis*: Treatment or time groups means are equal (no variation in means of groups)

H_{0}: μ_{1}=μ_{2}=…=μ_{p}*Alternative hypothesis*: At least, one group mean is different from other groups

H_{1}: All μ are not equal

## One-way repeated measures ANOVA

In one-way repeated measures ANOVA, there is only one factor (treatments or time) to study

The variation is divided into three components including variation within-subjects
(SS_{subjects}), variation among the treatments or time points (SS_{B}), and residual variance
(SS_{E}).

Source of variation |
degree of freedom (Df) |
Sum of squares (SS) |
Mean square (MS) |
F value |
Significance |
---|---|---|---|---|---|

Subjects | Df_{subjects} = n-1 |
SS_{subjects} |
MS_{subjects} = SS_{subjects} ∕ Df_{subjects} |
||

Treatments or time | Df_{B} = k-1 |
SS_{B} |
MS_{B} = SS_{B} ∕ Df_{B} |
MS_{B} ∕ MS_{E} |
p value |

Residuals or error | Df_{E} = (n-1)(k-1) |
SS_{E} |
MS_{E} = SS_{E} ∕ Df_{E} |
||

Total | Df_{T} = nk-1 |
SS_{T} |

Where, n = total subjects, k = level for the treatments or time

Example data ^{2} for one-way repeated measures ANOVA analysis,

Id | W1 | W2 | W3 | W4 | W5 |
---|---|---|---|---|---|

P1 | 4 | 5 | 6 | 8 | 10 |

P2 | 3 | 4 | 6 | 6 | 9 |

P3 | 6 | 7 | 9 | 10 | 12 |

P4 | 5 | 7 | 8 | 10 | 12 |

P5 | 5 | 6 | 7 | 8 | 10 |

Here, there are five plants (P1 to P5) in which the number of leaves per plants were measured over multiple time points (1, 2, 3, 4 and 5 weeks) on plants at the low nutrient level. The number of leaves per plant and time points are dependent and independent variables, respectively.

Let’s perform one-way repeated measures ANOVA in Python,

#### Load and visualize the dataset

Run the code in Python colab, R colab

#### Summary statistics

```
from dfply import *
df_melt >> group_by(X.time_points) >> summarize(n=X.leaves.count(), mean=X.leaves.mean(), std=X.leaves.std())
time_points n mean std
0 W1 5 4.6 1.140175
1 W2 5 5.8 1.303840
2 W3 5 7.2 1.303840
3 W4 5 8.4 1.673320
4 W5 5 10.6 1.341641
```

#### Perform one-way repeated measures ANOVA

Run the code in Python colab

From the repeated measures ANOVA results, we reject the null hypothesis in favor of the alternate hypothesis [F(4, 16) =
146.27, *p* <0.05, η_{p}^{2}=0.97]. We conclude that the mean of the number of leaves
on plants differs significantly at different time points at the low nutrient level.

The measure of effect size (Partial Eta Squared; η_{p}^{2}) is 0.97 and
suggests that there is a large effect of time points on a number of leaves on plants. Partial Eta Squared indicates that
97% of the overall variance (main effect and error) associated with the time points.

**Note**: In one-way repeated measures ANOVA, the
Partial Eta Squared is the same as to Eta Squared. The reference values for effect sizes for Partial Eta Squared: small
effect = 0.01; medium effect = 0.06; and large effect = 0.14

Now, we know that differences in a number of leaves on plants are statistically significant, but ANOVA output does not tell which time points are significantly different from each other. To know the pairs with significantly different time points, we will perform multiple pairwise comparison (post hoc comparison) analysis.

#### Post-hoc tests

Perform multiple pairwise comparisons (*t* test) and corrections
(Benjamini/Hochberg FDR correction),

Run the code in colab

The post-hoc tests with Benjamini/Hochberg FDR correction
results indicate that all time points pairs are statistically significant (*p-corr* < 0.05). This concludes that
the number of leaves on plants are significantly different between each time points.

#### Check repeated measures ANOVA assumptions

##### Assumption of sphericity

The assumption of sphericity can be tested using *Mauchly’s test of sphericity*. The violation of the assumption of
sphericity can lead to an increase in type II error (loss of statistical power)
and the *F* value is not valid.

Run the code in colab

As the *p* value (0.8883) is non-significant (*p* > 0.05), the data met the assumption of sphericity, and
variances of differences of independent variables are equal.

##### Assumption of normality

*Shapiro-Wilk test* can be used for checking the assumption for normality of each level of the within-subjects factor

Run the code in colab

As the *p* value is non-significant (*p*>0.05) for all levels of the within-subjects factor, we conclude
that data for each time point is normally distributed.

As discussed in standard ANOVA, the visual approaches (QQ-plots and histograms) perform better than statistical tests for checking the assumption of normality.

## Two-way repeated measures ANOVA (Within-within-subjects ANOVA)

In two-way repeated measures ANOVA, there are two within-subjects factors (treatments or time) to study

Let’s perform two-way repeated measures ANOVA (Within-within-subjects ANOVA) in Python,

#### Load and visualize the dataset

Run the code in colab

#### Summary statistics

```
from dfply import *
df >> group_by(X.year, X.time) >> summarize(n=X.num_leaves.count(), mean=X.num_leaves.mean(), std=X.num_leaves.std())
time year n mean std
0 Week1 Year1 5 4.6 1.140175
1 Week2 Year1 5 5.8 1.303840
2 Week3 Year1 5 7.2 1.303840
3 Week4 Year1 5 8.4 1.673320
4 Week5 Year1 5 10.6 1.341641
5 Week1 Year2 5 3.6 0.894427
6 Week2 Year2 5 4.2 0.447214
7 Week3 Year2 5 5.8 0.447214
8 Week4 Year2 5 7.4 0.547723
9 Week5 Year2 5 9.6 0.547723
```

#### Perform two-way repeated measures ANOVA

```
import pingouin as pg
res = pg.rm_anova(dv='num_leaves', within=['time', 'year'], subject='plants',
data=df, detailed=True)
res
#output
Source SS ddof1 ddof2 MS F p-unc p-GG-corr np2 eps
0 time 226.88 4 16 56.72 158.657343 1.177428e-12 0.000003 0.975408 0.421229
1 year 18.00 1 4 18.00 3.600000 1.306351e-01 0.130635 0.473684 1.000000
2 time * year 0.80 4 16 0.20 1.454545 2.620704e-01 0.291675 0.266667 0.441606
```

Run the code in colab

From the repeated measures ANOVA results, we have two within-subjects factors (time and year; main effects) and interaction effects (time*year) to analyze.

We reject the null hypothesis in favor of the alternate hypothesis for time (F_{4, 16}=158.65, *p*<0.001,
η_{p}^{2}=0.97).

We fail to reject the null hypothesis for year (F_{1, 4}=3.6, *p*=0.13,
η_{p}^{2}=0.47), and for time and year interaction effect (F_{4, 16}=1.45,
*p*=0.26, η_{p}^{2}=0.26).

__Note on F value__:

*F*value is inversely related to

*p*value and higher

*F*value (greater than

*F*critical value) indicates a significant

*p*value.

We conclude that the main effects (time) had significant effects on the number of leaves of plants at the low nutrient level.

## References

- Singh V, Rana RK, Singhal R. Analysis of repeated measurement data in the clinical trials. Journal of Ayurveda and integrative medicine. 2013 Apr;4(2):77.
- Scheiner SM, Gurevitch J, editors. Design and analysis of ecological experiments. Oxford University Press; 2001 Apr 26.
- Vallat, R. (2018). Pingouin: statistics in Python. Journal of Open Source Software, 3(31), 1026, https://doi.org/10.21105/joss.01026
- Seabold, Skipper, and Josef Perktold. “statsmodels: Econometric and statistical modeling with python.” Proceedings of the 9th Python in Science Conference. 2010.
- Schober P, Vetter TR. Repeated measures designs and analysis of longitudinal data: if at first you do not succeed—try, try again. Anesthesia and analgesia. 2018 Aug;127(2):569.
- Maher JM, Markey JC, Ebert-May D. The other half of the story: effect size analysis in quantitative research. CBE—Life Sciences Education. 2013 Sep;12(3):345-51.

If you have any questions, comments or recommendations, please email me at
**reneshbe@gmail.com**

This work is licensed under a Creative Commons Attribution 4.0 International License