Repeated Measures ANOVA using Python and R (with examples)

Renesh Bedre    6 minute read

Repeated measure ANOVA designs

Repeated measures ANOVA (Within-subjects ANOVA)

  • The repeated measures ANOVA is useful when the responses from the same subjects (experimental units) are measured repeatedly (more than two times) over a period of time or under different experimental conditions. For example, the number of leaves on plants measured every week for a month under certain treatment conditions.
  • In repeated measure ANOVA, each subject serves as its own control, and each subject experiences all the levels of the treatments or time.
  • The standard ANOVA is not appropriate for repeated measurements on the same subjects as the data violates the assumption of independence i.e. it does not interrogate within-subjects observation dependencies.
  • Repeated measure ANOVA is an extension to the Paired t-test (dependent t-test) and provides similar results as of Paired t-test when there are two time points or treatments.

Assumptions of repeated measures ANOVA

  • The responses from subjects (dependent variable) should be continuous
  • The responses between subjects should be independent
  • The subjects are randomly selected from the population
  • The subject’s responses (dependent variable) should be normally distributed for each level of time or treatment (within-subjects factor)
  • Assumption of sphericity: the variances of differences in responses between any two levels of the independent variable (within-subjects factor) should be equal (Mauchly’s test of sphericity). This assumption also known as homogeneity-of-variance-of-differences assumption.
  • There should be no outlier

Repeated measures ANOVA Hypotheses

  • Null hypothesis: Treatment or time groups means are equal (no variation in means of groups)
    H0: μ12=…=μp
  • Alternative hypothesis: At least, one group mean is different from other groups
    H1: All μ are not equal

One-way repeated measures ANOVA

In one-way repeated measures ANOVA, there is only one factor (treatments or time) to study

The variation is divided into three components including variation within-subjects (SSsubjects), variation among the treatments or time points (SSB), and residual variance (SSE).

Source of
variation
degree of freedom
(Df)
Sum of squares
(SS)
Mean square
(MS)
F value Significance
Subjects Dfsubjects = n-1 SSsubjects MSsubjects = SSsubjects ∕ Dfsubjects    
Treatments or time DfB = k-1 SSB MSB = SSB ∕ DfB MSB ∕ MSE p value
Residuals or error DfE = (n-1)(k-1) SSE MSE = SSE ∕ DfE    
Total DfT = nk-1 SST      

Where, n = total subjects, k = level for the treatments or time

Example data 2 for one-way repeated measures ANOVA analysis,

Id W1 W2 W3 W4 W5
P1 4 5 6 8 10
P2 3 4 6 6 9
P3 6 7 9 10 12
P4 5 7 8 10 12
P5 5 6 7 8 10

Here, there are five plants (P1 to P5) in which the number of leaves per plants were measured over multiple time points (1, 2, 3, 4 and 5 weeks) on plants at the low nutrient level. The number of leaves per plant and time points are dependent and independent variables, respectively.

Let’s perform one-way repeated measures ANOVA in Python,

Load and visualize the dataset

Load and visualize the dataset

Boxplot for one-way repeated measures ANOVA

Run the code in Python colab, R colab

Perform one-way repeated measures ANOVA

Perform one-way repeated measures ANOVA

Run the code in Python colab

From the repeated measures ANOVA results, we reject the null hypothesis in favor of the alternate hypothesis [F(4, 16) = 146.27, p <0.05, ηp2=0.97]. We conclude that the mean of the number of leaves on plants differs significantly at different time points at the low nutrient level.

The measure of effect size (Partial Eta Squared; ηp2) is 0.97 and suggests that there is a large effect of time points on a number of leaves on plants. Partial Eta Squared indicates that 97% of the overall variance (main effect and error) associated with the time points.

Note: In one-way repeated measures ANOVA, the Partial Eta Squared is the same as to Eta Squared

Now, we know that differences in a number of leaves on plants are statistically significant, but ANOVA output does not tell which time points are significantly different from each other. To know the pairs with significantly different time points, we will perform multiple pairwise comparison (post hoc comparison) analysis.

Post-hoc tests

Perform multiple pairwise comparisons (t test) and corrections (Benjamini/Hochberg FDR correction),

Post-hoc tests

Run the code in colab

The post-hoc tests with Benjamini/Hochberg FDR correction results indicate that all time points pairs are statistically significant (p-corr < 0.05). This concludes that the number of leaves on plants are significantly different between each time points.

Check repeated measures ANOVA assumptions

Assumption of sphericity

The assumption of sphericity can be tested using Mauchly’s test of sphericity. The violation of the assumption of sphericity can lead to an increase in type II error (loss of statistical power) and the F value is not valid.

Assumption of sphericity

Run the code in colab

As the p value (0.8883) is non-significant (p > 0.05), the data met the assumption of sphericity, and variances of differences of independent variables are equal.

Assumption of normality

Shapiro-Wilk test can be used for checking the assumption for normality of each level of the within-subjects factor

Assumption of normality

Run the code in colab

As the p value is non-significant (p>0.05) for all levels of the within-subjects factor, we conclude that data for each time point is normally distributed.

As discussed in standard ANOVA, the visual approaches (QQ-plots and histograms) perform better than statistical tests for checking the assumption of normality.

Two-way repeated measures ANOVA (Within-within-subjects ANOVA)

In two-way repeated measures ANOVA, there are two within-subjects factors (treatments or time) to study

Let’s perform two-way repeated measures ANOVA (Within-within-subjects ANOVA) in Python,

Load and visualize the dataset

Load and visualize the dataset two way

Boxplot for two-way repeated 
measures ANOVA

Run the code in colab

Perform two-way repeated measures ANOVA

import pingouin as pg
res = pg.rm_anova(dv='num_leaves', within=['time', 'year'], subject='plants', 
                  data=df, detailed=True)
res
#output
        Source      SS  ddof1  ddof2     MS           F         p-unc  p-GG-corr       np2       eps
0         time  226.88      4     16  56.72  158.657343  1.177428e-12   0.000003  0.975408  0.421229
1         year   18.00      1      4  18.00    3.600000  1.306351e-01   0.130635  0.473684  1.000000
2  time * year    0.80      4     16   0.20    1.454545  2.620704e-01   0.291675  0.266667  0.441606

Run the code in colab

From the repeated measures ANOVA results, we have two within-subjects factors (time and year; main effects) and interaction effects (time*year) to analyze.

We reject the null hypothesis in favor of the alternate hypothesis for time (F4, 16=158.65, p<0.001, ηp2=0.97).

We fail to reject the null hypothesis for year (F1, 4=3.6, p=0.13, ηp2=0.47), and for time and year interaction effect (F4, 16=1.45, p=0.26, ηp2=0.26).

Note on F value: F value is inversely related to p value and higher F value (greater than F critical value) indicates a significant p value.

We conclude that the main effects (time) had significant effects on the number of leaves of plants at the low nutrient level.

References

  1. Singh V, Rana RK, Singhal R. Analysis of repeated measurement data in the clinical trials. Journal of Ayurveda and integrative medicine. 2013 Apr;4(2):77.
  2. Scheiner SM, Gurevitch J, editors. Design and analysis of ecological experiments. Oxford University Press; 2001 Apr 26.
  3. Vallat, R. (2018). Pingouin: statistics in Python. Journal of Open Source Software, 3(31), 1026, https://doi.org/10.21105/joss.01026
  4. Seabold, Skipper, and Josef Perktold. “statsmodels: Econometric and statistical modeling with python.” Proceedings of the 9th Python in Science Conference. 2010.
  5. Schober P, Vetter TR. Repeated measures designs and analysis of longitudinal data: if at first you do not succeed—try, try again. Anesthesia and analgesia. 2018 Aug;127(2):569.

If you have any questions, comments or recommendations, please email me at reneshbe@gmail.com

This work is licensed under a Creative Commons Attribution 4.0 International License