Permutation ANOVA (distribution-free)

Coming soon

Nonparametric ANOVA

This test is implemented and is currently going through StatMinds’ production verification: every statistic is independently checked against a trusted reference (scipy / R), locked with regression tests, and the screen is exercised across assumption-met/violated and significant/non-significant scenarios before it opens up.

See what’s live now

Distribution-free omnibus test for 3+ independent groups.

Computes the classical one-way F statistic and references it against a Monte-Carlo null built by reshuffling group labels (default B = 10,000 permutations). Assumes exchangeability of residuals under H₀ — valid without normality, but NOT robust to severe heteroscedasticity. Reports the observed F, permutation p, and the empirical null distribution. Apt when sample size is too small for the classical F's CLT to bail out the normality assumption AND group variances are roughly comparable.

Worked example

Do 3+ groups differ without the ANOVA normality assumption?

A permutation one-way ANOVA rebuilds the F null distribution by shuffling group labels many times.

Result

Groups differed, F = 5.2, permutation p = .006 (10,000 shuffles).

How you'd report it (APA)

A permutation ANOVA showed a significant group difference, F = 5.2, p = .006 (10,000 permutations).

When to use it

  • 3+ independent groups with non-normal continuous DV
    A bench-science study compares enzyme activity across four catalysts with n ≈ 15 each.
  • Small-sample k-group test with no distributional assumptions
    Pilot bioassay: 4 catalysts × n = 6 reactions each (total N = 24).

When NOT to — use instead

Hypotheses

H₀: the k group distributions are identical (including means, variances, shapes).
Hₐ: at least one group distribution differs.

Parameter tested: set of population distributions

Assumptions (and what to do if they fail)

Under H₀ the group distributions are identical (including variance). Permutation p-values remain valid under modest heteroscedasticity; extreme variance differences bias them.medium

Check: Report the SD-ratio across groups.

If violated: Severe heteroscedasticity inflates the permutation Type I error because label shuffling smears variance structure.

Adequate total sample size (N ≥ 15 recommended) for a stable Monte-Carlo null.low

Check: Report N.

If violated: Small N makes the null discrete and coarse.

Ready to run a Permutation ANOVA (distribution-free) on your own data?

Guided setup, automatic assumption checks, effect sizes, figures and an APA write-up.

Run this test →