TOST Equivalence Test (two one-sided tests)

Coming soon

T-Tests

This test is implemented and is currently going through StatMinds’ production verification: every statistic is independently checked against a trusted reference (scipy / R), locked with regression tests, and the screen is exercised across assumption-met/violated and significant/non-significant scenarios before it opens up.

See what’s live now

Tests for EQUIVALENCE between two group means within a pre-specified equivalence bound ±Δ.

The null is non-equivalence (|μ₁ − μ₂| ≥ Δ); rejecting the null means the groups are statistically equivalent at the chosen tolerance, NOT that they are equal in any absolute sense. Two one-sided t-tests are conducted at α each, and equivalence is declared when BOTH reject (p_TOST = max(p_lower, p_upper) < α). Reports the mean difference + 90% CI (which by duality must lie entirely within ±Δ for equivalence at α = .05), the two one-sided p-values, and the equivalence verdict.

Worked example

Are two formulations equivalent — not just 'not significantly different'?

TOST (two one-sided tests) checks whether the mean difference falls within a pre-set equivalence margin (±2 units) — the proper way to argue equivalence.

Result

The 90% CI for the difference [−1.01, 1.22] sat entirely within ±2, so equivalence was established (TOST p = .003).

How you'd report it (APA)

A TOST equivalence test showed the two formulations were equivalent within ±2 units (90% CI [−1.01, 1.22], TOST p = .003).

Try it yourself: Load this ready-made sample and follow the run above.

When to use it

  • Equivalence between two independent group means
    Bioequivalence study: peak plasma concentration of a generic drug (n = 60) against the reference brand (n = 60).
  • Non-inferiority margin (one-sided equivalence)
    Non-inferiority RCT: new shorter chemo regimen vs standard.

When NOT to — use instead

Hypotheses

H₀ (of NON-equivalence): the absolute mean difference is at least as large as the equivalence bound, |μ₁ − μ₂| ≥ Δ. Decomposed into two one-sided nulls: H₀_lower: μ₁ − μ₂ ≤ −Δ and H₀_upper: μ₁ − μ₂ ≥ +Δ.
Hₐ (of equivalence): the absolute mean difference is smaller than Δ, i.e. −Δ < μ₁ − μ₂ < +Δ. Both one-sided nulls must be rejected to conclude equivalence.

Parameter tested: population mean difference μ₁ − μ₂ (equivalence bound Δ specified a priori)

Assumptions (and what to do if they fail)

No significant outliers on the DV within each groupmedium

Check: Per-group boxplot inspection.

If violated: Outliers distort the mean difference, CI, and TOST p-value.

The DV is approximately normally distributed within each groupmedium

Check: Shapiro-Wilk p > .05 or n ≥ 30 per group (CLT).

If violated: Severely non-normal data threaten TOST's Type I error rate.

Equal variances across groups (for Student TOST); the pipeline uses Welch's t by default so this is informationallow

Check: Levene's p > .05 → variances homogeneous.

If violated: Welch-style Satterthwaite df handles this case; no correction needed in the pipeline.

The equivalence bound Δ represents a difference that would be practically / clinically negligiblehigh

Check: Cite the source of Δ in the pre-registration or methods section.

If violated: A poorly chosen bound makes equivalence trivial to establish (large bound) or impossible (tiny bound) regardless of the true effect.

Ready to run a TOST Equivalence Test (two one-sided tests) on your own data?

Guided setup, automatic assumption checks, effect sizes, figures and an APA write-up.

Run this test →