Bonferroni Correction Calculator

Every hypothesis test has a small chance of a false positive. When one question is tested several times, those chances add up: with m independent tests, each run at α = 0.05, the probability of at least one false positive is 1 − (1 − α)^m. That is about 22.6% for 5 tests and 64.2% for 20 tests. This probability is the family-wise error rate (FWER).

The Bonferroni correction keeps the FWER at or below α. It rests on the union bound (Boole's inequality): the chance that at least one of m true null hypotheses is rejected cannot exceed the sum of the m individual chances. Test each hypothesis at level α / m, or equivalently multiply every p-value by m (capped at 1) and compare it with α. The two give the same decisions, and neither needs the tests to be independent.

1. Paste the raw p-values of your tests, separated by commas, spaces or new lines. Use the p-value your software reports (0.00031 or 3.1e-4), not a bound such as < 0.001.
2. Set the family-wise α, usually 0.05. For the two Benjamini methods this number is the false discovery rate you accept (often written q).
3. Press Adjust p-values. Read the corrected thresholds, the tests each method rejects and the table of adjusted p-values. A test is rejected when its adjusted p-value is at most α.
4. Only need the corrected α? Leave the p-values empty and type the number of tests: you get α / m (Bonferroni), the Šidák level and the uncorrected FWER.

With m tests and sorted p-values p(1) ≤ p(2) ≤ … ≤ p(m), every adjusted p-value is capped at 1:

• Holm (step-down) compares the smallest p-value with α / m, the next with α / (m − 1), and so on, and stops at the first p-value that fails.
• Hochberg (step-up) starts from the largest p-value, finds the first one that is at most α / (m − i + 1) for its rank i counted from the smallest, and rejects it and every smaller p-value.
• Benjamini-Hochberg finds the largest rank i with p(i) ≤ (i / m) · α and rejects the i smallest p-values.

• Bonferroni threshold (α/m) is the significance level each single test must beat. The Šidák threshold is its slightly larger, exact counterpart for independent tests.
• FWER if uncorrected is the chance of at least one false positive among m independent tests if you ignored the problem and used α for each.
• Tests rejected at α lists, for each method, how many tests are still significant after the correction and which ones (numbered in the order you entered the p-values).
• Adjusted p-values can be compared directly with α, whatever the method. An adjusted p-value of 1 means the correction pushed it to the maximum, not that the effect is proven absent.

A study measures five outcomes and gets the p-values 0.003, 0.012, 0.021, 0.048 and 0.31. Four of them are below 0.05, but with five tests the chance of at least one false positive is 22.62%, so the correction has to be applied. Press Load example to enter them.

At α = 0.05 the methods disagree in the way the theory predicts. Bonferroni (threshold 0.05 / 5 = 0.01) keeps only test 1. Holm keeps tests 1 and 2. Benjamini-Hochberg, which controls the false discovery rate instead of the family-wise error rate, keeps tests 1, 2 and 3.

Use an FWER method (Bonferroni, Holm, Šidák, Hochberg) when a single false positive is costly, as in a confirmatory study with a handful of pre-specified comparisons. Use an FDR method when you screen many hypotheses, such as thousands of genes, and can live with a small share of false discoveries in exchange for power. R's documentation of p.adjust notes that the unmodified Bonferroni correction is dominated by Holm's method, which is valid under the same arbitrary dependence, so Holm is the better default when you only need FWER control. Bonferroni remains the simplest rule to state, apply by hand and use for simultaneous confidence intervals (build each one at level 1 − α / m).

• Fix the family before you look. m counts every test that addresses the same question, including those you did not report. Choosing which tests to count after seeing which are significant defeats the correction.
• Bonferroni is conservative. With many tests, or tests that are strongly correlated, it rejects too little: at m = 100 each test must reach 0.0005. Holm or an FDR method keeps more power.
• Mind the dependence. Šidák, Hochberg and Benjamini-Hochberg are derived for independent or positively dependent tests. When the dependence is unknown or possibly negative, use Holm or Benjamini-Yekutieli.
• It does not repair the tests. The correction assumes each p-value is valid. It cannot fix a wrong test, biased data or selective reporting.
• One primary test needs none. A single pre-specified hypothesis is tested at α. Secondary endpoints can be handled with a fixed testing order instead.
• After an ANOVA with every pair of means compared, use Tukey HSD, which exploits the structure of the comparisons and is usually more powerful than Bonferroni. For factorial designs see the two-way ANOVA calculator.

For the reasoning behind these rules read multiple comparisons explained. A p-value of a single test comes from the p-value calculator, and the chance of missing a real effect with a stricter α is the topic of statistical power explained.

R also offers Hommel's method and statsmodels offers Holm-Šidák and two-stage FDR procedures; this calculator covers the six methods above and matches statsmodels' multipletests for each of them.