About this calculator
For example, if both counts used the same questions, wording, engine and place, 4 of 20 then 8 of 20 AI answers is +20 points. The change from 4 of 20 to 8 of 20 has a 95% range of −8 to +44 points, so it cannot be told apart from run-to-run variation.
Use this if you counted AI answers before and after a change to your site and want to know whether the rate moved.
How it decides
A spreadsheet can recompute any result here if it rounds to whole points before testing the range against 0.
| Step | Rule | Where it comes from |
|---|---|---|
| Inputs | For each period: answers collected, a whole number from 1 to 1,000, and answers that named you, a whole number from 0 up to answers collected. Anything else gets a message, not a result. | This calculator. |
| Range for each period | p = named ÷ n and z = 1.96. Low = (p + z²/(2n) − z × √(p × (1 − p)/n + z²/(4n²))) ÷ (1 + z²/n). High is the same with + before z. If named is 0, low is 0%. If named equals n, high is 100%. Shown in whole percent; the next step uses the unrounded values. | Wilson score interval. Wikipedia prints the formula and says it can be used with small samples. |
| Range for the change | p1 before and p2 after, each with its unrounded Wilson bounds: l1 and u1 for p1, l2 and u2 for p2. Change d = p2 − p1. Low = d − √((p2 − l2)² + (u1 − p1)²). High = d + √((u2 − p2)² + (p1 − l1)²). | Newcombe (1998) reports that combining the two Wilson intervals performs well. The abstract does not print the formula. Our code is checked against the statsmodels library's newcomb method (z = 1.96): the same whole-point results on 3,000 random pairs of counts. |
| Did it change? | Round d, low and high to whole points, halves away from zero. If the rounded low is above 0, the change is a clear rise. If the rounded high is below 0, it is a clear fall. Otherwise the change cannot be told from run-to-run variation. | This calculator: the verdict follows the whole points shown. |
| Count entered | Result |
|---|---|
| 4 of 20, then 8 of 20 | +20 points, range −8 to +44: no clear change |
| 4 of 20, then 10 of 20 | +30 points, range +1 to +53: a clear rise |
| 4 of 50, then 20 of 50 | +32 points, range +16 to +47: a clear rise |
| 1 of 6, then 9 of 14 | +48 points. The range before rounding is +0.44 to +71.32, shown as 0 to +71: no clear change, because the rounded low is 0. |
What it cannot tell you
- Same conditions. Both periods must use the same questions, wording, engine and place. Logged-in repeats can share history, so the true range is wider.
- Many comparisons. The 95% covers one comparison. Reporting only the question that moved makes false alarms likely.
- Cause. A clear rise or fall does not say why the rate moved, and a name is not a recommendation. For visitors, see how to see whether AI sends you visitors and what AI assistants cite for local businesses.
Sources
| Source | What it says | Date | Who publishes it |
|---|---|---|---|
| Wikipedia, Binomial proportion confidence interval | The Wilson score interval formula, and that it can be used with small samples. | Page last edited September 11, 2026. Read October 6, 2026. | Wikipedia, a nonprofit encyclopedia. Sells nothing. |
| Newcombe, Interval estimation for the difference between independent proportions | Abstract: a method combining Wilson score intervals for the two proportions “performs well, and is readily implemented irrespective of sample size”. The abstract does not print the formula. | Statistics in Medicine 17(8): 873 to 890, April 30, 1998. Read October 6, 2026. | A medical statistics journal. Sells no AI tracking. |
| statsmodels, confint_proportions_2indep | Lists newcomb as a method for the interval of the difference between two proportions, and cites Newcombe (1998) among its references. It does not print the formula. | No date shown. Read October 6, 2026. | The statsmodels developers, an open-source Python library. Sells nothing. |
Questions people ask
How do I check whether a website change moved my AI mentions?
Take one count before the change and one after it, under the conditions listed under What it cannot tell you, then enter both. This page cannot say how long an engine takes to pick up a site change, so it cannot say when the second count should start.
Can I judge a change by whether the two bars overlap?
No. The check compares the two counts directly, so overlapping bars do not decide it. With the same conditions both times, 2 of 20 then 9 of 20 has Before and After bars of 3% to 30% and 26% to 66%, which overlap, yet the change is +35 points, range +7 to +57: a clear rise.
How many answers do I need in each period?
A real rise in the share of answers naming you, from 20% to 40%, needs about 85 answers in each period to show as clear in about 80 of 100 repeats of both counts, and 20% to 30% about 325. These are our own calculation, not checkable in a spreadsheet, and assume independent answers to the same question. The AI answer sample size calculator takes your own rates.
The other two checks
For one count, use the AI mention rate confidence calculator, which gives the 95% range for the true rate. To plan how many answers to collect in each period, use the AI answer sample size calculator.
Last updated October 7, 2026