Statistical Significance Calculator Widget

Version: 1.0 Platforms: OSX (min 10.4.3) License: Apache

This is an OSX dashboard widget that enables quick calculation of data needed to assess statistical significance:

1. The margin of error for a specific result for a specific sample size. For example if you surveyed 100 people out of a total population of 100,000 and 85% said they did like chocolate then the margin of error is 7% at a confidence level of 95%, which means that if you repeated the survey enough times within the same population then 95% of the time you would get an answer between 78% and 92%.
2. The sample size needed to obtain a specific margin of error for a specific result. For example, if you have a population of 500 people and you want to know how many agree with a question to within 5% either way and you suspect the around 40% will agree then the sample size needed is 212, with the same restriction as above on this being only 95% likely. If you do not have any idea how many will agree with the question then use 50% as that gives the highest figure, to be on the safe side.
3. Whether the difference in two results is statistically significant or not. If you conduct two surveys and in one 35% of 2,000 people (from an infinite population) agree with you and in the other 42% of 85 people agree with you then is this difference significant or not? In this case it isn’t.

The maths behind it

For the formulae that follow

e is the margin of error returned z is the z value from the confidence slider as 1.645 (90%), 1.96 (95%), 2.576 (99%)
n is the sample size
p is the proportion
N is the population

Margin of error
For infinite populations this is calculated as

s = √p(1-p)
e = zs ÷ √n

which in Javascript is

s = Math.sqrt(p*(1-p));
e = z * s / Math.sqrt(n);

For finite populations the value of n is adjusted before being used in the formula above

n_adj = (N-1)n ÷ (N-n)

which in Javascript is

n = (pop - 1) * n / (pop - n);

Sample size
For infinite populations this is calculated as

n = (z² – p(1-p)) ÷ e²

which in Javascript is

n = Math.pow(z,2) * (p * (1 - p)) / Math.pow(e,2);

For finite populations the value of n is adjusted after the formula above has been calculated

n_adj = n ÷ (1+(n-1) ÷ N)

which in Javascript is

n = n / (1 + (n - 1) / pop);

Difference between two proportions
For the case where one sample is not a sub sample of the other and populations are infinite this is calculated as

p = (p₁n₁ + p₂n₂) ÷ (n₁ + n₂)
s = √p(1-p) ÷ n₁ + p(1-p) ÷ n₂
z_test = (p₁-p₂) ÷ s

which in Javascript is

p = (p1*n1 + p2*n2) / (n1 + n2);
s = Math.sqrt(p * (1 - p) / n1 + p * (1 - p) / n2);
ztest = (p1 - p2) / s;

If z_test > z where z is chosen from the confidence slider then the difference is significant.
For finite populations then n is adjusted before being used in the formula above

n_1adj = (N-1)n₁ ÷ (N-n₁)
n_2adj = (N-1)n₂ ÷ (N-n₂)

Where the second sample is a sub sample of the first, p₁ and n₁ are adjusted before being used in all the formulae above

p_1sub = (p₁n₁ – p₂n₂) ÷ (n₁ – n₂)
n_1sub = n₁ – n₂

which is Javascript is

p1 = (p1*n1 - p2*n2) / (n1-n2);
n1 = n1 - n2;