Statistical Significance Calculator Widget

This is an OSX dashboard widget that enables quick calculation of data needed to assess statistical significance:
 The margin of error for a specific result for a specific sample size. For example if you surveyed 100 people out of a total population of 100,000 and 85% said they did like chocolate then the margin of error is 7% at a confidence level of 95%, which means that if you repeated the survey enough times within the same population then 95% of the time you would get an answer between 78% and 92%.
 The sample size needed to obtain a specific margin of error for a specific result. For example, if you have a population of 500 people and you want to know how many agree with a question to within 5% either way and you suspect the around 40% will agree then the sample size needed is 212, with the same restriction as above on this being only 95% likely. If you do not have any idea how many will agree with the question then use 50% as that gives the highest figure, to be on the safe side.
 Whether the difference in two results is statistically significant or not. If you conduct two surveys and in one 35% of 2,000 people (from an infinite population) agree with you and in the other 42% of 85 people agree with you then is this difference significant or not? In this case it isn’t.
The maths behind it
For the formulae that follow
e is the margin of error returned z is the z value from the confidence slider as 1.645 (90%), 1.96 (95%), 2.576 (99%)
n is the sample size
p is the proportion
N is the population
Margin of error
For infinite populations this is calculated as
s = √p(1p)
e = zs ÷ √n
which in Javascript is
s = Math.sqrt(p*(1p));
e = z * s / Math.sqrt(n);
For finite populations the value of n is adjusted before being used in the formula above
n_{adj} = (N1)n ÷ (Nn)
which in Javascript is
n = (pop  1) * n / (pop  n);
Sample size
For infinite populations this is calculated as
n = (z² – p(1p)) ÷ e²
which in Javascript is
n = Math.pow(z,2) * (p * (1  p)) / Math.pow(e,2);
For finite populations the value of n is adjusted after the formula above has been calculated
n_{adj} = n ÷ (1+(n1) ÷ N)
which in Javascript is
n = n / (1 + (n  1) / pop);
Difference between two proportions
For the case where one sample is not a sub sample of the other and populations are infinite this is calculated as
p = (p_{1}n_{1} + p_{2}n_{2}) ÷ (n_{1} + n_{2})
s = √p(1p) ÷ n_{1} + p(1p) ÷ n_{2}
z_{test} = (p_{1}p_{2}) ÷ s
which in Javascript is
p = (p1*n1 + p2*n2) / (n1 + n2);
s = Math.sqrt(p * (1  p) / n1 + p * (1  p) / n2);
ztest = (p1  p2) / s;
If z_{test} > z where z is chosen from the confidence slider then the difference is significant.
For finite populations then n is adjusted before being used in the formula above
n_{1adj} = (N1)n_{1} ÷ (Nn_{1})
n_{2adj} = (N1)n_{2} ÷ (Nn_{2})
Where the second sample is a sub sample of the first, p_{1} and n_{1} are adjusted before being used in all the formulae above
p_{1sub} = (p_{1}n_{1} – p_{2}n_{2}) ÷ (n_{1} – n_{2})
n_{1sub} = n_{1} – n_{2}
which is Javascript is
p1 = (p1*n1  p2*n2) / (n1n2);
n1 = n1  n2;
This is beautiful!
very useful bravo
hi,
in the difference analysis part, i was not able to find “significant difference” cases.
I tried 5% of a sample 100 items and 95% of a sample of 100 items with a 90% significance and still the difference was “not significant…”
i have my doubts and you will understand.
regards,
It does look as though that code is no longer working. I’m not sure what version of OSX that stopped working with. Unfortunately the Apple development tools for widgets are pretty old so I’m not sure if I can fix this.