It is important to perform power and sample size calculations at the start of the study. Before the data is collected, the question that must be answered is: What is the sample size that I need in order to reject the null hypothesis? The answer to this question is multifaceted. First and foremost, it is important to know what power you are trying to achieve.

Power is the probability of rejecting the null hypothesis when the null hypothesis is false. That is, when you perform the test and reject the null hypothesis because the p-value is very small, you are saying that there is only a small chance that the null hypothesis is true. Sometimes the null hypothesis needs to be rejected because it is false, but the p-value is not low enough and so we erroneously state that the null hypothesis is not rejected. Many times this happens because the sample was not large enough. And so we come back to the main question: How large of a sample size do we need in order to have enough Power to reject the null hypothesis?

There are a few items that we need in order to answer that question.
1. What amount of Power do we feel comfortable with?
2. What α will we use to reject the null hypothesis?
3. What is the effect size that we expect to see in the analysis?

Here are the answers:
1. The gold standard for Power is 80%. That means that if we have 80% Power, we have an 80% chance that we will reject the null hypothesis if the null hypothesis is false.
2. α (alpha) is the probability of rejecting the null hypothesis when it is true, so we like to pick a small number. The gold standard is 5%. There is always a chance that the null hypothesis is true, but the values in the sample seem to correspond to the alternative hypothesis. Thus, if the probability of this situation is less than 5%, then we can say that it is unlikely that the event would have occurred if the null hypothesis is true; therefore, we reject the null hypothesis.
3. Effect size is a standardized measure of the difference between two groups. To standardize, we divide the difference by the standard deviation. This measure helps us decide how big of a sample we need in order to show that this difference did not occur by chance alone. For example, let's say you want to know if a change with an effect size of 3 on a scale from 0 to 100 is statistically significant. An effect size of 3 with only 10 people sampled may be due to chance alone, because large variations among 10 people are likely. However, if you sample 1000 people, then this effect size becomes more significant. The reason for this follows from the central limit theorem.

After these three questions are answered, the calculation for sample size is very easy and can be done in a very short amount of time. 

Contact Statistical Consulting Network for help with your Power and Sample size calculations.