An Interactive Graph for Choosing Sample Size
A good chart is the best way to understand the law of diminishing returns when it comes to sample size. So for our June 2011 newsletter we built an interactive graph for choosing sample size. It’s cool, educational, and useful. Moreover, it will show you just how mind boggling the numbers behind sampling can be. It may even give you more sympathy for the majority of people who just don’t “get it” or believe it when it comes to statistical sampling.
What does Versta Research recommend when it comes to sample size? Well, the answer always depends on the type of study, the objectives of the study, the audience to whom it will be presented, and of course budget. We can make legitimate statistical calculations based on any sample size, but there are dramatic trade-offs in precision and cost no matter what sample size is chosen.
Here, however, are some general guidelines:
If you have the budget for a large sample, don’t even consider going beyond a couple thousand, as you gain so little. Beyond a sample size of 2,000 (which gives you a margin of error of about ±2%) you would have to pull an additional 4,700 into your sample (for a total of 6,700) to gain just one more percentage point in precision. The benefit of doing this will almost never exceed the cost of including that many more people in your sample.
The exception: If you need to understand segments or subgroups within your sample, choose your sample size based on the precision you need for those smaller groups, not the sample overall.
With a small sample there is a substantial gain in precision for every random person you add to the sample. The difference between a sample of 1,000 and 1,075 is relatively small, decreasing the maximum margin of error by just a tenth of a percentage point. But the difference between a sample of 50 and 125 is dramatic, decreasing the maximum margin of error by more than five percentage points. Another twenty five, fifty, or one hundred respondents can make an important difference when you are looking at small samples.
Most sample sizes will range from about 100 and 1,200 and within this range, for each additional 50 or 100 people you include in your sample, you gain a decent improvement in the precision of your estimates. So the questions to consider are always (1) how much precision do you really need, and (2) what is the cost of including each additional person?
In academic journals you will find studies with sample sizes as small as thirty to fifty people. Some of the healthcare research we do relies on samples sizes of fifty to one-hundred. Most other research we do, including that for publication in media outlets, relies on samples ranging from 300 to 1,200.
Unfortunately there is no magic number for sample size, whether it be thirty, one hundred, three hundred, or one thousand. But there is a magic phone number worth trying: (312) 348-6089. Versta Research has a great deal of experience choosing sample sizes and consulting with research, marketing, and communications teams on the key questions that need to be answered within constraints of time and budget. As always, the magic is not in any number itself, but in the highly skilled way in which it is deployed and then turned from data into stories.
—Joe Hopper, Ph.D.
