Sampling
Before conducting a survey, you must choose a relevant survey population. And, unless a survey population is very small, it is usually impossible to survey the entire relevant population. Therefore, researchers usually just survey a sample of a population from an actual list of the relevant population, which in turn is called a sampling frame. With a carefully selected sample, researchers can make estimations or generalizations regarding an entire population's opinions, attitudes or beliefs on a particular topic.
Sampling Procedures and Methods
There are two different types of sampling procedures--probability and nonprobability. Probability sampling methods ensure that there is a possibility for each person in a sample population to be selected, whereas nonprobability methods target specific individuals. Nonprobability sampling methods include the following:
- Purposive samples: to purposely select individuals to survey.
- Volunteer subjects: to ask for volunteers to survey.
- Haphazard sampling: to survey individuals who can be easily reached.
- Quota sampling: to select individuals based on a set quota. For example, if a census indicates that more than half of the population is female, then the sample will be adjusted accordingly.
Clearly, there can be an inherent bias in nonprobability methods. Therefore, according to Weisberg, Krosnick, and Bowen (1989), it is not surprising that most survey researchers prefer probability sampling methods. Some commonly used probability sampling methods for surveys are:
- Simple random sample: a sample is drawn randomly from a list of individuals in a population.
- Systematic selection procedure sample: a variant of a simple random sample in which a random number is chosen to select the first individual and so on from there.
- Stratified sample: dividing up the population into smaller groups, and randomly sampling from each group.
- Cluster sample: dividing up a population into smaller groups, and then only sampling from one of the groups. Cluster sampling is " according to Lee, Forthofer, and Lorimer (1989), is considered a more practical approach to surveys because it samples by groups or clusters of elements rather than by individual elements" (p. 12). It also reduces interview costs. However, Weisberg et. al (1989) said accuracy declines when using this sampling method.
- Multistage sampling: first, sampling a set of geographic areas. Then, sampling a subset of areas within those areas, and so on.
Sampling and Nonsampling Errors
Directly related to sample size are the concepts of sampling and nonsampling errors. According to Fox and Tracy (1986), surveys are subject to both sampling errors and nonsampling errors.
A sampling error arises from the fact that inevitably samples differ from their populations. Therefore, survey sample results should be seen only as estimations. Weisberg et. al. (1989) said sampling errors cannot be calculated for nonprobability samples, but they can be determined for probability samples. First, to determine sample error, look at the sample size. Then, look at the sampling fraction--the percentage of the population that is being surveyed. Thus, the more people surveyed, the smaller the error. This error can also be reduced, according to Fox and Tracy (1986), by increasing the representativeness of the sample.
Then, there are two different kinds of nonsampling error--random and nonrandom errors. Fox and Tracy (1986) said random errors decrease the reliability of measurements. These errors can be reduced through repeated measurements. Nonrandom errors result from a bias in survey data, which is connected to response and nonresponse bias.
Confidence Level and Interval
Any statement of sampling error must contain two essential components: the confidence level and the confidence interval. These two components are used together to express the accuracy of the sample's statistics in terms of the level of confidence that the statistics fall within a specified interval from the true population parameter. For example, a researcher may be "95 percent confident" that the sample statistic (that 50 percent favor candidate X) is within plus or minus 5 percentage points of the population parameter. In other words, the researcher is 95 percent confident that between 45 and 55 percent of the total population favor candidate X.
Lauer and Asher (1988) provide a table that gives the confidence interval limits for percentages based upon sample size (p. 58):
Sample Size and Confidence Interval Limits
(95% confidence intervals based on a population incidence of 50% and a large population relative to sample size.)
Sample size |
Confidence interval limits for percentages |
10 + |
31% |
20 + |
22% |
30 + |
18% |
40 + |
16% |
50 + |
14% |
60 + |
13% |
70 + |
12% |
80 + |
11% |
90 + |
10.3% |
100 + |
9.8% |
150 + |
8.0% |
200 + |
6.9% |
250 + |
6.2% |
300 + |
5.6% |
400 + |
4.9% |
500 + |
4.4% |
1000 + |
3.1% |
Confidence Limits and Sample Size
When selecting a sample size, one can consider that a higher number of individuals surveyed from a target group yields a tighter measurement, a lower number yields a looser range of confidence limits. The confidence limits may need to be corrected if, according to Lauer and Asher (1988), "the sample size starts to approach the population size" or if "the variable under scrutiny is known to have a much [original emphasis] smaller or larger occurrence than 50% in the whole population" (p. 59). For smaller populations, Singleton (1988) said the standard error or confidence interval should be multiplied by a correction factor equal to sqrt(1 - f), where "f" is the sampling fraction, or proportion of the population included in the sample.
Lauer and Asher (1988) give a table of correction factors for confidence limits where sample size is an important part of population size (p. 60) and also a table of correction factors for where the percentage incidence of the parameter in the population is not 50% (p. 61).
Tables for Calculating Confidence Limits vs. Sample Size
Correction Factors for Confidence Limits When Sample Size (n) Is an Important Part of Population Size (N >= 100)
Sample percentage of population |
Correction factor |
5% |
.98 |
10% |
.95 |
15% |
.92 |
20% |
.89 |
25% |
.87 |
30% |
.84 |
35% |
.81 |
40% |
.78 |
45% |
.74 |
50% |
.71 |
55% |
.67 |
60% |
.63 |
65% |
.59 |
70% |
.55 |
(For n over 70% of N, take all of N)
From Lauer and Asher (1988, p. 60)
Correction Factors for Rare and Common Percentage of Variables
Percentage incidence |
Correction factor |
50% |
none |
40% or 60% |
.98 |
35% or 65% |
.95 |
30% or 70% |
.92 |
25% or 75% |
.87 |
20% or 80% |
.80 |
15% or 85% |
.71 |
10% or 90% |
.60 |
5% or 95% |
.44 |
2.5% or 97.5% |
.31 |
From Lauer and Asher (1988, p. 61)