USGS Home
Contact USGS
Search USGS

Randomization can occur at 2 levels. In both experiments and sample surveys, randomization means that the objects to be studied are randomly selected from some population (called a target population) for which inference is desired. Accordingly, each member of that population has some chance of being included in the sample. Chances may be the same for all members, but that is not necessary. At a second level, in a manipulative experiment, randomization means that the treatment each unit receives is randomly determined.

Randomization makes variation among sample units, due to variables that are not accounted for, act randomly, rather than in some consistent and potentially misleading manner. Randomization thereby reduces the chance of confounding with other variables. Instead of controlling for the effects of those unaccounted-for variables, randomization makes them tend to cancel one another out, at least in large samples. In addition, randomization reduces any intentional or unintentional bias of the investigator. It further provides an objective probability distribution for a test of significance (Barnard 1982).

While randomizing the assignment of treatments to units is crucial in experimentation, I suggest that randomization in selecting the units in an experiment or sample survey is less important than control or replication. First, the intended benefits of randomization apply only conceptually. Randomly sampling from a population does not ensure that the resulting sample will represent that population, only that, if many such samples are taken, the average will be representative. But in reality only a single sample is taken, and that single sample may or may not be representative.

Randomization does make variation act randomly, rather than systematically. However, this property is only conceptual, applying to the notion that samples were repeatedly taken randomly. The single sample that was taken may or may not have properties that appear systematic. Randomization ostensibly reduces hidden biases of, or "cheating" by, an investigator. But, if an investigator wishes to cheat, why not do so but say that randomization was employed (Harville 1975)?

What Does a Sample Really Represent?

Any sample, even a nonrandom one, can be considered a representative sample from some population, if not the target population. What is the population for which the sample is representative? Extrapolation beyond the area from which any sample was taken requires justification on nonstatistical bases. For example, studies of animal behavior (or physiology) based on only a few individuals may reasonably be generalized to entire species if the behavior patterns (or physiological processes) are relatively fixed (i.e., the units are homogeneous with respect to that feature). In contrast, traits that vary more widely, such as habitat use of a species or annual survival rates, cannot be generalized as well from a sample of comparable size. Consistency of a feature among the sampled and unsampled units is more critical than the randomness of a sample. Can one comfortably draw an inference to a population from a sample, even if that sample is nonrandom? In reality, most useful inferences require extrapolation beyond the sampled population. For example, if we want to predict the consequences of some action carried out in the future based on a study conducted in the past, we are extrapolating forward in time.

Is Randomization Always Good?

Suppose you want to assess the characteristics of vegetation in a 10-ha field. You decide to place 8 quadrats in the field and measure vegetation within each of those quadrats. Results from those 8samples will be projected to the entire field. You can select the 8 points entirely at random. It is possible that all 8quadrats will be within the same small area of the field, however, and be very different from most of the field. Choosing points at random ensures that, if you repeat the process many times, on average you will have a representative sample. But in actuality you have only 1 of the infinitely many possible samples; randomness tells you nothing about your particular sample. It might be perfectly representative of the entire field, or it might be very deviant. The chance that it is representative increases with sample size, so the risk of a random sample not being representative is especially troublesome in small samples.

There are methods for taking samples to increase the chance that they better represent the entire field. One method is to stratify, if there is prior knowledge of some variable likely to relate to the variable of interest. Another method is to take systematic rather than random samples. Hurlbert (1984) emphasized the importance of interspersion in experimental design, having units well distributed in space; this serves 1 goal of randomization, often more successfully. Such balanced designs diminish the errors of an experiment (Fisher 1971).

What Is Independence, and Is It Necessary?

Randomization provides a basis for probability distributions because the observations in a random sample generally are statistically independent. Independence is a mathematically wondrous property, since it facilitates the definition of distributional properties, such as the variance, test statistics, and P-values. But what is independent? Consider, as did Millspaugh et al. (1998), the assessment of habitat preference of animals that occur together. If the animals are inextricably tied together, such as a mother and her dependent offspring, then the locations of each certainly are not independent. If the animals occur together simply because they favor the same habitats, Millspaugh et al. (1998) argued that the individual animals are independently making habitat choices and thus should be treated as independent units. Then there are intermediate situations, such as a mother and her not-quite-dependent offspring. Ascertaining independence is not a simple matter; statistical independence can be evaluated only in reference to a specific data set and a specified model (Hurlbert 1997).

But what is the problem if data are not independent? Suppose you have 100 observations, but only 50 of them are independent, and for each of those there is another observation that is identical to it. So the apparent sample size is 100, but only 50 of those are independent. If you estimate the average of some characteristic of the individuals, the mean in fact will be a good estimator. But the standard error will be biased low. And a test statistic, say, for comparing the mean of that group with another, will be inflated and will tend to reject the null hypothesis too often (e.g., Erickson et al. 2001).

This is a fundamental problem for any test statistic from an individual study. There are ways to correct for the disparity between the number of observations and the number of independent observations. Dependencies among observations sometimes can be modeled explicitly, such as with generalized estimating equations (Liang and Zeger 1986). Dependencies, such as sampling from clusters of units, often result in overdispersion, in which the sample variance exceeds the theoretical value; in such cases certain adjustments to the theoretical variance can be made (McCullagh and Nelder 1989, Burnham and Anderson 2002). A similar issue arises with respect to temporally or spatially correlated observations.

I argue later that problems caused by a lack of independence, while affecting inferences from individual studies, are less consequential than they appear.

Independence and the Scope of Inference

Suppose you are investigating area sensitivity in grassland birds. That is, you wonder whether certain species prefer larger patches of grassland to smaller patches. Area sensitivity might be manifested by reduced densities (not just total abundance) in smaller habitat patches or by the avoidance of habitat edges (Faaborg et al. 1993, Johnson and Winter 1999, Johnson 2001b). Avoidance of edge means that birds are restricted to the interior portions of a patch, which results in reduced densities for the patch as a whole. To determine whether certain species are area-sensitive, you might compare densities of the species in patches of similar habitat but different size. Alternatively, you might examine the locations of birds (let's say nests, but we could consider song perches, etc.) within a habitat patch and determine whether there is evidence that densities of nests are reduced near edges compared to interiors.

For comparing densities, the sample units are patches. Those are the units to which a "treatment" (patch size) pertains; all birds in that patch have the same patch size. For examining edge avoidance, in contrast, the sample units are nests because each has its own, possibly unique, value of the "treatment" (distance to edge). Logically, then, the latter approach would be more powerful because a single patch might produce dozens of sample units (nests), resulting in much larger sample sizes.

Assuming there is no free lunch, what is happening here? The disparity is the scope of inference. If we study densities in patches, the studied patches can be considered a sample from some target population of patches, and inferences should apply to that population. If we study nests within a patch and examine distances to an edge, the inference is only to that single patch. You might conclude that birds avoid locating their nests near a habitat edge, but that conclusion applies only locally.

Replication Is Necessary for Randomization to Be Useful

The properties of randomization in the selection of units to study are largely conceptual; that is, they pertain hypothetically to some long-term average. For example, randomization makes errors act randomly, rather than in a consistent direction. But in any single observation, or any single study, the error may well be consistent. It is only through replication that long-term properties hold.