USGS Home
Contact USGS
Search USGS

Any imaginative wildlife biologist can easily list a dozen or more variables that could influence a response variable of interest, be it the number of squirrels in a woodlot, the nest success rate of bobolinks (Dolichonyx oryzivorus) in a field, or the survival rate of mallards in a particular year. An investigation of such a response variable will adequately determine the influences of only a few of the multitude of explanatory variables. The remainder will not be under the investigator's control and indeed may not even be known to the investigator, or may be known but not measured.

The extent to which these other variables influence the response variable confounds the observed relationship between the response variable and the explanatory variables under study. In addition, those unknown influences may restrict the scope of inference for the relationships that are discovered.

Consider again our example of estimating the effect on squirrel density of selectively logging woodlots. Suppose that, in general, such logging does reduce squirrel density. In any particular situation, however, that result might not follow because of the effects of other (possibly unmeasured) variables. Predators of squirrels in a logged woodlot might have been reduced, offsetting any population decline associated with logging. Or an outbreak of disease in the squirrels might have reduced their numbers in the unlogged woodlot, erasing any difference between that woodlot and the 1 that was logged.

Design control (restricting the range in variation of potentially confounding variables) reduces the influence of such variables, but that practice is not always feasible. Randomization tends to make variables that are not studied act, well, randomly, rather than in some consistent direction. With replication, those variables then contribute to variance in the observed relationship, rather than a bias. Nonetheless, in any single study, those unobserved relationships may give us a misleading impression of the true relationship between the response variable and the explanatory variables under study.

Metareplication provides us greater confidence that certain relationships are general. Obtaining consistent inferences from studies conducted under a wide variety of conditions will assure us that the conclusions are not unique to the particular set of circumstances that prevailed during the study. Further, by metareplicating studies, we need not worry about P-values, issues of what constitute independent observations, and other concerns involving single studies. We can take a broader look, seeking consistency of effects among studies. Consistent results suggest generality of the relationship. Inconsistency will lead us either not to accept the results as truth or to determine conditions under which the results hold and those under which they do not. That approach will lead to understanding the mechanisms.

If, indeed, most individual wildlife studies are flawed to some degree, why have we any confidence whatsoever in the science? Perhaps the errors are inconsequential. Or, possibly we don't really believe in those single studies anyway, and don't take action until a clear pattern emerges from disparate studies of the phenomenon. Our innate Bayesianism may be weighting results from an individual study with our prior thinking, based on other things we know or believe.

To conclude, we certainly should use the best statistical methods appropriate for a given data set to maximize the value of those data. However, as Hurlbert (1994:495) wisely noted, "lack of understanding of basic principles and simple methods by practicing ecologists is a serious problem, while under-use of advanced statistical methods is not." More important than the methods used to analyze data, we should collect the best data we can. We should use the principles of design-controls, randomization, and replication in manipulative experiments; matching and measuring appropriate covariates in observational studies. And, most critically, studies themselves need to be replicated to have confidence in the findings and their generality. Metareplication exploits the value of small studies, obviates concerns about P-values and similar issues, protects against claiming spurious effects to be real, and facilitates the detection of small effects that are likely to be missed in individual studies.