USGS Home
Contact USGS
Search USGS

Wildlife researchers seem to be doing everything wrong. Few of our studies employ the hypothetico-deductive approach (Romesburg 1981) or gain the benefits from strong inference (Platt 1964). We continually conduct descriptive studies, rather than the more effective manipulative studies. We rarely select study areas at random, and even less often do the animals we study constitute a random sample. We continue to commit pseudoreplication errors (Hurlbert 1984, Heffner et al. 1996). We confuse correlation with causation (Eberhardt 1970). Frequently we measure the wrong variables such as indices to things we really care about (Anderson 2001). And we may measure them in the wrong places (convenience sampling; Anderson 2001). We often apply meaningless multivariate methods to the results of our studies (Rexstad et al. 1988). We test null hypotheses that not only are silly but are known to be false (Cherry 1998, Johnson 1999, Anderson et al. 2000). We rely on nonparametric methods that are neither necessary nor appropriate (Johnson 1995).

Such problems permeate our field. In my early years as a hypercritical statistician, I read many articles in The Journal of Wildlife Management and related journals. In virtually every article, I found problems—often serious ones—in the methods used to analyze data. That experience was repeated later in a class in evolutionary ecology. During that class, we critically reviewed many key papers in evolutionary ecology. Some students were assigned to attack, others to defend those articles. We identified substantial problems in the design, analysis, or interpretation in nearly all of those influential and highly regarded studies. Despite all our transgressions, we must be doing something right. We have brought some species back from the brink of extinction. The bald eagle (Haliaeetus leucocephalus), whooping crane (Grus americana), Aleutian Canada goose (Branta canadensis leucopareia), and gray wolf (Canis lupus) were extremely rare over much or all of their range only a few years ago; now they are much more common. Many of us had given up on the black-footed ferret (Mustela nigripes) and California condor (Gymnogyps californianus), species that, while still at risk, appear to be recovering. And we can manage for abundance if we want to, such as we have done for white-tailed deer (Odocoileus virginianus) and mallards (Anas platyrhynchos). Recently, Jack Ward Thomas spoke of the "tremendous record of success" in our field (Thomas 2000:1).

Why this apparent inconsistency between our error-prone methods and the successes of our profession? I hope to address that question here by discussing what truly is important in scientific research. I first discuss causation, then manipulative experimentation as a powerful way of learning about causal mechanisms. The 3 cornerstones of experimentation are control, randomization, and replication. These features also are integral to observational studies and sample surveys, which are more common in our field. For those types of studies especially, I argue that the most important feature is replication. Further, I expand this concept to the level of metareplication—replication of entire studies—and suggest that this is the most reliable method of learning about the world. It is a natural way of human thinking and is consistent with a Bayesian approach to statistical inference. Metareplication allows us to exploit the values of small studies, each of which individually may be unable to reach definitive conclusions. Metareplication provides us greater confidence that certain relationships are general and not specific to the circumstances that prevailed during a single study.