Lincoln index

The Lincoln index is a statistical measure used in several fields to estimate the number of cases that have not yet been observed, based on two independent sets of observed cases. Described by Frederick Charles Lincoln in 1930, it is also sometimes known as the Lincoln-Petersen method after C.G. Johannes Petersen who was the first to use the related mark and recapture method.^[1]

Applications

Consider two observers who separately count the different species of plants or animals in a given area. If they each come back having found 100 species but only 5 particular species are found by both observers, then each observer clearly missed at least 95 species (that is, the 95 that only the other observer found). Thus, we know that both observers miss a lot. On the other hand, if 99 of the 100 species each observer found had been found by both, it is fair to expect that they have found a far higher percentage of the total species that are there to find.

The same reasoning applies to mark and recapture. If some animals in a given area are caught and marked, and later a second round of captures is done: the number of marked animals found in the second round can be used to generate an estimate of the total population.^[2]

Another example arises in computational linguistics for estimating the total vocabulary of a language. Given two independent samples, the overlap between their vocabularies enables a useful estimate of how many more vocabulary items exist but did not happen to show up in either sample. A similar example involves estimating the number of typographical errors remaining in a text, from two proofreaders' counts.

Formulation

The Lincoln Index formalizes this phenomenon. If E1 and E2 are the number of species (or words, or other phenomena) observed by two independent methods, and S is the number of observations in common, then the Lincoln Index is simply

$L={E_{1}E_{2} \over S}$

If S = 0 (that is, if there is no overlap), then the Lincoln Index is undefined and no estimate is justified. This can arise if the observers only find a small percentage of the actual species (perhaps by not looking hard enough or long enough), if the observers are using methods that are not statistically independent (for example if one looks only for large creatures and the other only for small), or in other circumstances.

Limitations

The Lincoln Index is merely an estimate. For example, the species in a given area could tend to be either very common or very rare, or tend to be either very hard or very easy to see.^[3] Then it would be likely that both observers would find a large share of the common species, and that both observers would miss a large share of the rare ones. Such distributions would throw off the consequent estimate. However, such distributions are unusual for natural phenomena, as suggested by Zipf's Law).

T. J. Gaskell and B. J. George propose an enhancement of the Lincoln Index that claims to reduce bias.^[4]

See also

• Sampling Theory

Further reading

• Lincoln, Frederick C. (May 1930). Calculating Waterfowl Abundance on the Basis of Banding Returns. Circular. 118. Washington, DC: United States Department of Agriculture. Retrieved 21 May 2013.
• Petersen, C. G. J. (1896). "The Yearly Immigration of Young Plaice Into the Limfjord From the German Sea", Report of the Danish Biological Station (1895), 6, 5–84.
• T. J. Gaskell; B. J. George (1972). "A Bayesian Modification of the Lincoln Index". Journal of Applied Ecology. 9 (2): 377–384. doi:10.2307/2402438.

Notes

1. Southwood, T.R.E. & Henderson, P. (2000) Ecological Methods, 3rd edn. Blackwell Science, Oxford.
2. "Estimating Population Sizes by Mark-recapture and Removal Sampling Methods". University of Texas.
3. T. Bohlin; B. Sundstrom (1977). "Influence of unequal catchability on population estimates using the Lincoln and the removal method applied to electro-fishing". OIKOS (28): 123–129. JSTOR 3543331.
4. Gaskell and George (1972)