Uncharacteristic Sample

Description:

The argument draws a conclusion from a sample that is not sufficiently diverse to give a fair representation of the population about which the conclusion is being drawn.

Comments:

Be careful not to confuse this fallacy with Hasty Generalization. A sample may be quite large, but still not be representative of the whole, not because it is too small, but because it is too narrow.

Examples:

"Judging by the opinions of people who call in to talk shows, most Americans believe that the F.B.I. is a bunch of jack-booted thugs."

"American cars are built to last. In a sample of one thousand American-made automobiles driven in San Diego, we found that most of them showed little corrosion or weathering, even if they had been on the road for over ten years."

Discussion:

Diversity in a sample is actually a much more important characteristic than mere size. A small sample can be quite predictive, if it is as diverse as the population it is called upon to represent. For example, in order to determine whether a bunch of grapes has seeds or not, it is usually enough to sample just one grape. This is because a bunch of grapes is exceptionally uniform: all the grapes are genetically identical to the others. They may vary with respect to size (which depends on growth, not genetics), but they are unlikely to vary with respect to having (or lacking) seeds.

On the other hand, a sample can be very large yet still be misleading if it is not sufficiently diverse. In the example above, a sample of automobiles in San Diego might involve thousands of cars, yet still be likely to lead to a false conclusion, since the weather is San Diego is uniformly temperate. Hence cars in San Diego are not subject to harsh winter weather and the salt on roads used to melt ice, which is characteristic of the general population of cars throughout the United States.

Source: I first became aware of this fallacy from W. Ward Fearnside and William B. Holther, Fallacy: the Counterfeit of Argument (1959). However, Fearnside and Holther name it "Unrepresentative Generalization." I coined the term "Uncharacteristic Sample," which seems to me to be a better description of the fallacy. I doubt that Fearnside and Holther's book is the earliest source for this fallacy.

WELCOME                     EXPLANATION OF PRINCIPLES                                     TABLE OF FALLACIES                        EXERCISES                     INDEX