Fishing for Data

Description:

The argument draws a conclusion from correlations observed in a sample, but only after the sample has already been drawn, and without declaring in advance what correlations the experimenter was expecting to find (i.e. without declaring an experimental hypothesis).

Examples:

"Three of my four children were born in February, and all three are left-handed. Apparently most people born in February are left-handed."

"We took a survey of our class and discovered that, out of thirty students, seven were born in June. We conclude that college students are much more likely to be born in June than in any other month."

Discussion:

Any sample is likely to have certain peculiarities. This is predicted by the laws of probability. Even so, it is unlikely that the peculiarities of a given sample will happen to support any particular pre-designated hypothesis. That is why good scientific method requires that the "experimental hypothesis" (and its opposite, the "null hypothesis") be clearly stated before a consideration of the data.

Actually, using the peculiarities found in samples to suggest new lines of research may not be a bad idea. Occasionally, such peculiarities may indicate some underlying causal mechanism, although usually they will be nothing more than the random variation that we should expect to see in a random sample. When we do use an odd regularity to suggest an underlying causal mechanism, we are using the peculiarity, not as the major premiss of an Induction, but as the minor premiss of a Retroduction. The conclusion should not be a generalization from the sample (asserting something about the population); rather, the conclusion should suggest a mechanism to explain the observed peculiarity.

Source: Charles S. Peirce, Collected Papers. Peirce regarded this as one of the most serious and common errors of Inductive logic.

WELCOME                     EXPLANATION OF PRINCIPLES                                     TABLE OF FALLACIES                        EXERCISES                     INDEX