Tainted Data

Description:

The argument draws a conclusion from observations that were obtained in a manner that would be likely to tilt or corrupt them.

Comments:

This is the fallacy that occurs when an experiment is conducted in a manner that causes the phenomenon to be observed to be changed (and distorted) by the manner in which the observation was made. It includes asking loaded or leading questions on a survey, or giving visual or auditory clues that indicate a preferred answer.

Examples:

"Our campaign workers, while passing out buttons and literature for Senator Fletcher, asked people which candidate they preferred. A large majority said they favored Senator Fletcher, so we naturally believe he is going to win."

"I don't think my Tommy is a behavior problem. When I visited your class he didn't do anything out of line the whole time I was watching him."

Discussion:

Scientists were very excited when a sample of gas trapped inside a Martian rock tested positive for methane, a gas produced by living organisms. On re-examination, the trace of methane was not detected, and it is now thought that the methane trace may have been due to contamination brought to Mars by the very instrument that performed the test. Tainted data occurs when an observation is affected by something brought to the obervation by the observer. Scientists must often be careful about the way in which evidence is handled. A blood sample from a murder case would be no good as evidence if it were to become contaminated with other materials. The notion of "tainting" usually refers to chemical or biological samples, but there is no reason why the concept should not also include survey data based on loaded or leading questions, or data from questionaires in which the respondents were not guaranteed anonymity.

Changing the phenomenon to be observed is a serious problem in several sciences from sociology to particle physics. In particle physics, "observations" usually involve bouncing photons off the small particles to be observed. This is like trying to find the location of a beach ball by throwing baseballs at it--which will, of course, cause the beach ball to change its location. In social science experiments, it is well known that experimental subjects will behave differently merely because they know they are being observed (which is known as the Hawthorne Effect) or because they are expecting a certain result (the Placebo Effect). Some of these effects may be impossible to entirely filter out. However, it clearly taints the results of a study to let the subjects of the experiment know what answer is "expected" of them. To prevent such tainting of the data, experimental subjects should be as "blind" as possible, i.e. they should not know precisely how their answer will be used as data. Ideally--where this does not violate other ethical rules!--they should not even know they are being observed.

Source: The term "tainted data" is widely used by researchers, and any good research methods text book should contain a discussion of the topic. I doubt that an original source for the term can be identified. However, I believe mine is the first list to explicitly include Tainted Data as a fallacy.

WELCOME                     EXPLANATION OF PRINCIPLES                                     TABLE OF FALLACIES                        EXERCISES                     INDEX