Inappropriate Operational Definition

Description:

The argument draws a conclusion about a property that cannot be observed directly based on a study in which an operational definition was used to replace the property of interest with a clearly observable and measurable property. However, the operational definition fails to adequately capture the property of interest.

Examples:

"Children spend more time playing than adults do. That proves that children are more creative than adults."

"Polish immigrants score poorly on I.Q. tests. Apparently Poles just aren't very bright."

Discussion:

In making observations on a sample, it is almost always necessary to specify precisely which property or behavior we are trying to observe. Are children more active than adults? To settle this question we need to observe some children and some adults. But what precisely should we be watching for? We need to know which behaviors count as "being active." Running and jumping, for example, may be suitably observable. In specifying precisely what we intend to observe we create what is called an "operational definition." It is in the nature of operational definitions to be more limited and behavioristic than the concepts they are intended to capture. Within reason, this is perfectly acceptable, and unavoidable in any case. However, sometimes the difference between the original concept and the operational definition becomes so great that we are not in fact observing what we claim to be observing. The most notorious example is the operational definion of "intelligence" as "scores well on a Stanford-Benet I.Q. test." It is known that success on the Stanford-Benet I.Q. test is heavily influenced by familiarity with the culture of white, middle-class America. People unfamiliar with that culture tend to score poorly on the test, even if they are very "intelligent" by a more usual understanding of the word.

Source: This error is well-known to researchers, and any good text book on research methods will contain a discussion of the issue. However, I believe I am probably the first to include this error on a list of logical fallacies.

WELCOME                     EXPLANATION OF PRINCIPLES                                     TABLE OF FALLACIES                        EXERCISES                     INDEX