Posted by

"Researcher Challenges a Host of Psychological Studies"
Science Times, New York Times
by Erica Goode, January 2, 2001.

Many psychological studies may be compromised by an error
having to do with the difficulty of measuring patients'
subjective responses. So cautions Linda Bartoshuk, a
psychologist at the Yale University School of Medicine.

The problem creeps in when researchers use a common
measurement technique, the rating scale, to compare different  people's
subjective experiences – creating distorted data.
For example, two patients may be asked to rate the degree of
their depression on a scale of one to 10. Both may put the
figure at a 6. But how is the researcher to know if they
actually feel the same depth of pain? Put another way, one man's pain
may be another's twinge.

   o   After reviewing eight volumes of the journal
Psychology and Behavior, Bartoshuk found the error in 17
of 64 human studies that used rating scales.

   o   In some cases the mistake completely invalidated the studies'
findings, she says.

   o   Ratings scales are highly effective when researchers want to find
out how the same subjects' feelings, attitudes or sensations change over
time or in different situations.

[Fred A. Baughman Jr., MD:
However, it is invalid to assume that ratings on a rating scale by
(1) 2 separate teachers of the same child, (2) by a teacher and a
parent, or (3) by a teacher and a school counselor, are comparable.
This is why such raters so often vary—with one judging the child ‘ADHD’
the other ‘normal.’ Such differences of opinion, being entirely
subjective, are the rule. If persons using rating scales
(psychologists, psychiatrists and others) acknowledge that all they are
rating/evaluating/measuring, are a person’s/child’s/subject’s feelings,
attitudes or sensations, that would be OK. Likewise, if those
diagnosing ADHD made no claim other than that they were measuring the
frequency of certain behaviors in normal children, their would be less
to object to. However, the claim that such a rating device at any
cut-off point reveals a physical abnormality, trait, or marker is never
valid. Such things require objective, physical or chemical criteria.
In medicine, there must always be a physical abnormality/finding (this
includes chemical). The fact that we speak of a biological organism, a
human, a dog or a horse—physical entities, is not what is meant. Rather
a physical abnormality must be found and must be demonstrable to others
by some technology—gross, microscopic, chemical, x-ray, or

· Adjectives also mean different things to different people, she points
out.  For example, a famine victim's "very hungry" probably indicates a
level of hunger far higher than the "very hungry" of a Fortune 500 CEO
questioned an hour before dinner.

· The mistake is common in other fields as well.  Monica Biernat of the
University of Kansas and Melvin Manus of the University of Michigan have
discovered the same problem in studies of sensory perception.  They
discovered that subjects do not always mean the same thing when they
apply descriptions such as "tall," "aggressive," or "financially
successful" to others.

Leave a Reply

  • (will not be published)