"Researcher Challenges a Host of Psychological Studies" (MEASURING IN A SUBJECTIVE SCIENCE) Science Times, New York Times by Erica Goode, January 2, 2001. Many psychological studies may be compromised by an error having to do with the difficulty of measuring patients' subjective responses. So cautions Linda Bartoshuk, a psychologist at the Yale University School of Medicine. The problem creeps in when researchers use a common measurement technique, the rating scale, to compare different people's subjective experiences creating distorted data. For example, two patients may be asked to rate the degree of their depression on a scale of one to 10. Both may put the figure at a 6. But how is the researcher to know if they actually feel the same depth of pain? Put another way, one man's pain may be another's twinge. o After reviewing eight volumes of the journal Psychology and Behavior, Bartoshuk found the error in 17 of 64 human studies that used rating scales. o In some cases the mistake completely invalidated the studies' findings, she says. o Ratings scales are highly effective when researchers want to find out how the same subjects' feelings, attitudes or sensations change over time or in different situations.
[Fred A. Baughman Jr., MD:
However, it is invalid to assume that ratings on a rating scale by
(1) 2 separate teachers of the same child, (2) by a teacher and a
parent, or (3) by a teacher and a school counselor, are comparable.
This is why such raters so often varywith one judging the child ADHD
the other normal. Such differences of opinion, being entirely
subjective, are the rule. If persons using rating scales
(psychologists, psychiatrists and others) acknowledge that all they are
rating/evaluating/measuring, are a persons/childs/subjects feelings,
attitudes or sensations, that would be OK. Likewise, if those
diagnosing ADHD made no claim other than that they were measuring the
frequency of certain behaviors in normal children, their would be less
to object to. However, the claim that such a rating device at any
cut-off point reveals a physical abnormality, trait, or marker is never
valid. Such things require objective, physical or chemical criteria.
In medicine, there must always be a physical abnormality/finding (this
includes chemical). The fact that we speak of a biological organism, a
human, a dog or a horsephysical entities, is not what is meant. Rather
a physical abnormality must be found and must be demonstrable to others
by some technologygross, microscopic, chemical, x-ray, or
imagingotherwise.]
· Adjectives also mean different things to different people, she points out. For example, a famine victim's "very hungry" probably indicates a level of hunger far higher than the "very hungry" of a Fortune 500 CEO questioned an hour before dinner. · The mistake is common in other fields as well. Monica Biernat of the University of Kansas and Melvin Manus of the University of Michigan have discovered the same problem in studies of sensory perception. They discovered that subjects do not always mean the same thing when they apply descriptions such as "tall," "aggressive," or "financially successful" to others.