REPORT OF A WORKSHOP ON EXPERIMENTAL PARAPSYCHOLOGY

• Document Type: CREST
• Collection: STARGATE
• Document Number (FOIA) /ESDN (CREST): CIA-RDP96-00792R000100130001-2
• Release Decision: RIFPUB
• Original Classification: K
• Document Page Count: 26
• Document Creation Date: November 4, 2016
• Document Release Date: September 5, 2003
• Sequence Number: 1
• Publication Date: February 22, 1989
• Content Type: REPORT

Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
Report of a
Workshop on Experimental Parapsychology
International Security and Commerce Program
Office of Technology Assessment
United States Congress
February 22, 1989
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
OFFICE OF TECHNOLOGY ASSESSMENT
International Security and Commerce Program
Workshop
September 30, 1988
Professor James Alcock
Glendon College
Toronto, Canada
Robert G. Jahn
Professor of Aerospace Studies
Princeton University
Princeton, NJ
Daryl Bem
Professor of Psychology
Cornell University
Ithaca, NY
Charles Honorton
Director
Psychophysical Research Laboratories
Princeton, NJ
Ray Hyman
Professor of Psychology
University of Oregon
Eugene, OR
Rapporteur
Courtland Lewis
Arlington, VA
John Palmer
Institute of Parapsychology
College Station
Durham, NC
Theodore Rockwell
Chevy Chase, MD
Marcello Truzzi
Department of Sociology
Eastern Michigan University
Ann Arbor, MI
Jessica Utts
Professor
University of California, Davis
Davis, CA
Chairman
Alan Shaw
Office of Technology Assessment
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
CONTENTS
.I. INTRODUCTION ........................................................... 3
Historical Background ................................................. 3
Nature of Research .................................................... 3
Recent Controversy .................................................... 4
Purpose of the OTA Workshop ........................................... 6
II. DESIGN AND CONDUCT OF EXPERIMENTS ...................................... 7
Methodology ........................................................... 7
Replicability ......................................................... 9
Criteria for Acceptable Experiments .................................. 11
III. ANALYSIS OF DATA ..................................................... 12
IV. THE MEANING OF EXPERIMENTAL RESULTS .................................. 16
Assessments of Meaning and Meaningfulness ............................ 16
What Would Constitute Proof of Psi? .................................. 18
Conflicting Orientations ............................................. 19
Need for a Theory of Psi ............................................. 20
V. RELATION OF PARAPSYCHOLOGY TO THE BROADER WORLD OF SCIENCE ............ 20
Is Parapsychology a Science? ......................................... 20
Resistance by "Establishment" Science ................................ 21
Choices for the Future ............................................... 22
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
The modern field of study known as parapsychology grew directly out of
19th-century "psychical research" into reported psychic phenomena such as
telepathy and clairvoyance. The field had its earliest roots in various
investigations into a wide range of supernatural, occult, and mystical topics
reaching far back into human cultural history. However, the field did not
become formalized as an arena of organized scholarly research until 1882, with
the establishment in London of the Society for Psychical Research (SPR). A
U.S. counterpart, the American Society for Psychical Research (ASPR), was
founded in 1885. Both societies still exist today.1
From the beginning, the field represented by these organizations has been
highly controversial. Although it has occasionally attracted and engaged
well-known scientists, it has generally encountered substantial resistance and
criticism from the scientific "establishment" on the basis of its aims and
methods. To its detriment, parapsychology has had difficulty in freeing
itself of association, in the minds of those outside the field, with extreme
and unsubstantiated claims, commercial ventures of questionable validity, and
a certain amount of quasi-scientific "research" carried out in its general
area of inquiry.
Throughout this century a considerable body of parapsychology research
has been conducted in a manner that attempts deliberately to follow scientific
methodology. The research approaches most often derive from and resemble
research in psychology, although a few programs based on engineering and
applied science have recently appeared. A seminal example of academic
research in the field was the work of J.B. Rhine and Louisa Rhine at Duke
University, beginning in the 1920s. The Rhines established many of the basic
concepts and protocols of modern parapsychology. In 1937, they founded The
Journal of Parapsychology, which is perhaps the leading journal in the field
today.
The general focus of parapsychological research is on "psychic
phenomena," or "psi," which is defined as "processes of information and/or
energy exchange which involve animate consciousness in a manner not currently
explicable in terms of known science."2 The field can be divided into several
major categories and subcategories, as shown in Table 1. Most (but not all)
of the parapsychological research in laboratory settings involves categories I
and II.
1 For a detailed examination of the development of parapsychology, see: Jahn,
R.G. & B.J. Dunne. Margins of Reality: The Role of Consciousness in the
Physical World. New York: Harcourt, Brace, Jovanovich, 1987.
2 Jahn, R.G. The persistent paradox of psychic phenomena: An engineering
perspective. Proceedings of the IEEE, 1982, 70(2): 136-170.
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
In 1957 a professional organization in the field, the Parapsychological
Association (PA), was formed. (Unlike the SPR, membership in the PA requires
formal recognition of professional status by the Association's Council.) It
was accepted as an affiliate by the American Association for the Advancement
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
I. Extrasensory Perception (ESP)
A. Telepathy
B. Clairvoyance
C. Precognition/Retrocognition
D. Animal ESP
II. Psychokinesis (PK)
A. Physical Systems (equipment, etc.)
B. Biological Systems
A. Reincarnation
B. Apparitions
C. Mediumship
IV. Out-of-Body Experiences (OBE)
SOURCE: Jahn, R.G. The persistent paradox of psychic
phenomena: An engineering perspective. Proceedings of the
IEEE, 1982, 70(2): 136-170.
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
of Science (AAAS) in 1969, signaling a tentative new status and recognition of
the field. In 1976 a group of philosophers=, magicians, science writers,
scientists, and others concerned about a widespread increase in popular interest
in the occult formed a Committee for the'-Scientific Investigation of Claims of
the Paranormal (CSICOP).3 CSICOP and a number of affiliated local groups have
since become a primary source of criticism and skepticism directed primarily at
highly publicized claims of the paranormal, but often extending more generally
to the field of parapsychology. A highly charged and long-running debate has
emerged, featuring charges and countercharges between members of CSICOP and the
scientific community, on the one hand, and members of the parapsychological
community, on the other.
One recent chapter of that debate appeared in the March 1985 issue of The
Journal of Parapsychology, in which a leading parapsychology researcher (Charles
Honorton) and a leading psychologist and-critic (Ray Hyman) presented their
views on the nature of one category of parapsychology experiments.4 In the
December 1986 issue of the same journal, the two published a "Joint communique"
in which they explored in a constructive manner areas of agreement and
disagreement regarding experiment design, documentation, and data analysis.5
(Both individuals participated in the OTA workshop.) Another landmark
publication appeared in 1987: a substantial portion of one issue of Behavioral
and Brain Sciences was devoted to articles and letters on the subject. 6
The most recent episode in the continuing controversy over the field
appeared in the form of a December 1987 report by the National Research Council
(NRC), which evaluated parapsychology along with other possible mechanisms for
enhancing the performance of U.S. Army personnel.7 The NRC's evaluation of the
potential of parapsychology was generally negative, prompting strenuous and
detailed claims of bias and unfairness from the parapsychological community.8
It was in this context that the OTA workshop took place. OTA examined
parapsychology at the request of its oversight body, the Technology Assessment
Board, which expressed interest in the human potential aspect of
3 It should be noted that few scientists were involved in the creation of
CSICOP. Although today many of its Fellows are scientists, the organization
has always been led and operated by non-scientists.
4 Hyman, R. The ganzfeld psi experiment: A critical appraisal. and Honorton,
C. Meta-analysis of psi ganzfeld research: A response to Hyman. The Journal
of Parapsychology, 1985, 49(l): 3-91.
5 Hyman, R. & C. Honorton. A joint communique: The psi ganzfeld controversy.
The Journal of Parapsychology, 1986, 50(4): 351-364.
6 Behavioral and Brain Sciences, 1987, 10(4).
7 National Research Council. Enhancing Human Performance: Issues, Theories,
and Techniques. Report of the Committee on Techniques for the Enhancement of
Human Performance, Commission on Behavioral and Social Sciences and
Education. Washington, DC: National Academy Press, 1988.
8 See, for example: Palmer, J.A., C. Honorton, & J. Utts. "Reply to the
National Research Council Study on Parapsychology." Special report prepared
for the Board of Directors of the Parapsychological Association, Inc.,
Research Triangle Park, N.C., 1988.
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
parapsychological research. This workshop and report are OTA's response to that
interest. OTA's intention in holding the workshop was not to continue the
debate described above, but to illuminate it by identifying the main points of
contention and the reasons for them. For that reason, background materials were
researched to provide a fuller elucidation of some issues not adequately aired
in the course of the one-day meeting.
Discussion was limited to the area of laboratory parapsychology.
Participants represented three groups: researchers in parapsychology,
constructive critics of the field, and knowledgeable individuals who follow the
field from a position of more neutral interest or involvement. Although the
meeting was to some extent an occasion for spirited debate, no attempt has been
made to "score" that debate. This report will describe the controversies only
to the extent that they bear on OTA's inquiry into the status of the field.
II. DESIGN. AND CONDUCT OF EXPERIMENTS
Controversy in the area of experiment design and conduct appears to center
around the two issues of flaws in methodology and replicability of experiments.
One of the most persistent and perhaps most damaging charges leveled by the
critics is that various flaws in the design and conduct of nearly all
parapsychology experiments render their results scientifically less meaningful
than they are reported to be -- and, in fact, often account for the results.
Parapsychologists dispute this charge, saying that most of the alleged flaws
have little or no effect on the experimental findings that indicate "psi," and
that much research in psychology (the home discipline of many critics) -- and
indeed in many other fields of science -- exhibits similar flaws without
encountering comparable criticism and questioning of the results. They say the
question of flaws is highly subjective, with individual views differing widely
over what distinguishes an experimental "flower" from an experimental "weed."
Further, the parapsychologists claim, this criticism is a red herring intended
to tarnish the image of parapsychology within the scientific community and the
public and to impede broader awareness of findings and progress in the field.
Those at the workshop asserted strenuously that they and their colleagues make
every effort to conduct their experiments in the most rigorous manner possible.
The real issue, they say, is what constitutes a flaw. The types of flaws that
have been brought into question include purported procedural flaws such as:
o inadequate precautions against "sensory leakage" (for example, allowing
subjects in a telepathy experiment to examine the same "target", such as a
photograph, that had been handled earlier by a person serving as "sender")
o inadequate security provisions (e.g., to prevent tampering with equipment)
o improper randomization techniques (such as hand shuffling of cards)
o feedback (such as failure to randomize targets before presenting them to
the subject for judging)
o incomplete documentation of experimental procedures
o inconsistency of conditions and procedures used in an experiment.
Also at issue are various purported analytical and statistical flaws, including:
o multiple testing or analysis (i.e., to find the test or analysis that gives
the most positive "effect size" for psi)
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
o underestimation of the effective error rate and overestimation of the
actual significance level
o erroneous use of particular statistical'procedures.
In their joint communique, Honorton argued that there is "no significant
correlation between indices of study quality [flaws] and study outcome." Hyman
agreed in most cases, but insisted that there is a positive correlation in the
case of poor randomization, feedback, and inadequate documentation.9 (This
assertion was rebutted in a later analysis.)10 Workshop participants generally
agreed that in any event the "flawless," perfect experiment is very rare in any
field, if it exists at all. The point of the issue, critics say, is that the
presence of a flaw usually implies inadequate controls for error. The most
stringent among them believe that an apparently successful experiment cannot be
considered to have demonstrated a true anomaly (i.e., a possible psi effect)
unless it can be shown that the experiment is completely flawless. The
parapsychologists counter that any such flaws should be an issue only if they
could have systematically influenced the result.
Another point made by parapsychologists is that, by and large, the critics
have done very little experimentation in parapsychology. Therefore, they have
little awareness of the difficulties encountered or of what the truly
significant flaws might be. Indeed, the parapsychologists note, there could be
additional flaws that have not yet been considered and which critics could
identify through actual experimentation.
The NRC report distinguished three types of criticisms relating to flaws in
experiment design and conduct.ll First is what is called the "smoking gun," in
which observed findings are said to be definitely attributable to a specific
flaw or other factor (such as deliberate fraud) that is shown to be present.
Second., and more common, is the "plausible alternative" allegation that a
particular flaw is present and co d have produced the reported results. Third
is the "dirty test tube" concept, which alleges not that a particular discovered
flaw produced the positive result but simply that the presence of flaws in the
experiment demonstrates a general sloppiness which brings the results into
question. In other words, if results have been obtained under conditions that
fail to meet "generally accepted standards" for scientific research (such as
clean test tubes), that fact alone casts doubt on the accuracy and validity of
the results. The authors of the NRC report claimed that, while they could find
no instances of the "smoking gun" or "plausible alternative," even the best
parapsychology experiments exemplify the "dirty test tube" problem (an
allegation that has been heatedly disputed by parapsychologists).
Participants agreed that the burden of proof in the first two categories
should be on the critic making the allegation, as it is in other sciences where
an experiment is challenged on these grounds. In the case of the "dirty test
tube," the critics feel that the burden of proof should be on the experimenter.
This position brings from the parapsychologists the objection that it is non-
9 Hyman & Honorton, op. cit., p. 353.
10 Harris, M.J. & Rosenthal, R. "Interpersonal Expectancy Effects and the
Human Performance Research." Washington, DC: National Academy Press, 1988.
(p. :3)
11 National Research Council, op. cit., pp. 199-200.
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
falsifiable -- that is, that there is no definitive defense against the charge
of "general sloppiness". Without detailed criteria for what constitutes
acceptable research, they say, criticism of an experiment cannot be anticipated
or refuted. In their view, it is an example of the unfairly high standards that
are set: for parapsychology research, in which no amount of care is ever
"enough."
The critics present at the workshop tended to agree in principle with the
unfairness of "non-falsifiability," but they believe that the "dirty test tube"
charge is nevertheless valid because the success of the experiments rests on
whether a small but significant departure from some statistical norm or observed
norm has in fact been seen, making it essential to eliminate extraneous factors.
Parapsychologists object that the charge itself is irrelevant -- that if the
alleged flaws or "dirt" have no demonstrable connection with the experimental
outcome, it is capricious to argue that the experiment is invalid.
The litany of charge and countercharge could be detailed much further.
Suffice it to say that the controversy surrounding parapsychology is nowhere
more heated and complex than in the area of experimental methodologies.
REPLICABILITY
For critics of parapsychology, a key argument against the existence of psi
phenomena is the difficulty of replicating (repeating) successful experiments --
a difficulty that also plagues many accepted areas of the social and behavioral
sciences. If the same effect could be produced many times by different
experimenters using different equipment but the same procedures, then the
likelihood that an extraneous variable is producing the effect would be much
lower and the validity level of the anomalous results would be higher. However,
positive results are in most cases maddeningly (to parapsychologists) difficult
to replicate, even with the same operator. (Because the effects seem to be
highly operator-specific, results are even more difficult to replicate with
different operators or groups of operators.)
Parapsychologists attribute this replicability problem to the elusiveness
and weakness of the psi "signal," and to its vulnerability to subtle subjective
factors, most of them unknown. Critics contend that the near non-replicability
merely demonstrates the effects of randomness combined with multiple flaws in
experiment design and conduct.
The parapsychologists are well aware that experimental flaws, both actual
and alleged, become less important as a phenomenon becomes more replicable.
Therefore, in recent years they have placed great emphasis on replicability. As
one noted in his presentation, "The final criterion of the success or failure of
my research program is the extent to which I'm able to develop procedures that I
can articulate with sufficient precision that other people are able to obtain
similar results."
In the case of ESP-oriented experiments (such as remote perception or
"ganzfeld" psi experiments), the number of these more standardized experiments
is still not large. There, parapsychologists have relied upon what is termed
"meta-analysis" of many studies, taken as a group, to give statistical
confirmation for a pattern of positive results. The rationale is that "an
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
experiment or an effect may be considered replicated if a series of replication
attempts provides statistically significant evidence for the original effect
when analyzed as a series."12 It is also'contended that this approach allows
the analyst to assess methodological flaws to determine their effect empirically
across the data base.
'Using the meta-analytic approach, parapsychologists have calculated an
overall "success" rate, in achieving statistically significant effects (p .05) are too often interpreted erroneously by critics
as insignificant results. An example w s given to demonstrate that sample size
(n) has a substantial impact on the statistical outcome; if n is small there is
very little chance for a "successful" outcome, even if relatively high actual
probabilities of success based on ESP are assumed. Meta-analysis of all the
forced-choice precognition experiments conducted from the 1930s on showed that
Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
 Approved For Release 2003/09/09 : CIA-RDP96-00792R000100130001-2
those with significant results had, on average, two and one-half times larger
sample sizes than those with nonsignificant results. Thus, large values of n
impart considerably greater "power" to the analysis. Power is, essentially, the
probability that the experiment is going to succeed. (This concept of power is
now being taught at the graduate level in statistics.) Table 2 illustrates how
power increases with increasing n.
At least one of the critics present argued that there is no correlation
between sample size and the probability of obtaining a significant outcome, but
Utts refuted that claim. It was agreed that the spread of data -- for example,
one or two "outliers" (data points far outside the general cluster) -- can
greatly weaken the correlation.
The statistician recommended that parapsychologists stop focusing on the
arbitrary, "sacrosanct" p