PROJECT EVALUATION

• Document Type: CREST
• Collection: STARGATE
• Document Number (FOIA) /ESDN (CREST): CIA-RDP96-00791R000200190041-2
• Release Decision: RIPPUB
• Original Classification: K
• Document Page Count: 2
• Document Creation Date: November 4, 2016
• Document Release Date: May 19, 2000
• Sequence Number: 41
• Publication Date: June 14, 1995
• Content Type: MEMO

SAIC-[phone]-Created: Wednesday, June 14,199522:38- Page 1 of 2 ---------------
------------- Appr4wed-Release-?{I0-3f@9M 6 rt:fA=FZDP56-'0D7911R660200190041-2
The Cognitive Sciences Laboratory
330 Cowper Street, Suite 200, Palo Alto, CA 94301
Voice: 415.327.2007 - Fax [phone]
~.
An Employee-Owned Company e-mail: [e-mail]
pQemorandum
SG1I
SG1I
Date: 14 June 1995
To:
From: Edwin C. May, Ph.D.
Re: Project Evaluation
o
1 axe
evaluation. I am asked to find "the 10 primary studies or reports developed under the
subject program ...that.make the best case for establishing the validity of the paranormal
phenomena known as Remote Viewing." I am happy to do this as soon as possible.
I have a concern, however, about a problem that may be raised that could call any results
into question.
Dr. Robert Rosenthal in the Psychology Department at Harvard and others have defined a
statistical circumstance known as the "file drawer" problem. It is standard practice in
behavioral sciences (including the study of anomalous cognition-remote viewing) to use
a p=0.05 criteria as a measure of statistical significance. That is, statistical significance is
claimed when, given that the null hypothesis (i.e. no RV in this case) is TRUE, there is a
5% chance that a repeated experiment of the same number of trials would yield a
statistical deviation as large as in the original, or larger. We use that criterion in our
laboratory.
The file draw problem is this: Under the null hypothesis 5% of the studies will be
statistically significant. If only the significant studies are published (or in our case given
to AIR), it is problematical to assess the validity of the research, because it might be that
for every study published there are 19 others (in the file drawer) that did not reach
statistical significance and were not published. If this were the case, there would be no
evidence for an anomaly. Rosenthal and others provide ways of assessing the potential
magnitude of this problem.
It might be argued that if I am allowed to pick the best 10 studies, and if it turns out that
there were 200 others that were not "good," one could be seriously mislead. This is a
worst case scenario; however, critics would correctly argue that there is an undetermined
file drawer problem with this new approach, and if it is not addressed, the validity of the
evaluation can be questioned. I can assure you that Ray Hyman will raise this point.
Might I suggest an alternative that address the concerns of the time constraint, yet avoids
the file drawer problem. In 1989, SRI published (now downgraded to unclassified) a
meta-analysis of all the SRI work from 1973 through 1989. This study was part of the
me new tasking with regard to providing support for the project
P. d t
Page 1
Approved For Release 2003/09/16 : CIA-RDP96-00791 R000200190041-2
 SG1I
ApprovFax I ransmlttalRCover'Sneet041-2
To: - ORD
From: Edwin C. May, Ph.D., SAIC
Fax Phone Number: [phone]
Date: Wed, Jun 14, 1995 - 22:38
Transmitting (3) pages, Including cover sheet.
If there Is difficulty with this transmission, please call: [phone]
Note:
Approved For Release 2003/09/16 : CIA-RDP96-00791 R000200190041-2