REVIEW OF PROJECT PROCEDURES

• Document Type: CREST
• Collection: STARGATE
• Document Number (FOIA) /ESDN (CREST): CIA-RDP96-00789R002900290001-9
• Release Decision: RIPPUB
• Original Classification: S
• Document Page Count: 13
• Document Creation Date: November 4, 2016
• Document Release Date: May 22, 2000
• Sequence Number: 1
• Publication Date: April 5, 1991
• Content Type: REGULATION

Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
STAR GATE
DEFENSE
INTELLIGENCE
AGENCY
DT-S-1030-SL
REVIEW OF PROJECT PROCEDUHES (11)
5 APRIL 1991
STAR GATE
?NOFORN
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
REVIEW OF PROJECT PROCEDURES
5 April 1991
This is a Department of Defense Intelligence Document prepared by the
Technology Assessment and Support Office, Directorate for Scientific
and Technical Intelligence, Defense Intelligence Agency.
Technology Assessment and Support Office (DT-S)
REPRODUCTION REQUIRES APPROVAL OF
ORIGINATOR OR HIGHER DOD AUTHORITY.
FURTHER DISSEMINATION ONLY AS DIRECTED
BY DT OR HIGHER DOD AUTHORITY.
CLASSIFIED 6Y: DIA/DT
DECLASSIFY ON: OADR
CCC:14OF0169 LIMBIS
Approved For RelealkT*STERDP96-00789R002900290001-9
 Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
TABLE OF CONTENTS
Ea
I.
PURPOSE ...........................................
1
Il.
SCOPE ......................................... ...
1
III.
SUMMARY ..........................................
1
IV.
BACKGROUND .......................................
1
V.
SHORTFALLS/REVISIONS ..............................
2
APPENDIX
A. EVALUATION FLOW CHART
B. REPORT FORMAT
C. SUMMARY OF INFORMATION FORMAT
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
I. (U) PURPOSE:
(U) Purpose of this report is to review initial. STAR GATE
procedures and to provide recommendations for revisions based on
recent experience including DESERT STORM tasking.
II. (U) SCOPE:
(S/NF) This report examines data development, evaluation and
assessment procedures for data generated by STAR GATE sources.
III. (U) SUMMARY:
A. (S/NF) Lessons learned from recent DESERT STORM tasking have
led to several immediate changes in STAR GATE procedures as discussed
in Section IV. Some recommendations for future revisions are also
identified.
B. (S/NF) Evaluations of future STAR GATE reporting will be
prepared in accordance with the criteria and provisions of DIAM 58-13,
Subject: The Department of Defense HUMINT Management System, dated 2B
March 1988.
IV. (U) BACKGROUND:
A. (S/NF) In December 1990 and January 1991, initial efforts
were made to provide an effective and viable system to evaluate STAR
GATE reporting. These procedures are illustrated in Figure 1. The
INCLASSIFIED
-1-
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002/Q5/~7_,: CIA-RDP96-00789ROO2900290001-9
initial proposals resulted in the implementation of procedures that
attempted to assess both the utility and the value of STAR GATE
reporting.
B. (S/NF) During the course of an off-site conference held on 7
and 8 March 1991 at Fort Dietrich, senior analysts from DT-5A, DT-38,
and DT-S, identified shortfalls in current evaluation procedures and
prescribed revisions that would allow for the implementation of an
objective and work--.able system. The senior analysts believed that
assessments relating to the accurate of STAR GATE reporting could
best be addressed by applying applicable scientific methodologies to
STAR GATE material. Procedures already exist and can be implemented
available project ADP methods. Specific accuracy assessments would be
determined by intelligence analysts in conjunction with appropriate
outside scientific review. The same analysts were of the opinion that
the intelligence value or utility of STAR GATE reporting could best be
determined by select intelligence analysts directly involved with
specific task areas.
C. (U) What follows are the results of a comprehensive group
effort conducted to identify procedural shortfalls and recommend the
implementation of the following revisions.
V. (S/NF/SG/LIMDIS) SHORTFALLS AND REVISIONS:
- SHORTFALL: Need to identify action officers with whom DT-18
can deal directly to expedite the evaluation process and insure that
such evaluations are accomplished in the most successful manner.
NOFORN/LIMDIS
STAR GATE
Approved For Release 2002/05/17 CIA-RDP96-00789R002900290001-9
 Approved For Release 20 RDP96-00789R002900290001-9
analytical evaluation process. Direct involvement with these (and
other) scientific analysts should help in better task definition and
in identifying ways for improving potential utility.
- SHORTFALL: Procedures to date appear burdensome and need
streamlining.
- REVISION: Procedures developed by the R&D community should be
used for assessing the accuracy of project reporting. The
intelligence value of STAR GATE reporting must be determined by
intelligence community analysts working from an all source
intelligence perspective and environment. This should be based on an
agreed-upon definition of utility and what target data would be
required to satisfy the utility criteria.
- SHORTFALL: Need to assess whether our quantifiable answer
categories would be expanded to reflect the range of uncertainty when
one is evaluating a DT-S report.
- REVISION: Specificity in the judicious selection of mission
requirements will go a long way towards streamlining the process.
Target selection must be based on targets against which STAR GATE has
NOFORN/LIMDIS
STAR GATE
-3-
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
a favorable track record. To this extent, limits to STAR GATE data
(or project type), need to be provided in advance to the tasking
elements.
- SHORTFALL: Need to re-accomplish certain original tasking,
updating it as necessary to the current situation.
- REVISION: Tasking should be levied within specific time
windows and not be left open-ended. Requirement originators should be
reminded of their responsibility to update requirements and changes of
collection emphasis.
SHORTFALL: Senior analysts need to insure that the
intelligence evaluations (ie, utility assessments) of project
reporting are timely, objective, substantive and prepared in
consonance with established intelligence community procedures.
- REVISION: STAR GATE reporting needs to be evaluated in
consonance with the DIA HUMINT evaluation scheme paraphrased
substantially as follows:
RATING VALUE
Of Major Significance Having Impact on National
Policy and Decision-Making
Of High Value Best Effort - First Report
in a Field of Endeavor
Of Value Supplements - Updates -
Confirms Known Data
OFORN/LIMDIS
STAR GATE
-4-
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release -RDP96-00789ROO2900290001-9
Of Low Value Poor Quality - Not Timely
Of No Value Erroneous - Misleading
There is another consideration. Proper assessment procedures for
either accuracy or utility require that criteria for evaluation be
specific in advance. Consequently, it should be appropriate for a
candidate user (or task originator) to clearly define what type of
data would have potential utility, and how it would be used if
provided and how it would be evaluated. This type of data could
include specific essential. elements of information (EEl) and required
timing. A troublesome problem may be encountered if the specific EEI
(if posed as a question) provides too much background information for
the source that could adversely influence session results. To avoid
such difficulty, it may be necessary to conduct the session in several
phases, time permitting. This issue is not easy to resolve and
remains as a future action item. (A basic review of this overall
issue can be found in DT-S-1010-S, 13 Dec 1990, Evaluation Methods).
- SHORTFALL: Need to reconsider acceptance of tasking based an
availability and timeliness of known ground truth.
- REVISION: The STAR GATE collection management system must be
selective in its acceptance of requirements for which there is a
likelihood of learning ground truth within a reasonable amount of
time. A procedure needs to be established that allows for rapid
follow-up for task refinement.
OFORN/LIMDIS
STAR GATE
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002/05/17: CIA-RDP96-00789ROO2900290001-9
SHORTFALL: Multiple source reporting in a combined summary
format is confusing precludes affixing responsibility to the
reporting.
- REVISION: The principle of "one source one report" must be
strictly adhered to so that reporting may be attributable to a
specific source. ConIsecquently, data from any source will be reported
separately or in easily identifiable and separated sections of an
overall summary report.
SHORTFALL: Current STAR GATE reporting is inconsistent with
other established HUMINT reporting formats.
- REVISION: Reports need to be prepared in accordance with the
criteria used in the preparation of Intelligence Information Reports
(IIR's). Each report will bear a numerical source designator that
will protect the identity of the source. An example of the reporting
format is at Figure 2. A Summary of Information (SOI) will be
prepared by the source. It will reflect a project number, source
number, session number, and date of report. The SOI will be attached
as an addendum to the formal report. An example of an SOI is at
Figure 3.
- SHORTFALL: STAR GATE predictions are rarely accurate.
- REVISION: Predictive reporting is indeed the project's
weakest feature. The results of searches are conducted at a slightly
higher rate of success. Historically, the project's track record
I~NOFORN/LIMDIS
STAR GATE
-6-
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
reveals a relatively high rate of success when mission tasking is
targeted against fixed, static targets located in past or current time
windows. Consequently, project tasking should be prioritized
according to estimated success probability.
- SHORTFALL: STAR GATE project reporting should not stand alone.
- REVISION: Senior analysts need to insure that STAR GATE is
an integral part of the all-source collection management system, a key
tip-off player acting in concert with other intelligence disciplines.
SHORTFALL: Highlights of singularly outstanding reporting
should be recorded and made readily retrievable.
- REVISION: A "KKudo File" will be established to facilitate the
retrieval of examples of high quality reporting. Factors that may
have contributed to these results will be identified whenever
possible.
OFORN/LIMOIS
STAR' GATE
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
9 *AFL
DATA DEVELOPMENT. EVALUATION ANI7 ASSF-qSMFNT
OUTPUT
EV LUq
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002f0A/17 :,P~tRqP96-00789R002900290001-9
I~?'rii..1~.1F:i:{:::..!.. I'~It.JI''Ili{G:::Fa,~ ;::it..lt..i!'a::f:::: I',!t..)t ::;:::. ??t ::
1::f:a:L(::)1`1;: D,'?1"I"f:;: (:)F:: 1=;f::::I:'.:11=;"'
..,t.... ? :::i..!..i?~!.:~..!.. M !::'!::::::3':>'I. (:7!'.I 1::::! .IC:)
C) 1\1 N Li l"1B !::::I
ric:::L (..tcI~.Ar-1 y J. n ?f (::) r7cy't (: i Y~~'tart:i c)(::I :i 1-1 cir.t(11in~.:ir y 04: 0 r?('()::a?t:L (::)r1 (~iil.1 )
!::)I??(::)Y':1. (..:!c:1 L,y Wi(_1t..tr-(::('iu
`i. ( i') EV01L_Uiil"1'.0N 1i::1 1t?.(::)14'? 11 t..t(rir-i.c::1. c:I7.gria-tt?.(:7r" I A W Di(1'1 5 a
Itl0 t F't(::1. s b'I ?1:0 f (::)r?-o:i clr) r?ra t.:L i)Ga:!. ~T:, %.:L...A T F :I: 1.) .- D:1: MC T
ON. MADF~;
APPENDIX ~ Figure 2
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9
 Approved For Release 2002/05/17; CIA-RDP96-00789R002900290001-9
EXAMPLE
..I..i..{L: (I:... ..I..f::} ..I..IIL::: I::il::
.._.._.........._ .................. ........_........................................ _...... __.... _.......................................... ....... _......................................
: ' F 1::. ', Ea:? BY 1 1 11 i:iC11..11 C::E:: :C I`d f:al..Ii::: is OF f
...................,..._
................................... ................ ......_.............._......................................................... ....... ............. ........ _............................ _...... ..._....._................
.................. . _....................... ............. _ . . ................... _ ...........
f:3Ca :L
.................
APPENDIX ClFigure 3
Approved For Release 2002/05/17 : CIA-RDP96-00789R002900290001-9