(UNTITLED)

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP91-00452R000100060030-9
• Release Decision: RIFPUB
• Original Classification: K
• Document Page Count: 4
• Document Creation Date: December 20, 2016
• Document Release Date: May 16, 2006
• Sequence Number: 30
• Publication Date: September 9, 1999
• Content Type: REPORT

Approved For Release 2007/10/23: CIA-RDP91-00452R000100060030-9
10 August 1977
The ORD Career Service Panel has asked MAG to look into
the question of program evaluation. During the past several
months we have looked into the matter and read many papers
and a book on the matter. We have not been able to produce
a formula for program evaluation. We have explored the
question of why it seems to be such a difficult problem.
One of the observations we had is that most R&D evaluation
is done by an analysis of return on investment (ROI). In
the intelligence community no value has been assigned to
intelligence reports or international estimates. That in
turn makes it difficult to use any ROI method.
One key reason we have come up with is that most of
our R&D work is removed once or more from the final consumer.
Many of our products go from us to an engineering organization
and we have a smaller control over the final customer use of
that product. A simplified process is exhibited in Figure
1. We find that we have to justify a project before the
process begins and then despite whatever point ORD's role
ends we have to explain or evaluate in terms of the ultimate
product produced whether by us or another organization.
This is a critical point especially when we realize when one
of the key criteria in determining project success is whether
the project results in ultimate adoption of the product for
consumer use.
Approved For Release 2007/10/23: CIA-RDP91-00452R000100060030-9
 Approved For Release 2007/10/23: CIA-RDP91-00452R000100060030-9
How do we increase the probability of ultimate consumer
How do we show a customer the possibility of a product?
How do we convince the customer that the technology
proposed makes sense?
How do we find what the customer's problems are?
What his needs are?
CONTACT
Contact has implications. It means:
A. Sensitivity to major consumer problems-they
will expect us to grasp their priorities; to know what
their most difficult jobs are.
B. Conscienceness of multi-disciplinary application
(matching) of technological possiblity to their problems--
they will expect us to be above internal ORD parochialism.
C. Delivery of the most cost effective solution
(product) to apply against the problem:
(1) It may not be the best technical solution.
(i) Better technology may take more-time
(ii) Better technology may cost more
(2) It may mean scrapping the RFD project
(i) To pursue .a different technological
path to the solution
(ii) To wait concerning this problem to
attack another problem because:
.(a) The cost may be high
(b) The technology may not be here yet
Approved For Release 2007/10/23: CIA-RDP91-00452R000100060030-9
 Approved For Release 2007/10/23: CIA-RDP91-00452R000100060030-9
Contact has advantages. It means:
A. A better chance of understanding the consumers
problem.
B. A better chance of applying the right technology.
C. A better chance of maintaining the customer
interest all the way.
D. A better chance of selling the product.
SUGGESTIONS AND RECOMMENDATIONS
1. Division staff chiefs should undertake internal
programs to assure systematic and increased contact. This
should be put into their respective LOI's by D/ORD and they
in turn should put such a contact requirement into the LOI's
of individual project officers.
2. The Management Staff should undertake a program of
basic project officer education. A requirement of each
project officer should be to take the course "Project
Officer in the Contract Cycle". Perhaps ORD could work out
an arrangement with OTR to offer this course on a part-time
basis in Ames Building. Then MS should make other information
available which will explain the process, the paper requirements
such as the notebook, the contract inspection reports, etc..
3. PAS should be a clearing house for interdisciplinary
projects or projects requiring a different discipline than
the main contact division.
4. We strongly recommend that the Director of Research
and Development implement a plan which will permit divisions
to cut projects off in midstream without penalty, i.e., the
division should have the capability to apply the remaining
Approved For Release 2007/10/23: CIA-RDP91-00452R000100060030-9
 Approved For Release 2007/10/23: CIA-RDP91-00452R000100060030-9
vow,
fund to another technological approach of the same problem
or to the next priority unfunded problem on that division's
list. We feel this is one the key reasons why certain
.projects are not cut off earlier. This would require that
the Contract Staff would have to write the type of contract
which would enable such cutoffs.
Approved For Release 2007/10/23: CIA-RDP91-00452R000100060030-9