Study Suggestions

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP84B00506R000100050014-5
• Release Decision: RIFPUB
• Original Classification: S
• Document Page Count: 5
• Document Creation Date: December 9, 2016
• Document Release Date: August 12, 2000
• Sequence Number: 14
• Publication Date: August 9, 1972
• Content Type: MF

Approved For Relee 2001/08/14: CIA-RDP84B00506R0W-00050014-5
NATIONAL SECURITY COUNCIL
WASHINGTON, D.C. 20506
SECRET August 9, 1972
MEMORANDUM FOR RAY S. CLINE!,
FROM: A. W. MARSHALL
SUBJECT: Study Suggestions
I have reviewed your three suggested studies (Chile, Yugoslavia, NSSM 108)
and have discussed them with interested members of the NSC staff. Based
upon the information now before me, I would prefer to see review efforts
conducted towards Yugoslavia., Chile and NSSM 108 in that order. _
Ranking the three studies is a. difficult task because they all have potential.
However, I do not favor starting with NSSM 108. Even if restricted to
reviewing the management process, it would still be a major undertaking.
Activities surrounding its development took nearly a year's time. To
assess the management process would require investigating not only the
link between policymakers and the community, but also the managerial
efforts within the community to bring together analysts on a number of
nations to develop a comprehensive picture of Latin America.
Chile and Yugoslavia seem more manageable and relevant to present needs.
Chile and Allende obviously are important influences on U. S. policies to-
wards Latin. America; analysis better attuned to policymakers' needs would
be helpful. Should the NSCIC Working Group chose this study, I recommend
the time frame be moved back to January 1970 (or thereabouts) in order to
evaluate the analytical output surrounding Allende's rise to power. The
Yugoslavia study seems marginally more important than Chile because of
the distinct possibility of political turmoil there, in the near future, which
could be exploited by other powers.
Without going into detail on any one proposal, I have some general sugges-
tions concerning the objectives, methodologies and the analytical or
management issues to be addressed that may be of some use to ycu:
SECRET
*NSC Declassification/Release Instructions on File*
Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5
 Approved For Rele 2001/08/14: CIA-RDP84B00506R0@00050014-5
SECRET
OBJECTIVES
The objectives of a study should be focused on presenting the NSCIC with
lessons learned from reviewing the topic and developing specific guidance
for the NSCIC to consider passing to the community. Detailed historical
records will normally have to be compiled to carry out these studies, but
this effort should not be allowed to overwhelm the study groups. Where
necessary, sampling of products could be used. Also, as one of your
studies suggests, comparing two years separated by four to five years
might reduce the workload and focus on the question; --Have we improved
over the last several years? What are the trends in key indicators of pro-
duct quality/ or analysis staff quality, etc.?
What is important is (a) the evaluation of the intelligence process and
products during the period, and (b) the presentation of a rigorously developed
set of conclusions, general and specific recommendations, and appraisals of
the likely impact of these recommendations on the quality of production and
the internal functioning of the community.
All three studies have a distinct management orientation in that they propose
to examine the interplay among output, guidance received and resources
available. Qualitative evaluation of a particular product apparently would
center on asking whether it clearly made its points and whether these points
were relevant to the needs of policymakers at the time. Little work will be
performed directly to evaluate the intrinsic quality of the product - i. e. ,
whether it represented an exhaustive perceptive analysis -- except as this
falls out from the study. This approach does not disturb me. I find it
perfectly appropriate to target the study efforts on the management issues
as well as on analytical ones. However, for some selected products an
assessment of the quality of the intelligence analysis, especially as regards
political and governmental behavior, would be of interest. What is impor-
tant is to insure that everyone participating in the study understands the
approach.
METHODS
I envision that the role of the working level participants in a study would go
beyond that of recognizing problems and making general recommendations.
SECRET
Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5
 SECRETved For Rele 2001/08/14: CIA-RDP84B00506R0QQ,100050014-5
Since these individuals will be the ones most familiar with the intelligence
effort surrounding the topic, and will have spent the greatest time thinking
about it, they should have some valuable thoughts about both the very broad
changes needed and the specific adjustments which could be made in the
present system of managing the intelligence process. Fully exploiting
their insights is essential to drawing maximum value from the work which
goes into studies. The working group can still be the final judge of the
weight of the case the study group makes.
I suggest the NSCIC Working Group select one of the topics to begin immedi-
ate work on. You should guide the study and I would be delighted to help you
in any way I can. The remaining two could be postponed until later in the
year when present studies will be nearing completion. At that time these
topics could be considered along with any others that have been submitted.
Past studies have experimented with panels, individual analysis and group
reports, and some of these methods may not be appropriate to the proposed
three topics. For example, a. panel may be of little value in evaluating a
management process if its members are unfamiliar with the community;
however, an appropriately selected panel could be very helpful in judging
whether a product presented a, clear message to the policymaker. If'the
management process is to be a key area, a working group should also
consider using management oriented individuals (such as OMB personnel)
who have no vested interests.
MANAGEMENT AND ANALYTICAL ISSUES
While the issues outlined in the scope of each of your proposals give a broad
view of each study's direction, a more detailed list of issues should be for-
mulated before a study actually begins. The following are some suggestions.
Study Execution
Prior to beginning the compilation of information I think a study group
should reach some tentative ideas about the scope of their effort. Any
pre-study theories should be discussed to decide how to test for them.
The time period and the actual intelligence products to be covered (e. g. ,
NIE's, daily briefings, etc.) should be laid out with special attention to
what is being ignored and why. A general format for the report should
be discussed.
SECRET
Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5
 Approved For ReI Fe 2001/08/14: CIA-RDP84B00506R0f 00050014-5
SECRET
Management Issues
An excellent first step to understanding the management of intelligence
during a period would be to study the linkage along which information
flowed within the community and between the community and the policy-
makers. Both the informal and formal aspects of these links could be
examined. Attention could be given to the flow of information, feedback
and guidance, and to how the flow was affected by the perceptions of dif-
ferent participants (either agencies or individuals), by time constraints,
and by procedures. However, I recognize that to develop such a detailed
perspective is extremely difficult when events occurred months in the past,
people have changed, etc.
A particularly interesting issue would be to identify the implicit standards
which were set for intelligence output (e. g., number of reports, response
time to questions, etc.) and how the community measured up to those
criteria. Furthermore, who set the criteria and why?
Consideration should be given to how the output of the community could
have been improved through better management. In the short term this
includes examining the role of middle managers in motivating analysts, in
communicating needs to analysts, and in matching the analysts' effort with
the needs of the consumers. The role of higher level intelligence managers
should-be subjected to the same review. (In the long term the working group
may become interested in such questions as how training influenced the roles
of both analyst and manager and how the system of personnel management
and motivation affect their behavior.)
Analytical Issues
The "quality's of analysis is a frequently discussed topic. In addressing it
an effort must be made to define what we mean by quality -- is it a rigorous
analysis, one that combines all viewpoints, one that reads well, etc. The
criteria of quality may vary markedly between individuals and agencies; in
fact the lack of consensus could well be a major part of the problem.
Examining the analytical output requires an appreciation for the raw data
being gathered aad the validity being implicitly given to it (simply because
we collect it covertly does not mean it is accurate, regardless of the
SECRET
Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5
 Approved For RelQpe 2001/08/14: CIA-RDP84B00506R04g:100050014-5
SECRET
collection device). The modification in the raw data as it is processed
may merit review in order to see how it influences the analyst. The
analyst's' work is then perhaps the key to the quality of the product. How
does he assemble data? What methods does he use to analyze it -- does
he just assemble data and make obvious inferences; does he test different
analytical frameworks against the data; what data does he dismiss? In
looking at the final written product one needs to consider whether major
points are clear, whether they respond to consumer needs, and how the
analysts try to communicate the appropriate level of uncertainty.
SECRET
Approved For Release 2001/08/14: CIA-RDP84B00506R000100050014-5