ESSAY ON KEY JUDGMENTS BY DR. KLAUS KNORR

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP98S00099R000501010028-3
• Release Decision: RIPPUB
• Original Classification: C
• Document Page Count: 6
• Document Creation Date: December 23, 2016
• Document Release Date: June 1, 2012
• Sequence Number: 28
• Publication Date: June 29, 1979
• Content Type: MEMO

Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 : CIA-RDP98S00099R000501010028-3
CONFJ.BE9TIAL
29 June 1979
Memorandum for Dr. Bowie
Subject: Essay on Key Judgments byl 25X1
has written an interesting Essay on
formulating the Key Judgments portion of an Intelligence
Estimate a copy of which is attached. You may wish to
consider the points the Essay raises.
Attachment:
As stated
2"
25X1
rt
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01_CIA-RDP98S00099R000501010028-3
 JI
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 CIA-RDP98S00099R000501010028-3
CONFIDENTIAL
18 June 1979
(revised 27 June)
Key Judgments:
Concept and Practice
(NIEs and IIMs)
1. The producer of an estimate is apt to regard the
main paper as the analytical centerpiece of his effort.
From the viewpoint of the policymaking consumer, who often
does not have the time or inclination to read the main paper,
it is the Key Judgments that are the most important part of
an estimate. It will only be interesting Key Judgments that
may induce him to read part or all of the main paper. Because
to stimulate intelligence consumption and serve the consumer
is the purpose of estimative products, the Key Judgments
should be designed with great care.
2. The literal use of Key Judgments was started in 1969,
and their importance apparently reaffirmed in 1973. I have
not so far been able to establish whether these initiatives
were accompanied by written or elaborate oral directives.
3. In any case, present practice exhibits a large range
of quality, and of conceptions of what the Key Judgments should
be about.
25X1
25X1
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 CIA-RDP98S00099R000501010028-3 ,.
 Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 : CIA-RDP98S00099R000501010028-3
CONFIDENTIAL
4. The relevant criteria of quality are hardly controversial
and can be simply stated. Key Judgments should be (a) clearly
articulated, (b) coherent, that is, logically and substantively
integrated and (c) reflective of the analysis in the main paper.
Although these criteria are unobjectionable, Key Judgments vary
inordinately in meeting them. While some variation in quality
is inevitable and acceptable, the average should be raised.
This is especially true regarding the extent to which Key
Judgments and main paper are appreciably at variance.1
5. Even more striking is the conceptual disagreement
on the purpose of Key Judgments. In my experience, many, if
not most authors simply write Key Judgments as if they were
meant to be a summary of the main paper, often with an apparently
indifferent selectivity on the points in the main paper that
are to be included.
6. There are very lengthy NIEs (for example, 11-14)
that feature a Preface, Key Judgments, Summary, Main Paper,
and Annexes. This practice, incidentally, obviously regards
Key Judgments and Summary to be different things. However,
most NIEs and IIMs are much shorter and, therefore, do not
require a summary in addition to the Key Judgments.
1 In some cases, the divergence results from the fact that
main paper and Key Judgments are written by different
people.
CONFIDENTIAL
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 CIA-RDP98S00099R000501010028-3
 L U
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 CIA-RDP98S00099R000501010028-3
` UUNFiUM4T1AL --
7. Key Judgments do, of course, summarize parts of the
main paper. But they should do so in a sharply selective
fashion. Key Judgments should present, and present exclusively,
estimative answers to the Key Questions that are presumably of
interest to the policymaker.2 Estimative answers are judgmental
statements about things we do not know, whether they are
predictive or related to presence or past.
8. Present practice in the formulation of Key Judgments
reflects another conceptual conflict that affects their
selectivity. At one extreme, Key Judgments present only the
best estimative guess or guesses. At the other extreme, Key
Judgments recognize different assumptions and contingencies
even if one estimate is favored, more or less, over another.
Practice varies greatly on this perspective.
9. Both concepts can be seriously defended. The more
"balanced" version is safer. It recognizes that estimating
is a hazardous business. It alerts the consumer to the
possibility of events held to be less probable than others
(which is especially important if these events would have
serious consequences to US interests.) It also informs the
consumer on how to think about the estimative problem rather
than giving him only the result of the author's thinking.
2 In general, these will be the key questions, formulated in
the Concept Paper, which prompted the need for an estimate
to begin with. Subsequent analysis, however, may engender
some revision, or additions, to these questions.
3
CONFIDENTIAL
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01_ CIA-RDP98S00099R000501010028-3 4
 Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 : CIA-RDP98S00099R000501010028-3
CONFIDENTIAL
On the other hand, the balanced version has the grave disadvantage
of the diminished usefulness of a hedging estimate.
The alternative version has the great virtue of being
truly estimative in the sense of giving the consumer forth-
rightly the best judgment that can be produced. Choosing
among assumptions, contingencies and judgments is the very
essence of the estimative process.
10. To be dogmatic about solving this conceptual conflict
seems unwise in view of intrinsic differences in estimative
difficulties. Yet it seems also unwise to leave this conceptual
conflict wholly unresolved. Three desiderata greatly commend
themselves. First, if the author is truly unable (as distinct
from being unwilling) to exercise judgment about various
possibilities, he should clearly say so, and present his
alternatives. Similarly, alternatives should be presented in
the Key Judgments if different estimating units arrive at
conflicting estimates about important matters. Second, forth-
right selective judgment should be regarded as the ideal outcome
of the estimative effort. Third, uncertainties about the
preferred estimate are better expressed - in the Key Judgments -
in terms of degrees of confidence with which they are offered
than by giving different estimative judgments.
11. In line with the estimative ideal, Key Judgments
should be as forthright as possible in the expression of
probabilities attached to an estimate. Whenever this is
the preferred estimate, to say that an event is virtually
certain to happen is better than to say that it is probable, etc.
4
CONFIDENTIAL
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01_CIA-RDP98S00099R000501010028-3
 1
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 CIA-RDP98S00099R000501010028-3
CONFIDENTIAL
12. If Key Judgments only give the policymaker key
estimate answers to the key estimative questions, then these
estimates deserve to be supported by a brief, concise,
explanatory statement, including estimative assumptions.
The brevity of the supportive statement depends on the number
of estimative answers (or ticks) that are presented--which
can involve very few or a great many.
13. Finally, it would seem useful, especially when the
estimative answers are many and supportive statements must be
very economical, that each answer show in parentheses the
supporting paragraphs in the main paper and the annexes. If
he feels challenged to do so, the policy consumer can then
quickly turn to the relevant portions of the main paper and
the annexes.
14. It seems that the use and usefulness of NIEs and
IIMs might well increase if the proposed practice became
reasonably uniform.
* I wish to thank my colleagues on the Senior Review Panel
for helpful comments.
CONFIDENTIAL
Declassified in Part - Sanitized Copy Approved for Release 2012/06/01 _ CIA-RDP98S00099R000501010028-3