HOW THREE ESTIMATES WENT WRONG

• Document Type: FOIA
• Collection: FOIA Collection
• Document Number (FOIA) /ESDN (CREST): 06887351
• Release Decision: RIPPUB
• Original Classification: U
• Document Page Count: 12
• Document Creation Date: July 31, 2024
• Document Release Date: June 25, 2024
• Case Number: F-2014-01314

Approved for Release: 2024/06/12 C06887351
_SEMI
Some lessons from case
histories of misfudgment
in the NIEs.
HOW THREE ESTIMATES WENT WRONG
Willard C. Matthias
The procedure by which National Intelligence Estimates are pre-
pared is designed to ensure that those responsible for policy decisions
receive an agreed intelligence judgment ( or a carefully delineated dis-
agreement) based upon the best information and most thorough re-
view possible. But the procedures only provide the framework within
which people function, and the estimates are only as good as they are
made by those who operate the machinery.
The Estimative Process
In this machinery the Board of National Estimates plays the most
important managerial and intellectual role. The draft of the estimate
is prepared by the Estimates Staff under the Board's direction; it
is discussed with representatives of the USIB agencies and CIA com-
ponents under the chairmanship of a Board member; and it is pre-
� sented by this Board member to the USIB. The Staff member who
prepares the initial draft also plays an important part: he is more
familiar with the information than most Board members are, and if
he is a skillful writer and convincing defender of his views he puts
an ineradicable stamp on the estimate. The Board chairman�par-
ticularly if he has had a long familiarity with the subject and is
temperamentally inclined to play a leading role�will likewise put
a strong stamp on it.
Nevertheless, it is the Board as a corporate entity which bears re-
sponsibility to the DCI for the form and substance of the estimates.
It is his Board, an instrument to formulate his views and to take account
for him of the information and judgments applicable to the subject
in question. While he personally may rely upon some Board mem-
bers more than others, in his official capacity he must have a Board
sufficiently competent and balanced in composition and experience
that he can be confident that all significant aspects of the subject
have been weighed judiciously.
27
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
Three Estimates
One cannot generalize about the method or technique Board mem-
bers use in arriving at estimative judgments. Each estimative situa-
tion is unique, and the estimator must lean upon a variei y of sup-
ports: the amount and persuasiveness of the evidence; the methods of
analysis used by the contributors; the judgment of others in whom he
has special confidence, whether because of their study, experience, or
sharpness of mind; his own background in the subject; and, for want
of a better word, his hunches.
By dictionary definition a hunch is a feeling or suspicion not based
upon evidence but upon premonition. I do not believe in premoni-
tions, but I admit of the hunch in some sense. It must be compounded
of something�a sense of the logic of a situation, a ring of authenticity
in certain evidence, an uneasiness because some factor in a situation
is unexplained or prima facie unexplainable, a sense of the general
weight of evidence though no individual piece of it is suffic] ently per-
suasive, a feeling that some leader or group is likely to act in a certain
way because of emotional or ideological predilections, however ir-
rational or illogical the course. Such factors enter in most Frequently
when there is no solid factual base for conclusions or when the evi-
dence is contradictory. A most notable case of correct hunch oc-
curred in 1962, when Director John McCone kept worrying the pos-
sibility that the Soviets might put strategic missiles into Cuba despite
the absence of reliable evidence to this effect and despite his Board's
judgment that they would not do anything so foolish.
In some ways the estimative job is easier than it was ten or fifteen
years ago. We have, for example, much more and better evidence
on many aspects of Soviet military capabilities than we did then;
collection methods have improved and analytical skills developed.
But there are still many problems which strain the estimator's
capacity. Some which cause the severest trouble are those predicting
likely courses of events ( a) in unstable areas or situations of tension,
( b) in situations where the strengths of competing forces appear
evenly balanced or are difficult to assess, and ( c) where the evidence
is contradictory, often through deliberate deception. In each of
these types of problems the Board of National Estimates has made
misjudgments in which this estimator has participated. I would
like to describe as best I can recall how three estimates over which
I presided came to render�if not explicitly, at least implicitly�
judgments that were wrong.
28 71�
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
Three Estimates $E CET
Arab-Israeli Tensions
Estimates of future developments in category a, in situations of
instability or tension, are both the most challenging and the most
discouraging kind. The first reaction is that one is being asked to do
the impossible, yet this kind of task can be the most rewarding if
done successfully ( and although I speak here of a failure, it often
is done successfully). Actually, it is impossible to predict the course
of events; but one can describe how the parties involved apparently
think about a situation, how they have so far acted toward it, and
how they might act toward hypothetical changes. The case I have
in mind is that of an estimate entitled "The Arab-Israeli Dispute:
Current Phase," published 13 April 1967.
This estimate was undertaken in the context of an increase in tensions
in the Palestine dispute and in the wake of an Israeli raid on Samu
in Jordan in late 1966 which shook the monarchy in Amman. These
developments raised the question of whether the modus vivendi that
had prevailed between Israel and its Arab neighbors since 1957 was
coming to an end. Most of what the estimate said was right:
a. Rivalries and disputes among the Arabs reduce their chances of doing
anything significant about their quarrel with Israel; these rivalries also create
some danger of precipitating crises from which large-scale Arab-Israeli hos-
tilities could develop.
b. The Israelis seem likely to continue existing policies, including occasional
retaliatory action; they would resort to force on a large scale only if they
felt their security seriously endangered.
c. [The Israelis] could best any one of their neighbors and probably all of
them collectively. Arab cooperation being what it is, Israel probably would
not be obliged to take them on all at once.
d. The Soviet leaders almost certainly view the Arab-Israeli dispute as
promoting their interests. . . . But the Soviets do not want an outbreak of
large-scale conflict in the area, since this would carry serious risk of a US-
Soviet confrontation and thus threaten the positions which the Soviets have
already won in the area.
But the estimate had one final conclusion which, though it was
technically correct, conveyed a sense of reassurance and was, in light
of the events of May and June 1967, misleading.
Although periods of increased tension in the Arab-Israeli dispute will occur
from time to time, both sides appear to appreciate that large-scale military
action involves considerable risk and no assurance of leading to a solution.
In any event, the chances are good that the threat of great power intervention
will prevent an attempt by either side to resolve the problem by military force.
'reit 29
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
Three Estimates
I say that this last conclusion was technically correct because the
Six Day War was not an attempt to "resolve" the Arab-Israeli problem
by military force. It fits under the rubric of the conclusion ( see a
above) that rivalries among the Arabs created the danger of "pre-
cipitating crises from which large-scale Arab-Israeli hostilities could
develop" and that (in d above) "the Soviet leaders almost certainly
view the Arab-Israeli dispute as promoting their interests." A review
of the available information shows that the Soviets had a role in
precipitating the crisis by passing intelligence information about Israeli
plans for a punitive expedition against Syria to the Syrians and
Egyptians. Nasser, who had been accused in the past by his Arab
rivals of hiding behind the skirts of the UN, this time sought to avoid
the charge. His mobilization and the events which followed then
led the Israeli leaders to conclude that their security was "seriously
endangered" (b above). Quite clearly, both the Soviets and the
Egyptians made some miscalculations about the consequences of their
actions.
One can thus exculpate oneself by this kind of textual exegesis.
But there was in the estimate a serious lacuna: we did not sufficiently
treat the possibilities arising out of terrorist activities, border raids,
troop movements, propaganda, political warfare, and the psychological
effects of these in Israel and the Arab world. Had we understood
these better, we should have ended the estimate by noting the danger
that they could lead to an explosion rather than asserting the unlikeli-
hood of a deliberate resort to force.
Why did we make this error? I think we were under two mis-
apprehensions. The first was that we overestimated the Soviets'
good sense, something we have done before ( e.g., when the question
was whether they would deploy missiles to Cuba, in 1962). It is,
think, a safe judgment that if the Soviets had thought in mid-May
what they knew on 5 June, they would have kept that provocative
intelligence information to themselves. The moral is that how the
Soviets may think about a particular area and what they may do
tactically may not be entirely consistent. We as estimators must
recognize more frequently ( as we often do in observing the tactical
moves of governments regarding which we have more complete
knowledge) that specific actions taken by the agents of a government
do not always flow from the general policy objectives or posture of
the leadership.
30 SECRET
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
Three Estimates
The other misapprehension, I believe, was a failure to keep up
with dynamic aspects of Near East politics. We did not realize how
much more confident in themselves both the Israelis and Egyptians had
become. We did not therefore realize how much lower was the
threshold of Nasser's readiness to expose himself to danger, and how
much lower was the threshold of Israel's readiness to fight against
creeping threats to its existence. I cannot say whether this was
a failure in intelligence reporting or in analysis; I suspect it was
a bit of both. U.S. personnel abroad are often too much absorbed
in the day-to-day business of their operations to detect a growing
change of mood; analysts in Washington are too often cynically prone
to think their foreign charges are the same feckless ( or scheming)
fellows they always were and that nothing much changes. I, for one,
am prepared to be a bit more cynical myself about area specialists.
Prospects in Vietnam
The estimate illustrating misjudgment in category b, when the
strengths of competing forces appear to be evenly balanced or are
difficult to assess, is one which had a long and tortuous history.
Initiated in October 1962, it was finally cleared by the USIB only in
April 1963; it was entitled "Prospects in South Vietnam." At that
time Diem was still president of South Vietnam and Madame Nhu
was riding high. The U.S. commitment was still in the form of
advisers and logistical support. The estimate was to assess how things
were going, what problems there were, what the prospects were. I
will not examine here all the conclusions of the paper, but only its
general statements about how the war was going and what the pros-
pects were for South Vietnam in the kind of struggle that was going
on then. I quote from some of the conclusions of the draft finally
approved by the USIB:
a. We believe Communist progress has been blunted and that the situation
is improving. Strengthened South Vietnamese capabilities and effectiveness,
and particularly US involvement, are causing the Viet Cong increased diffi-
culty, although there are as yet no persuasive indications that the Communists
have been grievously hurt.
b. Assuming no great increase in external support to the Viet Cong, changes
and improvements which have occurred during the past year now indicate that
the Viet Cong can be contained militarily. . . . However, we do not believe
that it is possible at this time to project the future of the war with any
confidence. Decisive campaigns have yet to be fought and no quick and easy
end to the war is in sight. . . .
31
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
,effK1 Three Estimates
c. Developments during the last year or two also show some promise of
resolving the political weaknesses, particularly that of insecurity in the country-
side, upon which the insurgency has fed. However, the government's capacity
to translate military success into lasting political stability is questionable.
The estimate thus rang no tocsin. To put it in simpler language:
things are not going to hell; we don't know how it will all come out,
but the South Vietnamese are not doing so badly; Diem is improving,
he might win the military struggle, though even if he does, don't
think the political troubles of South Vietnam will be over. Half a
year later Diem was ousted, and the political and military situation
degenerated to critical proportions by the end of 1964. What made
the estimate so wrong?
In this case the draft originally prepared by the Estimates Staff
was essentially correct, but it was fatally weakened during the process
of review and coordination. This was a long and painful process for
me as chairman, since I had helped the Staff prepare this draft. Let
me quote some of the original conclusions, c below being the final one:
a. There is no satisfactory objective means of determining haw the war is
going. The increased US involvement has apparently enabled the South Viet-
namese regime to check Communist progress and perhaps even to improve the
situation in some areas; however, it is impossible to say whether the tide is
running one way or the other ...
b. On the South Vietnamese side, new strategic concepts, such as the
fortified hamlet, and shifts in military and security organization training, and
tactics have strengthened the counter-guerrilla effort. Howev5r, very great
weaknesses remain and will be difficult to surmount. Among these are lack
of aggressive and firm leadership at all levels of command, poor morale among
the troops, lack of trust between peasant and soldier, poor tactical use of
available forces, a very inadequate intelligence system, and Dbvious Com-
munist penetration of the South Vietnamese military organization.
c. The struggle in South Vietnam at best will be protracted and costly.
The Communists are determined to win control, and the South Vietnamese
alone lack the present capacity to prevent their own eventual destruction.
Containment of the Communists and reestablishment of a modicum of security
in the countryside might be possible with great US effort i a the present
political context of South Vietnam, but substantial progress toward Vietnamese
self-dependence cannot occur unless there are radical changes ia the methods
and personnel of the South Vietnamese Government. Even should these take
place without mishap, this would only be a beginning; the Communists retain
capabilities and support which will require years of constructive effort to
dissipate.
Some of the process of dilution began in the Board itself. The
Board did not change the main thrust of the paper, or alter essentially
32 ET
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
Three Estimates
the wording of the conclusions cited in a and b above. But it did
eliminate some of the prescient words both from the text and from
the final conclusions cited in c. The final conclusion now read simply:
With US help, the South Vietnamese regime stands a good chance of at
least containing the Communists militarily. However, the modus operandi
of the Diem government, and particularly its measures to prevent the rise
of contenders for political power, have reduced the government's effectiveness,
both politically and militarily. We believe that unless radical changes are
made in these methods of government, there is little hope that the US in-
volvement can be substantially curtailed or that there will be a material and
lasting reduction in the Communist threat.
The serious weakness of this change was that it shifted the emphasis
from the inherent difficulty and long-term character of the problem
(to which Diem contributed) to an indictment of the Diem regime.
This led us into trouble at the coordination meeting with departmental
representatives.
Some of the military representatives at the coordination meeting had
served in South Vietnam and had been appalled at the South Vietnam-
ese military performance. The emphasis in the paper on political weak-
nesses as a major cause of the military failures quite naturally appealed
to their professional instincts as well as confirmed their own observa-
dons. The indictment of the Diem regime, however, no doubt because
it called into question the existing .U.S. policy of working with Diem,
caused the State Department representative to reserve his position
on this aspect of the paper. He also thought the estimate underesti-
mated the prospects for gains through an improved military effort,
although we had gone so far as to say, "With US help, South 'Vietnam
stands a good chance of at least containing the Communists militarily."
Thus the DCI and the USIB members were presented with a paper
which, although the Estimates Board had eliminated reference to the
gloomy long-term prospects, was still a fairly dolorous document.
But it was encumbered with a departmental reservation, and this
obliged the USIB to look at it carefully. The DCI, then John
McCone, was particularly uneasy about it, since it seemed to con-
tradict the more optimistic judgments reached by those in policy
circles who had been sent to Vietnam to make on-the-spot appraisals
and recommendations. He therefore decided to postpone USIB con-
sideration and asked the Board of National Estimates to consult with
some of those who had been on such missions. The Board proceeded
"CREK.. 33
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
Three Estimates
to meet with two high-ranking military officers and two civilians
in key policy-making positions.
None of these four consultations was particularly helpful. The
witnesses seemed reluctant to make a frontal assault on the judg-
ments of the paper but equally reluctant to endorse it. They showed
a general tendency to take issue with a particular sentence purport-
ing to state a fact, rather than an estimative judgment. This or that
was "too pessimistic," but there was no clear line of argument why.
All four held forth some degree of optimism, largely based upon the
belief that things were better than they had been. This indeed may
have been true, but it was not established how badly things had
been going before or how this degree of improvement stood up to
the task, namely to deal with a determined and resourceful op-
ponent who was immeasurably helped by the profound underlying
political weaknesses of South Vietnam. None of the consultants was
attempting to mislead, but the simple fact was that each cf them in
some way and to some degree was committed to the existing U.S.
policy, and none of them was intellectually free at that point or in
those circumstances to stand back and look at the situation in its
broadest aspects.
The drafters then returned to their desks and prepared a revised
draft. The Staff members, although increasingly weary of the con-
troversy, were nevertheless much inclined to stick to their guns. I,
however, had become inclined to shade the estimate in a more op-
timistic direction. I began to think that perhaps we had been too
gloomy; and at the same time I had to get an estimate i.hrough to
meet the DCI's new deadline. If we stuck to the original draft, the
DCI and other CIA components might not go along with it; even if
they did, this draft might now evoke still greater departmental dis-
sent than it had the first time (since high-ranking personnel had now
become engaged); in short, if we were so rigid that we invited debate
and amendment at the USIB, we might find ourselves with a paper
more offensive to our judgment than one which moved slightly to-
ward a less pessimistic view. What we now wrote, in spite of some
staff objection, embodied approximately the conclusion first cited
above. The estimate rode easily through the USIB with the DCI's
full concurrence.
Even so, this estimate was not calculated to give anyone a sense
of comfort. Indeed, very recently a senior official closely associated
with Vietnamese affairs, who had most likely seen only the finished
34
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
Three Estimates
product, remarked that it was too bad policy-makers had not paid
more attention to it. Nevertheless, it did not sound the alarm which
it should have and would have if the first draft conclusions had been
kept intact. A year or so after the date of the estimate, Mr. McCone
openly expressed regret for his own part in weakening what had been
"right the first time."
The lesson provided by this experience is to shun the advice of
those who in one way or another are committed to or responsible for
a particular line of policy. They are no doubt well informed, but it
is also theirs to be hopeful. Above all, their responsibility is to their
policy-making chiefs, and they can hardly be expected to recite before
an intelligence working group information or beliefs which implicitly
or explicitly might suggest that established policy is based upon un-
sound premises. Study of the premises of national policy is the
business of intelligence officers, and it is as unfair to ask the executors
of policy to testify on the soundness of those premises as it is unwise
to accord their views uncritical acceptance.
The Goa Invasion
The estimate that illustrates the difficulty of forecasting in cate-
gory c�when the evidence is contradictory, perhaps because of delib-
erate deception�was not very important in terms of its policy impact;
its conclusion was so equivocal that it provided the warning needed.
It was, however, wrong, and I who chaired it was among those who
thought it wrong at the time. It was a crash estimate, requested on
the morning of December 12, 1961 and approved by the USIB on
the afternoon of the following day, concerning the likelihood of an
Indian attack upon Goa. During the preceding few weeks Indian
troops had been concentrating in the Goa area, public opinion�
especially on the left�was clamoring for action, and a strong mo-
mentum in favor of invasion had developed. Yet the evidence was
conflicting, and it was possible that these activities were de:signed
purely to apply pressure and to bring about the incorporation of
Goa into India by peaceful means.
We thought that Nehru had not made up his mind and was being
subjected to contradictory pressures. We concluded:
Clearly there is strong evidence pointing to an invasion�the military and
political preparations have gone so far as to be difficult to reverse without
some loss of prestige to the Indian government. Although the Indians perhaps
still hope that their warlike activities will extract concessions from the Por-
SE � 35
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
,ret� Three Estimates
tuguese, we doubt that the Lisbon Government will move far enough�if at
all�to meet Indian requirements. However, relying chiefly on our judgment
of what Nehru conceives to be India's basic interests and our assessment of his
past behavior, we believe that the chances of a direct military invasion are
still about even.
Five days after this estimate was approved the Indians seized Goa
by military force.
The formulation in the final estimate was close to that of the staff
draft. In a post-invasion memorandum to the Chairman of the Esti-
mates Board, the head of the Near East Staff stated that initially he
and his colleagues had rated the chances as less than even, "relying
mainly on Nehru's restraint in previous crises over Goa and their
estimate of his attitudes, objectives, and ability to control develop-
ments," and that he had learned through informal contacts that the
State Department people went even further, calling the odds "con-
siderably less than even." Nevertheless, impressed by the evidence
of advancing preparations, the Staff was uncertain enough to qualify
the chances as only "slightly less than even."
The Estimates Board members, in their review, agreed generally
with the experts. It was difficult for me, as chairman, to dissent;
but I was impressed more by the evidence of preparations than by
the history of Nehru's political attitudes. The reports From people
who had seen the preparations and talked with the Indians sounded
as if the latter meant business, meant to finish off the Goa affair once
and for all. Argument along these lines succeeded in moving the
Board toward dead center�"The chances of a direct military inva-
sion are still only about even"�and implying that as preparations
continued and the Portuguese failed to give, the chances of invasion
might rise.
The coordination meetings did not help very much. One depart-
mental representative who wanted to raise the odds on invasion
wanted also to add a paragraph about the threat to U.S.-Portuguese
relations and U.S. base rights in the Azores if the Indians went ahead.
It seemed to me that his position derived more from departmental
interest than objective judgment, and this was not acceptable in an
intelligence estimate. We stayed with the Board's "still about even"
formula, and the USIB also agreed to this without dissent.
The day after the estimate was approved, the odds or, an invasion
rose perceptibly in reports from New Delhi. The Army attache said
he believed invasion would "take place very soon."
36 SEC
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
Three Estimates ,e16
(b)(1)
(b)(3) the country team believed that military action was
imminent unless the Portuguese promptly folded. /
/ The only person who did not seem to think
an invasion was on was the U.S. ambassador in New Delhi, who
expressed the belief that action was not imminent less than 24 hours
before it began.
The actual invasion gave a sense of vindication to those of us who
had thought it likely, but it also raised the question of why the others
had been wrong. The evidence, though conflicting�the ambassador
no doubt was the victim of some deceit�did include reports with a
decided ring of seriousness. Those who rated the chances of invasion
as even or less than even of course read these reports. They were
relying on Nehru's high-mindedness, and since this did not jibe with
the evidence, they had nowhere to go but to sit on the fence. The
lesson to be derived from this experience is not that one should look
only at the evidence and disregard the doctrines and attitudes of
leaders; that would be folly. It is that one should try to reconcile
the two; in so doing one might perhaps find that, as in the law, there
is more than one line of precedent.
No Rules
Other Board members could, I am sure, make an analysis of how
estimates in which they participated fell short and similarly draw
lessons from them. In time we could have enough lessons floating
around to keep us tongue-tied. We could fall into the tragic error
of the young man whose aggressive and fast-moving brother killed
himself by wrapping his car around a light pole; the surviving brother,
having taken this lesson to heart and resolved to plod about on foot,
was run over by a truck. Our job is to make estimates; we have to
take the plunge. This does not mean reckless diving, but it does not
mean standing on the end of the diving board helpless with worry
about every conceivable hazard to health and safety.
There is no alternative to regarding each estimative problem as a
new one and applying one's accumulated knowledge and experience
to it. It helps to try to determine why we were right or why we
were wrong and to use these determinations as signposts along the
way, but we must also remember that the specialist who misled us
on one estimate corrected our misapprehensions on another, that the
t 37
Approved for Release: 2024/06/12 C06887351
Approved for Release: 2024/06/12 C06887351
pd(- Three Estimates
political philosophy which a national leader seemed to negate in one
action he might never negate again. The problems we a::e dealing
with are too complex for simple rules or simple "lessons." The magic
words "estimate" or "judgment" are simply the exercise of good sense
in light of everything it is possible to learn or to ponder concerning
any particular matter.
38
Approved for Release: 2024/06/12 C06887351