ANALYTICAL PITFALLS AND STUMBLING BLOCKS

• Collection: FOIA Collection
• Collection: Declassified Articles from Studies in Intelligence: The IC’s Journal for the Intelligence Professional
• Document Number (FOIA) /ESDN (CREST): 0000622865
• Document Page Count: 7
• Document Release Date: July 30, 2014
• Case Number: F-2013-02322
• Publication Date: December 1, 1992

Approved for Release: 2014/07/29 000622865TITLE: Analytical Pitfalls and Stumbling BlocksAUTHOR:(b)(3)(c)VOLUME: 36 ISSUE: Winter YEAR: 1992Approved for Release: 2014/07/29 000622865pproved for Release: 2014/07/29 000622865TUNES 1INTELLIGENCEA collection of articles on the historical, operational, doctrinal, and theoretical aspects of intelligence.0All statements of fact, opinion or analysis expressed in Studies in Intelligence are those ofthe authors. They do not necessarily reflect official positions or views of the CentralIntelligence Agency or any other US Government entity, past or present. Nothing in thecontents should be construed as asserting or implying US Government endorsement of anarticle's factual statements and interpretations.Approved for Release: 2014/07/29 000622865Approved for Release: 2014/07/29 000622865 (b)(3)(n)-Overcoming obstaclesAnalytical Pitfalls and Stumbling Blocks (b)(3)(c)Soothsayers used to examine the entrails of animalsto predict the future. Today, we intelligence anaylstsrely on far more sophisticated sources to underpinour appraisals of what lies ahead. Nevertheless, weoften do not appear to obtain much better results thatour more primitive predecessors.On some occasions our seeming inability to figureout what comes next is caused by the way we ap-proach our work, rather than a failure of intellect orimagination. Moreover, there are times when we ac-tually get it right but somehow are unable to per-suade our superiors or the policymakers of the ac-curacy of our judgments.The following is a list and disucssion of what I con-tend are several barriers to good intelligence analy-sis. Most examples relate to life in CIA's Directorateof Intelligence (DI). I believe that they also aregenerally applicable to intelligence producers in thecommunity. I present this to alert junior people topotential problems and to remind seniors that thereare traps for even the most experienced.Defensive AnalysisThose analysts who are brave enough to make judg-ments usually make far more good ones than bad.The best and brightest analysts that I have knownhave never shirked from making a judgment, andpromotions go more often to analytic risk-takers. Ifone wants to join their ranks, the following pitfallsshould be avoided:? Echoing the opinions of collectors in the field,who usually are not privy to all-source intelli-gence.? Being comfortable as part of a consensus. You canhave a lot of company in making a wrong judg-ment.13? Defining prediction so narrowly that you really donot do it at all.? Setting up a series of distant hypotheses to avoida short-term prediction. For example, an assess-ment of how a united Korea would likely dealwith the US probably would not be helpful if itdid not offer insight on when and how unificationwould be likely to occur.? Establishing a straight-line projection in lieu ofreal analysis.The Use of HistoryA look at a nation's or region's history is often thelogical starting point for an intelligence officer. Fortoo many, however, it also seems to be the stoppingpoint, particularly in terms of analytical history. Thefact that something occurred a certain way 10 or 20years ago can easily become the basis for a judgmentthat the same thing is happening today. While suchan approach often provides the correct answer, italmost guarantees failure in predicting new develop-ments. In using history, always look for what is new.Are there real parallels, or are today's events beingforced into yesterday's mold?Applying LabelsTo simplify presentation, we type individuals and is-sues. Everyone is put into his or her group?conservatives and liberals, reactionaries and radicals,hardliners and softliners. Too often the reader is ex-pected to understand precisely what the terms mean.If there is a chance that your reader might be mis-lead, add a short definition.Con tialApproved for Release: 2014/07/29 000622865Approved for Release: 2014/07/29 000622865C>fidential   (b)(3)(n)Underestimating SituationsFor many years we put considerable stock in the per-manence of repression. The cant was that "the re-gime's pervasive and efficient secret police will beable to handle dissent." We also invented the "long-suffering peasant" who would put up forever with asituation that would send Americans to the barri-cades. Finally, we seemingly closed the loop with thejudgment that, in the absence of evidence of large-scale opposition to a system, the people had to havebought what their governments were selling. Butfrom time to time, and especially during the past fewyears, intelligence analysts have been brought upshort when highly controlled regimes rapidly fellapart. What did we miss?I would argue that the myth we created of the degreeof political control and the impact of societal inertiahas been a major barrier to our understanding ofchange and our ability to recognize early the signs ofan impending national or regional explosion. Oncewe conclude that a government is essentially un-challenged, virtually all resources and analytic effortcan be focused on military and technological de-velopments, leadership shifts, foreign policy adven-tures, and economic problems.If we want to be relevant, bread-and-butter issueshave to be at the top of our list. I do believe,however, that we have to pay more attention to un-derstanding societal change.This process often proceeds at a slow pace. To meas-ure it, we identify and weigh the importance of smallbreaks with the past and evaluate how such changewill affect a government's ability to survive.Examples abound. The children of the revolution be-gin to replace their parents. Ideology is ridiculed.People begin to focus on the disparity between thehaves and the have nots. Corruption erodes the lead-er's authority. Economic ineptness and bureaucraticbungling foster anger, resentment and frustration.Foreign wars fuel national rage as casualties mount.Young people reject the values of their own societyfor those of another. Although we have not ignoredsuch changes, we have not concentrated on develop-ing the analytic tools to the extent that they can helpus do a better job in anticipating landmark events.ConidentialBlocksIn looking at societal change, an analyst probablyshould not dwell too long on the differences between"them and us," to put much stock in "nationalcharacter." People are likely to have the same needs,goals and desires wherever they live. Even thoughpeople adapt to the mores of their particular societiesin order to survive, it does not mean that they haveto like them. If logic suggests that people should beunhappy, they probably are. And at some point theywill be likely to try to change their situation.Publish or PerishIntelligence officers use many different vehicles totransmit their messages. Many often appear to con-sider that the daily current intelligence publicationshave the most impact because the President, cabinetmembers, and other senior policymakers read them.Nevertheless, I see two serious problems stemmingfrom the way we produce current intelligence?publishing too fast and too often.In my view, many analysts and managers believe thatthe primary goal is to scoop the commercial compe-tition. This approach sometimes means that ideas arevetted prematurely, before full analytical resourcescan be applied. As electronic dissemination is in-creasingly used, analysts will almost certainly comeunder even greater pressure to get a story out.Some analysts have said that they also have beenpushed to publish because they are the recipients ofexpensive-to-obtain technical intelligence. Using thematerial in a product supposedly justifies the cost ofcollecting it, even though it adds little to the body ofknowledge.Getting Locked InOn really "hot" topics, the policymakers invariablywill want a continuing flow of information. The lackof new information or insights, however, can lead tothe repetition of analysis, with the consequence thata judgment too often stated becomes difficult tochange.14Approved for Release: 2014/07/29 000622865Approved for Release: 2014/07/29 000622865BlocksThis problem is compounded by a demand for con-tinuity and consistency. Most analysts have been ad-vised from time to time that it is bad form to keeptaking their consumers up one hill and then down thenext. There is, at least, an implicit command to stickto initial judgments until they are contradicted bystrong evidence. From a public relations standpoint,this probably is the right thing to do. But this posi-tion would seem to impede the prompt transmittal ofreports or messages to policymakers, thereby limitingtheir available options in dealing with the problemsat hand.At a minimum, analysts should do a periodic zero-based review of their key judgments to ensure thatpressures to write are not interfering with their abil-ity to determine The facts of a given situation.Policy RelevanceIn the Agency's early years, managers and analystsseemingly adopted the view that knowledge wassought for knowledge's sake. This led to a debate be-tween those who believed that remaining aloof frompolicymakers was the only way to keep their analyti-cal purity and those who believed that purity withoutrelevance made no sense. The latter group won theargument.Analysts and managers are now expected to establisheffective contact with policymakers, and we havelargely succeeded in plugging into the policyprocess. As a result of our efforts to march in stepwith policymakers, however, we appear less inclinedto produce premonitory intelligence. This perceptionis based on a sampling of intelligence publicationsover time. I have gained the clear impression thatmost papers that I read now seem to emphasize whatis happening and why, or what happened and why. Inthe interest of policy relevance we appear to betraining a generation of intelligence newspersons andhistorians.Overcoming BiasA fair amount of time, energy, and emotion recentlyhas been devoted to the debate on "politicization."There are a few points around the periphery of thediscussion that are worth noting.15Co-7iftcleg*1(b)(3)(n)First, managers in the DI are likely to draw similarconclusions from the same overt sources of informa-tion as policymakers. I would further argue that un-der most circumstances intelligence managers proba-bly are not eaptives of their consumers; they arelikely to share the same view of the world, and theyprobably would be as skeptical as their clients ifsomeone challenges what they see as reality.Assuming these thoughts are generally true, ananalyst meeting resistance from a supervisor to achange in judgment and seeking a higher probabilityof getting his or her new or different interpretationinto print should initially assume that the managerdoes not find the new evidence or analysis compel-ling. If there is good will on the part of the reviewer,the analyst has a reasonable chance at overcomingthe former's doubts and reservations. But if themanager's resistance is politically motivated?a farlower probability in my mind?then nothing willmove him or her. Consequently, an analyst shouldenvision a situation that will permit success.Second, we focus on mangers as politicizers, andoften forget that analysts can-also have agendas otherthan producing the best unbiased analysis. Therehave been times in the past when one could questionwhether a particular line of analysis was aimed atenlightening or directing US policy. Even the wayanalysts package their judgments can lead to uncer-tainty about motivation. For example, I recall theplaintive remark of a branch chief: "I want to tellpeople the truth, but do we have to rub their noses init?"As to inadvertent bias in analysis, there are numer-ous traps. On many occasions we DI militaryanalysts expressed delight that our views placed usdead center between the Defense Department ("onthe right") and the State Department ("on the left").That may have been the best place to be, but in20-20 hindsight, one wonders how much of our anal-ysis was influenced by our desire not to adopt thepositions of our competitors.While politicization can be a major impediment toproviding the best intelligence, do not focus on it tothe exclusion of equally important barriers. The keyis to understand the nature of the problem.Approved for Release: 2014/07/29 000622865Confi tialApproved for Release: 2014/07/29 000622865 (b)(3)(n)  Writing to CommunicateOne of the rites of passage for most new DI analystsis a briefing on the DI writing style. The ostensiblepurpose is to prepare people for formatting theirproducts, but the end result frequently is to intimi-date the new employee. The following is a compositeof some of the briefings I have overheard:"Good morning, Mr. Phelps. Your mission, ifyou choose to accept it, is to learn to write inthe DI style. This style is unique in the world;we believe that it came to the original intelli-gence analyst in a vision after he fasted for 40days in the desert. We put our judgments upfront, use topic sentences, keep all sentencesshort and to the point, avoid exotic words, andput an implications section at the back of thepaper to transmit the full impact of the mes-sage. It will probably take you years to learn it,and the odds are that you will never succeed. Ifyou fail, your branch chief will disavow everknowing you."I offer an alternative speech:"Good morning, Mr. Phelps. I have a missionfor you that is far from impossible. I want youto take those skills that you have been using allyour life, particularly analyzing and communi-cation, and apply them to your job. We have adifferent format, but it should not be aproblem. Put your judgments up front, and sup-port them with clearly written, informativeparagraphs. Some people can write better thanothers, but virtually all the people we hireshould be able to write."I have watched enough talented people stumble overintelligence writing to wonder whether our attitudetowards it erodes their confidence and impedes theirdevelopment. Why do analysts with advanceddegrees and presumably much experience in writingsometimes fall apart when trying to put together aparagraph for an intelligence publication? How manycases of writer's block have we noted?I suggest that as a test we modify our introduction tothe intelligence arts by delivering a simple messageat the outset: it is not mystical, it is communications!BlocksDeadly ProseWhen it comes to intelligence writing, analysts seemto be getting the message that boring is best, fol-lowed closely by dry, and uninteresting. Somehowthe notion that we are just one of many sourcesavailable to consumers and that if we do not grabtheir attention we will lose them does not appear tobe a prevailing view. The way our product is printedis first rate, our graphics outstanding, and even ourformats are quite good. I am concerned that once aconsumer starts to read, he or she may be turned offby soporific prose.Far too often, scholarly equates to indigestible, andour titles could cure insomnia. I believe that many aDI analyst writing about the beginning of World WarIII could title his or her paper, "Massive Exchangeof Thermonuclear Weapons Likely to Cause SevereEconomic Disruption and Tax Most MedicalFacilities." This is a straightforward, clear, andaccurate title?and deadly dull.I do not advocate using silly or irrelevant writing inour publications, but it seems reasonable to try anduse our prose to engage policymakers. If I had to sellmy paper, would anyone want to buy it?CoordinationMost analysts appear to treat coordination withalmost the same degree of enthusiasm as a trip to thedentist. One does it because it is required and notout of any expectation that the product might beimproved. The implicit goal of the analyst seeminglyis to take an article completely through the processwithout it being touched by other human hands.Pride of authorship is just one factor in the distastefor coordination. We may also be recreating thenorms of academic behavior. Professors and graduatestudents present their ideas and then defend them tothe death. Although everyone is polite, it has neverbeen clear to me whether anyone listens to anyoneelse?other than junior scholars who sit at the feet ofthe masters. On occasion, in convocations of aca-demics, I have felt like I was attending a Borgiafamily reunion.16Approved for Release: 2014/07/29 000622865Approved for Release: 2014/07/29 000622865BlocksIntelligence officers do not exist solely to impressanyone with their scholarship or to prove the fallacyof an opponent's views. Our responsibility is toprovide policymakers with the kind of informationand insight that will best enable them to do theirjobs. If that comes from an individual effort, so beit. But in my mind there should be equal credit forthe analyst who recognizes the clarity and relevanceof another's ideas and incorporates them in his or herproduct. Thus, coordination should be an opportunityto gain additional knowledge or to test the validityof one's thoughts, not an exercise in stonewalling orone-upsmanship.Assuming that being fair-minded and receptive arethe starting points for effective coordination, thereare a number orways to improve the chances ofgetting a good result:? Be clear in your mind why you have chosen yourparticular analytical approach. Explore the alterna-tive explanations and also be prepared to depictyour situation.? Treat every suggestion seriously. "That is the -stupidest idea I have ever heard" is not aresponse calculated to improve the coordinationprocess.? Be prepared to give ground. Ninety percent, oreven 75 percent, of something is better than 100percent of nothing.? Be willing to change words as an act of goodfaith. Words often offend when ideas do not. Youmay be able to keep the same analytical line, ifyou define it differently.? Be willing to be persuaded. Every old-timer canpoint to an occasion when he or she turned apiece 180 degrees on the basis of someone else'smore perceptive argument. It does not happenoften, but it is not a crime when it does.17CoritWntial(b)(3)(n)? Take seriously the comment, "I do not understandwhat are you saying." We are in the communica-tions business. If someone who is familiar with asubject is confused, consider the impact on thepoor polieymaker.? If you disagree with specific coordination com-ments, write alternative words before consultingdirectly with the coordinator. With words in hand,you are in a much better position to influence thecourse of discussion.? Be willing to concede several minor points to gaina major one. It is hard to resist, "I gave you whatyou wanted on these three ideas, can't you movecloser on this other one for me?"Still a Good ShowIn presenting these pitfalls and problems, I would beremiss if I did not put them in perspective. I believethat we in the DI and the Intelligence Community asa whole generally have the right to be proud aboutthe work we do and the printed and oral intelligencethat we provide. What I have pointed to as needingattention are deviations from a product norm that isof high quality, thoughtful and relevant.It has been a matter of pride to me, however, that in-telligence organizations never rest on their laurelsand are constantly be looking for ways to improve. Ifthis were not so, it would have been pointless towrite this article.This article is classified CO>713A<VTIALApproved for Release: 2014/07/29 000622865Confi tial(b)(3)(n)