THE DI'S PRODUCT REVIEW PROCESS

• Collection: FOIA Collection
• Collection: Declassified Articles from Studies in Intelligence: The IC’s Journal for the Intelligence Professional
• Document Number (FOIA) /ESDN (CREST): 0000622860
• Document Page Count: 7
• Document Release Date: July 30, 2014
• Case Number: F-2013-02322
• Publication Date: June 1, 1992

TITLE:AUTHOR:VOLUME:Approved for Release: 2014/07/29 000622860The DI's Product Review Process(b)(3)(c)36 ISSUE: Summer YEAR: 1992Approved for Release: 2014/07/29 000622860Sproved for Release: 2014/07/29 000622860STUDIES IINTELLIGENCEA collection of articles on the historical, operational, doctrinal, and theoretical aspects of intelligence. -All statements of fact, opinion or analysis expressed in Studies in Intelligence are those ofthe authors. They do not necessarily reflect official positions or views of the CentralIntelligence Agency or any other US Government entity, past or present. Nothing in thecontents should be construed as asserting or implying US Government endorsement of anarticle's factual statements and interpretations.Approved for Release: 2014/07/29 000622860Approved for Release: 2014/07/29 000622860How to improve itThe DI's Product ReviewProcess (b)(3)(c)". . .it is worth remembering that the basic purposeof the review process is to take the individual ideasor judgments of a single analyst or a few analysts,and turn those ideas or judgments into the institu-tional position of the Central Intelligence Agency."DDI Newsletter12 January 1984From the standpoint of intelligence analysisper se, the product review process is one ofthe central fixtures of professional life in theDirectorate of Intelligence (DI). It is also an aspectof intelligence work that deserves more attention andstudy than it has traditionally received. While atti-tudes toward review vary tremendously across thedirectorate, most observers agree that the processcould be improved. The last DI-wide survey on thesubject, conducted in late 1988 by the DI Manage-ment Advisory Group (MAG), found that the biggestsingle complaint from analysts was not with theprocess itself but with management's failure to fol-low the prescribed process. I would argue that as adirectorate we do not have any "prescribed" process,and that many of the problems with review can betraced to this basic lack of a conceptual framework.For new analysts, review can be an unexpected,humbling experience that seems to contradict manyof the norms of their academic experience. In thecase of complex or contentious pieces review can bea grueling exercise, even for experienced analysts. Insome cases, the outcome of the process may be a de-cision not to publish, with months of research andwriting seemingly wasted. One of the most commongrievances expressed by analysts is that there are toomany layers of review, leading to contradictory re-quirements and redundant or "circular" revisions.7_--C41141-144ENT-TAt-Managers have to deal with similar problems, thoughfrom a different perspective. Their greatest challengeis often to mediate conflicting positions and producecompromise language without gutting the analysis.And, finally, for all involved the process can some-times be protracted and exhausting. Many of us haveheard stories about items for the NationalIntelligence Daily (NID) that took a week to coor-dinate or "hardcover" papers that took a year fromfirst draft to final publication.In my view, the existing review process is necessaryand it is not fundamentally flawed. Many of itsproblems can be ameliorated, if not solved, by build-ing a clearer shared understanding of its goals andpurposes.First Things FirstThe directorate has no succinct, common definitionof review. Some DI offices have statements abouthow review is supposed to function. But these areusually mechanistic descriptions or productiontimetables detailing, for example, how much turn-around time each management echelon has for apaper. Because of differences in culture and subjectmatter, and the latitude given office directors by theDeputy Director For Intelligence, at this level theprocess can differ substantially from office to office.In general terms, review is the contribution of the or-ganization to the production of a particular intelli-gence product, as opposed to the contribution of theindividual analyst. (This discussion deals primarilywith the production of longer-term written intelli-gence products, though some if it applies to currentpieces and high-level briefings.) This may seem ob-vious, but it is an important characterization becauseit reminds us that intelligence in whatever form itApproved for Release: 2014/07/29 000622860-GeftPIVENTIAT:Approved for Release: 2014/07/29 000622860may take is essentially an amalgam of individual andorganizational efforts, both of which have to begiven their due. To depreciate the individual's contri-bution is to forget that all analysis has to begin withsome person's intellectual effort; ignoring the organi-zation's rightful role risks straying into the morass ofanalytical pride.At a more specific level, review can be seen as hav-ing four basic elements: coordination, validation,editing, and production. These are not, however, se-quential stages; they are logically, but not necessarilytemporally, distinct. Coordination, which some mis-takenly assume to be synonymous with review, is thequality-control part of the process. It consists of theexamination of a particular piece of intelligence byothers in the organization to eliminate errors of fact,to identify additional perspectives that might nothave occurred to the author, and to ensure that thejudgments drawn are congruent with those taken byother DI components.Validation is the adoption by the organization of theanalysis contained in the product as its corporate po-sition. It is the heart of the process. Validation for-mally occurs when the product receives final ap-proval from the appropriate management level, but itactually takes place throughout the process, begin-ning with management's decision to undertake theproject. Through the sometimes innumerable coordi-nation meetings and draft rewrites, the organizationgradually takes "ownership" of the product from theanalyst. This is often difficult for both parties: theanalyst has to learn to give up total control of sub-stance, while the organization has to take care not tomuddle or compromise the analytical message. Evenafter final approval, some may remain skeptical of apiece's analytical line. There is nothing wrong inthis. When the Agency puts its seal on a product, itsignifies only an organizational consensus, not a una-nimity of opinion. In cases where opposing views areheld by a significant minority, differences can andshould be reflected in the draft itself.Broadly speaking, the goals of the editorial stage ofreview are to improve the readability and organiza-tion of the text and bring it into conformance withthe DI writing style. Good editors are able to im-prove the clarity of a piece while leaving substanceConfitiontia  Processunaffected. Because editing plays a role in refiningthe presentation of analysis, a critical aspect of intel-ligence production, it is properly considered part ofreview. Unfortunately, a.basic problem with editingis that too many in the process view it as theirresponsibility, a fact that often leads to an unneces-sary duplication of,effort. For example, how many ofus can recall expending considerable effort on theprecise wording of a piece of current intelligence atthe office level and below, only to have the NIDStaff substantially reword it?The final phase of the review process is usuallydominated by production, during which the piece isprepared for publication by the Office of CurrentProduction and Analytic Support production staff,the author, and his or her home office editorial staff.Although some might consider it inappropriate to in-clude production as part of review, important deci-sions regarding the overall layout, the presentation ofmaps, figures and other visual aids, and the finaliza-tion of the piece's dissemination list are made onlyat this stage.Perspectives on the ProcessA majority in the DI regardless of position, seniority,or substantive speciality believe the review processis imperfect. In my experience, observers of itusually fall into one of three general categories. Thefirst could be labeled the "pessimist" school. Itsproponents argue that the existing process is systemi-cally corrupt and that no amount of tinkering willcure its ills. For example, they cite the fact that thoseclosest to "substance," the line analysts, have aweaker voice in deciding disputed intelligence judg-ments than their non-expert superiors. This, theysometimes hold, opens the doors to the politicizationof intelligence. Others claim that minority positionsare often submerged by the dominant corporate view,and that fresh, groundbreaking analysis cannot occurbecause of the demands of building consensus. Inview of the seriousness of these shortcomings, theyargue, the existing edifice of review should be razedand rebuilt.A second, more mainstream group might be dubbedthe "cynics." While disagreeing with the pessimists'contention that the process in inherently flawed, this8Approved for Release: 2014/07/29 000622860Approved for Release: 2014/07/29 000622860Processgroup agrees that it is a waste of time to try to im-prove it. Cynics have witnessed many attempts to"fix" the review process, most of which they judgeto have had little practical effect. They are, therefore,content to live with the prevailing process. It isdifficult to generalize about the predominant atti-tudes of this group because they cover such a widespectrum. Most seem to believe, however, that theshortcomings of review are a natural consequence ofthe conflict between the individual's desire for sub-stantive creativity and judgmental freedom and theorganization's demand for conformity and com-promise. If the cynics had a motto, it probably wouldbe: "When you can fix human nature, then you'll fixthe review process." In an extreme form, some cyn-ics have almost a "sink or swim" mentality: havingput up with and succeeded in spite of the challengesposed by the review-process, they see no reason whyothers, particularly new analysts, cannot do so aswell.I call the last group the "optimist" school. It be-lieves that the process is not fundamentally defectiveand that its deficiencies can be remedied to somesignificant degree. The mark of an optimist is thewillingness to entertain and at times to implementsignificant changes or experiments in the reviewprocess. Most are cautious, however, because theyrecognize that tinkering with a basically soundprocess has its risks. In a perverse way, the optimistsin the DI probably are responsible for most of theshortcomings of the process because they have beenin the vanguard over the years in refining the system.But this should not deter us from suggesting or at-tempting solutions.Tackling the ProblemTo my mind, all of the problems with the reviewprocess in the DI boil down to two: one substantiveand one perceptual. The substantive problem is thatthe functions of the individuals involved in theprocess are not drawn clearly enough. This leads toa duplication of effort and unnecessary delays. What,for example, is the difference between the functionsof the branch chief and the division chief? Shouldthere be any difference? Among peers, where doesthe burden of persuasion lie with regard to disputedjudgments, with those requesting change or withthose resisting change? What is the threshold forincluding dissenting opinions in publications? In thecase of editing, where does the branch chief's role9--Gernfitend and the office editorial staff's begin? All of ushave different answers to these questions but nocommon set of guidelines.This problem could be substantially reduced if eachof the participants in the process concentrated on thatfunction that they perform best. For example,analysts are best suited to perform research and draftbasic analysis. Branch chiefs have greater ex-perience, and they are still in touch with the subjectmatter on a routine basis; they are best equipped toperform a thorough substantive check. Higher-levelmanagers have greater experience, and they are morein touch with consumers; they are best able to exa-mine the product for focus and policy relevance. Andso it goes with all the other participants. Instead ofhaving the perspective of producers who "assemble"a product, with each contributing his or her share tothe as yet imperfect whole, we often seem more likeartists who feel that the work of art can only go for-ward in "Perfect" form (hence the stylistic editing atevery level of review). We need more producers andfewer artists.The perceptual problem with the review process isthat each one of us defines it in a different way. Some-times these differences are significant. The particularsof the process vary widely across the DI, and that isas it should be. But there exists no authoritative frame-work within which to structure the details. In theMAG survey, analysts felt most frustrated that manage-ment did not follow the prescribed process. This is adirect reflection of the varying expectations betweenanalysts and managers about what review is and howit should function. This has sometimes contributed toa perception among analysts that, while they are heldaccountable for following the prescribed process,management has no such strictures.When analysts join the DI, they are expected to beable to do research, analyze, and write. Such expec-tations are reasonable because these skills are testedon the PATB and because they are required to makeit through the university system. It is curious,however, that we also seem to expect them to under-stand the review process without ever having beenexposed to it. In the Office of Training andEducation's (OTE) Analyst Training Course, a greatdeal of effort is rightly devoted to teaching newanalysts how to write in the DI style. But the reviewApproved for Release: 2014/07/29 000622860_Ccu4141errttaiti?.!Approved for Release: 2014/07/29 000622860process receives little attention, even though an un-derstanding of it will be as important to an analyst'ssuccess as mastering the DI writing style. Similarly,in the Supervision of Analysis Course, newmanagers are taught how to review papers but notabout the review process itself. OTE is not responsi-ble, however, for this state of affairs; the DI has noformal review concept that could serve as a basis forsuch instruction.Recommendations for ChangeAs a first step, the DI should develop and promul-gate a directorate-level product review process con-cept, with which all individual office review proce-dures must be in accordance. The broad outlines ofsuch a concept have already been defined, most re-cently in the DI Notice Guidelines for ProductReview, Coordination, and Incorporation ofAlternative Views (DI N 20-205, 2 June 1992). Theynow need to be developed further and formally pub-lished in a manual or similar document. This effortshould involve both managers and analysts, and itshould focus on elaborating an overarching conceptof review that can be applied across all of the DIline offices, rather than establishing particular reviewprocedures, which should be left up to the individualoffices.Once developed, this concept should be integratedinto OTE training for analysts and managers in theDI. For analysts, this could be accomplished primar-ily through a block of instruction in the AnalystTraining Course, which is required for all newanalysts. It could also be presented in the Workshopfor DI Midlevel Analysts and the DI WritingWorkshop. Managers could receive instruction duringthe Supervision of Analysis Course, which is manda-tory for all new branch chiefs. The DI reviewprocess concept could also be included in an analysistradecraft manual, development of which was calledfor in another recent DI Notice. Over time, the con-cept would permeate the directorate and become itsinstitutional philosophy. And, unlike the current situ-ation, it would be explicit and commonly held. If ex-perience so warrants, the concept could and shouldbe modified as conditions change.ProcessToward a DI Review PhilosophyThe DCI's task force on Politicization andIntelligence Production .has already taken steps to ad-dress many of the concerns raised here. Their effortsare a good start toward developing a more clear-cutphilosophy of review in the DI. Some of mythoughts on the possible substance of such aphilosophy follow:? The central purpose of review is to produce aproduct that represents the institutional positionof the CIA.? All other things being equal, fewer levels ofreview are better than more. There should exist apresumptive requirement to demonstrate that anew level of review will significantly improvethe product before it is created.? Review begins at the conceptual stage of a paper,not when analysis and drafting are completed,and it carries through to the final physicalproduction stage.? For the analyst, review is as much about givingup his or her sole control of the product as it isabout getting others to take ownership of it.? Even though the analyst plays the primary role incarrying out the day-to-day procedures of review,management should bear the ultimate responsibil-ity for the success or failure of the process.? All parties should be held accountable to the es-tablished review process, except in the most un-usual circumstances.? There is no room for ego or literary hubris in thereview process. Intelligence production in the DIis a team effort in which compromise, next to thepursuit of truth, is the most highly valued com-modity.? Review is logically distinct from research but notfrom analysis. The intellectual scrutiny to whichproducts are exposed during the process is an in-tegral part of the final published analysis.10Approved for Release: 2014/07/29 000622860Approved for Release: 2014/07/29 000622860Process __Sizorifideatierl--- Greater emphasis should be put on timeliness.According to the recent DCI task force report onproduction, consumers believe that timeliness issometimes a problem with DI products, and theygenerally value it more highly than quality ofcontent.? To reduce duplication of effort and enhance time-liness, participants in the process should concen-trate on the task or function that they aloneamong the participants are best able to perform.This article is classifiedELDENT517.?,11 ,Gerriltia IApproved for Release: 2014/07/29 000622860