AUTOMATED COMMUNICATIONS TERMINAL (ACT) PROGRAM EVALUATION

Document Type: 
Collection: 
Document Number (FOIA) /ESDN (CREST): 
CIA-RDP79-00498A000200040005-0
Release Decision: 
RIPPUB
Original Classification: 
K
Document Page Count: 
37
Document Creation Date: 
December 9, 2016
Document Release Date: 
October 24, 2000
Sequence Number: 
5
Case Number: 
Publication Date: 
March 18, 1976
Content Type: 
REPORT
File: 
Body: 
Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 EXECUTIVE SUMMARY An evaluation of the Automated Communications Terminal (ACT) located in the Headquarters Signal Center has been completed using the generalized program evaluation criteria given in the Appendix. The ACT system is a computer-based system developed to auto- mate manual message processing activities. It has been recog- nized that the existing manual methods would require continual increases in personnel and hardware to ensure adequate communi- cations service. The basic objective was to achieve greater message processing speed and accuracy using fewer personnel than would be required if'the manual methods were continued. The ACT system was declared operational in March 1913 and has satisfied all technical objectives. The number of per- sonnel assigned to message processing functions was reduced from 150 to 127, while message volumes increased by 19% and message lengths by 20%. Message processing times were reduced by at least 65%, which resulted in the reduction of queues from 400-500 messages at peak periods before ACT to essentially none after its implementation. Additionally, it was shown that ACT-reduced the annual operating costs of the Headquarters Signal Center by $275,000. A substantial number of problems did develop during the program and are attributed to several factors: a. The contractor's inexperience in communications- oriented systems; b. The contractor's decision during the program to discontinue further efforts in computer-based systems development; c. The lack of adequate definition in the specification of all functional requirements. While the net result of the ACT program was considered to be beneficial, a number of recommendations were made that might enhance similar efforts in the future. These include more careful scrutiny of prospective contractors and the establishment of standardized, legally-based proposal evaluation criteria that Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 would more critically evaluate contractors' capabilities; consideration of incentive award type of contracts for programs of this magnitude; inclusion of penalties to cover additional costs to the Government if specified contractor support is not provided; and minimizing hardware specifications to those absolutely essential to maximize bidder latitude in system design. Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 s Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 AUTOMATED COMMUNICATIONS TERMINAL (ACT) PROGRAM EVALUATION Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 1. Introduction The Automated Communications Terminal (ACT) is part of a program to use technology to replace manual methods employed at the Headquarters Signal Center to handle narrative message traffic. The overall plan includes the automation of the relaying, processing, disseminating, repro- ducing, and distributing of narrative record traffic. The relaying functions have been modernized with the installation of the Message Automation Exchange, MAX-II, the message originating and terminating processing functions have been automated with the installation of ACT, and the disseminating function will be automated with the Cable Dissemination System scheduled for acceptance in 1976. A review of available systems has shown that replacing the present reproduction and distribution facilities cannot yet be done cost effectively. The ACT system was declared operational in March 1973, almost three years after the contract was signed and eighteen months behind schedule. Since then, a number of modifications have been. made to the system to enhance overall performance. The following pages describe and examine the evolution of ACT, from the initial planning back in the early 1960's through the present. 2. Background The original thinking about automating many of the manual procedures employed in the Signal Center for processing narrative messages started in the early 1960's, with. active planning beginning in the 1964 time period. It had been recognized that the increasing traffic volumes would continue to place greater demands on the Signal Center to ensure adequate service. The present methods would require perpetual expansion in terms of the number of personnel and equipment. It was decided. that plans shoul.d be formulated to automate what was then called the Washington. Terminal Facility (WTF), which consisted of the originating and terminating message processing functions for Headquarters' Staff traffic. Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Since most originating and terminating messages funnel. through the Cable Secretariat (CS) for dissemination, reproduction, and distribution, traffic flow is some- what distributed between it and the Signal Center. With the plans to automate most of the manual procedures in the Signal Center, the Cable Secretariat, which was then a staff function of the DCI, became concerned that their message processing queues would significantly increase. They requested that the Office of Joint Computer Services examine the feasibility and practicability of automating their activities. It was concluded and recommended in April 1966 that the only cost-effective approach would be to amalgamate the systems required for the WTF and CS. A joint task group, consisting of representatives of the Cable Secretariat, Intelligence Watch, Office of Joint Computer Services and Office of Communications was established to fully examine the problems and define an operational specification. This effort continued. until January 1969 when it was determined that the complexity of the combined system was such that budgetary limitations would preclude implementation. It was agreed, therefore, that a three-phase sequential program would be initiated. Phase I was to include the automation of the Washington Terminal Facility, Phase II would basically automate the analyst's portion of the Cable Secretariat (CS) and Phase III would automate the CS printing and reproduction facility. Much of the work accomplished before this separation of activities was used in preparing the specifications for both the Automated Communications Terminal and the Cable Dissemination System. The remainder of this evaluation only considers the ACT program as conceived in Phase I. 3. ACT Functional Description The ACT system is a multi-programmed computer system configured to automate the message processing operations of the Headquarters Signal Center. The ACT, consisting of an Originating and a Terminating Section within the system, basically handles incoming and outgoing cable traffic. Compared to the old manual systems, ACT provides much greater message processing speed and accuracy, greater message storage and retrieval capabilities, and greater operational flexibility. Approved For Release 2001/03/03: ?CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 The originating message processing segment of the system performs several primary functions. These are mes- sage input, validation, routing, accountability, assembly, logging, and output to the Message Automatic Exchange (MAX-II). Other functions, including the output of messages to local printers/paper tape punches, message retransmission and readdressal, logging, message retrieval, and other general system control operations, are performed as required in con- junction with the primary functions. Outgoing messages are processed by an Optical Charac- ter Reader (OCR) which scans and converts the typed letter to electrical signals. The electrical signals are sent to the outgoing computer which will assign numbers, build the message in the correct format, and check various parameters for validation. If corrections need to be made, the message will be displayed on an originating Visual Display Unit (VDU) for operator processing. After processing has been completed, the rr,~ssage is sent to MAX, then on to the addressed stations. Terminating messages are received by ACT from MAX-II. The system ensures that there are no format errors in the message and, once accepted, automatically validates, accounts for, and logs the incoming message. Processed messages are generally sent to one of the two high-speed printers located within the Cable Secretariat Branch for dissemination, reproduction and distribution. When operator intervention is necessary, messages are displayed on a VDU for operator processing. 4. ACT System Evaluation 4.1 Original Objectives The procurement specification that was the culmination of the many man-years of planning that went into the ACT system stated the following as the general objective: "ACT shall achieve greater message processing speed and accuracy than is presently attainable. ACT shall process present and projected traffic loads with fewer personnel than would be required if the existing manual system were to be continued." Specific objectives included in the planning documents for the Phase I automation of the WTF included the following: a. To fully automate those functions that were done manually, thereby eliminating heavy backlogs of message traffic. Approved For Release 2001/03/03 : IA-RDP79-00498A000200040005-0 -8- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 b. To eliminate the use of overtime needed to reduce backlogged message traffic. c. To be able to handle increased volumes with- out an increase in personnel. d. To speed the in-house handling times in the processing of messages. e. To effect other personnel savings wherever possible. 4.2 Did Objectives Change? The ACT performance requirements given in the procure- ment specification were based on satisfying the origii.al system objectives and remained valid throughout the contract. While numerous changes and amendments did occur during the contract cycle, these were basically to ensure compliance with stated performance specifications. 4.3 Were Objectives Attained? The ACT system was designed to satisfy the objectives cited in Section 4.1 above. The following paragraphs describe the extent to which each of the stated objectives was ful- filled. a. The primary purpose of the ACT system was to fully automate those functions in the WTF that prior to ACT were done manually and, in so doing, to eliminate the heavy backlogs of message traffic. This basic goal was certainly fulfilled. The Optical Character Reader replaced the M-28 teletype equipments used to manually prepare message tapes for transmission (poking). Disc files and magnetic tapes were used to replace the monitor reperforators for storing traffic and they also were used to replace all the card files required for manual logging of message traffic. Two high- speed printers installed in the Cable Secretariat area re- placed the multitude of M-28 teletype printers used to receive the traffic from each circuit for logging and. for- warding to CS. Visual display units are utilized to correct format errors vice the manual methods previously employed. Finally, the entire message processing functions that required, among other things, preparing the transmission header was replaced with a computer program that automatically prepares the transmission format and inserts routing indicators, message accountability numbers, etc. Approved For Release 2001/03/03 r-gLA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 The Optical Character Reader has eliminated 92.50 of the poking workload, that is, only 7.5% of the outgoing messages received in the Signal Center cannot be read by the OCR and require manual tape preparation. Of those that are read, 66% are error free and do not require any further manual processing; however, the remaining 340 do require operation intervention because of handwritten changes, preparation format errors, or just imperfections in or smudges on the paper. The 7.5% that have to be manually poked con- sist of 6.7% that are done by pre-arrangement (i.e., not prepared in OCR format, such as Intel cables), and only .8% that are OCR rejects. Some Signal Center activities still require some form of manual processing. For example, certain categories of traffic are routed by MAX-II and ACT to M-28 teletype receive reperforators for either distribution on dedicated "room" circuits to selected offices or insertion of additional routing indicators for expanded distribution. The effect of the above on traffic processing was significant. The average length of time it took to process a message and forward it for onward transmission was reduced by at least 65%, as shown in Table III in paragraph d. below. For example, the average elapsed time for priority messages was reduced from approximately three hours to one hour, and for routine messages, twelve hours to three hours. While no specific queue figures are avail able, it was general opinion that before ACT was implemented 400-500 messages would be in outbound queue on a Friday afternoon, and it would sometimes take a substantial portion of the weekend before the queue was eliminated. Today, Friday afternoon message backlogs are considered insignificant. Terminating message queues were usually not as evident as originating. Traffic, received on M-28 teletype printers, was processed and forwarded to the Cable Secretariat in an average three to four minutes per message, which is just a fraction of that required for originated traffic. Another basic reason for minimal queues is that traffic was received at the terminal facility on multiple low-speed lines from disc file storage at the relay (MZA.X-II) and sent to the Cable Secre- tariat via a pneumatic tube system. Today, most messages are sent directly to high-speed printers in the Cable Secretariat and the only queues in the Signal Center are contained in the MAX-II and ACT system disc files. -5- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 b. The elimination of the use of overtime needed to process backlogged message traffic also was accomplished. As noted in paragraph a. above, the messages in queue for processing have been reduced to insignificant levels. The specific overtime figures obligated for just this purpose were not identifiable, however, overall overtime expenditures were reduced from $39,745 be ore ACT (1972) to $25,294 after the implementation of ACT (1975), a decrease of approximately 36%. STATINTL STATINTL The above figur d be qualified, however. First, although they do include expenditures, over- time at that facility remained relatively constant over that period at approximately 800 hours per year. The 1972 figure has been adjusted for the approximate 5.5% annual increase in the Federal pay scale. Even with the indicated decrease in overtime, traffic volumes handled by the Signal Center increased by 17 percent during this period. In summary, then, the decrease in overtime STATINTL noted above is a conservative estimation and, while it is based on overall usage, the implementation of ACT STATINTL was the only significant c ange in _ operations over that period. c. The ability to handle increased volumes of traffic without an increase in personnel has been clearly achieved. Table I compares traffic volumes for 1972 and 1975 with the number of personnel assigned to the message processing functions within the Signal Center. As shown in the table, even with an increase in traffic volume of approximately 17 percent, and message length by 20 percent, a 15 percent reduction in personnel was attained. A slight off- setting factor, however, is the overall average grade increase from 8.7 to 9.0 refl increases in required job skills. STATINTL A study completed by in 1975 showed that if the manual methods used to proc ffic before ACT were still being utilized in 1974, 21 additional personnel would have been required. Approved For Release 2001/03/03 : CPA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 TABLE I Signal Center Volumes and Personnel Pre-ACT Messages processed in the Signal Center (WTF, SAF, PTF) Average Message Length (words) Operations personnel required for message processing Maintenance personnel required to support message processing equipments Automation Specialists (program- mers) required to support message processing Total Signal Center personnel associated with message processing Overall average GS grade level personnel supporting message processing Post-ACT STATINTL d. The fourth objective, to speed the in-house handling times for the processing of messages, was also successfully attained. Two levels of criteria were employed to show the improvements resulting from the ACT system. The first method compares average processing times for several categories of traffic to show that significant savings re- sulted. The second method of comparison shows the average elapsed times in the Signal Center for each priority of message; that is, the length of time it took a message to be sent to MAX-II at Headquarters for relay after being received in the Signal Center. -7- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 The first comparison is based on measuring the actual time required in each of the message processing functions. For example, the average time required to process an originating Agency staff message when using ACT is 4.2 minutes. This is based upon time and motion studies measured from receipt in the Signal Center through filing of the message after being read by the optical character reader. Included is an allowance for editing messages that are rejected by the ACT system and employee overhead (lunch, coffee breaks, supervision, etc.). The pre-ACT figures are based on the manual procedures required to process a message (validating, logging, poking, etc..), and also include employee overhead. Table II below compares pre- and post-ACT originating processing times for each of the indicated message categories: Originating Processing Times (Minutes) Category Agency Staff Special Designee Special Channel Telepouch Restricted Handling Intel 11.7 4.2 16.9 4.7 27.0 4.7 6.5 4.2 25.2 4.7 11.7 4.2 Post-ACT The second method used to evaluate the improvement in the originating process was to compare the average elapsed times in the Signal Center for each priority of message. Table III below shows the average time from when a message was received in the Signal Center for processing to when it was actually trans- mitted to the Message Automatic Exchange (MAX-II) for relay. These figures then include the queue times before processing, processing times, and queue times awaiting transmission to the relay (MAX-II). -8- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Priority Critic, Flash Immediate Priority Routine Telepouch Table III Average Elapsed Time - Outgoing Pre-ACT 14 minutes 1 hour, 35 minutes 2 hours, 52 minutes 11 hours, 50 minutes 72 hours Post-ACT 5 minutes 15 minutes 1 hour 3 hours 9 hours Terminating traffic comparison figures are difficult to obtain, primarily because in the ACT system most categories of received traffic are sent directly to two high-speed printers in the Cable Secretariat unless format errors occurred or special. handling was required. Even in the manual system, terminating traffic processing was not a significant problem with average mess- age processing time being just a fraction of what was required in the originating process. For example, an incoming Agency staff message required 3 minutes to process compared with 11.7 minutes in the originating process. Special Channel messages required 4.2 minutes on the receive side compared to 27 minutes on the send side. As a partial confirmation of the gains achieved by automating, a study done by showed that if. the manual system used prior to AC was used to handle 1975 receive traffic volumes, six additional personnel would be required to process the terminating traffic. The ACT system's ability to process and forward messages more rapidly also is an indication of the Signal Center's ability to handle crisis workloads without increases in personnel and without impacting other activities. e. The last objective was to effect further personnel savings whenever possible by automating other manual activities in the Signal Center. After the manual functions of the Washington Terminal Facility were replaced with ACT, the system was modified to include the major portion of the message processing functions in both the Special Activities Facility and the Project Terminal Facility. The personnel figures shown in Table I above include these reductions. STATINTL Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 4.4 What Was the Impact of ACT on Annual Operating Costs? The comparison of specific operating costs for Signal Center message processing functions can only be accomplished at a very gross level because many of the pre-ACT expenditures were combined with costs for other activities. Table IV below compares the expenditures for each of the indicated categories. TABLE IV Comparison of Signal Center Annual Message Processing Costs Category Pre-ACT Post-ACT Personnel services (excluding O/T) Overtime 39,746 25,294 Maintenance and service contracts 92,812 91,467 Supplies and materials 52,840 26,503 Technician and programmer training 0 18 4.61 Total ANNUAL SAVINGS $274,059 STATINTL STATINTL The costs shown for personnel services only include the salaries for the personnel, including supervisors, directly associated with the message processing functions in the Signal Center (ACT or WTF, SAF, and PTF). Personnel (operators, technicians and programmers) assigned to MAX - I not in- cluded, nor are logistical personnel or other Staff STATINTL pro-rated. It was necessary to include both S Activities and the Project Terminal because ACT did absorb significant portions of these facility workloads. The pre- and post-ACT dollar amounts are based on the number STATINTL of personnel shown in Table I (obtained from the Position Control Register and Staff input) and the 1975 salaries for the ave ade level calculated from the manning tables. Approved For Release 2001/03/03 : C11A_RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 The overtime figures were derived as noted in Section 4.3, paragraph a. Annual costs for maintenance and service contracts include the PRI teletype maintenance contract, as well as the service contracts covering ACT.hardware and peripherals. Costs before ACT were increased by an annual 5.5% to represent cost increases resulting from inflation. Supplies and material costs were derived from the 1972 budget. and were also increased by an annual 5.5% to more closely resemble 1975 dollars. Figures for after ACT were determined from actual current usage. Training before ACT was primarily accomplished internally; however, after the implementation of ACT, both technicians and programmers required external training. Comparing the overall expenditures for Signal Center message processing before and after the implementation of ACT, the above table shows an approximate annual savings of $275K, the bulk of which can be attributed to reduced person- nel costs. 4.5 Overall ACT System Cost The total fiscal obligations associated with the ACT system include a number of ancillary expenditures in addition to the contract cost. These include both personnel and hard- ware costs not part of the original contract and all upgrades after the system was implemented. The following paragraphs describe these additional expenditures, with paragraph 4.5.5 summarizing total obligations. 4.5.1 ACT Contract Cost STATINTL The original contract was amended eight times resulting primarily from c ari ications`^n the specifi- cation or enhancements to the proposed . syst,em,, The following summarizes the ACT contractual obligations: STATINTL Approved For Release 2001/03/03 : Cj RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 c. Amendment #2 - Provide instruction not included in original contract (for training in Washin ton, D.C., area vice contractor facility) . . . $3,400 d. Amendment #3 - Clarification of contract . . . N/C e. Amendment #4 - Provide maintenance training not included in original contract - $16,820; provide operator training - $8,550; provide mounting tables for use with VDU's - $2,340; provide additional spares - $10,600; Total . . . $38,310 f. Amendment #5 - Card reader to facilitate program changes . . . $4,670 g. Amendment #6 - To allow CPU #1 or #2 to have background operation when failover has occurred or one CPU is in standby. Will provide better maintenance support and allow use of on-line peripherals . . . $34,500 h. Amendment #7 - To provide a modification for five-level paper tape units to provide a means for software detection of a "low-tape" or "out-of-tape" condition . . . $3,800 i. Amendment #8 - Increases to contract resulting from scope clarification for originating and termina- ting software . . . $25,028 Total Contract Cost . . . $1,736,176 4.5.2 Ancillary Hardware Costs A number of additional hardware expenditures were required as a result of the ACT installation. The biggest single line item was the shielded enclosure costing $78,580. Two air handlers were procured for the enclosure to rovide the proper temperature and humidity at a cost of $12,500. Optical isolators had to be installed to interface with remote printers at a cost of $2,400. Finally, modifi- cations to th shielded enclosure were required at a cost of $3,913. Approved For Release 2001/03/03 : CIS-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Additional spare parts not covered in the original ACT contract were procured at a cost of $48,034 to facilitiate maintenance. These. included a separate disc assembly and a visual display unit, as well as a multitude of smaller components. Special time stamp machines were procured at a cost of $5,000 for inserting the Message Reference Number and the Date Time Group on outgoing messages in an OCR compatible type font. The compatibility of typewriters in the Headquarters area with the ACT Optical Character Reader was also necessary. Fortunately, it has already been recognized by an OL task group that the non-standardization of typewriters was becoming increas- ingly inefficient and changes were necessary. The numerous type styles in use in the Agency caused delays and frustrations when minor corrections or additions had to be made to corres- pondence when originated in a different location with a different type style. Furthermore, the Office of Joint Computer Services had already implemented an optical scanner for pro- cessing documents that required a unique type font. The decision was made, therefore, to standardize to the extent possible on the IBM Selectric II typewriters. Since there were a number of other parameters involved in this decision than just the implementation of ACT, direct cost apportion- ments are impossible to determine. For exam le, while the cost of the IBM Selectric was approximately M. more per unit than others being considered, the interchangeability of type fonts (each font costing $18) resulted in substantial savings at the numerous locations requiring multiple-tyne styles. In total, the additional hardware cost associated with the implementation of ACT not covered in the actual con- tract price is $140,427. 4.5.3 Ancillary Manpower Costs While every program requires a certain amount of manpower for system definition, specification writing, proposal evaluation, and contract monitoring, these costs are generally considered to be overhead, unless external assistance is used. Additional manpower costs included were those resulting from training the various maintenance, operational, and programming personnel who will be responsible for the system once it is declared operational, and staff personnel assigned to the contractor to assist in the development of the system. Approved For Release 2001/03/03 : CTA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 The basic contract included the training of personnel in the operation, programming, and maintenance of the ACT system. As the development progressed, the contract was amended as noted in Section 4.5.1 to include increased operational and technical trvining. It was subsequently decided to again expand training in all areas at a total cost of $46,000, including travel and per diem. It was also necessary to train Agency secretaries in the proper procedures for preparing cables and telepouches for electrical transmission with the new optical character reader. A special handbook and supplemental training were provided for this purpose using in-house resources. Three computer programmers were assigned to the contractor to assist in the development of the softwai-a of the ACT system. After the first year, one programmer was reassigned and the other two remained on-site for an additional ten months. Salaries for these programmers are estimated at $56K and per diem amounted to an additional $28K, bringing total programmer costs to approximately $84K. In total, the additional manpower cost associated with the implementation of ACT not covered in the actual contract price is $130,000. 4.5.4 ACT Upgrade Costs The ACT system configuration has been modified as necessary to absorb additional activities and increase system flexibility. The first modification was the integration of Project traffic into the ACT system. This was accomplished with in-house soft- ware changes and the procurement of a line printer at a cost of $42,000 for remote printing application. This action resulted in the dismantling of the old teletype equipment dedicated to the Project Terminal Facility. An additional disc file assembly was procured at a cost of $35,000 for on-line application. The standby processor had previously been of limited utility for performance of diag- nostics, assemblies, etc.; however, the provision of a dedicated disc file to this processor allowed these functions to be performed with less system disruption. Although it was initially decided to lease the Lundy- Farrington Optical Character Reader at a cost of $55,000 per year (including maintenance) because of anticipated changes in the OCR market and uncertainty regarding its overall performance, it was eventually concluded that the system was adequate and that substan- tial savings could be made if it was procured. An agreement was Approved For Release 2001/03/03: C4A-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 reached with the company to purchase the OCR for $67,000. The OCR portion of the ACT system provided no redundancy and it does require extensive maintenance. the designed interface between the OCR and the computer portions of the, fi AC.T,..system 3could not be readily accommodated within the planned Cable'blssemination System. Therefore., plans were initiated. in.Fl-74 to procure a new OCR subsystem that was easier tomain- .~v.,o.. #n, could be used with the existing ACT system, and would be interfaced with CDS. This system is presently being tested on=?gtte. A second identical system is planned for procurement the latter part of FY-76. Total cost of the upgrade is approxi- mately $130,000. A two-year contract has been established at a cost of $23,5^0 to upgrade the ACT visual display units in an effort to enhance overall system maintenance capabilities. Total cost associated with upgrading the ACT system since its implementation is $297,500. 4.5.5 ACT System Overall Cost The previous paragraphs highlighted the main expen- ditures incurred for the ACT system since the solicitation of bids. As noted previously, these costs only include the initial implementation costs and system upgrade costs, and l do not include annual o orating expenditures. The overal STATINTL ACT system cost is based upon the following: STATINTL a. Final ACT contract cost - b. Ancillary hardware cost c. Ancillary manpower cost d. System upgrade cost TOTAL 4.5.6 Comparison of Estimated Cost with Actual The ACT s stem as specified was estimated to cost approximately which compares ver favorably with both the origina contract cost of and the final amended contract cost of Also to be included, however, are the ancillary manpower and hardware costs STATINTL STATINTL STATINTL Approved For Release 2001/03/03 :.gl54-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 described above. Incorporating these fi raises the total cost of the ACT implementation to The $.3M ted with ACT system upgrade costs consists of items a STATINTL assocz required to both expand system capabilities and rectify defi- ciencies in the delivered system. Examining the items enumerated in Section 4.5.4, approximately $.12M can be attributed to basic system costs, bringing the total costs associated with STATINTL implementing the originally specified CAT system to approximately 43% above the estimated STATINTL 4.6 What Ancillary Impact Did ACT Have? While the Automated Communications Terminal certainly provided significant message processing enhancement, its implementation did impact other activities within the Signal Center as well as every office within the Agency involved in the electrical transmission of traffic. Also affected were the personnel involved in processing traffic and the environment in which they worked. The following paragraphs describe these effects: a. Impact On Other Activities The primary interface affected by the ACT system was that with the Cable Secretariat (CS). While the originating message procedures were basically unchanged, terminating traffic was now received electrically by CS on two high-speed printers. Since messages were no longer being manually processed in the Signal Center, CS had to cope with a potential increased backlog resulting from their manual operation. After ACT was placed on-line, CS initiated a number of procedural changes that minimized their queues. Some changes were also made to ACT to make the CS operation more efficient, including having the messages paginated and classified as received on the high-speed printers. A substantial benefit derived is the ability for CS to receive priority messages more rapidly than was possible with the manual methods previously employed in the Signal Center. The overall workload impact on the Cable Secretariat was minimal, with positive gains being made in terms of disseminating higher-priority messages. Approved For Release 2001/03/03 : CTA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 b. Personnel Impact The implementation of ACT followed the auto- mation of the relay functions at the Signal Center (MAX-II) by a number of years and, therefore, the im act of a STATINTL computer-based environment was familiar to iW personnel. Supervisors were briefed on the difficulties encountered as a result of the MAX-II installation and were thus able to avoid many of the problem areas experienced during the earlier automation effort. It was general opinion that employee morale has been enhanced by the automation of the manual message processing procedures. The basic problem encountered was that of motivating personnel not selected to initially operate the ACT system and were, thus, still engaged in the tedious, manual procedures utilized in other areas of the Signal Center. The positions assigned to ACT were considered to be more prestigious, primarily because of the association with a new computer system that still provided substantial interaction with the traffic but at a much more sophisticated level. This exacerbation of employee morale has virtually been eliminated since the implementation of ACT because personnel are being rotated between the various sections, and because advancing technology has resulted in the installation of more sophisticated equipment in most other operational areas. The boredom factor that sometimes occurs from the automation of procedures that had previously been accomplished manually did not result when ACT was implemented. A significant amount of operator interaction is still required by the system, especially on the originating side. Also, the differing procedures for each category of traffic require continual review and update. Any spare time is generally spent in self-training, either in ACT system operation or related areas of interest. Another area of significant impact is the operating environment in which personnel must function. A new computer installation generally provides a more conducive atmosphere in which to work (better lighting, fancier decor, etc.); however, inadequate human engineering can seriously reduce performance. Areas of concern include the interaction with equipment, climate, and noise. Approved For Release 2001/03/03 : 6A-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 The designed human interaction with the ACT system has resulted in relatively few problems. The main areas of interface occur at the optical character reader and the visual display units. The basic problem area in the OCR is the designed editing capability which requires excessive human movement to adequately correct messages. The new OCR's being implemented will rectify this shortcoming. The VDU's were specially designed for the ACT application and have been well received by all personnel. Both the temperature and humidity are controlled s-Q,mputer e3rViT~5i`iment and it is only the former that presents _a problem. _"Tl e equipment in the ACT shie1d-ed enclosure is cooled entirely by room air temperature rather than by direct chilled water piping to the major sub-assemblies. As a result, the temperature is kept considerably cooler than would other- wise be necessary. The situation is..__aggrava,tedby the manner in which thE: air is distributed, namely, forced air flow under the computer floor. Being uncarpeted,,.the communicators ax.e, directly exposed to the steel flooring for most of the eight-hour shift, which has resulted in claims of medical problems by some of,_th,e, personnel. Another problem area is the noise level resulting from the multitude of equipment operating in the closed environ- ment of the shielded enclosure. While soundproofing is used on the walls, the combined effect of the air handlers, high-speed printers, and multiple blowers, results in a relatively high noise level which has also resulted in some medical problems. c. Environment Impact In this era of declining natural resources, new programs should also be examined in terms of their environmental impact. The factors considered include space, power, and waste The equipments required in the Signal Center for the narrative message processing functions before ACT was implemented consisted of rows of teletype machines used to prepare paper tapes for transmission, print copies for dissemination and reproduction, and produce paper tapes for storage of messages. These functions were replaced with an optical character reader, computers, disc files, magnetic tapes, visual display units and high-speed printers. The resultant space required for the ACT system and the teletype machines still retained for backup in the event of an ACT Approved For Release 2001/03/03 : LIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 failure is approximately 2,000 square feet, as compared with approximately 1,000 square feet for the manual equipments. It should be noted, however, that the traffic volumes increased by 17% and, therefore, additional equipment would have been installed. Also, the square footage saved by absorbing the manual processing functions of the Project Terminal Facility and Special Activity Facility must also be considered. In total, it is estimated that the implementation of ACT resulted in 25% more space being utilized. The electrical power consumed processing staff traffic before the implementation of ACT was approximately 11 KW. This compares with 25 kW required by the ACT system. The pre ACT figure must also be modified to include the equipments required to handle the increased traffic volumes and the absorption of additional activities. It is estimated that there was at least a 50% increase in. electrical power consumption. Another power consideration is the air conditioning requirement. The heat load associated with the manual system was 30,J00 BTU/ hour compared with 109,000 BTU/hour today; however the pre-ACT figure should be increased by the factors mentioned above. It should be noted that the teletype equipments and personnel were previously cooled by the Headquarters building central air conditioning plant, whereas the cooling for the ACT hardware and personnel is obtained from a separate A/C system dedicated to MAX-II and ACT. In total, it is estimated that the air conditioning load increased by approximately 100%. The waste associated with Signal Center operations consists of the various forms of paper used by the equipments. As mentioned previously, the system before ACT required a paper tape for the transmission of each message, and a separate paper tape for storage. On the receive side, multiple copy paper was used for filing and dissemination. purposes; also required were numerous files and logs for message accountability. The ACT system. optically converts the message to electrical form and stores it on magnetic tape. On the receive side, message storage is accomplished using the MAX-Il disc files, and single copy paper is used for dissemination purposes. The files for accountability are now kept on higher capacity mag- netic storage, however, the printing of logs, system status reports, etc., is necessary to ensure system integrity and recovery after a failure. While amounts given in Section 4.4 for supplies and materials show a 50% reduction expenditures and should be a fairly accurate indication of the reduced waste. Approved For Release 2001/03/031tIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 4.6.2 What Was the Impact on the Customer? The implementation of ACT has affected the customer in a number of ways. The enhancements to message processing have enabled traffic to be delivered to the field recipient on a much more timely basis. On the receive side, the automated interface with the Cable Secretariat has allowed the more rapid dissemination and distribution of higher priority messages. While these are both intangible benefits, the improvements realized with ACT have significantly reduced the overall communication transmission time, especially during crisis situations. As a result of ACT, certain changes were necessary in the cable originating procedure. Not only were unique type fonts required to interface with the ACT optical character reader, but special message forms requiring careful. preparation were also necessary. The successful adaptation required writing a special message preparation handbook and providing supplemental training for each secretary. Today, over 900 of outgoing cable traffic is interfaced directly with the ACT OCR. 4.7 What Limitations Exist in the ACT System? While the implementation of the ACT system certainly has had a positive effect on Signal Center operations, the system., as implemented, did have certain limitations that still exist today. A number of modifications were necessary both during the contract and after to ensure adequate system perfor- mance. The primary changes to the system centered around the procurement of a third processor for backup, performance monitoring, and facilitating of program changes. The basic problems encountered were: lack of flexibility regarding use of on-line peripherals; inability to use the primary computers for the same functions as the standby unit in the event they failed and the standby unit was on-line; and inability to automatically switch over to the backup system in the event of an on-line failure. All problems, except the automatic switchover, have been resolved either through contract amendments or changes after the system was implemented. At present, a failure in either of the on-line processors results in an interruption of traffic flow and the initiation of manual recovery procedures. Adequate procedures have been established to minimize the disruption to traffic processing. Approved For Release 2001/03/03-:20IA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Another basic limitation in the system resulted from restrictions imposed in the procurement specification on the core size. This design constraint requires processing routines to be paged into and out of main core for functional execution. Almost all of the applications programs and system reference data reside on disc storage which results in relatively slow processing times for each message. More importantly, the limited core size basically precludes further system expansion. This has impacted on the CDS program by requiring certain functions to be absorbed by this new system. Another basic problem area is the designed interface of ACT components. The entire system is wired in such a manner (daisy-chain) that the adding of components and the reconfiguring of the system is an extremely difficult task. Also, maintenace and replacement of many components requires scheduling system downtime since most major components cannot be worked nn while the system is on-line. In summary, a number of software and hardware changes have been made to the system to satisfy expanded functional requirements and to overcome those limitations that resulted from both the lack of finite detail in certain portions of the procurement specification and the lack of understanding on the part of the contractor. At present, there does not appear to be any major improvement necessary aside from the inherent limitations described above. 4.8 What is the System Life Expectancy? The life expectancy of the ACT system is limited more by circumstance than by equipment obsolescence. The Cable Dissemination System, to be implemented in 1976, will require an interface with the ACT system. The previously mentioned hardware/software limitations made this interface difficult to design and, as a result, the CDS contractor elected to absorb the terminating function of the ACT system into CDS at no increase in cost. Therefore, upon successful implementation of the CDS, the ACT terminating subsystem will become spare equipment. The replacement of the ACT originating subsystem is programmed for FY-77, also dependent upon the successful implementation of CDS. This will provide a consolidated system for handling most terminal functions associated with the Signal Center, allowing common data storage, hardware, and programming, resulting in savings in personnel, space and maintenance. -21- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 a ti Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 As an interesting point to note, the initial study completed by the Office of Computer Services in 1966 showed that the most effective way in which to automate the Cable Secretariat was to consolidate it with the automation of the Washington Terminal Facility. Limitations in the budget, as well as the complexity of the system for that time frame, precluded consolidating the two operations. In retrospect, the experience gained with the procurement and implementation of ACT by itself certainly justified that decision and should enable the consolidation with CDS to be accomplished in a much more effective manner. 5. Procurement Evaluation 5.1 What Resources/Expertise Were Used to Prepare the RFP? The procurement specification for the Automated Communications Terminal evolved after many man-years were expended examining the functions of the Signal Center and the Cable Secretariat. Included in these analyses were repre- sentatives from the Cable Secretariat, Intelligence Watch, and Office of Joint Computer Services, as well as personnel from all areas of responsibility within the Office of Communications. A number of discussions were also held with other U.S. Government organizations, however, these were primarily with Department of State personnel involved in their Automated STATINTL Terminal System program (ATS). A contract was established to review the proposed procurement specs ication and provide a cost and size estimate for the joint CS/OC automation effort. This study effort resulted in a series of recommendations for defining the ACT system more ex licitly; it also estimated total system cost at As a result of budgetary STATINTL estimates and concern for system complexity, plans were formulated to separate the Cable Secretariat requirement from those of OC. In January of 1969, it was agreed that OC would start preparing specifications for the automation of just the Staff traffic message processing center (Washington Terminal Facility). These specifications were prepared in-house; however, they were the product of the many man-years of effort ex ended in the joint CS/OC effort, including the STATINTL external study. -22- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 1 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 5.2 What Type of Contract Was Selected? In discussions with the Office of Logistics, it was agreed that the contract for the ACT system would. be a purchase/lease arrangement. All of the ACT system, except the OCR, was procured under a firm fixed price contract because it was determined. to be the most advantageous to the U.S. Government since the specifications were felt to be adequately efin_c~ It was concluded, however, that the optical character reader would be leased because it was uncertain how effective it would be and because such equipment rapidly reaches obsolescence. Leasing would allow equipment changes in the future to be more readily accommodated. 5.3 How Was the Contract Awarded? On 27 February 1970, representatives of 17 qualified companies were briefed on the requirements for the ACT and presented copies of the specifications. The companies were selected in concert with the Office of Logistics and Office of Security. Separate conferences were held with each of the interested companies and by the deadline, 6 April 1970, eight proposals were received. Two of the proposals received were submitted by combinations of companies, those being STATINTL Evaluation criteria used to compare prospective contractors were developed by Office of Communications personnel. First, a group of OC personnel evaluated the technical pro- posals on a paragraph-by-paragraph basis using the format shown in Figure A. Once this was accomplished, the contractors were ranked on the basis of their numerical rating. The cost proposals were then revealed to the evaluation team and the following comparison table was formulated. Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 ACT Preliminary Proposal Comparison STATINTL based on reduced hardware in their proposed system. It was determined that this change did not affect their performance rating. It was then concluded that the five highest-ranked proposals, of the eight shown in Table V, would be examined further. Of the three not considered, two were eliminated because of price and the third for being non-compliant. OC personnel visited the plants of each of the remaining five proposers to view first-hand the available hardware and discuss details of the proposals with knowledge- able personnel. Upon completion of these visits, a second set of proposal evaluation criteria was developed to compare the proposal performance specifications with ancillary factors related to contractor understanding and experience. The results of this review, given in Figure B, were incor- porated into the performance/cost comparison chart shown in Figure C. Based. on. these evaluations, it was determined that the three proposers who should be considered for final During the course of these final technical evaluations, each of the above-named companies voluntarily submitted reduced cost proposals based on further refinements of their proposals, reduced price quotations from vendors, and/or corporate decisions to absorb part of the development costs. After further discussions to clarify technical points in each of the proposals, the three companies were given a common deadline to submit a final price proposal. After receipt Approved For Release 2001/03/03 4 t1A-RDP79-00498A000200040005-0 STATINTL Approved Foe Release 200.1/03103 : C1 -RDP79-00498A000200040005-0 Olt Outnt >>:.11rti Reo1,orr.c OR - 3 point R - 1tnc;,E~r~:~iro R .. 2 poi_r,tn PR -- 1. point IF( 1T. }'#c'.ti111.1;; Krr.por:li?,*D 2~on Jtoc pan~ivr lR 0 poi.i;'ta R PR 1T. 2..1.1. - 2.1.1,, .2 2.1.5.1 2.1.5.2.2 2.1.5.2.3 2.1.6 2.2.1.1. 2.2.1.2.2 2.2.1.2.3 2.2.1.3 2.2_.1.1: 2.2.2.1 2.2.2.3.1 2.2.2.4.1 2.2.2.4.6 . 2.2.2.5 2,2.2.5.3 2.2.2.6 2.2.2.7 2.2.2.8 2.2.2.8.3 r 2.2.2.9 2.2.2.9.9 2.2.2.10 Figure A Brackotn 1 r,araf--aj hI group of r, -;.,_.:;.aa cY, c~.y:n catagca;,'. Approved For Release 2001/03/03: CIA-RD 9-00498A000200040005-0 STATINTL Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 5.5 What Problems Occurred During the Contract? The contract, which was awarded on 15 June STATINTL 1970, progressed reasonably smoothly for approximately the first 15 months until pre-acceptance testing was initiated. Up until that time, a number of problem areas were surfaced by both the COTR and the contractor, however, these were resolved and resulted in the first seven amendments to the contract itemized in para ra h 5.4.1. Pre-acceptance testing STATINTL for the ACT System began on 22 September 1971 and was scheduled to be conducted over a six-week period. After approximately one-third of the testing was performed, certain STATINTL undesirable operating characteristics were Eleven specific problem areas were identified and claimed that five of the eleven were beyond the scope of the contract, while the COTR claimed that only two of the eleven were. Many of these problems resulted from a lack of contractor understanding of the field of communications. This resulted in many misunder- standings and misinterpretations of specific terms used in the specification. Also a factor, however, was the lack of adequate definition for many of the functional requirements. After a series of meetings were held to resolve these differences, a number of concessions were made by both parties and these resulted in Amendment #8 to the contract. The agreement reached covered most of the technical deficiencies, however, the following considerations were made: the contract completion date was extended to 17 May 1972 vice 10.October 1971; the parts warranty and OCR leased maintenance were reduced to a five month period after acceptance vice twelve months; and the liquidated damages penalty was reduced to $250 per day and a maximum of $15,000 from $500 per day and a $30,000 maximum. It was also agreed to restart pre-acceptance testing on 27 January 1972. Further problems arose as the agreed-upon technical changes were incorporated and pre-acceptance testing, which finally began on 4 April 1972, did not conclude until 2 May 1972. Arrangements were made to ship the system to Headquarters, with final acceptance testing planned to begin on 19 June 1972. A number of additional delays were incurred due to further technical problems, including some mandatory Engineering Change Orders issued by the computer manufacturer. Final acceptance testing was concluded on 15 October 1972 with five major problem areas still outstanding. -27- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 a result Agency personnel, primarily programmers, na assume the responsibility for debugging the ACT system. In further negotiations a reement was reached regarding all outstanding pro Mems.. would resolve two software problems and would provide the services of their head programmer to verify that software changes made by the OC programmers were valid, if the contract warranty period would be effective 15 October 1972, and all liquidated damages would be waived. The ACT system was finally activated on 12 March 1973 for incoming non-Agency traffic. Processing of outgoing Agency traffic began on 9 April and, on 26 April, outgoing SI and Restricted Handling traffic were accommodated. Within two months, approximately 80% of Agency non-project traffic was being processed in the ACT system. As of this date, 92.5% of all Headquarters Signal Center mes?ages are processed through the ACT system OCR. 6. Summary and Conclusion The ACT system as implemented and modified fulfilled the basic objectives associated with its performance specifi- cations. The specific objectives cited in Section-4.1 and evaluated in subsequent sections were and continue to be valid. The overall improvement in message processing speed and accuracy has enabled increased traffic volumes to be handled with fewer personnel and less overtime. Comparing major operating costs before and after the implementation of ACT showed an approximate annual savings of $275K. The significant gains shown in message processing capabilitie$ at reduced annual operating costs certainly makes the implementation of ACT a worthwhile accomplishment. While rall ACT system costs were shown to be approximately no attempt was made to apportion this figure over 7e expected useful life due to the impact of the CDS imple- mentation and because no dollar figure could be attributed to improved customer service. Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 STATINTL STATINTL STATINTL STATINTL STATINTL Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Next 1 Page(s) In Document Exempt Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 STATINTL The basic reason for the many difficulties encountered was that - was primarily experienced in computer-based STATINTL information systems and not communications. Of secondary significance, but also a contributing factor, was the specification itself which, in retrospect, did not adequately define all of the functional requirements. The specification also placed limitations on some of the hardware in an. apparent effort to reduce overall system cost, however, this resulted in reduced system performance capability and flexibility. Lastly, the contractor's decision to discontinue further efforts in the field of computer-based communications systems resulted in the acceptance of a system that required sub- stantial in-house technical, operational and programmer support before it could be declared operational. In conclusion, the ACT system has significantly improved. Signal Center operational capabilities and although many difficulties were encountered, it is felt that the benefits certainly outweigh the trials and tribulations experienced during the implementation period. 7. Recommendations The many problems discussed in this program review were a result of a multitude of independent contributing factors. While difficulties are to be expected in any program of this magnitude, there are areas where changes could be made that might enhance future efforts. These include: a. More careful scrutiny of prospective contractors,. Such factors as long range corporate objectives, prior -31- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 experience in related ?fields, and overall capabilities? sheuId be evaluate efox,e a company is allowed to bid b. Establishing standardized, legally based pro- posal evaluation criteria that would more critically evaluate contractor capabilities and proposed performance specifications, thereby precluding necessary acceptance of low bidder based solely on meeting minimal performance standards. c. Ensuring that functional specifications and generic communications terms are adequately defined for all prospective contractors. d. Considering incentive award type of contracts., for programs of this magnitude to emphasize desirability of meeting specifications, schedules, etc. e. Incorporating penalties into the contract for added costs to the Government if the contractor does not pro vide specified on-going support (e.g., warranty,_maintenance, future hardware expansion capability). f. Minimizing hardware specifications to those necessary for interfacing with existing systems or providing standardization; other specifications should only be given if budget limitations require it and, then, only if performance implications are established. g. Establishing common contractor proposal submission, deadlines to ensure that all bidders are given equal opportunity to modify their proposals or bids. h. Ensuring, to the extent possible, that Government personnel remain assigned to these large programs throughout the procurement cycle to maintain contract continuity. i. Ensuring that the latest human engineering factors are incorporated into the system design to enhance the physiological and psychological operating environment. -32- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Program Evaluation Criteria: 1. System Evaluation: a. What were the original objectives? b. Did the final system configuration satisfy the objectives? c. Are the original objectives still valid? d. What was the impact on annual operating costs? e. How did final system costs differ from estimated? f. What ancillary impact did the implementation have? 1) For the Office: a) On other activities? b) Personnel? c) Environmental? 2) For the Customer? g. What limitations exist in the final system? h. What is the system life expectancy? 2. Contract Evaluation: a. What resources/expertise were used to prepare the RFP? b. What type of contract was selected? C. How was the contract awarded? 1) What were vendor responses? 2) What contractor evaluation criteria were employed? -33- Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 d. What problems occurred during the contract: 1) Legal? 2) Technical? 3) Functional? e. How did contractor performance affect the program? f. Did any Agency action impact on satisfactory completion? 3. Program Summary a. What were the overall accomplishments of the new system? b. Was the system worthwhile? c. What were the basic problems, if any? d. What conclusions can be drawn? e. What recommendations can be made for future programs? Approved For Release 2001/03/03 : CIA-RDP79-00498A000200040005-0 ? k M