A.D. LITTLE INC. SAFE EVALUATION FINAL REPORT

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP86M00886R001700240005-6
• Release Decision: RIPPUB
• Original Classification: K
• Document Page Count: 14
• Document Creation Date: December 21, 2016
• Document Release Date: January 12, 2009
• Sequence Number: 5
• Publication Date: July 12, 1984
• Content Type: MEMO

Approved For Release 2009/01/12 : CIA-RDP86M00886R001700240005-6
12 JUL 1984
ODP-84-1056
SAF-E207-84
MEMORANDUM FOR: Director of Central Intellige
1984
erector, Consolidated SAFE Project Office, ODP
SUBJECT: A. D. Little, Inc. SAFE Evaluation Final Report
The results of the recent evaluation of the SAFE Project by
A.D. Little, Inc. were presented in a briefing on 15 June 1984
by The briefing was given to senior CIA and
DIA managers. A copy of the final report is attached for your
information.
L -ate
Approved For Release 2009/01/12 : CIA-RDP86M00886R001700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
1
1
1
r
1
t
s
I
Report to
Consolidated SAFE Project Office
July 1984
At Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
r
t
t
11,
11
1.1 User Reactions
1.2 SUL Vs. AIM
1.3 Contractor Roles
1.4 Release Schedules
1.5 Schedule Realism
1.6 Degree of Software Uniqueness
2.1 Future SAFE 7
2.2 Project Performance Compared to Original Goal 7
2.3 Organization Changes 9
AL Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
INTRODUCTION
1
During March-June 1984, Arthur D. Little, Inc. performed a review of
Project SAFE. Its purpose was to provide the Consolidated SAFE
Project Office with recommendations about the overall management and
future course of the Project. Oral presentations of the results were
-made separately to CSPO, CIA, DIA, and the IC staff; this report is a
brief summary of the presentations.
The report is in two sections:
- Observations and Implications, in which we record our obser-
vations and relate them to our experience with comparable
technology in industry; and
- Overall Recommendations which result from consideration of the
implications taken together.
1
I
Our review was not conducted in great depth. We talked with user
representatives and-read surveys of user reactions, but did not talk
to significant numbers of individual users. We talked with the
managers of contractor activities, but did not talk with individual
contractor employees or study their work product. It is therefore
possible that our results contain inaccuracies of detail. We doubt,
however, that we have missed any major factor that might affect the
overall course of Project SAFE.
In December 1982, Arthur D. Little, Inc. submitted the written report
of a more comprehensive review of Project SAFE. References to it are
made in this report, but this report is complete in itself.
AL Arthur D. Little, Inc. (iii)
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 1
t
1
1
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
1. OBSERVATIONS AND IMPLICATIONS
1.1 User Reactions
Users in both CIA and DIA are generally satisfied with the function-
ality provided by Delivery 1 of SAFE. They have had some difficulty
mastering all of the capabilities, particularly in designing interest
profiles that provide adequate discrimination while at the same time
covering a broad enough scope, but this is inevitable with such a
versatile system and will pass.
The users have a variety of desires for new services. Some of them
are ideas for enhancements of the present mail and text function
group; others range across a variety of processing tools that might be
made available either in subsequent modifications of SAFE in the
library of CMS-based tools that the more sophisticated analysts are
already accustomed to using. Generally, however, user desires for new
services relate more to communications scope and interfaces with other
systems than to enhanced or novel functions. The analysts were
anxious to talk to one another in their own branches, in other
branches, and in other agencies where comparable work is performed; to
data bases both inside and outside the intelligence community; and via
SAFE to other systems and communications nets throughout the community.
We also noted that in DIA the users tend to be frustrated with the
installation approach that has been followed (putting relatively small
numbers of terminals in a large number of branches). Frustration
arises from the inability to share communications with one's fellow
analysts working on the same problem, to obtain help with the opera-
tion of the system.
As a result of these observations, we repeat the recommendation from
our earlier report that greater emphasis be placed on increasing the
rate of terminal installation than on the addition of new features.
It is true, of course, that Deliveries 2, 3, and 4 of SAFE cannot be
delayed beyond their present dates, though Delivery 5 might be. In
any case, we hope that funds can be found to accelerate the rate of
installation and training, especially in DIA. To facilitate the
latter, it might be possible to streamline the training course into a
series of short increments, to provide on-line assistance to users,
and to rely in part on a "buddy system" in which already-trained
analysts are asked to help others working near them who are only
partially trained.
Shortly before our study was undertaken, the decision was made to
substitute the AIM software and interface language for the
previously-adopted SAFE User Language. Our review of this decision
indicated the following.
AL Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
t
1
1
1
First, during the course of the deliberations on this issue, the
specifications of the AIM interface language changed in many important
ways: in the end they became very similar to those originally speci-
fied for the SUL. In our judgment, the remaining omissions from the
SUL are unimportant and are offset by the fact that their omission
simplifies the interface with the user. For instance, the interface
that was adopted includes the simultaneous display of two windows to
-the user, while the original SUL calls for four. We believe from
experience with personal computers in industry that it is not prac-
tical to attempt to keep track simultaneously of more than two windows
at a time.
Second, we observed that the processing architecture developed for AIM
has been extensively redesigned to improve flexibility, modularity,
and to improve throughput. As a result of these changes, it is
possible to be more confident than before that Deliveries 2 and 3 of
SAFE can be implemented without major technical difficulty because:
Deliveries 2 and 3 are now a major evolutionary step from existing
software rather than an entirely new creation.
These observations lead us to conclude that the correct result has
been obtained: that AIM with extensive reworks and modifications.
represents a better and more reliable approach to meeting the
objectives of Project SAFE than the original top-down comprehensive
design.
Although Deliveries 2 and 3 are evolutionary rather than revolutionary
products, they are nevertheless a very major step forward, and will
constitute the first real challenge to the software integration,
validation and verification, documentation, and installation capabil-
ities of CSPO. These appear adequate to us to meet the tasks that lie
ahead, but have not yet been proved by installations of the magnitude
of Deliveries 2 and 3.
System responsiveness may also pose difficulties. In Deliveries 2 and
3, the number of users and the capabilities available to them will
both be much greater. It may be that the user-responsiveness at peak
periods will deteriorate to an unacceptable degree. A similar problem
was observed before with Early Capability SAFE. The DAP solved the
problem then, but no comparable improvement is contemplated for
Deliveries 2 and 3 after their initial delivery. Predictive perfor-
mance modeling is possible, and we reiterate the recommendation of our
earlier report that CSPO undertake such modeling.
With the additions to AIM, Deliveries 2 and 3 have become very
comprehensive. They include all of the file, text and mail-handling
functions which are common to all analysts in DIA and CIA. Most of
the major additions to SAFE beyond Deliveries 2 and 3 are specific to
one Agency or the other. Those changes that will be occurring to the
common SAFE will be more of an evolutionary or additive nature to the
AL Arthur D. Little, Inc. 2
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 1
1
1
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
existing Deliveries 2 and 3 than major new products of the magnitude
of Deliveries 2 and 3. We suggest that they be considered the first
release of the complete basic or core SAFE.
1.3 Contractor Roles
In the past there has been some difficulty in meshing the contractors
:so that they work together at full efficiency. Examples existed of
over-management, of attention to details that turned out to be
irrelevant or superseded, and of a certain amount of unnecessary
communicating and reiteration of presentations. This situation has
apparently improved markedly, and we think it will not recur.
The underlying reason for the difficulties of the past was uncertainty
about the course of the Project. As long as the choice of user
language was undecided the contractors were unable to proceed in the
straight line manner envisioned in the Project plan. In attempting to
do so, they did unnecessary and redundant work. Now that it has been
decided that the user interface will be the augmented AIM, all parties
have been able to swing into line behind it and make steady progress.
In the meantime, it appears that the contractor personnel have learned
more about the Project, and that there have been sufficient changes
and additions to their staffs. We see no reason to make significant
further changes in the roles, reporting relationships, or distribution
of responsibilities among the contractors.
Typically, when an information system is complete, accepted, and put
into routine use, the Project team that developed it dissolves, and a
maintenance contract is let that (with renewals) lasts for the life of
the system. We therefore reviewed the possibility that a contractor
might at some future time take over the further maintenance of the
SAFE system. We think this will probably not be possible. Even
though major deliveries of new capabilities of SAFE may stop, major
evolutionary change will not. New capabilities will continually be
added to the core of SAFE (first delivered as Deliveries 2 and 3), and
new interfaces to community systems and networks are likely to con-
tinue evolving for at least ten years into the future. What is needed
is a long-term source of support, smaller than that devoted to the
building of a new system, but larger than that devoted to the opera-
tion and maintenance of a mature system that changes very little. We
think that such a source of support can only be found within the
community itself, though much of the implementation required can be
contracted outside. This subject is discussed further in Section 2.
1.4 Release Schedules
In the past, delays in SAFE have been caused primarily
by
difficulties
in reaching final agreements on specifications. There
has
been little
trouble with on-time delivery of pieces of the system
once
those
pieces have been finally defined. The specifications
for
Deliveries 2
AL Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
1
t
and 3 are now clear and bounded (in the sense that any changes
occurring from now on will only be minor trade-offs in details within
the bounds of what has been agreed upon). It should therefore be
possible to go forward in an efficient manner to completion of
Deliveries 2 and 3. The DIA-critical requirements for Delivery 4 are
now also clear and bounded. The integrated data base, DoDIIS inter-
face and the like have been fully defined, but some loose ends remain
in the specifications for Delivery 4. These mostly relate to possible
enhancements to the core functions of SAFE which are defined as part
of Delivery 5. In order that Delivery 4 have the maximum chance of
being delivered on schedule, we believe that enhancements to the core
of SAFE should be delayed if there is any possibility of their inter-
fering with the critical delivery date for Delivery 4. The SAFE core
should be frozen at the level of Delivery 3 until Delivery 4 has been
satisfactorily completed, unless it is certain that enhancements can
be incorporated on the basis of finished and proved work that cannot
possibly cause a delay.
Delivery 5 appears to us to consist of two independent parts that can
be separately scheduled. One part consists of enhancements to the
core of SAFE, as noted above, mainly in the areas of improved indexes
to text files. A second part consists of the implementation of
CIA-specific interfaces and conversions, such as the transfer of the
central index files to SAFE, the transfer of AIM users to SAFE, and
the addition of wire service and NOMAD interfaces. The CIA-specific
developments can occur at any time CIA wishes them to without rela-
tionship to the installation of Delivery 4 or to the enhancements that
may prove to be desired to the core of SAFE. We therefore suggest
that Delivery 5 be redefined as two independently scheduled parts: a
new release consisting of enhancements to the core of SAFE, and a set
of conversions and interfaces to be implemented within CIA at CIA's
convenience.
1.5 Schedule Realism
As noted above, the pacing factor in SAFE's scheduling has apparently
always been the ability to converge upon final and bounded specifica-
tions. Convergence has occurred for Deliveries 2 and 3, and if the
changes recommended above are implemented, will also have occurred for
Delivery 4.
Our discussions with the contractors indicated that each of them
appears to be satisfactorily conversant with the task assigned to him,
adequately manned, adequately managed by CSPO, and therefore likely to
be on schedule. (As noted in the Introduction, we did not audit the
actual work in progress, so we cannot verify the state of each con-
tractor's work on the basis of personal observation. However, our
cumulative experience leads us to place confidence in the contractors'
representations. We also observed that the overall scheduling of the
deliveries appears to be conservative, particularly in the time
AL Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
1
1
1
allowed for system tests. We cannot be sure that system testing can
actually be completed in less time than has been allowed, but it seems
possible.
Finally, we note that should Delivery 4 slip for reasons we cannot
foresee, there will be an alternative available to DIA. Apparently an
IBM-based system incorporating MVS software and the DoDIIS modifica-
.tions to the model 204 data base management system will be separately
implemented within DoD, and it will be possible for DIA to develop a
contingency plan to adopt this system.for its IDB, if Delivery 4 of
SAFE should slip. Therefore, considering the probability that
Delivery 4 will be on schedule and the possibility of having an
alternative system available, we can forecast with some confidence
that the planned 1986 shut-down of the DIALOS system appears feasible.
1.6 Degree of Software Uniqueness
The objective of Project SAFE is to be as industry standard as
possible: to use industry standard hardware and systems programs and
to develop only that software which must be unique to the intelligence
community. In our report of December 1982, we recommended a steady
effort to reduce this degree of software uniqueness.
We observe that the present plans for Project SAFE involve about the
same amount of unique software as we saw in 1982. The operating
systems are essentially the same, incorporating the same amount of
CIA-developed modification as was planned then. The enhanced AIM is
just as unique as the SUL implementation software would have been.
Among the modified industry packages, the INQUIRE text data base
manager is probably going to be implemented in a more standard manner
than was envisioned earlier, but the model 204 data base manager will
have some additional unique software which was not anticipated then.
Perhaps more important, we are less optimistic than we were in 1982
that the SAFE software will be overtaken by commercial products in the
near term. We forecast in 1982 that within five years all of the
functionality of SAFE could be obtained via commercial products. We
still think this will eventually be the case, but it will take more
than five years. The evolution of imaginative packages for personal
computers is continuing rapidly, but the necessary host-based network
control capabilities are not. There is no software system in industry
that matches the mail routing, full text searching, and individual
profiling capabilities that have already been implemented in Delivery
1 of SAFE, and there will not be for several years. Industrial users
of office automation software are slowly involving increasing require-
ments for electronic mail, but are unlikely to reach those of the
intelligence community for some time. Commercial software will not be
able to replace Project SAFE software in its entirety, until both the
interface software for the personal computer and the network control
software comparable to SAFE's mail management functions are combined.
AL Arthur D. Little, Inc. 5
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
1
s
1
t
We think this combined set of capabilities will not be available from
industry until about 1990.
When the full capability of SAFE is available commercially, it will
probably be implemented in a different manner. By then intelligent
user terminals will probably all be personal computers with great
compute power and memory size: the user windows, user-specific files,
and maintenance of user states will probably be managed primarily
within the user-dedicated personal computers, rather than in the
central systems, as in SAFE. The central systems will be reduced to
providing resources: the switching and screening functions for mail
passing through the center, and the control of centralized multi-user
file resources. To put it simply, activities specific to the indi-
vidual user will be located at the user site, and resources that
support multiple users will remain centralized. Sometime around 1990,
then, the architecture of SAFE should be reimplemented in a more
distributed manner as part of a convergence on industry-standard
software.
We have assumed that the eventual standardization of Project SAFE will
be to purely commercial standards. This will never be entirely
possible, if only because the intelligence community will continue to
have certain unique communications protocols and encryption
requirements. It also begins to appear that standards for data
representations, packet switched networks, protocols, and command
languages are beginning to evolve in major segments of the intelli-
gence community: these may eventually become standards throughout it.
If this happens, the intelligence community will have its own set of
software and system standards which might form a more desirable focus
for the standardization of Project SAFE than purely commercial
standards will, while at the same time providing a large enough
economic base for the support of SAFE's evolution to obtain industry-
type economies.
For the foreseeable future, then, it will be necessary for the evolu-
tion of SAFE to be supported by resources internal to the intelligence
community. CIA and DIA can take over responsibility for their
specific interfaces, conversions, and implementations of SAFE within
their respective communities, but there remain the enhancements or
subsequent releases to the core SAFE software of Deliveries 2 and 3,
which should be performed by a single group. It appears to us that
this group should be associated with CIA for at least five years, not
so much because of the uniqueness of SAFE application software as
because of the CIA-specific nature of the system software base upon
which it is built. Section 2 of the report discusses this in further
detail.
AL Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
1
2.1 Future SAFE
After core SAFE is delivered (Deliveries 2 and 3), future evolution of
it should be in the form of new releases that add to rather than
.replace it (perhaps on an annual basis). Further, after Delivery 4
agency-specific additions and modifications to core SAFE should be the
responsibility of the respective agency.
Because of the importance and complexity of Delivery 4, CSPO should
retain responsibility for it. However, DIA has already taken
responsibility for the DoDIIS interface and query language, and after
Delivery 4 is installed and the conversion from DIAOLS has been made
it should be reasonable to expect DIA.to support its own needs (as
long as the people it has assigned to CSPO return to DIA at that time).
After Delivery 2, CIA should take responsibility for the CIA-specific
parts of Delivery 5 (interconnections to DIA systems, conversions, and
integration with CIA indices and networks).
The other part of Delivery 5, which constitutes changes and additions
to core SAFE, should be regarded as the first major additive release
to core SAFE, but not as a replacement. Project management for the
new release should be streamlined and simplified to reflect this view
of evolutionary releases rather than new products: it should be
possible to adopt a more "market-oriented" approach to the evolution
of core SAFE, in which user feedback is the primary source of direc-
tion and contracts are in relatively small increments, forming a
continual flow.
The result would appear as in Figure 1. Figure 1 shows calendar years
1983 through 1987, and the deliveries of Early Capability and Delivery
1. Deliveries 2 and 3 would form release 1 of core SAFE, with
responsibilities for Delivery 4 and the CIA-specific part of Delivery
5 moving to those Agencies after Delivery 4 occurs. Then, future
evolution of core SAFE would be in the form of additional releases.
As noted in Section 1, these may eventually involve a complete rework
of SAFE architecture with personal computers becoming the user state
machine, and/or with the resource providing portions of SAFE becoming
standard modules for networks used throughout the intelligence
community.
2.2 Project Performance Compared to Original Goal
In Project SAFE Report to Congress of 23 September 1982, it was
forecast that SAFE would be complete in fiscal year 1987 for an
additional $192 million investment. The evolutionary plan presented
in Figure 1 can be compared to this original goal.
AL Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 M M M M r Approved For Release 2009/01/12: CIA-RDP86M00886RO01700240005-6 ^ r
FUTURE SAFE
io
ti
DIA
CIA
DoD HS, Modules, interface
Early SAFE
Release 1
CIF, interfaces
(wire services)
IDB
Full SAFE/C
1983 1984 -Cy 1985 1986 1987
(PC User State
Machine?)
(IC Standard
Module?)
.l
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
1
1
1
The functions SAFE performs in 1987 will be somewhat different from
those originally envisioned. SAFE will have fewer computing or
manipulative features (number of windows simultaneously displayed,
graphic options, algebraic processes, etc.). Instead, SAFE will have
more interfaces to data sources, communication nets, and other intel-
ligence community systems. Roughly speaking, an even trade will have
been made between diversity of features and interoperability.
SAFE will continue evolving after 1987: it will not be "finished."
There will be additional releases of core SAFE after that time, though
it is difficult at this point to envision just what they will be.
Also, DIA and CIA versions of full SAFE will continue to add unique
capabilities, interoperability with other new systems and the like.
Such continuing evolution will, however, require a far lower level of
manpower and funding than the basic development of the system, which
will have been completed with Delivery 4.
It seems to us that these variations from the original plan are
desirable ones that do not reduce the original scope: it is fair to
say that SAFE will be completed on schedule. (If the existing
agreements on the functions of Deliveries 2-4 weaken, however, the
schedule will be threatened.)
We also think that the $192 million investment should be enough to
complete SAFE. As was recognized in 1982, the risk in the
"industry-standard" approach is not so much in software or contract
overrun as it is in requirements for computer power. Much of the
software is commercial; that which is not is broken into small modules
that are within the state-of-the-art, and individual overruns of time
or money should not be enough to affect the total much. However,. it
is possible that with thousands of terminals exercising the full
capabilities of core SAFE while interoperation with other systems is
occurring, the computer power required to provide good responsiveness
may be greater than forecast. As noted earlier, we suggest that CSPO
undertake performance modeling to test this question.
Fortunately, we expect that before 1987 the manufacturers will offer
substantial improvements in large IBM-compatible computers: we think
it likely that any overrun in computer power requirement will be
offset by improvements in the price-performance of processors avail-
able.
2.3 Organization Changes
We recommend that CSPO dissolve after Delivery 4 (by the end of
1986). When it does, the DIA employees should return there and
continue to support the evolution of SAFE within the DoD community.
CIA people should return there and exercise two responsibilities:
some should work on CIA-specific interconnections and conversions; the
others should retain the responsibility for the evolution of core
AL Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
 Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6
1
1
1
1
1
1
1
1
1
1
1
I
SAFE. This group (relatively few individuals supported by contrac-
tors) should retain close contact with SAFE users and formulate
reasonable release packages which they can propose as evolutionary
additions to core SAFE. There should be a single overseeing board,
representing all of the SAFE users, which responds to the proposals of
the core SAFE group and provides funds for the approved releases.
:To prepare for this dissolution of CSPO, we suggest that the DIA and
CIA people be slowly reassigned as opportunities permit to activities
related to their agency's releases. The QA group should become mostly
DIA, because the relatively rigorous project management disciplines
being practical by it are more applicable to DIA than to CIA. In the
other three major groups, agency personnel should (to the degree
convenient) be assigned by delivery: D2 and D5 manned by CIA people,
D3 and D4 by DIA people. Finally, the CIA group working on Delivery 2
should prepare to become the continuing support group for core SAFE
after the dissolution of CSPO.
After 1986, CIA would retain the obligation to support core SAFE for
all users, but this obligation would be much smaller and more limited
than the present obligation to support the full scope of Deliveries 4
and 5.
In approximately 1990, it should be possible for the responsibility to
leave CIA. It might move to a commercial vendor, which we still hope
will eventually provide systems that encompass all the functions of
SAFE. Alternatively, it might move to the Intelligence Community
Staff, if standards evolve across the community and if SAFE becomes a
modular system capable of widespread use and interoperability within
the Community.
AL Arthur D. Little, Inc.
Approved For Release 2009/01/12 : CIA-RDP86M00886RO01700240005-6