MONTHLY REPORT TO THE COINS COMMITTEE

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP80B01139A000100120001-9
• Release Decision: RIPPUB
• Original Classification: U
• Document Page Count: 4
• Document Creation Date: January 4, 2017
• Document Release Date: July 14, 2005
• Sequence Number: 1
• Publication Date: February 29, 1968
• Content Type: REPORT

0~L2U;,,C A I P$OBA1139A000100120001-9
29 February 1968
MONTHLY REPORT TO THE COINS COMMITTEE
From
Chairman, COINS Panel on System Evaluation
The COINS Panel on System Evaluation, as reconstituted
by the charter dated 30 January 1968, met twice during the month
on 6 February and 21 February.
The first meeting was devoted primarily to discussion of
the tasks and modus operandi of the panel, the potential utility
of the several COINS system logs as sources of data, and the
different points of view of the members toward various means of
gathering data on system utilization, performance, and user
reaction.
The second meeting focussed on consideration of a plan
drafted by NSA, most of which was aimed at measuring
the effectiveness o e OINS hardware/ software system; discussion
of two principal means of obtaining data on use and user reaction
(user log forms and interviews); and review of a DIA evaluation
in October 1967 of the DIA on-line formatted file system.
At the present time the panel is divided into subgroups,
each of which is developing the first draft of the following portions
of a plan for gathering and analyzing data.
a. Hardware/ software system performance
b. User log form for gathering a minimal amount of feedback
from users on each instance of interagency use
c. User survey to gather additional data on circumstances
of use and user reactions
d. Environmental conditions in each participating agency
that may affect use
Approved For Rele sl 0 ~ .: CI,~..1D 8QP 1139A000100120001-9
 Approved For Release 2005/07/20; CIA-RDP> OB 1I39A000100120001-9
COINS EVALUATION PANEL
Minutes of Meeting of 21 February 1968
1330-1545 hours, Ames Center Building, Roslyn
1. Information items. re
panel's information that Chairman of CODIB,
on 7 February had established a COD-Lb anel on Files for Use
in COINS with instructions to submit its draft report by 8 March
1968. Among its assigned'tasks was the following: "For evaluation
purposes, give consideration to the selection of a single area of
substantive intelligence to be included in COINS as a package of
properly prepared files. " I lof CIA is chairman
of the panel. I lalso reported that
in discussing with CODIB the problem of files for COINS, ha
mentioned the need to obtain suggestions from users as to how
files in the system could be improved -- a need that this Evaluation
Panel should keep in mind in developing the plans for gathering
n
.
data on User i .emu......-
the plan and the report on the evaluation conducted in October 1967
of the DIA on-line system of formatted files, which are also
COINS files. It was extremely helpful to the panel in its
discussion of means of obtaining data on COINS use and user
reaction to have this information on the questions asked and
responses obtained by DIA in its study.
2. draft evaluation pll.an. lopened
the discussion. witha s ort introductory briefing on the s a istician's
approach to the determination of systems effectiveness, defined
as the probability that a system will successfully meet an
operational demand within a given time when operated under
Approved For Rele se 2005t07/20: CIA=RDP80B01I 39A000100120001-9
 ApproM For Release 2005/07/20 : CIA-RDP86961139A000100120001-9
specified conditions. pointed out that for COINS, two
and a half of the three essentials for measurement of effectiveness
in this way are unknowns: there are no explicit operational demands;
the only time requirement included in the design of COINS, a
response within 15 minutes, was arbitrarily selected; and there
are no specified conditions. explained his belief
that it would be rather feeble to provide data on how COINS
performs in actual usage and to do no more, because the initial
trial period and the data contained in the system are not at all
typical. But tests could be planned and made to determine how
the system would perform under different conditions of operational
demand. agreed that such tests
could be conducted, that they would provide information on
limiting factors, but that the results would necessarily apply
only to the present hardware/ software configuration. After
further discussion, it was agreed that
F
will constitute a subgroup to develop the detailed
plan for gathering and analyzing data on the performance, or
effectiveness, of the COINS hardware and software. They will
consider whether use can be made of data from the switch and
agency system logs, and will design the tests to be made to
determine performance under a variety of conditions.
3. Pro osed COINS user iog_ Since the previous meeting,
Fad drafted a ''COINS User Log Form, " with
the thought that such a form could be used to obtain a small amount
of user feedback on each interagency interrogation; had discussed
it with I land they had then obtained
reactions to it from a number of other persons within CIA and NSA.
Copies of the form were distributed for panel discussion. Several
panel members felt that the six questions on it were too many.
Some thought it would be preferable to ask only the two principal
questions used in the DIA study concerning the "success" and the
format of responses. As for the distinction between use to obtain
needed data and use to gain knowledge of the COINS system, it was
suggested that it might be well to retain the distinction but to
require only those seeking needed data to answer the rest of
the questions.
STAT
STAT
4. Panel organization for developing the plan. As indicated
in paragraph 2, I Iwill proceed STAT
to develop the portion o the plan concerned with the ar ware/ software
system performance. will follow
through on the user log and will develop a final draft for panel
STAT
Approved For Release 2005/07/20, : CIA-RDP80B011'39A000100120001-9
 Apprord For Rel.eas 2005107/20:'C1A-RD;0801I39A000100120001-9
consideration. After some discussion of the pros and cons of
"in depth" interviews of users to obtain more information on the
circumstances of use and on their reactions than can be o'bca.ned
on a brief form, it was decided that Mr. Stein and
will concentrate on this question. nentio _ed
the need to gather certain descriptive information on the different
environmental conditions in the several agencies that may of ect
the extent of use of COINS. The c:,Lta that will eventually be presented
on system utilization and user reaction could be better undurs.ood
in the light of the conditions in each Agency with respect to measures
taken to encourage potential users to try COINS, training of
users, number and availability of terminals, ar_iouni and kind of
assistance available to users, and extent of users' familiarity
with a similar system. It should be possible to obta : from each
agency a descriptive account of such enfronmental conditions if
the panel's plan outlines the information needed.
agreed to draft this portion of the plan. It was agreed that, if possible,
preliminary drafts of each of the plan components will be available
for panel discussion at the next meeting.
5. The next meeting will be held on 7 March at 0930 hours
at the Ames Center Building, Rosslyn.
Approved For Release?2005/07/20.: CIA-RDP80BOl139A000100120001-9