REQUEST TO PARTICIPATE IN A PANEL DISCUSSION

• Document Type: CREST
• Collection: General CIA Records
• Document Number (FOIA) /ESDN (CREST): CIA-RDP84-00933R000100260003-2
• Release Decision: RIPPUB
• Original Classification: K
• Document Page Count: 24
• Document Creation Date: December 15, 2016
• Document Release Date: August 25, 2003
• Sequence Number: 3
• Publication Date: May 22, 1981
• Content Type: MF

Approved For Release 2003/11/05: CIA-RDP84-0093LR 900100260003-2
ODP-81-666
22 May 1981
MEMORANDUM FOR: Chairman, Publications Review Board
STAT FROM
Chief, Systems Programming Division
Deputy Director for Processing, ODP
Director of Data Processing
Deputy Director for Administration
a
Systems Programming Division, ODP
SUBJECT Request to Participate in a Panel Discussion
1. I request permission to participate in a panel discussion describing
Agency experience in solving a particular type of computing problem.
2. When approved, I intend to speak at the Conference in
STAT during the week of August 23rd. The audience is expected
to be comprised of about 400 persons from the United States and Canada
representing organizations running similar computer systems.
3. None of the material to be presented is classified or controversial. I
will discuss techniques that we have developed to allow users of IBM's
VM/CMS timesharing systems to share data. The architecture of our
unclassified Gimini database and AIM electronic mail systems will be described
in detail.
4. I am not under cover and will be identified as an Agency employee.
I will also give the standard disclaimer that the views expressed are my own
and not necessarily those of the Agency.
/signed/
STAT ODP/P/SPD/ISB 05-22-81/PRBREQ5
Distribution:
Original - addressee
1 - VM/SP Project
5 - Chronos
2 - DDA
Approved For Release 2003/11/05 : CIA-RDP84-00933R000100260003-2
 Approved Forease 2003/11/05 : CIA-RDP84-0093300100260003-2
SUBJECT: Request to Participate in a Panel Discussion
TITLE OF PANEL DISCUSSION: CMS File Sharing
I have reviewed the outline in paragraph 3 of this request, to the
best of my knowledge have found it to be unclassified, and approve it for
presentation.
William N. Hart
Bruce T. Johnson, D/ODP arry E. Fitzwater, DDA
2 1 AUG 1981
Date Date
/s/ Bruce T. Johnson
21 AUG 1981
Approved For Release 2003/11/05 : CIA-RDP84-00933R000100260003-2
 Approved Forelease 2003/11/05 : CIA-RDP84-0093000100260003-2
SHARING DATA IN CMS
The CMS Project of SHARE has submitted a requirement to IBM that
begins:
"It is not possible to share CMS files in a multiple-user, read/write
environment. "
The requirement explains that this deficiency makes it rather difficult to
implement applications such as electronic mail and general database systems.
We are acutely aware of the truth of this statement.
We have recently implemented two large applications under VM/CMS that
share data. Gimini is a refitted version of the GIMS database management
system that runs on a S/370 minicomputer. AIM is a facility that provides an
automated environment for creating, editing, sending, receiving and filing
documents and messages electronically.
GIMS is a hierarchical database management system that runs as a
problem program and supports multiple terminals, multiple databases,
transaction logging and concurrent transaction processing. It is dictionary
driven, allows multiple files within a database, multiple levels of security, and
has an EXEC-like capability. In other words, it is a full-function DBMS. We
are currently running GIMS under MVS on a 0 0331 Jn 3701168axed two -Amd-ah+
4 /VGs -6 3033 cIGSS *(ach fires.
GIMS began as a TRW research project back in 1965. The first
prototype for an IBM 7094/1410 came out in 1967. Two years later, GIM I
was released for DOS/360 and OS/360. GIM II was released in 1972. This is
essentially the version that we have been running on our MVS mainframes.
In 1979, we decided to refit GIMS to run on a 4331 minicomputer. VNI/CMS
was the logical choice for an operating system. The first Gimini system went
production in April of this year.
Gimini was implemented using a network of virtual machines that boggles
the mind. Each database user has an ordinary CMS virtual machine. The
software that runs in this virtual submits transactions to the database and
displays output on the user's terminal. Each GIMS user is assigned a
statement virtual that processes transactions. These virtuals are managed by
yet another that we know as the goddess virtual. There is also a tape
virtual that allows data to be extracted from, or bulk load to be made to, the
database. All communications between the user and the statement or goddess
virtuals is performed using the VMCF SENDX protocol. While the statement,
goddess and tape virtuals also use VMCF, they make particularly effective use
of shared writable discontiguous saved segments.
The data is stored in a small number of large CMS files. Each database
may consist of one or more CMS files, each of which corresponds to an
"extent." We have a small mod that allows in-place updating of records.
Gimini makes extensive use of discontiguous saved segments. Most of
the common code is re-entrant and resides in a 1.3 meg DCSS. There are
three shared writable segments containing database system tables. These
include the ENQ/DEQ/Wait entries that allow locking for groups of records,
compiled copies of the database dictionary and procedures, and lists of
Approved For Release 2003/11/05 : CIA-RDP84-00933R000100260003-2
 Approved Fgelease 2003/11/05 : CIA-RDP84-00993,000100260003-2
current generation counts for each record. Gimini performs its own lock
management, hence it can run in a tightly-coupled, multi-processor
environment.
A few functions have been omitted from the initial version of Gimini.
There is no history tape; it just isn't needed on a minicomputer. GIMS has
a function similar to spooling of consoles; an interface to the VM spooling
mechanism needs to be installed. The first version went to single user
systems, so multiple databases aren't fully supported. Also, record deadlocks
are so rare with only one or two users that the code to handle this condition
has been omitted.
Gimini is currently running on "trams IBM 4331-1s. It took a little over
two years for two system programmers to complete the refit. Of the 700
modules in GIMS, about 75 were affected. The changes applied principally to
interface routines. Most of the database routines were unchanged, and
software development and maintenance continues to be done under MVS. The
user's perception of the database is unchanged. Databases are also portable
between GIMS and Gimini using tape.
Over the next few months, support will be added for the missing
functions. GIM III should be out by June of next year, and it will also run
in a VM environment.
AIM provides an enviroment for the creation and management of
documents by non-data processing persons. It is similar in structure to
IBM's PROFS system, but places greater emphasis on document routing and
control. It is also TTY terminal oriented.
The AIM user is provided with more than twenty commands to create and
manage documents. Documents live in folders; documents are the logical
equivalent of files, while folders are equivalent to minidisks. Get and Put
privileges may be assigned to users for folders, while document access is
covered by Read, Modify and Append authority. Extensive controls, such as
From, To, Through and Carbon Copy, are provided to manage the flow of
documents.
The software that runs in the user's virtual machine comprises about 75%
of the code. This includes command processors, the database manager
interface, a CMS file system interface and SCRIPT subset, all of which run
under a tailored version of the editor. The central virtual machine manages
the database. It performs the actual manipulation of documents, as well as
transaction logging and user notification. Data is transferred between the
central virtual and user virtuals via VMCF SENDX.
The data is stored in a structure of CMS files on the central virtual's
data disk. Each user has a file that points to a list of folders. A folder has
one file that lists all of the documents in that folder, as well as a "shadow"
file that lists the users who have access to that folder. A document is
comprised of several files: a header that defines routing and symbolic
reference data, a text file containing the body of the document, a shadow
listing the users who have copies of the document, and annotates or appends
containing data added to the file after creation. Each document involves an
average of three CMS files. Regardless of the number of users who have
copies of a document, only a single copy is retained in the system.
- 2 -
Approved For Release 2003/11/05 CIA-RDP84-00933R000100260003-2
 Approved For lease 2003/11/05 : CIA-RDP84-0093 00100260003-2
AIM attempts to perform as much processing as possible in the user's
virtual to reduce the burden on the central server. There is a powerful
interpreter with an alias, or symbolic substitution, capability. All data is
double stored, just in case a disk drives goes out to lunch permanently.
Most of the code that runs in the user's virtual is re-entrant and resides in a
DCSS.
The "pilot project" has been running since January with a select group
of 200 users. Next month, AIM will enter "test production" and more users
will be permitted to climb aboard. It will enter full production in January
1982, and everybody will be allowed to use the system. We currently have
some 2,000 documents in AIM. One projection calls for 300,000 documents
when the system is fully operational.
Many additional features are on the drawing board for AIM. It should
ultimately interface directly to both XEDIT and Waterloo SCRIPT. It will have
to support multiple minidisks and multiple central servers to provide
acceptable performance. An archival facility is being developed, as are an
array of office automation goodies. It will also be expected to interface to a
variety of other applications.
What have we learned from all our efforts? Secure sharing of files
requires that the data management portion of an application be removed from
the user's virtual machine. Processing cannot usually be concentrated in a
single service virtual machine and still provide adequate response. Hence, a
careful division of labor between the user's virtual and the central, or
intermediary, virtual is necessary.
The performance of the CMS file system tends to crumble when
thousands of files are involved. The chief problem arises from the time and
storage required to repeatedly search the FST chains. The solution that we,
and PROFS, have chosen is to keep our own compressed directory and access
only slices of the whole database spread out over multiple minidisks.
We have yet to track any problems down to the performance of VMCF,
and it is believed that we could possible improve its performance with a minor
mod to the VM Control Program. Finally, shared writable segments have
proven to be a very powerful tool. It is, however, necessary to modify the
system slightly to prevent unauthorized mucking with the tables.
You can share large amounts of data in CMS. Unfortunately, today's
systems require that you spend substantial resources developing the support.
Approved For Release 2003/11/05 CIA-RDP84-00933R000100260003-2
 4 1%w Approved For elease 200 Pf/W s di K DP84-0093 00100260003-2
Central Intelligence Agency
Washington, DC 20505
Approved For Release 2003119~8gSS~1k-ADP84-00933R000100260003-2
 Approved For Re)ease 20039 Fd8 : j *DP84-009338000100260003-2
Problem:
"it is not possible to share CMS files in a multiple-user, read/write
environment. This deficiency makes it more difficult than should be
necessary to take advantage of the simplicity, efficiency and flexibility
of the CMS file system when building very commonly needed interactive
applications such as electronic mail, computer conferencing, transaction
processing and general database systems.
SHARE CMS Project Requirement 80023
Foil 2
Approved For Release 2003/9~)68':"lA-RDP84-00933R000100260003-2
 Approved Fo elease 20011 Ogs' . RDP84-0093 000100260003-2
Applications:
Gimini: a refitted version of the GIMS database management system
that runs on a S/370 compatible minicomputer.
AIM: a facility that provides an automated environment for
creating, editing, sending, receiving, and filing documents
and messages electronically.
Approved For Release 20031Pl1 o sg~ DP84-00933R000100260003-2
 Approved For Release 2003111'1@ ?Of DP84-00933R000100260003-2
GIMS:
- A hierarchical database management system that runs as a problem
program and supports multiple terminals, multiple databases,
transaction logging and concurrent transaction processing. It is
dictionary driven, allows multiple files within a database, multiple
levels of security, and has an EXEC capability.
- Currently running under MVS
Approved For Release 2003'f1bY?di c 2DP84-00933R000100260003-2
 Approved Forritelease 200319W~6fiA-c2DP84-00934&000100260003-2
Chronology:
1965 TRW research project
1967 Prototype for 7094/1410
1969 GIM I available for DOS and OS/360
1972 GIM 11 available
1979 Decision to use VM/CMS as a base
1981 Gimini system in production
fied
Approved For Release 2003Ym8r,6; IA DP84-00933R000100260003-2
 Approved For'R ease 20031f119Mts:I -*DP84-009338000100260003-2
Components:
Goddess virtual initializes and shuts down GIMS. Also assigns a
statement virtual to a user. Communicates using VMCF SENDX and
DCSS.
Statement virtuals process transactions. There is one for each user
signed on to GIMS. Communicates using VMCF SENDX and DCSS.
User virtuals submit transactions to the statement virtuals and display
output on the user's terminal. Communicates using VMCF SENDX.
This is the user's ordinary virtual machine.
Tape virtual allows extracting and bulk loads of a database.
Communicates using VMCF SENDX and DCSS.
Approved For Release 2003)I+ k CIA=RDP84-00933R000100260003-2
 Approved Fo4,pelease 200 )iVbV F iLIA4DP84-0093 000100260003-2
Data Organization:
Each database consists of one or more CMS files (extents) in GIMS
format. In-place updating of records is allowed.
Approved For Release 2003Y 'I)~W . DP84-00933R000100260003-2
 Approved For Release 200314 1FJ 4 tfk-I DP84-00933ROt0100260003-2
Discontinguous Saved Segments:
GIMSEG is a 64K, shared writable segment containing the database
"bootstrap," and ENQ/DEQ/Wait tables. Allows locking at
the group (of records) level.
GIIVISTATE is a 1.3 meg, shared R/O segment containing code.
GIMCOMP is a 3 meg, shared writable segment containing the
compiled dictionary and procedures.
GIMSHARE is a 512K, shared writable segment containing a list of
current generation counts for each record.
Approved For Release 2003 fib b dl'A-'~,DP84-00933R000100260003-2
 Approved Fo telease 200M)9 IiN*DP84-009338000100260003-2
Missing Functions:
History tape.
Directed output.
Multiple databases.
Record deadlock.
Approved For Release 2003Vl~~6%$:S*Ii DP84-00933R000100260003-2
 Result:
Approved For R ease 2003H9Yd3s:~Wr-YkDP84-00933M 0100260003-2
Gimini currently running on s4331s.
1.5 system programmers spent 27 months refitting software.
About 75 out of 700 modules affected. Changes principally applied to
interface routines. Database routines are unchanged. Most software
development and maintenance is still done under MVS.
User perception unchanged. Databases are portable between GIMS and
Gimini.
Approved For Release 2003/nl f/W"'c DP84-00933R000100260003-2
 Approved For.Release 200,W T7 ? ~i$ RDP84-0093&R000100260003-2
Future:
- Provide support for missing functions.
Unclassified
Approved For Release 2003/11/05 : CIA-RDP84-00933R000100260003-2
 Approved For Release 2003MWM5~ fA!ADP84-009338000100260003-2
AIM:
- Provides an environment suitable for the creation and management of
documents by non-data processing persons.
- Similar in structure to PROFS, but greater emphasis given to document
routing and control. Also is TTY oriented.
Approved For Release 20031i{1c/b` r L -RDP84-00933R000100260003-2
 Approved Fo,6Release 200 A i IAtDP84-009 000100260003-2
Externals:
26 user commands.
Documents (logical equivalent of files).
Folders (logical equivalent of minidisks).
- Get and Put access at folder level; read, modify and append at
Routing includes from, to, through, carbon copies, plus others.
Foil 13
Approved For Release 2003}' 4M88 S t DP84-00933R000100260003-2
 Approved For I'2elease 20031IMMsd1DP84-009338000100260003-2
Components:
- User virtual software comprises about 75% of code, including command
processors, database manager interface, CMS file interface and. SCRIPT
subset, all running under tailored editor. Communicates using VMCF
SENDX.
- Central virtual manages the database. Performs actual manipulation of
documents upon request of user virtual, as well as transaction logging
and user notification.
Approved For Release 200Mn1 A '6iA RDP84-00933R000100260003-2
 ? Approved Fo telease 200 1'iA i6i DP84-0093.38000100260003-2
- Folder List.
- Folders and shadows.
- Document headers, text, shadows, annotates and appends.
- All are CMS files (average 3 per document).
Approved For Release 20031P1505-S8LR DP84-00933R000100260003-2
 Approved For Rel 'ease 200314 VM9~?fi-I4DP84-00933ROOQ~0100260003-2
Features:
- Maximize processing in user virtual.
- Interpreter with alias capability.
- Double storing of data.
- Most user virtual code is re-entrant and resides in a DCSS.
Approved For Release 200'f/W?dL,&2DP84-00933R000100260003-2
 Approved Fc elease 200 'fV sLfIIRckDP84-0093i,R000100260003-2
Status:
- "Pilot project" running since January 1981.
- "Test production" in September 1981.
- Full production in January 1982.
- Currently 2,000 documents. 300,000 documents projected at full load.
Approved For Release 2003} q/ 8 Ss61b-RDP84-00933R000100260003-2
 Approved For Re ease 2003M4fdg9 sd DP84-00933R 100260003-2
Future:
- XEDIT and Waterloo SCRIPT.
- Multiple minidisks.
- Multiple central server virtuals.
- Archive capability.
- Office automation goodies.
- Interfaces to other applications.
Approved For Release 200 nl)dt '&IMRDP84-00933R000100260003-2
 Approved Fo telease 200i)/bVsdIAdRDP84-00931000100260003-2
Lessons Learned:
- Distribute processing.
- Distribute data when there are large numbers of CMS files.
- VMCF innocent until proven guilty.
- Shared writable segments are a very powerful tool.
Approved For Release 2003/ ff: CIA-RDP84-00933R000100260003-2