MEMORANDUM REPORT: EXTERNAL QUALITY CONTROL REVIEW OF THE SECURITY PROGRAM EVALUATION AND ASSESSMENT DIVISION, OFFICE OF SECURITY AND INTELLIGENCE OVERSIGHT (2003 - 0011 - AS)

• Document Type: FOIA
• Collection: FOIA Collection
• Document Number (FOIA) /ESDN (CREST): 06702674
• Release Decision: RIPPUB
• Original Classification: U
• Document Page Count: 13
• Document Creation Date: December 28, 2022
• Document Release Date: August 15, 2018
• Case Number: F-2016-02244
• Publication Date: July 25, 2003

�,1
Approved for Release: 2018/08/14 C06702674
SECRET/ / X1
Central Intelligence Agency
WitedingWOMM505 Inspector General
IG 03-0442 7034744555
25 July 2003
MEMORANDUM FOR: Anne M. Sigmund
Acting Inspector General
Department of State
SUBJECT:
(U//FOU0) Memorandum Report: External
Quality Control Review of the Security
Program Evaluation and Assessment Division,
Office of Security and Intelligence Oversight
(2003-0011-AS)
1. (U//FOU0) The Central Intelligence Agency Office of
Inspector General reviewed the system of quality control for the
Office of Security and Intelligence Oversight (SIO), Security
Program Evaluation and Assessment Division (SPEAD) in effect
during the fiscal year ending 30 September 2002. Our review was
intended to satisfy the requirement for external quality control
review established under Government Auditing Standards
promulgated by the Comptroller General.
2. (U//FOU0) The objectives of our review were to
determine whether SPEAD's internal quality control system is in
place and operating effectively to provide reasonable assurance
that policies, procedures, and applicable auditing standards are
being followed. Our review was conducted in conformity with
standards and guidelines established by the President's Council
on Integrity and Efficiency (PCIE) in its February 2002 policy
statement on internal quality control and external quality
control reviews.
3. (U//FOU0) In conducting our review, we interviewed
staff, evaluated SPEAD's policies and procedures, assessed the
internal quality assurance program, and reviewed selected audit
reports and supporting working papers. We selected for detailed
review two of the three audits completed by SPEAD during fiscal
year 2002.*
* (U//FOU0) We selected the following two audits for detailed review:
Management of the Armored Vehicle Program, February 2002; and Enhancing the
Protection of Classified Documents at State Department Headquarters, June 2002.
UNCLASSIFIED//FOU0 When
Separated From Attachment
SECRET/ /X1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X1
SUBJECT: (U//FOU0) Memorandum Report: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
4. (U//FOU0) In our opinion, the system of quality control
for SPEAD in effect for the fiscal year ending 30 September 2002
was designed in accordance with the quality standards established
by the PCIE and provided SPEAD with reasonable assurance of
material compliance with professional standards in the conduct
of its audits. Therefore, we are issuing an unqualified opinion
on your system of audit quality control.
5. (U//FOU0) Although an effective internal quality
control system has been established, we believe that the system
could be strengthened by periodic reviews of SPEAD's audit
function conducted by an appropriately staffed internal review
team as prescribed by the PCIE and by performance of external
quality control reviews on a regular basis. We also found that
SPEAD audit personnel do not always follow the internal policies
and procedures prescribed in the SPEAD Audit Handbook for
conducting and supervising audit assignments and preparing audit
working papers. Additionally, the SPEAD Audit Handbook needs
updating and consideration should be given to adopting the
Office of Audits Audit Manual for SPEAD use. These issues are
discussed in greater detail in the attached Letter of Comments,
which includes recommendations for corrective action. The letter
also addresses an issue unrelated to SPEAD's system of quality
control that was discussed in the prior external quality control
review but was not fully resolved. None of the issues discussed
in the letter affect our overall opinion.
6. (U//FOU0) Comments on a draft of this report and the
Letter of Comments were obtained from the Assistant Inspector
General for Security and Intelligence Oversight in July 2003.
The Assistant Inspector General stated that he and his staff
concurred with the recommendations included in the draft Letter
of Comments and are working to implement them in a timely manner.
The specific actions SIO intends to take to implement the
recommendations have been incorporated into our final Letter
of Comments.
/in L. Helgerson
Attachment
2
SECRET/ /X1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X1
SUBJECT: (U//FOU0) Memorandum Report: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
(b)(3)
Distribution:
Original - Anne M. Sigmund (w/att) IG 03-0442
1 - IG (w/att)
1 - D/IG (w/att)
1 - IG Counsel (w/att)
1 - AIG/Audit (w/att)
1 - DAIG/Audit (w/att)
1 - C IG/AS (w/att) (b)(3)
1 - EXO/IG (w/att)
1 - AICF---G/AS (w/att) (b)(3)
1 - IG/AS/Chrono File (w/att)
1 - IG/AS/Report File (w/att)
3
SECRET/ /X1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
Attachment
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET//X1
Central Intelligence Agency
Washington, D.C. 20505
IG 03-0442/1
25 July 2003
MEMORANDUM FOR: Anne M. Sigmund
Acting Inspector General
Department of State
SUBJECT:
Inspector General
703-874-2555
(U//FOU0) Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
1. (U//FOU0) The Central Intelligence Agency Office of
Inspector General reviewed the system of quality control for the
Office of Security and Intelligence Oversight (SIO), Security
Program Evaluation and Assessment Division (SPEAD) in effect
for the fiscal year ending 30 September 2002 and provided the
results in a memorandum report dated 25 July 2003. This letter
of comments should be read in conjunction with that report.
2. (U//FOU0) Our review was designed to evaluate SPEAD's
system of quality control and compliance with that system.
We conducted our review in conformity with standards and
guidelines established by the President's Council on Integrity
and Efficiency (PCIE). Our review, however, was not designed to
disclose all weaknesses in the system of quality control or all
instances of noncompliance.
3. (U//FOU0) The comments and recommendations in this
letter pertaining to strengthening SPEAD's internal quality
control system and compliance with internal policies and
procedures for conducting and supervising audit assignments were
considered in expressing the opinion set forth in our report.
This letter does not alter that opinion.
qFPRPT//X1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X.1
SUBJECT: (U//FOU0) Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and .
Intelligence Oversight (2003-0011-AS)
4. (U//FOU0) This letter also addresses an issue unrelated
to SPEAD's system of quality control that was discussed in the
prior external quality control review but was not fully resolved.
5. (U//FOU0) Comments on the recommendations contained in
the draft letter of comments were received from the Assistant
Inspector General for Security and Intelligence Oversight (AIG/SIO)
and his staff and are incorporated in this final letter.
(U) Internal and External Quality Control Reviews
6. (U//FOU0) SPEAD is not satisfying the objectives of
an independent internal review as prescribed by PCIE guidance.
PCIE guidance regarding the general standard on quality control
contained in the 1994 Government Accounting Office (GAO)
Government Auditing Standards requires periodic internal review
by an independent Office of Inspector General (OIG) team to
evaluate whether the audit function, as a whole, is carried out
in accordance with government auditing standards, Office of
Management and Budget circulars, PCIE audit policy statements,
and statutory provisions applicable to the audit organization.
SPEAD uses the PCIE Checklist for Review of Individual
Performance Audits in conjunction with an independent review
of draft audit reports as an alternative means of achieving the
objectives of independent internal review.' The independent
reviewer of SPEAD audit reports is supposed to determine whether
GAO standards for audit evidence have been met. In the two
completed audits we reviewed, however, we found no evidence that
the PCIE audit checklist was completed nor did the independent
reviewer attest in any discernable manner to compliance with
GAO standards for audit evidence.2
1 (U//FOU0) In our 1994 external quality control review, we recommended that
SPEAD implement quality assurance reviews for audits or develop and implement
an alternative means of achieving the objectives of the quality assurance
function. SPEAD opted to establish alternative procedures.
2 (U//FOU0) We selected the following two SPEAD audits for detailed review:
Management of the Armored Vehicle Program, February 2002; and Enhancing the
Protection of Classified Documents at State Department Headquarters, June 2002.
2
SECRET/ /X1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X1
SUBJECT: (U//FOU0) Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
7. (U//FOU0) In our 1998 external quality control review,
we noted that SPEAD's use of the PCIE audit checklist was not, in
all instances, effective in providing assurance that organization
policies and procedures and applicable auditing standards were
being followed. We recommended that SIO revise its directive on
quality assurance procedures to provide for internal review of
SPEAD's internal quality control system by an appropriately
staffed internal review team. That recommendation has not been
implemented.
8. (U//FOU0) In a recent memorandum concerning
implementation of the recommendations from our 1998 external
quality control review, SIO indicated that SPEAD does not have
sufficient staff to implement the internal quality control system
outlined in the PCIE Policy Statement on Internal Quality Control
and External Quality Control Reviews. SIO management pointed out
that the PCIE guidelines allow an alternative system of internal
controls if the organization's written policies and procedures
explain how alternative controls can be effective for that
organization. SIO management told us that they plan to update
SIO's quality assurance directive to ensure that it conforms to
PCIE guidelines.
9. (U//FOU0) SIO should reconsider its decision to employ
an alternative means of achieving the objectives of PCIE
guidelines on independent internal review. SIO is a subset of
the Department of State/OIG, which has auditors conducting audits
in both SIO and its Office of Audits. We believe that personnel
from the Office of Audits could provide the review team needed to
perform a comprehensive internal review of SPEAD as described in
the PCIE guidelines. The Office of Audits employs in excess of
30 audit personnel whom we were told possess security clearances.
Beyond that, both SPEAD audits we reviewed were "sensitive but
unclassified." In light of the fact that current procedures have
not been wholly effective, we believe SIO should consider use of
auditors from the Office of Audits to form internal review teams
to periodically review SPEAD's audit function and its products.
3
SECRET! /X1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X1
SUBJECT: (UNFOU0) Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
(W/FOU0) Recommendation #1: Use auditors from the
Office of Audits to perform the independent internal
review function for SPEAD, as called for by the PCIE
guidelines.
10. (U//FOU0) In commenting on a draft of this letter,
the AIG/SIO and his staff concurred with Recommendation #1 and
stated that SIO will work with the Office of Audits to establish
an agreement for using their staff to perform independent
internal reviews of SPEAD audit reports.
11. (U//FOU0) Government auditing standards prescribe
that audit organizations conducting audits in accordance with
those standards should have an external quality control review
at least every three years. The last external quality control
review of SPEAD's operation was conducted in 1998. Because
SPEAD is not included in the Office of Audits' external quality
control review program, it is important that SPEAD adhere to a
three-year schedule as prescribed by the standards. SPEAD can
better adhere to this schedule if it includes in its quality
assurance procedures the requirement for external quality
control review on a three-year cycle.
(UNFOU0) Recommendation #2: Establish a schedule of
external quality control reviews on a three-year cycle
and revise SPEAD's quality assurance procedures to
include the requirement for external quality control
review.
12. (U//FOU0) The AIG/SIO and his staff concurred with
Recommendation #2 and stated that SIO will revise its quality
assurance procedures to include the requirement for external
quality control review. SIO will also work with the appropriate
agency to establish a schedule of quality control reviews on a
three-year cycle.
4
RP.CIRRT / /x1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X1
SUBJECT: (U//FOU0) Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
(W/F01170) Adherence to SPEAD Audit Handbook Policies
and Procedures
13. (U//FOU0) SPEAD's auditors and managers do not
consistently adhere to the policies and procedures prescribed
in SPEAD's Audit Handbook in conducting and supervising audits.
The two audits we examined lacked memorandums of team meetings,
audit assignment checklists, and finding worksheets, which are
required by the Audit Handbook. For one of the audits, the audit
exit conference was not documented, and for neither audit were
audit programs referenced to working papers. When an audit
program is not referenced, it is difficult to determine whether
audit steps were completed and audit objectives satisfied.3
In addition, a number of the technical aspects of working paper
preparation�such as cross-referencing and initialing of working
papers by auditors and audit managers�were not consistently
performed. SPEAD personnel cited increased pressures to complete
audits quickly and lack of management continuity over an extended
period of time due to a reorganization within the OIG as factors
contributing to personnel not fully complying with SPEAD policies
and procedures in conducting and completing their work.
14. (U//FOU0) Our prior external quality control review
of SPEAD found similar inconsistencies. In that review, we
recommended that SPEAD emphasize the importance of adherence to
the policies and procedures prescribed in the SPEAD Audit
Handbook and develop and incorporate into the SPEAD system of
quality control review a checklist designed to assess compliance
with internal policies and procedures. The recommendation was
not implemented. In a recent memorandum concerning the
implementation status of the recommendations in the last external
3 (W/FOU0) SPEAD's Audit Handbook does not have a specific requirement
for referencing or cross-referencing the audit program to associated working
papers, but it does require related working papers to be cross-indexed.
Although not explicitly required in the 1994 revision of Government Auditing
Standards, we believe that cross-referencing the audit program to associated
working papers is a widely accepted standard for DIG audit components.
In fact, the Department of State/OIG Office of Audits, in its audit manual,
requires cross-indexing of the audit program to the working papers.
5
PRrPRT//X1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X1
SUBJECT: (U//FOU0) �Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
quality control review, current SIO management indicated that
they still intend to implement the recommendation. In order for
policies and procedures to be effective and promote quality work,
they must be followed on a consistent basis, and recommendations
that promote those goals must be implemented.
(U//FOU0) Recommendation #3: Develop and incorporate
into the SPEAD system of quality control review a
checklist designed to assess compliance with the
internal policies and procedures prescribed in the
SPEAD Audit Handbook.
15. (U//FOU0) The AIG/SIO and his staff concurred with
Recommendation #3 and stated that SIO plans to develop the
checklist when OIG/SIO quality control procedures are updated.
6
cw-p-mmi/y1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET! /X1
SUBJECT: (U//FOU0) Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
(U//FOU0) The Audit Handbook Needs To Be Updated
18. (S) In our review, we noted that the SPEAD Audit
Handbook has essentially remained unchanged since 1995.
7
SECRET! /x1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X1
SUBJECT: (U//FOU0) Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
19. (U//FOU0) SPEAD and the Office of Audits are a part
of the same OIG, and both organizations perform their work in
accordance with government auditing standards, but each
organization has its own audit manual. We reviewed both manuals
and found that, except for a few directives and procedures
specific to SIO, the Office of Audits Audit Manual represents a
comparable, or even more complete, resource for SPEAD auditors.
Any SIO-specific policies or procedures not included could be
easily appended to the Office of Audits manual. We believe that
if SPEAD adopted the Office of Audits Audit Manual for its own
use, it would promote consistency within the OIG, ease the
transition for auditors moving between audit organizations, and
relieve SPEAD of the burden of making revisions resulting from
changes in Government Auditing Standards.
(W/FOU0) Recommendation #5: Revise the SPEAD
Audit Handbook to remove outdated material and update
for changes to policies and procedures. Consider
adopting the Office of Audits Audit Manual for use
by SPEAD auditors in performing audits.
20. (U//FOU0) The AIG/SIO and his staff concurred with
Recommendation #5 and stated that SIO will work with the Office
of Audits to establish an agreement for adopting its audit manual
for SPEAD auditors to use when performing audits.
�u-6hn L. Heigerson
8
SECRET/ /X1
Approved for Release: 2018/08/14 C06702674
Approved for Release: 2018/08/14 C06702674
SECRET/ /X1
SUBJECT: (U//FOU0) Letter of Comments: External Quality
Control Review of the Security Program Evaluation
and Assessment Division, Office of Security and
Intelligence Oversight (2003-0011-AS)
(b)(3)
Distribution:
Original
- Anne M. Sigmund (IG
03-0442/1)
1
- IG
1
- D/IG
1
- IG Counsel
1
- AIG/Audit
1
- DAIG/Audit
1
- C IG/AS
(b)(3)
1
- EXO/IG
1
- AICF---IG/AS
(b)(3)
1
- IG/AS/Chrono File
1
- IG/AS/Report File
9
CPOPVM/P5C1
Approved for Release: 2018/08/14 C06702674