In chaos theory, it’s known as sensitive dependence on initial conditions, or, more popularly, as “the butterfly effect.” Basically, it stands for the theory that a hurricane can be caused by the flapping of a butterfly’s wings a thousand miles away, several weeks before. Looking at a recent DOD Inspector General audit report on the investigation of a hotline report made by a DCAA auditor, we might be able to discern that same effect in action.

We’re talking about DOD IG audit report number D-2011-6-011, entitled, “Report on Hotline Allegation Regarding Lack of Agency Guidance on the Currency of Audit Testing at the Defense Contract Audit Agency,” dated September 21, 2011. You can find a copy of the DOD IG’s report here.

An unnamed DCAA auditor in the Eastern Region began evaluating Northrop Grumman Navel Shipyard’s Earned Value Management System (EVMS) in December, 2008. The auditor’s 90-page audit report had 17 findings. To support his conclusions, the auditor reviewed Contract Performance Reports (CPRs) dated September 21, 2008 “that were available at the time the audit started,” according to the DOD IG report. The audit was completed in August, 2009, and the report was then submitted for supervisory review, which was completed in November, 2009. The report was then forwarded to the Easter Region’s Technical Programs Division, for review by a Technical Specialist.

And that’s when the trouble started.

According to the DOD IG report—

On November 9, 2009, the Eastern Regional Technical Programs Specialist telephoned the supervisory auditor and told him that she would like the auditor to perform ‘current’” testing on more recent Contract Performance Reports. The auditor stated that he selected the most current Contract Performance Reports available at the start of the audit. At that time, no written guidance or policy related to a 6-, 9-, or 12-month testing policy existed. However, the data tested was no longer current by the time the audit was completed. To be sufficient and current, evidence supporting the audit opinion should be reasonably current as of the date of the audit report.

The Eastern Regional Technical Programs Specialist was concerned with the ‘age’ of the Contract Performance Reports and related transaction testing performed by the auditor. [Which, by the time of the review, were more than a year old.] … the specialist stated that it is the Eastern Regional Director’s position based upon discussions held in DCAA Executive Steering Committee meetings that the data tested should be within a six- to nine-month period prior to the issuance of the audit report.

On November 17, 2009, a Program Manager from Headquarters … said that the testing should be updated if it is more than 12 months old. On November 18, 2009, the Eastern Regional Director decided that the testing should be updated for transactions that were tested and are older than nine months. … the Eastern Regional Director directed the auditor to perform additional testing and determine if the original deficiencies were still at issue. Subsequently the Regional Audit Manager advised the Resident Auditor that the opinion stated in the audit report cannot be based on testing performed on contractor Contract Performance Report data from September 2008.

So, in essence, the fact that the audit took more than nine months to complete meant that the evidence used to support the auditor’s conclusions was too dated and no longer acceptable. The auditor was told to get back into the field and continue testing. Those of us who have experienced audits lasting two or more years might look at this direction and think, “Isn’t this just creating a perpetual audit environment where audits never end because the supporting data becomes stale before management signs-off?”

Yes. Yes, it is.

And that’s what the DCAA auditor alleged when he called the DOD IG hotline and reported his management. As the GAO audit report stated—

The complainant alleged that DCAA lacks any written guidance or agency-wide policy regarding the ‘currency’ of audit testing which is causing audit reports on contractor business system reviews to be delayed as a result of retesting.

The DOD IG substantiated the auditor’s allegation that DCAA lacked agency-wide guidance, which contributed to auditor uncertainty regarding when evidentiary data was too old   The IG recommended that DCAA develop such guidance. However, the IG also agreed that, in this instance, the data was too old and that the auditor was properly directed to perform additional testing.

As the IG stated in its report—

The auditor should have tested a representative selection of transactions across the year and not just transactions from reports issued on just one day. We observed that for very large projects such as this, the data tested will never be current unless such audits are scoped and resourced adequately. This particular audit only had two auditors assigned. Cost Performance Reports are submitted monthly for the nuclear aircraft carrier and are submitted quarterly for the nuclear submarine. The data tested by the auditor was not current and did not consist of sufficient appropriate evidence to provide a reasonable basis for the audit conclusion.

The IG stated that the rule that should have been applied to the situation to be as follows—

GAGAS 6.04b requires the auditor to obtain sufficient and appropriate evidence to provide a reasonable basis for the conclusion that is expressed in the report. The evidence provided in the report is more helpful if it is current.

The IG noted that prior GAO reports had criticized DCAA for GAGAS violations. The IG recommended that DCAA implement policy guidance to address the gap and ensure GAGAS compliance. The IG audit report included the following statement—

We recommend that DCAA Headquarters develop written agency-wide policy and guidance on the need to test current data to support opinions on the contractor’s internal controls and business systems. The policy and guidance should include criteria when the auditor should expand testing and perform additional work.

On behalf of DCAA, Patrick Fitzgerald concurred with the IG’s recommendation. The DOD IG report stated that DCAA committed to undertake the following steps—

By November 2011, DCAA will issue guidance, which will include the requirement for auditors to (i) perform sufficient testing of data that is relevant to the audit objectives, including the period or point in time covered by the report, (ii) perform testing of data generated by the system throughout the period under audit, and (iii) issue timely audit reports. For audits of contractor business systems, DCAA will perform compliance attestation engagements and report on the contractor’s compliance during a period of time or as of a point in time, consistent with the applicable attestation reporting standards (AT 601.55b) in AICPA’s Statements on Standards for Attestation Engagements. Circumstances where auditors would need to expand testing to obtain sufficient evidence for the conclusions expressed in the report should be limited since the transactions being evaluated in the audit will coincide with the defined period covered by the audit. DCAA agrees with the guidance in GAGAS A8.02g, that the evidence provided in the report is more helpful if it is current and, therefore, timely issuance of the report is an important reporting goal for auditors.

Importantly, the above paragraph states that DCAA plans to issue guidance in the very near future that will provide a policy statement regarding the issuance of timely audit reports. Obviously, that’s the first step to addressing the concern about “perpetual audits”. For our part, we applaud any actions taken by DCAA to get audit reports out quicker than the current system permits.

So to our readers, keep an eye out for the promised November 2011 Memorandum for Regional Directors (MRD) implementing the promised policy guidance. We think it will be quite important.

And also notice how a single phone call to the DOD IG hotline by a concerned auditor led to potentially big changes at the DCAA. That’s the butterfly effect in action.