We here at Apogee Consulting, Inc. are not DCAA-haters. We admit, though, that it’s tough to find a complimentary word about the Defense Contract Audit Agency on this blog. We assert that situation arises not from any inherent bias on our part, but instead from an objective assessment of the factual reality of the current defense contract audit environment.
While DCAA may be in the process of implementing corrective actions that will lead to Solomon-like audit conclusions while streamlining quality review processes to permit issuance of audit reports faster than a speeding bullet, as of this date we have yet to experience the wonderful goodness that is the promised future state of the Defense Contract Audit Agency. Instead, we and our clients continue to be mired in the same old miasmic swamp of conclusions unsupported by evidence; misinterpretation of applicable statutory, regulatory, and contractual language with the apparent goal of generating as much questioned costs as possible; and auditors who seem far more interested in creating pretty working papers than in getting to the facts.
The next time your DCAA auditor refuses to accept your 5 year-old spreadsheet as sufficient transaction support (the one that was created by the accountant who retired two years ago, and that’s been stapled to the original Journal Entry in the box that you’ve been paying a third-party to store for you in anticipation of an eventual DCAA audit)—and instead requires you to rehost your entire legacy cost accounting system for the sole purpose of observing your personnel actually downloading accounting data into that spreadsheet—we suspect you’ll empathize with the experiences of our clientele. You perhaps may even start to feel our frustration and angst with the current DCAA approach to performing contract audits.
Regardless of our feelings on the matter, we were happy for DCAA that the Office of the Inspector General of the United States Postal Service had very complimentary things to say about DCAA audits performed on a reimbursable basis for the USPS over the past three Government fiscal years (2009-2012). The USPS OIG said very complimentary things about DCAA in its recent report. And the USPS OIG said those complimentary things a lot. There was so much repetition, it got irksome.
And we think the USPS OIG got it wrong. We looked at the same data the IG auditors looked at, and we would have reached different conclusions than the IG auditors did.
But let’s start with the happy words from the “Highlights” section of the IG report.
DCAA audits are cost-effective tools that help Postal Service contracting officials negotiate lower contract costs and manage contracts. These audits have consistently contributed to significant savings and averaged a return on investment of $105 for every dollar spent over the last 4 fiscal years. During FYs 2009–2012, DCAA audits identified more than $185 million in unallowable and unsupported contract costs; and disclosed internal control weaknesses related to contractors’ accounting systems, financial capabilities, and labor charges. These results assisted contracting officials in negotiating lower contract prices and settlements.
Now let’s review the wording from the “Conclusion” section of the IG report—
DCAA audits are cost-effective tools that help Postal Service contracting officials negotiate lower contract costs and manage contracts. These audits have consistently contributed to significant savings and averaged a return on investment of $105 for every dollar spent over the last 4 fiscal years. During FYs 2009-2012, DCAA audits identified more than $185 million in unallowable and unsupported contract costs. They also disclosed internal control weaknesses related to contractors' accounting systems, financial capabilities, and labor charges. These results assisted contracting officials in negotiating lower contract prices and settlements.
Now let’s look at Page 2 of the body of the IG report—
DCAA audits are cost-effective tools that help Postal Service contracting officials negotiate lower contract costs and manage contracts. These audits have consistently contributed to significant savings and averaged a return of $105 for every dollar spent over the last 4 fiscal years. During FYs 2009-2012, DCAA audits identified more than $185 million in unallowable and unsupported contract costs due to reviews of price proposals, termination claims, and equitable adjustments. They also disclosed internal control weaknesses related to contractors' accounting systems, financial capabilities, and labor charges. These results assisted contracting officials in negotiating lower contract prices and settlements.
The entire audit report (excluding Appendices) weighed in at a lengthy eight pages. Within that page count we found three Tables and many bulleted points. So as you can see, repeating the same conclusion over and over took up a significant portion of the total report page count. Plus (as we noted) it was irksome. Look USPS OIG report drafters, we heard you the first time. You didn’t need to tell us three times. (Oh, wait. Let’s go back and see to whom the report was addressed. Maybe the report drafters did know their audience….)
And the thrice-repeated conclusions are wrong, in our view. Let’s dig into some details, shall we?
Table 1 of the report lists the audits that DCAA performed for the USPS by year between 2007 and 2012. There weren’t that many of them. In the most hectic year (2009), DCAA performed 29 audits for USPS; in the least hectic years (2011 and 2012), DCAA performed 11 audits (per year) for USPS. The OIG noted that the decline in audit workload “may” be attributable to an overall decrease in Postal Service spending. However (as the report stated), “management indicated they did not strongly pursue additional DCAA audit requests during this period because of concerns regarding DCAA’s audit quality and timeliness.”
Appendix B of the report provides details regarding the types of audits that DCAA performed for the USPS. Between 2009 and 2012, DCAA performed a total of 58 audits. 14 of the 58 (24%) were audits of “price proposals”—i.e., audits of cost proposals submitted to the USPS for evaluation prior to award of a contract. Three of the 58 audits were audits of proposed contract “equitable adjustments”—which are contractor proposals to modify contracts based on changes made by the USPS. Let’s add to those two audits of Termination Settlement Proposals, which are contractor proposals to establish final contract prices after a Termination for Convenience. That makes a total of 19 audits out of 58. Roughly a full third of all the DCAA audits performed for USPS related to contractors’ proposed costs.
How did DCAA do in that area?
According to the USPS OIG, DCAA questioned “unallowable or unsupported costs” in the amount of $181,204,699—or roughly 98 percent of all the DCAA audit “return on investment” calculated by the OIG in its report. That $181.2 million was found on 12 of the contractor proposals audited. One proposal had no opinion because DCAA found the contractor’s proposal to be inadequate for audit, and another proposal has zero questioned costs. Here’s what the USPS IG reported about that $181.2 million in DCAA findings—
Six of the 12 proposals contained $153,723,825 in unsupported costs primarily because the contractor did not provide adequate supporting documentation for subcontractor costs, direct material, and overhead rates.
As you know, we have been skeptical of such claimed taxpayer savings. Our position is that questioned costs save taxpayers nothing, unless a contract is awarded that includes negotiated reductions based on the DCAA findings. Happily for us, the USPS IG report provided some details as to how the USPS Contracting Officers handled the DCAA findings in their negotiations with contractors.
The following is taken verbatim from the IG report. Our analysis is in parentheses.
- One price proposal disclosed unallowable costs of $10.9 million and unsupported costs of $143.5 million. The CO was able to negotiate the proposed price of $382.1 million down to a negotiated final price of $346.8 million, for a savings of $35.3 million. ($154.4 million questioned; $35.3 million sustained. Sustention rate = 22.86%.)
- Another price proposal contained unallowable costs of $3.9 million. Based on the DCAA findings, the CO negotiated the original proposed price of $49.8 million down to a final negotiated price of $47.1 million, for a savings of $2.7 million. ($3.9 million questioned; $2.7 million sustained. Sustention rate = 69.23%.)
- For a price proposal containing $6.2 million in unallowable costs, the Postal Service later cancelled negotiations and issued a separate solicitation, which it subsequently placed on hold indefinitely. ($6.2 million questioned; zero sustained. Taxpayer savings = zero.)
- One contractor submitted a termination claim for $702,230, $475,322 of which DCAA questioned based on the comingling of terminated proposal costs, unallowable fee costs, and disallowed subcontractor estimated-to-complete costs included by the prime contractor. The Postal Service terminated this contract for convenience and settled the questioned cost of $475,322. (While we are not sure what “settled” means in this context, we think if savings had been obtained, they would have been reported. So we are going with: $475,322 questioned; zero sustained. Sustention rate = zero.)
- Another contractor submitted a termination claim for $336,849 and the entire claim was questioned due to inadequate cost and pricing data. The Postal Service was able to negotiate a lower price of $73,475, a reduction of about 78 percent of the claimed cost. ($336,849 questioned; $263,374 sustained. Sustention rate = 78%.)
- A contractor submitted an equitable adjustment that resulted in $2,514,232 in questioned costs out of $2,639,618 total claimed costs. The questioned costs were primarily due to excessive labor costs claimed for holiday staffing, nonconforming mail, and excess staffing due to truck schedule changes. The DCAA report was a useful tool for the CO to use during claims negotiations and for substantially reducing contract settlement amounts. As a result, the CO negotiated the claim down to $16,562. ($2.514 million questioned; $2.514 sustained. Sustention rate = 100%.)
Looking just at the foregoing, we can see that the USPS realized savings of about $40.78 million. That’s a good result; there’s nothing to criticize anybody for. But it’s a far cry from the $185 million in repetitively claimed savings and “return on investment.” If the $185 million in DCAA questioned costs resulted in a “return on investment” of $105 for every dollar spent, then the real ROI (based on sustained findings) is more like $23 for every dollar spent. Again, that’s not bad—not bad at all. But it’s significantly different from the amount reported by the OIG. One is tempted to say it’s so far off, it’s actually misleading.
Looking at the totals for this area, we can see that the USPS CO’s sustained about $40.78 million out of $181.205 million initially questioned by DCAA, for an overall sustention rate of 22.5%--meaning that 87.5% of questioned costs were not sustained. We believe that the sustention rate is the one metric that best measures the quality of DCAA’s audit findings. In our view, that is not a superlative sustention rate; indeed, one is tempting to say it’s a pathetic sustention rate.
Regardless of our difference in opinion with that of the USPS OIG, it is important to report (again) that DCAA didn’t just question costs: the audit agency “also disclosed internal control weaknesses related to contractors' accounting systems, financial capabilities, and labor charges.” Let’s dig into those reported findings, shall we?
DCAA performed 18 “reviews” in this area (31 percent of all DCAA activity in the review period covered by the USPS OIG). 12 of the 18 were reviews of contractor financial capability; the less said about them, the better. (We will note for the record that, with respect to one review, the IG reported that “the CO felt the findings did not accurately reflect the supplier’s financial capability.”)
Of the other six reviews, four were reviews of USPS contractors’ “business systems” (3 Accounting System reviews and 1 Estimating System review). Before getting into the details, we need to note that the adequacy criteria were not disclosed by the OIG in its report. It’s not clear whether DCAA used the Accounting System adequacy criteria found in its standard audit programs for Internal Controls Adequacy (ICAPs)—which would be based on inapplicable DFARS criteria—or if the auditors tailored the audit program for USPS requirements. That matters, a lot.
In any case, the USPS IG reported—
The DCAA reports disclosed that two of the three accounting systems reviewed were unacceptable for accumulating and segregating costs on Postal Service contracts. One accounting system review was initiated prior to awarding a fixed-price contract.
It should be noted that the two Accounting System reviews that found the contractors’ systems to be inadequate were issued in 2009. 2009 was a banner year for DCAA; it was the year in which its external peer-reviewed quality control system opinion was pulled because of systemic quality failures—particularly in the area of business system adequacy reviews. We’re not saying that DCAA reached wrong conclusions in those two reports; but we are saying that one should be skeptical of DCAA’s opinions issued at that time.
In contrast, the report of the Estimating System review was issued in 2012, when all the corrective actions taken by DCAA since 2009 should have resulted in a solid, supportable, opinion as to the adequacy of the contractor’s Estimating System. The company upon which this helpful review was visited was Accenture.
Here’s what the DCAA reported to the USPS, with respect to Accenture’s Estimating System—
This audit disclosed several weaknesses in the system, including not:-- Using historical experience.
-- Monitoring and tracking estimates against actual costs.
-- Subjecting estimates to periodic internal reviews.
-- Requiring periodic training on the Estimating Manual.
-- Reviewing management documentation sufficiently.
-- Reviewing subcontractor costs adequately.
-- Documenting policies, procedures, and practices.
The funny thing about the foregoing is that they are, largely, policy deficiencies and not deficiencies in actual practice. The other funny thing is that they are deficiencies only when measured against DFARS adequacy criteria. The DFARS is not applicable to USPS contracts, and we would be shocked to see the DFARS Estimating System Adequacy contract clause (252.215-7002) in a USPS contract. Indeed, as the USPS IG itself noted in separate audit report, “While the Postal Service is not subject to the FAR or DFARS, it does not currently have specific policies and procedures or clauses that address supplier estimating systems.”
In other words, it appears that DCAA applied the wrong adequacy criteria to Accenture when concluding that the Accenture Estimating System was inadequate. That does not strike us as the basis for a high-quality conclusion.
Regardless of our opinion about the basis of DCAA’s conclusion, the USPS OIG used the DCAA conclusion to initiate its own probe of Accenture’s Estimating System, so that it could reach its own conclusion. The IG report on Accenture is right here. While the USPS IG did not, as DCAA did, use the DFARS Estimating System adequacy criteria as the basis for its audit conclusions, it did reprint them in full in Appendix C of its report—and call them out as being “best practices”. So it seems that the USPS OIG believes that the quasi-private, civilian, entity that is the United States Postal Service would be better served if it ran its contracting operations like the Department of Defense.
Words fail us.
Accenture’s response to the original DCAA audit report and/or to the follow-up OIG audit report, were not provided. We wonder what they said.
In conclusion, we offer the foregoing as constructive criticism of the USPS OIG and its decision to use DCAA. We suggest that DCAA still has a measure of distance to travel, in terms of improving its audit quality (as measured by the sustention rate) and in improving the timeliness of its audit reports. Although the USPS OIG was seemingly much taken with the use of DCAA as outsourced auditors, we thought that the love was not so very much warranted.
But perhaps that’s just us. If you think we’re wrong, feel free to leave your comment below.
| < Prev | Next > |
|---|
