With the new thresholds for EVMS compliance being rolled out, a new approach to how DCMA manages its Validation review workload is being rolled out as well. In the past, DCMA would perform validation and ongoing surveillance reviews on any contract exceeding $50M. Now with the new thresholds in place, DCMA will only perform validation reviews on contracts larger than $100M, but will use a data driven approach to identify when a contractor will have to submit to a surveillance review.
In the past, surveillance reviews were done based on the calendar. So like clockwork, contractors would support audits done by DCMA to show how effectively the customer was managing their EVM processes within the guidelines of the ANSI standard and their own internal management system.
The outcome of surveillance could range from a few small corrective actions to loss of the advanced agreement. However, due to resource constraints as well as to minimize costs to contractors, this method of performing surveillance based on a predetermined schedule is no longer effective.
DCMA’s New Data Driven Approach
The new approach DCMA will employ is a data driven approach. DCMA will use analytics to review contractor data and identify high-risk contracts. Those contracts that are determined to be high risk will be subject to a surveillance review, while those that are low risk will not be subject to surveillance reviews.
How exactly does this work? DCMA has derived a series of metrics that can show compliance to all of the ANSI-748 32 criteria. The contractor’s monthly data will run through all DCMA’s metrics to verify compliance to all of the criteria. Based on DCMA’s findings, contractors could be subject to surveillance reviews for specific segments of the ANSI criteria. This is another cost benefit, as surveillance will not necessarily cover all 32 criteria. Below is an example of the metrics for Criteria 30 (Retroactive Changes):
As shown above, these series of metrics will be used to identify if any retroactive changes have been made. If there has been one, that will be an indication that a process has failed or more investigation is required, potentially leading to a surveillance review.
As of this point in time, I have not seen all the metrics DCMA is using. That said, we can certainly make educated guesses on what kind of metrics are being used to identify deficiencies. This presents us with an opportunity to better perform analysis on our numbers and allows us to identify our own, audit inducing, deficiencies prior to delivering data to DCMA. This will help us to improve our processes, data, analysis and management while also reducing costs and minimizing our risks for audits.