The Changing Role of Test & Evaluation in DoD

June 27, 2011

Welcome to the Pacific Aerospace Consulting blog. Here is our first post!

Once upon a time, there was an expectation of receiving full funding for our DT&E projects. We maximized efficiency where we could, but were never required to adapt to a new DT&E business culture until now.

The DoD commissioned a Defense Science Board (DSB) Task Force to conduct a comprehensive review of how we do T&E. Their published results can be downloaded here. In summary they found:
– Lack of disciplined processes from eroding workforce and loss of senior T&E experience
– DT&E deficiencies identified but not implemented timely because of schedule and funding constraints
– Greater emphasis on integrated testing is required

The DSB findings have driven a shift in culture, as seen in the very recent (13 June 2011) update to the Defense Acquisition Guidebook (DAG). Among other changes, the DAG now places a greater emphasis on technical maturity risks, instead of cost and schedule risks. Also, the Guide to Incorporating T&E into DoD Acquisition Contracts now assumes a reader’s thorough understanding of DAG chapter 9 (integrating T&E into the acquisition process) for future DoD bids. Test agencies are also responding to the changing Defense Acquisition world. For example, here is a story on the change in organizational culture at the Army Test and Evaluation Command (ATEC).

For this blog discussion, please consider the following:

How do the DSB findings affect the industry’s view of DT&E as we strive to support our customer’s needs?

How can we in the industry help our government customer adapt (What key areas to focus on)?

How are foreign military sales efforts affected? Positive or negative?

Please post your thoughts and comments on any/all of the above. We need to be proactive, because these changes affect us all!

  1. Gday PAC Team, your comments on this area are probably very timely from an industry perspective but more so from the Defence agency side. Australia has just released a review by our National Audit office paste this link into your web browser:
    This paper in fact devotes a few pages to the whole concept of Systems Engineering and T&E in general and the failures (perceived or otherwise) to the current DMO processes for Navy. So I am not sure if this has added value to your questions posed, but I am sure the way government agencies work, this will have follow on reviews and effects to capital acquisition of platforms and equipment for Australia.

    • pacaero says:

      Nathan: Great information! Thanks for commenting. I think there is plenty to discuss regarding FMS and direct commercial efforts in the new era we’ve entered.

      I noticed the document in your link recommends, among other things, end-to-end assessments of performance. Sounds similar to the US Systems Engineering/T&E approach. We are also focusing more on assessing overal technical readiness of a weapon system. In US systems engineering, we are focusing more on Technical Maturity as a formal risk item, as opposed to a more traditional segmented approach to risk management.

      There is a benefit to deconstructing each potiential risk into individual risk contributors. We just can’t track the risk associated with each individual tree at the expense of improperly managing the health of the forest as a whole.

      We can discuss the FMS implications of this topic here, and in future posts. Thanks again for the information!

  2. Eric says:

    I had the opportunity to attend the Aviation Week webinar on Systems Engineering for the Age of Complexity. The webinar was led by Jeff Wilcox, Corporate Vice President for Engineering, Lockheed Martin Corp, and presented some interesting ideas as follows;
    1. From Performance-Based to Mission-Based
    2. From Clean-Sheet to Effective Reuse
    3. From Complex to Simple
    4. From Fixed to Flexible
    5. From Document-Based to Model-Based
    6. From Compartmentalized to Holistic
    As these changes/mindsets are integrated into the acquisition process the T&E component will likely require its own changes. Industry will have to stay abreast of such changes in the acquisition community to maintain an efficient, value added T&E process. Maybe the T&E community is already adapting due to updated project management methods such as agile and the changes discussed in the original blog are a result of this.

    Please let me know if you would like the link to Aviation Week for the webinar archive.

    • Todd Vinson says:

      Eric: Thanks for the info! Seems that the effort to come to terms with a reformed approach to Systems Engineering/DT&E is well underway!

      I’m sure there will be interesting discussion and fallout from upcoming SE and T&E conferences. I will provide a links in a separate post.



Leave a Reply

Your email address will not be published.

Leave a comment










Suite 111, 460 Pacific Highway
St Leonards NSW 2065
Ph: +61 2 9906 6595 

1940 Garnet Ave, Suite 200
San Diego CA 92109
Ph: +1 858 456 5296

U 2.02 Building E 1 Technology PL
Williamtown NSW 2318
Ph: +61 2 4081 2887

Unit 2.04, 17 Moore Street
Canberra ACT 2601
Ph: +61 2 4081 2887

ABN: 58 112 563 210