Examining the Statistical Rigor of Test and Evaluation Results in the Live, Virtual and Constructive Environment

The Department of Defense has mandated that weapons systems undergo persistent and realistic testing in a joint operational environment. Testing for new weapons systems is to occur early and often, in an operationally realistic environment, in order to identify and correct problems before resolution options become technically infeasible and/or cost prohibitive. Executing the appropriate fidelity of testing solely in the live environment is not always a viable course of action. Advances in distributed testing capabilities combined with the establishment of the technical infrastructure are producing a continually expanding group of distributed capable participants able to play a role in robust joint operational scenarios. However, obtaining statistically rigorous results from virtual tests, especially those pertaining to Operational Test and Evaluation, remains elusive. Several considerations associated with the Design of Experiments for virtual testing are outlined, and potential methodologies explored, with the aim to ensure rigorous and actionable results are produced when using live, virtual and constructed simulations for test purposes.