Summary: GAO reviewed selected Department of Defense (DOD) weapons systems simulations to determine: (1) what factors DOD should consider in assessing simulation credibility; (2) the results of assessing weapons systems simulations with respect to these factors; and (3) DOD efforts to reinforce simulation credibility.
GAO developed an assessment framework of 14 factors covering theory, model design, input data, correspondence between the model and the real world, management, documentation, and reporting, applied it to three Army air defense simulations, and found that: (1) the simulations' representations of combat reality were limited; and (2) DOD efforts to validate simulation results were weak. GAO also found that: (1) the Army support structures for simulation activities were very limited; (2) although DOD believes that the credibility of simulation results is important, it has not established a systematic and uniform system for determining credibility; and (3) although DOD issued directives and regulations on related topics, there was little guidance with direct relevance to simulations.
