What GAO Found
The Veterans Benefit Administration (VBA) within the Department of Veterans Affairs (VA) now measures and reports the accuracy of its claim decisions in two ways—by claim and by medical issue—but its approach has limitations. When calculating accuracy rates for either measure through its Systematic Technical Accuracy Review (STAR) program, VBA does not always follow generally accepted statistical practices. For example, VBA does not adjust its accuracy estimates to reflect that it samples the same number of claims for review from each regional office despite their varying workloads, and thus, produces imprecise estimates of national and regional accuracy. Further, because VBA does not clearly explain differences in how its two accuracy measures are calculated or their associated limitations, reported information about accuracy performance lacks clarity and may be confusing. Finally, VBA reviews 39 percent (over 5,000) more claims nationwide than necessary to achieve its desired precision in reported accuracy rates, thereby diverting limited resources from other important quality assurance activities, such as targeted reviews of error-prone cases. VBA could achieve its desired precision by reviewing fewer claims for offices with smaller workloads or higher performance levels.
Overall, VBA has made enhancements to its other quality assurance activities, although GAO identified implementation shortcomings that may detract from their effectiveness. To improve local quality, VBA recently created quality review teams (QRTs) comprised of certified staff in each regional office. QRTs review a sample of claims before they are finalized to identify specific error types and help prevent inaccurate decisions. QRTs also assess individual staff performance; however, claims processed during overtime were excluded from such reviews in three of the four offices GAO visited, which may undermine regional quality for a portion of claims. VBA officials told GAO they do not know the extent to which this occurs nationally. Recent VBA data shows that about 10 percent of claims were processed during overtime. Also, to help ensure claims processors make consistent decisions when presented with the same evidence, VBA began using a questionnaire approach to test for consistency. Although these questionnaires allow VBA to reach more staff and require fewer resources to administer, VBA did not pre-test them to ensure the clarity of questions or validity of the expected results. For each enhancement, their effectiveness is unclear because VBA has done little to date to assess their impact on improving accuracy.
VBA has also taken steps to coordinate its quality assurance efforts, but has not maintained centralized guidance and other key supports that might help prevent future errors. VBA coordinates quality assurance efforts by disseminating national accuracy and consistency results, trends and related guidance to regional offices, which use this information to train staff. Further, VBA uses the results of STAR reviews for other quality assurance activities, such as focusing QRT in-process reviews on commonly made errors. However, regional office staff told GAO that there are multiple sources of guidance, that searching these sources is time-consuming and difficult, and that VBA's policy manual and national training are not sufficiently updated to help claims processors avoid future errors. In addition, regional office data systems do not allow managers to readily track error trends, resulting in ad-hoc and inefficient work-arounds.
Why GAO Did This Study
With a growing workload of disability claims due, in part, to recent wars, VBA faces difficulties in improving the accuracy and consistency of its claims decisions by VBA staff in 57 regional offices. To help achieve its goal of 98 percent accuracy by fiscal year 2015, VBA recently implemented a new way of measuring accuracy, and changed several quality assurance activities to assess the accuracy and consistency of claims decisions and to provide feedback and training to claims processors.
In this statement, GAO discusses initial observations from its ongoing review of VBA's quality assurance efforts, addressing the extent to which (1) VBA effectively measures and reports the accuracy of its compensation claim decisions; and (2) VBA's other quality assurance activities are complementary and coordinated. For this work, GAO analyzed STAR accuracy and VBA claims data from fiscal year 2013 (the most recent fiscal year for which complete data are available); reviewed relevant federal laws, regulations, guidance and other documents relevant to quality assurance activities; and interviewed VBA staff from headquarters and four VBA regional offices (selected to achieve variety in geography, size of workload and reported accuracy rates), and Veterans Service Organization officials. GAO has no recommendations at this time. GAO plans to issue its final report later in 2014, along with any related recommendations.
For more information, contact Daniel Bertoni at (202) 512-7215 or bertonid@gao.gov.
