Summary: Pursuant to a congressional request, GAO provided information on how federal agencies should assess the quality of program performance data.
GAO noted that: (1) accuracy is the only one of several important areas to consider when examining the quality of agency performance data; (2) although certainly not exhaustive, the following key dimensions of data quality illustrate some of the different types of quality concerns that agencies consider when producing program performance data: (a) accuracy--the extent to which data are free from significant error; (b) validity--the extent to which the data adequately represent actual performance; (c) completeness--the extent to which enough of the required data elements are collected from a sufficient portion of the target population or sample; (d) consistency--the extent to which data are collected using the same procedures and definitions across collectors and times; (e) timeliness--whether data about recent performance are available when needed to improve program management and report to Congress; and (f) ease of use--how readily intended users can access data, aided by clear data definitions, user friendly software, and easily-used access procedures; (3) when assessing how good data need to be, it is important to recognize that no data are perfect; (4) data need to be good enough to document performance, support decisionmaking, and respond to the needs of internal and external stakeholders; (5) decisions on "how good is good enough" may depend on the uses of the data and the consequences of program or policy decisions based on those data; (6) this may involve trade-offs among the key dimensions of quality; (7) the key is for agencies to be aware of the data quality limitations of the performance data, understand the trade-offs involved, and reveal and discuss in performance reports the limitations and trade-offs; (8) determinations on how good data need to be also depend on other factors, such as the type of measure being used and the amount of change expected in the data; (9) different types of performance data may require different levels of accuracy; (10) developing standards for key dimensions of data quality is a matter for individual agency analysis and determination, taking into account: (a) the views of the users of the data (internal and external stakeholders); and (b) relevant professional standards and technical advice; (11) current users of performance data may have valuable experience with the strengths and weaknesses of existing data, therefore, they can provide insights into the data credibility with external audiences; and (12) professional standards provide another tool for agencies working to determine the appropriate level of data quality for their performance measures.
