Input:
	Background

		Use of Performance Information in the Federal Government

Concerned that the federal government was more focused on program  activities and processes than the results to be achieved, Congress  passed the Government Performance and Results Act of 1993 (GPRA).  GPRA sought to focus federal agencies on performance by requiring  agencies to develop long-term and annual goals, and measure and report  on progress towards those goals annually. Based on our analyses of the  act’s implementation, we concluded in March 2004 that GPRA’s  requirements had laid a solid foundation for results-oriented  management. At that time, we found that performance planning and  measurement had slowly yet increasingly become a part of agencies’  cultures. For example, managers reported having significantly more  performance measures in 2003 than in 1997, when GPRA took effect  government-wide.
However, the benefit of collecting performance information is fully realized  only when that information is actually used by managers to make  decisions aimed at improving results. Although our 2003 survey found  greater reported availability of performance information than in 1997, it  also showed managers’ use of that information for various management  activities generally had remained unchanged. Based on those results,  and in response to a request from Congress, in September 2005, we  developed a framework intended to help agencies better incorporate  performance information into their decision making. As shown in figure  1, we identified  five leading practices that can promote the use of performance  information for policy and program decisions; and  four ways agency managers can use performance information to  make program decisions aimed at improving results.
Our September 2005 report also highlighted examples of how agencies  had used performance information to improve results. For example, we  described how the Department of Transportation’s National Highway  Traffic Safety Administration used performance information to identify,  develop, and share effective strategies that increased national safety belt  usage—which can decrease injuries and fatalities from traffic accidents— from 11 percent in 1985 to 80 percent in 2004.
Subsequently, the GPRA Modernization Act of 2010 (GPRAMA) was  enacted, which significantly expanded and enhanced the statutory  framework for federal performance management. The Senate  Committee on Homeland Security and Governmental Affairs report  accompanying the bill that would become GPRAMA stated that agencies  were not consistently using performance information to improve their  management and results. The report cited the results of our 2007  survey of federal managers. That survey continued to show little change  in managers’ use of performance information. The report further stated  that provisions in GPRAMA are intended to address those findings and  increase the use of performance information to improve performance and  results. For example, GPRAMA requires certain agencies to designate a  subset of their respective goals as their highest priorities—known as  agency priority goals—and to measure and assess progress toward those  goals at least quarterly through data-driven reviews.
Our recent work and surveys suggest that data-driven reviews are having  their intended effect. For example, in July 2015, we found that agencies  reported that their reviews had positive effects on progress toward  agency goals and efforts to improve the efficiency of operations, among  other things. In addition, for those managers who were familiar with their  agencies’ data-driven reviews, our 2013 and 2017 surveys showed that  the more managers viewed their programs as being subject to a review,  the more likely they were to report their agencies’ reviews were driving  results and conducted in line with our leading practices. Recognizing the  important role these reviews were playing in improving data-driven  decision making, our management agenda for the presidential and  congressional transition in 2017 included a key action to expand the use  of data-driven reviews beyond agency priority goals to other agency  goals.
More broadly, our recent surveys of federal managers have continued to  show that reported government-wide uses of performance information  generally have not changed or in some cases have declined. As we  found in September 2017, and as illustrated in figure 2, the 2017 update  to our index suggests that government-wide use of performance  information did not improve between 2013 and 2017. In addition, it is  statistically significantly lower relative to our 2007 survey, when we  created the index.
Moreover, in looking at the government-wide results on the 11 individual  survey questions that comprise the index, we found few statistically  significant changes in 2017 when compared to (1) our 2013 survey or (2)  the year each question was first introduced. For example, in comparing  2013 and 2017 results, two questions had results that were statistically  significantly different:
The percentage of managers who reported that employees who report  to them pay attention to their agency’s use of performance information  was statistically significantly higher (from 40 to 46 percent).
The percentage of managers who reported using performance  information to adopt new program approaches or change work  processes was statistically significantly lower (from 54 to 47 percent).
As we stated in our September 2017 report, the decline on the latter  question was of particular concern as agencies were developing plans to  improve their efficiency, effectiveness, and accountability, as called for by  an April 2017 memorandum from OMB.

		The Administration’s Plans for Federal Performance Management

In early 2017, the administration announced several efforts intended to  improve government performance. OMB issued several memorandums  detailing the administration’s plans to improve government performance  by reorganizing the government, reducing the federal workforce, and  reducing federal agency burden. As part of the reorganization efforts,  OMB and agencies were to develop government-wide and agency reform  plans, respectively, designed to leverage various GPRAMA provisions.  For instance, the April 2017 memorandum mentioned above stated that  OMB intends to monitor implementation of the reforms using, among  other things, agency priority goals. While many agency-specific  organizational improvements were included in the President’s fiscal year  2019 budget, released in February 2018, OMB published additional  government-wide and agency reform proposals in June 2018.
The President’s Management Agenda (PMA), released in March 2018,  outlines a long-term vision for modernizing federal operations and  improving the ability of agencies to achieve outcomes. To address the  issues outlined in the PMA, the administration established a number of  cross-agency priority (CAP) goals. CAP goals, required by GPRAMA, are  to address issues in a limited number of policy areas requiring action  across multiple agencies, or management improvements that are needed  across the government.
The PMA highlights several root causes for the challenges the federal  government faces. Among them is that agencies do not consistently apply  data-driven decision-making practices. The PMA states that smarter use  of data and evidence is needed to orient decisions and accountability  around service and results. To that end, in March 2018, the administration  established the Leveraging Data as a Strategic Asset CAP goal to  improve the use of data in decision making to increase the federal  government’s effectiveness.

		Federal Performance Management Leadership Roles and Responsibilities

Over the past 25 years, various organizations, roles, and responsibilities  have been created by executive action or in law to provide leadership in  federal performance management. At individual agencies and across the  federal government, these organizations and officials have key  responsibilities for improving performance, as outlined below.
OMB: At least every four years, OMB is to coordinate with other  agencies to develop CAP goals—such as the one described earlier on  leveraging data as an asset—to improve the performance and  management of the federal government. OMB is also required to  coordinate with agencies to develop annual federal government  performance plans to define, among other things, the level of  performance to be achieved toward the CAP goals. Following  GPRAMA’s enactment, OMB issued guidance for initial  implementation, as required by the act, and continues to provide  updated guidance in its annual Circular No. A-11, additional  memorandums, and other means.
Chief Operating Officer (COO): The deputy agency head, or  equivalent, is designated as the COO, with overall responsibility for  improving agency management and performance through, among  other things, the use of performance information.
President’s Management Council (PMC): The PMC is comprised of  OMB’s Deputy Director for Management and the COOs of major  departments and agencies, among other individuals. Its  responsibilities include improving overall executive branch  management and implementing the PMA.
Performance Improvement Officer (PIO): Agency heads designate  a senior executive as the PIO, who reports directly to the COO. The  PIO is responsible for assisting the head of the agency and COO to  ensure that agency goals are achieved through, among other things,  the use of performance information.
Performance Improvement Council (PIC): The PIC is charged with  assisting OMB to improve the performance of the federal  government. It is chaired by the Deputy Director for Management at  OMB and includes PIOs from each of the 24 Chief Financial Officers  Act agencies, as well as other PIOs and individuals designated by the  chair. Among its responsibilities, the PIC is to work to resolve  government-wide or cross-cutting performance issues, and facilitate  the exchange among agencies of practices that have led to  performance improvements.
Previously, the General Service Administration’s (GSA) Office of  Executive Councils provided analytical, management, and administrative  support for the PIC, the PMC, and other government-wide management  councils. In January 2018, the office was abolished and its functions,  staff, and authorities, along with those of the Unified Shared Services  Management Office, were reallocated to GSA’s newly created Shared  Solutions and Performance Improvement Office.

	Agencies’ Use of Performance Information in Decision Making and Related Leading Practices Generally Has Not Improved

		Reported Use of Performance Information in Decision Making Generally Has Not Improved at Individual Agencies Since 2013

As at the government-wide level—where, as described earlier, the use of  performance information did not change from 2013 to 2017—managers’  reported use of performance information at most agencies also did not  improve since 2013 (illustrated in figure 3).
At the agency level, 3 of the 24 agencies had statistically significant  changes in their index scores—1 increase (National Science Foundation)  and 2 decreases (Social Security Administration and the Office of  Personnel Management). Also, in 2017, 6 agencies had results that  were statistically significantly different—4 higher and 2 lower—than the  government-wide average (see sidebar).
Throughout the report, we highlight two different types of statistically  significant results—changes from our last survey in 2013 and differences  from the 2017 government-wide average. The former indicates when an  agency’s reported use of performance information or leading practices  has measurably improved or declined. The latter indicates when it is  statistically significantly higher or lower than the rest of government.  These results suggest agencies have taken actions that led to  improvements in their use of performance information. For example,  when a result is a statistically significant increase since 2013, as with the  National Science Foundation index score in 2017, this suggests that the  agency has adopted practices that led to a measurable increase in the  use of performance information by managers.
When a result is statistically significantly higher than the government-wide  average, like GSA’s 2017 index score, this suggests that the agency’s  use of performance information is among the highest results when  compared to the rest of government. These agencies could also have  insights into practices that led to relatively high levels of performance  information use. Finally, when a result is a statistically significant  decrease since 2013, as with the Social Security Administration’s index  score in 2017, or statistically significantly lower than the government-wide  average, like the Department of Homeland Security’s 2017 index score,  this suggests the agencies face challenges that are hampering their  ability to use performance information. Appendix III provides each  agency’s index scores from 2007, 2013, and 2017 to show changes  between survey years.
When we disaggregated the index and analyzed responses from the 11  questions that comprise the index—which could help pinpoint particular  actions that improved the use of performance information—we similarly  found relatively few changes in agencies’ recent results. Specifically, we  identified 16 instances where agency responses on individual questions  were statistically significantly different from 2013 to 2017—10 increases  and 6 decreases. This represents about 6 percent of the total possible  responses to the 11 survey questions from each of the agencies.
In addition, we found 12 instances where an agency’s result on a  question was statistically significantly higher (11) or lower (1) than the  government-wide average in 2017. For example, the percentage of Social  Security Administration (SSA) managers reporting that their peers use  performance information to share effective approaches was statistically  significantly higher than the government-wide average. Although SSA’s  index score had a statistically significant decline in 2017 compared to  2013, the agency’s index score remains relatively high, as it has in prior  years. The scope of our work has not allowed us to determine definitively  what factors caused the decline in SSA’s index score and whether the  decline is likely to continue, although its result on this particular question  may indicate a continued strength. Each agency’s results on the 11  questions that comprise the index are presented in appendix I. The  agencies’ respective statistically significant results are identified in figure  4.
While some agencies had statistically significant improvements on  individual questions, and could point to actions that led to improvements  in their use of performance information, these improvements should be  considered in relation to the range of agency results and the government- wide average. In figure 4, there are five agencies with statistically  significant increases on responses to individual questions, where those  results were not statistically significantly higher than the government-wide  average (see arrows without plus signs for the Departments of  Agriculture, Defense, and Justice; the Environmental Protection Agency;  and the National Science Foundation). While these represent  improvements, they should be considered in relation to the range of  agency results and the government-wide average (provided in detail in  the agency summaries in appendix I). For example, in 2017, the  percentage of managers at the Department of Agriculture who reported  that upper management use performance information to inform decisions  about program changes was statistically significantly higher than in 2013.  However, the department’s 2017 result (37 percent) was relatively lower  when compared to the maximum agency result on that question (60  percent). Appendix I presents the results on the index and the 11  questions that comprise it for each of the 24 agencies.

		Individual Agencies’ Reported Use of Leading Practices Generally Remains Unchanged

When we compared government-wide and agency-level results on  selected survey questions that reflect practices that promote the use of  performance information, we found that results between 2013 and 2017  generally remained unchanged. As described earlier, there are 10 survey  questions that both reflect the five leading practices identified in our past  work and had statistically significant associations with higher index  scores. As shown in figure 5, government-wide results on 2 of the 10  questions were statistically significantly different, both increases, from  2013 to 2017. Despite these two increases, the overall results suggest  these practices are not widely followed government-wide. On most of the  10 questions, only about half (or fewer) of the managers reported their  agencies were following them to a “great” or “very great” extent.
When we analyzed agency-level responses to these 10 questions, we  also found relatively few changes in recent results. Specifically, our  analysis found 20 instances—16 increases and 4 decreases—where  agencies’ responses on individual questions were statistically significantly  different from 2013 to 2017. This represents about 8 percent of the total  possible responses to the 10 survey questions from each of the  agencies.
In addition, we found 10 instances where an agency’s result on a  question was statistically significantly higher (8) or lower (2) than the  government-wide average in 2017. Each agency’s results on these 10  questions are presented in appendix I, and the statistically significant  results are identified in figure 6.
Those agencies with results on individual questions that are either  statistically significantly higher than 2013, higher than the 2017  government-wide average, or both may have taken actions in line with our  leading practices for promoting the use of performance information. For  example, the National Science Foundation had both types of statistically  significant results on a question about having sufficient information on the  validity of their performance data. Here, the agency’s result increased 27  percentage points from 2013 to 2017. While the scope of our review does  not allow us to definitively determine the reasons for the National Science  Foundation’s higher results, they suggest the agency has taken recent  actions that greatly improved the availability and accessibility of  information on the validity of performance data. In both 2013 and 2017,  our analyses found this particular question to be the strongest predictor of  higher performance information use when we tested for associations  between the questions that reflect leading practices and our index.

	Managers Whose Programs Were Subject to Data- Driven Reviews Reported Greater Use of Performance Information and Leading Practices

Our 2017 survey results show that managers who reported their  programs were subject to data-driven reviews also were more likely to  report using performance information in decision making to a greater  extent (see figure 7). For the 35 percent of managers who reported being  familiar with data-driven reviews, those who reported their programs had  been subject to data-driven reviews to a “great” or “very great” extent had  index scores that were statistically significantly higher than those whose  programs were subject to these reviews to a lesser extent.
Similarly, we found that being subject to data-driven reviews to a greater  extent was also related to greater reporting of agencies following  practices that can promote the use of performance information. As figure  8 shows, managers who reported their programs were subject to these  reviews to a “great” or “very great” extent more frequently reported that  their agencies followed the five leading practices that promote the use of  performance information, as measured by the 10 related survey questions  associated with higher scores on the index. For example, of the estimated  48 percent of managers who reported their programs were subject to  data-driven reviews to a “great” or “very great” extent, 72 percent also  reported that managers at their level (peers) effectively communicate  performance information on a routine basis to a “great” or “very great”  extent. Conversely, for the 24 percent of managers who reported their  programs were subject to data-driven reviews to a “small” or “no” extent,  only 30 percent reported that managers at their level do this to a “great”  or “very great” extent.

	Opportunities Exist for the Executive Branch to Increase the Use of Performance Information within Agencies

		Disparities Exist in the Use of Performance Information by Senior Agency Leaders and Managers at Lower Levels

Our past work has found that the Executive Branch has taken steps to  improve the use of performance information in decision making by senior  leaders at federal agencies. However, our survey results indicate those  steps have not led to similar improvements in use by managers at lower  levels. Through its guidance to implement GPRAMA, OMB developed a  framework for performance management in the federal government that  involves agencies setting goals and priorities, measuring performance,  and regularly reviewing and reporting on progress. This includes  expectations for how agency senior leaders should use performance  information to assess progress towards achieving agency priority goals  through data-driven reviews, and strategic objectives through strategic  reviews. For example, GPRAMA requires, and OMB’s guidance  reinforces, that data-driven reviews should involve the agency head,  Chief Operating Officer, Performance Improvement Officer, and other  senior officials responsible for leading efforts to achieve each goal.
OMB’s guidance also identifies ways in which agency leaders should use  the results of those reviews to inform various decision-making activities,  such as revising strategies, formulating budgets, and managing risks. Our  past work also found that agencies made progress in implementing these  reviews and using performance information. In July 2015, we found that  agencies generally were conducting their data-driven reviews in line with  GPRAMA requirements and our related leading practices, including that  agency leaders used the reviews to drive performance improvement. In  addition, in September 2017, we reported on selected agencies’  experiences in implementing strategic reviews and found that the reviews  helped direct leadership attention to progress on strategic objectives.
Despite those findings, our survey results continue to show that the  reported use of performance information by federal managers has  generally not improved, and actually declined at some agencies. This  could be because of the two different groups of agency officials covered  by our work. GPRAMA’s requirements, and the federal performance  management framework established by OMB’s guidance, apply at the  agency-wide level and generally involve senior leaders. Our past work  reviewing implementation of the act therefore focused on improvements  in the use of performance information by senior leaders at the agency- wide level. In contrast, our surveys covered random samples of mid- and  upper-level managers within those agencies, including at lower  organizational levels such as component agencies. Their responses  indicate that the use of performance information more broadly within  agencies—at lower organizational levels—generally has not improved  over time. The exception to this was managers whose programs were  subject to the data-driven reviews required by GPRAMA. As described  above, those managers were more likely to report greater use of  performance information in their agencies. This reinforces the value of the  processes and practices put in place by GPRAMA. Our survey results  suggest that limited actions have been taken to diffuse processes and  practices related to the use of performance information to lower levels  within federal agencies, where mid-level and senior managers make  decisions about managing programs and operations.
Although OMB staff agreed that diffusing processes and practices to  lower levels could lead to improved use of performance information, they  told us they have not directed agencies to do so for a few reasons. First,  OMB staff expressed concerns about potentially imposing a “one-size-fits- all” approach on agencies. They stated that agencies are best positioned  to improve their managers’ use of performance information, given their  individual and unique missions and cultures, and the environments in  which they operate. We agree that it makes sense for agencies to be able  to tailor their approaches for those reasons. OMB’s existing guidance  provides an overarching framework that recognizes the need for flexibility  and for agencies to tailor their approaches. Moreover, given the long- standing and cross-cutting nature of this challenge, a government-wide  approach also would provide a consistent focus on improving the use of  performance information more extensively within agencies.
OMB staff also told us that they believed it would go beyond their  mandate to direct agencies to extend GPRAMA requirements to lower  levels. GPRAMA requires OMB to provide guidance to agencies to  implement its requirements, which only apply at the agency-wide level. As  noted earlier, however, GPRAMA also requires OMB to develop cross- agency priority (CAP) goals to improve the performance and  management of the federal government. The President’s Management  Agenda established a CAP goal to leverage data as a strategic asset, in  part, to improve the use of data for decision making and accountability  throughout the federal government. This new CAP goal presents an  opportunity for OMB and agencies to identify actions to expand the use of  performance information in decision making throughout agencies.

		Plan for New CAP Goal Does Not Yet Contain Required Elements for Successful Implementation

As of June 2018, the action plan for implementing the Leveraging Data as  a Strategic Asset CAP goal is limited. According to the President’s  Management Agenda and initial CAP goal action plan, the goal primarily  focuses on developing and implementing a long-term, enterprise-wide  federal data strategy to better govern and leverage the federal  government’s data. It is through this strategy that, among other things, the  administration intends to improve the use of data for decision making and  accountability. However, the strategy is under development and not  expected to be released until January 2019, with a related plan to  implement it expected in April 2019.
The existing action plan, released in March 2018 and updated in June  2018, does not yet include specific steps needed to improve the use of  data—including performance information—more extensively within  agencies. According to the action plan for the goal, potential actions  currently under consideration focus on establishing agency “learning agendas” that prioritize the development  and use of data and other evidence for decision-making; building agency capacity to use data and other evidence; and  improving the timeliness of performance information and other data,  and making that information available to decision makers and the  public.
Although developing learning agendas and building capacity could help  improve the use of performance information in agencies, improving  availability of data may be less effective. For example, as our past survey  results have shown, increasing the availability of performance information  has not resulted in corresponding increases in its use in decision  making.
We recognize that the CAP goal was created in March 2018.  Nonetheless, it is important that OMB and its fellow goal leaders develop  the action plan and related federal data strategy consistent with all key  requirements to better ensure successful implementation. The action plan  does not yet include complete information related to the following  GPRAMA requirements: performance goals that define the level of performance to be achieved  each year for the CAP goal;  the various federal agencies, organizations, programs, and other  activities that contribute to the CAP goal; performance measures to assess overall progress towards the goal  as well as the progress of each agency, program, and other activity  contributing to the goal; and  clearly defined quarterly targets.
Consistent with GPRAMA, Standards for Internal Control in the Federal  Government identifies information that agencies are required to include in  their plans to help ensure they achieve their goals. The standards state  that objectives—such as improving the use of data in decision making— should be clearly defined to enable the identification of risks. Objectives  are to be defined in specific terms so they can be understood at all levels  of the entity—in this case, government-wide as well as within individual  agencies. This involves defining what is to be achieved, who is to achieve  it, how it will be achieved, and the time frames for achievement.
Ensuring that future updates to the new CAP goal’s action plan includes  all required elements is particularly important, as our previous work has  found that some past CAP goal teams did not meet all planning and  reporting requirements. For example, in May 2016 we found that most  of the CAP goal teams we reviewed had not established targets for all  performance measures they were tracking. This limited the transparency  of their efforts and the ability to track progress toward established goals.  We recommended that OMB, working with the Performance Information  Council (PIC), report on actions that CAP goal teams are taking, or plan  to take, to develop such targets and performance measures. OMB staff  generally agreed and, in July 2017, told us they were working, where  possible, to assist the development of measures for CAP goals. However,  the recommendation has not been addressed and OMB staff said the  next opportunity to address it would be when the administration  established new CAP goals (which took place in March 2018). Following  the initial release of the new CAP goals, CAP goal teams are to more fully  develop the related action plans through quarterly updates. Given the  ongoing importance of meeting these planning and reporting  requirements, we will continue to monitor the status of actions to address  this recommendation as implementation of the new CAP goals proceeds.

		Our Survey Results Identify Additional Opportunities for the PIC to Improve Federal Use of Performance Information

While the PIC, which is chaired by OMB, has contributed to efforts to  enhance the use of performance information, our survey results identify  additional opportunities to further those efforts. The PIC’s past efforts  have included hosting various working groups and learning events for  agency officials to provide performance management guidance, and  developing resources with relevant practices. For example, the PIC  created a working group focused on agency performance reviews, which  was used to share recommendations for how agencies can implement  reviews, along with a guide with practices for effectively implementing  strategic reviews. In January 2018, staff supporting the PIC joined with  staff from another GSA office to create a new group called Fed2Fed  Solutions. This group consults with agencies and provides tailored  support, such as data analysis and performance management training for  agency officials, to help them address specific challenges related to  organizational transformation, data-driven decision making, and other  management improvement efforts.
Our survey results identify useful information related to potential  promising practices and challenges that OMB and the PIC could use to  inform efforts to enhance the use of performance information more  extensively within agencies (e.g., at lower levels). As was previously  described, the PIC has responsibilities to (1) facilitate the exchange  among agencies of proven practices, and (2) work to resolve government- wide or cross-cutting performance issues, such as challenges. Our  analyses of 2017 survey results identified instances where agencies may  have found effective ways to enhance the use of performance information  by agency leaders and managers in decision making, as well as  instances where agencies (and their managers) face challenges in doing  so.
Specifically, based on analyses of our survey responses, we identified 14  agencies that may have insights into specific practices that led to recent  improvements in managers’ use of performance information, or ways that  they maintain relatively high levels of use by their managers when  compared to the rest of the government. Figure 9 summarizes the  agencies identified earlier in the report that had statistically significant  increases, or results higher than the government-wide average, on our  index or individual survey questions. As the figure shows, several  agencies had statistically significant results across all three sets of  analyses and therefore may have greater insights to offer: the General  Services Administration, National Aeronautics and Space Administration,  and the National Science Foundation.
In addition, our analyses identified nine agencies where results suggest  managers face challenges that have hampered their ability to use  performance information. Figure 10 summarizes the agencies identified  earlier in the report that had statistically significant decreases, or results  lower than the government-wide average, on our index or individual  survey questions. As the figure shows, the Office of Personnel  Management had statistically significant decreases in all three sets of  analyses.
Four agencies—the Departments of the Treasury and Veterans Affairs,  the Nuclear Regulatory Commission, and the Social Security  Administration—were common to both of the figures above. That is, they  had results that indicate they may have insights on some aspects of using  performance information and face challenges in other aspects. As was  mentioned earlier, to provide proper context, these results should be  considered in relation to the range of agency results and the government- wide average (provided in detail in the agency summaries in appendix I).
Given the prioritization of other activities, such as the recent creation of  the Fed2Fed Solutions program, the PIC has not yet undertaken a  systematic approach that could improve the use of performance  information by managers at lower levels within agencies. Such an  approach would involve identifying and sharing practices that have led to  improved use, as well as identifying common or cross-cutting challenges  that have hampered such use. The results of our analyses could help the  PIC do so, and in a more targeted manner. By identifying and sharing  proven practices, the PIC could further ensure that agency leaders and  managers are aware of effective or proven ways they can use  performance information to inform their decisions across the spectrum of  activities they manage within their agencies. Those proven practices also  may help agency leaders and managers resolve any identified  challenges.
Furthermore, in September 2017, we found that, for the estimated 35  percent of managers who reported familiarity with data-driven reviews,  the more they viewed their programs being subject to a review, the more  likely they were to report the reviews were driving results and were  conducted in line with our leading practices for using performance  information. Despite the reported benefits of and results achieved through  data-driven reviews, they were not necessarily widespread. As noted  above, GPRAMA requires agencies to conduct such reviews for agency  priority goals, which represent a small subset of goals, and they are  required at the departmental level. These reasons may explain why most  managers reported they were not familiar with the reviews.
As a result, we recommended that OMB should work with the PIC to  identify and share among agencies practices for expanding the use of  data-driven reviews. OMB staff agreed with our recommendation but have  yet to address it. In June 2018, OMB updated its annual guidance to  agencies to explicitly encourage them to expand data-driven reviews to  include other goals, priorities, and management areas as applicable to  improve organizational performance. However, as of June 2018, OMB  and the PIC have yet to take any steps to identify and share practices for  expanding the use of these reviews in line with our recommendation.  Given the additional analyses we conducted for this report—which show  that being subject to data-driven reviews is related to greater reported use  of performance information and leading practices that promote such  use—we continue to believe these further actions would help agencies  implement these reviews more extensively. We reiterate the importance  of the September 2017 recommendation and will continue to monitor  OMB’s progress to address it.

	Conclusions

For more than 20 years, our work has highlighted weaknesses in the use  of performance information in federal decision making. While the  Executive Branch has taken some actions in recent years, such as  establishing a framework for performance management across the  federal government, our survey results underscore that more needs to be  done to improve the use of performance information more extensively  within agencies and government-wide. The President’s Management  Agenda and its related CAP goal to leverage data as a strategic asset  present an opportunity to do so, as it aims to improve data-driven  decision making. As OMB and its fellow goal leaders more fully develop  the action plan for achieving this goal, providing additional details for its  plans to improve data-driven decision making would help provide  assurance that it can be achieved.
As part of those initiatives, our survey results could provide a useful guide  for targeting efforts. Officials at each agency could use these results to  identify areas for additional analysis and potential actions that could help  improve the use of performance information across the agency and at  lower levels. Similarly, OMB and the PIC could use the results to identify  broader issues in need of government-wide attention. It will also be  important, however, for OMB and the PIC to go beyond this analysis and  work with agencies to identify and share proven practices for increasing  the use of performance information at lower levels within agencies, as  well as challenges that may be hampering agencies’ ability to do so.

	Recommendations for Executive Action

We are making the following two recommendations to OMB:  The Director of OMB should direct the leaders of the Leveraging Data as  a Strategic Asset CAP Goal to ensure future updates to the action plan,  and the resulting federal data strategy, provide additional details on  improving the use of data, including performance information, more  extensively within federal agencies. The action plan should identify  performance goals; contributing agencies, organizations, programs, and  other activities; those responsible for leading implementation within these  contributors; planned actions; time frames; and means to assess  progress. (Recommendation 1)
The Director of OMB, in coordination with the PIC, should prioritize efforts  to identify and share among agencies proven practices for increasing,  and challenges that hamper, the use of performance information in  decision making more extensively within agencies. At a minimum, this  effort should involve the agencies that our survey suggests may offer  such insights. (Recommendation 2)

	Agency Comments

We provided a draft of this report to the Director of the Office of  Management and Budget for review and comment. We also provided a  draft of the report to the heads of each of the 24 federal agencies covered  by our survey.
OMB had no comments, and informed us that it would assess our  recommendations and consider how best to respond.
We are sending copies of this report to congressional requesters, the  Director of the Office of Management and Budget, the heads of each of  the 24 agencies, and other interested parties. This report will also be  available at no charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-6806 or mcneilt@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of our report. Key contributors to this report are listed in appendix  IV.

Appendix I: Summaries of Agency Survey Results

	U.S. Department of Agriculture

(USDA) (Goverment-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Commerce

(Commerce) (Goverment-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to 5 Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Defense

(Goverment-wide) (DOD)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Education

(Education) (Goverment-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Energy

(Government-wide) (Energy)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Health & Human Services

(HHS) (Goverment-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Partices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

	Department of Homeland Security

(DHS) (Goverment-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Housing and Urban Development

(Government-wide) (HUD)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

	Department of the Interior

(Interior) (Government-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Justice

(DOJ) (Government-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Labor

(DOL) (Government-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

	Department of State

(State) (Government-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Transportation

(DOT) (Government-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of the Treasury

(Treasury) (Government-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Department of Veterans Affairs

(Government-wide) (VA)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	U.S. Agency for International Development

(Goverment-wide) (USAID)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

	Environmental Protection Agency

(Government-wide) (EPA)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	General Services Administration

(Government-wide) (GSA)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	National Aeronautics and Space Administration

(Government-wide) (NASA)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	National Science Foundation

(Government-wide) (NSF)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Nuclear Regulatory Commission

(NRC) (Government-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Office of Personnel Management

(Government-wide) (OPM)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Partices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Small Business Administration

(SBA) (Government-wide)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

	Social Security Administration

(Government-wide) (SSA)

		11 Questions that Comprise the Index

		Performance information is used to:

10. The individual I report to 11. Employees that report to me 0 Percent of managers reporting “Great” or “Very Great”

		10 Questions Related to Leading Practices That Promote the Use of Performance Information

		Leading Practices:

		Aligning agencywide goals, objectives, and measures

		Improving the usefulness of performance information

		Communicate performance information frequently and effectively

Appendix II: Objectives, Scope, and Methodology

This report responds to a request that we analyze agency-level results  from our 2017 survey of federal managers at the 24 agencies covered by  the Chief Financial Officers (CFO) Act of 1990, as amended, to determine  the extent agencies are using performance information. This report  assesses the extent to which:  1.  the reported use of performance information and related leading  practices at 24 agencies has changed compared to our prior survey in  2013;  2.  being subject to data-driven reviews related to managers’ reported  use of performance information and leading practices; and  3.  the Executive Branch has taken actions to enhance agencies’ use of  performance information in various decision-making activities.
From November 2016 through March 2017, we administered our online  survey to a stratified random sample of 4,395 individuals from a  population of 153,779 mid- and upper-level civilian managers and  supervisors at the 24 CFO Act agencies. The management levels  covered general schedule (GS) or equivalent schedules at levels  comparable to GS-13 through GS-15, and career Senior Executive  Service (SES) or equivalent. We obtained the sample from the Office of  Personnel Management’s Enterprise Human Resources Integration  database as of September 30, 2015—the most recent fiscal year data  available at the time. The sample was stratified by agency and whether  the manager or supervisor was a member of the SES. To help determine  the reliability and accuracy of the database elements used to draw our  sample of federal managers for the 2017 survey, we checked the data for  reasonableness and the presence of any obvious or potential errors in  accuracy and completeness and reviewed our past analyses of the  reliability of this database. We concluded in our September 2017 report  that the data used to draw our sample were sufficiently reliable for the  purpose of the survey. For the 2017 survey, we received usable  questionnaires from about 67 percent of the eligible sample. The  weighted response rate at each agency generally ranged from 57 percent  to 82 percent, except the Department of Justice, which had a weighted  response rate of 36 percent. The overall survey results are generalizable  to the population of managers government-wide and at each individual  agency.
To assess the potential bias from agencies with lower response rates, we  conducted a nonresponse bias analysis using information from the survey  and sampling frame as available. The analysis confirmed discrepancies in  the tendency to respond to the survey related to agency and SES status.  The analysis also revealed some differences in response propensity by  age and GS level; however, the direction and magnitude of the  differences on these factors were not consistent across agencies or  strata. Our data may be subject to bias from unmeasured sources for  which we cannot control. Results, and in particular estimates from  agencies with low response rates such as the Department of Justice,  should be interpreted with caution. However, the survey’s results are  comparable to five previous surveys we conducted in 1997, 2000, 2003,  2007, and 2013.
To address the first objective, we used data from our 2017 survey to  update agency scores on our use of performance information index.   This index, which was last updated using data from our 2013 survey,  averages managers’ responses on 11 questions related to the use of  performance information for various management activities and decision  making. Using 2017 survey data, we conducted statistical analyses to  ensure these 11 questions were still positively correlated. That analysis  confirmed that no negative correlations existed and therefore no changes  to the index were needed. Figure 11 shows the questions that comprise  the index.
After calculating agency index scores for 2017, we compared them to  previous results from 2007 and 2013, and to the government-wide  average for 2017, to identify any statistically significant differences. We  focus on statistically significant results because these indicate that  observed relationships between variables and differences between  groups are likely to be valid, after accounting for the effects of sampling  and other sources of survey error. For each of the 11 questions that  comprise the index, we identified individual agency results, excluding  missing and no basis to judge responses, and determined when they  were statistically significantly different from (1) the agency’s results on the  same question in 2013, or (2) the government-wide average results on  the question in 2017. In this report, we analyzed and summarized the  results of our 2017 survey of federal managers. Due to the limited scope  of the engagement, we did not conduct additional audit work to determine  what may have caused statistically significant changes between our 2017  and past survey results.
To further address this objective we completed several statistical  analyses that allowed us to assess the association between the index and  22 survey questions that we determined relate to leading practices we  previously found promote the use of performance information. See figure  12 for the 22 specific questions related to these five practices that we  included in the analysis.
When we individually tested these 22 survey questions (bivariate  regression), we found that each was statistically significantly and  positively related to the index in 2017. This means that each question,  when tested in isolation from other factors, was associated with higher  scores on the index. However, when all 22 questions were tested  together (multivariate regression), we found that 5 questions continued to  be positively and significantly associated with the index in 2017, after  controlling for other factors.
To conduct this multivariate analysis, we began with a base model that  treated differences in managers’ views of agency performance  management use as a function of the agency where they worked. We  found, however, that a model based on agency alone had little predictive  power (R-squared of 0.04). We next examined whether managers’  responses to these questions reflecting practices that promote the use of  performance information related to their perceptions of agency use of  performance information, independent of agency. The results of this  analysis are presented in table 1 below. Each coefficient reflects the  increase in our index associated with a one-unit increase in the value of a  particular survey question.
Our final multivariate regression model had an R-squared of 0.67,  suggesting that the variables in this model explain approximately 67  percent of the variation in the use index. We also tested this model  controlling for whether a respondent was a member of the SES and found  similar results.
As shown above in table 1, five questions related to three of the leading  practices that promote agencies’ use of performance information were  statistically significant in 2017. These results suggest that, when  controlling for other factors, certain specific efforts to increase agency use  of performance information—such as providing information on the validity  of performance data—may have a higher return and lead to higher index  scores.
With respect to aligning agency-wide goals, objectives, and  measures, we found that each increase in terms of the extent to which  individuals felt that managers aligned performance measures with  agencywide goals and objectives was associated with a 0.08 increase  in their score on the use index.
In terms of improving the usefulness of performance information, we  found that having information on the validity of performance data for  decision making was the strongest predictor in our model (0.18). As  measured here, taking steps to ensure the performance information is  useful and appropriate was associated with almost as large a change  in a managers’ index score (0.16).
In terms of developing agency capacity to use performance  information, we found that having sufficient analytical tools to collect,  analyze, and use performance information (0.07), and providing or  paying for training that would help link their programs to achievement  of agency strategic goals (0.10), were also statistically significantly  related to a manager’s reported use of performance information.
When we combined these results with what we previously found through  a similar analysis of 2013 survey results in September 2014, we identified  10 questions that have had a statistically significant association with  higher index scores. This reinforces the importance of the five leading  practices to promote the use of performance information. For each of  these questions, which are outlined in figure 13 below, we determined  when agency results were statistically significantly different from 2013  results or the 2017 government-wide average.
For the second objective, we examined, based on the extent they  responded their programs had been subject to agency data-driven  reviews, differences in managers’ use index scores and responses on  questions related to practices that promote the use of performance  information. We grouped managers based on the extent they reported  their programs had been subject to these reviews, from “no extent”  through “very great extent.” We then calculated the average index scores  for the managers in each of those five categories.
We also examined differences in how managers responded to the 10  questions reflecting practices that can promote the use of performance  information based on the extent they reported their programs had been  subject to data-driven reviews. We grouped managers into three  categories based on the extent they reported their programs had been  subject to these reviews (no-small extent, moderate extent, great-very  great extent). We then compared how these groups responded to the ten  questions.
For the third objective, we reviewed our past work that assessed Executive Branch activities to enhance  the use of performance information;  various resources (i.e., guidance, guides, and playbooks) developed  by the Office of Management and Budget (OMB) and the Performance  Improvement Council (PIC) that could support agencies’ use of  performance information; and  the President’s Management Agenda, and related materials with  information on cross-agency efforts to improve the use of data in  federal decision making.
Lastly, for the third objective we also interviewed OMB and PIC staff  about any actions they have taken, or planned to take, to further support  the use of performance information across the federal government.
We conducted this performance audit from October 2017 to September  2018 in accordance with generally accepted government auditing  standards. Those standards require that we plan and perform the audit to  obtain sufficient, appropriate evidence to provide a reasonable basis for  our findings and conclusions based on our audit objectives. We believe  that the evidence obtained provides a reasonable basis for our findings  and conclusions based on our audit objectives.

Appendix III: Comparison of 2007, 2013, and 2017 Agency Use of Performance Information Index Scores

Appendix IV: GAO Contact and Staff Acknowledgments

	GAO Contact

	Staff Acknowledgments

In addition to the above contact, Benjamin T. Licht (Assistant Director)  and Adam Miles (Analyst-in-Charge) supervised this review and the  development of the resulting report. Arpita Chattopadhyay, Caitlin Cusati,  Meredith Moles, Dae Park, Amanda Prichard, Steven Putansu, Alan  Rozzi, Shane Spencer, and Khristi Wilkins also made key contributions.  Robert Robinson developed the graphics for this report. Alexandra  Edwards, Jeff DeMarco, Mark Kehoe, Ulyana Panchishin, and Daniel  Webb verified the information presented in this report.

Related GAO Products

Results of the Periodic Surveys on Organizational Performance and  Management Issues  Managing for Results: Further Progress Made in Implementing the GPRA  Modernization Act, but Additional Actions Needed to Address Pressing  Governance Challenges. GAO-17-775. Washington, D.C.: September 29,  2017.
Supplemental Material for GAO-17-775: 2017 Survey of Federal  Managers on Organizational Performance and Management Issues.  GAO-17-776SP. Washington, D.C.: September 29, 2017.
Program Evaluation: Annual Agency-wide Plans Could Enhance  Leadership Support for Program Evaluations. GAO-17-743. Washington,  D.C.: September 29, 2017.
Managing for Results: Agencies’ Trends in the Use of Performance  Information to Make Decisions. GAO-14-747. Washington, D.C.:  September 26, 2014.
Managing for Results: Executive Branch Should More Fully Implement  the GPRA Modernization Act to Address Pressing Governance  Challenges. GAO-13-518. Washington, D.C.: June 26, 2013.
Managing for Results: 2013 Federal Managers Survey on Organizational  Performance and Management Issues, an E-supplement to GAO-13-518.  GAO-13-519SP. Washington, D.C.: June 26, 2013.
Program Evaluation: Strategies to Facilitate Agencies’ Use of Evaluation  in Program Management and Policy Making. GAO-13-570. Washington,  D.C.: June 26, 2013.
Government Performance: Lessons Learned for the Next Administration  on Using Performance Information to Improve Results. GAO-08-1026T.  Washington, D.C.: July 24, 2008.
Government Performance: 2007 Federal Managers Survey on  Performance and Management Issues, an E-supplement to  GAO-08-1026T. GAO-08-1036SP. Washington, D.C.: July 24, 2008.
Results-Oriented Government: GPRA Has Established a Solid  Foundation for Achieving Greater Results. GAO-04-38. Washington, D.C.:  March 10, 2004.
Managing for Results: Federal Managers’ Views on Key Management  Issues Vary Widely Across Agencies. GAO-01-592. Washington, D.C.:  May 25, 2001.
Managing for Results: Federal Managers’ Views Show Need for Ensuring  Top Leadership Skills.GAO-01-127. Washington, D.C.: October 20, 2000.
The Government Performance and Results Act: 1997 Governmentwide  Implementation Will Be Uneven. GAO/GGD-97-109. Washington, D.C.:  June 2, 1997.

Output:
Why GAO Did This Study

To reform the federal government and make it more efficient and effective, agencies need to use data about program performance. The benefit of collecting performance information is only fully realized when it is used by managers to make decisions aimed at improving results.
GAO was asked to review agencies' use of performance information. This report assesses, among other things, the extent to which: (1) 24 agencies' reported use of performance information and related leading practices has changed since 2013 and (2) the Executive Branch has taken actions to enhance the use of performance information.
To address the first objective, GAO analyzed results from its 2017 survey of federal managers, and compared them to 2013 results. The survey covered a stratified random sample of 4,395 managers from the 24 Chief Financial Officers Act agencies. The survey had a 67 percent response rate and results can be generalized to the population of managers government-wide and at each agency. For the second objective, GAO reviewed agency documents and interviewed staff from OMB and the PIC.

What GAO Found

Agencies' reported use of performance information to make decisions, and leading practices that can promote such use, generally has not improved since GAO's last survey of federal managers in 2013. However, GAO's survey results continue to point to certain practices that could help agencies improve managers' use of performance information. For example, as shown in the table below, GAO's survey found that managers whose programs were subject to data-driven reviews (regular reviews used to assess progress on select agency goals) to a greater extent reported statistically significantly greater use of performance information to make decisions.
The Executive Branch has begun taking steps to improve the use of performance information within agencies and across the government. For example,
In the President's Management Agenda and government-wide reform plan, released in March and June 2018 respectively, the administration acknowledged the need to do more, and announced a goal, among other actions, to improve the use of data in federal decision making. However, the Office of Management and Budget (OMB) and others responsible for this goal have yet to fully develop action plans to hold agencies accountable for achieving it.
The Performance Improvement Council (PIC), which is chaired by OMB, has undertaken efforts to improve the use of performance information by, for example, creating a working group on agency performance reviews. But it has not yet taken a systematic approach to identify and share proven practices that led to, or challenges that may be hampering, increased use of performance information by managers. GAO's survey results identified agencies that may have insights into such practices and challenges.
More fully developing action plans for the new goal, and identifying and sharing proven practices and challenges, could help ensure the Executive Branch takes further steps to improve the use of performance information by managers within agencies and across the federal government.

What GAO Recommends

To improve the use of performance information within agencies and across the federal government, GAO recommends that OMB work with (1) fellow goal leaders to more fully develop action plans for the new goal to improve the use of data and (2) the PIC to prioritize efforts to identify and share proven practices and challenges. OMB had no comments on this report.


Input:
	Introduction

Each of the four major federal land management agencies has maintenance responsibility for tens of thousands of diverse assets in dispersed locations. These agencies are the Bureau of Land Management (BLM), Fish and Wildlife Service (FWS), and National Park Service (NPS), all within the Department of the Interior (DOI), and the Forest Service (FS) within the Department of Agriculture. These agencies maintain assets to preserve their functioning and to repair and replace components as needed.
The infrastructure needs of the federal land management agencies have been a subject of significant federal and public attention for many years. Congressional and administrative attention has focused on deferred maintenance and repairs , defined as "maintenance and repairs that were not performed when they should have been or were scheduled to be and which are put off or delayed for a future period." "Maintenance and repair" include a variety of activities intended to preserve assets in an acceptable condition, including activities such as preventive maintenance and replacement of parts, systems, and components. These terms do not include activities intended to expand the capacity of assets to allow them to serve different purposes or significantly increased needs.
Deferred maintenance and repairs often are called the maintenance backlog . The agencies assert that continuing to defer the maintenance and repair of facilities accelerates the rate of these facilities' deterioration, increases their repair costs, and decreases their value. Debate has focused on varied issues, including the level of funds needed to reduce deferred maintenance, whether agencies are using existing funding efficiently, the priority of deferred maintenance relative to regular maintenance, and whether additional sources of funds should be directed to maintenance. Other issues include how to balance the maintenance of existing infrastructure with the acquisition of new assets, whether disposal of assets is desirable given limited funding, and how much to prioritize maintaining infrastructure relative to other government functions.
Another issue relates to the dollar amount of deferred maintenance and the reasons for fluctuations over time. This report focuses on these issues. It first provides agency deferred maintenance estimates for FY2018, the most recent fiscal year for which this information is available. It also discusses changes in deferred maintenance over the past decade (FY2009-FY2018) and then identifies some of the factors that likely contributed to these changes.

	Estimates

The agencies typically identify deferred maintenance through periodic condition assessments of facilities. FS currently reports an annual deferred maintenance dollar total composed of estimates for 10 classes of assets. These classes include roads, buildings, trails, bridges, and water systems, among others. DOI currently reports annual deferred maintenance composed of estimates for four broad categories of assets: (1) roads, bridges, and trails; (2) irrigation, dams, and other water structures; (3) buildings; and (4) other structures. The "other structures" category includes a variety of assets (e.g., recreation sites and hatcheries).
For each of the 10 years covered by this report, FS has reported the amount of deferred maintenance as a single figure. DOI agencies began reporting deferred maintenance as a single figure in FY2015. In prior years, DOI agencies reported estimates as a range. For FY2014, for instance, the range had an "accuracy level of minus 15 percent to plus 25 percent of initial estimate." According to DOI, a range had been used because "due to the scope, nature, and variety of the assets entrusted to DOI, as well as the nature of deferred maintenance itself, exact estimates are very difficult to determine."
FS estimates of deferred maintenance included in this report generally are taken from the agency's annual budget justifications to Congress. The DOI Budget Office provided the Congressional Research Service (CRS) with a deferred maintenance range for each DOI agency for each fiscal year from FY2009 to FY2014. From these ranges, CRS calculated mid-range figures. For instance, DOI estimated NPS deferred maintenance for FY2014 at between $9.31 billion and $13.70 billion. The CRS-calculated mid-range figure is $11.50 billion. This report reflects CRS's mid-range calculations for FY2009-FY2014 to facilitate comparison with FS estimates. Since FY2015, the DOI Budget Office has provided CRS with a single estimate for each DOI agency, and those figures are used in this report. They represent deferred maintenance as of the end of the fiscal year (i.e., September 30). For both FS and DOI agencies, the deferred maintenance estimates generally reflect project costs. Finally, totals shown in the body and in tables of this report may not add to 100% due to rounding.

		FY2018

The four agencies had combined FY2018 deferred maintenance estimated at $19.38 billion. The agencies had widely varying shares of the total. NPS had the largest portion, 62%, based on an estimate of $11.92 billion. The FS share was 27% of the total, with an estimated deferred maintenance of $5.20 billion. The FWS portion was 7%, reflecting the agency's deferred maintenance of $1.30 billion. BLM had the smallest share, 5%, based on a backlog estimate of $0.96 billion.
Each agency's deferred maintenance estimate for FY2018 consisted of various components. For FS, the single largest asset class was roads, which comprised 61% of the FY2018 total of $5.20 billion. The next largest asset class was buildings, which represented 24% of the FS FY2018 total. The next two largest asset classes were trails and bridges, each with 5%. Six other asset classes made up the remaining 6%.
For NPS, the largest asset category was roads, bridges, and trails, which comprised 57% of the FY2018 deferred maintenance total of $11.92 billion. The buildings category comprised 19% of the total, followed by 18% for other structures and 6% for irrigation, dams, and other water structures.
Roads, bridges, and trails also reflected the largest share of BLM's FY2018 deferred maintenance, with 69% of the $0.96 billion total. Two other categories of assets had relatively comparable portions, specifically 14% for buildings and 12% for other structures. The remaining 6% was for irrigation, dams, and other water structures.
Roads, bridges, and trails made up the smallest portion of FWS's FY2018 deferred maintenance ($1.30 billion), unlike for the other agencies. Moreover, the four asset categories had roughly comparable portions, as follows: 27% for buildings; 27% for other structures; 24% for irrigation, dams, and other water structures; and 22% for roads, bridges, and trails.

		Overview of Decade (FY2009-FY2018)

			Changes in Estimates in Current and Constant Dollars

As shown in Table 1 and Figure 1 , in current dollars, the total deferred maintenance estimate for the four agencies showed considerable variation over the 10-year period from FY2009-FY2018, with a peak in FY2012. It ended the decade relatively flat, with an increase of $0.36 billion overall, from $19.02 billion to $19.38 billion, or 2%. Both the BLM and NPS estimates increased, by $0.42 billion (80%) and $1.75 billion (17%), respectively. By contrast, both the FWS and FS estimates decreased, by $1.71 billion (57%) and $0.11 billion (2%), respectively.
Within these overall changes, there was considerable variation among agency trends. The NPS estimate increased fairly steadily for several years, fell in FY2016, then rose again. The FS estimate was similar at the beginning and end of the decade, although it fluctuated between $5.10 billion and $6.03 billion throughout the 10-year period. The BLM estimate also fluctuated, falling in the first few years of the decade, then rising, leveling off, and rising again to a new high at the end of the decade. The FWS estimate had a generally steady decline during the first several years, leveled off somewhat after FY2015, and reached a decade low in FY2018. Figure 1 depicts the annual changes in current dollars for each agency and for the four agencies combined. Factors that might have contributed to the changes are discussed in the " Issues in Analyzing Deferred Maintenance " section, below.
By contrast, as shown in Table 2 and Figure 2 , in constant dollars, the total deferred maintenance estimate for the four agencies decreased over the course of the ten-year period by $3.61 billion, from $22.99 billion to $19.38 billion, or 16%. Three agencies had overall decreases: $0.37 billion (3%) for NPS, $1.22 billion (19%) for FS, and $2.34 billion (64%) for FWS. However, the BLM estimate increased by $0.32 billion (50%) over the 10-year period.
As was the case for current-dollar estimates, the overall changes in constant dollars reflected various fluctuations. The BLM estimate fell and rose during the period, with the lowest estimate in FY2011 and the highest at the end (FY2018). The FWS estimate exceeded $3 billion for each of the first four years before dropping steeply over the next six years to roughly one-third of the FY2009 level. The NPS estimate peaked in FY2010, then mainly declined, until increasing in FY2018. The FS estimate exceeded $6 billion for the first half of the 10-year period. It ranged roughly between $5 billion and $6 billion during the second half of the period, reaching a low of $5.20 billion in both FY2017 and FY2018. Figure 2 depicts the annual changes in constant dollars for each agency and for the four agencies combined.

			Agency Shares of Deferred Maintenance in Current and Constant Dollars

Throughout the decade, agency shares of the deferred maintenance totals differed, as shown in Figure 3 and Figure 4 . In both current and constant dollars, in each fiscal year NPS had the largest portion of total deferred maintenance and considerably more than any other agency. FS consistently had the second-largest share, followed by FWS and then BLM. Moreover, in both current and constant dollars, each agency's portion of the total annual deferred maintenance changed over the decade. Specifically, the NPS portion of the annual total grew overall throughout the period, from 53% in FY2009 to 62% in FY2018. By contrast, the FS share of the total decreased over the 10-year period from 28% to 27%. The FWS component also declined, from 16% to 7%, whereas the BLM portion rose from 3% to 5%.
The asset class or category that included roads typically comprised the largest portion of each agency's deferred maintenance. Roads represented the largest portion of FS deferred maintenance from FY2009 to FY2018. Over the 10-year period, the NPS roads, bridges, and trails category had the highest share of the agency's deferred maintenance, and irrigation, dams, and other water structures had the smallest. In some years, the portion of NPS deferred maintenance for the "other structures" category exceeded the buildings portion, but in some years the reverse was the case. Roads, bridges, and trails also was the biggest category of BLM's deferred maintenance from FY2009 to FY2018. Although this category typically represented a majority of the FWS total deferred maintenance in the earlier part of the period, this has not been the case since FY2013. A decline in the dollar estimate for roads, bridges, and trails resulted in a sizeable drop in overall FWS deferred maintenance beginning in FY2013, as discussed below.

	Issues in Analyzing Deferred Maintenance

Fluctuations in deferred maintenance estimates are likely the result of many factors, among them estimation methods, levels of funding, and asset portfolios, as discussed below. The extent to which these and other factors affected year-to-year changes in any one agency's maintenance backlog is unclear, in part because comprehensive information is not readily available in all cases or has not been examined. Therefore, the data in this report may not fully explain the changes in deferred maintenance estimates over time.

		Methodology

Methods for assessing the condition of assets and estimating deferred maintenance have changed over the years. As a result, it is unclear what portion of the change in deferred maintenance estimates is due to the addition of maintenance work that was not done on time and what portion may be due to changes in methods of assessing and estimating deferred maintenance. With regard to facility assessment, agencies have enhanced efforts to define and quantify the maintenance needs of their assets. Efforts have included collecting comprehensive information on the condition of facilities and maintenance and improvement needs. For instance, the first cycle of comprehensive condition assessments of NPS industry-standard facilities was completed at the end of FY2006. However, through at least FY2018, NPS continued to develop business practices to estimate the maintenance needs of nonindustry-standard assets. This category presents particular challenges because it includes unique asset types.
Alterations in methodology have contributed to changes in deferred maintenance estimates, as shown in the following examples for roads. The FY2015 FWS budget justification states that
[i]n 2012, Service leadership concluded that condition assessment practices and policies in place at that time were unintentionally producing higher than appropriate [deferred maintenance (DM)] cost estimates for some types of constructed real property. DM estimates for our extensive inventory of gravel and native surface roads are a major contributor to this challenge. In response, the FWS is refining its practices and procedures to improve consistency of DM cost estimates and their use in budget planning. Significant reductions in the DM backlog are resulting from this effort.
Subsequent FWS budget justifications have elaborated on changes to methods of estimating deferred maintenance for roads. For instance, the FY2017 document states that "deferred maintenance estimates for our extensive inventory of roads were further classified to emphasize public use and traffic volume. As a result, minimally used administrative roads are now generally excluded from contributing to deferred maintenance backlog calculations." Of note is that the roads, bridges, and trails category of FWS deferred maintenance declined substantially (by $1.18 billion, 81%) in the past several years in current dollars, from $1.46 billion in FY2012 to $0.28 billion in FY2018. This decline is reflected in the smaller FWS deferred maintenance total for FY2018 ($1.30 billion). The FWS change in the method of estimating deferred maintenance for roads, bridges, and trails appears to be a primary reason for the decreased estimate for this category and total FWS deferred maintenance over the 10-year period.
Similarly, FS attributes variations in deferred maintenance partly to changes in the methodology for estimating roads. For example, in FY2013 and FY2014, FS adjusted the survey methodology for passenger-car roads, with the goal of providing more accurate estimates of the roads backlog. The FS estimate of deferred maintenance for roads fell in current dollars by $0.84 billion (22%) from FY2012 to FY2014, from $3.76 billion to $2.92 billion. The extent to which the drop is attributable to changes in methodology, including regarding the types of roads reflected in the estimates, is not certain.
Finally, in FY2014, the NPS first reflected deferred maintenance for unpaved roads as part of its total deferred maintenance estimate (in agency financial reports). The agency's total deferred maintenance increased in current dollars by $0.26 billion (4%) from FY2013 to FY2014, from $6.57 billion to $6.83 billion. DOI cited the inclusion of unpaved roads as among the reasons for changes in NPS deferred maintenance estimates, although the extent of the effect on NPS estimates is unclear.
Broader changes in methodology also occurred during the decade. For example, DOI agencies had been using an accuracy range of -15% to +25% to derive the estimated range of deferred maintenance for industry-standard assets. The change to a single estimate beginning in FY2015 would have affected DOI deferred maintenance estimates as reflected in this report.

		Funding

How much total funding is provided each year for deferred maintenance for the four agencies is unclear because annual presidential budget requests, appropriations laws, and supporting documents typically do not aggregate funds for deferred maintenance. Portions of deferred maintenance funding (for one or more of the four agencies) have come from agency maintenance and construction accounts, recreation fees, the Highway Trust Fund (Department of Transportation) for roads, the Timber Sale Pipeline Restoration Fund (for FS and BLM), NPS concession fees, and the NPS Centennial Challenge account, among other accounts.
In addition, funding figures are not directly comparable to deferred maintenance estimates because the estimates are limited to project costs and thus do not reflect indirect costs, such as salaries and benefits for government employees. Annual appropriations figures typically reflect indirect costs. Evaluations of the sufficiency of federal funding for deferred maintenance may be hindered by the lack of total funding figures and by the incomparability of appropriations and deferred maintenance estimates.
Deferred maintenance estimates might vary due to economic conditions that are not related to agency efforts or within the control of facility managers. For example, if deferred maintenance estimates reflect costs of needed materials, fuel, supplies, and labor, then the cost of deferred maintenance might change as the costs of these products and services change. Further, DOI has noted that NPS deferred maintenance estimates could fluctuate with market trends and inflation.
Moreover, consistent and comprehensive information on the effect of federal funding on the condition of facilities and deferred maintenance over the decade does not appear to be available in budget documents. In particular, information based on the facilities condition index (FCI) seems to be incomplete or inconsistent in agency budget justifications. In some cases, budget justifications either do not provide FCI figures for assets or provide figures only for certain years. In other cases, it is not clear whether the FCI figures cover all agency assets or a subset of the assets. Together, the budget justifications present a mix of FCI information using quantitative measurements; percentage measurements; and qualitative statements, such as that a certain number or percentage of structures are in "good" condition, but without corresponding FCI figures.
Although amounts and impacts of deferred maintenance funding may not be readily available, the agencies at times have asserted a need for increased appropriations to reduce their backlogs. As a recent example, the Interior Budget in Brief for FY2020 sets out a proposal for the establishment of a "Public Lands Infrastructure Fund," with revenues from energy development on federal lands, to be used for deferred maintenance needs of the four agencies (as well as the Bureau of Indian Education). As a second example, a 2017 audit report asserted that reducing the FS maintenance backlog "will require devoting the necessary resources over an extended period of time," and that "increasing wildfire management costs have left the agency without extra funding to concentrate on reducing deferred maintenance." Moreover, in the past, agencies sometimes attributed reductions in deferred maintenance (or slower rates of increase) in part to additional appropriations, such as those provided in the American Recovery and Reinvestment Act of 2009 (ARRA; P.L. 111-5 ). The FY2016 FWS budget justification notes the ARRA funding as one factor contributing to a reduction in the backlog from the FY2010 high, for instance.
Some observers and stakeholders have identified ways to potentially address deferred maintenance without solely relying on federal funding. For instance, a 2016 report by the Government Accountability Office (GAO) on NPS deferred maintenance listed various actions that NPS is taking at some park units. They include using donations, volunteers, and partnerships to assist with maintenance; leasing assets to nonfederal parties in exchange for rehabilitation or maintenance; and partnering with states in seeking transportation grants. As another example, a 2016 report by a research institute set out options including outsourcing certain agency operations to the private sector, establishing a franchising system for new park units, and disposal of assets.

		Assets

The asset portfolios of the four agencies vary considerably in terms of number, type, size, age, and location of agency assets. Although comprehensive data on these variables over the past decade are not readily available, it is likely that they affect agency maintenance responsibilities and maintenance backlogs. For instance, NPS has more assets than the other DOI agencies, a sizeable portion of which were constructed before 1900 or in the first half of the 20 th century. The 2016 GAO report assessed various characteristics of the NPS maintenance backlog, including the age of park units. The agency determined that of the total FY2015 NPS deferred maintenance, park units established over 100 years ago had the largest share (32%). Further, park units established more than 40 years ago collectively accounted for 88% of all NPS deferred maintenance. Moreover, some NPS assets are in urban areas or are iconic structures, which could affect maintenance costs.
The effect of changes in agency asset portfolios on deferred maintenance is not entirely clear. However, it could be asserted that the acquisition of assets, such as a sizeable number of large or iconic assets in relatively poor condition, would increase regular maintenance needs and the backlog, if maintenance is not performed when scheduled. For instance, the NPS asserted that "when parks are created or when new land is acquired, the properties sometimes come with facilities that are in unacceptable condition or are unstable for the park or partner organizations.... When facilities are excess to the park ... they also contribute to the deferred maintenance backlog." Similarly, it could be argued that disposal of assets, such as a large quantity of old assets in poor condition, could reduce deferred maintenance. For example, a 2017 audit of the FS recommended that the agency "establish goals and milestones to aggressively reduce the number of unused or underused assets in the agency's portfolio" as one way to reduce maintenance backlogs given limited resources.
Agencies examine whether to retain assets in their current condition or dispose of some assets, as the following examples indicate. FS has sought to reduce its maintenance backlog by conveying unneeded or underused administrative sites, as well as decommissioning roads, road and facility infrastructure , and nonpriority recreation sites. FWS has attributed reductions in deferred maintenance in part to "disposing of unneeded assets." NPS identifies assets that are not critical to the agency's mission and that are in relatively poor condition for potential disposal. In the past, the agency has noted that although the agency seeks to improve the condition of its asset portfolio by disposing of assets, "analysis of removal costs versus annual costs often precludes the removal option."

Output:
Each of the four major federal land management agencies maintains tens of thousands of diverse assets, including roads, bridges, buildings, and water management structures. These agencies are the Bureau of Land Management (BLM), Fish and Wildlife Service (FWS), National Park Service (NPS), and Forest Service (FS). Congress and the Administration continue to focus on the agencies' deferred maintenance and repair of these assets—in essence, the cost of any maintenance or repair that was not done when it should have been or was scheduled to be done. Deferred maintenance and repair is often called the maintenance backlog.
In FY2018, the most recent year for which these estimates are available, the four agencies had combined deferred maintenance estimated at $19.38 billion. This figure includes $11.92 billion (62%) in deferred maintenance for NPS, $5.20 billion (27%) for FS, $1.30 billion (7%) for FWS, and $0.96 billion (5%) for BLM. The estimates reflect project costs.
Over the past decade (FY2009-FY2018), the total deferred maintenance for the four agencies fluctuated, peaking in FY2012 and ending the decade relatively flat in current dollars. It increased overall by $0.36 billion, from $19.02 billion to $19.38 billion, or 2%. Both the BLM and NPS estimates increased, whereas the FWS and FS estimates decreased. By contrast, in constant dollars, the total deferred maintenance estimate for the four agencies decreased from FY2009 to FY2018 by $3.61 billion, from $22.99 billion to $19.38 billion, or 16%. The BLM estimate increased, whereas estimates for the other three agencies decreased.
In each fiscal year, NPS had the largest portion of the total deferred maintenance, considerably more than any of the other three agencies. FS consistently had the second-largest share, followed by FWS and then BLM. Throughout the past decade, the asset class that included roads comprised the largest portion of the four-agency combined deferred maintenance.
Congressional debate has focused on varied issues, including the level and sources of funds needed to reduce deferred maintenance, whether agencies are using existing funding efficiently, how to balance the maintenance of existing infrastructure with the acquisition of new assets, whether disposal of assets is desirable given limited funding, and the priority of maintaining infrastructure relative to other government functions.
Some question why deferred maintenance estimates have fluctuated over time. These fluctuations are likely the result of many factors, among them the following:
Agencies have refined methods of defining and quantifying the maintenance needs of their assets. Levels of funding for maintenance, including funding to address the maintenance backlog, vary from year to year. Economic conditions, including costs of services and products, also fluctuate. The asset portfolios of the agencies change, with acquisitions and disposals affecting the number, type, size, age, and location of agency assets.
The extent to which these and other factors affected changes in each agency's maintenance backlog over the past decade is not entirely clear. In some cases, comprehensive information is not readily available or has not been examined.