Input:
	Background

Nurse staffing is a critical part of health care because of the effects it can  have on patient outcomes and nurse job satisfaction. According to VHA,  its staffing methodology aims to maximize nurses’ productivity and  efficiency, while providing safe patient care by ensuring appropriate nurse  staffing levels and skill mix.

		VHA Nurse Workforce

VHA’s nurse workforce is primarily composed of RNs, licensed practical  nurses (LPN), and nursing assistants (NA). These nurses provide  care—ranging from primary care to complex specialty care—in inpatient,  outpatient, and residential care settings at 151 VAMCs across the  country. In addition to the size of the nursing workforce, the nursing skill  mix—i.e., the share of each type of nurse (RNs, LPNs, or NAs) of the  total—is an important component of nurse staffing. Units vary in their  nursing skill mix, depending on the needs of their patients. For example,  intensive care units require higher intensity nursing, and may have a skill  mix that is primarily composed of RNs compared to other types of nursing  units that may provide less complex care. (See table 1 for a general  description of the types of nursing staff position, responsibilities, and  educational requirements.)
Although the number of nurses at VAMCs increased from FY 2009 to   FY 2013, VHA ranked nurses as the second most challenging occupation  to recruit and retain. Specifically, the total number of nurses at VAMCs  increased 13 percent from 72,542 in FY 2009 to 81,940 in FY 2013, with  similarly proportionate increases within each position type—RN, LPN, and  NA. During the same time period, the annual nurse turnover rate at  VAMCs—the percentage of nurses who left VHA through retirement,  death, termination, or voluntary separation—increased from 6.6 percent  to 8.0 percent. Although RNs had the lowest turnover rate among nurses,  VHA noted particular difficulty recruiting and retaining for the position,  particularly for RNs with advanced professional skills, knowledge, and  experience, such as RNs that provide services in medical and surgical   VHA projects that approximately 40,000 new nurses will be  care units. needed through FY 2018 to maintain current staffing levels and to meet  the needs of veterans.
See Department of Veterans Affairs, Veterans Health Administration, 2013 Workforce  Succession Strategic Plan (Washington, D.C.: 2013).

		VHA’s Nurse Staffing Methodology

To help ensure adequate and qualified nurse staffing at VAMCs, in July  2010, VHA issued VHA Directive 2010-034: Staffing Methodology for  VHA Nursing Personnel. ONS, the VHA office responsible for providing  national policies and guidelines for all VHA nursing personnel, led the  development of the nurse staffing methodology, which began in 2007.  (See fig. 1.)
To implement the methodology, each VAMC is required to (1) develop a  VAMC-wide staffing plan for its nurse workforce, comprised of individual  unit-level staffing plans, and (2) execute that plan. (See figure 2 for an  outline of the process for implementing VHA’s nurse staffing  methodology.)
Each VAMC unit is to develop a staffing plan outlining recommendations  on the appropriate nurse staffing levels and skill mix needed in that unit to  support high-quality patient care in the most effective manner possible.  Specifically, staffing plans are to be developed using expert panels and a  data-driven analysis of nursing hours per patient day (NHPPD). VAMC  nurse executives—members of senior management within each VAMC— are responsible for implementing the staffing methodology in their  respective VAMCs.
Expert panels: advisory groups—at the unit and facility level—of  VAMC staff with in-depth knowledge of nurse staffing needs. The use  of expert panels is intended to apply principles of shared governance,  which allows nurses to have influence over the delivery of patient care  and involves stakeholders from across the VAMC. VAMC nurse  executives are responsible for ensuring that the unit-based expert  panels represent all nursing types (RN, LPN, NA) and developing the  VAMC’s facility expert panel.
Data-driven analysis of NHPPD: involves determining the number and  skill mix of nurses needed for each unit by calculating the number of  direct patient care nursing hours provided for all patients on that unit  during a 24-hour period. The use of NHPPD represents a move  away from the more traditional nurse-to-patient ratios that assign a  certain number of patients to each nurse. Some research suggests  that NHPPD can better capture changes in nurses’ workloads and  case mix resulting from admissions and discharges, as well as patient  acuity levels, which can impact the amount of time nurses spend with  each patient.
After developing the staffing plan, each unit-based expert panel presents  its plan, which includes staffing recommendations, to the VAMC’s facility  expert panel. Those staffing recommendations may include, for example,  initiatives to change the number and skill mix of nurses needed for each  shift; change the number of nurses required for coverage during predicted  absences, such as annual and sick leave; and develop support services  for nurses, such as designated individuals to transport patients to other  areas of the facility as needed. The facility expert panel—comprised of  staff from across the VAMC—reviews each unit-based panel’s staffing  plan and aggregates all of the unit plans into one VAMC-wide staffing  plan. The VAMC nurse executive reviews the VAMC-wide staffing plan  and forwards it to the VAMC director for review and approval. Once  approved, the VAMC then begins execution of the initiatives outlined in  the VAMC-wide staffing plan. The directive requires each VAMC to  conduct an ongoing staffing analysis to evaluate staffing plans annually,  at a minimum, and for VAMC directors to incorporate projected staffing  needs into their annual budget review.
The staffing methodology is being implemented in three phases.
In Phase I, VAMCs were to implement the staffing methodology in all  inpatient units no later than September 30, 2011.
In Phase II, VAMCs are to implement the staffing methodology for all  other units, including the operating room, emergency department, and  spinal cord injury units. ONS has completed the Phase II pilot for  operating room units, and VAMCs are expected to implement the  methodology in their operating room units by October 1, 2014.  Deadlines for the implementation in other Phase II units have not  been set.
In Phase III, VAMCs are to use an automated system developed by  VHA that (1) merges VHA staffing data used in the staffing  methodology and other VHA data, such as human resource data, into  one data system, and (2) incorporates the data into staffing-related  reports, such as quality-of-care reports. A deadline for Phase III  implementation has not been set.
In May 2014, the VA OIG found that VAMCs in its review varied in their  implementation of the staffing methodology. Specifically, the VA OIG  reported that 8 of the 28 VAMCs reviewed had not fully implemented all  components of the staffing methodology by September 2013, 2 years  past the implementation date required by the VHA directive. As these  findings were similar to those of its April 2013 report, the VA OIG stated  in its 2014 report, “We re-emphasize the need for all facilities to fully  implement the methodology and accurately address patient needs with  safe and adequate staffing.”

		Impact of Nurse Staffing on Patient Outcomes and Nurse Job Satisfaction

Adequate and qualified nurse staffing at VAMCs is required to provide  effective and continuous patient care and to maintain a stable and  engaged workplace. The importance of nurse staffing on patient  outcomes and nurse job satisfaction has been emphasized by various  entities, including The Joint Commission; American Nurses  Association; Institute of Medicine; and Agency for Healthcare  Research and Quality.and qualifications of nurse staffing to patient outcomes and nurse job  satisfaction. For example, studies have shown:   Additionally, research has linked the adequacy
A link between the adequacy of nurse staffing and patient outcomes,  particularly in inpatient units, such as intensive care and surgical  units. For example, medication errors, pressure ulcers, hospital  acquired infections, pneumonia, longer-than-expected stays, and  higher mortality rates each have been associated with inadequate  nurse staffing.
A link between the qualifications of nursing staff and patient  outcomes. For example, one study found that patients cared for in  units utilizing more licensed and experienced nursing staff (RNs and  LPNs) and fewer unlicensed aides (NAs) had shorter lengths of  stay. Other studies linked baccalaureate-prepared nurses to lower  mortality rates.
A link between nurse staffing and job satisfaction. For example, some  studies have linked low job satisfaction to heavy workloads and an  inability to ensure patient safety. Other studies found that improving  nurse staffing and working conditions may simultaneously reduce  nurses’ burnout, risk of turnover, and the likelihood of medical errors,  while increasing patients’ satisfaction with their care.
Non-VA health care organizations use various approaches to ensure  effective nurse staffing. For example, some use fixed nurse-to-patient  ratios while others use adjustable, unit-specific minimum staffing levels,   and there have been several efforts to address nurse staffing using these  different approaches.requiring regulations that mandate specific nurse-to-patient ratios that  limit the number of patients cared for by an individual nurse. Other states  have passed legislation or adopted regulations addressing nurse staffing  without mandating specific ratios or staffing levels. For example, some  states require hospitals to have committees responsible for developing  unit staffing plans or require public reporting of staffing.

	VAMCs in Our Review Implemented VHA’s Nurse Staffing Methodology, Experienced Problems Developing and Executing Staffing Plans, and Some Reported Improvements in Nurse Staffing

All seven VAMCs in our review developed staffing plans using VHA’s  nurse staffing methodology and have taken steps to execute them.  However, VAMCs experienced problems in both the development and  execution of their staffing plans. Improvements in nurse staffing were  reported by some of the VAMCs which had taken steps to execute the  staffing plans.

		VAMCs in Our Review Have Implemented VHA’s Nurse Staffing Methodology by Developing Staffing Plans and Taking Steps to Execute Them

The seven VAMCs in our review have implemented VHA’s nurse staffing  methodology; specifically, each of these VAMCs has developed a facility- wide staffing plan, comprised of unit-level staffing plans for inpatient units,  and has taken steps to execute it. Although each of the seven VAMCs  in our review developed a staffing plan for FY 2013, only one had  developed a plan per VHA’s directive—that is, used both expert panels  and analysis of NHPPD—by September 30, 2011, the deadline specified  in the directive. (See table 2.) Across all 151 VAMCs, according to ONS  officials, VAMCs’ implementation of the nurse staffing methodology  varied, with, for example, some VAMCs completing the development of  their staffing plans during FY 2013, and some only beginning the  development process.
In addition to developing staffing plans, all seven VAMCs in our review  had taken steps to execute their respective staffing plans. For example,  VAMCs had taken steps to execute initiatives to increase the number of  unit nurses or change the skill mix of nurses to address patient care  needs. (See table 3 for examples of VAMCs’ staffing plan initiatives.)  VAMC officials told us there are many factors that could affect the  execution of staffing plan initiatives, such as available resources, the  amount of time needed, and other strategic priorities.

		VAMCs Experienced Problems Developing and Executing Staffing Plans, and Many of These Problems Persist

Officials and nursing staff from the seven VAMCs in our review told us  they experienced problems developing and executing staffing plans. (See  table 4 for examples of problems.) Some VAMCs were able to devise  solutions; however in many cases, the problems have persisted.
Problems Developing Staffing Plans. Staff and officials from each of  the seven VAMCs in our review reported facing problems developing  staffing plans.
Lack of necessary data resources. Staff and officials at six of the  seven VAMCs in our review said they did not have the appropriate  data resources to effectively calculate NHPPD as required by VHA’s  staffing methodology directive. Specifically, the directive instructs  VAMC staff to calculate NHPPD using a wide range of data, such as  number of admissions, transfers, and discharges; hours used for  planning and treatment; and human resources data. We found that  staff and officials needed to use multiple sources to collect the  necessary data, in some cases manually, a process they said was  time-consuming and potentially error-prone, and required data  expertise they did not always have. For example, at one VAMC, the  staffing methodology coordinator—a VAMC official who assists with  the administrative tasks associated with the implementation process— told us she struggled with some data analysis techniques, such as  creating a spreadsheet to help track staffing data, but the VAMC did  not have the financial resources to hire additional data analysts to  support the methodology. In contrast, officials from two VAMCs in our  review told us staffing methodology coordinators were assigned in  part based on their data analysis expertise.
Difficulty completing and understanding training. Staff from six of the  seven VAMCs in our review said the ONS training on the  methodology was time consuming to complete, and difficult to  understand. In 2011, ONS switched from instructor-led, group training  to individual, computer-based PowerPoint training. Many unit staff  reported that because the computer-based training took many hours  to complete, it was difficult to find the time to complete it, while also  carrying out their patient care responsibilities. They told us they often  had to start and stop the training to attend to patients, which  diminished its effectiveness. Further, the course’s complex material  was hard to absorb through an individual, computer-based course,  with many staff suggesting their understanding would have been  greatly improved with an instructor-led, group course where they  could ask questions, ensure consistency of learning, and build  camaraderie among unit expert panel members. To address the  difficulties in completing and understanding the training, one VAMC  developed its own instructor-led, group training provided to all its  units.
Time required. Staff and officials at all seven of the VAMCs in our  review reported that developing staffing plans required a lot of staff  time due to the complexity of the process. In particular, they said  gaining an understanding of the methodology, collecting the  necessary data, convening the unit expert panels, and preparing  presentations for the facility expert panel were time-intensive tasks  that, in some circumstances, took time away from patient care. For  example, members from one unit expert panel estimated they spent,  in total, about 160 hours (4 weeks) developing the unit staffing plan  during the first year the staffing methodology was implemented in their  unit. Some VAMCs’ staffing methodology coordinators developed  specific processes designed to decrease the burden on nursing staff  and improve efficiency. For example, they created templates for unit  panel members to use in staffing plan development; such templates  improved efficiency because unit panel members did not have to  independently develop their presentation format. Further, facility  expert panel members had to orient themselves to only one template,  and were therefore able to more easily make facility-level  comparisons and decisions.
Lack of communication within VAMC. Unit expert panel members at  four of the seven VAMCs in our review said there was a lack of  communication between nurses and VAMC leadership regarding the  status of the staffing plans, including plans for execution of the staffing  plan initiatives. Staff at one of these VAMCs said they had not  received any feedback on their FY 2012 or FY 2013 unit staffing  plans; they added that developing the 2013 staffing plan without  getting any feedback on the prior year’s plan felt “frustrating.” In  contrast, at another VAMC, officials told us that all unit staff—not just  staff involved in the unit panel—received regular updates on the nurse  staffing process at their monthly unit staff meetings.
Difficulty integrating unit staff into expert panels. Staff and officials at  three VAMCs described challenges in integrating unit staff into expert  panels. Some unit panel members told us that although they were  considered members of their respective unit panels, they were not  significantly involved in the development of their units’ staffing plans.  For example, a unit panel member said the VAMC’s staffing  methodology coordinator calculated the unit’s NHPPD, developed the  corresponding unit staffing plan, and presented the unit staffing plan  to the respective facility expert panel almost entirely without her unit’s  input. As a result, there was limited involvement of the unit panel  members in the expert panel and, consequently, limited shared  governance. Officials at these VAMCs said that from their  perspectives, there was interest in the methodology among unit panel  members, but sometimes it was difficult for these staff to attend  relevant meetings because of patient needs. In contrast, unit panel  members at other VAMCs in our review described how they were fully  integrated into the unit panels. They described in detail the data  analyses they prepared, the meetings they participated in, and their  experiences presenting their unit staffing plans to the facility expert  panel. Members from one unit panel told us it was helpful to be able to  use data to validate the unit’s staffing and share this data with the  facility expert panel—VAMC staff “beyond the typical chain of  command.” Officials at this VAMC noted that unit panel members felt  “empowered” to present their work to the facility expert panels.
Problems Executing Staffing Plans. Staff and officials from six of the  seven VAMCs in our review noted problems executing staffing plans once  approved by the VAMC director.
Hiring delays. Staff and officials from six of the seven VAMCs in our  review said they often faced hiring delays that impacted their ability to  execute staffing plan initiatives. Some VAMC staff noted it could take  more than 6 months to fill unit vacancies. Although staff from one  VAMC said hiring was slowed by the dearth of qualified nurses in their  community, staff from other VAMCs in our review said the supply of  nurses was not the problem, but rather the problem was the VHA  hiring process, which took months to complete for each candidate.  Additionally, VAMC staff noted that new hires also needed to  complete necessary internal trainings before joining a unit full time,  which added to the delays, and that some new hires were hurried  through this training process because their units were so desperate to  have them on staff.
Budget constraints. Staff and officials from five of the seven VAMCs in  our review said their VAMCs were not able to fully execute their  staffing plans due to budget constraints. For example, at one VAMC,  one of the approved staffing plan initiatives was the hiring of a large  number of nurses for its units, in part, to address the VAMC’s inability  to increase their nursing staff over a period of years. The official told  us that, due to budget constraints, the VAMC was going to phase in  this hiring initiative over the next few years.

		Improvements in Nurse Staffing Were Reported When VAMCs Executed Staffing Plan Initiatives

Some VAMC staff reported improvements in the adequacy and  qualifications of their units’ nursing staff when nurse staffing plan  initiatives were executed. For example, at two VAMCs at which the  number of nurses was increased or support services for nurses, such as  patient transporters or sitters, were put in place,adequacy of the nursing staff had improved. Furthermore, improvements  in the qualifications of unit nursing staff were noted by staff in VAMC units  where, for example, skill-mix changes were made or the amount of  floating of nurses from their home unit to an unfamiliar unit was  decreased. Both VAMC officials and unit staff noted improvements in  staffing when nurses’ qualifications were more appropriately matched to  the right level of work (for example, having RNs rather than LPNs  available to provide more complex patient care) and to the right units (for  example, the units for which they were hired and trained).
Some VAMC staff said they also had seen improvements in patient  outcomes and nurse job satisfaction. For example, nursing staff at one  VAMC said that after creating sitter positions—as indicated by their  VAMC’s staffing plan—they saw a decrease in patient falls. The staff said  sitters were able to monitor patients more closely, and as a result,  patients were less likely to fall during walks to the bathroom, for example.  Similarly, nursing staff in a mental health unit at another VAMC said that  by having more staff they had decreased their restraint use because there  were more staff available to meet veterans’ needs. Additionally, nursing  staff we interviewed at one VAMC that had made staffing changes based  on staffing plans said they were better able to provide the type of nursing  care “veterans deserve,” and this made them feel more positive about  their work. Some nurses at this VAMC also said the shared governance  aspect of the methodology was empowering, which, combined with their  enhanced understanding of staffing at their VAMC, helped improve their  overall job satisfaction.
However, some VAMC unit staff reported that unit nurse staffing  continued to be inadequate and that nurse unit assignments and job  duties were not always appropriate for their qualifications. For six of the  VAMCs in our review, staff from at least one unit interviewed said their  unit staffing levels were inadequate. Staff said ensuring adequate staffing  was particularly challenging when there were unplanned staff absences  and they had to “scramble” to provide coverage. Some unit staff noted  that this situation often resulted in units forcing nurses to work overtime or  nurses floating to other units where they did not always have the  qualifications to provide care. At some VAMCs, staff said there were  increased staff injuries due to inadequate staffing. Furthermore, staff at  one VAMC reported that where there had not been any changes made  based on the unit staffing plans, their units continued to be understaffed  to the detriment of both patient care and their job satisfaction.

	VHA’s Oversight to Ensure Its Nurse Staffing Methodology Is Implemented, Administered Appropriately, and Contributes to an Adequate and Qualified Nurse Workforce Is Limited

Our review of VHA’s oversight of its nurse staffing methodology found  that some internal controls—those related to environmental assessment,  a plan for monitoring compliance, evaluation, timeliness of  communication, and organizational accountability—are limited. The  implementation of internal controls is necessary for ensuring initiatives  achieve intended outcomes and for minimizing operational problems.  Without these internal controls in place, VHA cannot ensure that its  methodology meets department goals, such as establishing a  standardized methodology for determining adequate and qualified nurse  staffing at all VAMCs, and ultimately, having nurse staffing that is  adequate to meet veterans’ health care needs.
Environmental Assessment. VHA did not comprehensively assess each  VAMC to ensure preparedness for implementing its methodology,  including having the necessary technical support and resources, prior to  the issuance of the methodology directive in 2010. Furthermore, as of  August 2014, VHA did not have a plan for assessing whether VAMCs  have the necessary resources to execute their approved nurse staffing  plans. Under federal internal control standards, successful organizations  monitor their internal and external environments continuously and  systematically, and by building environmental assessments into the  strategic planning process, are able to stay focused on long-term goals  even as they make changes to achieve them.
VHA did not assess VAMCs’ technical resources to determine if all  VAMCs would be able to successfully implement the methodology. For  example, the directive recommended that VAMCs use comparative data  from external sources, such as the National Database of Nursing Quality  Indicators (NDNQI) when analyzing unit-level staffing data. According to  some VAMC officials, due to the costs and complexity of contracting,  not all VAMCs had access to this data source. Each VAMC was  responsible for establishing its own contract to purchase access to  NDNQI data, which some VAMC officials said was expensive and time- consuming to set up, noting that it would have been helpful to have  assistance in coordinating the contracting process. Officials from ONS  reported that they are discussing the possibility of having a VHA-wide  contract so that all VAMCs would have access to NDNQI data. In addition  to access to comparative data, according to the directive, VAMCs need  appropriate data system capabilities—in particular an automated staffing  system for information such as patient admission, transfer and discharge  data, and human resources data—to facilitate implementation of the data- driven methodology and calculation of NHPPD. However, not all VAMCs  in our review had an automated staffing system in place even 3 years  after the release of the directive. Officials at a VAMC without an  automated staffing system told us staff were collecting and inputting data,  in many cases manually, into a spreadsheet to calculate NHPPD, and  that this process was extremely time-consuming and potentially error- prone. ONS officials said they knew VAMCs needed automated staffing  systems when the directive was published in 2010. However, they  thought Phase III—a national automated staffing system—would be  forthcoming, and did not fully review whether VAMCs had alternative data  capabilities to assist them in the interim.
When we asked how they assessed the readiness of VAMCs for  implementation of the methodology, ONS officials told us that they did not  do this as well as they should have for Phase I implementation in  inpatient units, despite its 2009 Phase I pilot evaluation to better  understand the potential capabilities and weaknesses of VAMCs.  According to ONS, it still has not conducted such an assessment of all  VAMCs even though it has moved forward with planning the national  rollout of Phase II in operating room, emergency department, and spinal  cord injury units. ONS, however, has assessed some of the available  resources of the sites that have participated in the pilots for Phase II in  spinal cord injury units. For example, ONS officials told us that they asked  these participating sites questions about their access to data and nurse  turnover within the pilot units to determine their ability to fully and  successfully participate in the pilot. According to ONS officials, all sites  reported that they were able to fully participate in the pilot. By not  comprehensively assessing the VAMCs’ technical support and resources  to determine if they were prepared to implement the methodology, VHA  had no assurance that the VAMCs would be successful.
Plan for Monitoring Compliance. ONS did not develop a plan for  monitoring VAMCs to ensure they were in compliance with the  implementation and ongoing administration of Phase I of the  methodology. Under federal internal control standards, plans should be  designed to ensure that ongoing monitoring occurs in the course of  normal program operations, and managers should identify performance  gaps in compliance with program policies and procedures.
ONS reported implementing two mechanisms for obtaining information  from VAMCs—a 2013 questionnaire sent to all VAMCs and monthly  methodology conference calls with VAMCs—but neither was an adequate  mechanism for comprehensively assessing the compliance of each  VAMC. The questionnaire, sent nearly 2 years after the deadline for  implementation of Phase I of the methodology, asked VAMCs to report  their status of staffing plan development, but because of lack of clarity in  the questions asked, inconsistency in medical center responses, and lack  of validation of the self-reported responses, it was not reliable for  determining the extent to which VAMCs had developed staffing plans.  ONS officials reported that they have no plans to survey VAMCs again on  their status of developing staffing plans. Furthermore, the monthly  methodology conference calls that started when the directive was  published in 2010 did not provide an adequate mechanism for monitoring  compliance because they too relied on VAMCs to self-report problems. A  VAMC official told us that participants were reluctant to raise problems,  such as not developing staffing plans on time, during these monthly calls.
In addition, the directive requires VAMCs to evaluate their staffing plans  for Phase I annually, or more frequently if needed, but ONS officials told  us that they did not have a systematic plan for monitoring compliance with  this evaluation beyond the 2013 questionnaire and the monthly  methodology conference calls. Moving forward, ONS officials said they  plan to review whether all VAMCs implemented both the unit and facility  expert panels, but, as of August 2014, had no detailed plan or timeline for  conducting this review or for monitoring VAMCs’ ongoing evaluation of  their staffing plans. The lack of a plan for monitoring VAMCs’ compliance  with the implementation and ongoing administration of the methodology  hinders VHA from being able to ensure that all VAMCs are staffing their  nurses using the same, standardized methodology.
Evaluation. There have been limited evaluations of the methodology, and  one of these evaluations has been significantly delayed. Under federal  internal control standards, measuring performance allows organizations  to track the progress they are making towards program goals and  objectives, and provides managers important information on which to  make management decisions and resolve any problems or program  weaknesses.
Evaluation of Phase I pilot (conducted in September 2009)—ONS  identified VAMC challenges with implementing the methodology— such as difficulties accessing data, and staff nurses having an overall  lack of knowledge of the methodology process. The evaluation  contained recommendations, such as developing a training guidebook  and providing guidelines on the role of the expert panels, to improve  the methodology process. According to ONS, most of the  recommendations from this 2009 evaluation have been addressed;  however, we found that weaknesses identified in the 2009 evaluation  still existed for all of the seven VAMCs included in our review.
Evaluations of Phase I national implementation and training (began  early 2014, preliminary results were expected August 2014). Similarly,  ONS did not begin an evaluation of the national implementation of the  methodology until January 2014, more than 2 years after VAMCs  were required to have implemented it, and, as of August 2014, had  still not been completed. According to ONS officials, the Phase I  national evaluation was to review VAMCs’ experiences during  implementation, including a review of the training provided to VAMCs  during that phase. The lengthy delay in the evaluation of Phase I was  potentially problematic because the ongoing difficulties that VAMCs  have experienced during implementation may have been avoided or  resolved more quickly if the evaluation results had been available and  corrective actions put into place. VAMC staff we interviewed told us  they have been struggling with components of the methodology since  the directive was issued. For example, some VAMC staff expressed  difficulty completing and understanding the data analysis process for  calculating NHPPD. An earlier evaluation of the methodology could  have helped identify this problem, as well as potential solutions to  address it. Furthermore, the delay limited ONS’s time to apply lessons  learned from Phase I evaluations to the implementation of Phase II,  portions of which are already nearly complete.
Phase II pilot training evaluation (began in early 2014 with results  expected November 2014)—ONS is conducting an evaluation of the  training that was provided to the VAMCs involved in the Phase II pilots  in operating room, emergency department, and spinal cord injury units  to determine if the training provided to these units needs to be  changed in preparation for the national rollout. ONS officials told us  that they have completed the operating room pilot; the national rollout  of the methodology in operating room units in all VAMCs began in  February 2014 and is expected to be completed by October 1, 2014.  ONS officials said that it has completed the pilot for the emergency  department units, but has not completed the pilot for spinal cord injury  units; ONS has not scheduled deadlines for their national  implementation.
VHA’s delays in completing evaluations of the methodology limit its ability  to identify and resolve VAMC implementation and administration  problems, and thus help to ensure successful rollouts of subsequent  phases of the methodology.
Timeliness of Implementation and Communication. The long timeline  for implementing the pilots and national rollouts of Phases I and II, as well  as evaluating Phase I of the staffing methodology—more than 7 years— and for communicating methodology-related information to VAMCs may  have hindered the ability of VAMCs to develop their staffing plans and to  execute the initiatives contained in those plans. Under federal internal  control standards, timeliness in the development of a program or  implementation of a policy is needed to maintain relevance and value in  managing operations and making decisions. When information regarding  a policy or program is not provided in a timely manner, there can be a  loss of stakeholder support, which can affect how stakeholders make  decisions. For example, staff from some VAMCs involved in the Phase II  pilot stated that they believed the data and reports generated from the  methodology were only a paper exercise because they had not gotten  any feedback from ONS on next steps. ONS officials told us they have  communicated information on the Phase II pilot, such as the status of the  pilot and feedback obtained from the training sessions, through their  monthly conference calls with VAMCs; however, based on our interviews,  this information did not reach many staff at the VAMCs in our review that  participated in the Phase II pilot.
Furthermore, ONS officials have not adequately communicated to VAMCs  the status of Phase III of the methodology—development of a national  automated staffing system. According to the directive, a national  automated staffing system was to be developed to support VAMCs in the  implementation of the methodology. Because this automated staffing  system has yet to be developed as per the directive, officials from two  VAMCs told us they bought their own systems, which helped to effectively  administer the methodology. ONS officials told us at the time the directive  was published in 2010, Phase III implementation was an aspirational  goal. ONS officials said they had expected VHA data system teams to  begin the process of developing a national automated system; however, it  was not made a department goal, and is not currently on the list of  projects under consideration for funding. Having a variety of staffing  systems, and thus inconsistent data variables across VAMCs, inhibits  ONS’s ability to adequately evaluate the effectiveness of the staffing  methodology. If an automated staffing system is eventually developed  under Phase III, VAMCs likely will have to dismantle the staffing systems  they have created and restructure their data analysis processes, which  likely will be time-consuming and costly. VHA’s long timelines for the  implementation and communication of methodology-related information  put stakeholder support of the methodology at risk and increase the  potential for duplication of efforts.
Organizational Accountability. VHA did not define areas of  responsibility or establish the appropriate line of reporting within the  framework of VA’s management structure for the ongoing administration  and oversight of the methodology. Under federal internal control  standards, an agency’s organizational structure should provide  management with a framework for planning, directing, and controlling  operations to achieve agency objectives; a good internal control  environment requires that the agency clearly defines key areas of  authority and responsibility. VHA does not require VAMCs to submit any  information or reports on the implementation and ongoing administration  of the methodology to ONS or the VISNs. Such information, if it were  shared, may have been used to inform ONS of any systematic problems  that necessitate changes to help ensure the continued viability of its  methodology, as well as identify any best practices that have been  implemented by VAMCs across the country. ONS officials told us that  they did not require the VAMCs to submit any such documentation to  ONS, because they made a conscious decision to not “micro-manage”  the local process of nurse staffing.
Furthermore, VHA has not sufficiently utilized the VISN-level  management structure in the implementation or ongoing administration of  the methodology. While the methodology directive described a role for the  VISNs, that role was limited to ensuring that resources are available to  VAMCs as they try to staff their units; the directive did not mention a role  in the implementation or ongoing administration of the actual  methodology. As a result, VISNs have not been consistently aware of  problems experienced by VAMCs in their region, and have not provided  support or education. In our interviews with VISN officials representing  each of the seven VAMCs in our review, we found that three of the VISNs  were not substantively involved in the implementation and ongoing  administration of the methodology. According to ONS, in many VISNs,  discussions of staffing methodology implementation were minimal, and  rather than VISN leadership, the nurse executives, in addition to their  responsibilities within their individual VAMCs, had the responsibility of  disseminating staffing methodology-related information to the VAMCs  within the VISN.
Staff from three VISNs that were more substantially involved in the  implementation of the methodology provided oversight for the nurse  staffing methodology and acted as liaisons for VAMC nurse executives for  network-level issues. One VISN official we interviewed was developing  oversight mechanisms for VAMCs in the region, including a requirement  for nurse executives to submit a quarterly staffing report. According to the  official, having such a reporting requirement at the VISN level would give  the right amount of emphasis to the process and provide support to nurse  executives implementing the methodology in the VAMCs. The quarterly  report could also help inform VISN officials about issues with the  methodology. This official was developing these mechanisms  independently of ONS, but they could be considered potential best  practices to be shared across all VISNs.
ONS officials told us they thought ideas or problems across VAMCs  related to the methodology would be shared through the VAMC nurse  executives. They also hoped that VISN leadership would be interested in  the methodology and, as a result, schedule VISN-level briefings to aid in  its implementation. VHA, however, did not specify either of these roles in  the directive or take steps to ensure that they were occurring. Moving  forward, ONS officials said they are considering developing a VISN-level  staff position that would specifically focus on educating VAMCs within the  region about the methodology, and assisting them with implementing it.  Without clearly defined roles and responsibilities within VA’s  organizational structure, VHA’s ability to improve its oversight of the  implementation and administration of the staffing methodology and  provide VAMCs with additional resources to assist with problems is  compromised.

	Conclusions

As the number of veterans requiring care in VAMCs and the complexity of  services needed by many of these veterans increase, the need for an  adequate and qualified nurse workforce is increasingly critical. Although  VHA’s nurse staffing methodology was intended to provide a nationally  standardized methodology for determining and ensuring adequate and  qualified nurse staffing at VAMCs, its ability to do so across all 151  VAMCs is not likely to be realized unless existing weaknesses are  addressed. Although some improvements in nurse staffing were reported  with the implementation of the staffing methodology, the seven VAMCs in  our review experienced problems developing and executing the related  staffing plans, including problems pertaining to data resources, training,  and communication. Many of these problems persist as the seven  VAMCs continue to administer the methodology.
We also found that VHA’s oversight of the staffing methodology is limited  and in many cases lacks sufficient internal controls, which could diminish  VHA’s ability to ensure an adequate and qualified nurse workforce. In  particular, VHA has not adequately assessed the needs or preparedness  of VAMCs to effectively implement the methodology, does not have a  formal mechanism to ensure VAMCs’ ongoing compliance with the  methodology, has not clearly defined a role in oversight for VISNs, and  does not regularly communicate with VAMCs or VISNs to cull and share  best practices system-wide. Furthermore, delays in VHA’s evaluations of  early phases of the staffing methodology have made them too late to be  useful in designing future phases or helping VAMCs with implementation.  Because the implementation and administration of the nurse staffing  methodology is ongoing, it is critical that VHA improve its oversight to  help ensure an adequate and qualified nurse workforce across all  VAMCs.

	Recommendations for Executive Action

To help ensure adequate and qualified nurse staffing at VAMCs, we  recommend that the Secretary of Veterans Affairs direct the Interim Under  Secretary for Health to enhance VHA’s internal controls through the  following five actions:  1.  Provide support to all VAMCs to meet the objectives of the VHA  a.  training that more clearly aligns with the needs of VAMC staff and  b.  a systematic process for collecting and disseminating staffing  2.  Conduct an environmental assessment of all VAMCs, including an  assessment of their data analysis needs, to determine their  preparedness to implement the remaining phases of the methodology,  and use that information to help guide and provide the necessary  support for the implementation of the remaining phases and for the  ongoing administration of the methodology;  3.  Develop and implement a documented process to assess VAMCs’  ongoing compliance with the staffing methodology, including  assessing VAMCs’ execution of staffing plans and more clearly  defining the role and responsibilities of all organizational components,  including VISNs, in the oversight and administration of the  methodology;  4.  Complete evaluations of Phase I and Phase II and make any  necessary changes to policies and procedures before national  implementation of Phase II in all VAMCs; and  5.  Improve the timeliness and regularity of communication with VAMCs,  including unit-level staff, regarding the status of the various phases of  the methodology.

	Agency Comments

We provided a draft of this report to VA for its review and comment. VA  provided written comments, which are reprinted in appendix I. In its  written comments, VA generally agreed with our conclusions and  concurred with all five of the report’s recommendations. To address the  recommendations, VA indicated that VHA will take a number of actions,  such as developing a written document specifying its process for  assessing ongoing compliance with the staffing methodology and  improving the timeliness and regularity of communication with VAMCs  through face-to-face regional training sessions. VA indicated that target  completion dates for implementing these recommendations range from  September 2015 through September 2016. Regarding the  recommendation that VA complete evaluations of Phase I and Phase II  before national implementation of Phase II in all VAMCs, VA indicated  that, by September 2016, it would complete its evaluations and determine  what opportunities exist to modify policies and procedures, but did not  explicitly state that the evaluations would be completed before national  implementation. We continue to emphasize the importance of completing  the evaluations before national implementation of Phase II in all VAMCs.
As agreed with your office, unless you publicly announce the contents of  this report earlier, we plan no further distribution until 30 days from the  report date. At that time, we will send copies of this report to appropriate  congressional committees, the Secretary of Veterans Affairs, and other  interested parties. In addition, the report will be available at no charge on  the GAO Web site at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-7114 or draperd@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix II.

Appendix I: Comments from the Department of Veterans Affairs

Appendix II: GAO Contact and Staff Acknowledgments

	GAO Contact

	Staff Acknowledgments

In addition to the contact named above, Janina Austin, Assistant Director;  Jennie Apter; Kathryn Black; Jacquelyn Hamilton; Kelli Jones;   Vikki L. Porter; and Karin Wallestad made key contributions to this report.

Output:
Why GAO Did This Study

GAO and others have raised prior concerns about the adequacy and qualifications of VHA's nurse staffing. In part to address these concerns, VHA issued a directive in 2010 requiring all VAMCs to implement a standardized methodology for determining an adequate and qualified nurse workforce, which includes developing and executing nurse staffing plans. It also requires VAMCs to use the methodology on an ongoing basis to evaluate staffing plans.
GAO was asked to provide information on nurse staffing at VAMCs. This report reviews the extent to which (1) VAMCs have implemented VHA's nurse staffing methodology, and (2) VHA oversees VAMCs' implementation and ongoing administration of the methodology. GAO reviewed documents and interviewed officials from VHA, seven VAMCs selected to ensure variation in factors such as geographic location, and regional offices for these VAMCs. GAO used federal internal control standards to evaluate VHA's oversight. GAO also interviewed representatives of veterans service organizations, nursing organizations, and unions.

What GAO Found

The seven Department of Veterans Affairs medical centers (VAMC) in GAO's review implemented the Veterans Health Administration's (VHA) nurse staffing methodology, experienced problems developing and executing the related nurse staffing plans, and some reported improvements in nurse staffing. Specifically, GAO found that each of the seven VAMCs had developed a facility-wide staffing plan—which outlines initiatives needed to ensure appropriate unit-level nurse staffing and skill mix—and taken steps to execute it. However, VAMCs experienced problems—such as lack of data resources and difficulties with training—in both the development and execution of their staffing plans. Some VAMC staff reported improvements in the adequacy and qualifications of their units' nursing staff when nurse staffing plan initiatives were executed. For example, at two VAMCs where the number of nurses was increased or where support services for nurses were put in place, such as a designated group of staff to assist in transporting patients to and from appointments off the unit, unit staff said the adequacy of the nursing staff had improved. However, some VAMC unit staff reported that unit nurse staffing continued to be inadequate and that nurse unit assignments and job duties were not always appropriate for their qualifications.
VHA's oversight is limited for ensuring its nurse staffing methodology is implemented and administered appropriately. GAO found the following internal controls were limited in VHA's oversight process:
Environmental assessment. VHA did not comprehensively assess each VAMC to ensure preparedness for implementing the methodology, including having the necessary technical support and resources, prior to the issuance of the directive requiring each VAMC to implement the methodology.
Monitoring compliance. VHA does not have a plan for monitoring VAMCs to ensure compliance with the implementation and ongoing administration of the methodology.
Evaluation. VHA has conducted limited evaluations of the methodology, and at least one of these evaluations has been significantly delayed.
Timeliness of communication. VHA's protracted timeline for communicating methodology-related information may have hindered the ability of VAMCs to appropriately develop their staffing plans and to execute the initiatives contained in those plans.
Organizational accountability. VHA did not define areas of responsibility or establish the appropriate line of reporting within VA's management structure for oversight of the implementation and ongoing administration of the methodology.
Without these internal controls in place, VHA cannot ensure its methodology meets department goals, such as establishing a standardized methodology for determining an adequate and qualified nurse workforce at VAMCs, and ultimately, having nurse staffing that is adequate to meet veterans' growing and increasingly complex health care needs.

What GAO Recommends

GAO recommends VA: (1) assess VAMCs' ability to implement the methodology, (2) monitor VAMCs' ongoing compliance with the methodology, (3) complete timely evaluations, (4) improve the timeliness of communication with VAMCs, and (5) define areas of responsibility and reporting within VA's management structure. VA concurred with the recommendations.