Input:
	Background

		Coast Guard Organizational Structure for TAP

Coast Guard staffing for the TAP program reflects the organizational  structure of its Health, Safety, and Work-Life Directorate, which oversees  TAP policy. The Coast Guard’s TAP managers are assigned to 13  installations where Health, Safety, and Work-Life offices are located.  One or two TAP managers are assigned to each of the Coast Guard’s  nine districts, which often span multiple states and territories, and these  TAP managers oversee operations both for the installation where they  work and for units stationed throughout the region (see fig. 1). For  example, the TAP manager assigned to Coast Guard Base Cleveland  oversees TAP implementation both for that installation and for Coast  Guard units serving in Coast Guard District 9—a region that  encompasses portions of eight states and the Great Lakes area. The  program manager in Coast Guard Headquarters manages Coast Guard’s  Transition Assistance Program. The Coast Guard protects and defends  over 100,000 miles of U.S. coastline and inland waterways, and  consequently, TAP-eligible Coast Guard servicemembers sometimes  work in small, widely dispersed units assigned to remote locations,  including on Coast Guard vessels. One aspect of the Coast Guard’s  mission—a first responder for maritime search and rescue in United  States waters—can require Coast Guard servicemembers to respond to  emergency situations at a moment’s notice.

		TAP Process and Timing

The Coast Guard, which is overseen by DHS, not DOD, generally  oversees TAP implementation for its servicemembers. Federal law  requires DOD and DHS to require eligible servicemembers under their  respective command to participate in TAP, with some exceptions. In  response to this statutory requirement, DOD has promulgated regulations  and developed issuances which require that servicemembers complete  the component parts of the TAP program, and that commanding officers  ensure that servicemembers under their command complete these parts,  with some exceptions. In contrast, according to Coast Guard officials,  Coast Guard has not promulgated any regulations to implement TAP.  Further, Coast Guard issued its most recent Commandant Instruction in  2003, approximately 8 years prior to TAP redesign in 2011. However,  Coast Guard issued policy guidance in 2014 that made some limited  updates to the Commandant Instruction. Coast Guard officials also said  the Coast Guard plans to issue a new TAP Commandant Instruction in  May 2018.
Under the redesigned TAP, Coast Guard servicemembers—like their  DOD counterparts—begin TAP by attending pre-separation or transition  counseling where they are briefed on TAP requirements and available  transition resources. Pre-separation or transition counseling can be  delivered by TAP managers, uniformed career counselors, or online (see  fig. 2). Coast Guard servicemembers are able to participate in TAP  either through the Coast Guard or at a DOD installation, if space is  available.
During or at the end of pre-separation or transition counseling,  participants register for and attend TAP courses. The core curriculum  includes three required courses—the Department of Labor (DOL)  Employment Workshop, unless exempt, and Department of Veterans  Affairs (VA) Benefits Briefings I and II—and other courses that focus on  aspects such as translating military skills and experiences into  credentialing for civilian jobs and preparing a financial plan. Participants  may also elect to attend additional 2-day classes either at a Coast Guard  or DOD installation or online through the Joint Knowledge Online  platform, according to agency officials. These additional 2-day classes  include Accessing Higher Education, Career Technical Training, and  Entrepreneurship. Federal law requires the Coast Guard to permit  servicemembers who elect to take these additional 2-day classes to  receive them.
Federal law establishes a time frame within which servicemembers with  anticipated separation or retirement dates should begin the program.  According to federal law, retirees with anticipated separation dates are  expected to begin TAP as soon as possible during the 24-month period  preceding that date, but not later than 90 days before separation.  Similarly, servicemembers with anticipated separation dates who are not  retiring are expected to begin as soon as possible during the 12-month  period preceding that date, but not later than 90 days before separation.  Servicemembers who learn that they will separate or retire from the  military fewer than 90 days before their anticipated separation or  retirement date are expected to begin TAP as soon as possible within  their remaining period of service.

		Interagency Collaboration

As we previously reported, officials from multiple federal agencies  collaborate to deliver and assess TAP. The TAP interagency  governance structure includes senior officials from DOD, VA, DOL, DHS,  the Department of Education, the U.S. Office of Personnel Management,  and the Small Business Administration (SBA), who participate in TAP  Senior Steering Group meetings at least every month and TAP Executive  Council meetings each quarter. Further, officials tasked to particular  interagency working groups focus on specific elements of TAP (e.g.,  curriculum or performance measures), meet more frequently (typically at  least once a month), and generally communicate weekly, according to  agency officials. The TAP program manager for the Coast Guard told us  that he participates in several of the working groups.
One such working group is the performance management working group  that oversees the interagency TAP evaluation plan, which includes  monitoring performance measures related to TAP requirements,  indicators of post-program outcomes, and formal evaluations sponsored  by interagency partners. While DOD tracks TAP-specific performance  measures, other interagency partners track indicators of how well  veterans fare after leaving military service. For example, DOD tracks  performance measures prior to servicemembers’ separation, such as TAP  participation and credential attainment rates, while other agencies track  post-separation indicators, such as unemployment rates among veterans  ages 18 to 24. The performance management working group also  reviews the formal evaluation efforts led by individual agencies and  provides feedback to help shape their efforts in accordance with the TAP  Evaluation Plan.

	Coast Guard Lacks Reliable Data and Cites Several Factors that Affect Participation Coast Guard Lacks Reliable Data on Servicemembers’ Participation in TAP

The Coast Guard does not have complete or reliable data on participation  levels in TAP. According to Coast Guard officials, a major reason why  the data are not reliable is that the Coast Guard lacks an up-to-date  Commandant Instruction that specifies when to record TAP participation  data. Consequently, the data are updated on an ad-hoc basis,  according to agency officials, and may not be timely or complete. For  example, one TAP manager said she updates the list of TAP participants  for her installation only once every few months because of her other  duties. According to federal internal control standards, management  should use quality information—including current and timely information— to achieve the entity’s objectives and to communicate quality information  to external parties. Given the lack of timely and complete data, we  determined the Coast Guard’s TAP data were not sufficiently reliable for an analysis of participation in TAP classes. Because it lacks policies and  procedures governing reliable data collection, including when data should  be entered and by whom, the Coast Guard cannot determine to what  extent its servicemembers attend TAP, although federal law mandates  that DHS ensure all TAP-eligible servicemembers participate in the  program.
In addition, the data collection system currently used to track TAP  participation is not sufficient to ensure reliable data. For example,  according to Coast Guard staff, TAP staff enter TAP participation data  into a shared spreadsheet that all TAP managers can edit. Specifically,  staff record the names of servicemembers they identify as TAP-eligible  and whether these individuals completed required portions of TAP. Coast  Guard officials said they are in the process of adopting a new data  system, in October 2018, to more reliably track TAP participation and that  they expect to fully adopt this system–DOD’s TAP-IT Enterprise  System—after a new Commandant Instruction is finalized, in May 2018.  In November 2016, DOD launched the new system to collect TAP-related  data for servicemembers in the Army, Navy, Air Force, and Marine  Corps. In addition to standardizing data collection and improving data  completeness and accuracy, the TAP-IT Enterprise System is expected to  track information related to the time frames of servicemembers’  participation. According to a senior DOD official, the military services will  not be able to use the system to generate unit-level or installation-level  reports until October 2018.

		Serving at a Remote Installation and Rapid Separations Hindered TAP Participation, As Did Limited Staff Capacity and Competing Priorities

According to our survey, the most common factor affecting TAP  participation, cited at 11 of the 12 Coast Guard installations we surveyed,  pertained to servicemembers assigned to geographically remote  locations. The next three most commonly cited factors–each cited by 7 of  the 12 installations surveyed—relate to the timing of TAP participation:  rapid separation from the military, not being sufficiently aware of the need  to attend TAP, and starting the transition process too late to attend. (See  fig. 3.) Headquarters-based TAP officials identified additional factors that  may affect servicemember participation, such as separating from the  Coast Guard Reserves or retiring with no plans to work after leaving the  military. However, the Coast Guard lacks participation data to verify  whether participation rates for these groups are in fact lower than for  other Coast Guard servicemembers.
Coast Guard installations we surveyed did not indicate that unit  commanders or direct supervisors affected participation in TAP’s required  courses or additional 2-day classes. However, Coast Guard  servicemembers and TAP officials we spoke with said unit commanders  or direct supervisors sometimes prevented participation. All three TAP  managers we spoke with (of 12 nationwide) told us that while  commanders generally allowed servicemembers to register for TAP  courses, they occasionally required them to return to their duties before  completing the courses. We observed this during a TAP class at a Coast  Guard installation we visited when a servicemember’s commander  ordered her to return to the unit during TAP training and she missed a  briefing she wanted to attend. Two of three TAP managers we  interviewed also said commanders sometimes required servicemembers  under their command to wait to take TAP classes until close to their  separation date because of mission priorities.
Two of three TAP managers interviewed said that commanders in the  Coast Guard face unique challenges in ensuring TAP participation. They  said commanders in all branches of the military must balance competing  demands, including their primary mission and the training needs of the  personnel they oversee. They said it can be particularly difficult for Coast  Guard commanders to juggle these priorities because Coast Guard  servicemembers are sometimes assigned to very small units or called to  return to duty for emergency situations during scheduled TAP classes.  One TAP manager said that a commander in a remote location had  collaborated with her to provide a classroom-based TAP class for  transitioning Coast Guard servicemembers within the commander’s unit,  but rescue efforts occurred during the class which resulted in most of  those servicemembers returning to their vessel to respond to the  emergency. In addition, all three TAP managers we spoke with said there  are limited resources for holding TAP in a classroom setting.  Consequently, classroom-based TAP may not be offered frequently in  remote locations, making rescheduling difficult. One TAP manager said  that her installation typically offers three or four TAP classes a year and  because classes are so infrequent, servicemembers are encouraged to  start TAP as soon as possible prior to separation.
Coast Guard staff we interviewed said that juggling competing priorities  affected the Coast Guard’s ability to implement TAP. Both the frontline  and headquarters staff who oversee TAP implementation said they  oversee at least three other programs in addition to TAP at their  installation and throughout their regions, including the Coast Guard’s  relocation and spousal employment programs.

		Coast Guard Relies on Online Delivery of TAP for Several Categories of Servicemembers

The Coast Guard relies on online delivery of TAP information and classes  for servicemembers who are rapidly separating and assigned to remote  and geographically dispersed units, according to our survey results and  several Coast Guard staff we interviewed. For example, all 12  installations we surveyed cited servicemembers facing rapid separations  as a reason for accessing TAP training online, and 11 cited  servicemembers being remotely stationed as a reason. Coast Guard staff  also said online TAP was used for servicemembers interested in  attending additional 2-day classes.
The three TAP managers we interviewed also identified several reasons  why installations had to rely on online TAP classes. For example, one  manager corroborated our survey results, saying that many Coast Guard  servicemembers worked in small units assigned to remote and  geographically dispersed locations, making it difficult to convene a  sufficient number of transitioning Coast Guard servicemembers to meet  minimum class size requirements. In addition, all three managers said  they used the online version of TAP for remotely stationed Coast Guard  servicemembers because the Coast Guard lacked the resources for them  to attend classes in person. Although they preferred that  servicemembers participate in live, classroom-based TAP classes, all of  the managers acknowledged that the online version of TAP played an  integral role in ensuring that more servicemembers could participate in  the program. However, two of them noted that while classroom delivery of  TAP classes provided an interactive learning environment that allowed  participants to ask questions and learn from their peers, online  participants generally clicked quickly through the slides and had difficulty  understanding the information being presented. Two managers told us  that they regularly used the online version to deliver parts of the TAP  curriculum. For example, one TAP manager said she required  participants to complete the crosswalk of military and civilian occupations  class online before attending required classes in person. Two managers  noted that additional 2-day classes were available online, and one noted  that some servicemembers attended these classes in a classroom setting  either on a Coast Guard base or a DOD installation. Finally, all three TAP  managers said that many participants in online TAP classes would benefit  from participating in a real-time virtual version of TAP led by live  facilitators. Two managers told us that having a remote facilitator  delivering TAP in real time would give participants more opportunity to  ask questions and better understand and absorb class content.

		Feedback About TAP Was Generally Positive

Despite these challenges, TAP managers and separating Coast Guard  servicemembers we interviewed provided generally positive feedback  about the TAP program. All of the 25 Coast Guard servicemembers we   spoke with said that the information they received during the courses was  useful and they liked the instructors. One Coast Guard servicemember  praised the classroom courses for being interactive, and several Coast  Guard servicemembers said they wanted the opportunity to retake TAP  before or shortly after they separated from the Coast Guard. However,  many said the volume of information presented in a short period of time  could be overwhelming and was like “trying to drink from a firehose.”

	Coast Guard Cannot Effectively Measure Performance or Monitor Implementation to Ensure Key TAP Requirements Are Met

		Coast Guard Has Not Set a Formal Performance Goal for TAP Participation and Cannot Effectively Measure Program Performance Because it Lacks Reliable Data

The Coast Guard has not set a formal performance goal for TAP  participation, according to a Coast Guard official, and as previously  discussed, does not have complete, reliable data. Without reliable  information, the Coast Guard cannot effectively monitor TAP  implementation or measure program performance. DHS is mandated to  ensure that all TAP- eligible servicemembers of the Coast Guard  participate in TAP before leaving military service. However, without  effective monitoring of program participation, the Coast Guard cannot  know to what extent its servicemembers receive the required training they  need to prepare for civilian life. According to federal internal control  standards, management should consider external requirements—such as  the laws with which the entity is required to comply—to clearly define  objectives in specific and measurable terms. In addition, establishing  goals can help agencies define expected performance and articulate  results. A Coast Guard official said the Coast Guard’s long-term goal is  for full compliance with TAP requirements, but in the interim, the Coast  Guard uses DOD’s 85 percent VOW compliance goal as an informal  benchmark against which to gauge the Coast Guard’s TAP  performance. However, the Coast Guard has not communicated a  specific, measurable goal to TAP staff implementing the program, or to  Coast Guard commanders who oversee separating and retiring Coast  Guard servicemembers, according to a Coast Guard official. Establishing  and communicating a formal goal could help the Coast Guard define  expected performance. The official also told us that, like DOD, the Coast  Guard tracks the elements of TAP mandated under the VOW Act— transition or pre-separation counseling, VA Benefits I and II, and the DOL  Employment Workshop.

		Coast Guard Does Not Monitor Compliance with Additional TAP Requirements

The Coast Guard does not monitor the (1) timeliness of participation in  TAP, and (2) access to additional 2-day classes. A Coast Guard official  said the Coast Guard does not currently monitor TAP beyond tracking  whether separating servicemembers participate in the required courses,  and currently lacks the capacity to undertake additional monitoring efforts.  However, he said additional monitoring would be possible once the Coast  Guard completed the move to the DOD TAP-IT Enterprise data system.

			Timeliness of TAP Participation

According to a Coast Guard official, the Coast Guard does not currently  monitor the timeliness of TAP participation although federal law  prescribes time frames for servicemembers to begin TAP participation.  Generally, separating servicemembers who are not retiring are to begin  TAP participation no later than 90 days before their separation date.  Without a systematic method for monitoring timeliness, the Coast Guard  cannot know whether its servicemembers begin the program on time or  account for the timeliness of TAP participation. As a result, the Coast  Guard cannot know whether its servicemembers are starting TAP early  enough to complete the training they need to adequately prepare for their  transition to civilian life.

			Access to Additional 2-Day Classes

The Coast Guard does not track which of its servicemembers participate  in the additional 2-day classes, according to a Coast Guard official we  interviewed, even though federal law requires that DHS ensure those who  elect to participate are able to receive the training. By not tracking which Coast Guard servicemembers participate in 2-day classes or requiring  transition staff to document when servicemembers ask to attend, the  Coast Guard cannot determine the extent to which servicemembers who  wished to attend these courses were able to do so, as required by law.

		Roles and Responsibilities Are Not Clearly Defined

Coast Guard commanders and TAP managers do not have clearly  defined roles and responsibilities in implementing TAP because of the  lack of an up-to-date Commandant Instruction, according to TAP staff we  interviewed. As previously discussed, the Coast Guard’s last  Commandant Instruction on TAP was issued in 2003, approximately 8  years prior to TAP’s redesign. According to federal internal control  standards, to achieve the entity’s objectives, management should assign  responsibility and delegate authority to key roles throughout the entity.  Without an up-to-date Commandant Instruction, TAP managers and  commanders may be unclear on who is ultimately responsible for  ensuring servicemembers attend TAP. Moreover, two TAP managers also  told us that an up-to-date Commandant Instruction might lead some  commanders to place higher priority on ensuring TAP participation. Coast  Guard officials said the Coast Guard was in the process of revising the  TAP Commandant Instruction and anticipated issuing the new instruction  in May 2018.

		Coast Guard Does Not Share Participation or Performance Data with Commanders or TAP Interagency Partners, Limiting Monitoring and Evaluation

The Coast Guard lacks the ability to share data with commanders, limiting  its ability to monitor TAP participation and ensure servicemembers attend  the program. According to a Coast Guard official, the Coast Guard’s  current data collection system also cannot generate installation or unit- level participation rates to share with commanders who oversee  transitioning and retiring servicemembers. Federal internal control  standards state that management should share quality information  throughout an organization to enable personnel to perform key roles, and  we have previously reported that by regularly sharing useful performance  information with leaders at multiple levels of an organization, agencies  can help leaders make informed decisions. Without this information,  individual unit commanders or the commanders’ supervisors cannot  determine whether Coast Guard servicemembers under their command  completed TAP or identify whether there is a need for corrective actions  to ensure they do so.
As we mention earlier in this report, the Coast Guard plans to adopt  DOD’s TAP-IT Enterprise System, which according to officials, could help  the Coast Guard ensure eligible servicemembers participate in the  program. According to a Coast Guard official, once the system is fully  implemented by the Coast Guard, commanders will be required to verify  and document whether Coast Guard servicemembers under their  command completed TAP, potentially making commanders more vested  in the process. We previously reported that a senior DOD official said that  the TAP-IT Enterprise System may be able to generate unit- and  installation-level reports for the four DOD-led military services by October  2018, and a Coast Guard official said he would work with DOD to identify  whether this capability could also extend to the Coast Guard. Once data  reliability improves, sharing installation and unit-level TAP performance  information with Coast Guard commanders could support monitoring  efforts.
The performance measures tracked by the TAP interagency working  group do not reflect TAP implementation broadly across all five military  services, according to a Coast Guard official we interviewed. The Coast  Guard does not currently share TAP data it collects with DOD or other  members of the interagency performance working group. While the  benefits of interagency data sharing cannot be realized without the Coast  Guard first improving the quality and completeness of its TAP data, we  have identified leading practices for interagency collaboration, including  that members of interagency working groups identify and share relevant  agency performance data. Moreover, federal internal control standards  call for management to communicate quality information to external  parties. Because the Coast Guard does not share TAP data, the  performance measures tracked by the interagency group do not reflect  Coast Guard servicemembers’ experiences and thus do not provide a  complete picture of TAP implementation across the five military services.  More specifically for the Coast Guard, without such data sharing, future  TAP evaluations may not be able to assess the effectiveness of TAP  delivery, hindering the Coast Guard’s ability to make program  adjustments to better prepare its servicemembers to successfully  transition to life after military service. Coast Guard officials said migrating  to DOD’s TAP-IT Enterprise System will facilitate information sharing with  interagency partners and that improving data completeness and reliability  is a top priority for 2018.

	Conclusions

Given the sacrifices servicemembers have made to serve their country, it  is imperative they are afforded every chance to adequately prepare for  civilian life before leaving military service. In order to make a successful  transition, servicemembers need to be well-positioned to get a job or  make an informed decision about whether to pursue additional education  or start a small business. As such, the Transition Assistance Program  (TAP) serves a critically important function—to give servicemembers the  tools and information they need to successfully transition to life outside  the military. Federal law requires that the Coast Guard ensure all eligible  servicemembers participate in the program, but thousands of Coast  Guard servicemembers may have transitioned without the support  provided by TAP. Reliably tracking participation has proven to be a  challenge for the Coast Guard, in part because it lacks a current  Commandant Instruction that defines the roles and responsibilities of staff  responsible for implementing TAP and ensuring complete and reliable  data are collected. In preparing to issue an updated Commandant  Instruction, the Coast Guard has taken a positive step toward addressing  the limitations of its current TAP data, and will be better positioned to  ensure compliance with VOW Act requirements using reliable data.
In addition to collecting reliable data, the Coast Guard could further  demonstrate its commitment to meeting TAP requirements by  establishing formal performance goals that measure the extent to which  Coast Guard servicemembers participate in TAP. By establishing interim  performance goals, the agency would be able to show its progress  towards achieving full compliance. Moreover, communicating  performance goals to unit and installation commanders could enhance  accountability and might spur progress toward meeting federal program  requirements.
By expanding its monitoring efforts beyond tracking participation in TAP’s  required classes, the Coast Guard could enhance its ability to ensure  other TAP requirements are met and that its servicemembers are able to  access additional transition resources. Monitoring the timeliness of  participation would help ensure Coast Guard servicemembers have  adequate time to complete TAP before leaving the military. Further, by  monitoring requests to participate in additional 2-day classes and 2-day  class attendance, the Coast Guard would be in a better position to identify  whether servicemembers who wish to attend the classes are able to do  so, to determine whether more classes are needed, and to communicate  this information to the interagency partners responsible for delivering  these classes.
Commanders can also play a key role in bolstering TAP participation.  Having an up-to-date written Commandant Instruction that explicitly  describes commanders’ roles and responsibilities could enhance  commanders’ ability to ensure TAP’s proper implementation and  compliance with VOW Act requirements. Moreover, once data quality  improves, providing commanders a mechanism to readily determine  whether servicemembers under their command have completed TAP  could help them monitor the program to ensure that all TAP-eligible  servicemembers receive the resources they need to successfully  transition to civilian life.
Finally, once more reliable data on Coast Guard servicemember  participation are available, sharing this information with interagency  partners could improve TAP implementation on a broader scale. Sharing  reliable data, such as participation figures for the Coast Guard, would  give TAP interagency partners a more complete picture of implementation  across all five military services. Sharing such information would also  enhance the interagency group’s ability to evaluate how well TAP serves  the entire population of servicemembers. Improving the reliability of the  Coast Guard’s TAP data will be essential for the benefits of data sharing  to be realized.

	Recommendations for Executive Action

To ensure that all eligible Coast Guard servicemembers are provided the  opportunity to complete the Transition Assistance Program (TAP), we  recommend the Commandant of the Coast Guard take the following  seven actions:  Issue an updated Commandant Instruction that establishes policies  and procedures to improve the reliability and completeness of TAP  data by including when and by whom data should be recorded and  updated. (Recommendation 1)
Establish a formal performance goal with a measurable target for  participation rates in VOW Act-mandated portions of TAP.  (Recommendation 2)
Monitor the extent to which Coast Guard servicemembers participate  in TAP within prescribed time frames. (Recommendation 3)
Monitor the extent to which Coast Guard servicemembers who elect  to participate in additional 2-day classes are afforded the opportunity  to attend. (Recommendation 4)
Issue an updated Commandant Instruction that defines the roles and  responsibilities of the personnel who administer the program and  ensure servicemembers’ participation. (Recommendation 5)
Once reliable data are available by installation or unit, enable unit  commanders and the higher-level commanders to whom they report  to access TAP performance information specifically for the units they  oversee so that they can monitor compliance with all TAP  requirements. (Recommendation 6)
Once reliable data are available, share TAP information with DOD and  other interagency partners, such as data on participation in required  TAP courses and additional 2-day classes. (Recommendation 7)

	Agency Comments and our Evaluation

We provided a draft of this report to the Departments of Homeland  Security, Defense, Education, Labor, and Veterans Affairs, the Office of  Personnel Management, and the Small Business Administration for their  review and comment. The formal written response of the Department of  Homeland Security (DHS) is reproduced in appendix II. In addition, DHS  provided technical comments from Coast Guard officials that we  incorporated into the report as appropriate. The other agencies did not  provide any comments.
In its written comments, DHS agreed with all seven of our  recommendations.
If you or your staff have any questions about this report, please contact  me at (202) 512-7215 or brownbarnesc@gao.gov. Contact points for our  Offices of Congressional Relations and Public Affairs may be found on  the last page of this report. GAO staff who made key contributions to this  report are listed in appendix III.

Appendix I: Objective, Scope, and Methodology

	Overview

This report examines (1) what is known about the reliability of Transition  Assistance Program (TAP) data on participation levels and the factors  that affect Coast Guard servicemembers’ participation, and (2) the extent  to which the Coast Guard measures TAP performance and monitors key  areas of TAP implementation. To address these questions, we surveyed  Coast Guard installations with full-time TAP operations; reviewed Coast  Guard data on TAP participation for fiscal years 2012 to 2017; visited one  Coast Guard installation and interviewed TAP managers from two  additional Coast Guard installations selected for diversity in location,  among other reasons; and interviewed Coast Guard officials responsible  for overseeing TAP implementation for the Coast Guard. We also  reviewed relevant federal laws, regulations, policies, documents, and  publications. Information in this report is current as of the date GAO  received formal agency comments from DHS.

	Survey

Our survey of Coast Guard installations with full-time TAP operations  asked about how TAP was being implemented. The survey included  questions about the accessibility of TAP components, challenges Coast  Guard servicemembers faced in attending the components, and the level  of commander support for participation. Our survey targeted front-line  TAP managers, who could draw on the expertise of TAP course  facilitators, transition counselors, career counselors, and other key TAP  staff as necessary.
After drafting the survey questions, we pretested them with a TAP  manager to ensure (1) the questions were clear and unambiguous, (2)  terminology was used correctly, (3) the survey did not place an undue  burden on agency officials, (4) the information could feasibly be obtained,  and (5) the survey was comprehensive and unbiased. We revised the  content and format of the survey based on the feedback we received.
We initially sent the survey to TAP managers at all 13 Coast Guard  installations at which TAP staff were located. We removed one installation  when we later found that the TAP manager position was vacant and  revised the total to 12 Coast Guard installations. The survey was  accessible online from October 31, 2016, through January 18, 2017,  through a secure server that recipients were able to access using unique  usernames and passwords. We sent an email announcement to TAP staff  at all 13 Coast Guard installations at which TAP staff are located on  October 24, 2016. We sent a second email on October 31, 2016 to notify  participants the survey was available online, and provided their unique  passwords and usernames. We sent two follow-up e-mails (November 14,  2016 and November 28, 2016) to those who had not responded. Finally,  we contacted all remaining nonrespondents by telephone starting  December 5, 2016. The survey was available online until we reached a  100 percent response rate.

	Interviews with Coast Guard Installation TAP Staff and Servicemembers

To increase our understanding into how TAP was being implemented at  installations and supplement our survey findings, we visited one Coast  Guard installation and interviewed TAP managers from two additional  installations. We selected the installations based on several factors,  including the size of the installation, proximity to Department of Defense  (DOD) installations, and diverse locations in the United States. (See table  1.) At Coast Guard Base Elizabeth City in North Carolina, the installation  we visited, we interviewed the TAP manager, uniformed career  counselors, and senior installation leadership. During our interviews with  TAP managers at all three installations, we asked about the extent to  which Coast Guard servicemembers participate in TAP’s required and  additional 2-day classes, including whether the servicemembers attended  classes online or in a classroom setting, challenges to ensuring Coast  Guard servicemembers participate in TAP, and the extent to which they  monitor Coast Guard servicemembers’ participation in TAP. At Coast  Guard Base Elizabeth City, we also interviewed 25 Coast Guard  servicemembers (both officers and enlisted personnel) to get their  perspective on how well TAP worked and any challenges they had  participating. To help guide the interviews with the Coast Guard  servicemembers, we asked them to complete a short questionnaire that  asked about their experiences with the TAP program.

	Interviews with Agency Personnel

We also interviewed TAP staff at Coast Guard headquarters to learn  about TAP policy, monitoring efforts, and performance measures for the  service overall. For example, we asked what policies and procedures  guide installations’ TAP implementation; what performance measures the  Coast Guard uses to monitor TAP; how performance results are reported  and shared with different levels of Coast Guard leadership; and to what  extent the Coast Guard uses results from TAP participant satisfaction  assessments. We also asked whether the Coast Guard plans to shift to  DOD’s new TAP-IT Enterprise System and how using the new system  could affect its monitoring efforts in the future. In evaluating the Coast  Guard’s performance measures, we focused on measures related to  servicemembers’ transition experiences before leaving the military. We  did not gather information on post-program evaluations and outcomes  because they were determined to be outside the scope of this review.

	Data Reliability Assessment

We reviewed DHS data on TAP participation for fiscal years 2012 to  2017. To assess the reliability of the Coast Guard’s TAP participation  data, we interviewed agency officials knowledgeable about the data. We  determined these data were not sufficiently reliable due to limitations with  the Coast Guard’s data collection system. Specifically, the system lacks  adequate controls to ensure TAP data are complete and accurate.
We conducted this performance audit from February 2016 to April 2018 in  accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix III: GAO Contact and Staff Acknowledgments

	GAO Contact

	Staff Acknowledgments

In addition to the contact named above, Meeta Engle (Assistant Director),  Amy MacDonald (Analyst-in-Charge), James Bennett, Holly Dye, David  Forgosh, Ying Long, Jonathan McMurray, Jean McSween, Andrew  Sherrill, Benjamin Sinoff, and Timothy Young, made significant  contributions to this report.
Also contributing to this report were Susan Aschoff, Jessie Battle,  Ramona Burton, Melinda Cordero, Elizabeth Curda, Dawn Hoff, Ben  Licht, Serena Lo, Sheila McCoy, Almeta Spencer, Christopher Schmitt,  James Whitcomb, and Jill Yost.

Appendix IV: Related Products

Transitioning Veterans: DOD Needs to Improve Performance Reporting  and Monitoring for the Transition Assistance Program, GAO-18-23.  Washington, D.C.: November, 8, 2017.
Transitioning Veterans: Improvements Needed in DOD’s Performance  Reporting and Monitoring of the Transition Assistance Program,  GAO-18-225T. Washington, D.C.: November 8, 2017.
Department of Defense: Transition Assistance Program (TAP) for Military  Personnel, GAO-16-302R. Washington, D.C.: December 17, 2015.
Veterans’ Employment: Need for Further Workshops Should Be  Considered before Making Decisions on Their Future, GAO-15-518.  Washington, D.C.: July 16, 2015.
Military and Veteran Support: DOD and VA Programs That Address the  Effects of Combat and Transition to Civilian Life, GAO-15-24.  Washington, D.C.: November 7, 2014.
Veterans Affairs: Better Understanding Needed to Enhance Services to  Veterans Readjusting to Civilian Life, GAO-14-676. Washington, D.C.:  September 10, 2014.
Transitioning Veterans: Improved Oversight Needed to Enhance  Implementation of Transition Assistance Program, GAO-14-144.  Washington, D.C.: March 5, 2014.
Military and Veterans’ Benefits: Enhanced Services Could Improve  Transition Assistance for Reserves and National Guard, GAO-05-544.  Washington, D.C.: May 20, 2005.
Military and Veterans’ Benefits: Observations on the Transition  Assistance Program, GAO-02-914T (July 18, 2002).

Output:
Why GAO Did This Study

Thousands of Coast Guard servicemembers have left the military and transitioned into civilian life, and some of these new veterans may face significant challenges, such as finding and maintaining employment. To help them prepare, federal law mandated that DHS provide separating Coast Guard servicemembers with counseling, employment assistance, and information on veterans' benefits through TAP. GAO was asked to examine TAP implementation.
This review analyzes (1) the reliability of TAP data on participation levels for Coast Guard servicemembers and the factors that affect participation, and (2) the Coast Guard's performance measures and monitoring efforts related to TAP. GAO interviewed Coast Guard headquarters staff; surveyed 12 Coast Guard installations that conduct TAP (100 percent response rate); collected and reviewed participation data for reliability; and interviewed TAP managers from three installations selected for size and location, and 25 Coast Guard servicemembers at one location. (For a companion report on TAP implementation for separating and retiring servicemembers in other military services, see GAO-18-23 .)

What GAO Found

The United States Coast Guard (Coast Guard), which is overseen by the Department of Homeland Security (DHS), lacks complete or reliable data on participation in the Transition Assistance Program (TAP), designed to assist servicemembers returning to civilian life. According to senior Coast Guard officials, a major reason why data are not reliable is the lack of an up-to-date Commandant Instruction that specifies when to record TAP participation data. Consequently, the data are updated on an ad-hoc basis and may not be timely or complete, according to officials. Federal internal control standards call for management to use quality information to achieve the entity's objectives. Until the Coast Guard issues an up-to-date Commandant Instruction that establishes policies and procedures to improve the reliability and completeness of TAP data, it will lack quality information to gauge the extent to which it is meeting TAP participation requirements in the VOW to Hire Heroes Act of 2011.
According to GAO's survey of Coast Guard installations, various factors affected participation, such as servicemembers serving at geographically remote locations or separating from the Coast Guard rapidly. TAP officials and Coast Guard servicemembers GAO interviewed said commanders and direct supervisors sometimes pulled servicemembers out of TAP class or postponed participation because of mission priorities. TAP managers also said they rely on delivering TAP online because many Coast Guard servicemembers are stationed remotely.
The Coast Guard cannot effectively measure performance to ensure key TAP requirements are met because it lacks reliable data and does not monitor compliance with several TAP requirements. Further, the Coast Guard has not established a formal performance goal against which it can measure progress, although federal internal control standards stipulate that management should consider external requirements—such as the laws with which the entity is required to comply—to clearly define objectives in specific and measurable terms. Establishing a goal could help the Coast Guard define expected performance. In addition, the Coast Guard does not monitor TAP requirements regarding the timeliness of servicemembers' TAP participation or their access to additional 2-day classes. Consequently, it cannot know whether servicemembers are starting TAP early enough to complete the program or those who elected to attend additional 2-day classes were able to do so before separation or retirement, as required by the Act. Finally, the Coast Guard lacks an up-to-date Commandant Instruction that establishes the roles and responsibilities of Coast Guard staff in implementing TAP. Federal internal control standards stipulate that management should assign responsibility and delegate authority to key roles throughout the entity. Issuing an up-to-date Commandant Instruction that defines roles and responsibilities would clarify who is ultimately responsible for ensuring Coast Guard servicemembers attend TAP, thereby facilitating accountability.

What GAO Recommends

GAO is making seven recommendations, including that the Coast Guard issue a new Commandant Instruction establishing data collection policies, set TAP performance goals, monitor timeliness and access, and define roles and responsibilities. DHS agreed with all of GAO's recommendations.


Input:
	Background

		U.S. Missions in Afghanistan

Since 2001, the United States has made a commitment to building  Afghanistan’s security and governance in order to prevent the country  from once again becoming a sanctuary for terrorists. To achieve its  security objectives, the United States currently has two missions in  Afghanistan: a counterterrorism mission that it leads and the NATO-led  Resolute Support train, advise, and assist mission, which it participates in  with other coalition nations. The objective of Resolute Support, according  to DOD reporting, is to establish self-sustaining Afghan security ministries  and forces that work together to maintain security in Afghanistan. The  United States is conducting these missions within a challenging security  environment that has deteriorated since the January 2015 transition to  Afghan-led security. The United Nations reported nearly 24,000 security  incidents in Afghanistan in 2017—the most ever recorded—and, despite a  slight decrease in the overall number of security incidents in early 2018,  the United Nations noted significant security challenges, including a spike  in high-casualty attacks in urban areas and coordinated attacks by the  insurgency on ANDSF checkpoints.
DOD provides both personnel and funding to support its efforts in  Afghanistan. DOD documents indicate that the United States contributes  more troops to Resolute Support than any other coalition nation. As of  May 2018, the United States was contributing 54 percent of Resolute  Support military personnel, according to DOD reporting. Of the  approximately 14,000 U.S. military personnel in Afghanistan as of June  2018, about 8,500 were assigned to Resolute Support to train, advise,  and assist the ANDSF, according to DOD reporting. For fiscal year 2018,  Congress appropriated about $4.67 billion for the Afghanistan Security  Forces Fund—the primary mechanism of U.S. financial support for  manning, training, and equipping the ANDSF. Other international donors  provided about $800 million, and the Afghan government committed to  providing about $500 million, according to DOD reporting.
Under Resolute Support and the International Security Assistance Force  mission that preceded it, CSTC-A is the DOD organization responsible for  (1) overseeing efforts to equip and train the ANA and ANP; (2) validating  requirements, including equipment requirements; (3) validating existing  supply levels; (4) submitting requests to DOD components to contract for  procurement of materiel for the ANDSF; and (5) ensuring that the Afghan  government appropriately uses and accounts for U.S. funds provided as  direct contributions from the Afghanistan Security Forces Fund. OSD-P  is responsible for developing policy on and conducting oversight of the  bilateral security relationship with Afghanistan focused on efforts to  develop the Afghan security ministries and their forces.

		U.S.-Purchased Equipment for the ANDSF

In August 2017, we reported that the United States had spent almost $18  billion on equipment and transportation for the ANDSF from fiscal years  2005 through April 2017, representing the second-largest expenditure  category from the Afghanistan Security Forces Fund. In that report, we  identified six types of key equipment the United States funded for the  ANDSF in fiscal years 2003 through 2016, including approximately:
600,000 weapons, such as rifles, machine guns, grenade launchers,  shotguns, and pistols;
163,000 tactical and nontactical radios, such as handheld radios and
76,000 vehicles, such as Humvees, trucks, recovery vehicles, and  mine resistant ambush protected vehicles;
30,000 equipment items for detecting and disposing of explosives,  such as bomb disposal robots and mine detectors;
16,000 equipment items for intelligence, surveillance, and  reconnaissance, such as unmanned surveillance drones and night  vision devices; and
208 aircraft, such as helicopters, light attack aircraft, and cargo  airplanes.

		ANDSF Organization and Force Levels

The Ministry of Defense oversees the ANA, and the Ministry of the Interior  oversees the ANP. According to DOD reporting, the authorized force level  for the ANDSF, excluding civilians, as of June 2018 was 352,000:  227,374 for the Ministry of Defense and 124,626 for the Ministry of  Interior. The ANA includes the ANA corps, Afghan Air Force, Special  Mission Wing, ANA Special Operations Command, and Ktah Khas  (counterterrorism forces). The ANP includes the Afghan Uniformed  Police, Afghan Anti-Crime Police, Afghan Border Police, Public Security  Police, Counter Narcotics Police of Afghanistan, and General Command  of Police Special Units.
The ANA Special Mission Wing, Ktah Khas, ANA Special Operations  Command, and ANP General Command of Police Special Units are  collectively referred to as the Afghan Special Security Forces. In this  report, we refer to the Afghan Air Force and the Afghan Special Security  Forces as specialized forces, and the other components of the ANDSF as  conventional forces. According to DOD reporting, the combined  authorized force level for the specialized forces as of June 2018 was  approximately 34,500, or about 10 percent of the ANDSF’s total  authorized force level of 352,000, compared with the conventional forces,  which make up about 74 percent of the total authorized force level for the  ANDSF. Figure 1 shows the ANDSF’s organization.

		Resolute Support Advising Strategy and Goals

U.S. and coalition advisors from Resolute Support focus on capacity  building at the Ministry of Defense, Ministry of Interior, and ANDSF  regional headquarters, according to DOD reporting. Ministerial advisors  are located at Resolute Support headquarters in Kabul. At the ministerial  level, advisors provide assistance to improve institutional capabilities,  focusing on several functional areas. Table 1 summarizes the indicators  of effectiveness that ministerial advisors are to use to measure ministerial  progress in developing functioning systems that can effectively execute  each of the functional areas.
Regional Resolute Support advisors from seven advising centers located  throughout Afghanistan provide support to nearby ANA corps and ANP  zone headquarters personnel, according to DOD reporting. Some  advisors are embedded with their ANDSF counterparts, providing a  continuous coalition presence, while others provide less frequent support,  based on proximity to and capability of their ANDSF counterparts.  Regional advisors are to track ANDSF capability development by  assessing the progress of the ANA corps and ANP zone headquarters  based on five capability pillars (see table 2). DOD and other Resolute  Support advisors are to document the results of these assessments each  quarter in an ANDSF Assessment Report.
According to DOD reporting, in addition to ministerial and regional  advising, two tactical-level advisory commands provide continuous  support for the ANDSF’s specialized forces: Train, Advise, and Assist  Command–Air (TAAC-Air) advises the Afghan Air Force down to the unit  level, and NATO Special Operations Component Command–Afghanistan  (NSOCC-A) primarily provides tactical-level special operations advising  for the Afghan Special Security Forces. TAAC-Air and NSOCC-A  assess capabilities at the headquarters level based on the five capability  pillars described above in table 2, and these assessments are included in  the quarterly ANDSF Assessment Report. Figure 2 shows the levels of  advising each Resolute Support advisory command type provides for the  ANDSF conventional forces and specialized forces.

		ANDSF Capabilities Reportedly Continue to Improve; DOD Has Identified Several Capability Gaps and Initiated Efforts to Address Them DOD Has Reported the ANDSF Generally Continue to Improve Their Capabilities but Rely on Coalition Forces to Fill Several Critical Capability Gaps

Since Resolute Support began, the ANDSF have improved some  capabilities related to the functional areas and capability pillars described  above, but face several capability gaps that leave them reliant on coalition  assistance, according to publicly available DOD reporting. DOD defines  capability as the ability to execute a given task. A capability gap is the  inability to execute a specified course of action, such as an ANDSF  functional area or a capability pillar (see tables 1 and 2 above). According  to DOD guidance, a gap may occur because forces lack a materiel or  non-materiel capability, lack proficiency or sufficiency in a capability, or  need to replace an existing capability solution to prevent a future gap  from occurring.
According to DOD reporting on the Afghan security ministries, ANA corps,  and ANP zones, the ANDSF generally have improved in some capability  areas since Resolute Support began, with some components performing  better than others. For example, DOD has reported that the Afghan  ministries have improved in operational planning, strategic  communications, and coordination between the Ministry of Interior and  Ministry of Defense at the national level. In general, the ANA is more  capable than the ANP, according to DOD reporting. According to DOD  officials and SIGAR reporting, this is due, in part, to the ANA having more  coalition advisors and monitoring than the ANP. DOD officials also noted  that the Ministry of Interior, which oversees the ANP, and Afghanistan’s  justice system are both underdeveloped, hindering the effectiveness of  the ANP. Corruption, understaffing, and training shortfalls have also  contributed to the ANP’s underdevelopment, according to DOD and  SIGAR reporting. The Afghan Special Security Forces are the most  capable within the ANDSF and can conduct the majority of their  operations independently without coalition enablers, according to DOD  reporting. DOD and SIGAR reports have attributed the Afghan Special  Security Forces’ relative proficiency to factors such as low attrition rates,  longer training, and close partnership with coalition forces. The Afghan Air  Force is becoming increasingly capable, and can independently plan for  and perform some operational tasks, such as armed overwatch and aerial  escort missions, according to DOD reporting.
However, DOD has reported that the ANDSF generally continue to need  support in several key areas. For example, as of December 2017, DOD  reported several ministerial capability gaps, including force management;  logistics; and analyzing and integrating intelligence, surveillance, and  reconnaissance information. DOD also reported that, as of December  2017, the ANA and ANP continued to have capability gaps in several key  areas, such as weapons and equipment sustainment and integrating fire  from aerial and ground forces. The ANDSF rely on support from  contractors and coalition forces to mitigate capability gaps in these key  areas. For some capability areas, such as aircraft and vehicle  maintenance and logistics, the ANDSF is not expected to be self- sufficient until at least 2023, according to DOD reporting.
According to DOD officials and SIGAR reporting, coalition and contractor  support helps mitigate ANDSF capability gaps in the immediate term but  may make it challenging to assess the ANDSF’s capabilities and gaps  independent of such support. For example, vehicle and aircraft  maintenance contractors are responsible for sustaining specific  operational readiness rates for the equipment they service. While this  helps ensure that ANDSF personnel have working equipment to  accomplish their mission, thereby closing an immediate capability gap, it  may mask the ANDSF’s underlying capabilities and potentially prolong  reliance on such support, according to DOD officials and SIGAR  reporting.

		DOD and the ANDSF Have Plans and Initiatives in Place to Address Some ANDSF Capability Gaps

DOD and the ANDSF have begun implementing plans and initiatives that  aim to strengthen ANDSF capabilities. These include the following,  among others:
ANDSF Roadmap. In 2017, the Afghan government began  implementing the ANDSF Roadmap—a series of developmental  initiatives that seek to strengthen the ANDSF and increase security  and governance in Afghanistan, according to DOD reporting. The  Roadmap is structured to span 4 years, but DOD has reported that its  full implementation will likely take longer than that. According to DOD  reporting, the Roadmap aims to improve four key elements: (1)  fighting capabilities; (2) leadership development; (3) unity of command  and effort; and (4) counter-corruption efforts.
Under the Roadmap’s initiative to increase the ANDSF’s fighting  capabilities, DOD and the ANDSF have begun implementing plans to  increase the size of the specialized forces. Specifically, DOD reports  that the ANDSF plans to nearly double the size of the Afghan Special  Security Forces by 2020 as an effort to bolster the ANDSF’s offensive  reach and effectiveness. The Afghan Special Security Forces are to  become the ANDSF’s primary offensive force, the conventional ANA  forces are to focus on consolidating gains and holding key terrain and  infrastructure, and the conventional ANP forces are to focus on  community policing efforts. In addition, to provide additional aerial fire  and airlift capabilities, the ANDSF began implementing an aviation  modernization plan in 2017. The aim is to increase personnel strength  and the size of the Afghan Air Force and Special Mission Wing fleets  by 2023.
Enhanced vehicle maintenance efforts. To help improve the  ANDSF’s vehicle maintenance abilities, DOD awarded a National  Maintenance Strategy Ground Vehicle Support contract, which,  according to DOD officials, became fully operational in December  2017. The National Maintenance Strategy Ground Vehicle Support  contract consolidated five separate vehicle maintenance and training  contracts into a single contract and contains provisions for building the  capacity of ANDSF and Afghan contractors to incrementally take  control of vehicle maintenance over a 5-year period.
Additional U.S. military personnel. As part of the South Asia  strategy, the United States committed 3,500 additional military  personnel to increase support to its missions in Afghanistan.  According to DOD reporting, most of the additional personnel will  support the Resolute Support mission, providing more advising and  combat enabler support to the ANDSF. Additionally, in March 2018,  the United States began deploying a Security Force Assistance  Brigade—a new type of unit made up of U.S. Army personnel with  expertise in training foreign militaries—to Afghanistan. The Security  Force Assistance Brigade will advise conventional and specialized  forces at and below the corps and zone levels and will accompany  and support ANA conventional forces at the battalion level in ground  operations as needed, according to DOD and SIGAR reporting.

	DOD Has Some Information on ANDSF Specialized Forces’ Ability to Operate and Maintain U.S.-Purchased Equipment but Has Limited Reliable Information on Its Conventional Forces

		DOD Advisors Embedded with Specialized Forces Provide Some Information on Those Forces’ Capabilities

DOD collects some reliable information about the operation and  maintenance abilities of ANDSF specialized forces, in part because  advisors are embedded at the tactical level with the specialized forces,  according to DOD officials. Specifically, U.S. and coalition forces advise  specialized forces at the tactical level under Resolute Support because  building ANDSF aviation and special operations abilities are considered  particularly important, according to DOD reporting. DOD officials told us  that since U.S. and coalition forces are embedded at the tactical level for  specialized forces, they can monitor, assess, and report on tactical  abilities, including the ability to operate and maintain equipment.
Our analysis of information provided by DOD about the Afghan Air  Force’s ability to operate and maintain MD-530 helicopters illustrates that  DOD has some detailed information about specialized forces. TAAC-Air  advisors help train Afghan pilots and maintainers and collect information  on their tactical abilities. For example, TAAC-Air advisors track the  percentage of maintenance performed by Afghan Air Force maintainers  and aircraft operational readiness rates, according to DOD officials.  According to DOD reporting and officials, as of December 2017, the  Afghan Air Force could independently conduct MD-530 helicopter  operations for short intervals without contractor support but relied on  contractors to perform the majority of maintenance and sustainment  activities. See appendix II for more information on the Afghan Air Force’s  ability to operate and maintain MD-530 helicopters.

		DOD Advisors Have Limited Contact with Conventional Forces in the Field, Yielding Little Information on Their Ability to Operate and Maintain Equipment

U.S. and coalition forces perform high-level assessments of the ANDSF  conventional forces’ capabilities at the corps and zone levels but do not  assess their tactical abilities, such as the ability to operate and maintain  equipment, according to DOD officials. For example, U.S. and coalition  forces assess the ANA and ANP conventional forces in quarterly ANDSF  Assessment Reports, but these reports are at the corps and zone  headquarters levels, and are not meant to provide an evaluation of the  entire ANDSF, according to DOD reporting. DOD officials stated that  other U.S.- and coalition-produced reports and assessments, such as  DOD’s semiannual Section 1225 reports to Congress, semiannual  periodic mission reviews, and annual Afghanistan Plans of Record,  provide some information on the ANDSF’s high-level capabilities.  However, according to DOD officials, these reports do not routinely  assess the conventional forces’ ability to operate and maintain  equipment.
According to DOD officials, DOD does not assess conventional forces’  tactical abilities because advisors have had little or no direct contact with  conventional units below the corps and zone levels, and thus do not  collect such information on conventional forces. Specifically, under  Resolute Support, U.S. and coalition forces have not embedded with the  conventional forces below the corps and zone levels except in limited  circumstances.
Since U.S. and coalition forces do not collect firsthand information on the  conventional units’ tactical abilities, they rely on those units’ self-reporting  for information on ANDSF abilities below the corps and zone levels,  which, according to DOD officials, may be unreliable. ANDSF reporting is  not verified by U.S. officials and can be unreliable in its consistency,  comprehensiveness, and credibility, according to DOD officials and  SIGAR. For example, the ANDSF produce a monthly tracker on vehicle  availability, maintenance backlog, repair times, and personnel  productivity, but DOD officials told us that the trackers are of questionable  accuracy.
Our analysis of information provided by DOD about the ANDSF’s ability to  operate and maintain tactical and nontactical radios illustrates the limited  amount of information DOD has on ANDSF conventional forces’ tactical  abilities. Specifically, DOD officials could not say how well ANDSF  personnel on the front lines operate radios in the field and had only  limited information on the ANDSF’s ability to maintain radios. For  example, the officials noted that the ANA conventional forces can perform  some unit-level radio repairs but that complex ANA radio maintenance  and all ANP radio maintenance is conducted by contractors. DOD officials  at Resolute Support headquarters told us that they provide ministerial- level advising on how to manage ANDSF radio systems and do not  provide tactical advising or inventory control for radios. See appendix III  for more information on the ANDSF’s ability to operate and maintain  radios.
Our analysis of information provided by DOD about the ANDSF’s ability to  operate and maintain Mobile Strike Force Vehicles (MSFV) highlights the  limited amount of information DOD has on ANDSF conventional forces’  tactical abilities compared with specialized forces. DOD officials were  able to provide operation and maintenance information for MSFVs that  had transferred to the specialized forces as of January 2018 but were  unable to provide operation and maintenance information for any other  MSFVs. The ANDSF began transferring one of the ANDSF’s two MSFV  brigades from the conventional to specialized forces in August 2017,  according to DOD officials. As part of this transfer, NSOCC-A  advisors—who provide tactical-level advising for the Afghan Special  Security Forces—assumed oversight for the first brigade from Resolute  Support headquarters advisors. DOD officials stated that the ANDSF’s  ability to operate and maintain MSFVs in this brigade prior to the transfer  was unknown, as neither Resolute Support headquarters nor the ANA  had assessed this. The operation and maintenance abilities of the second  brigade, which is still in the conventional forces, remains unknown. DOD  officials at NSOCC-A were able to provide information such as inventory  and mission capability rates for the MSFVs that had transferred, but only  for the short period of time the vehicles had been under the control of the  specialized forces. DOD officials told us that NSOCC-A plans to collect  more information on the specialized forces’ ability to operate and maintain  MSFVs as they are transferred. See appendix IV for more information on  the ANDSF’s ability to operate and maintain MSFVs.
In the absence of embedded advisors at the tactical level, DOD has not  implemented alternative approaches to collect reliable information about  the conventional forces’ ability to operate and maintain equipment.  Federal internal control standards state that U.S. agencies should obtain  and process reliable information to evaluate performance in achieving key  objectives and assessing risks. DOD officials acknowledged that some  of the plans described above that DOD and the ANDSF have begun  implementing to address capability gaps may provide opportunities for  DOD to collect more reliable information on the conventional forces’  ability to operate and maintain U.S.-purchased equipment. For example,  the National Maintenance Strategy Ground Vehicle Support contract  requires that contractors regularly report the total work orders received,  work in progress, and completed maintenance work performed by ANDSF  personnel as well as vehicle availability rates, which may be more reliable  than the ANDSF’s monthly report on vehicle availability. In addition, the  Security Force Assistance Brigade may be able to collect and report on  the tactical abilities of units they advise and accompany on missions  since they are being deployed at or below the corps and zone levels.  However, as of June 2018, DOD officials had not decided which, if any, of  these options to pursue. Without reliable information on the equipment  operation and maintenance abilities of ANDSF conventional forces, which  represent nearly 75 percent of the ANDSF, DOD may be unable to fully  evaluate the success of its train, advise, assist, and equip efforts in  Afghanistan.

	Conclusions

The United States invested nearly $84 billion in Afghan security in the 17- year period spanning fiscal years 2002 through 2018, but DOD continues  to face challenges to developing a self-sustaining ANDSF. While DOD  has reported the ANDSF have improved in several capability areas, they  continue to face critical capability gaps, impeding their ability to maintain  security and stability in Afghanistan independent of U.S. and coalition  forces. Moreover, DOD lacks reliable information about the degree to  which conventional forces—which make up about three-quarters of the  ANDSF—are able to operate and maintain U.S.-purchased equipment.  This limits DOD’s ability to fully evaluate the success of its train, advise,  assist, and equip efforts in Afghanistan.

	Recommendation for Executive Action

The Secretary of Defense should develop and, as appropriate, implement  options for collecting reliable information on the ANDSF conventional  forces’ ability to operate and maintain U.S.-purchased equipment.  (Recommendation 1)

	Agency Comments

We provided a draft of this report to DOD and State for comment. DOD  declined to provide written comments specifically on this public version of  the report, but DOD’s comments on the sensitive version of this report are  reprinted in appendix V. The sensitive version of this report included two  recommendations, which DOD cited in its comments on the draft of the  sensitive report. One of those recommendations related to information  that DOD deemed to be sensitive and that must be protected from public  disclosure. Therefore, we have omitted that recommendation from DOD’s  comment letter in appendix V. This omission did not have a material  effect on the substance of DOD’s comments. In its comments, DOD  concurred with the recommendation we made in this version of the report  and stated it will take steps to implement it. DOD also provided technical  comments, which we incorporated as appropriate. The Department of  State had no comments.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of Defense, and the Secretary of State. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov. If you or your staff has any questions about this  report please contact me at (202) 512-7114 or farbj@gao.gov. Contact  points for our Offices of Congressional Relations and Public Affairs may  be found on the last page of this report. GAO staff who made key  contributions to this report are listed in appendix VI.

Appendix I: Objectives, Scope, and Methodology

House Report 114-537 associated with the National Defense  Authorization Act for Fiscal Year 2017 included a provision for us to  review the Afghan National Defense and Security Forces’ (ANDSF)  capability and capacity to operate and sustain U.S.-purchased weapon  systems and equipment. This report is a public version of a sensitive  report that we issued on September 20, 2018. Our September report  included three objectives, including one on the extent to which DOD  considers ANDSF input and meets their needs when identifying  equipment requirements. DOD deemed the information related to that  objective to be sensitive, which must be protected from public disclosure.  Consequently, we removed that objective and a related recommendation  from this public report. This version includes information on the other two  objectives: (1) what has been reported about ANDSF capabilities and  capability gaps and (2) the extent to which DOD has information about  the ANDSF’s ability to operate and maintain U.S.-purchased equipment.  Although the information provided in this report is more limited, the report  uses the same methodology for the two objectives as the sensitive report.
To identify what has been reported about ANDSF capabilities and  capability gaps, we reviewed North Atlantic Treaty Organization (NATO)  and DOD documents and reports, such as DOD’s semiannual Section  1225 reports to Congress, produced after the start of the NATO-led  Resolute Support mission on January 1, 2015. To determine what steps  DOD and NATO have taken to try to address gaps, we reviewed reports  the Center for Naval Analyses produced for DOD, as well as DOD and  NATO documents and reports produced after January 1, 2015, and  reports from GAO, the Special Inspector General for Afghanistan  Reconstruction (SIGAR), and the DOD Inspector General. We also  interviewed Center for Naval Analyses representatives and DOD officials  in the United States and Afghanistan, including DOD officials at the  Combined Security Transition Command–Afghanistan (CSTC-A) and in  the Office of the Undersecretary of Defense for Policy (OSD-P) who  helped create the DOD reporting we reviewed.
To determine the extent to which DOD has information about the  ANDSF’s ability to operate and maintain U.S.-purchased equipment, we  reviewed DOD documents and reports and interviewed DOD officials in  the United States and Afghanistan, including DOD officials who advise the ANDSF. We also reviewed federal internal control standards to  determine what responsibilities agencies have specifically related to  information collection. To provide illustrative examples of information  DOD has about the ANDSF’s ability to operate and maintain U.S.- purchased equipment and what that information indicates about the  ANDSF’s abilities and challenges, we interviewed and analyzed written  responses from DOD officials, including DOD officials who provide  procurement and lifecycle management for some ANDSF aircraft and  vehicles, about three equipment types—MD-530 helicopters, Mobile  Strike Force Vehicles (MSFV), and radios. We selected these three  equipment types from a list that we developed, for an August 2017 report,  of key ANDSF equipment the United States purchased from fiscal years  2003 through 2016. We made our selections after reviewing DOD  documentation and interviewing DOD officials regarding a number of  considerations, such as (1) how critical the equipment is to the ANDSF’s  ability to achieve its mission; (2) which ANDSF component uses the  equipment (i.e., Afghan National Police, Afghan National Army, or both);  (3) whether DOD intends to continue procuring the equipment for the  ANDSF; and (4) whether the equipment had been in use at least 5 years.
We collected detailed information about the ANDSF’s ability to operate  and maintain MD-530 helicopters, MSFVs, and radios, as well as other  key statistics DOD provided about the equipment, such as inventory,  average lifespan, average cost, role, and training. This information was  based mainly on DOD responses collected from January 2018 to  February 2018 as well as DOD documents and reports produced after  January 1, 2015. The total amount of MD-530s and radios authorized for  procurement was based on DOD data that we collected for our August  2017 report on key ANDSF equipment the United States purchased in  fiscal years 2003 through 2016, which we supplemented with additional  data DOD provided on U.S.-purchased equipment from October 1, 2016,  through December, 31, 2017.
The performance audit upon which this report is based was conducted  from August 2016 to September 2018 in accordance with generally  accepted government auditing standards. Those standards require that  we plan and perform the audit to obtain sufficient, appropriate, evidence  to provide a reasonable basis for our findings and conclusions based on  our audit objectives. We believe that the evidence obtained provides a  reasonable basis for our findings and conclusions based on our audit  objectives. We subsequently worked with DOD from September 2018 to  October 2018 to prepare this public version of the original sensitive report  for public release. This public version was also prepared in accordance  with those standards.
Manufacturer: MD Helicopters,  Inc.
U.S. Program Management  Office: U.S. Army, Non-Standard  Rotary Wing Aircraft Project  Management Office  Program Advising: Train Advise  Assist Command–Air (TAAC-Air)
The United States originally procured 6 unarmed MD-530s for the AAF for  rotary wing training in 2011. In 2014, the United States purchased 12 armed  MD-530s and began retrofitting the 5 remaining trainer helicopters with  armament for operational missions to address a close air attack gap. MD- 530s were chosen to fill the gap over other aircraft, in part because they  could be delivered relatively quickly as the AAF awaited A-29 light attack  aircraft that were experiencing procurement delays, according to Department  of Defense (DOD) officials. The United States procured additional MD-530s  in 2015, 2016, and 2017 because of the aircraft’s positive impact on the  battlefield, according to DOD officials (see fig. 3).
Key Statistics   Variants: All can be armed with  .50-cal machine gun pods and/or  2.75 inch rocket pods.
Total Authorized for  Procurement: 60 as of December  31, 2017  Inventory: 25 as of January 2018  (30 are scheduled for delivery;  attrition of 5 due to crashes and  enemy fire)
Average Lifespan: Absent  mishaps, and with good  maintenance, there is no defined  lifespan limit for MD-530s,  according to DOD officials.
National Army and Afghan National Police, depending on the mission, in  all but one region of Afghanistan, which is supported by other aircraft.
MD-530s are typically tasked two at a time for missions, according to  DOD officials.
An MD-530 crew consists of a pilot and co-pilot, according to DOD.
Average cost: $6.3 million per  aircraft, including all electronic  devices, weapons management  systems, and weapons (excluding  ordnance), according to DOD  officials. o  Division of labor is based on the individual crew members'  capabilities, with one pilot handling navigation and communication  while the other identifies targets and operates the weapon systems.
Army pilot advisors at Kandahar Air Field, according to DOD officials.  o  MD-530 pilot training takes about 3 years (see fig. 4).

	GAO Comments

1. The GAO report number cited in DOD’s letter refers to a draft of the  sensitive version of this report, which we issued on September 20, 2018.  Prior to issuing that version, we changed its report number to GAO-18- 662SU to reflect its sensitive nature. That version of this report included  two recommendations. The second recommendation has been omitted  from DOD’s letter in this public version because it was related to  information that DOD deemed to be sensitive.

Appendix VI: GAO Contacts and Staff Acknowledgments

	GAO Contacts

	Staff Acknowledgments

In addition to the contact named above, Joyee Dasgupta (Assistant  Director), Kara Marshall, Katherine Forsyth, and Bridgette Savino made  key contributions to this report. The team also benefitted from the expert  advice and assistance of David Dayton, Neil Doherty, Justin Fisher,  Ashley Alley, Cary Russell, Marie Mak, James Reynolds, Sally  Williamson, Ji Byun, and J. Kristopher Keener.

Output:
Why GAO Did This Study

Developing independently capable ANDSF is a key component of U.S. and coalition efforts to create sustainable security and stability in Afghanistan under the North Atlantic Treaty Organization (NATO)-led Resolute Support mission. The United States is the largest contributor of funding and personnel to Resolute Support, providing and maintaining ANDSF equipment, along with training, advising, and assistance to help the ANDSF effectively use and sustain the equipment in the future.
House Report 114-537 included a provision for GAO to review the ANDSF's capability and capacity to operate and sustain U.S.-purchased weapon systems and equipment. This report addresses (1) what has been reported about ANDSF capabilities and capability gaps and (2) the extent to which DOD has information about the ANDSF's ability to operate and maintain U.S.-purchased equipment. To conduct this work, GAO analyzed DOD and NATO reports and documents, examined three critical equipment types, and interviewed DOD officials in the United States and Afghanistan.
This is a public version of a sensitive report issued in September 2018. Information that DOD deemed sensitive has been omitted.

What GAO Found

Since the Resolute Support mission began in 2015, the Afghan National Defense and Security Forces (ANDSF) have improved some fundamental capabilities, such as high-level operational planning, but continue to rely on U.S. and coalition support to fill several key capability gaps, according to Department of Defense (DOD) reporting. DOD has initiatives to address some ANDSF capability gaps, such as a country-wide vehicle maintenance and training effort, but DOD reports it does not expect the ANDSF to develop and sustain independent capabilities in some areas, such as logistics, for several years.
While DOD has firsthand information on the abilities of the Afghan Air Force and Special Security Forces to operate and maintain U.S.-purchased equipment, it has little reliable information on the equipment proficiency of conventional ANDSF units. U.S. and coalition advisors are embedded at the tactical level for the Air Force and Special Security Forces, enabling DOD to directly assess those forces' abilities. However, the advisors have little direct contact with conventional ANDSF units on the front lines. As a result, DOD relies on those units' self-assessments of tactical abilities, which, according to DOD officials, can be unreliable.
GAO's analysis of three critical equipment types illustrated the varying degrees of DOD's information (see figure above). For example, DOD provided detailed information about the Air Force's ability to operate and maintain MD-530 helicopters and the Special Security Forces' ability to operate and maintain Mobile Strike Force Vehicles; however, DOD had limited information about how conventional forces operate and maintain radios and Mobile Strike Force Vehicles. DOD's lack of reliable information on conventional forces' equipment operations and maintenance abilities adds to the uncertainty and risk in assessing the progress of DOD efforts in Afghanistan.

What GAO Recommends

GAO recommends that DOD develop options for collecting reliable information on conventional ANDSF units' ability to operate and maintain U.S.-purchased equipment. DOD concurred with this recommendation.