Input:
	Background

ARPA-E’s typical funding announcement and award selection process  begins with the agency hiring a program director responsible for  identifying a gap in energy technology research and developing a  program to fill that gap. ARPA-E is required by statute to achieve its  goals through energy technology projects that, among other things,  accelerate transformational technological advances in areas that industry  on its own is not likely to undertake because of technical and financial  uncertainty, while also ensuring that its activities are coordinated with,  and do not duplicate the efforts of, programs and laboratories within DOE  and other relevant research agencies. ARPA-E’s efforts to identify  existing energy technology research gaps and to design a program to  address those gaps involve research; consultation with scientific experts,  including a workshop with outside experts; and internal discussions within  ARPA-E. From this process, program directors develop funding  opportunity announcements that describe the technical requirements  specific to each program’s technology area that applicants have to meet,  as well as the four standard criteria that ARPA-E uses to guide its merit  selection process. Following the issuance of a funding opportunity  announcement, ARPA-E employs the following multi-stage process to  merit review applications, make funding award decisions, and monitor  projects:
Concept paper. Applicants initially submit a 4- to 7-page abstract of  their projects. Scientific experts from government, industry, and  academia serve as reviewers.
Full application. After reviewing concept papers, ARPA-E  encourages some applicants to submit full applications. Full  applications are generally quite extensive, requesting information on  the technical and financial aspects of the proposed project, among  other things. ARPA-E officials we interviewed noted that these  applications are frequently more than 100 pages and can take 30 to  45 days for the applicant to develop. Full applications are reviewed  against the selection criteria by leading scientific experts in the  relevant field and assigned numerical scores.
Reply to reviewer comments. After reviewing a full application,  reviewers provide comments and questions to the applicants, who  then have the opportunity to respond.
Selection. A three- to four-person panel, chaired by the relevant  ARPA-E program director, considers the reviewers’ comments and  numerical scores and recommends applications for an award. The  final decisions on which applicants to select for award negotiations  are made by the selecting official, usually the Director of ARPA-E.
Award negotiations. Once selections are made, ARPA-E program  directors work closely with selectees to negotiate the terms and  conditions of their award. These negotiations include, among other  things, developing a project plan with technical milestones that are to  be met during the 2 to 3 years that the award is being funded, a  budget and management plan, and an intellectual property and data  management plan. Funds are awarded once negotiations regarding  the terms and conditions of the award are concluded. ARPA-E seeks  to complete negotiations regarding the terms and conditions of an  award within approximately 100 days of sending a letter notifying an  applicant that they have been selected for award negotiations.
Selectees may be allowed to begin spending money to start work on  their projects up to 90 days prior to the completion of award  negotiations. However, these expenditures are made with the risk that  applicants may not be reimbursed if award negotiations are  unsuccessful and ARPA-E does not fund the award.
Monitoring. ARPA-E monitors and supports the projects it funds  through quarterly reviews and site visits. At any point during the  award, ARPA-E may decide whether to continue or terminate the  project based on whether agreed-upon project milestones are being  met.

	DOE Developed and Implemented a New Process to Assess the Department’s Financial Assistance against the Administration’s Priorities

In 2017, DOE developed and implemented a new review process to  assess DOE financial assistance for new work against the current  administration’s priorities, including financial assistance for which ARPA- E had already made award selections. DOE reviewed and approved  ARPA-E’s opportunities for financial assistance on a rolling basis from  May to September 2017, and nearly all were approved to proceed.
The formal review of DOE financial assistance officially began on May 4,  2017, when DOE’s Chief of Staff issued a memorandum stating that  funding opportunity announcements and determinations of non- competitive financial assistance would be reviewed to ensure consistency  with the administration’s priorities. According to the memorandum, DOE  agencies that award financial assistance—referred to in this report as  funding organizations—were to provide information about each  competitively selected funding announcement and determination of non- competitive financial assistance by May 15, 2017. This information  included, for example, a brief description of the financial assistance, the  number and amount of planned awards, and the technology readiness  level of the projects being funded. DOE Office of Management officials  told us that the agency’s financial assistance review lasted through  September 2017, as some DOE organizations continued to submit new  financial assistance for review, but that the review was largely completed  by August 10, 2017. However, while the formal review of DOE financial  assistance began in May, award negotiations for ARPA-E-funded projects  were suspended nearly 1 month earlier. Specifically, according to ARPA- E officials, DOE’s Deputy Chief of Staff verbally directed ARPA-E on April  6, 2017 to stop all ongoing award negotiations. Figure 1 shows the  timeline of DOE’s review of ARPA-E financial assistance.
Pursuant to the DOE Chief of Staff’s May 4th memorandum, ARPA-E and  other DOE funding organizations submitted the requested information to  the DOE review team, which was coordinated and facilitated by the  Director of DOE’s Office of Management. Other members of the financial  assistance review team included DOE’s acting Chief Financial Officer;  deputy assistant secretaries, chiefs of staff, and senior advisors at several  DOE funding organizations; and members of the department’s  congressional affairs and public affairs staff.
According to DOE Office of Management officials we interviewed, the  review team assessed the department’s financial assistance against five  criteria:
Whether the financial assistance was statutorily mandated;
Whether the financial assistance was described in congressional
Whether the financial assistance was consistent with administration  priorities, as identified in budget documents and other statements  from the President and Secretary of Energy, among other things;
What technology readiness level the financial assistance was  intended to fund; and
Whether the technology encompassed by the project was already  being funded by the private sector or others.
DOE Office of Management officials stated that the review team did not  use the above criteria to assign quantitative scores to evaluate the  department’s financial assistance; instead, the team collaboratively  discussed each opportunity for assistance. In most cases, the review  team was able to reach consensus on whether the financial assistance  aligned with the administration’s priorities. DOE Office of Management  officials also noted that they met with ARPA-E leadership to obtain  additional information about ARPA-E financial assistance on three  occasions during the course of the review. ARPA-E officials said that, in  addition to those three meetings, they provided written information to  address questions received from the review team and to provide  additional context regarding ARPA-E financial assistance.
In total, DOE’s review team assessed 6 ARPA-E fiscal year 2017 or prior- year funding opportunity announcements for which applicants had been  selected for award negotiation, 7 fiscal year 2017 announcements in the  earlier stages of the merit review and selection process, 2 fiscal year  2017 announcements that had not yet been released, and 17  opportunities for financial assistance where ARPA-E funded renewals or  new work under a determination of noncompetitive financial assistance.  According to DOE Office of Management officials, the review team  worked as quickly as possible to review all of DOE’s financial assistance  to minimize potential disruptions for recipients and DOE’s funding  organizations. Once the review team approved an opportunity for financial  assistance, DOE funding organizations were allowed to resume work,  DOE Office of Management officials told us. Figure 2 shows the total  cumulative funds for ARPA-E financial assistance approved by the review  committee at various stages in the review. For example, as shown in  Figure 2, the review team approved roughly $158.3 million (55.6 percent)  of ARPA-E’s proposed financial assistance on May 18, 2017, 3 days after  the deadline for DOE funding organizations to submit information to the  review team. The remaining proposed financial assistance was approved  in several stages from June through August 2017. As of August 25, 2017,  all of ARPA-E’s competitively selected funding opportunity  announcements, renewals, and determinations for noncompetitive  financial assistance, where selectees had been selected for negotiation,  were approved by the review team, representing roughly $265 million, or  93.1 percent, of all ARPA-E funding reviewed by the team.
DOE Office of Management officials also stated that the financial  assistance review team made a decision early in the course of the review  to honor all existing DOE commitments to fund new work. These officials  said that this extended to commitments made to entities that had been  selected for award negotiations, even though the department does not  officially commit to providing funds until such negotiations are completed  and the award is finalized. However, according to ARPA-E selectees we  interviewed, this message was generally not communicated to them,  which led to uncertainty about whether their projects would be funded. In  contrast, the review team recommended that the DOE Chief of Staff  cancel ARPA-E’s Facsimile Appearance to Create Energy Savings  funding announcement, which had accepted full applications but had not  selected any applicants for award negotiation. This opportunity would  have funded the development of advanced information technology that  could allow for three-dimensional digital representation of a person in a  room nearly indistinguishable from the person being there in real life,  which might allow for increased telecommuting. DOE Office of  Management officials told us that the review team reached this  recommendation in part because this technology was already being  funded by the private sector. As of November 2017, DOE Office of  Management officials said the review team had cancelled 3 other DOE  funding announcements as a result of the review.
According to information we collected, DOE’s review of ARPA-E financial  assistance, as part of the DOE-wide review process, did not require the  President to send a special message under the Impoundment Control  Act. Specifically, the delay in obligating ARPA-E funds for financial  assistance examined through DOE’s review process was for  programmatic reasons. DOE officials explained that the purpose of the  review was to ensure that the agency’s financial assistance aligned with  the priorities of the current administration.

	DOE’s Financial Assistance Review Created Uncertainty for ARPA-E Selectees, Which Led to Delayed Project Timelines and Staffing Difficulties, among Other Impacts

According to the 10 ARPA-E selectees we interviewed, DOE’s financial  assistance review process created uncertainty, which led to a variety of  impacts—the most frequently cited of which were potentially delayed  project timelines and difficulties staffing project teams. Selectees told us  that they received little communication from ARPA-E during the review  process, and they indicated that additional information about review  timelines and potential effects on their awards would have helped them  manage some of the uncertainty they experienced during the review  process. DOE Office of Management officials said the fiscal year 2017  review process helped to better identify and coordinate future financial  assistance department-wide on crosscutting issues. DOE is conducting  the fiscal year 2018 review process prior to publicly issuing funding  announcements. As a result, DOE Office of Management officials said,  the delays and uncertainty that selectees experienced in fiscal year 2017  should be reduced.
In our structured interviews with ARPA-E selectees, the most frequently  cited impact of the uncertainty caused by DOE’s financial assistance  review was the potential need to delay project timelines. All of the ARPA- E selectees we interviewed told us that they might need to extend their  project timelines because of uncertainty caused by DOE’s review. Four of  these selectees noted that the delay caused by DOE’s review could  cause additional, cascading delays in their timelines. For example, 1  selectee we interviewed said that it would need to re-issue a hiring  announcement it had publicized prior to the review because the review  prevented it from hiring someone. In addition, the selectee would need to  resubmit the hiring announcement to the university and state human  resources departments for approval, which could take months to process.  Another selectee said that it missed 2 months of a 3-month planting  season because it could not start project work, and had the delay lasted  any longer, the selectee would have missed an entire year of data  collection on the project.
Selectees also cited challenges to staffing project teams as a result of the  uncertainty caused by the review. Selectees stated that delays caused by  the review affected their ability to hire team members they had planned to  hire based on their original schedule, as potential members moved on to  other projects or took different jobs. For example, 9 selectees told us that  they delayed hiring new project team members while DOE’s review was  occurring. Four selectees we interviewed said that they had difficulty  retaining staff during the review process. For example, 1 selectee had to  lay off 2 of the company’s 15 staff members because of the delay in  receiving funding, and several other staff members left voluntarily.  Furthermore, the selectee said laying off these staff members resulted in  an increase in the company’s unemployment taxes, which was expensive  for a small-sized company. Four other selectees that we interviewed said  they had to assign existing project team members to other funded work or  general activities because they could not begin work on their ARPA-E  project until they received funding.
Selectees we interviewed cited additional impacts associated with the  uncertainty caused by DOE’s financial assistance review. These impacts  included:
Delaying equipment purchases. Four selectees reported that they  had to delay purchasing important equipment needed to execute their  project. One selectee noted that the delay caused by the review was  long enough that price quotes it had received from equipment sellers  expired, and that prices could increase in later quotes.
Changes to project scope. Two selectees told us that they might  need to limit the planned scope of their projects to be able to complete  them in the proposed timeline. For example, 1 selectee said its project  involves helping to scale up three to four different technologies a year,  which it might not be able to do if it has to adhere to its initial  timeframes.
Loss of advantage against potential competitors. Four of the  selectees we interviewed said that the delay may have caused their  technology to fall behind their potential competitors in some way. For  example, 1 selectee noted that it was working in a competitive  environment for its technology, with ongoing efforts in multiple  countries, and reported that its project might have fallen behind  others’ efforts as a result of delays associated with DOE’s review.  However, 4 other selectees said that the review was not likely to  cause any loss of competitiveness.
Impacts on external project partners. Three selectees noted that  DOE’s review caused uncertainty for partners on their projects,  including partners that provide external funding. For example, 1  selectee told us that private investors in its technology area are most  active in the fall and that its project team might not be able to seek a  second round of funding if it could not demonstrate the necessary  technical results of the project by then.
Impacts on pre-award spending reimbursements. One selectee  reported that it had to cease certain pre-award spending. The  selectee said that it spent roughly $10,000 on equipment and 150  hours of labor prior to DOE’s financial assistance review, but it could  not submit invoices for these expenditures to ARPA-E while the  review was ongoing and would not be able to if its project was  ultimately not approved. Furthermore, the selectee said that even if  the award was approved, the delay might result in expenditures falling  outside the 90-day window of allowable pre-award expenditures,  which would require obtaining approval from ARPA-E to be  reimbursed.
Selectees we interviewed also stated that they received little  communication from ARPA-E during the review, which contributed to the  uncertainty about the status of their projects. Specifically, 6 of the  selectees said that they would have liked additional information from  ARPA-E on a variety of topics related to the review. For example, 4  selectees said they would have liked additional information about the  review timeline and when it was planned to be completed. One of these  selectees told us that a written document from ARPA-E indicating a rough  time frame and next steps would have helped facilitate better planning for  their project team. Two selectees we interviewed wanted additional  information about whether they could renegotiate their timelines once the  review was completed. Three selectees told us that they would have liked  additional information about whether the review would cause them to lose  their funding. ARPA-E officials we interviewed told us that they made  three separate requests to DOE’s Deputy Chief of Staff to learn what they  could communicate to selectees about the April 6, 2017, verbal order and  the review process. ARPA-E officials told us that they were directed by  the Deputy Chief of Staff not to communicate with selectees about the  verbal order until receiving guidance from his office. ARPA-E developed  proposed language to share with selectees but did not receive approval  from the Deputy Chief of Staff to distribute it. DOE Office of Management  officials we interviewed told us that they did not issue guidance to ARPA- E or other DOE funding organizations about how the organizations should  communicate with selectees during the review.
In contrast to its fiscal year 2017 review, DOE began its 2018 financial  assistance review in August 2017, prior to publicly issuing funding  announcements. On August 10, 2017, DOE’s Office of Management sent  an email to DOE funding organizations directing them to submit  descriptions of their proposed financial assistance by September 8, 2017.  Because the review will occur prior to publicly issuing funding opportunity  announcements, and thus before any recipients apply or are selected,  DOE Office of Management officials said the delays and uncertainty that  selectees experienced in fiscal year 2017 should be reduced.
DOE Office of Management officials told us that—aside from changing  the timing of its 2018 financial assistance review—the review team’s  membership and evaluation criteria will be largely the same as for the  fiscal year 2017 review. The officials said that they discussed the review  process with senior leaders in DOE’s funding organizations to help  ensure that they understood the priorities, expectations, and steps of the  review process. The officials also told us that the review team developed  additional guidance to clarify certain issues that arose during the fiscal  year 2017 review. This additional guidance included:
On August 17, 2017, funding organization managers were informed  that continuation awards—those where the activity is presently being  funded—would be exempt from submission to the review team and  can continue to move forward.
On August 29, 2017, funding organizations were informed that they  should identify financial assistance that falls under one of seven  crosscutting research issue areas. According to DOE Office of  Management officials we interviewed, a key benefit of the fiscal year  2017 review process was that the review team noticed that DOE had  several funding announcements at multiple funding organizations  related to these areas. DOE’s funding organizations may be able to  coordinate to issue a consolidated funding announcement in these  crosscutting research issue areas, to ensure efforts are  complementary and not duplicative. Furthermore, DOE Office of  Management officials we interviewed said that knowing which funding  organizations are funding work in these areas will support DOE  meetings on crosscutting issues.

	Agency Comments

We provided a draft of this report for review and comment to the  Secretary of Energy. DOE provided technical comments, which we  incorporated as appropriate.
We are sending copies of this report to the appropriate congressional  committees, the Secretary of Energy, and other interested parties. In  addition, the report is available at no charge on the GAO website at  http://www.gao.gov.
If you or your staff members have any questions about this report, please  contact me at (202) 512-3841 or neumannj@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made key contributions to  this report are listed in appendix I.

Appendix I: GAO Contact and Staff Acknowledgments

	GAO Contact

	Staff Acknowledgments

In addition to the individual named above, Chris Murray (Assistant  Director), Perry Lusk (Analyst-in-Charge), Antoinette C. Capaccio, John  Delicath, Justin Fisher, Kimberly McGatlin, Dan Royer, Tind Shepper  Ryen, Lauren G. Sherman, and McKenna Storey made key contributions  to the report.

Output:
Why GAO Did This Study

ARPA-E provides funding for research to overcome long-term and high-risk technological barriers in developing energy technologies. Since 2009, ARPA-E has awarded approximately $1.3 billion to universities, public and private companies, and national laboratories to fund energy research projects. Starting in May 2017, DOE began reviewing its financial assistance department-wide, including ARPA-E's, to determine if it met the administration's priorities.
GAO was asked to examine this review process as it pertained to ARPA-E. This report describes (1) how DOE implemented the financial assistance review process; and (2) the perspectives of ARPA-E selectees on the impacts of the review process.
GAO reviewed documents and interviewed officials at ARPA-E and DOE's Office of Management, which coordinated the review. GAO also interviewed a nonprobability sample of 10 of the 68 ARPA-E award selectees whose financial assistance was evaluated under the review. GAO identified selectees to interview based on representation across ARPA-E's recipient types, including universities, private companies, and national laboratories, among other criteria. While the views of selectees GAO interviewed cannot be generalized to all affected ARPA-E selectees, they provide illustrative examples of the effects of DOE's review.

What GAO Found

The Department of Energy (DOE) developed and implemented a new process to review its financial assistance to ensure that all new work funded by the department—including by DOE's Advanced Research Projects Agency-Energy (ARPA-E)—was consistent with the current administration's priorities. The review process covered funding opportunity announcements as well as certain other types of financial assistance. New awards were delayed until the review of the underlying financial assistance opportunity was completed. DOE reviewed and approved ARPA-E's financial assistance on a rolling basis from May through September 2017, and nearly all ARPA-E financial assistance was approved. DOE Office of Management officials met with ARPA-E officials on several occasions to discuss their review of ARPA-E financial assistance. DOE officials GAO interviewed said they wanted to complete the review as quickly as possible to minimize effects on DOE programs. GAO determined that the delay was not reportable under the Impoundment Control Act. The Impoundment Control Act requires the President to notify Congress if an agency wants to withhold the obligation of funds. GAO has separately informed Congress of an impoundment of $91 million in funds that were not allocated to any financial assistance awards, and was not related to DOE's review process.
According to the 10 ARPA-E project selectees GAO interviewed, DOE's financial assistance review process created uncertainty, which led to a variety of project impacts. The impacts most commonly cited by selectees included potentially delayed project timelines, as well as difficulties in staffing their project teams, among other impacts as shown below.
DOE officials GAO interviewed said that they are reviewing DOE financial assistance in fiscal year 2018. DOE officials said that a key benefit of the fiscal year 2017 review process was an opportunity to better identify and coordinate future financial assistance department-wide on crosscutting issues. However, DOE plans to review fiscal year 2018 financial assistance prior to issuing funding opportunity announcements to the public, and thus before any recipients apply or are selected. As a result, DOE officials said, the uncertainty that ARPA-E selectees experienced during the fiscal year 2017 review process should be reduced.

What GAO Recommends

GAO is not making any recommendations. DOE provided technical comments, which GAO incorporated as appropriate.


Input:
	Background

The FMS program, which transfers defense articles and services to  international partners and organizations, is essentially an acquisition  process through which the U.S. government procures military equipment,  training, and other services on behalf of foreign customers. Multiple  organizations have a role in the FMS program. The Department of State  has overall responsibility for the program, including approving what  defense items and services can be sold to specific countries. DOD  administers the FMS program and manages the procurements executed  within the military departments on behalf of foreign governments. Within  DOD, DSCA carries out key functions such as supporting development of  policy for FMS. The military departments carry out the day-to-day  implementation of FMS procurements which can include providing price  and availability data at the customer’s request.
Typically, defense items—such as weapon systems—made available for  transfer or sale to foreign customers are systems that have completed  operational testing and are entering or have entered full rate production.  In addition, DOD also sells non-standard items, which are defined as  items that DOD does not currently manage and may include items that (1)  are commercially available, (2) DOD previously purchased and have  since been retired, or (3) were purchased in a different configuration for  DOD components. For example, a customer may express interest in  buying tanks that DOD no longer buys for its own needs. A customer may  also express interest in buying a tank that DOD currently procures but  with a radio communications configuration that is different from what DOD  uses.

		FMS Price and Availability Process

A single DOD entity may not have full responsibility for all aspects of  responding to a foreign customer’s request to purchase U.S. defense  items and services. Under DSCA policy, FMS procurements must  generally be managed at “no cost” or “no profit” to the U.S. government.  DOD’s work related to developing price and availability data and other  FMS operations is generally paid for through the administrative charges  collected from foreign customers. Depending on the complexity of the  customer’s request, coordination within and across DOD components  may be necessary to obtain complete information on pricing and  availability. DOD may also need to coordinate with defense contractors  who ultimately develop and provide the equipment or services.
The FMS process generally begins when a foreign government submits a  letter of request to the Department of State or DOD to purchase defense  articles or services. In the letter of request, the foreign customer may  express interest in obtaining preliminary price and availability data for the  capabilities it seeks. While DOD describes price and availability data as  rough order of magnitude estimates, DSCA’s guidance does not define  the precision of these estimates. According to DOD, FMS price and  availability data are non-binding estimates for the defense items and  services and are not intended to be budget-quality estimates. Requests  for price and availability data can signal to DOD and defense contractors  the potential for future sales. DOD and contractors may also draw upon  these requests to forecast staffing needs and production line availability.
DOD security cooperation organizations working in U.S. embassies  around the world can assist potential customers with defining and refining  their requirements prior to submitting a request for price and availability  data. The security cooperation organizations engage in this early  coordination to help customers articulate their capability needs. This early  coordination also gives DOD components advance notice of upcoming  requests so they can initiate technology security and foreign disclosure  processes for the timely release of information.
Requests for price and availability data represent an optional step in the  process. Customers may forgo the price and availability process and  instead submit a formal assistance request for a letter of offer and  acceptance, which when signed by the customer and U.S. government  becomes an executable FMS case. Figure 1 illustrates where the option  to request price and availability occurs in the overall FMS process.

		DOD Is Reconsidering Options to Implement Recent Legislative Change for FMS Price and Availability Process

The National Defense Authorization Act for Fiscal Year 2017 required  DOD to establish a process for defense contractors to provide input on  any differences regarding the appropriateness of government price and  availability data prior to delivery of formal responses to customers. In  response, DSCA issued a policy memorandum in October 2018 that was  rescinded 2 months later due to concerns about the sensitivity of  information to be shared with contractors. The policy memorandum had  instructed DOD components to formally request rough order of magnitude  estimates from the prime defense contractor if (1) the total value of the  primary article or service requested exceeds $50 million, and (2) the  customer has a preference for a non-competitive sole source acquisition  or only a single source exists for the primary defense item. Additionally,  the memorandum stated that DOD components will allow the prime  contractor 5 business days to provide feedback on the appropriateness of  the estimate for its items that is included in the price and availability  response prior to the customer receiving this response. The  memorandum had established a formal process to obtain contractor  feedback and resolve issues that may occur, such as differences between  the program office’s and prime contractors’ estimates, and emphasized  the importance of being aware of program deadlines when following the  process to coordinate with contractors. According to a DSCA official, this  new policy would have helped alleviate industry concerns about how  DOD incorporates estimates provided by industry to develop price and  availability responses provided to foreign customers. However, according  to DSCA officials, when implementing the process, DOD found that the  potential level of detail and precision in price and availability estimates  could provide an unfair competitive advantage to contractors coordinating  with DOD on price and availability responses to foreign customers. As  discussed in further detail later in the report, in some instances we found  price and availability estimates DOD offered included more precise  information than rough order of magnitude estimates. According to DSCA  officials, such information could offer the contractor insight into the  government’s pricing methodologies. DSCA subsequently rescinded the  October 2018 policy memorandum. DSCA plans to conduct a 120-day  review to reassess options to find a solution, if any, on what information  can be shared with contractors to satisfy the legal requirement to obtain  contractor input and feedback on price and availability estimates before  DOD responds to customers.

	DOD Received about 3,000 Requests for Price and Availability Data over the Past 5 Years

From fiscal years 2014 through 2018, DOD reported receiving 3,038  requests for price and availability data from foreign customers from 93  countries and the North Atlantic Treaty Organization. Foreign customer  requests included services and items such as training and support  services for weapon systems, missiles and ammunition, aircraft, and  communication equipment. We found that most requests came from the  same foreign customers. Specifically, 10 customers accounted for 56  percent of requests, with one customer accounting for 28 percent of all  requests during the 5-year period within our review. Customers in the  Indo-Pacific region accounted for the largest share of requests, as shown  in figure 2.
Among DOD components, the military departments—Army, Navy, and Air  Force—received almost all price and availability requests, as shown in  figure 3. The Army received slightly more requests over the 5-year period,  closely followed by the Navy.
Foreign customers we obtained information from noted that they request  price and availability data to inform their acquisition strategy, obtain a  sense of affordability, and for budget planning. For example, when  considering potential acquisition strategies, some customers may request  data for different options, variants, or quantities of similar items or  services, resulting in multiple requests for price and availability data to  inform a potential purchase. In cases when a customer is interested in  procuring a specific item, the customer may request data to obtain  information about prices and lead times to determine affordability. The  customer may also request the data when considering whether to  purchase from the United States or from foreign countries.
Requesting price and availability data can also provide foreign customers  with information on whether the U.S. government will make the requested  defense item or service available for sale. While preliminary estimates are  not an official acknowledgement that the item or service will be made  available to the customer, the request can trigger a U.S. government  review that includes application of policies that govern the release of  certain technologies or systems and a discussion with the customer about  the item or service. In some cases, customers can receive responses with  partial information if some requested items are not available for release.
DOD does not collect data on which customers’ requests for price and  availability data resulted in a formal request to purchase defense items or  services under FMS. Army security assistance officials told us it can take  years between when price and availability data are provided and when a  customer submits a request for a letter of offer and acceptance, if at all.  For their part, customers we obtained information from noted that there  may be numerous reasons for why they might choose not to pursue a  potential sale. For example, the item or service could not be made  available within a timeframe to meet their needs; the overall capability  was not affordable; or price and availability estimates were higher than  estimates from other foreign sources.
The military departments do not consistently track information on the  status of responses sent to foreign customers. We found the Navy and  Army generally captured the status of a response in the system,  identifying when a response is in development, has been sent to the  customer, or has been canceled but, according to security assistance  officials, this information may not be entered consistently. In addition, the  Air Force does not generally update the status of a response in the  system. Further, Air Force security assistance officials told us the  department does not update data in the system to reflect that the Air  Force provided price and availability data to the customer. According to  DSCA and military department officials, there is no requirement that DOD  components record when a response is sent to a customer. A DSCA  official told us that DSCA does not have a specific need to monitor the  status of price and availability responses, in part because these are not  formal offers, and DOD prioritizes data collection for formal FMS cases— cases for which a signed agreement between the U.S. government and  foreign customer is in place.

	DOD’s Guidance Allows for Flexibility in Developing Price and Availability Data and Reflects Leading Practices for Using Quality Information

DSCA has established DOD-wide guidance—the Security Assistance  Management Manual—for responding to foreign customers’ requests for  information on defense items and services available for purchase through  the FMS program. The manual includes some guidance on developing,  documenting, and communicating price and availability data to foreign  customers, but largely pertains to a customer’s request for a letter of offer  and acceptance with the intent to buy. Security assistance officials from  across the military departments told us they rely on the manual to guide  their efforts throughout the price and availability process, and that  DSCA’s guidance provides a framework for the process and is not always  prescriptive, allowing military departments latitude in how they implement  it. DSCA and military department officials we spoke with said that a  flexible process is needed to account for various circumstances specific  to each request. The price and availability process outlined in guidance  and described by DSCA and military department officials involves input  from numerous organizations within and external to DOD, as shown in  figure 4. The guidance states the process should be completed within 45  days.
Generally, we found that DSCA’s guidance reflected attributes conducive  to using quality information as called for by federal internal control  standards. For example, the standards call for agencies to define  information requirements and obtain relevant data from reliable sources.  DOD’s guidance reflects this, stating that price and availability data  should serve as rough order of magnitude estimates of the cost and  availability of defense items or services and are for rough-order planning  purposes. The guidance also  instructs officials to assess whether a foreign customer’s request  contains the necessary information to develop price and availability  data, such as the major item or service, quantity, anticipated delivery  schedule, and other specifications;  suggests that price and availability data also provide customers with  information about costs for not only buying equipment but also the  related operation and sustainment costs; assumes responses will include standard items—nonstandard items  identifies relevant data sources that the military departments can  consult to develop price and availability data, such as last contract  award, stock price, or information from defense contractors;  states that military departments and DSCA should use the Defense  Security Assistance Management System to prepare responses to  price and availability requests;  suggests that data should be itemized by separating main equipment  from training, technical publication, transportation costs, and other  elements, as applicable; and  states that responses should be developed and communicated to  customers within 45 days from when DOD receives the request.

	In Selected Examples, DOD Included Comprehensive Data on Ownership Costs When Developing Price and Availability Responses

When selling defense items and services to foreign customers, military  department officials indicated that they strive to offer a complete and  sustainable capability, referred to as the total package approach. Using  this approach, DOD takes into account the related support, such as  training, logistics, spare parts, warranties, contractor support, and other  considerations necessary for operating and sustaining the defense items  or services being purchased. The total package approach represents the  initial and follow-on cost of owning and supporting the capability. For  example, a DOD program official may develop a cost estimate for the  capability, including several years of technical support for maintaining it.  DOD may also provide a customer with cost estimates for maintaining the  capability over the course of its expected lifetime.
Specifically, in the five examples we reviewed, we found that DOD  officials generally used a total package approach when developing price  and availability data. For example, military department officials developed  price and availability data that not only included the items and services  requested by the customer, but also included rough order of magnitude  estimates for additional costs to reflect the expected ownership costs.  Ownership costs may include development, procurement, operation, and  sustainment costs for the defense item, as part of a total package  approach. The timeframe of ownership costs provided may vary.  According to a DSCA official, ownership costs generally cover the first 2  years. In four of the five cases we reviewed, the customer requested a  capability and, in response, the program office provided estimates for not  only the equipment but also the support needed to achieve the desired  capability ranging from one week of training to five years of technical  support. For example, in one case, a customer requested data for a  complex naval weapon system that they had not previously used. Navy  program officials provided estimates for the system, spare parts, training,  and other items as requested by the customer. Program officials also  included estimates on radio navigation equipment and software that are  essential for the system to function as intended, but were not part of the  customer’s initial request. Officials stated that they included these  additional costs to give the customer a comprehensive view of the costs  to acquire, operate, and maintain the weapon system. In the fifth case,  program officials told us they did not have to include training or support as  this customer was replacing missiles in their inventory, previously  purchased through FMS. However, in considering the foreign customer’s  ownership costs, the officials said they included costs for containers for  storing the missiles.
For the selected examples, program officials obtained data from defense  contractors and previous sales, adjusting estimates from data sources to  ensure the price and availability estimate reflected what the customer  could expect to pay for the item or service—initial and follow-on cost of  owning and supporting the capability—if the customer decided to proceed  with the purchase. Defense contractors responsible for providing data for  four of the five examples told us they consider the quantity and specific  requirements of the request, such as training, spares, and support; as  well as inflation and anticipated production and delivery schedules in  some cases. We found that for the selected examples program officials  adjusted estimates from contractors and other data sources for a number  of reasons, such as to account for potential changes in production  schedules and adding program management support provided by the  U.S. government to administer system upgrades. By accounting for these  likely costs, program officials stated that they were providing the customer  with estimates that would more closely reflect expected costs if the  customer proceeded with the sale. For example:  In two of the responses we reviewed for missiles and communication  systems, Navy and Air Force officials increased contractors’  estimates, in part, to account for possible changes to production  plans. In the Navy response, for example, program officials increased  the contractor’s estimate for the missiles by approximately 14 percent.  Officials told us this was to account for possible changes in the  production schedule and quantity. Contractor representatives told us  that their estimate was based on a specific number of missiles being  produced in a certain production lot. Program officials told us that the  customer would not likely have a signed agreement in place to receive  missiles from that specific production lot. According to program  officials, this means the price per missile could be higher than  forecasted in the contractor’s initial estimate because there may be  fewer quantities in production, resulting in fixed production costs  spread among fewer missiles.
In an Army response we reviewed for non-standard upgrades to  several hundred tanks, the program official used estimates provided  by the contractor to develop the price and availability data. These tank  upgrades are considered non-standard because the U.S. government  no longer uses these tanks. In light of this, the program official  included costs for program management support provided by the U.S.  government because he said the magnitude of the program would  likely require an Army office to execute and manage the upgrades,  which is projected to last up to 10 years.
In an Air Force response we reviewed for a warning system, program  officials considered historical data from similar DOD contracts. The  program officials increased the price by $2.4 million dollars from past  procurements based on the customer’s request to add a new full-time  onsite engineer to support the warning system. This price also  included costs for housing, living allowance, and travel expenses.
Further, in our review of selected cases, we found that program officials  may include other charges in price and availability data, such as nonrecurring costs that are unique one-time program-wide  expenditures for certain major defense equipment sold under the FMS  program; a contract administration charge—generally, 1.2 percent of the value  of procured items—for services such as quality assurance and  inspection;  transportation costs for delivery of the item, which are generally  calculated based on rates established by DSCA; and an administrative charge—currently set at 3.2 percent of the total  value of the sale to recover civilian employee salaries and operational  costs for administering the FMS acquisition.

	Various Factors Can Influence DOD’s Approach and the Timeliness of DOD’s Responses

Military department officials told us that various factors influence the level  of effort and information involved in developing price and availability data,  some of which may also affect how long a response takes and whether  the 45-day timeframe suggested by DSCA’s guidance is achieved. When  a foreign customer requests price and availability data, DOD and defense  contractors, if involved, expend time and resources to provide a  response, all without any certainty that a sale will materialize. As such,  DOD officials and defense contractors determine what level of response  is appropriate, given the nature of the customer’s request and whether it  includes non-standard items or items that require customization, among  other things. DSCA and Navy program officials said customers are  interested in receiving price and availability responses quickly and  recognize that timeliness is an area of concern with the FMS process, in  general. Over half of the 12 foreign customers we obtained information  from noted that they are concerned with the length of time DOD’s  responses can take. Lengthy response times could result in customers  missing opportunities to consider potential requests in upcoming budget  cycles. Several customers communicated that some responses took  considerably longer than 45 days, with some taking anywhere from 6 to  12 months. Among the five examples we reviewed, responses took from  45 to 320 days, as shown in table 1.
Program and security assistance officials we interviewed told us they  consider the following factors:
Customer interest and commitment. Insight into the degree of  customer commitment to purchase through FMS may influence the  time and resources military departments expend on developing a  price and availability response. For example, Air Force security  assistance officials told us that they may develop a more detailed  response if advised by in-country personnel that a request for price  and availability data will likely become a request for an actual  purchase.
Clarity and completeness of customer’s request. Customers may  submit requests that lack the clarity and details needed to develop  accurate data and estimate delivery timeframes. Several military  department officials told us that when reviewing the customer’s  requests for price and availability data, they often have discussions  with customers to clarify requirements and in some cases estimated  delivery schedules before developing a response. Defining the  customer’s requirement—even at this early stage—can be an iterative  process that requires multiple interactions between the foreign  customer and DOD officials. In one of the examples we reviewed, the  defense contractor was also involved. These discussions to clarify the  customer’s requirements can prolong the process, according to  several program officials.
Existing policy to release price and availability data. The U.S.  government’s relationship with the foreign customer and the type of  defense item or service being requested—such as a weapon system  with protected critical technologies versus medical evacuation  equipment—can influence the length of time to obtain necessary  approvals for the release of price and availability data, according to  Navy program and Air Force security assistance officials. Requests  for price and availability data may spur the U.S. government to review  the current list of countries that have access to particular critical  technologies, as shown in one Navy response to a request for a  ballistic missile defense system. Initially, the Navy’s Foreign  Disclosure Office determined the system would not be available for  potential release and the Navy program office excluded it from the  price and availability data. About a year later, according to Navy  officials, following a change in U.S. policy, the Foreign Disclosure  Office approved the release of price and availability data for the  system and the Navy included it in a subsequent price and availability  response.
Complexity of the request. Requests for a non-standard system,  integration with foreign components, or a complex system may cause  program offices to spend additional resources and time to develop  price and availability data. For example, in response to a request for a  complex weapon system to be integrated into a foreign customer’s  ship, Navy program officials said that they needed several months to  develop price and availability data due to the complexity of this  request, which required program officials to work with multiple  contractors and DOD entities to develop price and availability data. In  contrast, Army security assistance officials said that they generally  aim to conserve resources and time by developing price and  availability data based on standard items, even in instances when  customers may request non-standard or complex systems.
Existing workload. The volume of requests and competing priorities  can also affect the timeliness and the level of effort applied to the  response. For example, Army security assistance officials stated that  they may prioritize a customer’s request for a letter of offer and  acceptance, which initiates an executable FMS case, over a request  for price and availability data because there are not resources  available to do both at the same time.
Availability of requested item or service. When obtaining the items  from defense contractors, for example, military department officials  consider production schedule and quantity—both of which require  additional assumptions to estimate unknown costs. For items that are  in DOD’s inventory and will not be replaced, officials are to take into  account the item’s actual value when developing price and availability  data, according to a DSCA publication.
External factors. In cases where a customer is requesting price and  availability data to decide whether to purchase defense items or  services from the United States or another foreign government,  military departments may expend additional resources to develop  detailed price and availability data. For example, a Navy security  assistance official stated that when officials are aware the customer  plans to hold competitions between U.S. and foreign defense  contractors, they solicit more detailed technical and cost information  from defense contractors to present a competitive estimate.
Individually and combined, these factors, as well as the overall process,  can influence response times.

	Agency Comments

We provided a draft of this report to the Department of Defense (DOD) for  comment. DOD’s response letter is reproduced in appendix II. DOD  separately provided technical comments, which we incorporated as  appropriate.
We are sending copies of this report to the appropriate congressional  committees and the Acting Secretary of Defense. In addition, the report is  available at no charge on the GAO website at http://www.gao.gov.
If you or your staff have any questions about this report, please contact  me at (202) 512-4841 or makm@gao.gov. Contact points for our Offices  of Congressional Relations and Public Affairs may be found on the last  page of this report. GAO staff who made key contributions to this report  are listed in appendix III.

Appendix I: Objectives, Scope, and Methodology

In this report, we (1) described foreign military sales (FMS) price and  availability requests Department of Defense (DOD) received from fiscal  years 2014 through 2018, (2) assessed DOD’s guidance on developing  price and availability data, (3) described how DOD develops price and  availability data for the requested capability, and (4) identified factors that  can influence the timeliness for DOD to provide price and availability data  to the customer.
To describe requests for price and availability data DOD received from  foreign customers, we analyzed data from the Defense Security  Cooperation Agency (DSCA). We reviewed data for fiscal years 2014  through 2018, the most recent 5-year period available. DSCA and other  DOD components, including the military departments, use the Defense  Security Assistance Management System as a workflow resource to  process price and availability data requests, among other things. The  system does not track which of the estimates result in a letter of offer and  acceptance. To assess the reliability of Defense Security Assistance  Management System data, we tested for missing data, duplicates,  inconsistent coding, and compared data for five examples to price and  availability documentation we received from the Army, Navy, and Air  Force. We interviewed DSCA officials responsible for the data system to  identify the quality controls in place to help ensure the data are accurate  and reliable and discussed military department practices for using the  system with security assistance officials. We found that generally the  documentation for the five selected preliminary estimates matched the  data DSCA provided and requests matched across multiple datasets we  received from DSCA. Based on these steps, we determined the data  were sufficiently reliable to report examples of the types of items and  services requested and the number of requests DOD received by region,  DOD component, and foreign customer. We did not report the number of  responses DOD provided for these requests or how long it took DOD to  provide a response to foreign customers using this data because military  departments do not consistently update information in the Defense  Security Assistance Management System to track the status of responses  or dates when a response is provided to the customer.
To assess available guidance, we reviewed DSCA and Army, Navy, and  Air Force guidance for developing preliminary estimates in response to  requests for price and availability data. We compared the DOD-wide  guidance—the Security Assistance Management Manual—to the  Standards for Internal Control in the Federal Government, which call for  agencies to use quality information collected from relevant and reliable  sources. Specifically, we reviewed the guidance to determine if it  contained attributes that contribute to quality information such as  identifying the information requirements and relevant data sources  needed to develop the price and availability data.
To describe factors that DOD considers when developing price and  availability data and illustrate how these factors influence the process, we  selected a non-generalizable sample of five responses from fiscal year  2017 data provided by the military departments. Fiscal year 2017  represented the last complete year of data available when we selected  this sample. Because the sample is not generalizable, we cannot report  whether practices used among the responses are used across DOD for  all price and availability responses. However, these examples provide  useful insight into the process and the assumptions used when  developing price and availability data. We selected the five examples— one from Army, two from Navy, two from Air Force—to obtain a variety of  responses, including median and large case values and a median  response time. We determined there were inconsistencies in the data  provided, but that the data were sufficient for our purposes of selecting a  non-generalizable sample from across the military departments.
For each selected example, we collected and analyzed the letter of  request, price and availability data, DOD’s response to the customer,  supporting documentation if provided such as clarification of the  customer’s request, and data collected from defense contractors or  program offices. We reviewed the assumptions and factors used in  developing the data and the various elements that make up the data,  such as administrative charges and costs for training and spares. We  interviewed relevant DOD security assistance and program officials, and  defense contractor representatives to understand the context and  decisions made in developing, documenting, and communicating the  price and availability data.
To identify the factors that can influence the timeliness of responses, we  interviewed officials from DSCA and the Army, Navy, and Air Force. We  also obtained information from defense contractors and foreign customers  who, as stakeholders in the FMS price and availability process, have  broad insights and perspectives on the process. To gather input from  foreign customers, we interviewed representatives from the Foreign  Procurement Group who also solicited information from its consortium of  46 member countries on our behalf. We received responses from 12  countries—one of which was also a customer for one of the examples  included in our review. To obtain contractor’s perspectives, we gathered  information from five companies through interviews and attended a  meeting hosted by the National Defense Industrial Association. Three of  the companies we obtained information from were involved in providing  cost and schedule data for four of the examples in our sample. The  information we obtained from these foreign customers and defense  contractors is not generalizable to all foreign customers and defense  contractors. As mentioned previously, we did not assess the timeliness of  DOD’s responses because DOD does not consistently track when price  and availability data responses are provided to customers in the Defense  Security Assistance Management System. However, the information we  gathered for the five examples in our sample provided some insight about  how long it took DOD to provide a response to the customer.
We conducted this performance audit from June 2018 to February 2019  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix II: Comments from the Department of Defense

Appendix III: GAO Contact and Staff Acknowledgments

	GAO Contact

	Staff Acknowledgments

In addition to the contact named above, Candice Wright (Assistant  Director) and Leslie Ashton (Analyst-in-Charge) managed this review.  Bruna Oliveira, Carmen Yeung, Kurt Gurka, Robin Wilson, and Emily  Bond made significant contributions to the work.

Output:
Why GAO Did This Study

DOD manages the procurement of billions of dollars in defense items and services on behalf of foreign customers through the FMS program. These sales help support the defense industrial base and are vital to U.S. foreign policy and national security interests.
The FMS process generally begins with a request by a foreign government for information about a U.S. defense item or service. Requests for price and availability data are an optional step in the process. DOD guidance is to generally respond to such requests within 45 days.
The fiscal year 2018 National Defense Authorization Act included a provision for GAO to review DOD's process for developing price and availability data for foreign customers. This report addresses, among other objectives, (1) price and availability requests DOD received from fiscal years 2014 through 2018, (2) how DOD develops price and availability data, and (3) the factors that can influence the timeliness of DOD's responses to foreign customers with price and availability data.
GAO analyzed DOD price and availability data for fiscal years 2014 through 2018, the latest data available; and reviewed documents for a non-generalizable sample of five price and availability responses—varying by estimate value—provided to foreign customers by the Army, Navy, and Air Force. GAO also interviewed defense contractors and DOD officials.
GAO is not making any recommendations at this time.

What GAO Found

The Department of Defense (DOD) reported receiving 3,038 requests for Foreign Military Sales (FMS) price and availability data in fiscal years 2014 through 2018 from 93 countries across six geographic regions, as shown in the figure. Foreign customer requests included services and items such as training and support services for weapon systems, missiles, aircraft, and communication equipment.
Not all countries in each region submitted a price and availability request.
DOD officials indicated they generally strove to offer price and availability data that reflected rough order of magnitude estimates of total anticipated costs for a complete and sustainable capability. Contractors often provide input to DOD for these cost and schedule estimates. In the five examples GAO reviewed, DOD officials considered factors such as possible production delays and included anticipated costs for support services, operations, and sustainment, when needed. DOD officials also included FMS administrative charges and, as applicable, nonrecurring and transportation costs. GAO found that when DOD considered these factors in developing the response to the customer, at times, they made adjustments to the estimates provided by contractors to more fully reflect expected costs if the items are purchased.
Among the five examples, GAO found that response times ranged from 45 to 320 days and that a number of factors can affect timeliness. For example, the complexity of the system or capability the customer is interested in acquiring may require involvement from multiple program offices and defense contractors, requiring more time than the 45 days suggested by DOD's guidance.