Input:
	Background

TSA is responsible for implementing and overseeing security operations  at roughly 440 commercial airports as part of its mission to protect the  nation’s civil aviation system.

		Screening Technologies

TSA is responsible for ensuring that all passengers, their carry-on bags,  and their checked baggage are screened to detect and deter the  smuggling of prohibited items, such as explosives, into the sterile areas of  airports and onto aircraft. Agency procedures generally provide that  passengers pass through security checkpoints where their person,  identification documents, and carry-on bags are screened by  transportation security officers (TSO). TSA uses a variety of screening  technologies—screening systems, as well as software and hardware for  those systems—to carry out its mission. Figure 1 depicts the various  screening technologies a passenger may encounter in primary and  secondary security screening.

		Process for Acquiring and Deploying Screening Technologies

TSA develops detection standards that identify and describe the  prohibited items—such as guns, knives, military explosives, and  homemade explosives—that each technology is to detect during the  screening process. The standards, which are classified, also identify  how often the technology should detect prohibited items (referred to as  the required probability of detection) and the maximum rate at which the  technology incorrectly identifies prohibited items (the probability of false  alarm). For explosive materials, the standards also identify what the  screening technology is to be able to detect in terms of (1) the minimum  amount or weight of the material (the minimum detection mass) and (2)  the chemical and physical makeup of the material (density range of the  explosive material).
S&T supports TSA in the development of standards by, among other  things, analyzing the characteristics (threat mass, or the amount of  material that constitutes a threat, and density) of explosive materials. The  agency uses the resulting data to develop detection standards that are  specific to each screening technology.
After a detection standard is approved, TSA decides whether to  operationalize—put into effect—detection standards by acquiring and  deploying technologies to update detection capabilities to meet the  standard. That is, it decides whether to take steps to develop new  technology capable of meeting the standard and put the new technology  in place at commercial airports. Technology can mean new software to  upgrade existing screening systems as well as entirely new screening  systems. TSA does not always or immediately operationalize detection  standards, for reasons which are explained later in this report.
To operationalize a detection standard, TSA must acquire technology  capable of meeting the standard. TSA officials told us they follow DHS  acquisition policies and procedures when acquiring new screening  technologies. Officials said they adapt detection standards as detection  requirements to guide the acquisition process, meaning the specifications  described in the standards are incorporated into the requirements  manufacturers must meet when developing new technology. Once  manufacturers have developed new technologies that meet detection  requirements, the technologies undergo a test and evaluation process,  known as the qualification process. The following are key steps in that  process:  1.  Certification – Certification is a preliminary step in TSA’s qualification  process. For TSA to certify that a screening technology meets its  detection requirements, S&T’s Transportation Security Laboratory  conducts certification testing on a manufacturer’s initial submission of  its proposed screening technology to determine whether it meets  TSA’s detection requirements (i.e., the rate at which it must accurately  detect each category of explosive it is designed to detect, among  other things).  2.  Integration/Implementation Testing – TSA Systems Integration Facility  administers qualification testing to test system performance against  additional requirements, such as reliability, availability, and  maintainability. TSA also conducts field testing to ensure readiness for  operational test and evaluation.  3.  Operational Test and Evaluation - TSA deploys units to selected  airports to conduct operational testing. Operational testing allows TSA  to evaluate the operational effectiveness, suitability, and cyber  resiliency of the technology in a realistic environment.
After new technologies have been tested and approved, TSA can  purchase and deploy them to commercial airports. When a deployed  screening system can no longer be updated to meet new detection  standards, TSA considers it obsolete and generally designates it for  replacement with a newer version of the technology.
Figure 2 shows TSA’s process for acquiring and deploying new screening  technologies to meet detection standards.

		DHS Risk Management

DHS guidance provides that its components, including TSA, use risk  information about security threats and analysis to inform decision-making.  Risk management helps decision makers identify and evaluate potential  risks so that actions can be taken to mitigate them. DHS defines a risk  assessment as a function of threat, vulnerability, and consequence.  DHS guidance also says that risk assessments and transparency are key  elements of effective homeland security risk management.

	TSA Has a Process for Developing Detection Standards, but Has Not Updated Its Guidance or Documented Key Decisions

		TSA Has Consistently Followed Testing Protocols in Developing Detection Standards

TSA has a process to develop new explosives detection standard in  response to emerging, credible threats involving a homemade explosive  (see sidebar for more information on homemade explosives). According  to TSA officials, the first step in the process is to determine whether a  new detection standard is needed, which they do by working with S&T  and other federal partners to ”characterize” the threat material—that is,  identify the chemical and physical properties of the material, such as the  threat mass and density. Below is the process (steps) TSA and S&T  officials told us they use to characterize a threat material and determine  whether a new detection standard is needed.
Homemade Explosives   Homemade explosives are designed to cause  destruction when used in improvised  explosive devices. The picture below shows  damage to an aircraft panel from a homemade  explosive. Beginning in the early 2000s,  homemade explosives replaced military and  conventional explosives as the preferred tool  of terrorists, and challenged the capabilities of  existing screening technologies. Unlike  conventional threats, homemade explosives  are often made of common commercial items  and it can be challenging to distinguish them  from innocuous gels and liquids stored in  personal baggage or cargo. They also have  different detonation patterns from  conventional explosives in that they often  release energy much more slowly, which may  lead to incomplete or delayed detonation. This  pattern is not well understood, which makes it  much more difficult to predict the resulting  damage. of the explosive—the minimum amount of the material that constitutes a  threat to civil aviation.
Material down selection (selection of possible mixtures for testing).  Because the exact formulation of the explosive can vary, S&T must test  and model various formulations in different proportions to gain an  understanding of the homemade explosive. In this step, TSA determines  the representative formulations and preparations that are to be prepared  and tested, based on data provided by S&T.
Synthesis, formulation, and preparation of materials. S&T establishes  how the threat material could be made, including its chemical synthesis  (as applicable), possible formulations or mixtures of the material with  other components, and the preparation of those mixtures. S&T uses this  information to develop samples of the threat material for testing.
Data acquisition and analysis. S&T examines the samples using micro- computed tomography and explosives detection system, and the resulting  data are sent to S&T’s Transportation Security Laboratory for verification.  The verified data are then sent to the U.S. Department of Energy’s  Lawrence Livermore National Laboratory for analysis.
The Transportation Security Administration  and the Science and Technology Directorate  have ranked 300 conventional and homemade  explosives that pose the most likely threat to  aviation security based on factors such as  availability, stability, performance, and method  of initiation.
Region of responsibility. Lawrence Livermore National Laboratory  generates preliminary results in the form of the “region of responsibility,”  which is a map or explosive detection “window” outlining the  characteristics of the threat material in terms of density and effective  atomic number. These preliminary results are discussed among TSA and  S&T stakeholders, with TSA determining the final region of responsibility.  The region of responsibility data are used to develop software algorithms  that will allow screening technologies to recognize explosive materials  whose characteristics fall within the region of responsibility.
Detection standard. TSA and S&T also use the region of responsibility  data to determine whether the explosive material can already be detected  by deployed screening technologies. If screening technologies can  already detect the material, TSA will not contract with technology  manufacturers to develop a new software algorithm or screening  technology. But regardless of whether a new software algorithm or new  technology is needed, TSA will draft a new detection standard for the  material that, generally, will specify the minimum threat mass and density  range to be detected, the acceptable probability of detection, and  probability of false alarm. The draft standard is reviewed by TSA senior  management before being approved.
We found that the work S&T and other stakeholders performed to  characterize explosive threat materials was consistent across the threat  materials. Specifically, we found that S&T consistently followed the  process described to us (as outlined above) for characterizing a threat  material in the seven material threat assessments we reviewed. We  also reviewed documentation regarding additional testing and analysis  S&T performed on select threat materials, and found the additional testing  and analyses were performed consistently.

		TSA Has Not Updated Its Guidance for Developing Detection Standards to Reflect Required Procedures, Key Stakeholder Roles, and New Organizational Structure

TSA has not updated its 2015 guidance for developing new detection  standards to reflect key changes in their procedures. In December 2015,  TSA issued the Detection Requirements Update Standard Operating  Procedure, which a senior official told us served as the agency’s  approved guidance for developing detection standards. Our review of  the document found that, as of August 2019, it did not accurately reflect  (1) designated procedures for developing detection standards, (2) the  roles and responsibilities of key stakeholders such as S&T, and (3) TSA’s  organizational structure. For example, one way in which the 2015  guidance has not been updated is in the designated procedures it  describes for reviewing available intelligence information. Specifically, the  guidance calls for an annual assessment of emerging threats, which a  senior TSA official told us TSA no longer conducts because relevant  emerging threats are now occurring more frequently and intelligence  information is processed on an ongoing basis.
In another example, the guidance specifies that TSA will form working  groups composed of agency officials and stakeholders to assess potential  threat materials and develop an analysis plan, and that each working  group will define the roles and responsibilities of its members. According  to a senior TSA official, the agency does not convene working groups to  assess intelligence or develop an analysis plan, although officials  regularly meet with stakeholders to discuss the steps needed to  characterize new threat materials and document the minutes from these  meetings.
Finally, while the guidance discusses in detail which TSA offices and  management positions are responsible for implementing and overseeing  the process, the agency has since reorganized and these offices and  positions no longer exist. Therefore, the 2015 guidance is no longer  relevant in terms of which offices and positions are responsible for  implementing and overseeing the approval of detection standards.
Officials told us that, as of August 2019, they had begun revising the  guidance to reflect existing standard operating procedures for developing  detection standards, but had yet to finalize a draft of the new guidance or  document plans or timeframes for completing and approving it. Further, it  is not clear to what extent the revised guidance will address designated  procedures for developing detection standards, the key roles and  responsibilities of stakeholders, and TSA’s new organizational structure.  Officials said they had not updated the guidance earlier because both  TSA and S&T had been undergoing agency reorganizations.
Standards for Internal Control in the Federal Government provides that  agencies should identify, on a timely basis, significant changes to internal  conditions that have already occurred, such as changes in programs or  activities, oversight structure, and organizational structure. Additionally,  agencies are to develop and maintain documentation of internal controls,  such as policies and procedures necessary to achieve objectives and  address related risks. By documenting the processes and procedures  TSA uses to develop detection standards, clarifying the roles and  responsibilities of stakeholders, and documenting organizational changes,  TSA could have better assurance that detection standards are developed  in accordance with established policies and practices.

		TSA and S&T Did Not Document All Key Decisions Regarding the Development of Detection Standards

Our review of TSA’s steps to develop detection standards from fiscal  years 2014 through 2018 found that TSA and S&T did not document all  key decisions—those that could potentially affect outcomes—regarding  the testing and analyses (characterization) of explosive threat materials  and the development of explosives detection standards. We found that  TSA and S&T produced a series of detailed material threat assessments  to document the characterization of threat materials and consistently  developed action memos to justify proposed detection standards.  However, we also found that in five of the seven material threat  assessments we reviewed TSA and S&T did not consistently document  key steps in the testing and analyses of materials, such as how selected  samples were prepared for testing. For example, one S&T material threat  assessment we reviewed did not document the method used to  synthesize (chemically produce) material samples used for testing. Not  documenting the method could prevent officials from fully understanding  the results of the analysis. Specifically, the assessment noted that there  are multiple methods of synthesis, and that the chosen method could  affect the makeup of the resulting material and therefore the ability of the  screening technologies to detect it. Additionally, while two of the seven  material threat assessments cited standard operating procedures for  sample preparation for all participating laboratories, three did not cite  standard operating procedures for at least one laboratory and two stated  that sample preparation information had not been provided by one or  more of the participating laboratories. Without documentation, TSA might  not have all the necessary information to address future issues involving  detection of these materials.
We also found four instances in which TSA did not clearly document why  select materials were sent for additional testing or did not document key  decisions regarding the development and consideration of detection  standards. For example, S&T performed additional testing and analysis  on select threat materials after the material threat assessment was  finalized. However, the documentation of this additional testing left out  key elements regarding how and why the additional testing was needed  and conducted. The action memo documenting new standards based on  the results of the additional testing did not include a justification for why  specific threat materials were selected for additional data collection.  While a test plan for equivalency testing of one material stated that the  additional testing was conducted because data reported in the literature  were not considered representative of current threat configurations,  similar justification was not included in the action memo justifying  changes to the new standard based on the additional testing. Finally, a  senior TSA official told us he requested the additional equivalency testing  because the values in the previous detection standards appeared to be  more conservative than expected and there was no documentation  explaining how TSA had arrived at those numbers. According to the  official, the previous detection standard was approved before his tenure  and the determining officials were no longer with TSA. He also stated that  he did not know whether TSA required documentation of testing and  analysis when the previous detection standard was being developed.
We found that TSA did not document key decisions regarding the  development and consideration of detection standards. For example,  officials could not provide documentation of conclusions reached on  specific key decisions, such as the consideration and decision not to  approve a proposed explosives trace detection standard. A senior TSA  official said he did not know why the decision had not been documented  because the officials involved were no longer with the agency.
According to Standards for Internal Control in the Federal Government,  documentation is required for the effective design, implementation, and  operating effectiveness of an agency. Documentation also provides a  means to retain organizational knowledge and mitigate the risk of having  that knowledge limited to a few personnel, as well as a means to  communicate that knowledge as needed to external parties. By  documenting key decisions regarding the development of detection  standards, including instances in which draft standards are not approved,  TSA could better ensure that effective decisions are made and that  organizational knowledge is retained regardless of changes in personnel.

	TSA Operationalizes Detection Standards by Updating its Screening Technologies, Which Can Take Years to Complete

TSA officials said one way to operationalize detection standards—acquire  and deploy technologies to update detection capabilities and meet the  detection standard—is to update existing screening systems with new  technology, such as software or firmware. When possible, the agency  installs software as part of routine maintenance. TSA can also deploy  new hardware or replace screening systems to update detection  capabilities. According to officials, the agency applies an incremental  approach to updating existing screening technologies—it updates  technologies when manufacturers are able to develop the requisite  capabilities and as resources allow—which can take years to complete.
According to officials, all fully deployed TSA screening technologies had  detection capabilities that met detection standards approved from 2006  through 2012. That is, as of August 2019, TSA’s fleet of screening  technologies met detection standards that were approved in 2012 or  earlier. For example:
Bottled liquid scanner units met a detection standard that was
Advanced technology x-ray units met two detection standards,  depending on their manufacturer, that were both approved in 2010;  and
Explosives trace detection units met a detection standard that was  approved in 2012.
Further, for each screening technology, the agency has approved two to  three new detection standards that have not been operationalized, as of  August 2019. For example, in addition to the 2006 detection standard for  bottled liquid scanner, TSA approved standards for bottled liquid scanner  in 2012 and in 2017 that have not been operationalized.
TSA officials said they were working to operationalize some of the  detection standards approved since 2012. Officials said they were  working with manufacturers to develop new technologies to operationalize  some of these standards. In other cases they were in the process of  deploying new technologies that meet these standards.  For example, as  of August 2019, TSA was in the process of updating and replacing  explosives detection systems to meet a detection standard that was  approved in 2014. Officials said they expected to have the entire fleet  updated by September 2023. TSA officials said they were also in the  process of updating deployed advanced technology x-ray units for one of  its two manufacturers to meet a standard that was approved in 2014. For  more information about the detection standards TSA had approved for  each technology as of August 2019, and the status of TSA’s progress in  operationalizing them, see appendix I.
TSA shares information about the capabilities it needs with manufacturers  through requests for proposal, requests for information, and broad agency  announcements. The agency places approved technologies on a qualified  products list—a list of technologies that have been tested and certified as  meeting requirements by TSA and DHS—and the agency can then award  a contract to one of the manufacturers to purchase and deploy the  technology. Before deploying technologies to airports, TSA conducts  testing to ensure consistency in the manufacturing process, system  configuration, and functionality following production, and then again after  the technology is installed at airports.
Our analysis of the acquisition information TSA provided found it took  from 2 to 7 years to fully develop, certify, test, and deploy screening  technologies to airports. For example, when operationalizing explosives  trace detection standard 5.0, it took one manufacturer 4 years and a  second manufacturer 7 years to develop, and for TSA to deploy, the  software needed to update the capability of existing explosives trace  detection units to meet the new standard.
Figure 3 provides our analysis of TSA’s timeline for operationalizing  advanced imaging technology detection standards approved from 2010  through 2016. TSA officials said they approved detection standard 3.3 for  advanced imaging technology in October 2010 and began deploying  technology that met that standard to airports in August 2011. Officials  said they approved a subsequent standard, 4.1, in January 2012, began  deploying technology to meet it in October 2014, and completed the  deployment in September 2017. Officials said it took 3 years to complete  deployment because the demand for advanced imaging technology  increased over time as airports experienced an increase in passenger  volumes, among other reasons. Since 2012, TSA approved two additional  detection standards for advanced imaging technology—4.3 in February  2016 and 4.3.1 in August 2016. TSA officials said they have not  operationalized these two standards because the manufacturer has not  been able to develop the requisite technology. As such, deployed  advanced imaging technology units meet standards approved in 2010  and 2012.
TSA officials stated that they do not always, or immediately,  operationalize detection standards after they are approved. They said  they make these decisions on a case-by-case basis, depending on many  factors. These include whether: (1) manufacturers have the technological  ability, (2) a new technology is in development, and (3) screening  technologies already have the capability.
Manufacturers do not have the technological ability. TSA officials said  manufacturers do not always have the technical ability to meet detection  standards. According to officials, it can be challenging for manufacturers  to develop the technology necessary to detect new threats as presented  in a detection standard, and in some cases impossible without further  research and development. For example, TSA officials said that  manufacturers have been unable to develop the requisite technology to  meet the most recent detection standards (4.3 and 4.3.1) for advanced  imaging technology. However, TSA officials said they have expanded  their research and development efforts to try to develop the technology.  TSA officials told us they plan to continue developing detection standards  irrespective of the capabilities of currently deployed technologies so that  they can focus on identifying emerging threats. The new detection  standards then serve to set expectations for manufacturers about the  capability to which they should aspire and justify research and  development necessary to realize that capability. To better manage the  difference between the capabilities of deployed technologies and the  capabilities described in detection standards, TSA officials said they are  in the process of developing a new position of Capability Manager, who  would be responsible for managing the development of mission-essential  capabilities—such as carry-on baggage screening—from start to finish.  Officials said they expect this position will help bridge the gap between  approved detection standards and the detection capabilities of deployed  screening technologies over time, because the managers will have cross- cutting oversight of the process.
A new technology is in development. Officials said that they may not  operationalize a detection standard if they expect a new type of screening  technology will replace an existing one. For example, officials said that  TSA is exploring new alarm resolution technologies—that is, screening  technologies that are used to determine whether alarms are false  positives. Officials said new alarm resolution technologies may replace  the bottled liquid scanner in the future, and therefore they have not  pursued operationalizing detection standard 2.3.
Screening technologies already have the capability. According to TSA  officials, new detection standards do not always add significant detection  capabilities. For example, officials decided not to operationalize bottled  liquid scanner detection standard 3.0 when it was approved in 2017  because the deployed units already had most of the capabilities called for  in the detection standard; TSA developed the new standard to better align  with standards for other technologies.

	TSA Deployment Decisions are Generally Based on Logistical Factors, and the Extent to Which TSA Considers Risk Is Unclear Because Decision- Making Lacks Documentation

		TSA Assesses Risks and Capability Gaps When Determining Acquisition Needs

Our review of TSA acquisition documents found that TSA considers risk  at the beginning of the screening technologies acquisition process..  Specifically, the agency considers risk in two phases—(1) a risk  assessment developed from intelligence information and modeling tools,  and (2) an annual capability analysis that analyzes and prioritizes  capability gaps and determines mitigation options. Figure 4 provides an  overview of TSA’s acquisition process for new screening technologies.
Risk assessment. TSA uses intelligence information and modeling tools,  such as the Risk and Trade Space Portfolio Analysis, to assess risk to the  aviation system. The Risk and Trade Space Portfolio Analysis was  developed in 2014 to analyze the security effectiveness of alternate  combinations of some aviation security countermeasures. Officials said a  recent example of a risk-informed deployment decision influenced by the  Risk and Trade Space Portfolio Analysis was TSA’s 2017 deployment of  141 advanced imaging technology units to category III and IV airports.  Officials said that around 2014, TSA received intelligence about a  potential terrorist threat to airports, as well as the results of covert testing  at airports that identified screening vulnerabilities. Officials said a 2014  Risk and Trade Space Portfolio Analysis also identified disparities in  screening capabilities at smaller airports. In part because of the  vulnerability identified by these three factors, as well as ongoing  conversations between TSA senior leadership, the DHS Inspector  General, and members of Congress, officials said TSA procured and  deployed additional advanced imaging technology units to some category  III and IV airports that did not have them.
Capability analysis. TSA uses the Transportation Security Capability  Analysis Process, a structured decision-making tool, to identify and  prioritize capability gaps and help direct agency resources towards  closing critical gaps to an acceptable level. When existing screening  capabilities do not fully meet TSA’s mission needs, the associated  capability gap presents a security risk. As part of the Transportation  Security Capability Analysis Process, TSA produces Capability Analysis  Reports that identify and recommend solutions to closing capability gaps.  Recommendations have included procedural changes, such as new  training for TSOs, and investments in new technology. TSA’s investment  in computed tomography technology for checkpoint screening of carry-on  baggage is an example of TSA’s implementation of the Transportation  Security Capability Analysis Process to validate capability gaps and  identify recommended courses of action. Officials said that in some cases  the agency may identify a capability gap that cannot be resolved to an  acceptable level with commercially available screening technology, in  which case it will pursue additional research and development.

		TSA’s Approach to How Risk Informs Deployment Decisions Lacks Documentation

TSA officials told us that they operate under the assumption that every  airport is a possible entry point into the aviation system for a terrorist, and  they do not consider there to be a significant difference in vulnerability  among airports when deploying screening technologies. However,  officials did not provide analysis or documentation that supported this  conclusion. Officials noted the exception to this assumption is a handful of  airports that are consistently considered to be the highest risk because of  known threats and a high volume of international travelers.
Further, officials said that if they had information about a threat to a  specific airport that would be mitigated by deploying a screening  technology, they would modify their plans for deployment accordingly.
However, TSA’s process for how it would change its deployment plans to  specific airports based on risk lacks transparency. For example, officials  said that as part of the acquisition process they have ongoing discussions  with stakeholders about their deployment strategies, including security  and intelligence officials who would inform them of any relevant risk  information. Officials said these discussions are generally informal and  not documented—it was unclear how these discussions have  incorporated information about risk in the past, and officials could not  provide an example of when risk information at specific airports had  directly influenced deployment of technologies to airports in the recent  past.
In 2018, the agency released its Transportation Security Administration  Systems Acquisition Manual, which called for deployment plans to be  written documents, and officials said they began documenting their plans  for deploying screening technologies in the last two years. TSA officials  provided us with one deployment plan—for their 2018 deployment of  explosives trace detection units—but we found that it was not transparent  about how risk was a factor in officials’ methodology for determining the  order of airports to receive the technology. The explosives trace  detection plan documented TSA’s schedule of deployment and the roles  and responsibilities of relevant stakeholders, among other things.  However, while the plan indicated that officials would coordinate with  relevant offices within the agency for information about risks that might  impact their deployment strategy, we found that the plan did not  document how risk had informed their decisions about where and how to  deploy the technology, including the assumptions, methodology, and  uncertainties considered.
Additionally, TSA officials did not document, and could not fully explain,  how risk analyses contributed to and factored into the following specific  deployment decisions.
Deployment of advanced imaging technology to smaller airports.  Officials said many factors influenced their decision to deploy advanced  imaging technology units to category III and IV airports, including  information about threats and a related 2014 risk analysis. However,  officials did not document their decisions and could not fully explain their  risk analysis, including their process for analyzing and weighing relevant  factors. According to officials, the decision was made during discussions  with senior leadership, which were risk-informed and supported by  whiteboard analyses and classified documents. Additionally, officials told  us that, for practical reasons, they deployed units to those category III  and IV airports that had the space to accommodate them, but did not  further assess the priority of deployment among the smaller airports  because they had determined that the risk was uniform and because they  planned to deploy the units within a short timeframe. Officials did not  document the risk assessment that led to this determination, and could  not explain how the three elements of risk—threat, vulnerability, and  consequence—were used or assessed.
Deployment of targeted threat algorithm. In 2016, TSA deployed a  targeted threat algorithm—software to improve detection capabilities—to  a limited number of advanced imaging technology units in response to a  specific threat. After testing the operational impacts of the software  algorithm, the agency decided to stop further deployment. The  documentation TSA provided did not explain how officials had analyzed  the risk-mitigation benefits of the algorithm, including the underlying  assumptions and uncertainty, or how they had weighed those benefits  against the operational impacts and costs when they made their decision  not to fully deploy the algorithm.
TSA officials said they follow the DHS acquisition process to acquire and  deploy technologies and their deployment decisions are based on, and  informed by, their initial assessments of capability gaps, as well as their  understanding that every airport offers equal entry into the aviation  system. However, officials had not documented the rationale for these  decisions and could not fully explain how risk had informed their decisions  about where and in what order to deploy screening technologies.
DHS’s Risk Management Fundamentals states that components should  consistently and comprehensively incorporate risk management into all  aspects of the planning and execution of their organizational missions.  Additionally, it says transparency is vitally important in homeland security  risk management, and documentation should include transparent  disclosure of the rationale behind decisions, including the assumptions,  methodology, and the uncertainty considered. By fully disclosing what risk  factors are weighed and how decisions are made, TSA officials can better  ensure that their deployment of screening technologies matches potential  risks (threats, vulnerabilities, and consequences). This is of particular  importance given the agency’s limited resources and the fact that  screening technologies are not easily relocated.

		TSA Generally Deploys Screening Technologies Based on Logistical Factors

TSA officials said that absent a specific risk to an airport or category of  airports that would be mitigated by deploying a screening technology,  they consider a number of logistical factors that are aimed at maximizing  the operational efficiency of the screening process. These factors  influence the number of units of a technology the agency deploys to  airports, the order in which they deploy them, and where they are  deployed.
Officials said they use modeling software to determine the most efficient  number of units to allocate to an airport for each type of screening  system. This analysis takes into account variables such as the number  of flights at an airport, airport passenger volumes, items per passenger,  and secondary search rates. Additionally, agency officials said the layout  of an airport is a significant determining factor for the number of units it  receives. For example, an airport that has centralized checked baggage  screening areas will need fewer explosives detection systems than an  airport that has checked baggage screening areas dispersed in different  locations.
Additionally, TSA officials said that logistical and funding factors can  influence the order of deployment, including the manufacturer’s ability and  resources to develop and deliver technologies. For example, as of June  2019, officials said the agency was in the process of updating the  detection capabilities of 62 percent of its advanced technology x-ray fleet  because one of its two manufacturers had completed testing and  certification of the new technology, but the second manufacturer’s  technology had yet to be certified. Officials said they also try to plan  their deployment schedule around minimizing disruptions to airport  operations, so if an airport could not absorb a full deployment of a  technology because it would affect too many passengers, TSA would  schedule the deployment in phases to minimize disruptions.
Further, TSA officials said that, as a result of these many logistical  considerations, they generally fully deploy new screening technologies to  category X airports first—generally, airports with the highest passenger  volumes—and then proceed in order down to the airport with the lowest  passenger volume. Officials said larger airports generally have the  infrastructure in place to incorporate new technology without extensive  disruption to operations, and they will screen the most passengers by  deploying screening technologies to the largest airports first.

	TSA Does Not Ensure That Screening Technologies Continue to Meet Detection Requirements after Deployment to Airports

TSA practices do not ensure that screening technologies continue to  meet detection requirements after they have been deployed to airports.  According to agency officials, the agency uses certification to confirm that  technologies meet detection requirements before they are deployed to  airports, and calibration to confirm that technologies are at least minimally  operational while in use at airports. Officials stated these processes are  sufficient to assure TSA that screening technologies are operating as  intended. However, while certification and calibration serve important  purposes in the acquisition and operation of screening technologies, they  have not ensured that TSA screening technologies continue to meet  detection requirements after they have been deployed.
Certification occurs prior to deployment. TSA’s certification process is  designed to ensure screening technologies meet detection requirements  during the acquisition process, prior to the procurement and deployment  of the technologies, but it does not ensure screening technologies  continue to meet detection requirements after deployment. As previously  described, manufacturers provide an initial submission of the screening  technology to TSA for certification testing as part of the acquisition  process. During the certification process, S&T’s Transportation Security  Laboratory tests the technology under controlled conditions to determine  whether it meets TSA’s detection requirements. After TSA certifies that a  screening technology meets detection requirements and it undergoes  additional testing to determine whether it meets other TSA requirements  in controlled testing facilities, TSA may deploy it to select airports for  operational testing and evaluation to determine how it performs in an  airport setting. Certification testing demonstrates that a manufacturer’s  screening technology meets detection requirements during the acquisition  process, which allows TSA to determine whether it should continue to  consider the technology for acquisition.
Certification does not ensure that deployed technologies continue to meet  detection requirements because it does not account for the possibility that  performance of technologies can degrade over time throughout the  technologies’ lifecycles after deployment. For example, in 2015 and 2016,  DHS removed a sample of deployed explosives trace detection and  bottled liquid scanner units from airports for testing in the Transportation  Security Laboratory. The laboratory concluded that some deployed units  for each technology it tested no longer met detection requirements— either the required probability of detection for certain explosives or the  required rate for false alarm, or both. One explosives trace detection unit  that was tested was found to have a probability of detection much lower  than required. According to TSA officials, the units did not meet detection  requirements because they were not adequately maintained, which  affected their performance. In light of this, officials stated that they  introduced better controls to ensure that routine preventative  maintenance is performed as required. However, because TSA does not  test the units after they are deployed to airports, it cannot determine the  extent to which these controls ensure technologies continue to meet  detection requirements. Officials noted that TSA uses a layered security  approach at airports, so if one layer should fail—such as a deployed  technology—the agency can still rely on other security measures among  the various layers of security to detect threats. We have previously  reported on the importance that TSA ensure each measure is effective to  make the best use of its limited resources, in order to serve its aviation  security mission.
Calibration does not test whether technologies meet detection  requirements. TSA officials stated that daily calibration also helps ensure  that screening technologies continue to meet detection requirements after  deployment. However, while calibration demonstrates that the screening  technology is at least minimally operational, it is not designed to test  whether the screening technology meets detection requirements. For  example, each explosives detection system is calibrated with an  operational test kit that contains items of various densities. To calibrate  explosives detection systems, a TSO must run the operational test kit  through the unit and verify that the item is correctly displayed on the  monitor (see figure 5 below). This process demonstrates whether the  system can identify the known items’ densities, but it does not ensure that  the system meets detection requirements. As a result, calibration could  indicate that the unit is functioning even when its detection capabilities  have degraded—that is, calibration determines that the technology is  functional, but it does not ensure that the technology is meeting detection  requirements.
TSA officials stated that they plan to develop a process to review  screening technologies on an annual basis to analyze their performance,  including detection over time. TSA officials stated that, as of August 2019,  they were actively working on developing a review process for the  explosives detection system but did not have a date for when they  planned to complete it. TSA officials for the passenger and carry-on  screening technologies stated that they had not yet started developing a  review process for those technologies and the timeline for developing a  review process will depend on funding.
TSA officials also noted that there are challenges in designing a process  to ensure that screening technologies continue to meet detection  requirements after deployment. For example, TSA and S&T officials  stated that it is not feasible to conduct live explosives testing in airports.  Further, according to TSA officials, while it is relatively easy to temporarily  transfer smaller screening technologies, such as explosives trace  detection and bottled liquid scanner units, to a controlled setting for live  explosives testing, it would not be feasible to transfer larger installed  units, such as advanced imaging technology. Although testing with live  explosives in an airport poses undue risks and transferring larger  machines for testing may be costly, TSA could develop other measures.
TSA officials stated that there is no requirement to ensure that its  screening technologies continue to meet detection requirements after  deployment to airports. However, Standards for Internal Control in the  Federal Government calls for agencies to establish and operate a system  to continuously monitor the quality of performance over time. Without  taking additional steps to ensure screening technologies are meeting  detection requirements, TSA may run the risk that its deployed screening  technologies are not detecting explosives and other prohibited items.  Developing and implementing a process to monitor screening  technologies’ detection performance over time would help provide TSA  assurance that screening technologies continue to meet detection  requirements, as appropriate, after deployment. In doing so, TSA would  also be better positioned to take any necessary corrective actions if or  when screening technologies no longer operate as required.

	TSA Spent an Estimated $3.1 Billion to Purchase, Deploy, Install, and Maintain its Fiscal Year 2018 Inventory of Screening Technologies

We estimate that TSA spent $3.1 billion to purchase, deploy, install, and  maintain its inventory of screening technologies as of the end of fiscal  year 2018, based on agency estimates of costs. Of this $3.1 billion, we  estimate that TSA spent 71 percent to purchase screening technologies,  9 percent to deploy, about 12 percent to install, and, for fiscal year 2018,  about 9 percent to maintain them for 1 year. The highest estimated total  expenditures on a per-technology basis were for explosives detection  systems ($2.1 billion, or 68 percent), advanced technology x-ray ($443  million, or 14 percent), explosives trace detection ($227 million, or 7  percent), and advanced imaging technology ($197 million, or 6 percent).  Table 1 provides information on estimated expenditures for TSA’s  September 2018 inventory of screening technologies, by screening  technology and life-cycle phase (i.e., purchase, deploy, install, and  maintain). Appendix III provides additional information on estimated TSA  expenditures, such as prices per unit of technology and estimated  expenditures by airport category.
TSA has also incurred costs, or has plans to incur costs, for additional  actions related to screening technologies. Specifically, it has also incurred  costs for modifications to commercial airport facilities to accommodate  screening technologies. Further, TSA estimates additional life-cycle costs  of $804 million to acquire, deploy, and maintain computed tomography  systems through fiscal year 2026. The following provides more  information on these estimated expenditures.
Airport modifications. TSA incurs costs related to modifying commercial  airports to accommodate certain screening technologies, such as  checked baggage screening systems (e.g., explosives detection  systems). In December 2017, we reported that TSA had obligated at least  $783 million from fiscal years 2012 through 2016 to reimburse airports for  the allowable design and construction costs associated with installing,  updating, or replacing screening technology. For example, TSA may  enter into agreements to reimburse airport operators for a percentage of  the allowable design and construction costs associated with facility  modifications needed for installing, updating, or replacing in-line  explosives detection systems. In-line screening systems use conveyor  belts to route checked luggage through explosives detection systems,  which capture images of the checked baggage to determine if a bag  contains threat items not permitted for transport, including explosives.  From fiscal years 2012 through 2016, agreements for TSA  reimbursements to airports for checked baggage screening systems  generally ranged in value from $50,000 to $150 million. As we reported  in December 2017, in general, depending on the airport’s size, TSA will  reimburse 90 or 95 percent of the allowable, allocable, and reasonable  cost of certain projects. For other projects, TSA may provide 100 percent  reimbursement—for example, where existing systems require the  correction of security or safety deficiencies.
Computed tomography. In addition to its fiscal year 2018 inventory, TSA  is currently in the process of deploying computed tomography to  commercial airports to replace advanced technology x-ray systems.  Computed tomography technology applies sophisticated algorithms to  detect explosives and other prohibited items and creates a 3D image of  carry-on baggage that a TSO can view and rotate 360 degrees. In fiscal  year 2018, TSA determined that computed tomography is the best  technology available to address rapidly evolving threats in the  transportation sector, and plans to eventually deploy it to all checkpoints  and replace advanced technology x-ray technology. As recorded in  TSA’s Deployed Locations Report, TSA had deployed 11 computed  tomography systems to category X and I airports as of September 24,  2018. According to TSA’s September 2018 life-cycle cost estimates, the  agency plans to field 883 units by fiscal year 2026. As shown in table 2,  TSA also planned to spend $805 million to purchase, deploy, and  maintain this new technology through fiscal year 2026. However, in  August 2019, TSA officials told us that they expect this estimated total  procurement cost of $805 million to likely decrease as the per unit cost  had decreased from $400,000 to $233,000 in the initial fiscal year 2019  contract for computed tomography.

	Conclusions

TSA has invested billions of dollars in screening technologies as it  responds to terrorists’ attempts to use homemade explosives to disrupt  and damage civil aviation. Forecasted increases in passenger volumes  and ongoing terrorist threats make it imperative that TSA employ  recommended management and internal control practices. TSA could  help ensure that critical detection standards are developed in accordance  with approved practices, and that agency goals are effectively met by  updating its guidance for developing standards. Additionally, by  documenting key decisions in the development of detection standards,  TSA could better assure the effectiveness of decision-making and the  retention of organizational knowledge in the face of inevitable changes in  personnel.
Similarly, when making technology deployment decisions, incorporating  DHS-recommended practices for risk management would improve TSA’s  ability to effectively fulfill its mission to secure the nation’s civil aviation  system. While TSA assesses risk when deciding whether to invest in a  new technology to address an identified capability gap, it is unclear the  extent to which it considers risk when determining where and in what  order to deploy approved screening technologies to airports. DHS  guidance for homeland security risk management calls for risk to be  considered consistently and comprehensively in all aspects of an  agency’s work. Additionally, risk management includes transparent  disclosure of the rationale behind decision-making so that stakeholders  can understand how key factors were weighed. Incorporating these risk  management principles into its decision-making for deploying screening  technologies to airports would allow TSA to align its deployment  strategies with potential threats, vulnerabilities, and consequences.
Lastly, TSA cannot ensure that its screening technologies continue to  meet detection requirements after they have been deployed to airports.  Developing and implementing a policy to ensure that TSA’s screening  technologies continue to meet their respective detection requirements  after deployment may assure the agency that its deployed screening  technologies are effectively detecting explosives and other prohibited  items that they are designed to identify, which is a critical part of TSA’s  mission.

	Recommendations for Executive Action

We are making the following five recommendations to TSA:  The TSA Administrator should update TSA guidance for developing and  approving screening technology explosives detection standards to reflect  designated procedures, the roles and responsibilities of stakeholders, and  changes in the agency’s organizational structure. (Recommendation 1)
The TSA Administrator should require and ensure that TSA officials  document key decisions, including testing and analysis decisions, used to  support the development and consideration of new screening technology  explosives detection standards. (Recommendation 2)
The TSA Administrator should require and ensure that TSA officials  document their assessments of risk and the rationale—including the  assumptions, methodology, and uncertainty considered—behind  decisions to deploy screening technologies. (Recommendation 3)
The TSA Administrator should develop a process to ensure that  screening technologies continue to meet detection requirements after  deployment to commercial airports. (Recommendation 4)
The TSA Administrator should implement the process it develops to  ensure that screening technologies continue to meet detection  requirements after deployment to commercial airports. (Recommendation  5)

	Agency Comments and Our Evaluation

We provided a draft of this product to DHS for comment. We provided a  draft of this report to DHS for review and comment. DHS provided written  comments, which are reproduced in full in appendix IV. DHS concurred  with our five recommendations and described actions undertaken or  planned to address them. TSA also provided technical comments, which  we incorporated as appropriate.
With regard to our first recommendation that TSA update guidance for  developing and approving screening technology explosives detection  standards, DHS concurred and stated that TSA has included updated  guidance in its Requirements Engineering Integrated Process Manual,  which was completed in September 2019. According to DHS, the  update provides TSA’s process for developing and approving explosives  detection standards, including designated procedures and roles and  responsibilities of stakeholders, and reflects organizational changes to  TSA. TSA provided us with the Requirements Engineering Integrated  Process Manual in November 2019, concurrent with DHS comments. We  will review the update and the extent to which it addresses the  recommendation. This action, if fully implemented, should address the  intent of the recommendation.
DHS concurred with our second recommendation that TSA ensure that  officials document key decisions supporting the development of screening  technology explosives detection standards. DHS stated that the updated  Requirements Engineering Integrated Process Manual describes the  process for documenting key decisions, including testing and analysis  decisions, in the development of new detection standards. We will review  the update and the extent to which it addresses the recommendation.  This action, if fully implemented, should address the intent of the  recommendation.
DHS also concurred with our third recommendation that TSA document  its assessments of risk and the rationale behind its decisions to deploy  screening technologies. According to DHS, TSA has instituted an  improved process for documenting elements that contribute to  deployment decisions—TSA’s August 2019 deployment plan for  computed tomography is an example of the process. DHS stated that  TSA will continue to include a comparable level of documentation in  future deployment plans for screening technologies. We agree the  computed tomography deployment site selection strategy is an example  of how TSA can document the rationale governing the deployment of a  screening technology. Future plans can further benefit by explaining the  risk analysis itself along with the role that risk considerations played in the  selection of airports for deployment. Formalizing guidance that directs  TSA officials to document risk assessments and the rationale behind  deployment decisions would help TSA ensure that its deployment of  screening technologies matches potential risks.
DHS concurred with our fourth and fifth recommendations that TSA,  respectively, develop and implement a process to ensure that screening  technologies continue to meet all detection requirements after  deployment to commercial airports. DHS stated that TSA will develop  recurring individual post implementation reviews (PIR) for all screening  technologies in accordance with DHS Directive 102-01, to assess multiple  aspects of system performance, including detection over time. DHS also  stated that TSA intends to examine the component performance of the  detection chain rather than a direct measure of detection requirements,  due to the limitations of using live explosives and simulants. DHS stated  that because the detection chain for each technology is unique and will  require individual reviews, TSA is developing a policy on the PIR  development process, which it estimates will be completed by March 31,  2020. We appreciate the limitations live explosives and simulants present  in testing and the need for reviews that are tailored to meet the unique  characteristics of each screening technology. TSA plans to implement the  review process on the first screening technology by December 31, 2020.  These actions, if implemented across all applicable screening  technologies, should address the intent of the recommendations.
We are sending copies of this report to the appropriate congressional  committees and to the Acting Secretary of Homeland Security. In  addition, this report is available at no charge on the GAO website at  http://gao.gov.
If you or your staff have any questions concerning this report, please  contact me at (202) 512-8777 or russellw@gao.gov. Contact points for  our Offices of Congressional Relations and Public Affairs may be found  on the last page of this report. GAO staff who made significant  contributions to this report are listed in Appendix V.

Appendix I: Transportation Security Administration (TSA) Screening Technologies

This appendix presents additional details on the TSA screening  technologies we reviewed, including their function and the number of  units deployed.

Appendix II: Objectives, Scope, and Methodology

This report addresses Transportation Security Administration’s (TSA)  processes for developing and deploying screening technologies to  airports regulated by TSA (i.e., “commercial” airports). Specifically, we  examined  1. the extent to which TSA has a process for developing explosives  detection standards for screening technologies in response to identified  emerging threats;  2. how TSA operationalizes detection standards to update detection  capabilities;  3. the extent to which TSA has considered risk when deploying screening  technologies to commercial airports;  4. the extent to which TSA ensures screening technologies meet the  requirements for detection standards after deployment; and  5. TSA estimated expenditures to purchase, deploy, install, and maintain  its inventory of screening technologies as of the end of fiscal year 2018.
To address all of our objectives, we identified 11 screening technologies  TSA used to screen passengers’ identification documents, person, carry- on bags, and checked baggage at commercial airports as of September  24, 2018, as recorded in TSA’s Government Property Management  database. The seven screening technologies in use at commercial  airport passenger checkpoints were advanced imaging technology,  advanced technology x-ray machine, bottled liquid scanner, boarding  pass scanner, chemical analysis device, threat image projection x-ray,  and walk-through metal detector. The credential authentication  technology and computed tomography, also used at checkpoint  screening, were deployed and in use at select airports as TSA pilot  projects. The two TSA screening technologies in use at commercial  airports for checked baggage were explosives detection systems and  explosives trace detection (TSA also uses explosives trace detection for  checkpoint screening).
We assessed the reliability of TSA’s inventory data by interviewing  agency officials and reviewing related documentation, such as the  database user manual, among other things. We determined the data were  sufficiently reliable to determine the type and number of TSA screening  technologies deployed as of September 2018. To better understand how  TSA screening technologies have been used, we reviewed reports from  the U.S. Department of Homeland Security (DHS) Office of the Inspector  General, the Congressional Research Service, past GAO reports, and  relevant DHS and TSA documentation, such as DHS and TSA strategic  documents and acquisition plans. To observe TSA screening procedures  and the operation of screening technologies in the airport setting, we  conducted site visits to seven commercial airports. During these visits we  discussed screening technology issues with TSA federal security  directors or their representatives. We selected these airports to reflect a  range of airport categories, technologies, and geographic diversity.  The  results of these site visits and interviews cannot be generalized to all  commercial airports, but they provided us with important context about  the installation, use, and maintenance of TSA screening technologies  across the different types of airports that TSA secures. We also  conducted a site visit to the TSA Systems Integration Facility to better  understand how screening technologies are tested and evaluated prior to  deployment. Further, we interviewed officials from two industry  associations and one screening technology manufacturers association  based on input from TSA and DHS Science and Technology Directorate  (S&T) officials.
To determine the extent to which TSA has a process for developing  explosives detection standards, we examined TSA documents such as  approved detection standards, action memos summarizing support for  proposed detection standards, the Detection Requirements Update  Standard Operating Procedure, and briefing slides describing TSA’s  process, as of August 2019, for assessing threat materials and  developing detection standards. We also evaluated Material Threat  Assessment reports that summarized the testing and analyses performed  by S&T’s Homemade Explosives Characterization Program, in  coordination with S&T laboratories, to characterize (identify the physical  density and mass of) explosive materials for detection standards  developed from fiscal years 2014 through 2018. We evaluated S&T’s  testing and analyses in accordance with TSA and S&T guidance to  determine the extent to which these steps were consistent across  materials; we did not analyze the sufficiency of the testing and analyses.  We also assessed TSA and S&T processes and the extent to which they  were documented in accordance with Standards for Internal Control in the  Federal Government, and discussed the details of steps taken to develop  standards with relevant TSA and S&T officials. In addition, we conducted  a site visit to S&T’s Commercial Aircraft Vulnerability and Mitigation  Program testing site at the U.S. Army Aberdeen Test Center, Maryland, to  better understand how S&T tests the vulnerability of commercial aircraft  to explosive materials.
To understand TSA’s process and timelines for operationalizing—putting  into effect—detection standards, we requested information from TSA  about screening technologies subject to explosives detection standards,  deployed as of September 24, 2018: advanced imaging technology,  advanced technology x-ray, bottled liquid scanner, explosives detection  systems, and explosives trace detection. We requested information about  the detection standards that deployed screening technologies met, as of  August 2019, as well as subsequently approved detection standards,  including the date the standards were approved, the dates when TSA  achieved certain acquisition milestones when developing and deploying  the associated technologies, and the status of ongoing and upcoming  efforts to update detection capabilities to meet new standards. We  identified the acquisition milestones by reviewing a past GAO report on  TSA’s acquisition process and in consultation with GAO acquisition  experts. We also reviewed a classified TSA report that evaluated the  performance of a particular algorithm in order to understand TSA’s  process for developing new screening technologies to meet detection  standards. In addition, we reviewed relevant acquisition documents, such  as DHS’s Acquisition Management Instruction 102, the 2018  Transportation Security Administration Systems Acquisition Manual,  acquisition decision memos, acquisition plans, and Operations  Requirements Documents. To understand TSA’s process for deciding  whether to operationalize detection standards, we requested and  reviewed available documentation for the standards that TSA had not  operationalized, such as an operational status transition memo for bottled  liquid scanner, and interviewed TSA officials about those decisions.
To understand how TSA had considered risk in its approach to deploying  screening technologies at airports, we reviewed available documentation  related to TSA’s deployment decisions. These included decision memos  from acquisition review board meetings and action memos to TSA  leadership; risk registers for checked baggage and checkpoint acquisition  programs; available deployment plans, such as the agency’s Action Plan  for deploying explosives trace detection units to airports in 2018; and  acquisition guidance. To understand how TSA assesses capability  needs and gaps, we interviewed agency officials about TSA’s  Transportation Security Capability Analysis Process and reviewed  capability analysis reports from 2018 and 2019, as well as TSA’s  prioritized list of capability gaps and needs. We also interviewed  acquisition officials, including TSA’s Component Acquisition Executive,  about the role of risk in deployment decisions and requested written  responses to specific questions. We assessed TSA’s decision-making  process for deploying and updating screening technologies, generally,  against DHS risk management criteria, such as DHS’s Risk Management  Fundamentals. We also reviewed related areas of risk management and  decision-making to understand the context in which TSA makes  deployment decisions. Specifically, we reviewed the 2017 Transportation  Sector Security Risk Assessment and the Cities and Airports Threat  Assessment reports to understand the risks facing the nation’s aviation  system. We also reviewed TSA’s enterprise risk management  framework, such as the Enterprise Risk Management Policy Manual, to  understand the role it played in TSA’s deployment decisions. We also  interviewed an official from TSA’s Enterprise Performance and Risk office  and the Executive Risk Steering Committee. To understand how TSA  categorizes airports, we reviewed a 2017 Nationwide Airport  Categorization Review memo from TSA’s Security Operations office and  interviewed Security Operations officials.
To understand how TSA deploys screening technologies across airports  and categories of airports, we analyzed TSA’s Deployed Locations  Report, which reported on technologies that were in use or available for  use at commercial airports from September 24 through September 30,  2018. We also reviewed TSA’s standardized methodology for determining  the most efficient number of screening technologies at an airport.  Additionally, we reviewed TSA’s Strategic Five-Year Technology  Investment Plan from 2015 and the 2017 Biennial Refresh to understand  TSA’s plans for ongoing investment in screening technologies. We  reviewed various throughput data, such as annual passenger throughput  for all commercial airports for fiscal year 2018 and enplanements data for  calendar year 2017, to understand and compare TSA’s allocation of  screening technologies with throughput data across airports and airport  categories. We used this analysis to identify airports that had an  unusually large or small number of screening technologies within a  category, and interviewed TSA officials to understand the decisions that  led to the allocation of screening technologies across airports and airport  categories.
In addition, we reviewed the status of TSA’s limited deployment of  computed tomography units to checkpoints. Specifically, we reviewed  TSA’s 2018 Deployment Site Selection Strategy, which described the  airports to which TSA would deploy computed tomography units and the  methodology it used to select them, slides from recent conferences TSA  held with industry representatives where it shared its plans for  transitioning to computed tomography, and relevant Operational  Requirements Documents. We also interviewed agency officials about  their plans for the limited deployment and TSA’s transition from advanced  technology x-ray to computed tomography for checkpoint screening.
To determine the extent to which TSA ensures its screening technologies  continue meeting detection requirements after deploying them to airports,  we reviewed TSA acquisition detection requirements for each screening  technology as well as TSA guidance related to the testing and evaluation  of screening technologies identified by TSA officials in interviews. We  also interviewed TSA and S&T Transportation Security Laboratory  officials about TSA requirements to test screening technologies, both  prior to and after deployment, to determine the extent to which they meet  detection requirements. We also observed transportation security  officers and a transportation security specialist for explosives conduct  verification and calibration procedures on screening technologies at the  airports we visited. We reviewed TSA guidance to determine the extent  to which its procedures ensure that screening technologies continued to  meet detection requirements in airports. We then evaluated the  procedures against Standards for Internal Control in the Federal  Government for monitoring.
To identify TSA’s estimated expenditures to purchase, deploy, install, and  maintain its inventory of screening technologies as of the end of fiscal  year 2018, we reviewed TSA programs’ life-cycle cost estimates, which,  for the purposes of acquisition planning, provide per unit estimates of the  cost to purchase, deploy, install, and maintain passenger and checked  baggage screening technologies. We chose this methodology in  consultation with TSA officials and after determining that historical  records of obligations and expenditures do not provided consistent and  sufficient detail for the purposes of our analysis.
The life-cycle cost estimates include relevant phases for each screening  technology (i.e., purchase, deploy, install, and maintain), although not all  technologies have cost estimates for each phase of the life cycle. For  example, some screening technologies may not specify deployment costs  because such costs are included in the initial purchase price of the unit. In  other cases, the technology does not have a deployment cost because  the unit is small and portable, and placement of the unit is therefore  handled by TSA airport staff at no charge. Estimated expenditures for  installation also include costs associated with site acceptance testing,  which is performed when a system is installed at its operational location.  Unlike the purchase, deploy, and install unit prices, the maintenance unit price is the yearly cost of maintenance for one unit, and therefore recurs  every year.
We assessed the reliability of the life-cycle cost estimates by reviewing  documentation on the development of the estimates and interviewing TSA  officials, among other things, and determined the estimates were  sufficiently reliable for the purpose of estimating the amount of funds  spent on acquiring, deploying, installing, and maintaining TSA’s inventory  of screening technologies as of the end of fiscal year 2018. Because the  life-cycle cost estimates were developed in different years, we used TSA  guidelines to adjust costs for inflation and convert our estimates to 2018  dollars. We multiplied these estimates against the number of screening  technologies deployed to commercial airports as of September 24, 2018,  using data from TSA’s Government Property Management database. For  computed tomography, we also obtained information on price and  quantity from the technology’s life-cycle cost estimate and TSA officials.  We also reviewed prior GAO work on TSA cost sharing programs for  airport facility modification related to installation of some of the  technologies in our review.
We conducted this performance audit from April 2018 to December 2019  in accordance with generally accepted government auditing standards.  Those standards require that we plan and perform the audit to obtain  sufficient, appropriate evidence to provide a reasonable basis for our  findings and conclusions based on our audit objectives. We believe that  the evidence obtained provides a reasonable basis for our findings and  conclusions based on our audit objectives.

Appendix III: Transportation Security Administration (TSA) Estimated Expenditures for Screening Technologies

We estimate that TSA spent $3.1 billion to purchase, deploy, install, and  maintain its inventory of screening technologies, as of the end of fiscal  year 2018, based on agency estimates of costs. Tables 3 through 5  provide information on estimated TSA expenditures by screening  technology, life-cycle phase, and airport category. To analyze TSA’s  estimated spending to purchase, deploy, install, and maintain its inventory  of screening technologies as of the end of fiscal year 2018, we reviewed  TSA life-cycle cost estimates, which, for the purposes of acquisition  planning, provide per-unit estimates of the cost to purchase, deploy,  install, and maintain passenger and checked baggage screening  technologies at TSA-regulated airports (i.e., “commercial” airports).  Because the life-cycle cost estimates were developed in different years,  we used the same guidelines used by TSA to adjust costs for inflation to  convert our estimates to 2018 dollars. We multiplied these estimates  against the number of screening technologies deployed to commercial  airports as of September 24, 2018.

Appendix IV: Comments from the Department of Homeland Security

Appendix V: GAO Contact and Staff Acknowledgments

	GAO Contact

	Staff Acknowledgments

In addition to the contact named above, Kevin Heinz (Assistant Director),  Barbara Guffy (Analyst in Charge), Kelsey Burdick, Jonathan Felbinger,  Tyler Kent, Thomas Lombardi, Erin O’Brien, Kya Palomaki, Rebecca  Parkhurst, and Dina Shorafa made key contributions to this report. In  addition, key support was provided by Chuck Bausell, Richard  Cederholm, Dominick Dale, Aryn Ehlow, Michele Fejfar, Eric Hauswirth,  Richard Hung, and Alexis Olson.