earch?
• Finding 5. Facilitators of research translation include a focused mission to improve care through research; research infrastructure, such as the colocation of researchers and clinicians at Clinical Centers of Excellence and the collection, m


ons. The scope and heterogeneity prevented a focused look within population and condition groups for summary effect estimates or strength of evidence assessments. Finally, as with all scoping reviews, some data items may be subject to misclassificati


nt.
Exposure characterization can be accomplished through measurement or modeling or, in some cases, a combination of both methods.
Examines associations between one or more exposures and health outcomes.
 xii
Gray literature
Group or population stud


 to submit a petition in accordance with the “Policy and Procedures for Handling Submissions and Petitions to Add a Health Condition to the List of WTC-Related Health Conditions” (WTC Health Program, 2021a).
Data compiled on a defined population, suc


survivors, referring to those present in the New York City disaster area in the dust or dust cloud on September 11, 2011; those who worked, resided, or attended school, childcare, or adult daycare in the New York City disaster area during or after Se


ers want to understand the impacts of research investments to date and how to maximize those impacts going forward. In 2017, NIOSH engaged the RAND Corporation to conduct a four-year, independent assessment of the translational impacts of Program-sup


answer the second research question (RQ2: To what extent is WTC health-related research translational?), we synthesized qualitative data from gray literature, Program documentation, focus groups, and interviews. From focus group and interview transc


rogram research.
 Although this report is of interest to a broad range of stakeholders, the primary audience is
 Program leaders who are charged with implementing the Program’s research mission. Therefore,
 we offer a concise report that directly con


ed full-text articles with adjudication and quality review. Data were extracted using a customized electronic form implemented in DistillerSR (DistillerSR, undated). Data analyses were performed using DistillerSR, Python (Python, undated), and Micros


y types and funding sources have evolved over time. Figure 2.2 shows the distribution of study types by funding source (NIOSH and non-NIOSH) in three-year increments. Regardless of funding source, exposure-outcome linkage studies have predominated (b


ublications supported by other funding were less likely to report on studies of responders than of survivors (responders: N = 181 of 515, 35.1 percent; survivors: N = 316 of 515, 61.1 percent).
Many studies included both populations, and because the


sychiatric conditions when considering all funders (physical: N = 182, 39.7 percent; neuropsychiatric: N = 294, 64.1 percent). Among all publications that addressed a covered neuropsychiatric condition, we found high concentrations of research on PTS


t research should cover every ring and node of the diagram, nor that it should start at fundamental questions before proceeding to questions on the outer rings. Rather, study of a topic may start on any ring, may skip rings, and may follow a nonlinea


ng can be seen as an impact of the research, including its use for decisions about care, program planning, and policy. Using findings from stakeholder focus groups and interviews, Program documentation, and gray literature, we found that WTC health-r


porated into iterative cycles of program planning. Program leaders describe using research findings to “make better decisions in terms of programmatic efforts or clinical procedures or clinical protocols within our unique population.” One Program lea


l and mental health conditions related to the September 2001 terrorist attacks, as well as diagnosis and treatment options for those enrolled in the Program (WTC Health Program, undated a). Specific emphasis was given to studies of biomarkers of expo


   WTCHP Funding Announcement (August 2020)
SOURCE: Author analysis of Program documentation.
NOTE: WTCHP = World Trade Center Health Program. This figure shows that the rationale and priorities for the Program’s research mission were established and


ralized as a disaster population.” An FDNY focus group participant had a similar perspective: “It’s really important for similar disasters down the line that we’ve become the advocates for this . . . because of what we went through, it’s going to be


udy establishes a link between 9/11 exposure and a certain outcome, that is considered a translational milestone in the Fundamental Questions ring of the framework and is denoted in Figures 2.6, 2.7. and 2.8 with a “1” next to the node that most clos


C disaster?” We identify research on this question as translational milestone 1 in the Fundamental Questions ring (see Figure 2.6). A smaller proportion assessed clinical, health services, or policy interventions for mental and behavioral health, wit


d people across their life course looking at sort of the trajectories of PTSD and depression. And I would love to see that continue because I think—just from pure research, nothing to do with the World Trade Center per se—there’s very little out ther


or practice.
The other group of publications on women’s health addressed a program or intervention aimed at modifying health outcomes (outcome modification studies, N = 9). These publications described treatment programs and are indicated as translat


g. This large body of literature has also led to the publication of several synthesis studies focused on the same topics.
Translational Milestones in Research on Youth
This body of work addresses the question, What are the health outcomes of WTC-expo


me after adult mental health and respiratory disease (WTC Health Program Scientific/Technical Advisory Committee, 2016).
The SSC has continued to urge the Program to prioritize research on children, young adults, and women. During the SSC’s September


lth, and a 2020 evaluation report shed light on the feasibility of creating a youth-focused cohort. Ongoing discussions will be needed to chart the best path forward for WTC youth-related health research.
Conclusion
In this chapter, we explain our co


search mission, infrastructure to support research, and investments in research communication and dissemination.
A Focused Research Mission
The Program’s research mission is to support translational bridges from research to improved clinical care. A


Es and the collection, management, and analysis of data by the Data Centers and Registry—serves as a facilitator of translation.
CCE-based research and care inform each other directly. Several researchers and CCE representatives who participated in f


requently contact Members to ascertain their preferred forms of engagement with the Program.
The Program has placed an increased emphasis in the past few years on using several communication channels and modalities for disseminating research findings


r, 2019).
• For a study on longitudinal changes in PTSD symptoms in WTC survivors, a researcher identified a need for clinical trials to study different modalities of PTSD treatment and the effectiveness of those treatments (Rosen, 2019).
1 These mee


ing medical education credit (Howard et al., 2018).
Peer-Reviewed Publications
Program leaders emphasize the importance of disseminating Program-funded research in the peer-reviewed literature, recognizing that rigorous peer review of the quality of


tion: methodological limitations, fragmentation of the U.S. health care system, funding constraints, variable definitions of research translation, and misaligned expectations of research.
Methodological Limitations
All health-related research—in any


stions there are with respect to 9/11 and health impacts and how challenging it is for researchers to secure funding to conduct important studies. As with any research program, NIOSH leadership must make decisions about the research it will prioritiz


at WTC-related research can and should achieve, as well as what the Program could do with better evidence. We identified four types of expectations for WTC health-related research.
Some Members who participated in our focus groups expressed a desire


s, and limitations of research, this misalignment leads to understandable frustration when different stakeholders perceive that they are not heard, understood, and responded to. The Program can enhance its efforts to engage stakeholders to overcome t


s to research translation include methodological limitations, fragmentation of the health care system, funding constraints, variable definitions of translation, and misaligned expectations about research.
 36
RQ4: To What Extent Has WTC Health-Relate


grant progress reports or presentations at the biannual research grantee meeting could be mined to identify translational accomplishments and nominate new research priorities. The Program can also consider analyses of the connections between related


 decisions can be more effectively communicated to various internal and external Program stakeholders.
We identified the NIEHS framework as a promising model for describing and assessing research translation in the environmental health context. In Ch


rch along different translational pathways. These are all potential actions the Program could take to more actively, and more proactively, manage translational bridges.
NIOSH and Program stakeholders might work together to develop case studies of tra


arch lifecycle, from planning research, to conducting research, to using it in communications and decisionmaking. An examination of the Program’s efforts to use established engagement frameworks and guidance will help illuminate new engagement activi


                   Example: Research to Care community events are held to review care implications of recent intervention studies.
     Example: Research case studies using the NIEHS framework are codeveloped by researchers and the Program.
 Using r


ay of examining a body of research, is novel and may need extensive testing and validation. It is unknown whether this framework on its own can capture all the distinctiveness and complexities of the Program’s research activities.
Conclusion
More tha


ygiene, the Registry is an ongoing epidemiological cohort study that was established in 2002. The largest registry in the history of the United States to follow the health effects of a disaster, it includes waves of surveys conducted approximately ev


ugh the FDNY Responder Health Program, the CCEs, and the NYC Health and Hospitals System WTC Environmental Health Center
• Health system leadership
• Policymakers (state, local, and federal [WTC Health Program science
team, CDC, other federal agencie


eting, etc.)
Independent data: peer-
reviewed literature
Independent data: gray
literature
Independent data: in
progress research (NIH
reporter)
Primary data collection:
groups and interviews)
stakeholder views (focus
Other (political context?)
Zadro


thesis of themes emerging from coded data. In this appendix, we also describe our approach to integrating quantitative and qualitative findings through the development of case studies of research impact.
Research Question Development and Refinement
T


 (5) collating, summarizing, and reporting the results (Chapter 2, “Impacts of World Trade Center Health-Related Research”).
Record Search
We used a systematic search strategy to find records in databases of peer-reviewed publications and gray litera


rocess are reported in the Preferred Items for Reporting on Systematic Reviews and Meta-Analyses (PRISMA) flowchart in Figure 2.1 (Liberati et al., 2009).
Full-Text Review
To guide the abstraction of data from peer-reviewed publications, we developed


ation studies describe physical or chemical stressors in the environment. Exposure-outcome linkage studies examine associations between one or more exposures and health outcomes (including mental health). Outcome modification studies examine characte


r, other similar acute traumatic injuries. Cancer: Any cancer (cancer type not specified); blood and lymphoid tissue, digestive system, eye and orbit, female breast, female reproductive organs, head and neck, respiratory system, skin (melanoma and no


akeholders have, according to several different models for visualizing a body of research. Their selection does not imply an evaluation standard: for instance, that WTC evidence should equally cover every evidence type from prioritization to dissemin


decisionmakers compare the benefits and harms of various interventions and strategies for preventing, diagnosing, treating, and monitoring health conditions in real-world settings.
• Evidence synthesis: The process of systematically reviewing, evalua


f these free-text options, creating a categorization scheme that we iteratively refined as we reviewed each free-text entry. The items include
• intervention types under the categories of screening or testing, treatment or management, or other interv


his initial round was to test and improve the appropriateness of the preliminary codebook and identify the need for additional documentation. The preliminary codebook was refined on the basis of these findings and again compared with the focus group


nalysis. We used the focus group discussion guide, which was based on our research questions, to develop a priori codes. This codebook was developed in parallel with another codebook for Program documentation, described below. The two were frequently


engine of more than 1,200 think tanks and research centers that is maintained by the Harvard Kennedy School (Harvard Kennedy School, 2021). In Google, we carried out three searches, two with limits to domain type (.org and .gov) and one without domai


us group transcripts, Program documentation, and gray literature documents. This text was then processed using a “bag of words” natural language processing approach, in which each page is transformed into a list of commonly used words and two-word ph


findings from multiple data sources across the same condition or population of interest (e.g., WTC youth, PTSD), using the research questions as a guide. For instance, to address subquestions about the extent to which the research portfolio addresses


steps and connected directly to Program planning and policy. As detailed in Chapter 2, translational steps are depicted in this framework on five “rings” corresponding to different kinds of research questions (fundamental questions, application and s


OR attack* OR catastroph* OR disaster* OR crash* OR (building* adj3 collaps*) OR (tower* adj3 collaps*) OR (building* adj3 burn*) OR (tower* adj3 burn*) OR (building* adj3 fire*) OR (tower* adj3 fire*) OR plane* OR airplane* OR jet* OR aircraft* OR “


RMS: TI terror* OR AB terror* OR KW terror* OR TI attack* OR AB attack* OR KW attack* OR TI catastroph* OR AB catastroph* OR KW catastroph* OR TI disaster* OR AB disaster* OR KW disaster* OR TI crash* OR AB crash* OR TI crash* OR TI(building* N3 coll


g* N3 burn*) OR TI(tower* N3 burn*) OR AB(tower* N3 burn*) OR MW(tower* N3 burn*) OR TI(building* N3 fire*) OR AB(building N3 fire*) OR MW(building N3 fire*) OR TI(tower* N3 fire*) OR AB(tower* N3 fire*) OR MW(tower* N3 fire*) OR TI plane* OR AB plan


OR aircraft* OR burn* OR crash* OR hijack*) OR ABS(terror* OR attack* OR catastroph* OR disaster* OR collaps* OR tower* OR building* OR fire* OR plane* OR airplane* OR jet* OR aircraft* OR burn* OR crash* OR hijack*) AND (EXCLUDE (SUBJAREA , “SOCI”)


d Sep* to Sept*.)
Statement #1: Common References to the Attack
TS=(“world trade center”) OR TS=(sept* NEAR/0 11) OR TS=(sept* NEAR/0 11th) OR TS=(“ground zero”) Exclude: RESEARCH AREAS: (PLANT SCIENCES OR ENGINEERING OR REMOTE SENSING OR COMPUTER SC


ES OR INSTRUMENTS INSTRUMENTATION OR PHYSICS OR REMOTE SENSING OR FORESTRY OR HISTORY PHILOSOPHY OF SCIENCE OR ENERGY FUELS OR THERMODYNAMICS OR ENTOMOLOGY OR WATER RESOURCES OR ASTRONOMY ASTROPHYSICS OR AUTOMATION CONTROL
       69
SYSTEMS OR ZOOLOG


Application of Translational Research Frameworks
• Evidence type (e.g., generation, synthesis)
• NIEHS Translational Research Framework
• NIH Translational Science Spectrum
Free-Text Capture
• Study objectives
• Findings
• Implications
• Limitations


ard E., Joseph A. Boscarino, and Sandro Galea, “Alcohol Use, Mental Health Status and Psychological Well-Being 2 Years After the World Trade Center Attacks in New York City,” American Journal of Drug and Alcohol Abuse, Vol. 32, No. 2, 2006, pp. 203–2


radeep, Daniel Del Prado, Steven Karidas, Richard Neff, Michael Sadler, “Penetrating Axillary Trauma in a Survivor of the World Trade Center Disaster,” Emergency Radiology, Vol. 9, No. 5, 2002, pp. 300–302.
24. Albert, Pradeep, Jeremy Handel, Paula Y


g Rosen, Kim Davis, Greg Smith, and Malachy Corrigan, “‘Stay Connected’: Psychological Services for Retired Firefighters After 11 September 2001,” Prehospital and Disaster Medicine, Vol. 22, No. 1, 2007, pp. 49–54.
34. Ammar, Ali, Zahia Esber, Sunil


asilios Christodoulou, Michael D. Weiden, and David J. Prezant, “Accelerated Spirometric Decline in New York City Firefighters with α1- Antitrypsin Deficiency,” Chest, Vol. 138, No. 5, 2010, pp. 1116–1124.
43. Banauch, Gisela I., Atiya Dhala, Dawn Al


2. Beebe, Beatrice, Christina W. Hoven, Marsha Kaitz, Miriam Steele, George Musa, Amy Margolis, Julie Ewing, K. Mark Sossin, and Sang Han Lee, “Urgent Engagement in 9/11 Pregnant Widows and Their Infants: Transmission of Trauma,” Infancy, Vol. 25, No


 Child and Adolescent Psychotherapy, Vol. 10, Nos. 2–3, 2011, pp. 286–295.
62. Bergman, Anni, K. Mark Sossin, Suzi Tortora, Lydia (Ryan’s Mother), Phyllis Cohen, and Beatrice Beebe, “The Team Approach to the Treatment of a Traumatized Mother and Chil


ncy Dodson, Jeanne M. Stellman, Steven Southwick, Vansh Sharma, Robin Herbert, Jacqueline M. Moline, and Craig L. Katz, “Stories Behind the Symptoms: A Qualitative Analysis of the Narratives of 9/11 Rescue and Recovery Workers,” Psychiatric Quarterly


ster: Implications for Mental Health Care,” General Hospital Psychiatry, Vol. 26, No. 5, 2004, pp. 346–358.
82. Boscarino, Joseph A., Richard E. Adams, and Charles R. Figley, “A Prospective Cohort Study of the Effectiveness of Employer-Sponsored Cris


ndro Galea, Jennifer Ahern, “Utilization of Mental Health Services Following the September 11th Terrorist Attacks in Manhattan, New York City,” International Journal of Emergency Mental Health, Vol. 4, No. 3, 2002, pp. 143–155.
93. Boscarino, Joseph


Grande, and James E. Cone, “Gender Differences in Probable Posttraumatic Stress Disorder Among Police Responders to the 2001 World Trade Center Terrorist Attack,” American Journal of Industrial Medicine, Vol. 53, No. 12, 2010, pp. 1186–1196.
103. Bow


Horowitz, and Joanne P. Reinhardt, “The September 11th Attacks and Depressive Symptomatology Among Older Adults with Vision Loss in New York City,” Journal of Gerontological Social Work, Vol. 40, No. 4, 2004, pp. 55–71.
112. Breysse, Patrick N., D’An


ter the September 11th World Trade Center Attacks,” Environmental Science and Technology, Vol. 38, No. 13, 2004, pp. 3514–3524.
122. Buyantseva, Larisa V., Mark Tulchinsky, George M. Kapalka, Vernon Chinchilli, Zhengmin Qian, Robert Gillio, Arthur J.


–362.
131. Caruana, Dennis L., Po Hsuan Huang, Jonathan C. Li, Keely Cheslack-Postava, and Anthony M. Szema, “Allergy and Lung Injury Among Rescue Workers Exposed to the World Trade Center Disaster Assessed 17 Years After Exposure to Ground Zero,” Jo


51, No. 1, 2002, pp. 1–5.
87
143. Centers for Disease Control and Prevention, “Potential Exposures to Airborne and Settled Surface Dust in Residential Areas of Lower Manhattan Following the Collapse of the World Trade Center—New York City, November 4


 to the World Trade Center Attacks, PTSD Symptomatology, and Suicidal Ideation,” Journal of Traumatic Stress, Vol. 24, No. 5, 2011, pp. 526–529.
154. Chemtob, Claude M., Yoko Nomura, and Robert A. Abramovitz, “Impact of Conjoined Exposure to the Worl


Chiu, Sydney, Mayris P. Webber, Rachel Zeig-Owens, Jackson Gustave, Roy Lee, Kerry J. Kelly, Linda Rizzotto, Rita McWilliams, John K. Schorr, Carol S. North, and David J. Prezant, “Performance Characteristics of the PTSD Checklist in Retired Firefig


lementi, Emily A., Angela Talusan, Sandhya Vaidyanathan, Arul Veerappan, Mena Mikhail, Dean Ostrofsky, George Crowley, James S. Kim, Sophia Kwon, and Anna Nolan, “Metabolic Syndrome and Air Pollution: A Narrative Review of Their Cardiopulmonary Effec


r Subsequent Decline in Function in a Sample of World Trade Center Responders,” Scientific Reports, Vol. 9, No. 1, 2019.
183. Clouston, Sean A. P., Jack Guralnik, Roman Kotov, Evelyn Bromet, and Benjamin J. Luft, “Functional Limitations Among Respond


chard E. Peltier, Judith Zelikoff, and Lung-chi Chen, “Acute High-Level Exposure to WTC Particles Alters Expression of Genes Associated with Oxidative Stress and Immune Function in the Lung,” Journal of Immunotoxicology, Vol. 12, No. 2, 2015, pp. 140


. Cone, James E., Sukhminder Osahan, Christine C. Ekenga, Sara A. Miller-Archie, Steven D. Stellman, Monique Fairclough, Stephen M. Friedman, and Mark R. Farfel, “Asthma Among Staten Island Fresh Kills Landfill and Barge Workers Following the Septemb


Services, Psychiatric Services, Vol. 57, No. 9, 2006, pp. 1313–1315.
95
212. Cowell, Whitney J., Sally A. Lederman, Andreas Sjödin, Richard Jones, Shuang Wang, Frederica P. Perera, Richard Wang, Virginia A. Rauh, and Julie B. Herbstman, “Prenatal Ex


pi, Robert, Elizabeth Brondolo, “Posttraumatic Stress Disorder Post 9/11: A Review of the Evidence and Implications for Public Health Policy,” TPM-Testing, Psychometrics, Methodology in Applied Psychology, Vol. 24, No. 3, 2017, pp. 363–378.
223. Cruz


. Crane, Julia R. Kaplan, Roberto G. Lucchini, Benjamin J. Luft, Jacqueline M. Moline, Susan L. Teitelbaum, Usha S. Tirunagari, et al., “Cohort Profile: World Trade Center Health Program General Responder Cohort,” International Journal of Epidemiolog


 Artery Diameter Is Associated with Reduced FEV1 in Former World Trade Center Workers,” Clinical Respiratory Journal, Vol. 13, No. 10, 2019, pp. 614–623.
244. de la Hoz, Rafael E., Xiaoyu Liu, Juan C. Celedón, John T. Doucette, Yunho Jeon, Anthony P


ckbill, Steven D. Stellman, and Mark R. Farfel, “Comparison of Health Outcomes Among Affiliated and Lay Disaster Volunteers Enrolled in the World Trade Center Health Registry,” Preventive Medicine, Vol. 53, No. 6, 2011, pp. 359–363.
254. Dekel, Sharo


rld Trade Center Attacks,” Psychological Trauma, Vol. 3, No. 1, 2011, pp. 1–7.
266. Dewart, Tracey, Blanche Frank, and James Schmeidler, “The Impact of 9/11 on Patients in New York City’s Substance Abuse Treatment Programs,” American Journal of Drug


 pp. 264–273.
275. DiMaggio, Charles, Sandro Galea, and Michael Emch, “Spatial Proximity and the Risk of Psychopathology After a Terrorist Attack,” Psychiatry Research, Vol. 176, No. 1, 2010, pp. 55–61.
276. DiMaggio, Charles, Sandro Galea, and Guohu


 City 6 Months After the WTC Attack,” Journal of Traumatic Stress, Vol. 24, No. 5, 2011, pp. 553–556.
103
287. Durmus, Nedim, Yongzhao Shao, Alan A. Arslan, Yian Zhang, Sultan Pehlivan, Maria- Elena Fernandez-Beros, Lisette Umana, Rachel Corona, Shei


and Robin Herbert, “Quality of Spirometry Performed by 13,599 Participants in the World Trade Center Worker and Volunteer Medical Screening Program,” Respiratory Care, Vol. 55, No. 3, 2010, pp. 303–309.
297. Eskenazi, Brenda, Amy R. Marks, Ralph Cata


Christina C. Ekenga, Sukhminder Osahan, Sharon E. Perlman, Lisa M. Gargano, Jennifer Imasuen, and Mark R. Farfel, “Relationship Between Persistent Posttraumatic Stress Disorder and Human Remains Exposure for Staten Island Barge and Landfill Recovery


enihan, Marcella M. Johnson, Vandana Karri, and C. V. R. Reddy, “Cardiac Sequelae in Brooklyn After the September 11 Terrorist Attacks,” Clinical Cardiology, Vol. 29, No. 1, 2006, pp. 13–17.
316. Fernandez, William G., Sandro Galea, Jeff Miller, Jenn


o. 1, 2006, pp. 5–14.
325. Freedman, T. G., “Voices of 9/11 First Responders: Patterns of Collective Resilience,” Clinical Social Work Journal, Vol. 32, No. 4, 2004, pp. 377–393.
326. Friedberg, Fred, Jenna L. Adamowicz, Indre Caikauskaite, Anthony N


r, Raluca M., Jeffrey S. Simons, Gerald A. Jacobs, David Meyer, and Erika Johnson-Jimenez, “Coping Motives and Trait Negative Affect: Testing Mediation and Moderation Models of Alcohol Problems Among American Red Cross Disaster Workers Who Responded


88–795.
346. Garcia, Evangeline, and Denise A. Horton, “Supporting the Federal Emergency Management Agency Rescuers: A Variation of Critical Incident Stress Management,” Military Medicine, Vol. 168, No. 2, 2003, pp. 87–90.
347. Gard, Betsy A., Josef


“Review of Non-Respiratory, Non-Cancer Physical Health Conditions from Exposure to the World Trade Center Disaster,” International Journal of Environmental Research and Public Health, Vol. 15, No. 2, 2018.
358. Gargano, Lisa M., Ho Ki Mok, Melanie H.


h Wicks, Michaeline Bresnahan, Cristiane S. Duarte, and Christina Hoven, “Comorbidity Classes and Associated Impairment, Demographics and 9/11-Exposures in 8,236 Children and Adolescents,” Journal of Psychiatric Research, Vol. 96, 2018, pp. 171–177.


Glaser, Michelle S., Neomi Shah, Mayris P. Webber, Rachel Zeig-Owens, Nadia Jaber, David W. Appel, Charles B. Hall, Jessica Weakley, Hillel W. Cohen, Lawrence Shulman, Kerry Kelly, and David Prezant, “Obstructive Sleep Apnea and World Trade Center E


ional Journal of Cancer, Vol. 145, No. 6, 2019, pp. 1504–1509.
387. Graeff-Martins, Ana Soledade, Christina W. Hoven, Ping Wu, Fan Bin, and Cristiane S. Duarte, “Use of Mental Health Services by Children and Adolescents Six Months After the World Tra


edy: Placing Children’s Reactions to Trauma in a New Context,” Cognitive and Behavioral Practice, Vol. 9, No. 4, 2002, pp. 286–295.
399. Haen, Craig, “Rebuilding Security: Group Therapy with Children Affected by September 11,” International Journal o


treach and Peer-Delivered Mental Health Services in New York City Following September 11, 2001,” Psychiatric Rehabilitation Journal, Vol. 32, No. 2, 2008, pp. 117–123.
409. Harville, Emily, Xu Xiong, and Pierre Buekens, “Disasters and Perinatal Healt


efighters,” Chest, Vol. 153, No. 1, 2018, pp. 114–123.
118
420. Henley, Robert, Randall Marshall, and Stefan Vetter, “Integrating Mental Health Services into humanitarian Relief Responses to Social Emergencies, Disasters, and Conflicts: A Case Study,


Brennan, and Lisa Colarossi, “Event-Exposure Stress, Coping, and Psychological Distress Among New York Students at Six Months After 9/11,” Anxiety, Stress, and Coping, Vol. 23, No. 2, 2010, pp. 153–163.
431. Hill, Matthew N., Linda M. Bierer, Iouri M


7.
440. Hoven, Christina W., Cristiane S. Duarte, Wu P, et al., Parental Exposure to Mass Violence and Child Mental Health: The First Responder and WTC Evacuee Study, Clinical Child and Family Psychology Review, Vol. 12, No. 2, 2009, pp. 95–112.
441.


450. Jack, Katherine, and Sherry Glied, “The Public Costs of Mental Health Response: Lessons from the New York City Post-9/11 Needs Assessment,” Journal of Urban Health, Vol. 79, No. 3, 2002, pp. 332–339.
451. Jackson, Carlos T., George Allen, Susan


nn Difede, Ellen Klausner, and Lisa Spielman, “Are Male Disaster Workers With Vietnam Military Service at Greater Risk for PTSD than Peers Without Combat History?” Journal of Nervous and Mental Disease, Vol. 194, No. 11, 2006, pp. 859–863.
462. Jayas


ty Among Rescue and Recovery Workers and Community Members Exposed to the September 11, 2001 World Trade Center Terrorist Attacks, 2003–2014,” Environmental Research, Vol. 163, 2018, pp. 270–279.
470. Jordan, Hannah T., Steven D. Stellman, Alfredo Mo


al, Vol. 111, 2018, pp. 268–278.
480. Katz, Craig L., Natalie Gluck, Andrea Maurizio, and Lynn E. DeLisi, “The Medical Student Experience with Disasters and Disaster Response,” CNS Spectrums, Vol. 7, No. 8, 2002, pp. 604–610.
125
481. Katz, Craig L.,


rigan, Steven B. Markowitz, Jacqueline M. Moline, David Z. Savitz, Andrew C. Todd, Iris G. Udasin, and Juan P. Wisnivesky, “Increased Rates of Asthma Among World Trade Center Disaster Responders,” American Journal of Industrial Medicine, Vol. 55, No.


imenez, Adam Gonzalez, Kathryn Guerrera, Julia Kaplan, Jacqueline Moline, et al., “Posttraumatic Stress Disorder and the Risk of Respiratory Problems in World Trade Center Responders: Longitudinal Test of a Pathway,” Psychosomatic Medicine, Vol. 77,


ack,” Social Service Review, Vol. 93, No. 1, 2019, pp. 96–128.
510. Kung, Winnie W., Xinhua Liu, Emily Goldmann, Debbie Huang, Xiaoran Wang, Keon Kim, Patricia Kim, and Lawrence H. Yang, “Posttraumatic Stress Disorder in the Short and Medium Term Fol


yed H. Haider, George Crowley, Erin J. Caraher, Dean F. Ostrofsky, Angela Talusan, Sophia Kwon, David J. Prezant, Yuyan Wang, Mengling Liu, and Anna Nolan, “Synergistic Effect of WTC-Particulate Matter and Lysophosphatidic Acid Exposure and the Role


errorist Attack,” International Journal of Emergency Mental Health, Vol. 8, No. 4, 2006, pp. 267–274.
529. Lederman, Sally Ann, Mark Becker, Stephen Sheets, Janet Stein, Deliang Tang, Lisa Weiss, Frederica P. Perera, “Modeling Exposure to Air Polluti


en Attending Prenatal Care,” Journal of Prenatal and Perinatal Psychology and Health, Vol. 19, No. 4, 2005, pp. 275–288.
539. Li, Jiehui, Howard E. Alper, Lisa M. Gargano, Carey B. Maslow and Robert M. Brackbill, “Re-Experiencing 9/11-Related PTSD Sy


r Tissue Samples from World Trade Center Responders,” Journal of Translational Medicine, Vol. 16, No. 1, 2018.
548. Lieberman-Cribbin, Wil, Stephanie Tuminello, Christina Gillezeau, Maaike van Gerwen, Rachel Brody, David J. Mulholland, Lori Horton, M


ast of the World Trade Center (WTC) in Lower Manhattan After the Collapse of the WTC 11 September 2001,” Environmental Health Perspectives, Vol. 110, No. 7, 2002, pp. 703–714.
134
558. Lipkind, Heather S., Allison E. Curry, Mary Huynh, Lorna E. Thorp


orld Trade Center Exposure on the Timing of Diagnoses of Obstructive Airway Disease, Chronic Rhinosinusitis, and Gastroesophageal Reflux Disease,” Frontiers in Public Health, Vol. 5, No. 2, 2017.
568. Locke, Sean, Angela-Maithy Nguyen, Liza Friedman,


No. 5, 2012, pp. 1069–1079.
577. Lurie, Jacob M., Hazel Lever, Lucy Goodson, Daniel Jack Lyons, Robert T. Yanagisawa, and Craig L. Katz, “Instilling Hope and Resiliency: A Narrative Photo-Taking Intervention During an Intercultural Exchange Involving


n, Mana, Jiehui Li, Mark R. Farfel, Carey B. Maslow, Sukhminder Osahan, and Steven D. Stellman, “Adolescent Behavior and PTSD 6–7 Years After the World Trade Center Terrorist Attacks of September 11, 2001,” Disaster Health, Vol. 2, Nos. 3–4, 2014, pp


l Qualitative Study,” Social Work Research, Vol. 31, No. 2, 2007, pp. 121–126.
597. Matthieu, Monica M., André Ivanoff, Sarah Lewis, and Kathryn Conroy, “Social Work Field Instructors in New York City After 9/11/01: Impact and Needs Resulting from t


l and Professional Experiences of 9/11 Mental Health Workers,” Social Work in Mental Health, Vol. 18, No. 6, 2020.
609. Mendelson, David S., Mark Roggeveen, Stephen M. Levin, Robin Herbert, and Rafael de la Hoz, “Air Trapping Detected on End-Expirato


G. Crawford, S. James, C. L. Katz, L. E. Crowley, et al., “Post-Traumatic Stress Disorder Dimensions and Asthma Morbidity in World Trade Center Rescue and Recovery Workers,” Journal of Asthma, Vol. 54, No. 7, 2017, pp. 723–731.
141
620. Moir, William


dies, Vol. 15, No. 6, 2006, pp. 733–743.
629. Moynihan, Patrick J., Jeffrey M. Levine, and Orlando Rodriguez, “The Experiences of Project Liberty Crisis Counselors in the Bronx,” Community Mental Health Journal, Vol. 41, No. 6, 2005, pp. 665–673.
630


nd Sandro Galea, “Patterns and Predictors of Trajectories of Depression After an Urban Disaster,” Annals of Epidemiology, Vol. 19, No. 11, 2009, pp. 761–770.
640. Napier, Camille O., Ositadinma Mbadugha, Laura A. Bienenfeld, John T. Doucette, Roberto


e,” New Solutions, Vol. 18, No. 1, 2008, pp. 3–22.
649. Nicosia, Gregory J., Lorna Minewiser, and Amanda Freger, “World Trade Center: A Longitudinal Case Study for Treating Post Traumatic Stress Disorder with Emotional Freedom Technique and Eye Movem


ol. 163, No. 6, 2009, pp. 531–539.
658. Norris, Fran H., Sheila A. Donahue, Chip J. Felton, Patricia J. Watson, Jessica L. Hamblen, and Randall D. Marshall, “A Psychometric Analysis of Project Liberty’s Adult Enhanced Services Referral Tool,” Psychia


yees of New York City Companies Affected by the September 11, 2001 Attacks on the World Trade Center,” Disaster Medicine and Public Health Preparedness, Vol. 5, Supp. 2, 2011, pp. S205–S213.
669. Offenberg, J. H., S. J. Eisenreich, L. C. Chen, M. D.


Heyde, Dori Reissman, and Vansh Sharma, “The Enduring Mental Health Impact of the September 11th Terrorist Attacks: Challenges and Lessons Learned,” Psychiatric Clinics of North America, Vol. 36, No. 3, 2013, pp. 417–429.
678. Paasivirta, Maria E., C


 709–714.
689. Perera, Frederica P., Deliang Tang, Virginia Rauh, Kristen Lester, Wei Yann Tsai, Yi Hsuan Tu, Lisa Weiss, Lori Hoepner, Jeffrey King, Giuseppe Del Priore, and Sally Ann Lederman, “Relationships Among Polycyclic Aromatic Hydrocarbon-DN


garet Altemus, Moonseong Heo, and Hong Jiang, “Salivary Cortisol and Psychopathology in Children Bereaved by the September 11, 2001 Terror Attacks,” Biological Psychiatry, Vol. 61, No. 8, 2007, pp. 957–965.
700. Pfeffer, Cynthia R., Margaret Altemus,


R. Ozbay, V. Sharma, et al., “Dimensional Structure and Course of Post-Traumatic Stress Symptomatology in World Trade Center Responders,” Psychological Medicine, Vol. 44, No. 10, 2014, pp. 2085–2098.
151
711. Pietrzak, R. H., A. Feder, R. Singh, C. B


mas K. Aldrich, and Kerry J. Kelly, “Cough and Bronchial Responsiveness in Firefighters at the World Trade Center Site,” New England Journal of Medicine, Vol. 347, No. 11, 2002, pp. 806–815.
722. Pridemore, William A., Adam Trahan, and Mitchell B. Ch


 Community Members Exposed to World Trade Center Dust and Fumes,” International Journal of Environmental Research and Public Health, Vol. 17, No. 18, 2020.
732. Reibman, Joan, Nomi Levy-Carrick, Terry Miles, Kimberly Flynn, Catherine Hughes, Michael


ng, and Joanna Legerski, “Engagement in Trauma-Specific CBT for Youth Post-9/11,” Journal of Emotional and Behavioral Disorders, Vol. 21, No. 1, 2012.
743. Rodriguez, Jorge J., and Robert Kohn, “Use of Mental Health Services Among Disaster Survivors,


4.
753. Rubacka, Joseph M., James Schmeidler, Yoko Nomura, Rohini Luthra, Khushmand Rajendran, Robert Abramovitz, and Claude M. Chemtob, “The Relationship Between PTSD Arousal Symptoms and Depression Among Mothers Exposed to the World Trade Center At


“Genetic Markers for PTSD Risk and Resilience Among Survivors of the World Trade Center Attacks,” Disease Markers, Vol. 30, Nos. 2–3, 2011, pp. 101–110.
763. Sauler, Maor, and Mridu Gulati, “Newly Recognized Occupational and Environmental Causes of C


 Curve Analysis,” BMC Psychiatry, Vol. 16, 2016.
773. Sciancalepore, Roseann, and Robert W. Motta, “Gender Related Correlates of Posttraumatic Stress Symptoms in a World Trade Center Tragedy Sample,” International Journal of Emergency Mental Health,


rs Associated With Mental Health Outcomes Among World Trade Center Disaster Evacuees,” Disaster Medicine and Public Health Preparedness, Vol. 11, No. 3, 2017, pp. 326–336.
159
784. Sherrieb, Kathleen, and Fran H. Norris, “Public Health Consequences o


. Singh, A., R. Zeig-Owens, C. B. Hall, Y. Liu, L. Rabin, T. Schwartz, M. P. Webber, D. Appel, and D. J. Prezant, “World Trade Center Exposure, Post-Traumatic Stress Disorder, and Subjective Cognitive Concerns in a Cohort of Rescue/Recovery Workers,”


ggi, “Potential Carcinogenic Effects of World Trade Center Dust After Intratracheal Instillation to Sprague-Dawley Rats: First Observation,” American Journal of Industrial Medicine, Vol. 56, No. 2, 2013, pp. 155–162.
805. Sofka, Carla J., “Assessing


e Barlev, Michael Marmor, and Marc Wilkenfeld, “Neurologic Evaluations of Patients Exposed to the World Trade Center Disaster,” Journal of Occupational and Environmental Medicine, Vol. 58, No. 11, 2016, pp. 1150– 1154.
816. Stecker, Mark, Jacqueline


irbrother, “Behavior Problems in New York City’s Children After the September 11, 2001, Terrorist Attacks,” American Journal of Orthopsychiatry, Vol. 75, No. 2, 2005, pp. 190–200.
826. Stuber, Jennifer, Heidi Resnick, and Sandro Galea, “Gender Dispar


 Allergy and Clinical Immunology, Vol. 113, No. 3, 2004, pp. 420–426.
835. Szema, Anthony M., Khalil W. Savary, Benjamin L. Ying, and Kevin Lai, “Post 9/11: High Asthma Rates Among Children in Chinatown, New York,” Allergy and Asthma Proceedings, Vol


ental Health Perspectives, Vol. 116, No. 10, 2008, pp. 1383–1390.
845. Thorpe, Lorna E., Shervin Assari, Stephen Deppen, Sherry Glied, Nicole Lurie, Matthew P. Mauer, Vickie M. Mays, and Edward Trapido, “The Role of Epidemiology in Disaster Response


r, “A Deese- Roediger-McDermott Study of Trauma Memory Among Employees of New York City Companies Affected by the September 11, 2001, Attacks,” Annals of Clinical Psychiatry, Vol. 27, No. 3, 2015, pp. 165–174.
167
856. Trout, Douglas, Ashok Nimgade,


re Science and Environmental Epidemiology, Vol. 27, No. 3, 2017, pp. 306–312.
865. van Gerwen, Maaike A. G., Stephanie Tuminello, Gregory J. Riggins, Thais B. Mendes, Michael Donovan, Emma K. T. Benn, Eric Genden, Janete M. Cerutti, and Emanuela Taio


juana Among Manhattan, New York, Residents After the September 11th Terrorist Attacks,” American Journal of Epidemiology, Vol. 155, No. 11, 2002, pp. 988–996.
875. Vossbrinck, Madeline, Rachel Zeig-Owens, Charles Hall, Theresa Schwartz, William Moir,


e,” Journal of Adult Development, Vol. 9, No. 4, 2002, pp. 241–250.
886. Warren, Tiffany, Shannon Lee, and Stephen Saunders, “Factors Influencing Experienced Distress and Attitude Toward Trauma by Emergency Medicine Practitioners,” Journal of Clinica


. 3, 2005, pp. 355–374.
896. Webber, Mayris P., Michelle S. Glaser, Jessica Weakley, Jackie Soo, Fen Ye, Rachel Zeig-Owens, Michael D. Weiden, Anna Nolan, Thomas K. Aldrich, Kerry Kelly, et al., “Physician-Diagnosed Respiratory Conditions and Mental


ger, et al., “Obstructive Airways Disease with Air Trapping Among Firefighters Exposed to World Trade Center Dust,” Chest, Vol. 137, No. 3, 2010, pp. 566–574.
905. Weiden, Michael D., Sophia Kwon, Erin Caraher, Kenneth I. Berger, Joan Reibman, Willia


m the World Trade Center Health Registry,” Drug and Alcohol Dependence, Vol. 140, 2014, pp. 1–7.
914. Welch, Alice E., Kimberly Caramanica Zweig, Jonathan M. McAteer, and Robert M. Brackbill, “Intensity of Binge Drinking a Decade After the September


nals of Allergy, Asthma and Immunology, Vol. 126, No. 3, 2020.
175
924. Wisnivesky, Juan P., Susan L. Teitelbaum, Andrew C. Todd, Paolo Boffetta, Michael Crane, Laura Crowley, Rafael E. de la Hoz, Cornelia Dellenbaugh, Denise Harrison, Robin Herbert,


phen M. Friedman, and Hannah T. Jordan, “Probable Posttraumatic Stress Disorder and Lower Respiratory Symptoms Among Rescue/Recovery Workers and Community Members After the 9/11 World Trade Center Attacks: A Longitudinal Mediation Analysis,” Psychoso


ticulate that Settled Inside the Surrounding Buildings and Outside on the Streets of Southern New York City After the Collapse of the World Trade Center, September 11, 2001,” Journal of the Air and Waste Management Association, Vol. 54, No. 5, 2004,


on/Irritants, Asthma Control, and Health-Related Quality of Life Among 9/11-Exposed Individuals with Asthma,” International Journal of Environmental Research and Public Health, Vol. 16, No. 11, 2019.
952. Yurt, Roger W., Palmer Q. Bessey, Nicole E. A


lan, et al., “World Trade Center Disaster and Sensitization to Subsequent Life Stress: A Longitudinal Study of Disaster Responders,” Preventive Medicine, Vol. 75, 2015, pp. 70–74.
961. Zvolensky, Michael J., Roman Kotov, Clyde B. Schechter, Adam Gonz


her end users (child code)
– Policymakers external to
Program (e.g., Congress or state governments) (child code)
 181
   Program Documentation A Priori Codebook
     Focus Group Transcripts A Priori Codebook
     Interview Transcripts A Priori Codebo


rs, and advocacy on behalf of the Program (child code)
– Informing Program/research planning (child code)
– Informing individual health care decisions (child code)
– Responding to future disasters (child code)
– Guiding future research (child code)
–


fect Term* (“World Trade Center” OR WTC OR “September 11” OR “ground zero”) AND Exposure Term* (“World Trade Center” OR WTC OR “September 11” OR “ground zero”) AND Translation Term*
  184
*Terms were searched one at a time
govinfo
Limit 2001–present


lly think about the dust and everything else, cleanup, when young kids are 19, 20 years old now, that’s to me I think about it constantly because I can’t even imagine a juvenile rheumatoid arthritis.
There was general agreement across stakeholder com


ents most commonly described three criteria. They would prioritize research according to
• what is most clinically relevant
• who is affected
• the “feasibility of being able to answer the question.”
One researcher described what is meant by clinical


ntific question that’s raised in the grant proposal or the contract proposal. It’s about the feasibility of being able to answer the question. That’s nothing new for the Federal Government or for any funding agency, but it will be of particular impor


Center Health Program Research Grantee Meeting, New York, June 2019.
Connected Papers, homepage, undated. As of September 21, 2021: https://www.connectedpapers.com/
Cooper, Simon, Robyn Cant, Michelle Kelly, Tracy Levett-Jones, Lisa McKenna, Philippa


activity/hearings/hearing-on-examining-hr- 1786-james-zadroga-911-health-and-compensation
House Committee on Energy and Commerce Democratic Staff, “Hearing on ‘Examining H.R. 1786, the James Zadroga 9/11 Health and Compensation Reauthorization Act,’”


’Brien, “Scoping Studies: Advancing the Methodology,” Implementation Science, Vol. 5, 2010.
Liberati, Alessandro, Douglas G. Altman, Jennifer Tetzlaff, Cynthia Mulrow, Peter C. Gøtzsche, John P. A. Ioannidis, Mike Clarke, P. J. Devereaux, Jos Kleijne


1, 2020: https://www.niehs.nih.gov/research/programs/translational/resources/index.cfm
National Institutes of Health, “Cooperative Research Agreements Related to the World Trade Center Health Program (U01),” PAR-12-126, March 23, 2012. As of Septembe


Categorization,” ACM Computing Surveys, Vol. 34, No. 1, March 2002, pp. 1–47.
Selker, Harry P., Laurel K. Leslie, June S. Wasser, Andrew G. Plaut, Ira B. Wilson, and John L. Griffith, “Tufts CTSI: Comparative Effectiveness Research as a Conceptual Fr


er 21, 2021: https://www.webtrends.com/
Wilczynski, Nancy L., R. Brian Haynes, John N. Lavis, Ravi Ramkissoonsingh, and Alexandra E. Arnold-Oatley, “Optimal Search Strategies for Detecting Health Services Research Studies in MEDLINE,” Canadian Medica


rking Group of New York City, 2011 Annual Report on 9/11 Health, New York, November 2011. As of September 20, 201: http://www.nyc.gov/html/doh/wtc/downloads/pdf/news/wtc-mwg-annual-report2011.pdf
WTC—See World Trade Center.
Yip, Jennifer, “Mental Hea


Suggestions for Estimating the Effects of State Gun Policies
Commentary on Four Methodological Problems in the Current Literature
TERRY L. SCHELL, ROSANNA SMART, ANDREW R. MORRAL
 Sponsored by Arnold Ventures
 CORPORATION
For more information on this


ernative explanation is that different researchers may be using methods that are more or less appropriate to the gun policy topics they are investigating. Use of flawed or inappropriate methods can lead to incorrect conclusions, even among the most n


ve are publicly available on the project website at www.rand.org/gunpolicy.
Justice Policy Program
RAND Social and Economic Well-Being is a division of the RAND Corporation that seeks to actively improve the health and social and economic well-being


). In this report, we discuss four common methodological problems that we observed in the gun policy literature and offer suggestions for how future research on gun policies should be conducted, reported, and evaluated.
This report is not intended as


. Some of these challenges are a result of the complexity of the policies being studied or limitations inherent in the available data. For example, classifying policies or laws involves trade-offs between increasing homogeneity among the policies who


or prespecified time points. If the policy effect is operationalized with multiple parameters (e.g., in hybrid models), researchers should present results as the joint effect of the multiple coefficients used to describe the effect. They should not t


 biased—if there are heterogeneous effects across units or time-varying policy effects.
The recommendation to control for confounders, however, should be balanced against the harms associated with overly complex models. Models that are more complex t


ch the treated unit on the outcome and other covariates in the pre-policy period, and the causal effect is estimated by assuming that, in the absence of treatment, the outcomes of the treated and weighted control units would have been identical.
In t


of independent, identically distributed residuals). Although some papers acknowledged these potential issues, they rarely provided empirical evidence to support the authors’ preferred model specification. More often, papers reported several alternat


ings to represent a significance test. Moreover, blocking on the implementation year and the quality of the match, which is what has typically been done in this literature, leaves unaddressed
xi
several known sources of heteroscedasticity (e.g., matc


 reviewed.
• Recommendation 15. Some of the proliferation of alternative tests might be attributed to the curiosity of journal editors and reviewers who wish to see their own preferred specification reported and so require that it be added to article


s are needed to identify the effects of gun policies in a persuasive and replicable way. In this report, we identified four areas that we think may be most ready for these improvements and proposed some ways that these improvements might be made.
xii


in-differences (DID) regression model, most commonly two-way fixed-effects models (i.e., including fixed effects for both the temporal and geographic units of analysis). Less
1 Although not all guns are firearms, in this report, we follow conventiona


ression-based models to identify the effects of gun policies, synthetic control studies have become increasingly common. They also have a somewhat unique set of limitations and interpretational challenges that have not been clearly articulated in the


stard, 1997; Rosengart et al., 2005; Vernick and Hepburn, 2003) have published or shared their coding of laws, which allows for debate and improvement of the coding schemes, such coding often is not transparent and cannot be reviewed to determine acc


re being developed. These include
• the RAND State Firearm Law Database, which, as of this writing, covers 1979 through January 2020 and includes 17 broad classes of state gun policies and many subcategories (Cherney et al., 2020)1
• Boston Universit


used only about 20 times a year to remove firearms from someone who presented an imminent risk to themselves or others. It was not until after the Virginia Tech shooting in 2007 that the law started to be more widely used. With this pattern of uptak


there is no single effect size to report. In fact, for the linear splines used in much of the extant gun policy literature, these models assume that the policy of interest can have effects on the outcome that are extremely large in magnitude—in some


hen the only effect of the policy is an immediate surge in crime following implementation within the treated units, the slope parameter will be negative, not positive.3 Standard frameworks for inferring causality from observations (e.g., Mill, 1843)


tion would be expected to become progressively more extreme after the policy is implemented—the opposite of regression to the mean. In contrast, in data showing regression to the mean, a null hypothesis that the trend before a given
 8
date equals th


e justified as the phase-in period of the policy’s effect (e.g., as used in the preferred specifications in Donohue, 2004, and Schell, Cefalu, et al., 2020). Such splines are similar to dummy-coded variables but with a gradual rather than abrupt tran


ets these three requirements for interpreting the effects. However, in 2020, we and other researchers published a study of the effects of three gun laws that we believe does satisfy these criteria (see Schell, Cefalu, et al., 2020). In that paper, we


the model results as revealing policy effects. Although we focus this chapter on issues of confounding that are most relevant to estimating the causal effects of state firearm policies, these issues are not unique to the study of firearm policies an


2014; Ayres and Donohue, 2003; Donohue, 2003; Donohue, Aneja, and Weber, 2019; Grambsch, 2008; Kovandzic and Marvell, 2003; Kovandzic, Marvell, and Vieraitis, 2005; Lott and Mustard, 1997; Manning, 2003; Plassmann and Whitley, 2003; Rosengart et al.,


ects of firearm policy and requires alternative approaches to those used most commonly in this literature. There are several methods that do not require parallel pre-trends in outcomes but instead assume that expected outcomes for states exposed and


resent in the outcome in the prior year. Thus, the model implies that law effects propagate forward in time even if the model does not contain an explicit effect of the law on the outcomes in those later years.
In general, if researchers include auto


d variable bias) in gun policy research (see, for example, Ayres and Donohue, 2009; Moody and Marvell, 2019), and many candidate confounders have been proposed as potentially important for ensuring valid estimates of a policy’s effects. These candida


Determining which variables are potential confounders and which are
2 In this context, collider refers to candidate covariates that may themselves be affected by implementation of the law and by changes in the outcome. Including colliders as covariat


r all predictors in the model. These problems are greatly exacerbated if one prunes variables based on standard levels of statistical significance (the default in many software packages) rather than on a criterion, such as cross-validated error or th


(2010)—is the synthetic control method, which estimates the causal effect by comparing (1) one analytic unit (e.g., a state) that implemented a policy with (2) a weighted average of other comparison units that did not implement the policy. These wei


nt and control groups.
However, in the gun policy research literature, the synthetic control studies that we examined did not attempt to balance on covariates that are thought to be associated with the treatment. Unless there are a large number of co


ng method in which treatment and control states are matched on those factors (Rubin, 1974). The Donohue, Aneja, and Weber (2019) synthetic control analyses achieved relatively good balance on the pre-implementation trends in the outcomes, but the tre


 In fact, Kaul et al. (2018) showed that the optimization method used to estimate synthetic control weights results in all other covariates being ignored when all pre-treatment outcomes are included as predictors.
Researchers using synthetic control


D, that combines features from both synthetic control and DID methods. Their approach (1) estimates unit-specific weights that define the synthetic control unit based on pre-treatment data, (2) estimates time-specific weights that define the syntheti


ss a variety of modeling approaches commonly used in gun policy evaluations to identify the approach that made the most-appropriate assumptions about the firearm mortality data. By design, that simulation study removed the possibility of selection bi


o severely underestimate the true uncertainty that two-thirds or more of effects known to have no systematic association with the outcome variable appear to be statistically significant, a share far higher than the 5 percent expected for significance


mber of states (or jurisdictions), in which case the standard clustering corrections may not yield accurate estimates of uncertainty (Conley and Taber, 2011; MacKinnon and Webb, 2017). Although a growing methodological literature has sought to derive


 the pre-implementation period
• for states with more-prevalent count outcomes (e.g., for states with higher homicide
rates) or with dichotomous outcomes with prevalence closer to 50 percent.
In addition, the variance will depend on the specific year


on all these factors and still have sufficient permutations to get a useful estimate of the p value in an analysis of state-level panel data of gun violence. Without blocking on those factors, a gun law passed in a small state with a high prevalence


 two- tailed). In addition to the mathematical limit on the number of possible quantiles that can be estimated in the data, there is a broader question about the accuracy of those quantiles, given their estimation in such a small sample.
2 Models tha


rarely used in the studies that we examined in The Science of Gun Policy. Additionally, such procedures further compromise statistical power, which might contraindicate their use in this field even if they are generally useful elsewhere. Because ther


ignificant effects may often get the direction of the effect wrong (Gelman and Carlin, 2014). These serious problems are common whenever the effects of interest are small relative to other sources of variation in the outcomes (Cohen, 1988).
Neverthel


re generally, significance tests conducted on studies in which the effect size of interest is small relative to the standard error of the estimate provided by the available data have almost no chance of improving public policy and have a non-negligib


o discover are likely to be exaggerated in their magnitude and, in the worst cases, are almost as likely to be in the wrong direction as the right one (Gelman and Carlin, 2014). Insufficient power may be slightly less of a concern in studies that eva


licy literature that we reviewed often have substantially lower power because of even less-precise estimates of causal effects. The permutation test results from the synthetic control method studies included in our literature review correspond to an


to the lives of so many.
Furthermore, we have not offered a comprehensive primer on estimating gun policy effects but instead focused our discussion on a few common problems that we have seen repeatedly in the gun policy research studies that we have


ven those with a good number of studies seemingly confirming their effects.
Yet these limitations of the available evidence should not be seen as an excuse for inaction by policymakers. When strong empirical evidence of the effects of policies is ava


ssion equation that takes on values of zero before the policy was implemented (as well as in states that never implemented it) and then takes on values that increase linearly over time for a given state once the policy is implemented in that state. F


1 2000 30 10 1 2001 31 11 1
    36
References
Abadie, Alberto, “Using Synthetic Controls: Feasibility, Data Requirements, and Methodological Aspects,” Journal of Economic Literature, Vol. 59, No. 2, 2021, pp. 391–425.
Abadie, Alberto, Alexis Diamond,


vi Feller, and Jesse Rothstein, “The Augmented Synthetic Control Method,” Journal of the American Statistical Association, Vol. 116, No. 536, 2021a, pp. 1789–1803.
———, Synthetic Controls with Staggered Adoption, Cambridge, Mass.: National Bureau of


,” Causal Inference: The Remix, March 8, 2021.
Daw, Jamie R., and Laura A. Hatfield, “Matching and Regression to the Mean in Difference-in- Differences Analysis,” Health Services Research, Vol. 53, No. 6, December 2018, pp. 4138– 4156.
de Chaisemarti


tational Statistics, Vol. 15, No. 3, September 2000, pp. 373–390.
Gibbons, Charles E., Juan Carlos Suárez Serrato, and Michael B. Urbancic, “Broken or Fixed Effects?” Journal of Econometric Methods, Vol. 8, No. 1, 2019.
Giffords Law Center to Preven


d Kevin Lang, “The Promise and Pitfalls of Differences-in-Differences: Reflections on 16 and Pregnant and Other Applications,” Journal of Business & Economic Statistics, Vol. 38, No. 3, 2020, pp. 613–620.
Kaul, Ashok, Stefan Klößner, Gregor Pfeifer,


ess, 2003, pp. 331–341.
Manski, Charles F., and John V. Pepper, “How Do Right-to-Carry Laws Affect Crime Rates? Coping with Ambiguity Using Bounded-Variation Assumptions,” Review of Economics and Statistics, Vol. 100, No. 2, 2018, pp. 232–244.
McShan


sis of Research Evidence on the Effects of Gun Policies in the United States, Santa Monica, Calif., RR-2088-RC, 2018. As of December 30, 2021:
https://www.rand.org/pubs/research_reports/RR2088.html
Riley, Richard D., Kym I. E. Snell, Joie Ensor, Dani


orporation, TL-354-LJAF, 2020. As of May 29, 2021: https://www.rand.org/pubs/tools/TL354.html
Shadish, William R., Thomas D. Cook, and Donald T. Campbell, Experimental and Quasi- Experimental Designs for Generalized Causal Inference, 2nd ed., Boston,


shington, D.C.: Brookings Institution Press, 2003, pp. 345–402.
Wasserstein, Ronald L., Allen L. Schirm, and Nicole A. Lazar, “Moving to a World Beyond ‘p < 0.05,’” American Statistician, Vol. 73, Supp. 1, 2019, pp. 1–19.
46
Wegman, Edward J., and Ia


Spending Trajectories After Age 65
Variation by Initial Wealth MICHAEL D. HURD, SUSANN ROHWEDDER
 Sponsored by Insight Investment
 CORPORATION
For more information on this publication, visit www.rand.org/t/RRA2355-1.
About RAND
The RAND Corporation i


iversity of Michigan Health and Retirement Study. The report presents estimates of the trajectories of spending after age 65 among single and coupled households stratified by wealth holdings observed at or closely following age 65. The estimated traj


recognize that spending needs are not constant over the course of retirement. Furthermore, there is heterogeneity across households with respect to the evolution of spending with age. Many households may be able to anticipate the cost of their desire


hange to help determine adequate saving rates during the working life and affordable spending levels during retirement.
Figure S.1. Estimated Spending Trajectories for Single Households
     Percent of total spending at age 65
100
 90
 80
 70
 60
 50


the surviving spouse: At advanced old age, the marginal utility of wealth to the surviving spouse declines because of their short life expectancy, causing the couple to consume more earlier. Furthermore, the spending path of the household will decli


ending on some categories will decline with age, they will bring spending forward, which would steepen any decline due to mortality risk.1
An alternative explanation for declining budget shares is a worsening of economic resources with age, which wou


mortality and possible cohort differences. A long-running panel is necessary because of macro shocks that could dominate over short periods. Furthermore, we use data on the detailed composition of spending to study budget shares and what those might


 categories were split to distinguish spending on goods versus spending on services. CAMS waves 2005 through 2019 queried spending in 39 categories.
  4
Analytic Sample and Derivation of Initial Wealth Quartile
We derive our analytic sample in two st


seholds in our analytic sample come from multiple birth cohorts, spanning birth years 1917 to 1952 (median: 1936); therefore, the year when individuals in our sample were observed near retirement (ages 65 through 69) varies. To summarize households’


.1 shows real two-year spending changes for couples and singles, by age band, over years 2005 to 2019, or seven changes. To reduce the influence of outliers, we show medians within age-band-wave cells of the rates of change. The changes are calculate


west quartile as reference. The three specifications differ in how age is modeled. In the columns labeled “Age Bands,” age is entered as categorical variables with ages 65 through 69 as reference. In the “Age Continuous” columns, age is entered linea


088 0.0097
0.0102
       2nd 3rd Highest
Male spouse’s age 65–69
70–74 75–79 80+
Male spouse’s age—65
Female spouse’s age is 2+ years less than male spouse’s age Constant
0.0246 0.0185 0.0101 0.0176 0.0099 0.0183
(Ref.) (Ref.) 0.0309* 0.0176
−0.0142


uding age bands as controls
  Percent of spending at age 65
          65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90
  100
 90
 80
 70
 60
 50
 40
 30
 20
 10
  0
OLS estimation, linear age control
  Percent of total sp


icted spending path for households in which female spouse’s age is less than two years younger than the male.
  Percent of spending at age 65
65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 lower mid upper
          18
C


d a fixed effects estimator and modeled spending as a quadratic in age. However, the fitted shape of the spending path in a quadric specification will be dominated by the large sample sizes at younger ages, thus obscuring the spending declines at old


ctories, are based on historical rates of change. Although our choice of functional forms and explanatory variables are informed by economic reasoning, our data are not fit to a structural model based on utility maximization with explicit modeling of


g panel of the HRS, we estimated survival differentials by age, sex, marital status, and education. According to our estimates, a single female lacking a high school education has a 50 percent chance of surviving to about age 80; a married female wit


dual trajectories would average out to the population average of −0.04. Separating the models requires linking together individual-level rates of change over more periods and the direct modeling of individual heterogeneity. This in turn requires a mo


dget constraints because even those in the top quartile, who are the least constrained, exhibit a declining path. We used the variation in budget shares to support an alternative hypothesis: Worsening health associated with aging causes declining mar


  sizes across age bands are insufficient for robust statistical analyses.
24
Appendix B. CAMS Spending Categories
The CAMS elicits household spending in 39 categories, which we aggregated into broader categories (see Table B.1).
Table B.1. Aggregat


tion.
Table C.1. Persistence of Wealth Quartile Between Ages 65 Through 69 and 70 Through 74
Wealth Wealth Quartile, Ages 70 Through 74 Quartile,
Ages 65
Through 69 Lowest 2nd 3rd Highest All
Lowest 20.8 5.1 1.2 0.2 27.4 2nd 6.3 13.8 5.1 1.1 26.3 3rd


Single Households
  spending at t in each age band
• median spending ratio at the population level: S med , where S med is the population median
St t+2
Smed t t
   100
 90
 80
 70
 60
 50
 40
 30
 20
 10
  0
                    female
male female Low


irement Study: Data Quality and Early Results, 1995.
RAND Corporation, “RAND HRS CAMS Data File 2019 (V1),” dataset produced by the RAND Center for the Study of Aging, January 2022a.
RAND Corporation, “RAND HRS Longitudinal File 2018 (V2),” dataset p


Selecting and Evaluating Case Studies of the Economic Benefits of Research and Services at the National Institute for Occupational Safety and Health
Case Studies on Personal Dust Monitors for Coal Miners, Improved Ambulance Design, and Amputation Sur


mental Policy
RAND Social and Economic Well-Being is a division of the RAND Corporation that seeks to actively improve the health and social and economic well-being of populations and communities throughout the world. This research was conducted in t


and Productivity Losses Attributable to Individual
Cases of Fatal and Nonfatal Respiratory Disease ................................................................. 43 Table 2.9. Estimated Avoided Medical Costs and Productivity Losses for Cases of Fa


h three exploratory case studies. This report builds upon that work by developing a process for selecting case studies for evaluation, applying that selection process to a list of ten potential case studies, and selecting three case studies from this


al cases avoided by age 73 or 85 and whether a 3-percent or 7-percent discount rate is used.
In the Improved Ambulance Design case, NIOSH’s impact on ambulance design has been the result of a steady stream of partnerships with stakeholders. NIOSH has


assachusetts. The program also provided better information on the extent of amputations in Michigan and in which industries and firms amputations had occurred that were not reported in other sources.
 While our estimates are not causal, if we assume


ntion coal mine dust personal sampler unit chronic obstructive pulmonary disease continuous personal dust monitor
Coal Workers’ Health Surveillance Program
coal worker’s pneumoconiosis, a.k.a. “black lung” Department of Transport
Enhanced Coal Worker


e difficult to accurately quantify the impact of activities on the underlying risk or associated outcomes. Second, benefits may not accrue until many years in the future. Third, much of NIOSH’s work is done in collaboration with many government, acad


tween them. RAND decided which case studies to pursue, although as the methodology clearly notes, NIOSH’s institutional priorities were one of several important factors in determining which cases to evaluate in detail. Given the need for detailed inf


f activities aimed at producing outputs that will help mitigate the hazard. Examples of NIOSH activities include research, surveillance, and training. These activities lead to outputs such as new information about the causes of a hazard, the identifi


s is exposed, such as a case study that evaluates the impact of several activities aimed at reducing a certain type of injury for a certain type of worker. Alternatively, a case might group several related hazards, such as a case study that evaluates


observed an increase in prevalence of coal workers’ pneumoconiosis (CWP, or “black lung”) in 2005, 2006, and 2007 (and more cases in younger miners).
Oil and gas workers can be exposed to hydrocarbon gases and vapors (HGV) and oxygen (O2)-deficient a


loss is the most common occupational disease among workers in the mining industry. Hazardous noise exposures are more prevalent in this sector than in any other major U.S. industrial sector. NIOSH studies have shown that ca. 90% of coal miners and 4


s provided funds to start regional safety training programs for commercial fishermen, although NOAA funding ended many years ago. Today, most funding comes from the Coast Guard, NIOSH, course fees, and locally raised funds.
    7
Method for Selecting


t areas of particular interest to NIOSH. Another institutional factor might be the consideration of which case studies NIOSH can complete in-house versus which case studies are best suited for evaluation by an external independent research organizati


of nine recorded fatalities (and an unrecorded number of injuries) between January of 2010 and March of 2015. New industry standards and proposed changes to federal regulations could promote safer methods for crude oil measurement, but even the large


eletal Diseases in Poultry Processing Plants, Reduction in Exposure to Hazardous Drugs in Health Care, and Alaska Commercial Fishing Training—were deemed likely to face significant feasibility challenges, and thus appear on the right-hand side of the


that would be selected should each evaluate different groups of workers to meet priorities over balance; hence we would need to choose between the two cases related to mining. We selected Personal Dust Monitors for Coal Miners over Mining Hearing Los


Co., Inc. (R&P), allows miners to take corrective actions based on these measurements to reduce occupational exposure to respirable coal mine dust. NIOSH, through an informal partnership with industry, labor, and MSHA, collaborated in product develo


e coal mine dust. Finally, we provide estimates of the economic benefits of the illnesses and fatalities likely to be averted due to the adoption of CPDMs and consider NIOSH’s role in contributing to these benefits.
The Cost of Black Lung
The costs t


9), which established the Mining Enforcement and Safety Administration within the Department of the Interior. The Coal Act established a mandatory federal exposure limit for the average concentration of respirable dust in underground coal mines of 3.


 exams to miners in the field and provide early detection of dust-related pulmonary disease, as well as target additional areas for prevention. Both surveillance programs are provided at no cost to miners, but participation is voluntary.
Through the


 industry, labor, and MSHA to develop and test the
prototype devices in laboratory and mine settings.
2004–2008 NIOSH published two reports demonstrating the CPDM provided accurate readings of a miner’s dust exposure and was wearable and durable duri


 monitors—and once the technology has been verified, to use these monitors in conjunction with other sampling methods for surveillance.
In response to the first report, NIOSH’s Pittsburgh Research Center (PRC) initiated the development of a fixed-sit


o Fisher Scientific conducted regional workshops in 2010 to help coal miners learn how to use the new instrument. NIOSH awarded another contract to Thermo Fisher Scientific in 2012 to revise the design and make the device lighter to carry. A modified


 CPDMs in April 2016. In August 2016, MSHA lowered the PEL for respirable coal mine dust from 2.0 mg/m3 to 1.5 mg/m3 of air for underground and surface coal mines, and from 1.0 mg/m3 to 0.5 mg/m3 for intake air at underground mines and for Part
90 mi


 furthest away when
downwind of the continuous miner
 Staying out of the return air when not bolting
 Alternating changing of dust collection bags so that
one person is not getting all of this specific unavoidable dust exposure
 Rock dusting more


s of NIOSH’s technological contribution to reducing the risk of exposure to high concentrations of respirable coal mine dust, we compare the reduction in negative health outcomes associated with observed reductions in exposure levels and the reductio


tion into categories that will later be used as the framework for estimating changes in exposure. Not all miners are exposed to high concentrations of respirable coal mine dust. In the regulatory analysis for the coal mine dust rule, MSHA estimated t


ile we do not have sufficiently detailed information to identify coal rank for each individual mine in MSHA’s Mines dataset, we use the distribution of WLs by coal rank within each occupation from MSHA’s QRA (MSHA, 2013).8 To estimate the distributio


nce
Low-/Med.- High-Rank
Rank Bituminous Anthracite
Mid-Recurrence
Low-/Med.- High-Rank
Rank Bituminous Anthracite
High-Recurrence
   Auger Operator
Continuous Miner Operator
Cutting Machine Operator
Drill Operator
Electrician and Helper
Laborer
Load


 July 2016, operator samples were collected using the CPDM, and the PEL was 2.0 mg/m3 of air (1.0 mg/m3 for Part 90 miners).
3. From August 2016 to December 2018, operator samples were collected using the CPDM, and the PEL was 1.5 mg/m3 of air (0.5 m


ample these occupations during mine inspections. Prior to 2016, operator samples accounted for slightly more than half of all samples collected, but due to increased sampling requirements operator samples now account for more than 80 percent of all s


y selectively sampling on lower production shifts/days because a higher number of valid samples was required for each calendar quarter.
Table 2.5. Number (and Percentage) of Coal Dust Samples Exceeding the PEL
 Population
Surface workers
Underground


rue for occupations that collect relatively few samples each period. For groups such as “other underground workers,” the distribution of measurements varies considerably over time because it reflects a different mix of job titles and tasks each perio


were able to continually monitor the device to make decisions to continue working.
An alternate explanation is that the data reflect a downward bias in the operator samples prior to 2016. In practice, while equivalent in accuracy to CMDPSUs, CPDMs le


ay, this implies that sampling using CPDMs may occur for as little as two to four weeks for a designated occupation during a calendar quarter. However, the benefits of lessons learned through end-of-shift measurements and adoption of better work prac


ary to July 2016 because we use quarterly and not monthly data due to sampling limitations and because most mines delayed the adoption of CPDMs until April 2016. Therefore, we use the second and third calendar quarters of 2016 to estimate the impact


ponse to using CPDMs (Haas and Colinet, 2018).
The before and after estimation is calculated as follows:
           (Equation 1)
where Ct is the average dust concentration at time t, 0 is a constant, PDM equals 1 if a sam


.e., areas with a high number of exceedances).21 This effect is attributable to the CPDM providing both continuous real-time measurement that allows miners to move further away from the source and end-of-shift measurements that allow miners to take c


disease per 1,000 miners. Based on these models, we estimate impacts for five major adverse health conditions:
1. CWP category 1 (CWP 1+): Based on the International Labour Organization (ILO) International Classification of Radiographs of Pneumoconio


 NMRD.
First, we estimate the number of excess cases of morbidity and mortality due to respiratory disease over a 45-year job tenure, from age 20 to 65, under baseline mining conditions compared with the probability of death without occupational expo


equivalent to a 5.4 percent reduction in exposure to respirable coal mine dust. We also present a sensitivity analysis showing the effect of using a relatively higher estimate of a 9.2 percent reduction, which is between the 5.4 and 13.0 percent est


and new workers entering the workforce.26 The benefits of using a single probability distribution are twofold: it avoids the need to make additional assumptions about workforce replacement and disease latency and progression, and estimates a smooth,


nek, 2019). Many miners wait until near retirement age (around their late 50s) before seeking diagnosis or treatment for CWP or starting the process of qualifying for federal black lung benefits.29 Therefore, we use 20 as an approximation for years o


is not considered to be disabling, but likely will result in a significant health impediment for coal miners. PMF and severe emphysema can both result in totally disability and inability to work. For cases of CWP 2+, PMF, and COPD, we use estimates


r cases of CWP 1+ based on the value of lost workday injury. For cases of CWP 2+, we use an estimate of approximately $490,000 (in 2018 dollars) from the EPA’s Office of Air and Radiation for avoided cases of chronic bronchitis (EPA, 2011b). For case


attribute a specific percentage of the monetized benefits of MSHA’s coal mine dust rule to NIOSH. Furthermore, it is challenging to separate the impacts of requiring the use of CPDMs and mandating a reduction in the PEL as both provisions of the rul


e risk models described in this chapter, this larger estimated reduction results in approximately a 40-percent increase in the number of cases of fatal and nonfatal respiratory disease prevented and a corresponding 40- to 45-percent increase in monet


eved and after 40 years approximately 95 percent of the steady state benefits will be achieved.
 47
monetized benefits by approximately 25 to 45 percent. This range is relatively large because it represents the estimated monetized benefits using disc


ed) if they deactivate or disable their proximity reader to ensure they get valid dust samples using the CPDM. Current regulations require both detectors and CPDMs be worn. NIOSH is also conducting research on techniques that can be used to mitigate


ons linked to CPDMs would have been less likely to occur. It is unlikely similar technology would have been developed without NIOSH. The technology has now been available for more than a decade and no other countries, or other manufacturers within th


ure 3.2, those individuals make up the majority of injuries. This report focuses on ambulance occupants.
51
Figure 3.1. The Number of Ambulance Crashes Is Slowly Increasing over Time


heir jobs while wearing standard safety restraints (Larmon, LeGassick, and Schriger, 1993). This has led many to work unrestrained in the patient compartment. In serious crashes investigated by NHTSA, 84 percent of EMS providers in the patient compar


: A Survey on Ambulance Patient Compartment Safety,” Journal of Emergency Medical Services, Vol. 32, No. 3, 2007, pp. 86–90.
Current, R. S., P. H. Moore, J. D. Green, J. R. Yannaccone, G. R. Whitman, and L. A. Sicher, “Crash Testing of Ambulance Chas


uctural Integrity Test to Support SAE J3027 Compliant Litter Systems”
NIOSH, Improving EMS Worker Safety in the Patient Compartment, Washington, D.C.: Department of Homeland Security, 2017.
Reichard, Audrey A., Suzanne M. Marsh, Theresa R. Tonozzi, S


stry-standard 30 mph crash test setting. The researchers performed frontal, side, rear, and rollover crash tests. The resulting data allowed researchers to reconstruct and publish the “crash pulse” for each crash type. The published crash pulses now


ted, manufacturers would provide new, stronger prototype components as test assets. This iterative process allowed the test procedures to be tested and refined while also providing manufacturers with the opportunity to
41 For more information, see NI


.S. Army Tank Automotive Research, Development and Engineering Center
     42 Assessment based on an interview with Jim Green of NIOSH on June 21, 2019.
43 Assessment based on an interview with an equipment manufacturer on April 10, 2019.
44 Assessme


ated there were 240,800 full-time employed EMTs and paramedics (NIOSH, 2019b). In addition to EMS workers, members of the general public ride in the patient compartment as patients or when accompanying a patient.
An alternative way to think about the


 truck chassis and Type III is mounted on a van chassis. Type II ambulances do not have a box-shaped patient compartment, but instead use a van chassis with a raised roof.
 59
several data sources to reach reasonable assumptions on these issues. Howe


on is included in the estimate of 22.9 million trips, and if the average air ambulances transport just as many patients per day as the average ground ambulance, then
22.4 million of the trips are ground transportation ambulances. This is a conservati


t there were approximately 36.1 million (22.4 million + 13.7 million) ambulance trips in 2016.
Step 3: Estimate the Current Rate of Injuries and Fatalities in the Patient Compartment
We use data from NASS-GES and FARS to measure the current rates of


om 2002 to 2015.54 An estimated additional 291 front seat ambulance passengers were injured and an average of 1.1 killed per year from 2002 to 2015. Patient compartments accounted for an estimated 332 injuries and an average of 5.8 fatalities per yea


 Injuries are indeed more severe when the ambulance is traveling faster, although high-speed crashes are rare. Some crashes occur when the ambulance is at a complete stop and is struck by another vehicle.
65
We estimated urban and nonurban injury rat


whether the road on which the crash occurred is urban or rural. We categorize crashes that occur on urban roads as attributed to “urban” areas, and crashes that occur on rural roads as attributed to “nonurban” areas. These allocations are imperfect a


ering all the redesign elements in aggregate. The alternative restraints were more successful than traditional lap belts in preventing injuries, but in some cases testing dummies still contacted other interior elements of the ambulance during crash s


es in replacement rates are caused by differences in usage rates; urban ambulances accumulate miles far faster than rural vehicles due to more frequent usage. Based on the date SAE standards were issued and the fact that redesigned ambulances are now


Serious Injury
Fatal Injury
Injured, Unknown Severity
Alternative Distribution 3:
Seat Belt Literature (NonUrban)
84.6%
8.6%
4.3%
1.4%
0.5%
0.6%
Alternative Distribution 3:
Seat Belt Literature (Urban)
84.8%
8.7%
4.3%
1.5%
0.1%
0.6%
Alternative Dis


mbulances reduce the number of injuries in each severity category by a fixed percentage. The standards introduced by NIOSH, as well as the design changes made by the manufacturers NIOSH partnered with, made a wide variety of changes to improve the sa


than 10-percent reduction in fatalities to over 80-percent reduction in fatalities. We base our assumption for fatality risk on two more recent studies. NHTSA (2000) found seat belts reduce fatalities in light trucks by 60 percent, with uncertainty r


not have sufficient data to differentiate between patients and workers in modeling reductions in injury associated with new restraint designs. As such, we make the following simplifying assumption: We assume that 35 percent of patient compartment oc


eets updates and population growth, and the third where benefits increase due to population growth only.
 73
Figure 3.10. Avoided Injuries per Year Will Increase as New Ambulances Become More Prevalent over Time
  Number of avoided injuries predicted


                                2049
                                                                       2050
Alternate Distribution 1
Alternate Distribution 2
Alternate Distribution 3
Consistent with Figure 3.9, if redesign results in the patient


.6 million to $10.1 million per year in avoided direct costs by 2020, and $8.0 million to $14.9 million by 2036. For comparison to the case presented in Chapter 2, these result in an annualized value of $4.3 million to $8.0 million from 2017 to 2050


SH research, rather than re-creating the same research themselves. Indeed, some of the industry partners NIOSH worked with are encouraging Canada to adopt the same SAE standards, as common global standards can help companies streamline manufacturing


ers, and includes acute, obvious injuries such as amputations.68
66 Over 90 percent of amputations in Michigan in recent years are finger or partial finger amputations, although amputations may involve other extremities including hands, arms, toes, f


ent of the problem. Further, it is difficult to measure progress on reducing work-related amputations if accurate, stable estimates of the current number do not exist (Rosenman, 2013).
The Burden of Work-Related Amputations
Most work-related amputati


ogram does not receive any other outside funding.73 The surveillance program specifically began reviewing medical records for patients treated for amputations and referring cases meeting designated criteria to MIOSHA. Beginning with data received in


lted in 1,566 violations cited and $652,755 in penalties assessed.77
75 See MSU, undated-a, for the yearly reports for the amputation surveillance program. From 2006 to September 2008, the industries were based on Standard Industrial Classification (


iffer slightly from their study. For example, between 2006 and 2012, our data record 233 inspections based on referrals with total initial violations of 1,971 and total initial penalties of $804,840. It is not clear why these differences exist but so


in 2003–2009 and 42 in 2011–2018, showing that the process is becoming more efficient over time. A shorter time between amputation and inspection in recent years provided an opportunity for quicker remediation.
 82
Better Targeting of Inspections
The


 phone March 28, 2019; interview with Adrian Rocskay of the General Industry Safety and Health Division of MIOSHA, conducted via phone April 29, 2019. See also Berkowitz, 2018.
80 Interview with Doug Kalinowski conducted via phone on June 6, 2019.
81


 than most other injuries . . . Furthermore,
82 NIOSH supports 26 states to do some kind of state-based surveillance program (Rosenman interview). See also Davis et al., 2014, and Friedman et al., 2013.
83 Although the Massachusetts program has used


86 See also Michaels, 2016. Although the Michigan surveillance program is not mentioned in the documentation
to the new rule, influence of the Michigan and Massachusetts programs on the federal regulation was noted by
Dr. David Michaels in his interv


e BLS data we see the number and rates of amputations decrease through time both in Michigan and in the rest of the United States. Amputations decreased substantially in Michigan from the 2003–2006 period (average 390/year) to the 2008–2017 period (a


 surveillance program did not exist (i.e., we do not know the counterfactual), we cannot claim that the program caused inspections to find
87
more violations and penalties than they otherwise would have. However, we find that the surveillance inspect


the employer belongs to (six digits); the type and scope of the inspection; whether the employer is privately owned; information on related activities; whether it was an inspection for health or for safety; count of serious, willful, repeat, other, a


 Note that in the data inspections are referred to as “planned” while in the manual they are called “programmed.”94 Planned inspections make up
73 percent of the observations in the Michigan data, followed by complaint at 14 percent, referral at 4 pe


tions and 44 percent partial inspections.
94 The reason for the difference in terminology is not clear.
95 Although we have generally used the word “injury” to describe the event that results in an amputation, the OSHA data use the word “accident.”


t Current Penalty Other Current Penalty Unclass Current Penalty Total Current Penalty Total FTA Amount
Serious Initial Violations Willful Initial Violations Repeat Initial Violations Other Initial Violations Unclass Initial Violations Total Initial V


nzero observations and observations with violation counts less than 30.
  97
Figure 4.4. Distribution of the IHS of Total Initial Penalty Size from Surveillance Program Referrals and All Other Inspections in Michigan
SOURCE: Author analysis based on


se are good outcomes for the purposes of the current study. We believe that MIOSHA inspectors are not incentivized to inflate these numbers, and their pay and prestige within the agency are not based on them. Assuming variation in hazardous condition


with the highest number of hazardous conditions, thus leading to a more effective use of resources. Thus, if (1) violations and unsafe conditions that could lead to injuries exist and are unevenly distributed across worksites, (2) MIOSHA inspectors r


is present. Log-linear models also cannot incorporate observations with zeros in the outcome variable. Both Poisson and negative binomial regression analyses are commonly used to model count data and to deal with the problem of zero-valued dependent


2, but MIOSHA still had high-priority industries in its strategic plan. Regressions using priority industries based on the 2014–2018 Strategic Plan produced an estimate of 1.60 for the Michigan surveillance program. Using an indicator for being in a


ccident Type = Unprog Rel Type = Prog Related Type = Follow-up Type = Prog Other Type = Monitoring Type = Other
Type = Un Prog Other Type = Variance
Type = Fat/Cat
Safety Inspection
(1)
3.24*** (0.38)
(2)
2.68*** (0.33)
2.21*** (0.15)
1.54*** (0.06)


ploited a cutoff of OSHA’s Site-Specific Targeting plan to estimate the effects of inspections with a fuzzy regression discontinuity using panel data on 61,702 unique establishments between 1996 and 2011. The authors reported that inspections decreas


the mean) compared with the 24 surveillance inspections that were performed. Viewed through this lens, the program “saved” 14 (72) inspections per year along with their associated costs and still found the same number of violations. Calculating simi


uld have led to no reductions in injuries, then the 24 additional surveillance inspections per year would lead to $9.72 million dollars of saved costs and lost earnings per year of the program. To the extent
that the very act of conducting an inspect


e geography or over many years to have enough statistical power to see the effects of the program on amputations.
 110
There is a further value in the information provided by the Michigan surveillance program. Although it is well known that the BLS S


federal rule change. Documenting and detailing exactly how the Michigan program influenced the Massachusetts surveillance program and how important the public comments from the Massachusetts Department of Health were on the creation of the federal la


se studies, and RAND traced the elements of these case studies from the input hazard to the potential changes in health outcomes for workers and other affected individuals. In parallel, RAND identified attributes of cases studies that NIOSH might use


nts, both by working directly with manufacturers and by collaborating with standard-setting organizations. NIOSH continues to play a pivotal role in these partnerships, and many of the design changes may not have happened without NIOSH’s involvement.


t resources utilized for these inspections would have been used to conduct inspections similar to the average inspection measured in our data had the program not existed, then we can provide some rough numbers of the effects of the program on these p


ting Evidence on Cancer Risk Among Firefighters found NIOSH research was associated with an annualized value of $71 million in 2016 dollars in avoided medical costs and productivity losses, and approximately $1 billion per year in VSL estimates for a


be subtracted from the baseline levels in Tables A.1–A.5 to calculate the impact of CPDMs on the excess risk of respiratory disease due to coal mine dust (measured per 1,000 workers).
  108 Due to the relative complexity of the exposure risk models,


ever, we note that the additional excess risk may be significantly higher for workers smoking two packs a day as opposed to workers smoking less than half a pack per day, for example.
119
Table A.1. Estimated Excess Risk of CWP by Age 73 After 45 Yea


 Operator Longwall Jacksetter Longwall Tailgate Operator Mechanic and Helper
Mobile Bridge Operator
Roof Bolter
Scoop Car Operator
Section Foreman
Shuttle Car Operator Uni-Hauler Operator
Utility Man
All Other Underground Jobs Part 90 Miners
Rank Bit


n and Helper
Laborer
Loading Machine Operator
Longwall Headgate Operator
Longwall Jacksetter
Longwall T ailgate Operator
Mechanic and Helper
Mobile Bridge Operator
Roof Bolter
Scoop Car Operator
Section Foreman
Shuttle Car Operator
Uni-Hauler Operato


e A.8. Estimated Excess Risk of COPD by Age 73 After 45 Years of Occupational Exposure at Reduced Exposure Levels (per 1,000 Workers)
 Occupation
Low-Recurrence
Mid-Recurrence
High-Recurrence
 Auger Operator 35.1 Continuous Miner Operator 28.0 Cuttin


 Uni-Hauler Operator
Utility Man
All Other Underground Jobs Part 90 Miners
Low-/Med.- High-Rank
Rank Bituminous Anthracite Rank
Bituminous Anthracite
Rank
21.8 15.5 17.7 –
– 26.0 – 18.8 21.3 20.5 – 13.2 16.9 24.0 45.8 19.6 – – 42.2 9.5
Bituminous Ant


Blackburn, M. L., “The Relative Performance of Poisson and Negative Binomial Regression Estimators,” Oxford Bulletin of Economics and Statistics, Vol. 77, No. 4, 2014, pp. 605–616.
Blackley, David J., Cara N. Halldin, and A. Scott Laney, “Continued I


lic Health Referrals to OSHA,” September 2011. As of July 15, 2019:
http://www.cste2.org/webpdfs/occupational/OSHAreferral922011.pdf
Cromartie, John, and Dennis Vilorio, “Rural Population Trends,” Amber Waves, February 15, 2019. As of July 24, 2019:


ria/transport_ria_final-csapr_2011-06.pdf
EPA—See Environmental Protection Agency.
Federal Transit Administration, “2001 Table 24: Number of Injuries–Collisions and Non- Collisions,” webpage, 2001. As of November 5, 2019: https://www.transit.dot.gov/


fects of Publicizing Violations of Workplace Safety and Health Laws,” August 30, 2019. As of November 5, 2019: https://drive.google.com/file/d/1HcKpGXZuFWNNLa1YTl0A4Hte1BiJabT-/view
Kica, Joanna, and Kenneth D. Rosenman, “Multisource Surveillance Sys


hn Wood, David J. Blackley, and David N. Weissman, “Coal Workers’ Pneumoconiosis—Attributable Years of Potential Life Lost to Life Expectancy and Potential Life Lost Before Age 65 Years—United States, 1999–2016,” Morbidity and Mortality Weekly Report


e Operators’ Dust Control Plans and Compliance Sampling for Respirable Dust,” Federal Register, Vol. 68, No. 44, March 6, 2003, pp. 10784–10884.
———, “Lowering Miners’ Exposure to Respirable Coal Mine Dust, Including Continuous Personal Dust Monitors


ght Trucks, Publication No. DOT HS 809 199, Washington, D.C.: NHTSA, 2000.
———, “The National Highway Traffic Safety Administration and Ground Ambulance Crashes,” presentation, April 2014. As of November 5, 2019: https://www.ems.gov/pdf/GroundAmbulan


ault.html
———, Enhanced Coal Workers’ Health Surveillance Program, Publication Number 2019-112, Washington, D.C.: Department of Health and Human Services, October 2018c.
———, “The Work-Related Injury Statistics Query System,” webpage, last updated Se


 Paul Moore, and Roger Levine, “Safety in Numbers: A Survey on Ambulance Patient Compartment Safety,” Journal of Emergency Medical Services, Vol. 32, No. 3, 2007, pp. 86–90.
Reynolds, Laura, Cara N. Halldin, A. Scott Laney, and David J. Blackley, “Co


s, “Impact of Differential Injury Reporting on the Estimation of the Total Number of Work-Related Amputations,” American Journal of Industrial Medicine, Vol. 57, 2014, pp. 1144–1148.
United States Census Bureau, “Statistics of U.S. Businesses (SUSB),


  CORPORATION
BENJAMIN M. MILLER, DEBRA KNOPMAN, LIISA ECOLA, BRIAN PHILLIPS, MOON KIM, NATHANIEL EDENFIELD, DANIEL SCHWAM, DIOGO PROSDOCIMI
U.S. Airport
Infrastructure Funding
and Financing
Issues and Policy Options Pursuant to Section 122 of the 20


endations on the actions needed to upgrade the national aviation infrastructure system to meet the grow- ing and shifting demands of the 21st century.” RAND, a nonprofit and nonpartisan policy research organization, was selected by the FAA through a


logy, and commu- nity organizations and institutions that affect well-being. For more information, email chep@ rand.org.
Contents
 Preface........................................................................................................... iii


PTER FIVE
Federal Role in Airport Infrastructure Funding: Airport Improvement Program. . . . . . . . . . . . . . . Airport and Airway Trust Fund. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .


an Index for Three Airline Industry Metrics, 1974–2018 . . . . . . . . . .
2.3. Herfindahl-Hirschman Index According to Enplanements at Large- and Medium-Hub Airports, 2018 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .


. . . . . . . . . . . . .
4.8. Trends in Commercial Service Airport Expenses, 2009–2017. . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.9. Commercial Service Airport Expenses, by Airport Size, 2009–2017 . . . . . . . . . . . . . . . . .


. . . . . . . . . . . . . . . .
4.3. Funding Sources for Average Rural and Urban Commercial Service Airports, 2017 . . .
4.4. Expenses for Average Rural and Urban Commercial Service Airports,
2017 (millions of dollars) . . . . . . . . . . . . . . . .


tion (FAA) Reauthorization Act of 2018 directed the FAA to contract with an independent research organization to address questions and provide recommendations related to infrastructure funding and financing at commercial service airports. These airpo


s an important determinant in an airport’s funding preferences. Depending on the “color of money,” airports also are required to line up approvals from the appropriate combina- tion of federal regulators, local and state governments, and airlines bef


ts over local land-use policy and public priorities.
Finding 3: There Are Known Areas in Which Infrastructure Investment Is Needed
Airport runways are generally in good repair. This reflects the priority given to airside infrastructure—i.e., infrastr


orts that choose to participate in the federally authorized passenger facility charge (PFC) program. With reference to the color of money in Finding 1, airlines cannot veto FAA-approved PFC-funded projects. The cap on PFCs has been set at $4.50 per p


small to support airport construction of any conse- quence. Airports, however, are permitted to defer their annual entitlements over several years to accumulate sufficient funds to undertake a project.
Finding 6: The Airport and Airway Trust Fund Has


by outside observers, including the FAA. The FAA has effective enforcement options available for discouraging the second form of revenue diversion, but the FAA’s existing enforcement mechanisms are largely unable to prevent grandfathered airports fr


 100 percent of AIP primary entitlements for medium- and large-hub airports
that choose to raise their PFC above $4.50.
We are not aware of compelling evidence or data justifying a particular level for a new cap. Any number could be chosen, but we no


ace demands on infrastructure.
Not all airports will choose to seek an immediate or longer-term PFC increase. To increase transparency regarding the intentions of airports in maintaining cash reserves beyond those required by bond rating agencies, we


ct funds to current high-priority projects at spe- cific airports.
We recommend that Congress remove the triggered primary entitlement increase that occurs when Congress appropriates at least at $3.2 billion to the AIP. Those airports not vol- untari


 as determined by the FAA.
Congress Should Include Ancillary Fees in the Domestic Passenger Ticket Tax
Ancillary fees are charges for airline-provided services or products that some airlines sell sepa- rately from tickets, such as checked baggage, ad


airports, cities, and routes are served by only one or two airlines. Lack of competition among airlines at a given airport can result in higher costs for travelers. In practice, PFC proj- ects provide airports with a means of accommodating new entra


rastructure, air traffic control towers, and other air traffic facilities, and there is a need for an assessment of their current capacity, functionality, and physical condition. An up-to-date inventory and assessment of infrastructure conditions wou


tus, regional and local economic conditions, and other factors.
As noted in (c)(1), 20 airports account for 96 percent of delays measured by the FAA’s OPSNET. These airports create delays for passengers that propagate throughout the NAS.
Separately,


als, while AIP grants are generally focused
on airside infrastructure. Expanding airports’ revenue bases to include the taxing of nonairport local businesses and residents
is unattractive to local governments and would increase the cost of local good