excluding benign and unspecified neoplasms (i.e., limited to ICD-9 codes 140–209 and 230–234) in three or more consecutive calendar quarters. This definition was chosen to select cases with a pattern of medical care over time that was very unlikely


 is likely to be missing from our data, limiting our ability to definitively identify the full population of cancer cases on the basis of medical bills reported to the WCIS.
Despite these limitations, the WCIS medical bills provide us with the best a


ver two years before injury and two years after injury to measure preinjury earnings and postinjury earnings. We assigned employment outcomes (both employment at any firm in California and employment specifically at the at-injury employer) from eight


 at-injury employer)
• firm size (full set of indicators for seven categories of firm size, with cutpoints at 10, 50, 100, 500, 1,000, and 5,000 employees)
• injury date (full set of indicators for calendar quarter of injury, from 2005Q1 to 2015Q4).


n cannot be tested directly because we do not see what would have happened to these workers in the absence of the cancer claim, but it can be tested indirectly because we have eight quarterly observations prior to the date of injury. Specifically, we


th a cancer claim to 1 and set the weight for each worker with a medical-only claim to 𝑤𝑤 = 𝑝𝑝 /1 − 𝑝𝑝 , the
predicted odds of a cancer claim, which measures how much more likely it is that the worker was a cancer case relative to a non-cancer case.


with a substantial delay after the date of injury raises the possibility that some of these claims are filed, at least in part, in response to workers’ labor market outcomes. In this case, our estimates of earnings losses could be biased due to reve


tions
–11,417*** (1,512) 635,450
           NOTE: * p < 0.10, ** p < 0.05, *** p < 0.01. Standard errors appear in parentheses. “Nature of injury” definition includes claims for which nature of injury on FROI is 74 (Cancer). “Medical claims (loose)”


Cancer: Findings from a Longitudinal Study of Breast and Prostate Cancer Survivors Employment and Cancer,” Cancer Investigation, Vol. 25, No. 1, 2007, pp. 47–54.
Bureau of Labor Statistics, “Labor Force Statistics from the Current Population Survey,”


 in T. L. Guidotti, ed., Health Risks and Fair Compensation in the Fire Service, New York, N.Y.: Springer International Publishing Switzerland, 2016,
pp. 265–280.
Hauglann, Beate, Jūratė Šaltytė Benth, Sophie D. Fosså, and Alv A. Dahl, “A Cohort


 on the Employment of Older Workers,” Health Services Research, Vol. 43, No. 1, 2008, pp. 193–210.
Stuart, Elizabeth, Haiden A. Huskamp, Kenneth Duckworth, Jeffrey Simmons, Zirui Song, Michael E. Chernew, and Colleen L. Barry, “Using Propensity Score


Wage Loss Monitoring for Injured Workers in California’s Workers’ Compensation System
2016–2017 Injury Year Findings (Third Interim Report)
STEPHANIE RENNANE, NICHOLAS BROTEN, MICHAEL DWORSKY
  SOCIAL AND ECONOMIC WELL-BEING
Sponsored by the Californ


er an accurate and up-to date estimate of wage loss for all injured workers in the state.
RAND Institute for Civil Justice
The RAND Institute for Civil Justice (ICJ) is dedicated to improving the civil justice system by supplying policymakers and the


r After Injury:
All Indemnity Injuries........................................................................................................... 87 Figure A.3a. Earnings Loss by Industry for Workers with Permanent Partial Disability
Benefits (consta


r before
2015
• comparison of earnings losses due to 2016 and 2017 injuries and earnings loss for earlier
injury years
• comparison of differences in earnings loss levels and changes across subgroups of
injuries.
As noted earlier, trends in earnings


d control workers to estimate wage loss one and two years after injury.
Key Findings
Using this methodology, we estimated earnings losses, relative employment, and post-injury employment at the same firm as prior to the injury for workers injured in


econd full year after injury (1–4 quarters after quarter of injury).
Relative Earnings Increased in Los Angeles but Declined in the Bay Area
We estimate that relative earnings in the first year after injury increased in Los Angeles by 2.1 percentage


y, given long reporting lags, but show large declines in relative earnings during the Great Recession and modest improvements beginning in 2014.
Following this report, RAND will produce a final policy report that will provide a more
 sustained analys


tcomes—leading to more substantial earnings losses—also make it more difficult and costly for the workers’ compensation system to meet its objective of providing adequate disability benefits to workers.
The overall objective of this project is to pro


d injury or illness is defined as one that prevents a worker from doing his or her usual work for more than three days or that requires an inpatient hospital admission.4 Workers with a temporary condition will collect weekly temporary total disabilit


filed by workers who have already retired or separated from their employer for reasons unrelated to their injuries. This may be particularly likely for occupational disease claims, which have a longer claim filing window than injury claims. In other


e similar to those observed for workers injured in 2010–2012 (Dworsky, Rennane, and Broten, 2018a, 2018b), consistent with this assumption. Although the findings in this report suggest that outcomes for injured workers have begun to improve for injur


ers (ASCs) and payment for spinal hardware.
PPD benefits also increased sharply because of changes to the disability rating formula and increases in the weekly maximum benefit. In addition to these changes, a $120 million Return to Work Fund was esta


and affect care during the first two years after injury for the 2016 and 2017 injured workers at the focus of this report and potentially have an indirect effect on labor market outcomes. However, the potential impacts may be limited in the first fe


jury and increase the risk of poor labor market outcomes. At the same time, increased labor demand due to economic growth means that employers may be more motivated to accommodate workers after an injury, or that workers who end up leaving a job afte


on to our methods should reference that study.
Data for Wage Loss Monitoring
We make use of several administrative data sources for this project, all of which have been used for prior RAND studies (e.g., Peterson et al., 1998; Reville et al., 2005; D


that captures all market segments (including self-insured and public-sector employers). Given the absence of any more reliable benchmark, this study will follow Dworsky et al. (2016) in treating FROI reported to the WCIS as a representative sample of


ng report.9 Additionally, this report provides an opportunity to update the data for injuries occurring in 2014 and 2015 now that more time has elapsed since the date of injury.
Also because of the longer follow-up period in this report, we present r


n the Base Wage File. The Base Wage File represents the most accurate and comprehensive source of data on quarterly wage and salary earnings in California. The industries covered by UI are virtually identical to the industries covered by workers’ com


ceptual Framework
We define earnings loss due to a workplace injury as the difference between what a worker earns following an injury and what he or she would have earned had he or she not been injured. We refer to the counterfactual amount that a wo


es data to tell us how each injured worker’s earnings would have evolved over time. A major
12
advantage of our method is that the data on the matched controls can capture subtle differences between workers in the dynamics of potential earnings, incl


to non-employment are highest in agriculture, ranging over the year from 10.9 percent to 27.7 percent.
Overall, the share of all separations that lead to a period of non-employment for the worker is higher during recessions and lower during periods o


pool to workers who were employed at the same firm as the injured worker during the quarter of injury. To further limit the donor pool, we discarded potential controls whose annual earnings are not close to those of the injured worker. Specifically,


 before the quarter of injury and the eighth quarter after injury.13
Finally, we selected control workers for injured workers with complete records on key FROI variables and valid EDD wage histories. As shown in Table 2.1, we successfully matched abo


clines before and after the date of injury are driven by the fact that we are sampling workers at a point when 100 percent of both injured and control workers are employed: The lower employment rate (and, as a result, earnings) before and after the p


or to the four-quarter match window provides an indirect evaluation of our matching methodology. Figure 2.1 illustrates clearly that injured and control worker earnings are extremely close over the year before the match window (pre-injury quarters –8


we primarily emphasize trends in earnings losses. However, it is useful to see how total employment and at-injury employment evolve for both injured and control workers in order to interpret the relative measures that we focus on in our presentation


rnings loss a useful tool for monitoring changes in worker outcomes.
However, a serious drawback of focusing on overall trends is that the relatively limited earnings losses experienced by the majority of medical-only cases can make it difficult to d


rnings is the difference between injured and control worker real earnings over the first (for 2016 and 2017 injuries) or second (for 2016 injuries only) year following the quarter of injury, adjusted by the difference between injured and control work


he year before the injury, and that the control workers’ average earnings in the second post-injury year are $36,200. Because the injured worker earned $200 less than his or her control workers prior to the injury, counterfactual earnings are adjuste


economic recovery [2013–2015], and the 2016 and 2017 injury cohorts). Details of our regression specifications are presented in the appendix. We ran these regressions on first-year post-injury outcomes, including the 2016 and 2017 injury cohorts, and


ups experience changes that are different from the overall average. For instance, if relative earnings for manufacturing workers injured in 2014–2015 were four percentage points less than the systemwide average, these estimates would tell us whether


y year 96% Relative employment
First post-injury year 100% Second post-injury year 102% Relative at-injury employment
First post-injury year 91% Second post-injury year 86%
Recession Pre–SB 863
Post–SB 863
Pre- Recovery,
     2008–2009 2010–2012 2013


ured in 2016 and 2017 relative to prior cohorts, with a mean of $706 (compared to a mean pre-injury weekly wage ranging between $656 and $675 for prior cohorts) and a median of $540, increasing from $499 for the 2013–2015 injury cohorts. Given that t


y Area, 20 percent in the Inland Empire, and another 24 percent in Los Angeles (a two-percentage-point decline from the 2013– 2015 injury cohorts). Eleven percent of injured workers lived in the Central Valley, and the remainder were spread fairly ev


d 77 to 78 percent between 2008 and 2012. Still, the share with at least one year of job tenure is higher than the 70 percent observed in the 2005–2007 injury cohorts.
30
Table 3.3. Industry, Firm Size, and Job Tenure by Year of Injury Year of Injury


the second post-injury year also increased from 88 for the 2013–2015 injury cohorts to 89 percent for 2016 injuries. Relative at-injury employment during the first post-injury year also increased by one percentage point relative to 2013–2015, to 79


ins a constant-maturity sample of claims where the worker received PPD benefits within 36 months of injury.
Table 3.5 shows the results on relative earnings, total employment, and at-injury employment for the constant-maturity PPD sample. Relative ea


loyment also remained stable for the overall 2016 and 2017 injury cohorts but increased by two percentage points, to 87 percent, in the first post-injury year for workers with indemnity payments from the 2016 and 2017 injury cohorts, and one percenta


ensation system, as well as trends estimated separately for medical-only and indemnity claims, and some preliminary analyses of outcomes for workers with permanent disability payments through 2015.
In this chapter, we estimate trends in earnings loss


ups of workers, or whether certain industries, regions, or other populations have had experiences that deviate from the statewide average trend.
The analysis presented here is descriptive and is not intended to test hypotheses about what mechanisms e


year after injury, relative earnings declined by 1.9 percentage points for workers in retail who were injured in 2016. None of the other subgroups experienced statistically significant changes in relative earnings during the second year after injury


ngs than the statewide average in larger firms. However, relative earnings in firms with over 5,000 employees declined by 1.6 percentage points for workers with permanent disability injured between 2013 and 2015. On the other hand, relative earnings


e quartiles had relative earnings ranging between 50 and 60 percent, approximately a 30-percentage-point difference. Relative earnings for workers with PPD in the lowest wage quartile began to improve, increasing by four percentage points for workers


in relative earnings during the first year after injury for workers with injuries of a specific nature in 2016 and 2017. We did not detect any statistically significant breaks in the relative earnings during the first year after injury for any of the


ve earnings. Two exceptions to this general rule are the fact that workers from North State–Shasta had significantly lower earnings relative to the average worker over the entire time frame, and workers in San Diego had significantly higher earnings


: Relative earnings increased by an average of 3.8 percentage points during the first year after injury for workers injured in 2016 and 2017, and by 2.7 percentage points during the second year after injury for workers injured in 2016. Relative earni


ty payments did increase for injured workers in 2016 and 2017. Relative earnings in the first year after injury increased by one percentage point, to 78 percent, for injured workers in 2016 and 2017 and increased by one percentage point, to 82 percen


ings increased by 3.8 percentage points for injured workers in 2013–2015, compared with prior cohorts. Because of the long follow-up period required to analyze outcomes for permanent disability, we caution that these outcomes are preliminary and will


 matched to an injured worker, an average of all matched control characteristics was used.
To identify the combination of parameters that produces the best match for the full population of injured workers, we tested 12 variations of the match on subs


tead. Finally, we also present the elementwise absolute imbalance, which sums the absolute value of the difference in quarterly earnings between injured and control. Formally, these are
defined as follows, with 𝑖𝑖 indexing injured workers and 𝑡𝑡
inju


ies
–0.54% 0.04% 21.70% 20.86%
26.16% 25.24%
2010–2012 Injuries
–0.16% 18.73%
22.67%
2013–2015 Injuries
–0.59% 19.07%
22.97%
             54
Table A.3c. Overall Imbalance
    Time Period
Match window
(1–4Q before injury) Second pre-injury year (5–8Q


 administrator weight, is 1.311, and its maximum value is 42.60. A summary of the sample construction and weighting scheme is shown in Table A.4.
55
      % female Average age Weekly wage Self-insured
All FROI in WCIS 0.42 40.45 706.16 0.33
Good Clai


 injury date. The primary
level of relative earnings from the level in the preceding time period. For example, 𝛽𝛽Y: captures
and workers injured in 2005–2007, while 𝛽𝛽FY captures the seasonally adjusted difference in
the seasonally adjusted differenc


es are compared with 2013–2015 injuries. The group-specific trend break coefficients capture whether the difference in relative earnings between each group and the statewide average changed relative to the previous time period. For example, 𝛽𝛽x,Fs fo


uries includes injuries through 2017. Sample for two-year indemnity injuries includes injuries through 2016.
 61
Table A.8a. Permanent Differences in Relative Earnings by Firm Size Category
   One Year Post-Injury
Two Years Post-Injury
        Firm S


) [0.1990]
 64
Table A.8d. Permanent Differences in Relative Earnings by Nature of Injury
   One Year Post-Injury
Two Years Post-Injury
        Nature of Injury
Cumulative
Multiple
Other
Specific
Indemnity b/se/p –0.1118 (0.0026) [0.0000]**
–0.033 (0


017. Sample for two-year indemnity injuries includes injuries through 2016.
 66
Table A.8f. Permanent Differences in Relative Earnings by Quartile of Pre-injury Earnings
   One Year Post-Injury
Two Years Post-Injury
        Quartile of Earnings
1st p


pared to Overall Average by Region
One Year Post-Injury
Two Years Post-Injury
           Trend Break*(Subgroup)
2008*Bay Area 2008*Central Coast 2008*Central Valley 2008*Eastern Sierra 2008*Inland Empire 2008*Los Angeles 2008*N. Sac Valley 2008*North


         Trend Break*(Subgroup)
2013*1st pre-injury wage quartile 2013*2nd pre-injury wage quartile 2013*3rd pre-injury wage quartile 2013*4th pre-injury wage quartile 2016*1st pre-injury wage quartile 2016*2nd pre-injury wage quartile 2016*3rd pre-i


ded in the analysis as a base period, so no coefficients are reported for these injury dates. The p-values presented above are unadjusted. ** indicates Holm-adjusted statistical significance based on a benchmark of α = 0.05. PPD (CM) indicates injuri


Subgroup)
2016*Southern California, cumulative
Indemnity b/se/p 0.0381 (0.0089) [0.0001]**
PPD (CM) (2005–2015) b/se/p
     NOTE: Table shows linear regression coefficients, standard errors in parentheses, and unadjusted p-values in brackets. 2005–20


e A.3d. Earnings Loss by Job Tenure at Injury for Workers with Permanent Partial Disability Benefits (constant maturity sample), 2005–2015 Injuries
 SOURCE: 2005–2015 matched WCIS-EDD data.
NOTE: Relative earnings defined as ratio of injured worker e


 for Workplace Safety,” Journal of Health Economics, Vol. 30, No. 4, 2011, pp. 764– 773.
Brooks, Ward, Study of Geographical Differences in California Workers’ Compensation Claim Costs, Oakland, Calif.: Workers’ Compensation Insurance Rating Bureau o


 Its Second Century, Kalamazoo, Mich.: W. E. Upjohn Institute for Employment Research, 2017.
Hyatt, Henry R., and James R. Spletzer, “The Shifting Job Tenure Distribution,” Labour Economics, Vol. 41, 2016, pp. 363–377. doi:10.1016/j.labeco.2016.05.00


 CORPORATION
LYNN A. KAROLY
Assessing the
Costs and Benefits
of LifeSet
The Youth Villages Program Model for Transition-Age Youth
  For more information on this publication, visit www.rand.org/t/RR2839
  Library of Congress Cataloging-in-Publication


as well as those interested more generally in the costs and benefits of programs serving young adults transitioning out of the foster care system.
RAND Social and Economic Well-Being is a division of the RAND Corporation that seeks to actively improv


es
Table S.1. Summary of the Estimated Cost and Benefits of LifeSet per Participant Under Alternative Assumptions, 2015 Dollars .................................................................................. x Table 2.1. Summary of Estimated Impac


i.e., a randomized control trial [RCT]), for a cohort of participants in Tennessee who enrolled in the program between October 2010 and October 2012, with findings reported one year and two years after randomization. Based largely on survey data, the


his summary, we highlight key findings and discuss the implications of the analysis for understanding the economic returns from LifeSet.
Key Findings
The BCA of LifeSet presented in this report produced the following key findings (see Table S.1):
• L


per Participant Under Alternative Assumptions, 2015 Dollars
   More Conservative
Summary Result Estimate
      Base-Case Estimate
    Less Conservative Estimate
  Present-value cost net of control group services Present-value of outcomes and deadweig


economic values of the outcomes that could be included in the analysis. In just over one-half of the simulations, present-value net benefits exceeded zero, indicating a positive economic return.
• If all outcomes could be valued, the estimated net be


he evidence regarding the program’s economic benefits, in particular. These include the following:
xi
• Collect comprehensive information about the cost of program implementation for both the services provided to LifeSet participants and the “busines


benefit-cost analysis
C control
CPI Consumer Price Index
CPS Current Population Survey GED general equivalency diploma NSC National Student Clearinghouse RCT randomized control trial
SNAP Supplemental Nutrition Assistance Program SSI Supplemental Sec


ured additional outcomes in three of the six domains: education and training, employment and earnings, and criminal involvement.
1 A growing body of research documents the challenges and adverse outcomes faced by youth transitioning to adulthood from


gh School
Diploma/General Equivalency Diploma
 Post-Secondary Education Plans
 Employment Skills
Community Partnerships
 Community
Collaboration
 Accessing Support
 Customer
Satisfaction
Team/Staff Development
 Career/Leadership Opportunities


the evaluation represent either benefits or costs, depending on the sign of the impact coefficient, as well as whether the perspective is that of the public sector (i.e., taxpayers), the participants themselves, or the rest of society. In performing


measured, Chapter 3 presents the approach taken in this study to value the outcomes examined in the impact evaluation. The resulting estimates of benefits and costs are presented in Chapter 4. Conclusions and implications of the findings are the subj


vices offered by the LifeSet staff. The frequency and duration of services utilized were close to expected levels, participation rates on the part of young adults were high, and participants were highly engaged in services.
3 This chapter provides a


ng stability and economic well-being, health and safety, social support, and criminal involvement.
Table 2.1 provides a summary of the findings in the form of a stoplight chart, with each row summarizing the impacts for a given outcome measure at thr


 safety, continued
 Binge drinking in past month
   0
        Used illegal drugs
 –
    Condom use during last sexual encounter
  +
      Was robbed or assaulted
  0
      Partner violence
   –
        Social support
  Score on social support scale


nefits), depending on the viewpoint of the decisionmaker. In such cases, the cell is shaded in a red and green pattern and viewed as indeterminant when judging whether the impact is advantageous or not.
Finally, Table 2.1 also indicates statistical s


cally significant only for the mental health problems scale, the indicator of not receiving health care when needed, and the indicator of experiencing partner violence.
9
• Criminal involvement. The category of criminal involvement shows mixed findin


had a higher incidence of having a high school diploma and enrollment in vocational training, but the estimates of postsecondary enrollment showed a lower incidence by the end of two years for LifeSet participants relative to the control group (see T


ved SSI*
Year 1
    Received SNAP*
Year 1
    Received WIC*
Year 1
    Received public housing/rental assistance*
Year 1
    Received Families First (TANF)*
Year 1
              12
      Outcome
Impacts in MDRC Evaluation
Source for Shadow Price
   H


Set participants from their observed higher rate of high school attainment as of the first year. We focus on receipt of a high school diploma and assume no benefit from attaining a GED.5 To estimate the lifetime earnings gain of a recipient of a high


regular diploma, after accounting for the selectivity of who receives which type of diploma and other biases introduced by the available data. The CPS, our source of estimated lifetime earnings, does not differentiate between those who completed a h


ther, the projected earnings gains are based on the impacts for educational attainment, discussed earlier.
Housing Stability
The MDRC evaluation reported the impact of LifeSet, in the first year, on a housing stability scale composed of four indicato


of the transfer benefit as it would apply for a single adult (or family with one adult and one dependent child, in the case of WIC and TANF) in Tennessee as of 2015 (see Table 3.2). Because the evaluation measured any use of the benefit program duri


t for housing, which is an estimated average value of the housing benefit that varies by local rental rates and family income.
Health
The MDRC evaluation assessed various measures of health status, health care utilization, and health insurance covera


rime costs associated with the effect on being robbed or assaulted (victimization increased by a very small amount) and the effect on intimate partner violence (victimization decreased by a statistically significant amount) (see Table B.6). In both c


self-reported crime, with only one statistically significant reduction in the incidence of criminal behavior—for receiving cash or goods in exchange for sex (see Table B.8). Although the incidence of arrests was lower (but statistically insignificant


alysis for LifeSet
In this chapter, we present the results of our BCA, drawing on the cost and impact estimates summarized in Chapter 2 and employing the approach to valuing outcomes reviewed in Chapter 3. Given the uncertainties in both cost and the


et per Participant Under Alternative Assumptions, 2015 Dollars
   More Conservative Summary Result Estimate
 Base-Case Estimate
     Less Conservative Estimate
       Present-value cost 8,763 8,357 7,951
LifeSet cost
Offset for service utilization by


group had higher rates of enrollment in postsecondary education, the projected future benefits from this source of impact are negative and almost offset the high school productivity gains.
Among the observed outcomes, the largest contributor to posit


is less than 1 (0.42), indicating that costs exceed benefits.
Accounting for Uncertainty in the Included Impacts
The base-case estimates rest on assumptions about the economic values of the outcomes affected by LifeSet, as well as the value of the re


ing insecurity, reduced economic insecurity, and greater social support; improvements in mental health and reduced substance misuse; reduced costs associated with having a handgun; and greater adherence to safer sex practices. If these outcomes could


 the short-term impacts were sustained.
Even with these sources of uncertain net benefits or costs, under the more conservative assumptions, the addition of the currently unmeasured benefits could be sufficient to generate positive net benefits overa


ojected to the future, beyond the point of last follow-up. Because the impact estimates were not always favorable for the treatment group, the impacts of LifeSet were associated with both positive benefits and negative benefits (or costs). Given unce


rease in taxes required to pay for LifeSet, net of any increase or decrease in taxes based on the program impacts. A more conservative estimate produces a deadweight cost of about $4,400 per participant, while a less conservative estimate has no such


use the magnitude of the impacts on the outcomes that are not valued
28
was typically small and statistically insignificant, these omitted sources of potential benefits are not likely to generate net benefits that are substantially larger than those


feSet is funded partially through private funders, more funding may be specifically dedicated toward tangible goods and
29
financial support around housing, clothing, and basic necessities. In other areas, staff may
have access to limited wraparound


uld experience even better outcomes from participating in LifeSet than those realized for the Tennessee participants. This would occur if the program model were effective at serving youth with even greater challenges. (Program effects could be smalle


gs and private benefits.
• Capture the associated magnitude (or quantity) of the impacts to go beyond simply measuring the incidence of any given outcome. For example, for homelessness, measure the number of nights and type of homelessness for each e


 in December 2018 (WSIPP, 2018a). Before we review the BCA findings, we discuss the following BCA components: program cost, program impacts, and valuation of program impacts and other assumptions.
WSIPP Estimate of LifeSet Program Cost in Washington


erall effect on net benefits from this omission is unclear.
Even for the year 1 findings, the WSIPP model does not account for all of the outcomes measured in the MDRC evaluation. In particular, as shown in Table A.1, the WSIPP model accounts for jus


adult
Year 1
    Score on familial closeness scale*
Year 1
    Biological family member would offer place to stay*
Friends would offer place to stay*
Year 1
    Year 1
      Criminal involvement
     Criminal behavior scale
   Got in gang fight or as


pendent organization, and used the gold-standard design, an RCT. Thus, the WSIPP-adjusted effect sizes are the same as the unadjusted effect sizes.
The WSIPP model also allows for effect sizes to change over time, based on either a longitudinal follo


dology produces a somewhat higher effect size of 0.104. And the effect is assumed to persist. Had the result for arrests been used instead, the impact would have been negative (a favorable outcome), with an effect size of –0.040, as reported by MDRC.


assumption in the WSIPP model is the inclusion of a parameter to capture the efficiency cost of raising taxes to pay for government transfers and other benefits, known as deadweight loss. In effect, every dollar of public revenue required to pay for


. One- and Two-Year Impact Estimates: Employment and Earnings
       Control Impact Effect
Outcome Source Group (T – C) Size p-value
       Earnings from formal work—year 1, year 2
Earnings—year 1 ($) 12-mo. survey 3,488 611 0.12 0.043
 Earnings—year


* (%) 12-mo. survey 6.9 2.0 0.08 0.230
 Received SNAP* (%) 12-mo. survey 51.4 6.1 0.12 0.036
 Received WIC* (%) 12-mo. survey 16.2 0.8 0.02 0.710
 Received public housing/rental assistance* 12-mo. survey 7.1 1.6 0.06 0.337 (%)
 Received Families Firs


 times a week to almost every day 54.2 –1.6 –0.03
  Score on familial closeness scale (range: 12-mo. survey 7.9 0.1 0.01 0.801 0–18)*
 Biological family member would offer 12-mo. survey 0.497 respondent a place to stay if needed* (%)
Not at all likel


t the cost of delivering the LifeSet program was $11,841 per participant in 2015 dollars (see Table C.1). These costs consist of staff salaries and benefits, Youth Villages overhead, travel, operations and maintenance, and direct support to youth.
Ta


ager (%) 12-mo. survey
More than once per week 21.9 8.2
 Once per week   38.5 12.1
More than once per month 4.6 6.2
 About once per month 6.7 10.4
 Less than once per month 2.5 6.5
 Did not have a case manager 25.7 56.5
  Received help preparing for


ants, we assume that those sessions were also one hour, on average. Thus, we estimate about 15 hours of contact with a case manager for control group members.
Hourly Cost for Case Worker Contact
The final step is to estimate the cost per hour of cont


n the estimate of program services used by the control group. (One exception was the estimated earning effect, in which the smaller estimate was taken from the two-year impact based on the administrative survey and the larger estimate was taken from


r and upper bounds used in the more and less conservative assumptions
(Chapter 3)
    Administrative cost rate for transfer programs Uniform Range of 10% to 20%
    Deadweight-loss parameter Triangle Minimum of 0, midpoint of 0.25, maximum of 0.50
 N


, John Eric Humphries, and Nicholas Mader, “The GED,” in Eric A. Hanushek, Stephen J. Machin, and Ludger Woessmann, eds., Handbook of the Economics of Education, Vol. 3, Amsterdam: North-Holland, 2011, pp. 423–484.
Heckman, James J., and Paul A. LaFo


of Two Interventions,” Journal of Studies on Alcohol, Vol. 63, No. 2, 2002, pp. 219–228.
Steurle, Eugene, and Leigh M. Jackson, eds., Advancing the Power of Economic Evidence for Making Informed Investments in Children, Youth, and Families, Washingto


  CORPORATION
MALCOLM V. WILLIAMS, LAURIE T. MARTIN, JESSICA SOUSA
Early Learnings from
the Hospital Community Cooperative of the American Hospital Association
Health is increasingly understood to be shaped by more than individual genetics, clinical


rganizations together to collectively address key SDOH in their communities and pro- mote health equity. The effort builds on the AHA’s #123forEquity Pledge to Act Campaign (American Hospital Association, undated-a), which has chal- lenged hospitals


on, this report should be of use to foundations or other funding agencies that might support similar partnered approaches to improve health inequity in their communities.
Approach
We collected data from each of the ten teams at two points in time—in


dress SDOH.
Teams provided examples of how their capac-
ity to address SDOH had expanded. These included increases in executive leadership support and funding for addressing SDOH and health equity; identifica- tion of new partners and a strengthening


ed at least one partner. But at baseline, only two teams felt that they had “the right types of partners to address health equity”; the other eight felt they had “some of the right partners.” At the end of the first year, eight teams felt that they h


ess encampment within its community during the HCC project period. Although this effort was independent of the HCC project, the team was able to quickly mobilize in support of its community partners to support housing placement for seven individuals


sing policy change (data not shown), others were poised to take advantage of policy win- dows and opportunities for alignment.
Teams Appreciated the Flexibility of HCC Funding and Protected Time to Work on Projects
Each team received $10,000 and valu


ised to take advantage of opportunities when they arose.
  7
Lessons Learned
This inaugural cohort generated some helpful lessons learned for hospitals and health systems looking to develop partnered approaches to addressing SDOH and health equity in


H. Second, because this pilot involved only ten sites, these results may not be representative of what other hospital or health system partnerships could achieve in the same period. Further compounding this issue of generalizability is that teams cam


partners:
• CHRISTUS St. Vincent
• Grady Health System
• Holy Name Medical Center
• Hurley Children’s Hospital
• MultiCare Health System
• Parkland Health and Hospital System
• Sharp HealthCare
• Truman Medical Center Behavioral Health • University o


8 during which teams worked with technical assistance providers to define and refine their projects. RAND researchers collected data at two time points: January 2019 (baseline, allowing time for teams to finalize projects) and October 2019 (follow-up


   INTRODUCTION
  CHALLENGES FOR THE U.S. JUSTICE SYSTEM
  RESULTS
We identified the following four categories of chal- lenges that come from changes in society as a whole:
• shifts in the amount and nature of crime
• changing needs of contact popula


local criminal justice agencies and fostering the adoption of new technologies, policies, and
2
practices that can improve agencies’ performance. To advance that mission, NIJ sponsored the Priority Crimi- nal Justice Needs Initiative,
a five-year coo


ities Workshop
June 2016: Law Enforcement Advisory Panel Mid 2016: Corrections Futures Interview Panel
February 2017: Potential Impact of Electronic Media Use on the Right to a Fair Trial Workshop
March 2017: Workforce Issues in Corrections Workshop


m change. The issues and problems in many of these areas of concern have been the focus of other research. To link the results of this effort to that broader foundation of scholarship, we provide examples of such studies throughout. In the remainder


ond to crimes where the perpetrators might be thousands of miles away and might have entered their jurisdiction only virtually. Cybercrimes also create challenges for court practitioners in both maintaining the expertise in technology needed to under


ed the need for model policies
and procedures that are specific to such new types of crime as “SWATing,” or calling the police from a falsified telephone number to trigger a law enforcement tactical response to another address. The law enforcement an


h response, courts wrestle with the right approach for individuals whose offending might be driven by a treatable illness, and corrections agencies have become the nation’s de facto primary institutions for mental health care delivery (Al-Rousan et a


ed that research and measures also are
The needs of the multilingual populations served by criminal
justice agencies were the source of a very specific technology requirement across the three sectors: technology for real-time language translation.
ne


 agencies deploying technologies that might have the potential for misuse or abuse.
fore, for the viability of such strategies as community policing, which rely on collaborative approaches to solve problems (Slocum, 2018; Murphy, Madon, and Cherney,


tage and other law enforcement surveillance data. Clear guidelines were felt to be needed to distinguish between data that should be published to improve transparency and fight crime and data that, if released, could be readily exploited by criminals


g police use of force and the disparate impacts on communities of such strategies as order- maintenance policing (Tyler, Jackson, and Mentovich, 2015).
It also was mentioned in our discussion with respect to future technology-focused public safety st


court proceedings, such as alternative dispute resolution, restorative justice, arbitration and mediation, and diversion- ary programs. Law enforcement also prioritized the need for research and measures to assess the effectiveness of different


st successful.
• Develop better tools to sort cases and match them with the process that is most likely to get to an outcome efficiently and effectively (e.g., negotiation, trial, diversion, specialty court), including collecting data to inform the a


e services (Bureau of Justice Statistics, undated).
Decentralization has significant consequences for criminal justice innovation. This atomization can be a boon to innova-
Information sharing and contractual mechanisms were recommended as options fo


rity of proposed solutions for this issue focused on procurement policies and contractual requirements,
  Practitioners across all sectors of the justice system complained that their agencies are drowning in a flood of data.
which could be potentiall


across the country, among different levels of government, or even international. Even within single geographic jurisdictions, the justice system requires information sharing to work effectively. For example, there is a need for data to be shared bet


. Among the types of tools mentioned were mobile devices that securely deliver inmate information and body-worn comput- ers to improve the effectiveness and efficiency of community corrections. Agencies also highlighted the need for procedural and te


reconstituting community correc-
 tions operations in a locally disruptive event that might affect both staff and supervisees.
Common Innovation Needs and Opportunities
Solutions to enduring issues raised largely by corrections
and law enforcement pa


imes argued that raising standards for recruitment—for example, requiring post–high school education for public safety officers—would be valuable in making the justice system more effective, they also pointed out that recruitment difficulties led to


on improving equipment to better protect criminal justice staff, including making changes to personal equipment and disseminating effective practices for selecting and carrying gear. Participants also described the need for novel tools to allow offi


oncerns about rates of criminal
recidivism. Most criminal justice offenders are released back into their communities at some point, and the rate at which they fall back into crime calls into question the capability of current efforts to promote behav


tive.
• Hold more interagency exercises to build relationships between agencies and bolster preparedness for large- scale incidents, where success depends on using common resources effectively.
• Address shortfalls in the ability to notify individual


Opportunities
Although many participants argued that the justice system requires more resources or different funding streams and that leaders should better advocate for those needs, a wide variety of cross-sector needs focused on how justice agencies


 and viable solutions can be quite different for agencies that are responsible for geographically dispersed and sparsely populated rural areas than for those that are responsible for densely populated urban jurisdictions.
Notes
1 The Initiative was a


anges that could be made using existing technology and business practices that possibly could improve performance in the relatively near term.
Although the inputs from our large group of workshop participants over the course of the project provided a


-Tech-Brings-New- Issues-to-Public-Safety-Agencies.html
Collom, Lindsey, “When Police Officers Use Social Media, Their Departments See Risks,” Arizona Republic, April 17, 2012.
Cummings, Scott, and Deborah Rhode, “Access to Justice: Looking Back, Thi


kson, Annie Brothers, Mark Barrett, Dulani Woods, and Michael J. D. Vermeer, A Database of Criminal Justice Needs for Innovation: Requirements for Developers and Funders—User Guide, Santa Monica, Calif.: RAND Corporation, TL-352-NIJ, 2020. As of Marc


orithmic Management,” Big Data and Society, 2018, pp. 1–16.
Link, Nathan W., “Criminal Justice Debt During the Prisoner Reintegration Process: Who Has It and How Much?” Criminal Justice and Behavior, Vol. 46, No. 1, 2019, pp. 154–172.
Lofstrom, Magnu


RAND Corporation, “Priority Criminal Justice Needs Initiative,” webpage, undated. As of February 1, 2020: https://www.rand.org/well-being/justice-policy/projects/priority- criminal-justice-needs.html
Robbins, Ira P., “Lessons from Hurricane Katrina:


on Team Model of Police Response to Mental Health Crises: A Primer for Mental Health Practitioners,” Best Practices in Mental Health,
Vol. 8, No. 2, 2012, pp. 71–79.
Wood, Emily F., Monica K. Miller, and Tatyana Kaplan, “Specialty Courts: Time for a


unen, Andrew Lauland, Kristin Leuschner, Jocelyn Lofstrom, Mauri Jun Matsuda, Heather McCracken, Rosa Meza, Paul Ng, Cordaye Ogletree, Rachel Ostrow, Bryce Pardo, Catherine Plummer, Laura Poole, Carter Price, Rajeev Ramchand, Adrian Salas, Jessica Sa


on science and technology policy, criminal justice, national security, and emerging technologies and innovation. His recent research involves the policy, procedure, and technology needs of criminal justice agencies, development planning and program


the RAND Corporation. Her research focuses on national security, performance measure development, program evaluation, emerging technologies, public health systems, and workforce training. She received her master’s degree in public health.
 About This


 prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest.
RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors. R® is a registered trademark.
 CORPORATION
RR-4242-NIJ


Breaking Barriers: A Rapid Rehousing and Employment Pilot Program for Adults on Probation in Los Angeles County
Evaluation Report
SARAH B. HUNTER, ADAM SCHERLING, MELISSA FELICIAN, SANGITA M. BAXI, MATTHEW CEFALU
  SOCIAL AND ECONOMIC WELL-BEING
Spon


 recommendations regarding future programming and research. This report should be of interest to state and county governments serving populations on felony probation and other organizations serving criminal justice–involved populations.
RAND Social a


g options in Los Angeles
• logistical barriers from the geographic spread of the county
• caution among landlords and property managers regarding the temporary subsidy model
and serving the criminal justice–involved population
• staffing concerns fro


ogram (i.e., whether participation in the program had an effect on recidivism, housing stability, employment, or income).
Although this evaluation cannot make strong conclusions about the effects the program might have had on the outcomes of interest


ome point
- Fifty-five (24 percent) had a job at program entry, 52 (23 percent) started without a job and gained employment, and 20 (9 percent) started with a job and lost employment.
- The average wage rate among those employed was $12 per hour, the


example, most participants did not achieve great gains in employment and associated income during the two-year program period. More information might be helpful in determining how to achieve these goals. There are many possible barriers to employmen


stakeholders moving forward. Our data suggest, and our interactions with stakeholders confirm, that many participants will not be able to support the full rental amount at the end of the 24-month subsidy period, particularly as rents in Los Angeles


sing obtainment, employment, recidivism) suggest that there are important participant and program factors to consider when scaling up such a program, in that the program as initially designed was associated with more progress in terms of job attainme


nt that put them at risk of reoffending and returning to jail or prison (Bradley et al., 2001; Petersilia, 2001; Petersilia, 2005; Gideon and Sung, 2011). Many of the individuals face difficulties accessing housing because probation conditions may re


rs initiative. More specifically, the Los Angeles County Probation Department—in collaboration with the Los Angeles County Department of Health Services (DHS), the Corporation for Supportive Housing (CSH), the Conrad N. Hilton
 1
Foundation, and two


hat rapid rehousing may be effective in helping families and individuals exit homeless shelters more quickly and reduces returns to shelters (Cunningham, Gillespie, and Anderson, 2015). However, it does not solve the housing affordability problem, wi


a sense of hope for the future and a way back to self-sufficiency (Shaheen and Rio, 2007; Marrone, 2005). Contrary to prevalent stereotypes, research has shown that people experiencing homelessness do want to work, including those who have disabiliti


nificant effects on recidivism (Miller and Ngugi, 2009). One recent evaluation of a small pilot project—entitled Jail in Reach, a program to rapidly rehouse individuals who were newly released from a county jail or prison in the state of Texas with a


e Breaking Barriers pilot program. We conducted a formative study during the first year of program implementation and a summative evaluation three years after the program was launched. The primary research questions were the following:
1. How does th


ations.
The primary goals of the Breaking Barriers pilot program were to (1) reduce recidivism, (2) improve participants’ housing stability, and (3) improve employment incomes sufficiently for individuals to take over their own rental payments by the


reed that a referral was appropriate for Breaking Barriers, they were offered an opportunity to enroll in the program.
If an individual was determined ineligible, they were often referred to a more suitable program that could better meet their needs.


rs, starts immediately and is followed by coordination of the employment and housing services, which we describe in more detail in the following sections.
8
Figure 2.1. Breaking Barriers Program Logic Model
    INPUTS
• Funding: Hilton Foundation ($2


 • duration of housing subsidies
Income and employment outputs (BC, Chrysalis):
• % of eligible participants applied for benefits (i.e., TANF, General Relief, SNAP)
• frequency of engagement with BB program services (i.e., housing stabilization, case


. The case manager, in collaboration with the participant, develops an individualized case management service plan that identifies short- and long-term goals. The case manager supports the participant through the housing search and move-in process an


cipants to prepare them for interviews and help them search for, apply for, and access jobs on a part-time or full-time basis. Some participants also benefited from employment with Chrysalis’ in-house contracting and temp agencies on a short-term bas


ing their full rental amount over time. If a participant’s contribution to rent exceeded their total unit rent, then the participant graduated from the program.
13
Chapter 3. Evaluation Design
The evaluation was conducted in two stages. During the fi


aunch of the program. The 19 interviewees all played key roles in program design or implementation. For example, participants included direct service staff, such as case managers, employment specialists, housing coordinators, housing specialists, and


rt.
We did not focus on or attempt to assess fidelity to the Housing First model during this qualitative assessment. No standardized scales were used to rank or rate interviewees’ agreement with concepts and ideas about aspects of implementation, and


ng change in program outcomes in the less rigorous design options (i.e., intervention group–only designs). Stakeholders concluded that it was not feasible to implement a randomized controlled trial because there were not enough resources to screen mo


rcation of roles and responsibilities across functions. All interviewees expressed very clearly where their roles began and ended, and clearly articulated the roles of other program staff. No overlapping roles or duplication of effort was found. Even


noted that the program was individualized to the client, making it tailored to the specific needs of program participants. For example, case managers meet clients close to where they work or live, and participants who are parents receive additional r


ards (for public transportation use) to alleviate transportation barriers, and employed several clients directly through Chrysalis Works, an internal service company, to give them an opportunity to start building an employment history.
   20
 Facilit


ew terrain around housing clients with criminal histories, often working harder to find housing in locations that clients think would be safer for them based on their personal history or that clients have indicated as their preference. Case managers


ored differently to better engage them in the program.
22
Table 4.2. Stakeholder-Reported Program Challenges Challenges to Implementation
External to the program but central to program design
Housing market and high and rising rents in Los Angeles: A


sing coordinators. The increase in clients brought several logistical and time management challenges. With such a client-oriented program tailored to meet the clients where they are living and work with them toward specific goals (which are developed


s the housing affordability crisis and size of Los Angeles County. When these results were shared with stakeholders in response to a more-recent review of the study findings, the stakeholders reported ways in which they addressed many of the cited ch


lthough individuals could receive up to a 24- month rental subsidy that started sometime after program enrollment, by using the two-year program period, we balanced inclusion of participants who had adequate opportunity to receive the different progr


61 (18 percent of those referred) did not attend an eligibility meeting. Of those who attended an eligibility meeting, 54 (16 percent of those referred) were not enrolled or were enrolled after November 2016 and thus were excluded from the analysis


or child support.
• Only 1 percent were reported as veterans.
During the six months prior to program entry, participants reported residing in a variety of different settings. The following is a breakdown of places in which participants reported resid


using or residential substance use treatment, or whereabouts were unknown at exit, or participant had died); or (3) recidivism exit (i.e., exited program because of incarceration). Using these criteria, 11 participants (27 percent) were classified as


evel appeared to be related, such that individuals with a higher education at program entry were more likely to obtain employment during the two-year program period, but this result was not statistically significant after correcting for multiple test


r of the program (i.e., months 1–6) and this number rose to more than 46 percent in the second half of the first year, then increased to 51.9 percent in the first part of the second year, and up to 58 percent in the last quarter of the program period


 days) compared with men (166
days).
• Individuals who were Hispanic were more likely to have a permanent-tenure position
(on average 75 percent) compared with black participants or individuals of other races/ethnicities (49 percent).
35
No other ass


had information about the client’s portion of income being applied to rent, their rental subsidy amount, and total rent. At housing exit or the end of our evaluation period (whichever came first), we had access to the same data as at housing move-in


 Move-In Median
At Two-Year Mark or Program Exit
        Rental Exit Status Information
Stably housed at exit
Client portion
BC subsidy
Total rent Incarcerated at exit
Client portion BC subsidy Total rent
n Mean 19 $108.27
19 $899.49 19 $1,006.06
25


 findings, if all participants were black, then 14.1 percent would have received a felony conviction; if all the participants were white, then the model estimates that 38.2 percent of them would have received a felony conviction.
Next, we explored ti


quarter of participants started the program without a job and ended the program employed; less than 10 percent started off employed and ended up without employment. The majority of participants were not employed at program entry or at program exit or


under probation or parole in California have two-year felony reconviction rates of around 22 percent (Bird, Goss, and Nguyen, 2019) with rates varying depending on whether an individual was initially sentenced to prison (25 percent), jail (28 percen


on gathering, program planners may want to tailor the program based on what they learn. For instance, this might entail enhancing the employment support arm of the program or developing more-specific program participant eligibility criteria. One exam


age of these opportunities and potentially expanding the opportunities available for participants to gain a better compensated skill set are necessary.
Study Limitations
Several limitations to this evaluation should be noted. Regarding the formative


o understand the objectives of the Breaking Barriers program, the way the program is being implemented and any challenges faced to date in implementing the program.
Today we want to talk about the Breaking Barriers Program from your point of view. I


m thinking of things like your organization’s mission, the clients you serve, and range of services the agency provides generally (not just to Breaking Barriers).
3. What services does your agency provide to Breaking Barriers and what is your specifi


aining Employment Among Those Unemployed at Program Entry
Table B.1 shows the model of employment status on the last day (at program exit or at two years) among clients unemployed on first day (n =166): 173 were unemployed on the first day and six pa


less than a high school diploma/unknown, AB 109 supervision status, and in the program for months 0–6.
50
Table B.3. Number of Days Employed
            (Intercept)
Age
Female
Race/ethnicity–Hispanic
Race/ethnicity–white
Race/ethnicity–other
Educatio


Education–postsecondary, some college, bachelor’s or higher
Education–high school diploma Supervision status–ATC Program months 7–12 Program months 13–18 Program months 19–24
Estimate Std. Error
14.717 1.990 -0.061 0.045 0.061 0.863 -0.580 0.870 -1.


0.358 -0.903 -0.012 1.600 1.225
Pr(>|t|)
0.028 0.909 0.881 0.001a 0.028 0.155 0.305
0.721 0.368 0.991 0.111 0.222
      Race/ethnicity–Hispanic
Race/ethnicity–white
Race/ethnicity–other
Education–postsecondary, some college, bachelor’s or higher
Educ


hnicity–other
Education–postsecondary, some college, bachelor’s or higher
Education–high school diploma Supervision status–ATC
Coef Exp(coef)
-0.026 0.974 0.254 1.289 0.263 1.301 0.249 1.282 -0.589 0.555 -0.709 0.492
0.234 1.264 -0.058 0.944
Se(coef)


ncisco, Calif.: Public Policy Institute of California, 2019. As of October 27, 2019: https://www.ppic.org/wp-content/uploads/recidivism-of-felony-offenders-in-california.pdf
Bradley, Katharine H., R. B. Michael Oliver, Noel C. Richardson, and Elspeth


: Housing Decisions Among Families Exiting Shelter,” Housing Policy Debate, Vol. 24, No. 2, 2014, pp. 364–386.
Flynn, Kathleen Nye, “Putting Teeth into AB 109: Why California’s Historic Public Safety Realignment Act Should Require Reentry Programming


ychiatric Rehabilitation, Vol. 8, No. 1, 2005, pp. 13–35.
Miller, Marna, and Irene Ngugi, Impacts of Housing Supports: Persons with Mental Illness and Ex-Offenders, Olympia, Wash.: Washington State Institute for Public Policy, Document No. 09-11-1901


 SECOND EDITION
The
Science
of Gun
Policy
A Critical Synthesis of Research Evidence
on the Effects of Gun Policies in the United States
Rosanna Smart, Andrew R. Morral, Sierra Smucker, Samantha Cherney, Terry L. Schell, Samuel Peterson, Sangeeta C. A


es; and the first edition of this report, which synthesized the available scientific data on the effects of 13 types of firearm policies on a variety of outcomes related to gun ownership. This second edition builds on that 2018 report, adding five ne


....... 109
CHAPTER SEVEN
Extreme Risk Protection Orders. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 State Implementation of Extreme Risk Protection Orders . . . .


.. 196
CHAPTER THIRTEEN
Firearm Sales Reporting, Recording, and Registration Requirements. . . . . . . . . . . . . . . . . . 197 State Implementation of Firearm Sales Reporting, Recording, and Registration
Requirements................................


ios Associated with the Effect of Stand-Your-Ground
Laws on Violent Crime. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 244
16.3. Incidence Rate Ratios Associated with t


o support claims about the effects of various policies.
Methodology
We used Royal Society of Medicine guidelines for conducting systematic reviews of a scientific literature (Khan et al., 2003). We focused on the empirical literature assessing the ef


lass of policies being considered; presents and rates the avail- able evidence; and describes what conclusions, if any, can be drawn about the policy’s effects on outcomes.
In many cases, we were unable to identify any research that met our criteria


same direction, and contradictory evidence was
not found in other studies with equivalent or stronger methods.
5. Supportive evidence. This designation was made when at least three studies not compromised by serious methodological weaknesses found su


   I
  Child-Access Prevention Laws
      êL
 êM
  êS
  I
   I
    I
 êL
  I
  Stand-Your-Ground Laws
   I
I
    éM
 éS
  I
 I
    Bans on Low-Quality Handguns
        I
  I
      I
   I
            Bans on the Sale of Assault Weapons and High


es
NOTE: I = inconclusive; L = limited; M = moderate; S = supportive. When we identified no studies meeting eligibility criteria, cells are blank.é= the policy We concluded that there is limited evidence that licensing and permitting requirements de


ale of Assault Weapons and High-Capacity Magazines
    Firearm Sales Reporting, Recording, and Registration Requirements
    Lost or Stolen Firearm Reporting Requirements
    Firearm Safety Training Requirements
    Waiting Periods
    Licensing and


ides among youth and limited evidence that the laws reduce total (i.e., firearm and nonfirearm) suicides among youth. In addition, there is limited evidence that these laws may reduce unintentional firearm injuries and deaths among adults. There is s


they reduce total suicides and firearm homicides.
• Recommendation 5. States without waiting-period laws should consider adopting them as a strategy for reducing suicides and homicides.
Conclusion 6. There is limited evidence that licensing and permi


 an important new federal initiative that, if sus- tained in future budgets, could make enormous contributions to knowledge about how to prevent gun-related injuries and deaths.
Summary xxv
xxvi The Science of Gun Policy: A Critical Synthesis of Rese


l victimization estimates. The current goal of gen-
erating such estimates for 22 states is a reasonable compromise between cost and the public’s need for more-detailed information. However, the bureau should con- tinue to expand its development of m


ce of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of U.S. Gun Policies
duce more readily detectable effects on public safety, health, and industry outcomes. The United States has a large stock of privately owned guns in circu


 Regina Kunz, Jos Kleijnen, and Gerd Antes, “Five Steps to Conducting a Systematic Review,” Journal of the Royal Society of Medicine, Vol. 96, No. 3, 2003, pp. 118–121.
National Research Council, Firearms and Violence: A Critical Review, Washington,


der
ERPO extreme risk protection order
FBI Federal Bureau of Investigation
FFL federal firearms licensee
IRR incidence rate ratio
K–12 kindergarten through grade 12
N/A not applicable
NIBRS National Incident-Based Reporting System
NICS National Insta


ferences.html).
 3
4 The Science of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of U.S. Gun Policies
Entrenched disagreements on gun policy are not surprising, given the number and variety of contested and contradictory studi


lence of gun ownership has an effect on crime or suicide, problems in the measurement of defensive gun use, and what we can learn from Australia’s experience banning some semiautomatic weapons. Those essays remain available on the project website. Th


ales and transfers
6. background checks
7. licensing and permitting requirements
8. waiting periods
9. firearm safety training requirements
10. lost or stolen firearm reporting requirements
11. firearm sales reporting, recording, and registration req


occurs is not known. Thus, for laws that increase or decrease firearm suicides, the effects on total suicides are likely smaller and harder to detect but are fundamentally of greater interest for public policy. For this reason, we examine the effects


, and murder or nonnegligent manslaughter. The last category excludes deaths caused by suicide, negligence, or accident, as well as justifiable homicides (such as the killing of a felon by a peace officer in the line of duty) (FBI, 2016e).
Gun polici


gency departments for assault; of these, 60,470 (3.8 percent) were firearm-related (CDC, 2017c).6
Unintentional Injuries and Deaths
Like official statistics on suicide, those on unintentional injuries and deaths in the United States are compiled by t


See Campbell and Nass, 2019.
Introduction 9
 10 The Science of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of U.S. Gun Policies
Often, the attention of researchers, the public, and policymakers is focused on mass shooting inc


es with available NVDRS data (Conner et al., 2019). Several years ago, the FBI announced plans to begin a new data collection effort designed to track all incidents in which law enforcement seriously injure or kill citizens (Kindy, 2015), but results


in using data on justifiable homicides to understand how specific firearm policies (e.g., stand-your-ground laws) influence defensive gun use because the policies themselves change what is classified as justifiable. The literature on defensive gun us


had active licenses to sell firearms in the United States. Just more than 47 percent of these licenses were held by dealers or pawnbrokers, 41 percent were held by collectors,8 about 10 per- cent were held by manufacturers of ammunition or firearms,


de Reports,” American Journal of Public Health, Vol. 106, No. 5, May 2016, pp. 922–927.
Barber, Catherine, and David Hemenway, “Too Many or Too Few Unintentional Firearm Deaths in Official US Mortality Data?” Accident Analysis and Prevention, Vol. 43


r, Clayton E., “Why the FBI’s Justifiable Homicide Statistics Are a Misleading Measure of Defensive Gun Use,” University of Florida Journal of Law and Public Policy, Vol. 27, 2016, pp. 505–514.
Duwe, Grant, “The Patterns and Prevalence of Mass Public


M., “On the Sources of Ordinary Science Knowledge and Extraordinary Science Ignorance,” in Kathleen Hall Jamieson, Dan Kahan, and Dietram A. Scheufele, eds., Oxford Handbook of the Science of Science Communication, Oxford, UK: Oxford University Pres


creation and Tourism, Lansing, Mich.: Michigan Land Resource Project, Public Sector Consultants, 2001.
Niforatos, Joshua D., Alexander R. Zheutlin, and Richard M. Pescatore, “Public Interest in Gun Control in the USA,” Injury Prevention, Vol. 25, Sup


 National Survey of Fishing, Hunting, and Wildlife-Associated Recreation, Washington, D.C., FHW/16-NAT, April 2018.
World Health Organization, Preventing Suicide: A Global Imperative, Geneva, 2014.
Introduction 17
CHAPTER TWO
Methods
 In this second


luding the White House and other U.S. government organizations, advocacy organizations focused on gun policy (such as the National Rifle Association and the Brady Campaign to Prevent Gun Violence), academic organizations focused on gun policy or gun


di- tional policies, programs, or strategies that had been noted by advocacy groups, had been implemented by states, or had garnered media or legislative attention since we assembled the original list. From this updated list, each member of our resea


, Papachristos, and Hureau, 2014; Braga and Weisburd, 2012; Braga, Weisburd, and Turchan, 2018). Accordingly, we offer no conclusions on the efficacy of such approaches. However, we recognize the potential importance of these other interventions and


ference Service
Social Science Abstracts
EconLit
Business Source Complete
WorldCat Scopus
LawReviews (LexisNexis)
Details
National Library of Medicine’s database of medical literature. Not used for gun industry or hunting searches.
Journal articles,


 outcomes included the following:
• gun OR guns OR firearm* OR handgun* OR shotgun* OR rifle* OR longgun* OR machinegun* OR “machine gun*” OR pistol* OR “automatic weapon” OR “assault weapon” OR “semi-automatic weapon*” OR “automatic weapons” OR “ass


al step, the next steps in the study review process used stan- dardized review criteria to identify all studies with evidence for policy effects meeting minimum evidence standards. Table 2.2 describes our inclusion and exclusion criteria for the titl


rm policy, or include quantita- tive analyses. In addition, we excluded studies if they were commentaries or conceptual discussions, systematic reviews or meta-analyses, case studies, dissertations, confer- ence abstracts, legal statutes or congressi


nerally requires three types of evidence (Mill, 1843):
• The cause and effect regularly co-occur (i.e., association).
• The cause occurs before the effect (i.e., precedence).
• Alternative explanations for the association have been ruled out (i.e., e


imates, standard errors, confidence intervals). One reviewer extracted data of interest from each study and entered them into the standardized form. A second reviewer independently extracted information on estimated effects and checked all other fiel


existing literature may have low power, poorly calibrated standard errors and Type 1 error rates, and other problems. Nevertheless, the criteria that we selected for evaluating individual studies represent threats to validity that we could evaluate o


of available studies as a whole to make a determina- tion about the level of evidence supporting the effect of the policy on each outcome.
Methods 29
30 The Science of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of U.S. Gun P


the estimates are not necessarily identical, they are not independent and thus cannot be combined as a single estimate.
While the selection of p < 0.20 as the criterion for rating evidence as suggestive is arbitrary, this threshold corresponds to eff


tinct data sets reflects the fact that many gun policy studies use identical or overlapping data sets (e.g., state homi- cide rates over several years). Chance associations in these data sets are likely to be identified by all who analyze them. There


011)
3 Aneja, Donohue, and Zhang (2011)
4 Aneja, Donohue, and Zhang (2014)
5 Anestis and Anestis (2015)
6 Ayres and Donohue (1999)
7 Ayres and Donohue (2002)
8 Ayres and Donohue (2003a)
9 Ayres and Donohue (2003b)
10 Ayres and Donohue (2009a)
11 Ayre


l. (2015)
107 Sen and Panjamapirom (2012)
108 Shi and Lee (2018)
109 Siegel et al. (2017b)
110 Steidley and Kosla (2018)
111 Strnad (2007)
112 Swanson et al. (2013)
113 Swanson et al. (2016)
114 Vigdor and Mercy (2003)
115 Vigdor and Mercy (2006)
116


fficer- Involved Shootings
                         Mass Shootings
  79
         46, 57, 79
  16, 79
  72, 79
     16
 16, 44, 46
    Unintentional Injuries and Deaths
  43
                       Violent Crime
 86, 102, 106
  107, 112, 113
 28, 100,


come of interest was determined. We then used the base rate to transform the regression esti- mate, β, to an IRR using the following formula:
IRR = (average base rate+β). average base rate
However, if the linear model used a logged dependent variable


thodological rigor. These limitations in the existing literature led us to pursue a more qualitative evaluation of the conclusions that available studies can support. As more research or relevant databases become available, meta-analyses may become f


and Daniel S. Nagin, “Do Right-to-Carry Laws Deter Violent Crime?” Journal of Legal Studies, Vol. 27, No. 1, 1998, pp. 209–219.
Blau, Benjamin M., Devon H. Gorry, and Chip Wade, “Guns, Laws, and Public Shootings in the United States,” Applied Economi


in Permit-to- Purchase Handgun Laws in Connecticut and Missouri on Suicide Rates,” Preventive Medicine, Vol. 79, 2015, pp. 43–49.
Crifasi, Cassandra K., Keshia M. Pollack, and Daniel W. Webster, “Effects of State-Level Policy Changes on Homicide and


European Economic Review, Vol. 81, 2016, pp. 32–67.
Duwe, Grant, Tomislav Kovandzic, and Carlisle E. Moody, “The Impact of Right-to-Carry Concealed Firearm Laws on Mass Public Shootings,” Homicide Studies, Vol. 6, No. 4, 2002, pp. 271–296.
Edwards, G


rrok, “Using Placebo Laws to Test ‘More Guns, Less Crime,’” Advances in Economic Analysis and Policy, Vol. 4, No. 1, 2004.
Hempstead, Katherine, and Antonio Rodríguez Andrés, Gun Control and Suicide: The Impact of State Firearm Regulations, 1995–20


un Violence 1994–2003, Washington, D.C.: National Institute of Justice, 2004.
Koper, Christopher S., and Jeffrey A. Roth, “The Impact of the 1994 Federal Assault Weapon Ban on Gun Violence Outcomes: An Assessment of Multiple Outcome Measures and Some


2, June 2003, pp. 185–198.
———, “Abortion and Crime: Unwanted Children and Out-of-Wedlock Births,” Economic Inquiry, Vol. 45, No. 2, 2007, pp. 304–324.
Luca, Michael, Deepak Malhotra, and Christopher Poliquin, The Impact of Mass Shootings on Gun Poli


ds 45
46 The Science of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of U.S. Gun Policies
Morral, Andrew R., Terry L. Schell, and Margaret Tankard, The Magnitude and Sources of Disagreement Among Gun Policy Experts, Santa Moni


Handgun Laws on Crime: Beyond the Dummy Variables,” International Review of Law and Economics, Vol. 23, No. 2, 2003, pp. 199–216.
Rudolph, K. E., E. A. Stuart, J. S. Vernick, and D. W. Webster, “Association Between Connecticut’s Permit-to-Purchase Ha


4.
Webster, D., C. K. Crifasi, and J. S. Vernick, “Effects of the Repeal of Missouri’s Handgun Purchaser Licensing Law on Homicides,” Journal of Urban Health, Vol. 91, No. 2, 2014, pp. 293–302.
Webster, Daniel W., and Marc Starnes, “Reexamining the A


ons (e.g., age 18 is the minimum legal age for handgun possession at the federal level). Other policies establish mechanisms for the removal of firearms from individuals who are prohibited possessors or expand firearm prohibitions to apply to individ


d 22–27; 8.4 percent for those aged 28–33) (calculated using data from the Centers for Disease Control and Prevention [CDC], 2019a). In theory, therefore, stricter age limits on purchasing or possessing a firearm could reduce the incidence of defensi


e retail sources.
However, the effectiveness of minimum age laws is likely to be tempered by the ease with which many youth can obtain firearms from sources other than legal retail- ers. For example, an analysis of data from the National Fatality Rev


equently used than long guns in firearm suicides and violent crime, so, in theory, raising the mini- mum age for such weapons could decrease violence without impacting lawful activi- ties, such as hunting (Tritch, 2014). More-restrictive minimum age


m age requirements that exceed the federal minimum for handgun possession.3 Several states also have minimums that are higher than the federal law for long gun purchase and possession.
Eight states and the District of Columbia restrict all handgun sa


e District of Columbia restrict long gun possession to those aged 21 or older.8 The minimum age for possession of a long gun in Alaska, Minnesota, and New York is 16,9 and it is 14 in Montana.10
Effects on Suicide
Research Synthesis Findings
We ident


un possession that were in place prior to the 1994 federal minimum age restriction, (2) state laws prohibiting juvenile handgun possession enacted the same year as the 1994 federal minimum age restriction, and (3) the federal ban on juvenile handgun


empirical model but focusing on suicides by males and the 1995 to 2004 period after the federal minimum age law, Andrés and Hempstead (2011)
assessed how state bans on the purchase of firearms by minors affected suicide rates in four age groups (15–


ne that raised its minimum age for posses- sion did so the same year it implemented a minimum purchase law, making the effects of these laws confounded. Moreover, the statistical model had an unfavorable ratio of covariates to observations (less than


passage of the background check law. Conversely, an effect size of more than 1.00 indicates that the law appears to increase the outcome by a factor equivalent to the effect size value. When the CIs do not include the value of 1.00, the estimated ef


s by minors were associated with significantly lower rates of suicide among males aged 15–44.
Considering these findings and an assessment of the relative strengths of these studies, we find inconclusive evi- dence for how minimum age requirements fo


8 to 1999. Using fixed-effects models that controlled for national trends, average state effects, a lag of the dependent variable, and state-specific linear trends, the author jointly exam- ined the effects of (1) state laws prohibiting juvenile hand


the study period; in addition, every state but one that raised its minimum age for possession did so the same year it implemented a minimum purchase age law, making the effects of these laws confounded. Moreover, the statistical model had an unfavor


 these results, and in consid- eration of the relative strengths of these studies, we find inconclusive evidence for how minimum age requirements for pur- chasing a firearm affect total and firearm homicides.
Minimum age requirements for possessing a


3.
Figure 3.3
Incidence Rate Ratios Associated with the Effect of Minimum Age Requirements on Unintentional Injuries and Deaths
NOTE: IRR values marked with blue squares indicate that methodological concerns are discussed in the text. See Appendix B


e.g., on how these policies were coded), it is clear that the analysis used a linear model to pre- dict a rare dichotomous outcome. Therefore, model assumptions were likely violated, making model estimates and CIs unreliable.
Figure 3.4 displays the


web/ncipc/mortrate.html
Duda, M. D., and K. C. Young, Factors Related to Hunting and Fishing Participation in the United States, Harrisonburg, Va.: Responsive Management, report for the U.S. Fish and Wildlife Service, Grant # 14-48-0009-92-1252, 1993


vior, Vol. 4, No. 1, 1999, pp. 59–75.
Office of Juvenile Justice and Delinquency Prevention, OJJDP Statistical Briefing Book, Washington, D.C.: U.S. Department of Justice, May 25, 2016.
Puzzanchera, C., G. Chamberlin, and W. Kang, “Easy Access to the


 Young People in California: Descriptive Epidemiology and Gun Ownership,” Journal of Adolescent Health, Vol. 43, No. 6, 2008, pp. 619–622.
Minimum Age Requirements 71
CHAPTER FOUR
Prohibitions Associated with Mental Illness
 Federal law prohibits the


The Science of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of U.S. Gun Policies
Epidemiological evidence suggests that a diagnosis of mental illness alone has little relation to risk of interpersonal violence (Swanson et al.,


s than perpetrators of violence (Desmarais et al., 2014). One study of persons with severe mental illness (in treatment at mental health agencies in Chicago) found that their annual exposure to violent crime victimization was more than four times hig


suggests that some of these suicides might have been prevented if this population had been prohibited from obtaining a firearm.
To assess the effects of expanded mental health–related prohibitions, the ideal data would distinguish outcomes between th


or example, Alabama prohibits “anyone of unsound mind” from owning, possessing, or controlling a fire- arm. Arkansas prohibits those adjudicated as “mentally ill,” and Illinois’ law applies to anyone adjudicated to have a “mental disability.” See Ala


ealth–related prohibitions on suicide. Examining state-level data from 1995 to 2004, Andrés and Hempstead (2011) assessed how men’s suicide rates were related to different categories of firearm restrictions, including prohibitions for persons with a


r regression models included state-level time-varying covariates, state-level suicide rates in 1990, and fixed effects for year and census subregion. Sen and Panjamapirom (2012) found that, compared with back- ground checks that investigate only crim


associated with mental illness can reduce total suicides and firearm suicides. A third study reported finding no effect of expanded NICS reporting for mental illness–related prohibitions on suicide but did not provide detailed results.
Considering th


13) obtained data from 2002 to 2009 for individuals in Connecti- cut who had been hospitalized for schizophrenia, bipolar disorder, or major depressive disorder. The authors estimated changes in violent crime arrests for individuals with at least one


sed in the text. See Appendix B for details.
Prohibitions Associated with Mental Illness 81
  Conclusions
We identified two qualifying studies that estimated how laws prohibiting gun purchases by those with a mental illness affect violent crime or ho


y Support Fund, “Mass Shootings in the United States: 2009–2017,” December 6, 2018. As of May 15, 2019: http://everytownresearch.org/reports/mass-shootings-analysis/
Federal Bureau of Investigation, “Federal Denials: Reasons Why the NICS Section Deni


., and A. Panjamapirom, “State Background Checks for Gun Purchase and Firearm Deaths:
An Exploratory Study,” Preventive Medicine, Vol. 55, No. 4, 2012, pp. 346–350.
Substance Abuse and Mental Health Services Administration, Results from the 2017 Nati


ciety more broadly. In 1994, Congress enacted the Violent Crime Control and Law Enforcement Act (Pub. L. 103-322), making it illegal to pos- sess or receive a firearm while subject to a nontemporary restraining order protecting an intimate partner or


idents are killed by their unmarried partner. In 2008, the pro- portion of intimate partner homicides committed by a spouse (46.7 percent) was nearly equal to the proportion committed by a boyfriend or girlfriend (48.6 percent) (Cooper and Smith, 201


e to keep guns illegally.
Domestic violence–related firearm laws may also affect suicide rates. Research suggests that intimate partner homicides in which firearms are the primary weapon are more likely to result in the suicide of the perpetrator tha


ith firearms (Hemenway et al., 2019a).
Alternatively, the process or threat of removing a firearm may increase tensions in a domestic violence situation, which may increase the risk of violence to the victim, officers responding to the situation, and


rth Dakota, Ohio, Oregon, Pennsyl-
 Prohibitions Associated with Domestic Violence 89
possession when an order is issued ex parte.3 Twenty-eight states have expanded the applicability of the DVRO firearm prohibition beyond the definition in federal l


aler, he or she should be unable to complete the purchase. At least one state’s laws require notifying the person who filed the restraining order or was the victim of the misdemeanor crime of domestic violence that the abuser attempted to purchase a


cioeconomic and demo- graphic variables, and hunting licenses per capita, the authors found that prohibitions related to domestic violence conviction were associated with significantly lower rates of suicide among men aged 45–64; the estimated relati


 violence have
 uncertain
effects on total and firearm suicides.
Evidence for this relationship is
inconclusive.
 Prohibitions Associated with Domestic Violence 93
Effects on Violent Crime
Research Synthesis Findings
We identified six qualifying stud


sian (2016) examined the 1996 Lautenberg Amendment, federal legislation that extended prohibited-possessor status to those convicted of misdemeanor domestic violence offenses and that stipulated a mechanism for requiring that they surrender all firea


of misdemeanor crimes of domestic violence; (3) prohibitions against persons convicted of misdemeanor stalking; and (4) laws designating some stalking crimes as felonies, which could serve to prohibit firearm purchase and possession for domestic vio


for year and census subregion fixed effects, baseline state suicide rates, and numerous state-level time-varying covariates, the authors found that checks on restraining orders were associated with significantly reduced rates of total and firearm ho


Evidence for this relationship is
moderate.
 98 The Science of Gun Policy: A Critical Synthesis of Research Evidence on the Effects of U.S. Gun Policies
stantially similar data sets with largely overlapping time periods. One additional study used a


es (Zeoli et al., 2018). Considering these results, we find that there is inconclusive evidence for how state firearm prohibitions associated with stalking convictions influence firearm-related intimate partner homicides, and there is limited evi- de


ez, Carolina, Rachel P. Kurland, Emily F. Rothman, Megan Bair-Merritt, Eric Fleegler, Ziming Xuan, Sandro Galea, Craig S. Ross, Bindu Kalesan, Kristin A. Goss, and Michael Siegel, “State Intimate Partner Violence–Related Firearm Laws and Intimate Par


cides?” Journal of Policy Analysis and Management, Vol. 35, No. 1, Winter 2016, pp. 67–93.
Saltzman, Linda E., James A. Mercy, Patrick W. O’Carroll, Mark L. Rosenberg, and Philip H. Rhodes, “Weapon Involvement and Injury Outcomes in Family and Intima


icies, Alcohol Taxes and Police Staffing Levels on Intimate Partner Homicide in Large U.S. Cities,” Injury Prevention, Vol. 16, No. 2, 2010, pp. 90–95.
CHAPTER SIX
Surrender of Firearms by Prohibited Possessors
 Federal law bans the sale of firearms


s stipulate roles for judicial officers or law enforcement to ensure that firearms are surrendered or confiscated. In addition, 29 states require the surrender of firearms pursuant to domestic violence restraining orders, to last for the duration of


800. Upon a showing by clear and convincing evidence that a party has used, displayed, or threatened to use a firearm or other dangerous weapon in a felony or is ineligible to pos- sess a firearm under § 9.41.040, the court shall require that the par


restraining orders or those convicted of misdemeanors. Among the policies they examined were state laws allow- ing police to confiscate firearms from a domestic violence incident (they simultaneously examined state laws that allow police to make war


equirements for firearm relinquishment. Using data from 1991 to 2015 and negative binomial regression models with controls for the lagged inti- mate partner homicide rate, national trends, and region fixed effects, the authors found that the effects


omicides and firearm intimate partner homicides to be uncer-
tain. Additional analyses by Vigdor and
Mercy (2006) that focused on other
types of violent crime found signifi-
cant effects of confiscation laws indicat-
ing that they increase assaults,


vels on Intimate Partner Homicide in Large U.S. Cities,” Injury Prevention, Vol. 16, No. 2, 2010, pp. 90–95.
CHAPTER SEVEN
Extreme Risk Protection Orders
 Extreme risk protection order (ERPO) laws, sometimes known as gun violence restraining order or


le individuals to petition for an ERPO in emergency cases without waiting to provide notice of a hearing to the respondent. Like ex parte orders related to domestic violence (see Chapter Five), these laws are designed for situations in which the wait


their behavior rather than relying on specific criminal or mental health histories, then ERPO laws could decrease suicides and homicides among this population over and above the suicides and homicides prevented by existing prohibited purchaser laws (


ition for an ERPO to law enforcement officers. However, some laws extend that right to family members and medical professionals. In general, by expanding the ability to request an ERPO to family members and others with knowledge of the high-risk indi


l event. How- ever, applying information on suicide case fatality rates by method and assuming a counterfactual pre-intervention probability of using a gun in a suicide attempt that was informed by the relationship between state-level gun ownership a


7-2510.04.
3 California, Colorado, Delaware, Hawaii, Illinois, Maryland, Massachusetts, New Jersey, New York, Oregon, and Washington. See Calif. Penal Code § 18150; Colo. Rev. Stat. § 13-14.5-104; Del. Code Ann. Tit. 10, § 7701, 7704 (family and hous


 date in 1999 and the second of which is stated to correspond with enhanced enforcement of the law following the Virginia Tech mass shooting in 2007.
Based on the 1999 enactment date of Connecticut’s ERPO law, the analyses showed small and uncertain


res indicate that methodological concerns are discussed in the text. See Appendix B for details.
Conclusions
One study that examined the effects of ERPOs on suicide rates generated mixed find- ings. Specifically, Kivisto and Phalen (2018) found a sug


irsch, Charles M. Belden, Michael A. Norko, Allison G. Robertson, Linda K. Frisman, Hsiu-Ju Lin, Marvin S. Swartz, and George F. Parker, “Criminal Justice and Suicide Outcomes with Indiana’s Risk-Based Gun Seizure Law,” Journal of the American Academ


ew, eight operate by establishing restrictions or requirements on the sales and transfers of firearms in legal markets. Each policy is reviewed in the subsequent chap- ters in Part C, as follows:
• background checks (Chapter Eight)
• licensing and pe


y individuals who already own a fire-
1 The Bureau of Alcohol, Tobacco, and Firearms (2002, p. A-3) defined crime gun as “any firearm that is ille- gally possessed, used in a crime, or suspected to have been used in a crime. An abandoned firearm may


disqualifi- cations for high-risk individuals and whether these disqualifications correctly target individuals who present greater danger to themselves or others. As of December 2018, the Federal Bureau of Investigation (FBI)’s National Instant Crimi


ome cases (e.g., restraining orders), an individual may be only temporarily prohibited from possessing a firearm, and, in the case of crime outcomes, details on the criminal offender can be known only if the perpetrator is known. (See the discussion


itation of the Ludwig and Cook (2000) study is that it had an unfavorable ratio of estimated parameters to observations (less than one to six), meaning it could have mis- leading parameter estimates and confidence intervals (CIs) due to model overfit


esis of Research Evidence on the Effects of U.S. Gun Policies
lower firearm suicide and total suicide rates. Sen and Panjamapirom’s estimates sug- gest the post-policy firearm suicide rate to be 96 percent of the expected rate had this policy not bee


es likely are not reliable.
Finally, Kagawa et al. (2018) used synthetic control methods to estimate how suicide rates were affected by the 1998 repeals of comprehensive background check laws in both Indiana and Tennessee. For each state separately,


in many states when the Brady Act was implemented. The second study, Sen and Panjamapirom (2012), examined components of background checks, finding significant effects indicating that checks on mental illness and checks on fugitive status reduce tot


 on the Effects of U.S. Gun Policies
Effects on Violent Crime
Research Synthesis Findings
Of studies that examined the relationship between background checks and violent crime, we identified 14 that met our inclusion criteria.13 Ludwig and Cook (2000


ividuals with a felony arrest and did not provide evidence for background checks per se.
14 Ludwig and Cook (2000) also tested the effects of background checks specifically (separate from waiting periods, which were also imposed by the Brady Act) by


ffects of the types of background checks conducted by states on homicides. They noted that the supply of state and local records to the NICS is voluntary and that substantial variation exists in state laws regarding the categories of records included


ratio of estimated parameters to observa- tions (less than one to ten), meaning the estimated effects and significance values may be inaccurate because of model overfitting. Examining a shorter time frame (1979 to 1998) and using negative binomial m


mes as involv- ing firearms (Swanson et al., 2016).
Since the first edition of this report, four more studies provide mixed evidence for the effect of background checks on violent crime, although the studies adopt different policy definitions or clas


eal of the background check law. The authors found uncertain effects for both policies on firearm homicide rates, while the estimate from Indiana showed that the repeal was associated with a significant increase in nonfirearm homi- cide rates. Althou


ing evidence on the effects of background checks, or some component of background checks, on violent crime. Six of these studies provided an overall effect of either dealer background checks or private-seller background checks on total homicide rates


ects on violent crime and total homicides.
Evidence for this relationship is
Background Checks 139
    Background checks have
uncertain
inconclusive.
      decrease
firearm homicides.
Private-seller background checks have
uncertain
effects on firearm


aws and the probability of at least one mass shooting event occurring (see Figure 8.3). However, assessing the effects of gun policies on mass shoot- ings was not the primary focus of the study, and the authors intended the estimates to serve solely


ion over the study period), and because the models did not control for other potential differences across states, the authors’ estimates most likely reflect differences between states that have the laws and states that do not rather than differ- ence


characteristics and potential proxy indicators of gun trafficking (e.g., the number of Bureau of Alcohol, Tobacco, Firearms and Explosives [ATF] gun traces linked back to a given FFL) and (2) dealer dropout rates during a period of reforms to the fed


rujillo-Baute, “How Do US State Firearms Laws Affect Firearms Manufacturing Location? An Empirical Investigation, 1986–2010,” Journal of Economic Geography, Vol. 17, No. 4, 2017, pp. 753–790.
Buchanan, Larry, Josh Keller, Richard A. Oppel, Jr., and D


lia, Jon S. Vernick, Daniel Webster, Cassandra Crifasi, Kara E. Rudolph, Magdalena Cerdá, Aaron Shev, and Garen J. Wintemute, “Repeal of Comprehensive Background Check Policies and Firearm Homicide and Suicide,” Epidemiology, Vol. 29, No. 4, 2018, p


” Preventive Medicine, Vol. 55, No. 4, 2012, pp. 346–350.
Swanson, Jeffrey W., Michele M. Easter, Allison G. Robertson, Marvin S. Swartz, Kelly Alanis- Hirsch, Daniel Moseley, Charles Dion, and John Petrila, “Gun Violence, Mental Illness, and Laws Th


a firearm, and the policies could facilitate firearm removal from owners who become ineligible. Requiring permits to purchase ammunition makes it more difficult for prohibited possessors to use their illicit firearms. Where no such checks occur, proh


arm. Azrael et al. (2017) found that, on average, gun owners had close to five firearms each, and a large majority (62 percent) purchased their most recent weapon from a licensed gun dealer. For those who already own guns, licensing and permitting re


crime through dis- rupting illegal firearm trafficking, causal inference could be strengthened by exam- ining crime gun trace data and changes in homicide rates.2 Specifically, if permit-to- purchase laws restrict trafficking operations from in-stat


strict of Columbia have implemented licensing or permitting regimes for all firearms,9 and seven states have done so for handguns.10 Michigan’s law has a broad exemption for individuals who purchase handguns from licensed dealers following a backgrou


§ 2C:58-3.
16 R.I. Gen. Laws § 11-47-35.
17 Hawaii Rev. Stat. § 134-2; N.J. Stat. Ann. § 2C:58-3; N.C. Gen. Stat. § 14-403 (handguns only).
18 Connecticut, Illinois, Massachusetts, New Jersey, and the District of Columbia. See Conn. Gen. Stat.
§§ 29-


percentage change in total suicide and firearm suicide in Connecticut before and after the state established a permit-to-purchase law in 1995, as well as before and after the repeal of Missouri’s permit-to-purchase law in 2007. This approach enabled


. Therefore, the estimates reported in Crifasi et al. (2015) may not be reliable indicators of the direction or magnitude of the true effects of permit-to-purchase laws on suicide.
Finally, a 2017 study of waiting periods and background checks includ


s were associated with significantly reduced rates of total and firearm suicides for adults aged 21 or older.
Considering these studies, we find limited evidence that licensing and permitting requirements decrease total suicides and firearm suicides


 law in this study, it is not possible to conclude that the changes were a result of the permit-to-purchase por- tion of the law as opposed to other factors influencing homicides in the state around the same time.
Crifasi, Pollack, and Webster (2016)


). Both studies that examined total homicide rates—either among adults aged 21 or older or among intimate partners—found uncertain effects of the laws. The same two studies also found uncertain effects for firearm-specific homicides among the populat


yielded two studies that met our inclusion criteria and examined the effects of licensing and permitting requirements on mass shootings in the United States.20 Using a two-way fixed-effects linear probability model, Luca, Malhotra, and Poliquin (201


tes (i.e., population size and aggregated personal income). The authors found uncertain effects of permit-to- purchase regimes on the likelihood of a public shooting event. Although their esti- mates of the effect of permit requirements for firearm o


examined how a policy index that captured the stringency of state laws regarding comprehensive background checks and permit-to-purchase requirements related to variation in the number of firearm manufacturing plants in a state and the number of back


Braga, Anthony A., Garen J. Wintemute, Glenn L. Pierce, Philip J. Cook, and Greg Ridgeway, “Interpreting the Empirical Evidence on Illegal Gun Market Dynamics,” Journal of Urban Health, Vol. 89, No. 5, 2012, pp. 779–793.
Brauer, Jurgen, Daniel Monto


ell, Terry L., Beth Ann Griffin, and Andrew R. Morral, Evaluating Methods to Estimate the Effect of State Laws on Firearm Deaths: A Simulation Study, Santa Monica, Calif.: RAND Corporation, RR-2685-RC, 2018. As of October 1, 2019: https://www.rand.or


self-harm, giving angry or distraught buyers time to “cool off” or gain perspective. While it is plausible that this cooling-off period could reduce impul- sive interpersonal gun violence, some evidence exists for the potential effects of this mecha


crime (Webster and Wintemute, 2015; Braga et al., 2012).2 However, a series of provisions attached to Bureau of Alcohol, Tobacco, Firearms and Explosives (ATF) appropriations (com- monly known as the Tiahrt Amendments) has denied most researchers acc


present elevated risk of violence. As discussed in Chapter Eight, the major- ity of prohibited possessors who perpetrate gun violence acquire their firearms from social acquaintances or the black market; thus, a large portion of violent gun crime is


such policy heterogeneity will depend on the extent to which different types of firearms are substitutes and the marginal effect of requiring an additional day or days of delay before transfer can occur. State waiting-period laws applying to only a


rates before and after the implementation of the Brady Act in 1994, which initially imposed waiting periods and background checks for purchases from licensed firearm dealers. When the Brady Act was implemented, 18 states and the District of Columbia


des and with suggestive negative effects on total suicides, and longer waiting periods (one week or more) showed some evidence of larger effects.8 Although the authors’ preferred model with quadratic state-specific trends had poten- tial issues with


a decline in firearm suicide induced by the waiting-period law might have reduced suicide attempts by other lethal means.
 Figure 10.1
Incidence Rate Ratios Associated with the Effect of Waiting Periods on Suicide
NOTE: IRR values marked with blue sq


 of the true causal effects of the laws.
We identified five studies that specifically examined the effect of waiting periods on violent crime. Mustard (2001) evaluated how waiting-period and shall-issue laws (see Chapter Eighteen) affect felonious de