sometimes taking years to complete the path from initial investigation to conviction, and can involve complexities not always present in criminal matters related to individual claimants or employers. They can also result in stopping large-scale schem


 rules or mislead others for the purpose of financial or personal gain. In recent years, these entities have all taken significant steps toward combating the problem of fraud through the use of advanced analytics, a
rapidly developing field of infor


le, one such rule commonly noted in materials describing warning signs for possible workers’ compensation fraud is that adjusters should more closely examine any claim first filed on a Monday, presumably due to an increased likelihood that the injury


pe of this report to cover all methods by which advanced analytics is conducted. Moreover, few large-scale applications of
advanced analytics in the private and public sectors use a single technique. A single application often uses an array of tools


ntify (i.e., describe) clusters and patterns  14 containing similar  records based on criteria that the application discovers.20 Unexpected clusters
or patterns could suggest that a previously unknown variety of fraud scheme had developed. For
exam


ires identifying at least some instances of fraud. In sociocentric SNA, the network as a whole is examined to identify patterns of

20 Nian  et al., 2016; Kou et al., 2004.
21 Bolton and Hand, 2002.


er social welfare systems because they automatically collect “accurate, real-time, and largely labeled data” in the routine course of business.
26 There are few instances in which human beings are

24


 business purposes, each source employing different formats and definitions.
28 Indeed, one
experienced observer in the area of governmental use of analytics to combat fraud has noted the many challenges created by the “considerable investment upfro


yments
• prepayment edits: computer edits that revise or suspend all or part of a suspect claim
• autodenial edits: computer edits that automatically deny a suspect claim without making
any payments
• provider revocation: elimination of a provide


ge information, and develop best practices.

32 GAO, 2015a.
33 Pub. L.  111-240, 2010, § 4241.
34 GAO, 2012; Censer, 2012.
35 Dickson, 2016.   19 Successes
According to CMS, OIG, and GAO, FPS h


, undated.
38 Congressional Budget Office, 2014.
39 Sullivan, 2012; U.S. House of Representatives, 2015;  Muchmore, 2016.
40 Citizens Against Government Waste, 2015.  20 University of Minnesota has also been a vocal critic of the program’s imple


verall system. States license the health care providers, establish payment policies, contract with

41 Parente et al., 2012.
42 HHS, 2016b.
43 GAO, 2015b, p.  16.
44 GAO, 2015b, pp.  29–31.
45 GA


 Administration is the target of several forms of fraud, including identity
theft in applications for Old-Age and Survivors Insurance and Supplemental Security Income
and, notably, from physician-assisted fraud regarding claims for disability insur


ss or bank account numbers. Despite reportedly
preventing $16 billion in revenue loss in 2012, the EFDS  was felt to have “fundamental
limitations in technology and design.”59 In 2011, the IRS awarded SAS Institute a $6.25 million
contract to prov


yment Security, and Rate Bureau.65 Reportedly, the
state has been able to increase total annual penalty collections about fivefold as a result of using this tool.
66
U.S. Postal Service employees receive the equivalent of workers’ compensation bene


icare context, we could find little indication of significant dissatisfaction with how the tools are being implemented or what benefits they are achieving. In our opinion, there are no external reasons that DIR should not begin to plan for the incorp


proceedings associated with a single workers’ compensation claim and all claims associated with a single WCAB case or proceeding can be readily identified without eyes-on effort. Similar

70 See, e.g


y submitted as an electronic document. DWC can no longer afford to warehouse mountains of paper forms even if it has good intentions to scan them at some future
but uncertain point.
The paper-form issue might be a minor one compared with a more pre


ential criminal investigation materials.   29 Chapter Four. Prevention: Bringing Postemployment  Claims Back
into the System
Background
Some participants at the June 2016 roundtable sessions voiced their suspicion that some
providers were engag


incident o r exposure, for the consequences of which
compensation is claimed” (Labor Code  § 5411). For such injuries, the maximum possible limit for filing a claim  30 realization that he or she has some sort of medical condition must also be accom


st under a year after leaving the employment, but only under the unlikely scenario that the incident or
exposure happened on the very last day on the job.
78 Employment Mut. L. Ins. Co. v. Indus. Acc. Comm’n , 41 Cal. 2d 676 (1953).
79 Zenith Ins


e necessary diagnosis linking the condition to the prior work would be made by a physician of the former employee’s own choosing. The former employee is then required to submit the initial claim form to the employer within certain time limits after l


im with WCAB, the former employee now has an essentially unlimited right to control the identity of those providing postdenial medical care, although the expenses associated with such care would be covered only if there is a favorable result after WC


r the liens themselves is believed to be weak . And finally,
they implied that, in at least some instances, the claimed value of some postemployment liens when presented appear to be inappropriately inflated beyond what might be expected given the n


re service plans, publicly funded benefit programs, and the like) filed each month
with WCAB has more than doubled in less than two years and is now holding at about 30,000
per month.
96 Not all of these liens involve postemployment claims, but mos


der the OMFS reflect policies designed to tightly control medical expenditures in California workers’ compensation claims. A similar result occurs when a private health insurer pays only what it considers to be the reasonable, usual, and customary ch


it of the doubt when
advancing a claim for compensation, absent good-faith knowledge that the claim does not involve a covered injury. That assumption is well  represented by rules about when a claim for
CT or occupational disease can be presented


selects falls directly on the employer’s shoulders because of “neglect or

105 A similar requirement would be in place if the former employer had a contract with a health care organization to
provide


al evidence, but it is hard to ignore the tale that one participant told at the
June 2016 roundtables. The participant described accidentally receiving a new-patient package in the mail from what might be described as a lien mill, a package that con


ontinued medical expenditures providing the proper incentives for a more aggressive defense.
We think that the special circumstances surrounding postemployment claims merit this type
of option. If the complaints of some of the June panelists are ac


ights currently available to him
or her to challenge the employer’s choice of provider or the type of care being received.
Note that we are not suggesting that Labor Code § 5402 should be amended regarding claims
made by current employees. There d


ll claims would be potential candidates for deny-but-control treatment. Presumably, employers would pick and choose among those candidates to target only the most-problematic providers, so the proportion of claims ultimately affected would be smaller


 WCAB. A recently indicted medical management corporation’s chief executive, accused of participating in kickback schemes, nevertheless advanced liens worth $58 million in payments for the same medicated “pain
creams” that were alleged to be the foc


on with other agencies and/or law enforcement.”
• Allegations and complaints are considered credible if  they have “indicia of reliability”
and the state has reviewed all facts and evidence carefully and acts judiciously.
• When a determination of


termine whether to condition the sentence on, among other things, forfeiting the right to seek payment). And rights for due process would still be in place, given that Chapter Five in California’s version of the Administrative Procedures Act  provide


s
• has exceeded the limits of his or her professional competence in rendering medical care
or in conducting independent medical examinations under the law
• has made materially false statements regarding his or her qualifications in his or her
a


fraud will trigger a Medicaid suspension, while a wide range of behaviors (sexual battery, for example) will constitute misconduct in New York.  46 Table  5.1. Medicaid and New York State Workers’  Compensation Board  Approaches
Characteristic  Me


in managing health care provider behavior. The New York model would certainly give DIR considerable discretion to address a variety of physician issues that go beyond mere fraud, including failures to comply with any law or regulation related to the


ion until alternative care could be arranged.
119 Our assumption is that DIR would use
its power to suspend payment sparingly and never to the point at which wide-scale  disruptions of
the health care delivery system would result. It should be note


ly issued by a provider who subsequently became the target of a fraud investigation. Making it clear that the taint of the original issuer follows the accounts receivable wherever they go will close that potential loophole from almost certain repeate


ional statutory language would be needed for suspending payments to not only the original provider but also to any holders of factored liens originating with that provider.
Comparison of Medicaid Approach with Recent Legislative Initiatives
Senate


it
pertains to patient care.
(iii) It is a financial crime that relates to the Medi -Cal program,
Medicare program, or workers ’ compensation system.
(iv) It is otherwise substantially related to the qualifications, function s,
or duties of a


 the disposition of the criminal proceeding fails to specify the disposition to
be made of lien filings in the workers ’ compensation system as set forth in
paragraph  (1), all liens pending in any workers ’ compensation case in any district
offic


ave ten days to submit a written request for a hearing to argue that
the criteria for suspension are not applicable.
• Requests for such a hearing would temporarily stay the suspension.
• The hearing must be held within 30 days of the submission o


a eligible for such sanctions, but one indicator might be the number of people and entities that have been excluded (essentially a type of suspension) by the HHS OIG from participating in Medicare, Medicaid, and other federal health care programs. OI


viction and are therefore not valid
(3) Liens generated for servi ces
after suspension are always
invalid .
Duration of
sanctions  Until investigation determines
insufficient evidence of fraud or upon completion  of legal
proceedings  Until d


surrendered or revoked.” Although such adverse credentialing actions would not require a criminal prosecution to trigger a workers’ compensation suspension, only about 100 licenses for all California providers were surrendered or revoked in 2016, and


of a medical license revocation outside California, it would appear that none of the outstanding liens filed with DWC would be voided. If California workers’ compensation fraud or abuse was involved in
the conviction that led to the suspension, the


in California. There are obvious and  57 well-founded reasons for that high standard, given the serious implications of imposing criminal
fines and imprisonment, but, in California, workers’ compensation is part of the civil justice
system and is d


of what they do is essentially undetectable by administrative or law enforcement agencies.
Part of the reason for fraudsters’ ability to hide in what is essentially plain sight is a problem
we noted in Chapter Three: A Balkanization of information


uired if the power and promise of advanced techniques to detect fraud and identify those who commit it are to be realized. Moreover, those data have to be easily and readily available to the analysts on an ongoing basis, without the need to repeatedl


, one in which employers have the option of denying a questionable claim
while continuing to offer medical care under their control, can be applied in just those instances likely to involve certain employee-selected providers who repeatedly generate


in place to adequately notify current patients of a provider who is anticipated to be the subject of a payment suspension. And third, the decision to suspend should be closely coordinated with the efforts of district attorney staff overseeing current


w Way to Find Cheaters,” Forbes, January 28,
2016. As of July 29, 2016: http://www.forbes.com/sites/metabrown/2016/01/28/ analytics-and-the-irs-a-new-way -to-find-cheaters/#5218dbf75b8b
California Department of Industrial Relations, Division of Wor


’ Compensation Covering Uninsured Medical
Costs? Evidence from the ‘Monday Effect,’” Industrial and Labor Relations Review, Vol. 49, No. 4, 1996, pp. 690–706.
CDI—See California Department of Insurance.  65 Censer, Marjorie, “Two Local Contractors


y, the Opportunity, and a Model for Change, Franklin, Tenn.: Coventry Workers’
Comp Services, May 2012.
GAO—See U.S. Government Accountability Office.
Gartner, “Advanced Analytics,” IT Glossary, undated. As of November 16, 2016:
http://www.gartne


“Investment in
Data Analytics by the Massachusetts Office of the State Auditor,” slide deck, January 15, 2014. As of July 31, 2016: http://montgomeryaga.org/images/meeting/011514/slides_for_fraud_january_webinar.pdf
McBirnie, Thomas J., Report on t


elease, March 16, 2015. As of July 28, 2016: http://www.ic.nc.gov/NCICFraud-031615NewsRelease.pdf
Parente, Stephen, Brian Schulte, Allen Jost, Thomas Sullivan, and Allan Klindworth,
“Assessment of Predictive Modeling for Identifying Fraud Within th


dit/full/pdf/A-08-13-13038.pdf
Stevens, Howard, “The Downside of Denial: A California Perspective,” LexisNexis Legal
Newsroom: Workers Compensation Law, July 14, 2009. As of September 15, 2016: https://www.lexisnexis.com/legalnewsroom/workers-compe


ive Analytics System, but Needs to Define Measures to Determine Its Effectiveness ,
Washington, D.C., GAO-13-104, October 15, 2012. As of April 21, 2017:
http://www.gao.gov/products/GAO-13-104
———, Standards for Internal Control in the Federal Gov


ber 2010. As of April 21, 2017: http://www.sentencingproject.org/wp-content/uploads/2016/01/ Deterrence-in-Criminal-Justice.pdf
Yasin, Rutrell, “How MassHealth Cut Medicaid Fraud with Predictive Analytics,” GCN,
February 24, 2014. As of July 26, 20


Using Video Analytics and Sensor
Fusion in Law Enforcement
Building a Research Agenda That Includes Business
Cases, Privacy and Civil Rights Protections, and Needs for Innovation
SUMMARY
On July 12–13, 2017, on behalf of the National Institute of J


ement by monitoring officer per -
formance and protecting officers’ health and safety.
•The r
isks of VA/SF technologies are significant, with security,
privacy, and civil rights protections needed if these technologiesare not to be misused or abused


eras that are monitored by an opera-tions center.
•
 The B
altimore, Maryland, Police Department’s CitiWatch,
which has an operations center with several analysts moni-toring over 700 cameras (as of early 2016). The program also includes a partners


fficers to reduce information overload• R&
D on architectures and APIs to integrate
VA data with RMS/CAD into unified records; should include best practices and model con-tracts in addition to technology
•
 R&
D on having dispatch and routing
 int


e. Donald, Donald, and Thatcher (2016) provide a review of research on capabilities to monitor cameras. In contrast, there have been estimates that a large city agency may have access to hundreds of thousands of video feeds from all over a particular


to determine whether a cartoon robot is present based on presence of such features as cartoon claws and robot-like heads.
Importantly, the aforementioned GPUs are capable of
processing large arrays of pixels in images simultaneously. Thus, a comput


tions Subsampling Convolutions Fully connected SubsamplingInputFeature maps
f.mapsf.mapsReduce the images or videos Classify the
resulting
features
Output
Figure 2. Example Multilayer Neural Network
SOURCE: Salatas, John, “Multilayer Neural Network,”


ly classify it—person, vehicle, etc.—and track its move-ment on camera. Test data sets for VA of human activity, such as the Activity, Event, and Action Databases library, main-tained by Rensselaer Polytechnic Institute’s Intelligent Systems Lab (und


ore such methods are broadly implemented. Second, once these core concepts have been identified, specific R&D needs should be determined.
Methodology
On July 12–13, 2017, on behalf of NIJ and the Priority Crimi-nal Justice Needs Initiative, the RAND


 cases were developed during a facilitated discussion, with participants weighing in on inputs, activities, and desired outputs of analytic and fusion efforts with the cases built in real time. Needs —defined as problems or opportunities coupled
wit


was designed to identify and rank needs based on the assessments of practitioners with experience in the field. Business Cases for Video Analytics and
Sensor Fusion in Law Enforcement
To provide an initial answer to the question of what the public


 officers on scene, court officers, and possibly the public. They must include explicit recognition and display of the uncertainties involved in an artificial intelligence judg-ment.
2 Such explanations would use narratives to explain an
alert in or


mphasized that alerts must be tailored to
support later review and decisionmaking. This could be accom -
plished at a basic level by employing pan-tilt-zoom (PTZ) in
response to sensor hits, such as gunshot detection, anomalies in health telemetry,


uto-indexing
for a caseIncident
searchPrepare and distribute
to court officers• Face detection and
enhancement
• Plate enhancement• Other• Face recognition• License plate
recognition
• Other
• Find “interesting” video and
sensor data for case
• La


and case number (if applicable), searching the structured metadata would also be much faster. However, this implies the need for a standardized dictionary describing how key data about the video, and what was found on it, are to be coded.
Business C


also be neces-sary, starting with the type of incident, history of what was observed, and ability to assign speech to people and accurately transcribe it. This should ultimately take the form of narrative descriptions of the observed events. Finally,


m video and sensor data could be used to evaluate, develop, and test polic-ing strategies. There is a need for models to recognize delayed reinforcement and to test whether outcomes improve over time. Such models could also provide an experimental ca


nts of interest for
diagnoses and evaluations
- Disorder- Accidents- Crimes- OtherCount and track events of interest
• Trend charts and control
charts that link to supporting labeled video and sensor data
13include presenting the decisionmaker with


any new VA/SF toolset deployment. Figure 9. A Conceptual Video Analytics and Sensor Fusion Processing Network
1. Real-time monitoring 2. Post-event investigation and reporting
Monitoring
for alerts
Review alerts
and get additional
information
Act on


ata must not be provocative, humiliating, or exploitative. These tools should also not be used to instruct an officer to, for example, draw a gun, escalate, de-escalate, or in any other way make critical decisions for an officer in the field.
Data I


 to interfaces or exchanges between police assets and city or jurisdiction assets, as well as with training, awareness, and resources of any partnering agen-cies.15Community Partnership and Acceptance
Finally, the panel discussed necessary procedures


tify and reduce false positives
over time.
• The
re is a general need to identify, categorize, and filter out
events (e.g., lengthy stretches of video and sensor feeds) that are not of interest to law enforcement.
•
 Tr
aining of VA algorithms nee


e Tier 1 needs are in a cluster with the highest EV and include 11 needs—23 percent of the total needs identi-fied by the group.
The second type of priority need had a median importance
score of 9, the highest possible rating. These are high-value (


nd suspicious potential indicators of ter-rorism (Nationwide SAR Initiative, 2016).17The needs are presented in Table 1. Within categories,
priority needs are presented in order of their expected value
scores, with higher-scoring needs presented fi


or relatively quickly disposed of for privacy and civil rights purposes.Facilitate the creation of a (continuously refreshed) service for cataloging and sharing data sources that can be used for training algorithms (e.g., a public library, Github.co


rastructure
Issue or Current Limitation Need Tier–Value
Existing data sources (e.g., LE databases) should
be integrated with VA systems to improve the recognition and prediction processes.Develop best practices and model contracts that consider arch


already carrying.2
Forensics: Video archives and real-time feeds are difficult to search for operationally relevant data.Video archive searches should be coordinated with related data (e.g., CAD, weather,)2
Monitoring—agencies: VA systems could be


oca-
tion of objects in videos.
Computational infrastructure•
 Asse
ss different processing models for VA/SF
(e.g., contracted cloud, government cloud).
• As
semble technical advisory groups to help
agencies considering sensor networks with VA/SF.


porating the complex treatment of nuance, along with automatic detection and adjustment for changes in behav -
iors over time. VA/SF networks should also incorporate semantic search and automatic infrastructure control, with the control-rule algorith


st-priority use case for these technolo-gies, potentially offering major benefits to society. The panel also felt that VA/SF technologies could be of great benefit in investigating crimes and major incidents, could provide major
 ti
me-savers throug


equaling the same benefit of prior “game changing” technologies such as body armor and crime hotspot analysis. Likelihood of success was bracketed as 1 equaling about a 10 percent chance of success and 9 equaling about a 90-percent chance of success


. These high-value needs are all marked with “HV” on the dendrogram. This approach also allows identifi-cation of low hanging fruit (LHF): needs that were rated very high in terms of probability of success, even if they are lower in payoff. Because t


ions).
1 Issue: CAD, RMS, mobile data, and VA systems are generally not interoperable with each other. Need:  Integrate dispatch and routing with what is
already known from VA systems.251 Issue:  There are limited skill sets within agencies
to oper


s to identify relevant evidence for the same event. Need:  Conduct research into video search
methods that would allow similar or related videos are also returned (or immediately available).
2 HV Issue:  A large volume of collected and stored
vide


r these systems (assess the business case).
2 Issue:  When algorithmic learning occurs,
organizations need a way to ensure that additional learning is coevolutionary with the learning occurring within organizations and among practitioners.
Need:


systems to reduce bias and improve auditability (e.g., use descriptive tags and include narratives as to why a tag was assigned).
2 Issue:  There is a lack of understanding of how
artificial intelligence can and should be applied to law enforcement


ble A.1—Continued31ENDNOTES
1 Here, we use the term business case rather than the term use case , as
the latter typically means a detailed list of interactions between a user
and a system to achieve some objective. Use cases are typically cap -
tur


eve, Paul Bramble, and Alistair Cockburn, Patterns for Effective Use Cases , Boston, Mass.: Addison-Wesley Longman
Publishing, 2003.
Aphex34, “Typical CNN Architecture,” December 16, 2015. As of
November 15, 2018: https://commons.wikimedia.org/wik


John S., Dulani Woods, Andrew Lauland, Sean E.
Goodison, Thomas J. Wilson, and Brian A. Jackson, Using Future Broadband Technologies to Strengthen Law Enforcement , Santa Monica,
Calif.: RAND Corporation RR-1462-NIJ, 2016. As of July 10, 2017: http


www.nytimes.com/2018/02/09/technology/facial-recognition-race-artificial-intelligence.html
Mitchell, Anna, and Larry Diamond, “China’s Surveillance State
Should Scare Everyone,” Atlantic , Feb. 2, 2018. As of October 1,
2018:  https://www.theatlant


 Visualization,” Deep Learning Workshop , Lille, France: 31st
International Conference on Machine Learning, 2015.
Acknowledgments
The authors would like to acknowledge the participation and
assistance of the members of the expert panel listed in t


 This Report
On behalf of the U.S. Department of Justice, the National Institute of Justice (NIJ), the RAND Corporation, in partnership with
the Police Executive Research Forum, RTI International, and the University of Denver, is carrying out a rese


mitted to the public interest.
RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.
R® is a
registered trademark.
RR-2619-NIJCORPORATION© Copyright 2018 RAND Corporation
www.rand.orgThis publication was


CORPORATION
Improving Outcomes
for Children Exposed to Violence
Safe Start Promising Approaches
Dana Schultz, Lisa H. Jaycox, Lynsay Ayer, Claude Messan Setodji,
Ammarah Mahmud, Aaron Kofner, Dionne Barnes-ProbyLimited Print and Electronic Dis


d build on our prior work evaluating the first round of 15 SSPA sites, reported in
• Lisa H. Jaycox, Laura J. Hickman, Dana Schultz, Dionne Barnes-Proby, Claude Messan
Setodji, Aaron Kofner, Racine Harris, Joie Acosta, and Taria Francois, National


ector
at RAND_Health@rand.org.
Questions or comments about this report should be sent to the project leader, Dana Schultz
(Dana_Schultz@rand.org).  v Contents
Preface ...............................................................................


es’ Planned Research Designs ................................................................................. 23
Table 2.5. Site Research Designs, Expected Effect Sizes, and Sample Sizes for Power ............. 26
Table 2.6. Analysis and Inferen


n’s exposure to violence have been declining, in 2011, 58 percent of
children in a nationally representative sample had been exposed to violence in the past year, with
48 percent exposed to multiple types of violence (Finkelhor, Shattuck, et al., 2


provements in specific outcome domains at a particular  xii site. The evaluation’s intent-to-treat analysis approach involved analyses of all those who were
offered participation in Safe Start, regardless of how much of the program they actually rec


Target
Enrollment
for 80%
Power Actual
Enrollment
Aurora, Colo. RCT: Strategi c enhancement to an intensive
dyadic therapy (Traum a-Focused Cognitive–
Behavioral Therapy + Let’s Connect)
compared with Trauma-Focused Cognitive–Behavioral Therap


ve medium intervention effects enrolled more families than the target needed for 80-percent power, while all of the studies that expected small intervention effects enrolled far fewer families than needed for 80-percent power to be able to detect the


parison group also improved, we found no evidence of significant differences observed in most cases between the intervention and comparison groups using difference-in-differences models. One exception was in El Paso’s primary outcome, child self-cont


ion group did not improve more than the comparison group. The interventions in Philadelphia and  xviSpokane operated within existing programs for families such that both the intervention and
comparison groups received a robust array of usual service


vailable individually online, include detailed results.  xviiConclusion and Next Steps
The OJJDP SSPA initiative sought to improve the evidence base for interventions for CEV as
delivered in community settings. This important work aimed to fill a g


ted prevention approaches. Targeted services would be used for the minority of children who have prolonged adjustment problems related to violence exposure. There is a growing evidence base about what works in terms of the more-intensive services for


been evaluated to see whether they do, in fact, produce a more resilient child, family, or community. Thus, this area is ripe for additional exploration, particularly when combined with the full array of services within a public health model. Clearly


on Scale RADS-2 Reynolds Adolescent Depression Scale, 2nd ed. RCT randomized controlled trial RYDS Rochester Youth Development Study SD standard deviation SE standard error S.E.L.F. Safety, Emotions, Loss, and Future SFCR Strengthening Family Coping


hort-term risk for depression and anxiety (Lansford et al., 2002) and behavior problems (Grogan-Kaylor et al., 2008), as well as high levels of depression, anxiety, and distress in adulthood (Lindert et al., 2014). Children exposed to domestic violen


argeted prevention programs would be reserved for the minority of children who experienced violence and are also displaying some lasting adverse effects. 3Figure 1.1. Tiered Public Health Model
To date, most of the treatment development and research


ain them (Wiltsey Stirman et al., 2012). Key factors related to implementation and sustainability include organizational elements, staffing, training and ongoing staff support, data systems, and ongoing fidelity monitoring and quality assurance (Fixs


lanning for quality assurance, and ensuring human subject protections.
Overview of the Current Safe Start Sites
Like the initial set of 15 sites, the ten SSPA sites that are the focus of this report differ by
community size, location, age range se


y phone calls
Honolulu, Hawaii Enhancement to an existing group therapy (Haupoa) + individualized clinical child assessment and individual
and family therapy (modular CBT) 3–17 Quasi-experimental  design comparing
Haupoa + modular CBT with usual


tcomes (posttraumatic stress, depression, behavior problems, social–emotional competence, school behavior and attitudes, family functioning, and caregiver mental health) data.
Funding for implementation and evaluation was modest. Sites received up t


site to document its specific capabilities and plans in each area through an iterative process with the national evaluation team. The Green Light process also involved establishing targets for study enrollment and service delivery based on power ana


s, targeted consultation with national experts), which, in some cases, resulted in changes to sites’ original implementation plans. Some  11 project leaders needed more support regarding the data collection and evaluation issues. Sites
also needed t


minister the measure.
• To maximize credibility with sites and the ultimate audience for the evaluation results, the measure had been widely accepted and widely used in the field.
• The measure or subscale of a measure adhered closely to the outcom


y  11–18
Child Substance use and gang
involvement 11–18
Social–emotional competence Caregiver Child social–emotional
competence 0–2
Caregiver Child cooperation,
assertion, and self-control 3–12
Child Child cooperation,
assertion, and self-con


a different study. We again received permission from the publishers to use those existing translations. We also used our own Spanish translations from the first phase for the caregiver and child information items, the caregiver report (CR) of the Ju


through I).  16 Table 2.3. Prioritized Outcomes, by Site
Site Caregiver
Resource and
Personal
Problems Child PTSD
Symptoms Child
Depression Child
Behavior and
Conduct
Problems Child Social–
Emotional
Competence Child School
Behavior and
A


 collection experience and prior training. In
some sites, evaluation supervisors were trained, experienced researchers who were able to draw
on their experience in conducting other longitudinal data collection efforts with children and families. At


loped its own eligibility criteria and established referral procedures with each of its referral sources. This means that sites differed a good deal from one another in terms of the initial level of need for services for children and caregiver. Once


 group assignments until after the baseline
assessment with the caregiver was complete. The SSTAR system used the stratified urn randomization procedure for group assignment that sought to equalize the groups on child age (Wei and Lachin, 1988). Bec


 completed caregiver assessment. One site offered a bonus incentive of $10 for completion of each intervention session.
For the baseline assessments, every instrument in the caregiver assessment needed to be
completed for the family to be fully enr


leted, and the status of completion of the surveys. On a monthly basis, RAND research staff reviewed assessment and service survey data and worked with sites to address any missing data or clarify any inconsistent responses. When we detected a patter


different aspects
of implementation, including receiving referrals, enrolling families in the study, child and
family status at time of enrollment, providing services, engaging families in the
intervention, quality assurance, and costs.
• structu


cells of the output tables in the site reports. This section gives a general overview of these strategies, with site-specific details appearing in each of the respective site reports (see Appendixes B through I).
Overview of Site Research Designs
T


ists.
Thus, higher levels of statistical power allow us to draw firmer conclusions about the
effectiveness of the site’s interventions. With inadequate statistical power, the results of statistical tests become hard to interpret. For example, lack


udy so as to be able to detect a clinically meaningful effect. In the Safe Start context, however, the size of the effect anticipated for some of the different site interventions was difficult to gauge because of the lack of existing research on thes


a site could have a certain amount of power to detect differences for measures used for its full sample but lower power for measures that were completed by a subsample in a particular age range. Second, we used multiple statistical tests in this stud


r the sites with large enough analytic samples, we compiled two summary dosage scores (one for dosage at six months and one for cumulative dosage up to 12 months) and, based on the distribution, created a variable to indicate whether families who rec


rvention services  for five of the sites. First, we conducted descriptive statistics to examine
changes over time for those in the intervention group who received different levels of dosage, mostly categorized into low, medium, and high dosages, of


 different between groups and
whether changes can be attributed to the intervention (assuming that the
groups are comparable)
At least 20
per group Multiple linear regressions,
adjusting for demographics Determine whether there is an in terventi


each set of analyses, depending on the number of tests conducted. In each site report, we give information about the p-values required to determine
significance. However, we also present information about nonsignificant trends observed between p < 0


Boys Mean
Child
Age,
in
Years Child Race and Ethnicity
Percentage
of Families
with Annual
Household
Income of
<$10,000 Average
Violence
Exposures
of the
Target
Child Percentage
of Children
with CRs in
the
Significant
Range for


ing referrals into the comparison group. Because of implementation challenges, OJJDP decided to end Queens’ formal participation in the evaluation, although the site continued to collect baseline and follow-up data on its own (as shown in the orange


nroll only 412 families in the study.
Figure 3.3. Required Versus Actual Enro llment for Sites with Comparison Groups
* Not included in the remainder of the analysis in this chapter because of implementation challenges.
In the previous two figures


ate the enrollment process into the normal flow of activity at the shelter, which made participation easier for families. Further, because families were living at the shelter, there were multiple opportunities for outreach about the study.
The other


rt. This might have helped these programs to maintain contact with families throughout the study. Other sites tried these and similar strategies to improve retention but did not have the same degree of success. Worcester had difficulty tracking famil


er for each study at 0 100 200 300 400 500 600
Worcester Denver Detroit El Paso Aurora (child outcomes) Philadelphia Spokane ARC Spokane COS Total N needed for power
Comparison N retained
Intervention N retained
Expected medium intervention effe


egarding the effect size the intervention would likely have on outcomes (Aurora for intervention retention, Detroit, El Paso, and Worcester). These studies had the sample sizes needed for 80-percent power to detect the minimum observable effects orig


 families.
• family readiness for and engagement in services
− Many sites worked to understand family concerns so as to acknowledge that families
face a myriad of issues that require a holistic approach to care, and to communicate
flexibility and


For example, the two programs that achieved the
highest enrollment rates delivered services in the community (Detroit) and the clinic
(El Paso), whereas other programs in these same settings (e.g., Queens in a
community setting and Aurora in a cli


study was a strategic enhancement
designed to test the effectiveness of Let’s Connect on retention in the TF-CBT intervention. An
examination of the intervention and comparison groups at baseline found no differences on any of the outcome measures,


ound each estimate.
Among Detroit’s 44 secondary outcomes, there were many statistically significant changes
for one or both groups after adjusting for multiple comparisons:
•intervention group only
− At six months, the intervention group, but not


e on Detroit’s primary outcome of family conflict. We divided the intervention families into three dosage groups: low (zero to three SFP sessions), medium (four to nine SFP sessions), and high (ten or more SFP sessions). Because we expected that chil


e assessment. For both groups, we observed a 50 statistically significant improvement in scores within groups on the primary outcome variable of
CR of child self-control, with both groups showing the expected increase in child self-control from base


on group only
− At 12 months, the comparison group, but not the intervention group, improved on SR
of family conflict and SR of family involvement.
• both intervention and comparison groups
− At six months, both groups improved on CR of total str


intervention and comparison groups in any of the dosage categories (Table 4.4).
Table 4.4. Changes in Mean Number of Sessions,  by Dosage Group, for Caregiver Report of Child
Self-Control in El Paso Safe Start: Six Months
Dosage Group N Baseline S


es among the 24 secondary outcomes. There were within-group differences for
several of the 27 tertiary outcomes. At six months, both groups had significantly fewer caregiver total stressors, caregiver resource problems, CRs of the child’s total vict


 Group N Baseline Six Months Mean
Difference Mean SD Mean SD
CR of child social–emotional
competence (ages 3–12)  None Intervention 8 31.88 18.89 33.75 27.22 —
 Comparison 8 30.63 15.68 29.38 18.02
Any Intervention 12 38.75 22.58 37.08 29.35 7.


se robust comparison conditions so that all families would receive some help, making the comparison condition ethical and acceptable to families in the study. However, in this case, families in the comparison groups fared nearly as well as the famili


e, and this might have made it difficult to demonstrate changes in children over time because there might not have been room to improve for children who were not experiencing many symptoms or problems at baseline. Here, services could potentially pro


ng evidence base for this intervention. For the primary outcome in Philadelphia and for both Spokane interventions, there was significant improvement for the comparison group only. In Philadelphia, the comparison group had a significant decrease in c


ntervention effects has little meaning because we do not have enough power to detect the expected effects.
Service Utilization and Dosage
We also looked at the variation in service across these interventions (Table 4.8). The vast
majority of Auror


sage group. For Philadelphia, the statistical test comparing the two groups on changes in mean scores between baseline and six months and baseline and 12 months revealed no statistically significant difference between the intervention and comparison


ntrol), the comparison group improvement was significant. The robustness of the usual services that both intervention and comparison groups received likely played a role in the results for Philadelphia and Spokane. The interventions at these sites op


PTSD. The other two studies produced
small effect size changes ranging from –0.15 to –0.20.
• CR of positive involvement: We were able to estimate effect size changes for CR of
positive involvement in four of the eight studies. Philadelphia and Sp


Child PTSD
Symptoms CR of Positive
Involvement Caregiver
Depression CR of Child
Self-Control CR of Family
Conflict CR of Child Total
Behavior Problems
Aurora –0.91*
(–1.33, –0.48)a –0.72*
(–1.10, –0.34)a 0.32b
(0.00, 0.64) –0.31b
(–0.61, 0


ild age, child gender, child race and ethnicity, and child’s exposure to violence). Among the sites, only Aurora had a small effect size for its primary outcome of CR of child PTSD symptoms when comparing the intervention group change with the compar


vidence of their effectiveness or have not been tested in real-world settings. SSPA was designed to grow the evidence base for interventions for CEV by rigorously evaluating the effectiveness of community-based programs. We conducted 11 evaluations a


h groups. For Philadelphia and Spokane, Safe Start was embedded within existing and robust programs for families, which likely made it difficult to see differential change between the groups that could be attributed to the Safe Start intervention.
E


hing interventions
relevant to CEV in their settings. The sites started from different places, with some sites more
experienced providing behavioral health services and others starting from scratch to integrate a new type of services into an existi


imitation to the national evaluation is that the assessment battery used in the
evaluation might not have been ideal for all sites. Despite each site implementing a different intervention, we used a common set of measures. It is possible that some o


one might be more efficient or less costly. However, we did not include a cost analysis in the national evaluation.
  71 Future Directions
The SSPA initiative set out to build the evidence base for programs for CEV. Although the
initiative added to


 health pyramid are services for families who have been
identified as exposed to violence but who are experiencing only mild or moderate symptoms. Future exploration of services at this level might try to pin down the necessary ingredients for less


s well as efforts within child welfare,
juvenile justice, and mental health systems ( Donisch, Bray, and Gewirtz, 2016). However, most
of these models have not been evaluated to see whether they do, in fact, produce a more resilient child, family,


ne outcome domains:
• background and contextual factors
• PTSD
• depression
• behavior and conduct problems
• family functioning
• social–emotional competence
• school behavior and attitudes
• violence exposure
• caregiver mental health.
We


— — 10 10
APQ 42 42 42 42
FES Conflict scale — — 9 9
Violence exposure JVQ — 19 19 19
Total number of items 59 78 133 184
NOTE: — = Measure did not apply to this age range.

Background and Contextual Factors
Caregivers completed five measures


used the weighted-least-squares-with-adjusted-means estimator (Mplus, undated). The analysis identified a two-factor solution that was easily interpretable and resulted in high internal consistency within each factor. Namely, from the 21 items, we c


s subscales for PTSD, depression,
and anxiety. With permission from the developers, we used only the PTSD subscale. The TSCYC PTSD scale includes 27 items that tap things the child does, feels, or experiences (e.g., bad dreams or nightmares, being b


so been demonstrated, with correlates in the expected direction between the RADS-2 and other depression measures (Reynolds, 1998; Reynolds, 2002). In the present study, the Cronbach’s alpha was 0.28.
Behavior and Conduct Problems
To assess internal


used IRT to create a single score for the entire age range that combined data from the two scales. The pattern of eigenvalues, predominantly strong loadings, and reasonable fit for the single-factor model provided evidence that the item set is suffi


were then asked follow-up questions about their extent of current involvement (e.g., are you currently a member of a gang? and on a scale of 1 to 5, how far from the center of the gang are you if 1 is leadership of the gang and 5 are people who just


of the SSIS (Gresham and Elliott,
2008) to assess assertion, self-control, and cooperation. Questions asked how true (e.g., not true,
little true, a lot true, or very true) and how important (e.g., not important, important, or critical) the child p


aregiver
and youth self-report scale on school functioning. We also asked teenagers one question about their grades.
Caregiver and Child Self-Reports of Child School Functioning
We used one scale from the BERS-2 (Epstein and Sharma, 1998) to asses


4 for inconsistent discipline, and 0.55 for corporal punishment. For child respondents, the Cronbach’s alpha was 0.82 for parental involvement, 0.82 for positive parenting, 0.79 for poor monitoring and supervision, 0.61 for inconsistent discipline, a


specific values were reported. We computed a total score for each subscale, as well as a total score for the entire measure, with higher scores indicating more violence exposure on each dimension. At baseline, the survey asked about these experiences


experiences. At baseline, the survey asked about these experiences in the past year, and, on the follow-up surveys, the survey queried about the prior six months and since the prior assessment.
Caregiver Mental Health
We used two measures to captur


 “Controlling the False Discovery Rate: A Practical and
Powerful Approach to Multiple Testing,” Journal of the Royal Statistical Society , Series B:
Methodological , Vol. 57, No. 1, 1995, pp. 289–300.
Blaustein, Margaret E., and Kristine M. Kinnib


y P. Mannarino, and Robert A. Steer, “A Multisite,
Randomized Controlled Trial for Children with Sexual Abuse–Related PTSD Symptoms,” Journal of the American Academy of Child and Adolescent Psychiatry , Vol. 43, No. 4, April
2004, pp. 393–402.
Coh


” Aggression and Violent Behavior , Vol. 13,
No. 2, March 2008, pp. 131–140.
Finkelhor, David, Richard Ormrod, Heather Turner, and Sherry L. Hamby, “The Victimization
of Children and Youth: A Comprehensive, National Survey,” Child Maltreatment , V


6–128.
Gewirtz, Abigail H., and Jeffrey L. Edleson, “Young Children’s Exposure to Intimate Partner
Violence: Towards a Developmental Risk and Resilience Framework for Research and Intervention,” Journal of Family Violence , Vol. 22, No. 3, April 20


h, Dawna Gabowitz, Margaret E. Blaustein, and Joseph
Spinazzola, “Development and Implementation of Trauma-Informed Programming in Youth
Residential Treatment Centers Using the ARC Framework,” Journal of Family Violence ,
Vol. 28, No. 7, October 2


herapy for Physically Abused
Children and Their Offending Parents: A Comparison of Clinical Outcomes,” Child
Maltreatment , Vol. 1, No. 4, November 1996b, pp. 322–342.
Kroenke, Kurt, Tara W. Strine, Robert L. Spitzer, Janet B. W. Williams, Joyce T


, “Cognitive–Behavioural
Interventions for Children Who Have Been Sexually Abused,” Cochrane Database of
Systematic Reviews , Issue 4, October 18, 2006, Art. CD001930.
Marvin, Robert, Glen Cooper, Kent Hoffman, and Bert Powell, “The Circle of Secu


mmunity Violence,” Child Development ,
Vol. 73, No. 4, July–August 2002, pp. 1265–1282.
Patterson, Gerald R., Marion S. Forgatch, and David S. Degarmo, “Cascading Effects Following
Intervention,” Development and Psychopathology , Vol. 22, No. 4, N


m Response Theory, New York: Springer, 1997,
pp. 85–100.
Sastry, Narayan, and Anne R. Pebley, Non-Response in the Los Angeles Family and
Neighborhood Survey , Santa Monica, Calif.: RAND Corporation, DRU-2400/7-LAFANS,
2003. As of November 14, 201


 for Social–Emotional Behaviors , Baltimore, Md.: P. H. Brookes Pub. Co., 2002.

Stein, Bradley D., Lisa H. Jaycox, Sheryl Kataoka, Hilary J. Rhodes, and Katherine D. Vestal,
“Prevalence of Child and Adolescent Exposure to Community Violence,” Clin


Landscape Survey
to Support Flood  Apex National Flood Decision Support Toolbox
Definitions and Existing Tools
Aaron Strong, Debra Knopman
CORPORATIONLimited Print and Electronic Distribution Rights
This document and trademark(s) contained herein a


first summarize definitions of resilience found in the peer-reviewed literature and government and other organizations’ reports. Second, we sum -
marize the literature on conceptual frameworks to guide understanding and actions to build resilience an


cused on resilience. In particular, the topic areas included (1) various definitions of resilience, (2) system-of-systems frameworks for con -
ceptualizing resilience, (3) indicator and metric systems for resilience, and (4) examples of how resilienc


screen -
ing the many self-identified DSTs, we applied several criteria to reduce the number to a man -
ageable set and focus our efforts on those that had the potential of mattering most to the Flood Apex program:
•
 field a
pplication: Was the DST


 and its own economic composition and needs.
As a result, decision analysis and support frameworks and indicators need to be tailored to each individual community. Resilience thus needs to be framed in terms of the particular circumstances of each i


o on investments in natural and social capital that enable communi -
ties to mitigate impacts of flood events and recover more quickly.
At present, jurisdictions vary considerably in terms of adopting flood DSTs for use in
preparing action plans and


sting DSTs.
Finally, how DSTs are disseminated to communities needs particular attention. Com -
munities need guidance through the decisionmaking process. The agricultural extension net -
work and the National Oceanic and Atmospheric Administration S


HSDC RAND Homeland Security and Defense Centerxviii    Landscape Survey to Support Flood Apex National Flood Decision Support Toolbox
IFRC International Federation of Red Cross and Red Crescent Societies
LUCI Land Utilisation and Capability Indicato


 but often finding it difficult to link output from these models to broader community goals and other tools in common use for land use, trans -
portation, water resources, and other infrastructure planning. Further, different institutional and govern


disaster management. Given the recent emphasis on resilience throughout the country, it is difficult to distinguish between simply using the term resilience to rename status quo behavior and a fundamental shift in action. We identify institutions th


arch terms: decision sup -
port flood , decision support tools flood , flood tools , decision support system flood , and flood decision
making . Our initial search resulted in a list of approximately 100 models and tools.
In screening the many self-


an the one for
which it was originally designed?
• U.S
. application: Has the DST been implemented in the United States, or could the tool
be easily transported to a U.S. context?
• usa
bility and transparency: Could well-trained planners found in


 have an associated approach for measuring resilience or indi -
cators of resilience, and many reviews exist of indicators and metrics of resilience. In Chapter Four, we focus on how these metrics relate to the frameworks and how they succeed or fail


(1) the magnitude of a stressor, as mea -
sured by the system’s movement from one equilibrium state to another, and (2) the length of time it takes for the system to move into homeostasis once the stressor has been removed. These ideas were first tra


not locked into specific strategies) that have diverse resources” (pp. 27, 28).
Resilience Alliance, undatedEcological system“[T]he capacity of a system to absorb disturbance and reorganize while undergoing change so as to still retain essentially th


tem from three separate
concepts: vulnerability, risk assessment, and adaptive capacity. All of these ideas are linked to a common goal of reducing a community’s risk to external forces (Lei et al., 2014). As Miller et al., 2010, and others have not


stressor will reduce the distance between A and B. That is, the system has roughly the same functionality as it did prior to the disruption. The idea underlying absorptive capacity is that, if a community has made an effort to reduce the impact of a


dicators in Chapter Four.
Additionally, there has been some emphasis on the role that resilience plays in the com -
pression of the time to postevent functioning (Olshansky, Hopkins, and Johnson, 2012). From a practical standpoint, much of the fundin


mmunities are communities that are self-sufficient in that they do not rely on external support after an event takes place (see, for example, Hall, 2014; Anh et al., 2013; and 12    Landscape Survey to Support Flood Apex National Flood Decision Suppo


s after a disaster occurs, there is a risk that strategies or actions to reduce vulnerabilities will be undervalued or ignored entirely. Alternatively, if the focus is solely on hazard mitigation, important capacities for the recovery process might b


nternational Development focuses on subsystems rather than on the system as a whole. This leaves the interdependencies that arise mostly unconsidered. In contrast, NIST explicitly has a chapter on dependencies across systems.
From the Arup Internatio


 a
bility for rapid rebound, to reestablish function quickly and avoid long-term disrup -
tions.
The Rockefeller and Arup International Development lists of characteristics are very
much aligned. These characteristics provide a means of developing c


inished capacity, using the pre-event techniques resulting in persistent dysfunction and vulnerability to future stress -
ors, or the system adapts to an altered environment. This characterization of resilience shows that resilience enters not only i


ocial and natural environments (although the framework could easily be adapted to incorporate such considerations). In particular, Rose develops his framework with two questions in mind: (1) How can the economic system be modeled in order to predict


 Toolbox
resilience context. There are five main components to the Francis and Bekera framework (see
Figure 3.4):
1. sys
tem identification
2. vul
nerability analysis
3. res
ilience objective-setting
4. sta
keholder engagement
5. res
ilience capacit


iple, as opposed to a struc-ture organized around social, economic, natural, and built systems (see Figure 3.5). The start -
ing point for the Bruneau et al. analysis is that the four dimensions of resilience are technical (T), organizational (O), so


ystems frameworks when applied to resilience are how the subsystems are segmented: along infrastructural lines (electric or water) or functional lines (e.g., social, built, natural). Additionally, how the interdependencies across systems are incorpor


 participants to engage in the decisionmaking process.
Interdependencies within a system can matter. To determine the total effect of invest -
ments in resilience made in a subsystem, one needs to measure the direct value to that sub -
system, as wel


ers
The levers are feasible actions or strategies that decisionmakers have available to reduce vul -
nerabilities, increase resilience, or improve adaptive and other capacities. Decisionmakers can control levers, in contrast to the external uncertain


s:
1.
 Defi
ne the problem.
2. Iden
tify feedback.
3. Dev
elop the model.
4. Per
form a sensitivity analysis.
First, the problem needs to be assessed and identified. This can be accomplished by
answering two questions: (1) What aspect of the system


n of the framework should simplify its use and highlight where touch points between systems exist rather than focusing on all the potential relationships that can take place in a community.27CHAPTER FOUR
Indicators and Metric Systems
Many alternative


ort Flood Apex National Flood Decision Support Toolbox
Table 4.1
Indicator Systems Reviewed in Cutter, 2016
Measure and Source TypeaSpatial UnitbFocuscDomaindMethode
Atlas of Social Protection: Indicators of
Resilience and Equity (World Bank, 2017)T


nd bottom-up approaches by how the system was developed. Top-down approaches are nomothetic, and bottom-up approaches are idiographic. Top-down approaches allow for comparison across communities, whereas bottom-up approaches are spe-cifically tailore


pex National Flood Decision Support Toolbox
Indicator Systems Not Considered in Cutter, 2016
In addition to Cutter, 2016, there are several other reviews, including Brooks, Aure, and
Whiteside, 2014; Link et al., 2015; and Winderl, 2014. Brooks, Aur


e vulnerability literature that measures a system’s vulnerability in the absence of considerations about exposure and hazards. A community might be vulnerable when measured using the Social Vulnerability Index (Cutter, Boruff, and Shirley, 2003), but


ary metric of resilience. Rather than relying on simple averaging or factor analysis to provide weights, the system’s actual characteristics and network could be used to form an index of resilience.33CHAPTER FIVE
Resilience in Practice
This chapter r


esilience in communities,
including the use of disaster-loss data to foster long-term commitments to enhancing resilience
•
 dev
eloping and deploying tools or metrics for monitoring progress toward resilience
• bui
lding local, community capacity b


in Practice    35
levels of government and by firms, such as utilities and telecommunication companies. Even
within the federal government, responsibilities are sufficiently distributed that fully centralized rulemaking for resilience is impractical


 (NFIP).
The first objective is critical to ensure that, to inform their investment decisions, local
decisionmakers have as much information as possible about the risks they face. This effort is
explicitly intended to incorporate future risks influ


 its emphasis on preparedness
and resilience in the United States. In 2012, the Red Cross undertook major community resil -
ience pilot efforts in New Orleans, Mississippi, Miami, San Francisco, and Denver, followed by similar programs in smaller lo


d programs developed within the network be adopted as common practice in a much larger number of cities and other contexts.
Although the 100RC program has global scope, it has a strong presence in the United
States. Its first two cohorts of cities (


is project has been discontinued, many of the ideas can be seen in the Rockefeller approach to 100RC.
Urban Resilience Planning
To understand how the ideas and theory of resilience have been brought to bear on practice
and planning, we provide a ser


r period. Addressing
these problems requires money and engineering but also close cooperation and coordination between the many agencies and private firms that are involved in one way or another with the Los Angeles water supply. The report also ide


orts, in some detail.
The report identifies multiple classes of hazards and assesses the city’s vulnerability to
each. Assessed hazards include coastal erosion; coastal storms; disease outbreaks; drought; earthquakes; extreme temperatures; flooding;


The Seattle postdisaster recovery framework emphasizes the difference between initial disaster response, which focuses on command and control issues and is often government-led, and disaster recovery, which is an inherently bottom-up, collective proj


mproves the community’s overall well-being” (Boulder Office of Emergency Man -
agement, 2013, p. 27).
The plan centralizes coordination during the recovery phase with a recovery coordinator,
who is part of the staff of the Boulder Office of Emergenc


isory Control and Data Acquisition tools, geographic information system (GIS)–based data management systems, outage management systems, distribution manage-ment systems, mobile workforce management systems, and modern business analytics tools. These


s essential to the functioning and recovery of a modern city; second, it is illustrative of a class of cross-jurisdiction networked infrastructure systems, including telecommunications and water supply, that are often owned and operated by private-se


al focus rather than individual community. At present, perverse incentives might exist that stem from the way that funding occurs in such programs as the Hazard Mitigation Grant Program and the NFIP. The incentives that communities face provide an un


relied on the first two—less stringent—screening
criteria.
We consciously made a decision not to include any H&H model by itself in the absence of
a well-documented field application that illustrated its use in real-world conditions. Addition -
all


hinking about the means by which decisions are teed up and made, as opposed to computational or modeling methods used to support decisionmaking.
Our goal in segmenting the DSTs by function is to show the breadth of tools that deci -
sionmakers have u


luation of structural interventions. Other than Hazus, the most commonly used risk assessment tool is MIKE FLOOD, which is already part of DHI’s flood toolbox (DHI MATLAB Toolbox). Table 6.1 provides a summary of risk assessment tools.48    Landscape


astructure, and
other physical assets, other tools focus on its effects on natural systems and ecosystem services. Human and natural systems are intimately linked, so this distinction in categorization is a matter of emphasis and framing. These tool


 tool and DST that helps to identify evacua -
tion decision times and the potential for significant storm effects, such as wind and storm surge. HURREVAC translates live-forecast track and wind extent information into interactive maps and reports des


rios or simulations of synthetic storms to better understand the impacts of specific projects or suites of projects. Most importantly, when multiple projects are considered, these tools allow a better understanding of the trade-offs that the decision


 consideration of alternative projects or project parameters. Many of the modeling approaches in the risk assessment category could be modi -
fied to evaluate projects. Table 6.5 lists project evaluation tools.
Integrated Decision Support Tools
Tools


en its architecture, it is likely that this tool could be adapted to explore site-specific alternatives, but it does not appear to have been used in this manner to date.
THESEUS DSS (THESEUS, undated [a]) represents yet another European entry in the


ration, undated (b). This decisionmaking framework uses the XLRM frame-work discussed in Chapter Three to help frame the decision problem and then uses hydro -
logic modeling, in the case of flood-related applications, together with uncertainty analy


 RAND Ho Chi Minh City, Louisiana, New
York, Colorado River Basin
CPRA Planning Tool RAND Louisiana
Risk Mapping, Assessment and
PlanningFEMA Nationwide, United StatesDecision Support Tools    55
Summary of Findings
Risk Identification and Assessme


r. That is, these kinds of tools are designed to illuminate the trade-offs that decisionmakers face and combinations of interven -
tions that might best achieve progress toward multiple public goals in the presence of a budget -
ary constraint.56


sts and benefits provides a structured way to develop
and evaluate alternatives. Many jurisdictions engage specialized engineering firms to help with the development of flood damage mitigation plans using DSSs of this sort. The Marin County (Califor


mitigation action database and scoring system. Although the approach was systematic, it focused strongly on its deliberative process, using computerized tools to assist with various parts of the process. The result has been well received, but the pro


f thousands of potential storm events to build up a detailed statistical portrait of which combinations were likely to be most in line with community goals and, often more importantly, which combinations were least likely to produce catastrophic fail


e resilience of the whole. The key is finding a balance between detail and parsimony to understand how different investments and dis -
ruptions cascade through the entire system. The design of the framework should simplify its use and highlight where


ces in information technology and control systems to build more-resilient networks that are capable of handling the extremely dynamic loads and potentially larger storms that the coming decades might bring.
The larger perspective provided by framing


recent decades, DSTs have moved from basic physical models, including maps and table-
top demonstrations, and informed by historical flood patterns and basic geophysical mea -
surements, to sophisticated hydrodynamic simulations that can produce deta


allow for deep exploration of flooding scenarios and uncertainty. For this reason, flood risk viewers are likely to remain fundamental tools for building risk literacy among decisionmakers and community members alike. In more-sophisticated integrated


d Greece, Thailand, worldwide
Motivation for development • “Re
liable flood modelling tools . . . enable us to analyse flood events
and develop flood protection measures or flood mitigation strate-
gies in the attempt to prevent the losses of human


Used across a variety of jurisdictions and agencies already
Potentially valuable to DHS? Currently being used by DHS Science and Technology Directorate
SOURCES: National Center for Computational Hydroscience and Engineering, undated; National Center


rea caused by riverine floods in plain settings.”
•
 Vis
ualize the following:
• Simp
le interface
• Flo
od simulation graphs
• Inu
ndation maps
Input data requirements • Targ
et area
• Top
ography data
• Prec
ipitation data
• Hyd
rologic data
• Econ


ercritical, and mixed-flow regimes. Steady and
unsteady flows.
• Uns
teady-flow module handles hydraulic structures, including dam-
break analysis, levee breaching and overtopping, pumping stations, navigation dam operations, and pressurized pipe sy


/R3” (p. 5).
Functionality •
 ArcG
IS extension
• Req
uires Spatial Analyst extension
• Pro
vides tools for visualization and risk compilation from various
flood projection data sets
• Use
s existing government flood data sets
• Too
l modeling is li


k to real-time and near-real-time atmo-
spheric, oceanic, and hydrologic information on the Internet
Input data requirements • Met
eorological, oceanographic, and hydrologic observing station
data
• Cri
tical facilities
• Dem
ographic and business d


d Decision Support Toolbox
Table A.13
Advanced Circulation
Aspect Description
Lead organization or author University of North Carolina at Chapel Hill
Locations where applied Nationwide, United States
Motivation for development • Hurr
icane forecastin


-tion” (Sustainable Coastal Communities, undated).
Functionality “CommunityViz offers many ways to present the analysis including real time demonstrations, Internet reports, and WebShots (a dynamic presentation tool). Google Earth and SiteBuilder3D p


ng hazards and sea level rise. This website should be used to promote enhanced preparedness and land use planning decisions with considerations for possible future conditions.”
Functionality Visualization of
•
 SLR (
bathtub model, 0 to 6 ft.)
• Con


ul visualization but limited decision support
SOURCES: Office for Coastal Management, 2017b; Environmental Dataset Gateway, 2012.Summary Tables    83
Table A.18
National Oceanic and Atmospheric Administration Sea Level Rise Planning Tool
Aspect Descr


rds
• Cre
ates a collection of maps to download or share online to communi-
cate flood exposure
• Pro
vides guidance for using the maps to engage community mem-
bers and stakeholders in conversations about potential coastal flood impacts
•
 Off
ers a


thor FLOODsite
Locations where applied Europe
Motivation for development “[E]nables decision makers to integrate multiple and complex relationships
between natural hazards, social and economic vulnerability, the impact of measures and instruments fo


sion-makers weigh potential conflicts among spatially-explicit management options that involve new activities or new infrastructure.”
User interface •
 Int
eractive website
• Simp
le interface
• Ove
rlays various basic boundary maps
• “[S
]patially-e


It compares the services [provided] by the current utilisation of the landscape to estimates of its potential capability, and uses this information to identify areas where change might be beneficial, and where maintenance of the status quo might be


applicable tool
• Use
s widely used GIS tools
SOURCE: NHDES, 2012.Summary Tables    91
Table A.26
Watershed Resources Registry Riparian Zone Restoration Suitability Model
Aspect Description
Lead organization or author Maryland Watershed Resources Reg


ous state and local sources to assist the local emergency manager
Input data requirements •
 Nati
onal Hurricane Center’s forecast advisories
• Hur
ricane evacuation study for the county or parish of interest
Outputs • War
ning time for evacuation
•


del connections, Delft-FEWS has also been applied in a wide range of different operational situations. Examples are water quality forecasting, reservoir management, operational sewer management optimization, and even peat fire prediction” (Deltares,


vision offices
Locations where applied
Motivation for development “This mission encompasses the regulation of river flow through more than
700 reservoirs, locks, and other water control structures located throughout the Nation.”
Functionality Integr


t Flood Apex National Flood Decision Support Toolbox
Table A.33
Flood Integrated Decision Support System, Melbourne
Aspect Description
Lead organization or author Flood Integrated DSS
Locations where applied Melbourne Water, Melbourne, Australia
Moti


d Apex National Flood Decision Support Toolbox
Project Evaluation
Table A.35
Coastal Louisiana Risk Assessment Model
Aspect Description
Lead organization or author RAND Corporation
Locations where applied Coastal Louisiana
Motivation for development


uitability for DHS and Flood APEX
Uncertainty and climate change Full risk analysis
Portability Designed to be used around the world
Potentially valuable to DHS? Might be difficult for communities to use the tool to consider trade-offs but
provides


X
Uncertainty and climate change Not explicitly but through which storms are used
Portability Yes
Potentially valuable to DHS? Yes, ability to both visualize impacts and estimate costs
SOURCE: USACE, undated.Summary Tables    103
Table A.38
Hydrologi


ping solution for the EU Flood
Directive. The package comprises 5 modules including flood estimation, risk and damage assessment as well as map generation” (p. 1)
Functionality •
 Inc
ludes MIKE FLOOD tool
• Han
dles tasks of the EU Floods Directive


4; Nujic, 1995; Neubert, Naumann, and Deilmann, 2009.106    Landscape Survey to Support Flood Apex National Flood Decision Support Toolbox
Table A.41
THESEUS Decision Support System Software Tool
Aspect Description
Lead organization or author THESEUS


n of risks
• Floo
d area
• Pro
bability of inundation plot
• Pri
oritization of risk areas
User interface
Suitability for DHS and Flood APEX
Uncertainty and climate change • “MD
SF2 . . . can explore a large number of scenarios very quickly,
making


 objectives in many plausible futures, rather than optimal in any single best estimate of the future” (p. 2).
Functionality •
 Pre
sented as a World Bank publication
• Use
s XLRM analytical framework
• X: exo
genous uncertainties (e.g., population, c


rge geography
SOURCE: Groves and Sharon, 2013.Summary Tables    111
Table A.46
Risk Mapping, Assessment and Planning
Aspect Description
Lead organization or author FEMA Risk Mapping, Assessment and Planning Program
Locations where applied Widely used


e, “Sea Level Rise Planning Tool (NOAA, FEMA, Army Corps),” June 2013.ADCIRC— See Advanced Circulation.
Adger, W. Neil, “Social and Ecological Resilience: Are They Related?” Progress in Human Geography , Vol. 24,
No. 3, 2000, pp. 347–364.Advanced Ci


: Evaluation of State Hazard Mitigation Plans Under the Disaster Mitigation Act,” Natural Hazards Review , Vol. 13, No. 2, May 2012, pp. 139–149.
Berry, Christine, Josh Ryan-Collins, and Tony Greenham, Financial Systems Resilience Index: Building a S


 Central, “Surging Seas,” undated. As of June 23, 2017:
http://sealevel.climatecentral.org/
Coastal Protection and Restoration Authority, “Welcome to CPRA’s Master Plan Data Viewer,” undated. As
of October 1, 2016: http://cims.coastal.louisiana.gov


r G. Burton, and Christopher T. Emrich, “Disaster Resilience Indicators for Benchmarking Baseline Conditions,” Journal of Homeland Security and Emergency Management , Vol. 7, No. 1,
2010, art. 51.
Deltares, untitled about page, undated (a). As of Ju


. As of June 18, 2017:
https://www.fema.gov/media-library/assets/documents/23194———, FEMA Strategic Plan 2014–2018 , Washington, D.C., July 18, 2014. As of June 18, 2017:
https://www.fema.gov/media-library/assets/documents/96981———, “Region II Coas


Vol. 530, February 18, 2016, pp. 307–312.
Gardner, Mark R., and W. Ross Ashby, “Connectance of Large Dynamic (Cybernetic) Systems: Critical Values for Stability,” Nature , Vol. 221, November 21, 1970, p. 784.
Global Risk Information Platform, United


Fifth International Conference on Intelligent Human–Machine Systems and Cybernetics , IEEE Computer Society, 2013, pp. 50–53.118    Landscape Survey to Support Flood Apex National Flood Decision Support Toolbox
Hughes, Karl, and Helen Bushell, A Mult


r-floodplain-section/documents/
flood-hazard-management-plan-update.aspxKirby, David, Flood Integrated Decision Support System for Melbourne , c. 2015. As of October 1, 2016:
http://www.floodplainconference.com/papers2015/David%20Kirby%20Full%20Pa


CI— See Land Utilisation and Capability Indicator.
Maryland Department of the Environment, “Final 2010 Integrated Report of Surface Water Quality in
Maryland,” April 2, 2010. As of June 28, 2017: http://www.mde.state.md.us/programs/Water/TMDL/Integr


San Diego, CA
Workshop , February 11, 2015a. As of July 13, 2017:
https://www.nist.gov/sites/default/files/documents/el/building_materials/resilience/
Framework_LineNumbered_75-25_11Feb2015.pdf
———, Community Resilience Planning Guide for Buildin


cal Infrastructure Security and Resilience , Washington, D.C.: White House, Presidential
Policy Directive 21, February 12, 2013. As of June 21, 2017: https://obamawhitehouse.archives.gov/the-press-office/2013/02/12/presidential-policy-directive-crit


e of Coastal Resilience Indicators—A Final Report , Coastal Services Center and National Oceanic and
Atmospheric Administration, December 2010.122    Landscape Survey to Support Flood Apex National Flood Decision Support Toolbox
Pearce, David W., an


nd Michel
Bruneau, A Framework for Defining and Measuring Resilience at the Community Scale: The PEOPLES Resilience
Framework , University at Buffalo, State University of New York, Multidisciplinary Center for Earthquake
Engineering Research, Tech


gency Management, Seattle All-Hazards Mitigation Plan , July 2009. As of June 19, 2017:
http://www.seattle.gov/Documents/Departments/Emergency/PlansOEM/HazardMitigation/
SeattleMitigationPlan.pdf
———, Comprehensive Emergency Management Plan: Base Pl


a Disaster-Resilient Community: A Guidance Note , version 2, November 2009.
UK Environment Agency— See United Kingdom Environment Agency.
UNISDR— See United Nations Office for Disaster Risk Reduction.
United Kingdom Environment Agency, Framework and


ronmental Effects Research Laboratory, EPA/600/R-15/059,
November 2015. As of July 14, 2017: https://www.epa.gov/sites/production/files/2016-03/documents/wmostv2_userguide_508final.pdfReferences    125
———, “Exposure Assessment Models: WMOST 2.0 Doc


l, and Philippa Howden-Chapman, “Understanding Resilient
Urban Futures: A Systemic Modelling Approach,” Sustainability , Vol. 5, No. 7, 2013, pp. 3202–3223.
Ziyath, Abdul M., Melissa Teo, and Ashantha Goonetilleke, “Surrogate Indicators for Assessin


TRACY C. KRUEGER, SEAN ROBSON, KIRSTEN M. KELLER
An Examination of
Recruiting and Selection Practices to Promote Diversity for Colorado State Troopers
The Colorado State Patrol (CSP) has recently increased investments to attract, select, and
train


atives to improve recruit -
ing and selection at CSP, trends in the demographic composition of recent CSP hires, and a discussion of potential barriers to hiring qualified troopers while also diversifying the CSP workforce. The third section provides


ies
RJP realistic job preview
SME subject-matter expert3
 Ȥ describe current policies and practices for
re
cruiting and selection
 Ȥ ide
ntify potential barriers to recruiting
and selecting a qualified and diverse set of
troopers.
• We r
eviewed t


 jeopardy, assisting other agencies, disaster mobilization, and crowd control. A cadet learns to formulate action plans that anticipate continuously changing circumstances and how to resolve emergencies.
•
 Att
end training and certification courses


econd classes of 2018 (i.e., 2018-1 and 2018-2).
11
We emphasize that timelines are specific to each class or cohort hired. Efforts are under way to shorten the duration of the background investigation.  2. Findings: Ongoing Initiatives,
Demographi


 evaluation—developed by an external consulting firm—assesses basic educational skills, emotional outlook, interpersonal skills, and practical skills.
b Candidates are given three hours and 15 minutes to
complete the multiple-choice test, which is s


and two CSP commissioned officers. The interview is conducted in person or via video chat and is expected to take 15 to 25 minutes. The panel provides independent ratings on a five-point Likert scale (from 1= less desirable/poor to 5 = most desirabl


tant Contact emails In 2016, CSP designed and implemented the distribution of a standardized email to all individuals who express interest in CSP and provide their email address. This email communication offers a structured and systematic way to enga


hnic minority applicants and an increase of 17 percent-age points in racial/ethnic minority appointees. Additionally, when compared to 2014, there has been an average increase of 3 percentage points in female applicants and female appointees.
Despi


attracted to organizations whose members are similar to themselves in terms of personality, values, interests, and other attributes, and (2) organizations are more likely to select those who possess knowledge, skills, and abilities (KSAs) similar to


asons not to join. From our review, there has been limited investigation of propensity to apply in law enforcement.
Nature of the jobThe CSP trooper position emphasizes patrolling highways and roads, which may not appeal to everyone interested in law


ected if they have particularly strong family ties and may subsequently be less inclined to relocate. There is an emerging literature regarding the “family-relatedness” of work decisions
l and understanding how to
better promote “sustainable careers


t a highly qualified and diverse pool of applicants.One of the strongest factors influencing increases in organizational diversity is establishing positions with responsibility for diversity efforts.
p
a David Rollins, Executive Accreditation Report


 determine the accuracy of these perceptions. In some cases, the data needed to evaluate a finding may not be available; therefore, CSP may need to initiate efforts to collect additional data.
A review of these key findings suggests that CSP
may ha


sources
HirePropensity to
apply to CSPProcess and
resource
management
SOURCES: Adapted from Greg Ridgeway, Nelson Lim, Brian Gifford,
Christopher Koper, Carl F. Matthies, Sara Hajiamiri, and Alexis K.
Huynh, Strategies for Improving O fficer Recrui


d labor market conditions.
Policy factors Recruiting resources • Al
ign key features of the role and CSP with marketing materials.
• En
gage in precision recruiting of highly qualified women and racial/ethnic
minorities.
• Co
ntinue building resou


nd we encourage future efforts to continue identifying best practices for using social media in order to recruit a highly qualified and diverse pool of applicants to positions in law enforcement. Propensity to Apply to CSP
Although CSP has basic demo


orkers, job).29 These
scales could be combined with other custom items for a survey that can be used to develop profiles that characterize low and high performers and describe characteristics of the job and the working environ-ment. These profiles c


 target the relevant population, including the use of web-based, phone-based, and traditional paper-and-pencil surveys.
Monitor Demographic Trends and Labor Market
Conditions
Using the information collected from the propen -
sity analysis, CSP sho


tics emphasized are representative of CSP and are important to the targeted applicant pool. For exam-ple, prior research for police departments in Southern California indicated that applicants not only wanted to serve their communities but also had a


 increase perceived attrac-
tiveness of the job and the organization, CSP should use women and racially/ethnically diverse recruiters to hold recruiting events that target female and racial/ethnic minority applicants (e.g., use a female recruiter at


ocation of recruiting resources.  17
Adjust Application Windows
Currently, when a new position at CSP is advertised,
an individual must submit an application within 14 days. In addition to the short window to complete an application, the dates for p


net as the place where they heard about the position. Over 30 per-cent of responses contained a term related to the Colorado government website (e.g., “Colorado gov,” “Colorado.gov,” “state website,” “Colorado jobs,” “state agency website”). Over 10


e hired. These data could also be supplemented with open-ended com-ments from recruiters to share additional insights on the event or suggestions for future events.
The data collected from this type of tracking
system would allow CSP to develo


 in lower passing rates for a subgroup. Although lower passing rates for a subgroup does not necessarily indicate that 19
the test is biased, additional steps should be followed
to ensure that each test section is related to the job. If the test sec


do not know how to gather the information requested. To address this concern, CSP could imple-ment a preliminary background assessment as an early indicator of potential issues. The Los Angeles Police Department encourages applicants to complete a pr


gible and interested in working for CSP. The results will also need updating regularly to correspond with potential changes in the demographic composition and interests of the Colorado workforce.
TABLE 3.2
Prioritized Project Recommendations
Priorit


, the recruiters also conduct background investigations.
10 Justin Mullins, 2016 Analysis Report; 2017 Recruitment, Affir-
mative Action and EEO Plan , Colorado State Patrol, undated.
11 The label “2018-1” signifies the year and training academy
s


pendent on passing this assessment. CSP explained that this change was inspired by annual trooper requirements: Troopers are required to take an annual physical fitness test, but they are not required to pass. Nevertheless, several CSP SMEs suggested


limate,” International Journal of Management , Vol. 26,
No. 3, 2009, p. 347.
41 Adopting a longer-term perspective, we emphasize the impor-
tance of ensuring that efforts to recruit more women and racial/ethnic minorities do not take away from the c


ce Brand Equity,” Journal of the Academy of Marketing Science , Vol. 28, No. 1, 2000,
pp 128–137; and Kevin Lane Keller, “Conceptualizing, Measur-ing, and Managing Customer-Based Brand Equity,” Journal of Marketing , Vol. 57, No. 1, 1993, pp. 1–22.


an Educational
Research Association, 2014.
Berry, Leonard L., “Cultivating Service Brand Equity,” Journal of
the Academy of Marketing Science , Vol. 28, No. 1, 2000, pp. 128–
137.
Breaugh, James, “Employee Recruitment: Current Knowledge
and Import


e-74
Greenhaus, Jeffrey H., and Ellen Ernst Kossek, “The
Contemporary Career: A Work-Home Perspective,” Annual Review of Organizational Psychology and Organizational Behavior , Vol. 1, 2014, pp. 361–388.
Greenhaus, Jeffrey H., and Gary N. Powell, “


s/monographs/MG881.html
Manhardt, Philip J., “Job Orientation of Male and Female College
Graduates in Business,” Personnel Psychology , Vol. 25, 1972,
pp. 361–368.
Matthies, Carl F., Kirsten M. Keller, and Nelson Lim, Identifying
Barriers to Diver


er Rees, Tom Jordan, and Jason Cheney, Cop Crunch: Identifying Strategies for Dealing with the Recruiting and Hiring Crisis in Law Enforcement , Washington, D.C.: National Institute of Justice,
NCJ 213800, 2005.
van Saane, N., Judith K. Sluiter, Ad


n this publication, visit  www.rand.org/t/RR2999.
© Copyright 2019 Community Oriented
Policing Services (COPS)About This Report
The Colorado Department of Public Safety continues to seek ways to improve
recruiting and hiring practices across its wo


Eric HellandThe Role of Health
Care Liens in Litigation and RecoveryCORPORATIONLimited Print and Electronic Distribution Rights
This document and trademark(s) contained herein are protected by law. This representation
of RAND intellectual property


ore and smaller liens than private lien holders.iv    The Role of Health Care Liens in Litigation and Recovery
RAND Institute for Civil Justice
ICJ was founded in 1979, and it has been dedicated to making the civil
justice system more efficient and


the same rights as the Center for Medicare Ser  vi
ces with
regard to settling parties. Thus, Medicare, Medicaid, and Medicare Part C plans now all have super lien rights.
This study examines how frequently health care liens appear in
large litigat


par ti
c u la
r lien type dominates the recovery.
Although lien rights for Medicare Part C and Medicaid are ex -
panding,  th
ere is no evidence of a trend in the number of health care
liens. However, our data cover only 2008–2017, so some care mu


nsation
CMS  Ce
nter for Medicare Services
ERISA  Em
ployee Retirement Income Security Act
FEHBA  Fe
deral Employees Health Benefits Act
ICJ RA
ND Institute for Civil Justice
MAIS  Me
dical Assistance Intercept System
MDL  mu
ltidistrict litigation
M


ly aimed for mass settlement.
 Th
ere have been numerous discussions of the prob  le
ms with pro -
ce
ssing a massive number of claims through a court system that has
traditionally been focused on providing individualized justice (see Nagareda, 2008


ple,
 un
less the lien is resolved, it is often difficult to
 set
tle a claim simply  bec
ause the plaintiff may have liens larger than
a settlement offer, thus making the plaintiff indifferent when faced with the choice between taking the settlem


nsion of such “super lien” rights into Medicaid and other statutory liens has compounded the complex -
ity of settlement (see Dixon and Kuznitsky, 2017).
To understand how burdensome litigants find medical liens, con -
sider the Reno Air disaster. On


less frequently, plaintiffs themselves. Increasingly, however, not only plaintiffs and their
 la
wyers but now defendants and third-  pa
rty
insurers must deal with liens against the injured parties for failure to pay child support, reimbursement f


unable to pay for some other reason, the expense falls to the secondary payer.
3 Although the origin of the Medicare liens and King Kong analogy is somewhat obscure,
Schmidt (2006) cites Tate and Holloway (2005) as first using this characterization.


tes v. Baxter Int’l, Inc. , 345  F.3d 866, 875 (11th  Cir.
2003) and Thompson v. Goetzmann , 337 F.3d 489, 498 (5th Cir. 2003) as examples of the
differing approaches across cases. In Goetzmann  the Fifth Cir  cu
it denied Medicare’s right
to coll


.S.C. 1395y(b)(3)(A))—
 he
nce our characterization of them as a bundle of rights.
9 The fines can be up to $1,000 per day for failure to report or erroneous reporting. See 42
U.S.C. 1395y(b)(2)(B)(iii):
In order to recover payment made  un der this


mented a three-
 ye
ar statute of limitations for CMS to pursue recovery.Background on Lien Rights for Health Care Providers    11
that  were settled years ago—  th ey  we re the plaintiffs’ and the plain -
tiffs’  la
wyers prob  le
m. (Dixon and Kuz


as limited to recovering only the portion of a
 le
gal judgment
specifically allocated to medical bills.11 Heidi Ahlborn of Arkansas
was injured in an accident and Medicaid paid for her treatment. Subse-quent to the payment, she sued the driver of


  we
re placed in a  le
gal position inferior to their counter  pa
rts’ in
Medicare. If the plaintiff  ’s
medical costs had been paid by Medicare—
an
d not by Medicaid, as was actually the case—  th
e plaintiff would have
clearly owed the full a


 Supreme Court denied certiorari on the plaintiff
 ’s a
ppeal. The decision allows Part C plans
essentially the same rights as CMS with regard to settling parties.15 Thus,
Medicare, Medicaid, and Medicare Part C plans now all have super lien rights


not covered by insurance,
and hence would only be compensated via the settlement, have highest priority. Fi  na
lly, Cali -
fornia has a common fund rule  un
der which liens are reduced by the same percentage as the
contingent fee the client is bei


ly assign lien rights via contract; however, a
number of states hold that a settling party can be held liable if they have knowledge of the health plan’s claim before settlement. In
 th
ese
19 Federal employees are covered by FEHBA (1959).
20 In 2


 hold
 th
ose funds as a fiduciary, for the benefit of the client and third party.
This duty includes a duty to notify the client and third party, and to deliver the property to the client and third party. This rule exists, in large part, to protec


r the Reno Air Show injuries would each represent a compensation event. Although each might involve hundreds of plaintiffs across multiple law firms, we aggregate the claims for the purposes of this study. The earliest claims date from 1992, although


imately 2,900 claimants and 3,500 cases, but the smallest event has only 2 claimants (and cases), while the larg -
est has over 48,000 claimants and over 56,000 cases.
1
To give a better sense of the temporal dimension of the data and
the size of th


er the Sample Period (96 compensation events)
NOTES: Sample size = 96 compensation events in which the ﬁrst case was
resolved after January 2008. Area of symbol proportional to number of cases.
RAND RR2393-3.1Earliest settlementJan-2008 Jan-2011 Jan


n’s prominence among federal MDL cases, the larg -
est compensation event type in the Garretson data. Exposure cases rank second in terms of number of events, ahead of device implantation cases. Interestingly, device implantation cases are the larges


ursing smaller Medicare liens that the private plans, for example, would not find lucrative to attempt to recover. Confounding this interpretation is the fact that private insurance fee schedules are generally higher than Medicare, so we would expect


xtensive recovery pro  ce
ss,
which includes statutory reductions for attorney fees and expenses.3
Medicaid, by contrast, is the second most frequent lien type, but the average Medicaid lien recovery value is less than the average Medicare lien rec


r Medicare’s recovery.Figure 4.4
The Distribution of Resolved Liens
05101520Frequency of Occurrence
0 0.5 12 1.5
NOTE: Sample size = 96 compensation events.
RAND RR2393-4.4Average number of liens per case (all types)34    The Role of Health Care Lien


 This is the gross amount, so attorneys’ fees (typically one-
 th
ird), expenses,
and non–  he
alth care liens would need to be deduced from this amount
to capture the net amount to clients. As shown in the chart, liens, on average, make up less th


s, we control for the
6 The fixed effects are an indicator variable equal to one if the first case in the event was
resolved in a given year (i.e., the indicator variable for year 2009 equals one if the first case for
event X was resolved in 2009)


The Role of Health Care Liens in Litigation and Recovery
to resolve health care liens associated with the claim. For this reason,
changes in the composition of the Garretson Group’s business could mask any trends. Fi
 na
lly, it is worth reiterating


ce Supreme Court denied cert in In re Avandia  in
2013No,
 u
nless
 th
ere is a
contract provision with individual insurer Varies by state and individual insurance contractNo,
 u
nless
 th
ere is a
contract provision with individual insurerApp


Data for Figure 4.8
Program Type Medicare MedicaidOther
Government PLRP Private
Consumer product 34.40 28.16 19.52 0.00 17.92
Exposure 71.48 14.31 7.09 0.17 6.96
Malpractice 79.20 18.43 0.00 0.00 2.37
Medical device implantation 25.78 13.42 28.42 5.


Resolution Group, “What Is a Private Lien Resolution Program?”
September 30, 2013. As of April 30, 2018:  https://web.garretsongroup.com/claimant-education-posts/claimant-education/private-lien-resolution-program
———, memo, March 23, 2016.Heaton, Pa


2, 2006, pp. 469–489.
Shavell, Steven, Foundations of Economic Analy  si
s of Law , Cambridge, Mass.:
Harvard University Press, 2009.
SMART Act—  See S
trengthening Medicare and Repaying Taxpayers Act.
Spier, Kathryn E., “Litigation,” in A. Mitchell


SOCIAL AND ECONOMIC WELL-BEING
Wage Loss Monitoring
for Injured Workers in California’s Workers’ Compensation System
2014–2015 Injury Year Findings (Second Interim
Report)
Michael Dworsky, Stephanie Rennane, Nicholas Broten
Sponsored by the Califor


n, motivate the need to gather an accurate and up-to-date estimate of wage loss for all injured workers in the state.
RAND Institute for Civil Justice
The RAND Institute for Civil Justice (ICJ) is dedicated to improving th e civil justice system
b


................... 18
Figure 2.3. At-Injury Employment Rates for Injured and Control Workers ................................ 19
Figure 2.4. Trajectory of Injured to Control Earnings Ratio, 2014–2015 Injured Workers ......... 21
Figure 3.1. Tr


.......................................................................... 16
Table 3.1. Average Earnings Losses and Labor Market Outcomes by Year of Injury ................. 28
Table 3.2. Injured Worker Sociodemographics, Injury Characteristics,


portant
objective for workers’ compensation policy. This is  true for several reas ons. Above all, the
employment and earnings of disabled workers can reflect successful reco very, rehabilitation, or
adaptation, and thus may be viewed as indicator


 employers and self-insure as a private group or as a joint
public authority.
Nonfatal workplace-related injuries or illnesse s may be classified as medical-only, temporary
disability, or permanent disabi lity. Medical-only claims involve no lost


 the number of new workers’
compensation claims reported to date has reached 670,000 in the 2017 injury year.
6 About one-third of workers with paid or settled perm anent disability benefits do not receive any temporary
disability benefits. 3 inju


,
8 including the

7 The Workers’ Compensation Rating Bureau of California, which develops rates for commercial insurers that serve
the California market, reports that PPD benefits were 41 pe rcent o


 system was
relatively modest between 2013 and 2014. Bills affecting benefits or compensability have largely
focused on peace officers, public safety workers, or narrower groups of workers (such as horse
racing jockeys or professional athl etes).


ead, the most substantial changes to the workers’ compensation system in 2014 and 2015
were most likely those associated with cont inued implementation of SB 863. Increases in
permanent disability (PD) benefits under SB 863 be gan to phase in for i


ry and increase the risk of poor labor ma rket outcomes. At the
same time, increased labor demand due to economi c growth means that employers may be more
motivated to accommodate workers after an inju ry, or that workers who end up leaving a job


hapter 2 of Dworsky, Rennane, and
Broten (2018); readers interest ed in a more complete introdu ction to our methods should
reference that study.
Data for Wage Loss Monitoring
We make use of several administrative data sour ces for this project,


idual-level claims data in  California that captures all
market segments (including self-insured and pub lic-sector employers). Given the absence of any
more reliable benchmark, this study will follow Dworsky et al. (2016) in treating FROI reported


concern in this report than they we re in Dworsky, Rennane, and Broten (2018).
To examine the potential impact of right-censori ng on the trends reported here, we used data
on the start date of indemnity be nefits to identify all claims from the 20


y benefits have not started yet and claims that have
not yet been filed. 11 in the Base Wage File. Both workers’ compensa tion and UI are nearly universal in California,
and so the wage and salary earnings of nearly all workers filing worker s’ com


s for a shorter window of time (one year pre-injury to two years post-injury) than in the earlier study. The
procedures used to define consistent names are otherwise identical to those described in Dworsky et al. (2016). Name consistency flags were


ol wo rkers to the analysis ma kes it possible to rely
on much weaker assumptions than would be necessa ry if we had data on only injured workers’
earnings. In this case, our key a ssumption is that, in the absence of the injury, the earnings of
i


kers who have been
lucky enough not to experience a job separation or an earnings decline for reasons unrelated to
workplace injury. This might seem reasonable in  some employment settings with high job


s, effectively letting the data tell us what level of earnings growth
or employment risk should be expected give n the experience of other workers at the same
employer. A valid control group such as the one used in this study allows us to isolate t


 ag e, gender, self-insured status,
and the pre-injury weekly wage.4 The pre-injury weekly wage is  by far more likely than these
other variables to be missing. A bout 11 percent of FROIs from good claims administrators are
casewise-deleted based