he world of Michigan indigent defense might be changing and for updating whatever standards are adopted at this time.
We then calculated mean and median hours for each of the nine study case types, using a mild form of Winsorization to address outlie


reached an agreement with the office wherein RAND would be provided with an extract of its CMS data conforming to our needs, and, in exchange, the office’s employees would not have to participate in the time study, primarily to avoid asking attorneys


 elsewhere (Table 2.9).
Table 2.8. Estimated Attorney Time Expenditures for Misdemeanor Cases
   Program
North Carolina public defenders Missouri public defenders Michigan
Michigan
Brooklyn Legal Aid Society & Brooklyn Defender Services
New York (fiv


ices primarily serves a single location, while the members of the private bar might have cases spread across multiple counties.
Table 2.10. Percentage of Recorded Case-Related Time Spent, by Activity
    Activity Type
Client communication
Discovery r


cated in the Upper Peninsula and the Northern Lower Peninsula, and to a random selection of 322 other attorneys in the roster who were not identified as public defenders. The decision to make a special effort to reach out to members of the indigent d


for comparison purposes. The survey concluded with a free-text field that asked the respondent to provide any comments related to caseloads, indigent defense issues in Michigan, or ways in which attorney resources affect the delivery of an adequate a


—Detroit metro Southeast—other Southeast Southwest
Upper Peninsula
Western NOTE: N = 323.
Activities
  A 45-hour workweek is common among survey participants, though 10 percent reported that they typically work 60 or more hours each week (Table 3.3).


dants (essentially functioning as “private public defenders,” presumably as a result of high volume contracts with local indigent defense systems), and (3) private attorneys for whom indigent defendants take up a relatively smaller part of their time


d retained) Appointed cases (all types)
Retained cases (all types)
Non–case-related time
PD Other–Heavy
Other–Light
    70.3 66.1 20.7 70.9 76.3 38.6
77.9 80.8 50.7 81.1 74.4 35.7 2.3 18.1 54.2 16.6 7.5 10.1
    NOTE: N = 48 for public defenders, 90


ght” of an NAC misdemeanor is 0.375 relative to an NAC felony, using a 150-to-400 ratio, the 128 misdemeanors would count as an extra 48 felonies on top of the 179 average felonies for Michigan public defenders). The average counts for the other two


day misdemeanors
Probation violations
Other adult criminal indigent defense trial court–level matter
Total
PD Other–Heavy Other–Light All Attorneys
4 5 2 3 20 10 10 10 20 15 12 15
100 35 30 40 200 50 44 75
200 75 50 75 200 200 50 100 150 50 25 61
10


eys are much more likely than others to perform their responsibilities without the aid of any support staff whatsoever (administrative staff, such as clerks, secretaries, or receptionists; investigators; paralegals or legal assistants; interns and ex


markedly in its needs, complexities, and challenges.” The recommended average should be “the average of all cases of that type taken together, including those that are pled out at an early stage with only modest time expenditures, those that require


9.2 65.6 86.4 85.3 78.2
          NOTE: Depending on the case type category, Ns for all attorneys range from 103 to 170, 22 to defenders, 27 to 53 for Other–Heavy attorneys, and 54 to 93 for Other–Light attorneys.
All Attorneys
44.7 31.9 50.4 42.5 45


enting a client in one of the study case types), the approach was quite different. We employed the Delphi Method, a feedback consensus method first developed by RAND in the aftermath of World War II as a way to systematically coalesce expert opinions


ttend in person, so we set up a speakerphone for audio participation, and an Adobe Connect web meeting so that the panelist could see what was displayed at the session, and our Delphi tool was hosted on a website that could be accessed from anywhere.


-Law, Law Offices of Walton & Scally, PC, Utica (Macomb County)
Melissa K. Wangler, Attorney-at-Law, The Law Office of Melissa K. Wangler, PLLC, West Branch (Ogemaw County)
Christopher B. Wickman, Attorney-at-Law, Nichols Law Firm, Lansing (Ingham Co


eir assumptions about the amount of time now spent on average for different case types and their own recommendations about necessary average hours on a paper form we provided. This was done to help the panelists minimize the effects of anchoring bias


by the mean).
When the panelists had responded to all of the questions, we projected a similar graphic and statistical summary of their responses on to a screen in the meeting room. The summaries were
78 Zipfinger, 2007.
  66
updated periodically as


d deviation of 4.98 (rounded). The CoV is the standard deviation divided by the mean of the original distribution of numbers, or, in other words, 4.98 divided by 11 (0.45, rounded).
80 See Gracht, 2012, and Kalaian and Kasmin, 2012.
67
 As indicated


pically required an attorney to fully investigate the behavior of the alleged victim, while the 4th-degree cases often lacked that particularly difficult aspect. There appeared to be general consensus that the CSC category was overbroad, so the follo


es was that such matters, as is true for all CSC cases, require
81 It should be kept in mind that though the panelists were providing their estimates of average (mean) attorney hours, each value entered into the Delphi application was simply a number


dard that has helped make the 40-hour workweek part of the ideal conception of the American way of life.82 But attorneys are likely to be classified as salaried professionals under FLSA and, as such, would receive no additional compensation for time


days per year
Duty days per year
Work hours per day
Total duty hours per year
Average reported non–case-related percentage of workweek Non–case-related duty hours
ANNUAL CASE-RELATED DUTY HOURS
Calculating the Recommended Standards
365.25 A 7 B 52.2


he input we received from attorneys in all three data collection efforts, we attempted to have the participants step out of their current roles as indigent defenders (if indeed that was what they were primarily doing at the time) and think more broad


n a year. As suggested by the calculations above, the two values—average time needed (often referred to as a case weight) and a maximum annual caseload—are related to each other mathematically by whatever estimate is used for annual case-related duty


seload and to do so with enough time to deliver effective defenses. To calculate the equivalent of an annual maximum caseload in such instances, we divided those absolute case weights into 1,855.8 hours or
111,348 minutes, depending on how they were


orado Maryland Colorado Texas
Virginia Washington Texas
NAC, 1973 Colorado North Carolina New Mexico Colorado Virginia
Case Type Measure
Effective Annual Maximum
    Felony 2
Felonies,upto10yearsimprisonment Caseweight(28.3hrs.) 66
Standard 64
  Felo


n.) 453 Case weight (225 min.) 495 Case weight (217 min.) 513
Case weight (191 min.) 583 Standard 672 Case weight (147 min.) 757 Case weight (137 min.) 813
                           Case weight (93 min.)
1,197
 78
Jurisdiction
Rhode Island Idaho Mar


20, respectively (essentially just the average hours for each of the two types), while the relative case weights for the two categories would be 4.0 (200 hours for type A cases divided by the 50-hour average for all cases) and 0.4 (20 hours for type


B, C, D) felonies (655.8 ÷ 40-hour case weight), or three murder or manslaughter cases and
80
seven CSC cases ([3 × 120-hour case weight] + [7 × 40-hour case weight] = 640 expected hours).
What about a situation where an attorney devotes less than fu


pical” indigent defense case in Michigan.98 Thus, a one-year misdemeanor has a relative
98 There is some evidence from both the time study and the attorney survey that at least for the participants in these two data collection exercises, 93-day misde


t say with certainty how many are currently handling a number and mix of cases that would exceed the recommended annual maximums. We
82
strongly urge the MIDC, local indigent defense systems across the state, and the Michigan trial courts to work tog


ge is that, based on what we learned in the survey, Michigan attorneys do not always limit their criminal law activities to just the nine categories of cases chosen for the caseload standards development effort. Indeed, with the exception of public d


 exceeds the recommended threshold.
As is true for any voluntary survey, however, not all participants answered all questions. Some, for example, provided information sufficient to calculate the percentage of all case-related time devoted to indigent


ast double the maximum.
Table 5.7. Weighted and Proportionally Adjusted Caseloads of Attorney Survey Respondents Percentage of Reporting Attorneys
      Level
Does not exceed standard Up to 24% over standard 25% to 49% over standard 50% to 74% over s


nt defender caseloads, at both the local system level and the individual attorney level, will be needed going forward to make sure that the standards continue to accurately reflect the evolving challenges and the unique local legal cultures facing de


from Participants in the Attorney Survey
The comments below were submitted by defenders responding to the attorney survey. The submissions, in alphabetical order, have been lightly edited for clarity and to avoid providing individually identifiable i


 in other counties, an appointed attorney is often required to be in more than one place at one time and judges should be understanding about that situation and/or not create such
         88
 conflicts to begin with (as far as practicable); some jud


lines, and impeachment issues. Furthermore, Westlaw & Lexis allow for running background checks on potential witnesses which allows for the defense attorney to obtain impeachable information in order to do their jobs effectively. Note the prosecutor


rsonal protection orders). It also allows defense counsel access to the other public records and PPOs of potential witnesses or alleged victims—that may undercut their credibility and be used to impeach them (or to engage in additional investigation)


ed to testify to impeach a witness. A defense attorney cannot conduct the investigation and later be a both a witness and defense attorney; the ethics rules don’t allow it.
PARALEGALS & INTERNS. They could be used to streamline costs if properly trai


ed by time; the time taken should be whatever is necessary to properly and thoroughly defend any client.
11. Every case is different, of course; e.g., in the CSC 4th case I am handling now, the assistant prosecutor was able to leverage a plea out of


for child support.
             92
 Our office has anywhere from 25–70 interns depending on the term. None of the interns are full-time. Many are not even half time, coming in only 1–2 days a week. Our office has one full-time support person. The rem


ttorneys to research and print as well as other attorneys who work and can assist, that would be helpful.
31. In our office, all felony matters are handled by rotating judges in one central location. As a result, multiple attorneys may have worked on


how many jail visits are compensated make it difficult to represent some clients when they do not comprehend the ramifications or impact to their life. Also the time necessary for forensic evaluations delays the ability to adequately communicate to c


s a day, by the time you add it all up, YOU ARE WORKING. When smart-minded attorneys bill for a retained case, they charge by the hour for wait time.
Why are we broke? Hidden reasons. We run up miles on our car shockingly fast, and thus have to repla


rance.
46. Murder cases require an extensive amount of preparation to appropriately provide effective and zealous counsel representation. At a minimum, counsel should allocate 350 hours to appropriate legal and factual investigation, in addition to p


n sharing statewide for special issues.
52. Not enough money for investigators, research, or other things that the state get but as defense attorneys we don’t.
53. Our lack of funding inhibits our access to experts and investigators. Adding arraignme


er form it comes in such as staff, equipment, etc.
63. The biggest problem is that there is not adequate compensation for attorneys accepting court appointments. The pay should be hourly, at least $150.00 per hour and travel time and research should


her than the public hallway outside the courtroom.
71. This time study should be modified into two separate categories. 1. Those offices that do not have vertical representation at the time of the study and; 2. those offices that do have vertical rep


ke in simply attempting to keep in touch with their clients. While I recognize that there are finite resources related to all publicly funded projects, including the delivery of indigent defense, the issue in developing caseload projections should re


cket, and I think you guys know about it. You must because the data exists. Why can’t this be talked about publicly?
78. We are given no resources or money for experts or additional testing. Our time is sucked away sitting in a judge’s courtroom aski


gard to that case type). We have edited the original comments for clarity and dropped some that appear to only have been notes intended for the sole use of the submitter.
Murder or Manslaughter
• I expect a great deal of discovery in these cases, as


t development.
• Will always conduct prelim, investigation, lots of discovery, and even if it is “just a CSC
4,” there are still lifelong consequences.
• Need to deal with experts and tons of story development.
• Even within CSC 1, a child vs. adult


ime is meeting with client; typically less is for discovery.
• Usually not that complicated.
• Less discovery for most cases, easier to negotiate.
• Based on talking to client, investigation.
• Probably less intense discovery, client will most likely


egal_aid_indigent_defendants/ ls_sclaid_def_train_eight_guidelines.authcheckdam.pdf
———, Ten Principles of a Public Defense Delivery System, February 2002. As of September 8, 2018:
https://www.americanbar.org/content/dam/aba/administrative/legal_aid_


ange, Vol. 79, No. 8, October 2012, pp. 1525–1536.
Hall, Daniel J., A Workload Assessment Study for the New Mexico Trial Court Judiciary, New Mexico District Attorneys’ Offices, and the New Mexico Public Defender Department: Final Report, Denver, Col


54zfszmcj4flssj3ni5vgk))/mileg.aspx?page=getObject&o bjectName=mcl-780-991
Michigan State Bar, “The Eleven Principles of a Public Defense Delivery System,” 2002. As of September 9, 2018:
http://sbmblog.typepad.com/files/11_priniciples2.pdf
108
Michig


w, John S. Hollywood, Dulani Woods, Mikhail Zaydman, and Brian A. Jackson, Visions of Law Enforcement Technology in the Period 2024–2034: Report of the Law Enforcement Futuring Workshop, Santa Monica, Calif.: RAND Corporation, RR- 908-NIJ, 2015. As o


Law Enforcement Efforts to Fight the Opioid Crisis
Convening Police Leaders, Multidisciplinary Partners,
and Researchers to Identify Promising Practices and to
Inform a Research Agenda
Sean E. Goodison, Michael J. D. Vermeer, Jeremy D. Barnum, Dulani


reatment promo- tion opportunities.
• Provide same-day, low-barrier access to treatment with a medication-first model of care.
• Provide syringe services to reduce associated harms and create treatment intervention opportunities.
• Use syndromic surv


aw enforcement officers experience additional physical dangers, mental trauma, and stressors. Needs related to officers’ mental health were seen as pressing, and participants gave high priority to strategies such as providing mental health interventi


elated over- dose deaths in 2010. The deadliest wave of the epidemic began in 2013, with the introduction of synthetic opioids into the illicit drug supply. Today, synthetic opioids—primarily fentanyl and its analogs—remain the primary driver of drug


ech given by then–Attorney General Jeff Sessions), the science of addiction and treatment, law enforcement–led efforts to combat opioids, key areas of cooperation among governmental and community stakeholders, law enforcement safety and wellness, and


riety of recommendations to address opioid use, addiction, and its consequences (White- house.gov, undated; Christie et al., 2017). The report acknowl-
edges that effective solutions to a crisis of this scope and scale will require significant invest


ces, the amount of dopamine in the body is naturally regulated by another chemical, gamma-aminobutyric acid (GABA). However, opioids bind to the part of the nerve that reduces GABA release, resulting in a prolonged release
of dopamine. Opioids are re


opioid use interferes with the brain’s reward system by impairing an individual’s ability to experience pleasure. This compounds compulsive opioid use as individuals seek to both re-achieve the rewarding effects of drug use and relieve withdrawal sy


ses to the crisis. The topics of interest were identified based on literature review and discussions among RAND, PERF, and NIJ experts. Each panel included a set of prepared presentations on different facets of the topic, followed by moderated discus


icers can rapidly administer naloxone, especially in rural jurisdictions where emergency medical services might not be as readily available (Davis et
al., 2014; Davis et al., 2015). Equipping law enforcement with naloxone can be especially important


ner et al., 2016). Fewer studies have examined the efficacy of law enforcement serving as an entry point to treatment. One nonrandomized controlled evaluation of a LEAD program in Seattle, Wash- ington, found favorable outcomes with regard to recidiv


Drug Trafficking Areas (HIDTA) program. The goal of the initiative is to address the basic causes of drug use, rooted in positive childhood development and family influ- ences on decisionmaking. Richards noted that West Virginia has been hit especia


viduals with OUD alive for long enough to eventually get them into treat- ment. As stated by another law enforcement expert, a naloxone deployment functions as a “handshake” with individuals with OUD and helps to build trust and rapport with officers


m. Attendees argued that it is counterproductive for law enforcement to arrest individuals for possession of MAT drugs, such as buprenorphine, because individuals should not be taken off of MAT pre- or postincar- ceration. One expert explained that,


nt these programs. For example, the Deaths Avoided with Nalox- one (DAWN) program was developed by the Ohio Department of Health for the purpose of promoting naloxone distribution among law enforcement officers and training those officers in its use


Matthew Stefanko, Balti- more City Health Department
• RxStat—Applying the NYPD’s Data-Driven Crime Fighting Principles of Compstat to Reduce Drug Over- doses and Save Lives, by Chauncey Parker, New York/ New Jersey HIDTA
• Overdose Fatality Reviews,


 the belief that overdoses are pre- ventable through comprehensive information sharing, proac- tive responses, and strategic focus on identifiable risk. Using
a multiagency, systematic process to examine the underlying dynamics of overdose cases, it


enerally agreed, one participant noted that an exclusive focus on a single indicator can mask other important issues that stakeholders should address. As
an example, this participant explained that naloxone could be considered successful based solely


However, attendees noted that information sharing can sometimes be limited by the Health Information Portability and Accountability Act (HIPAA).
Overall, experts expressed the need to start thinking about novel ways to attack the problem as a system


 (Pagane et al., 1996; Burgess, Barnhart, and Checkoway, 1996; Herbert et al., 2006;
 13
Riediker, 2007). Although dermal transmission from brief
skin contact is not likely to be a significant risk, according to the National Institute for Occupationa


, storage, and eventual disposal. Standardized training, policies, equipment, and oversight practices are critical to safely handling these nar- cotics but, in some cases, are still lacking because of the rapid spike in opioid use since 2013. Additio


appropriate safety and preventative measures are needed. Attendees also commended the brief, easy-to-understand, educational video that was pro- duced by BJA to provide clear and consistent information about officers’ risk of accidental exposure to o


are on the front lines in terms of assess- ing the shifts in drug use trends and their consequences. These substances are identified through samples submitted by law enforcement from drug seizures or through medicolegal death investigations (Morgan,


ed National Forensic Science Technology Center, the Phoenix Police Department was able to conduct a validation project for a field-deployable Raman spectrometer, a device that allows
for the potential identification of opioids, in addition to other n


 As a result, better information sharing is required to allow laboratories to best assess which drugs are currently in communities and which permutations are caus- ing damage that necessitates greater law enforcement or public health interventions.
T


ad show that individuals with OUD prefer to be notified about dangerous substances and will positively adjust drug
use behaviors when they are given such information. Both
law enforcement and public health experts stated that they
are obligated to sh


egies they thought appropriate, split among the action categories in whatever manner they saw fit. The final contents of each card were defined as a “need,” and each need is composed of both the title describing a strategy for mitigating the opioid c


ber of votes they received, we find eight top-tier needs, 15 middle- tier needs, and 30 lower-tier needs. Of the eight top-tier needs, one was in the cooperation and data sharing category, and
the remaining seven were in the overdose response and tre


heir effectiveness. Finally, three of the
13 top-tier needs had more than 20 percent of their votes on the research, replicate, and evaluate results action category. We consider these high-priority needs to be potentially promising strategies, but th


y Category
  Category
Overdose response and treatment promotion
Related High-Priority Need
• Broaden the use of MAT in the general population and increase accessibility. • Promote nonenforcement police outreach to connect individuals to treatment. •


s for Immediate Implementation
Four of the top-tier needs from the in-group clustering were identified for which more than half of their votes had been placed in the disseminate and promote broad application action category. These needs are classifie


 and effects.†
Use data from rapid analysis of seized materials to inform public health and law enforcement interventions.†
7.4
0 10 20 30 40 50 60 70 80 90 100
Percentage
NOTE: All of these needs were in the first tier within their groups. Totals mi


ness among officers about stress and trauma from the crisis. Participants also were concerned that officers who might take advantage of these interventions could experience stigma by other officers or even be penalized by reduced job duties as a resu


that law enforcement support for the strategy would be criti- cal, because police presence might otherwise counteract the harm-reduction approach. They suggested that more education is needed around the harm-reduction approach generally and that add


uch an early warning system could be implemented. They noted that being associated with occupational stress or trauma from policing the opioid crisis could carry stigma and result in workplace consequences. Therefore, participation in the system woul


ation innovations). This need was derived from the discussion fol- lowing the first two panels. This was the only top-tier need
not mentioned in the previous two sections, because approxi- mately 40 percent of votes were in the disseminate and promot


 MAT use, expand access and funding for it, and foster collaborations that can more-effectively direct indi- viduals with OUD to MAT options.
The next common theme among the high-priority needs and throughout the workshop discussion was the need for


ipants used coins to vote for needs. Figure A.1 shows photos of the voting process and Figure A.2 is a mockup of one of the voting sheets.
At the end of the voting, the number of coins was tallied for each category and clustered overall and within ea


ington, 2014. As of July 23, 2019: https://soc.washington.edu/research/reports/seattles-law-enforcement- assisted-diversion-program-lessons-learned-first-two-years
Beeson, Jeff, “ODMAP: A Digital Tool to Track and Analyze Overdoses,” webpage, May 15,


er S. Lonczak, and Seema L. Clifasefi, “Seattle’s Law Enforcement Assisted Diversion (LEAD): Program Effects on Recidivism Outcomes,” Evaluation and Program Planning, Vol. 64, 2017, pp. 49–56.
Community Oriented Policing Services, “Officer Safety and


egn, Michael E. Andrew, and John M. Violanti, “Health Disparities in Police Officers: Comparisons to the U.S. General Population,” International Journal of Emergency Mental Health, Vol. 13, No. 4, 2011, pp. 211–220.
Hartley, Tara A., Khachatur Sarkis


umberg, Don Stablein, Geetha Subramaniam, and John Rotrosen, “Comparative Effectiveness of Extended-Release Naltrexone Versus Buprenorphine-Naloxone for Opioid Relapse Prevention (X:BOT): A Multicentre, Open-Label, Randomised Controlled Trial,” The L


cription Opioids, Bethesda, Md., June 2018b. As of July 23, 2019: https://www.drugabuse.gov/publications/drugfacts/ prescription-opioids
National Institute of Justice, “Improving the Reliability of Drug Tests Done by Officers,” webpage, November 20,


e of Death, Police, Sheriffs, and Health Agencies Must Step Up Their Response, Washington, D.C., September 2017. As of July 23, 2019: https://www.policeforum.org/assets/opioids2017.pdf
Police Executive Research Forum, Building and Sustaining an Offic


https://www.justice.gov/opa/pr/attorney-general-sessions-announces- new-tool-fight-online-drug-trafficking
U.S. Food and Drug Administration, “Information About Medication-Assisted Treatment (MAT),” webpage, February 14, 2019. As of July 23, 2019: ht


rch, research methodology, program evaluation, police use of technology, and national data-collection efforts. Prior to join- ing PERF, Goodison was a law enforcement analyst and civilian researcher for the Washington, D.C., Metropolitan Police Depar


ring together practitioners and subject-matter experts to highlight promising practices and develop a prioritized research agenda for future law enforcement efforts to combat the opioid crisis. This report and the results it presents should be of int


Research Report
Understanding the Economic Benefit Associated with Research and Services at the National Institute for Occupational Safety and Health
An Approach and Three Case Studies
Benjamin M. Miller, David Metz, Troy D. Smith, Jesse Lastunen, Er


Resilience and Environmental Policy
The research reported here was conducted in the RAND Infrastructure Resilience and Environmental Policy program, which performs analyses on urbanization and other stresses. This includes research on infrastructure


.................................................................... 40 Table 3.4. Estimated Annualized Benefits of Avoided Fatal and Nonfatal Cancers for
Firefighters Due to the Adoption of Control Measures over a 60-Year Time Horizon
Using Willingn


arch and in intended users. The first case study examines research to develop, test, and support implementation of engineering control measures to limit exposure to silica among road construction workers and offers an example of NIOSH’s intervention


value associated with research conducted in partnership with industry and labor to develop, test, and support implementation of engineering controls to limit road construction workers’ exposure to silica dust. Based on the willingness-to-pay and VSL


illion to $11 million in new streams of annual productivity gains per year, and almost $700,000 to more than $16 million in avoided uncompensated wage losses per year.
There are limitations to this analysis. First, given limitations in available data


yan and P. Timothy Bushnell, who helped ensure that we had timely access to key documents and stakeholders. We also thank Philip Armour of RAND and John Mendeloff of RAND and the University of Pittsburgh for providing careful reviews of earlier draft


nding the benefits associated with this or any agency’s research activities. The first is that the benefits of research can occur many years in the future. Also, it is often difficult to determine whether NIOSH made a significant contribution to valu


oth. Throughout, we considered suggestions made by NIOSH and incorporated these into the analysis and report, where deemed appropriate. However, we alone developed the methodology and conducted the analysis, and the terms of the contract for this pro


tion, we briefly describe these cases.
Case Study 1: Supporting Development of Silica Controls in Asphalt Pavement Milling
This case study focuses on NIOSH research conducted in partnership with industry and labor to develop, test, and support implem


eletal disorders. The case study examines research that was used to justify an expansion of Ohio’s program and appears to have encouraged other states to adopt the interventions. It provides an illustration of NIOSH conducting intervention research i


ten lacking for many key parameters of the economic benefit models we estimate. Thus, the case study chapters walk the reader through a transparent set of logical steps, marshaling quantitative estimates where possible, and providing transparent disc


matic overall evaluation of NIOSH’s research investments. Additionally, we note that the benefits of research should ideally be evaluated in the context of the cost of producing it. We initially set out to include estimates of costs as part of this p


ries and the effectiveness of these measures in reducing workers’ exposure to silica. Finally, we provide estimates of the economic benefit of the illnesses and deaths likely averted by use of these control measures and consider NIOSH’s potential rol


a partnership that included the National Asphalt Pavement Association (NAPA),8 NIOSH assisted the five largest asphalt paving machine manufacturers in developing prototype fume control measures for their hot-mix asphalt pavers and then independently


pational exposure to respirable crystalline silica, proposing a new PEL of 50 μg/m3.
2014 All major asphalt milling machine manufacturers sign letters of commitment to install dust control measures on half-lane or larger drivable milling machines by


ey disease” (OSHA, 2016b, summary). In response to the NPRM, and as a result of the evidence from NIOSH’s field surveys through the partnership, all manufacturers of asphalt pavement milling machines sold in the United States signed letters of commit


kely that ventilation controls will be installed on the vast majority of cold milling machines several years before 2024.
Approach for Estimating the Economic Benefit of Silica Control Measures
Having described NIOSH activities and documented their l


ion control measures needed to reduce exposure from cold milling machines. Manufacturers would have had to independently develop these controls.
We note that it is likely that awareness of the risks of silica exposure during road milling would have e


very five years, is the most complete official measure of U.S. businesses and the economy.12
Step 2: Estimate the Number of Affected Workers
For this analysis, we start with the entire population of road construction workers using portable or mobile


ers
Total
Affected Workers
1,977 4,943 6,921
    NOTE: The total is not precisely the sum of the two categories because of rounding.
Step 3: Estimate the Reduction in Exposure
We rely on NIOSH research and field testing to estimate baseline exposure


he reduction in fatal and nonfatal illnesses.
Table 2.4 summarizes the estimated number of avoided fatal and nonfatal illnesses associated with reducing exposure levels from baseline to postintervention levels, as described in step 3. For example, th


s according to the gender and age distribution of road construction workers.18 We inflation-adjusted medical costs to 2016 dollars using BLS’s Consumer Price Index for medical care.
To calculate average medical costs for the affected population, we t


tic product (GDP) implicit price deflator.
To construct an average total lifetime cost for lung cancer fatalities, we combined medical costs and productivity losses. For each avoided fatality, we estimate total medical costs, productivity losses asso


s. Our analysis estimates a stream of benefits over time. Benefits will phase in over several decades because health risks depend on cumulative occupational exposure over a 45-year working lifetime based on occupational tenure in road construction. A


 these benefits over 60 years on an annualized basis to estimate the total benefit attributable to NIOSH research activities.27
Summary of Assumptions Made in Estimating Benefits
Given gaps in available data, we made several assumptions in estimating


hat highway, street, and bridge construction workers using half-lane and larger cold milling machines benefit from significantly reduced exposure to respirable crystalline silica due to the adoption of ventilation control measures. With the WTP and V


ple, tend to be significantly larger than avoided medical costs because they are based on individuals’ WTP for small reductions in the probability of dying, while medical costs include actual costs incurred by individuals and, in some cases, third-pa


osure levels is roughly proportional to the reduction in estimated benefits. However, given nonlinearities in the cumulative risk models used in the analysis, this effect could be more pronounced if the differences in measurement were greater.
Third,


ualized basis for fatal lung cancers.29 We do not have sufficient data to monetize benefits for other fatal and nonfatal diseases on a medical cost basis. These approaches are not directly comparable; rather, policymakers can use them to answer diffe


the link between cancer risk and firefighting and the ways in which NIOSH’s efforts have built on earlier efforts in this domain. Next, we present estimates of the effectiveness of the control measures developed in response to the original studies an


r” (TriData Division, 2009, p. vi). However, there was widespread agreement among all parties on the need for more research.
NIOSH Research and Related Activities
In 2010, NIOSH researchers began a multiyear study of career firefighters in Chicago, P


about how firefighters could better protect themselves and what fire departments and equipment manufacturers could do to reduce firefighters’ risk of contracting cancer. National firefighter associations developed and published guidelines and proced


ead trade journals and gave presentations and workshops at conferences (Fent, Horn, and DeCrane, 2015; Fent, Horn, Kirk, et al., 2015; Jackson et al., 2014; Fent, Evans, Couch, et al., 2012; International Association of Fire Fighters [IAFF], 2016).38


nts: Firefighter Case Study Year Event or Activity
2010 NIOSHbeginsthelargeststudyoffirefightersandcancerconductedtodate,includingalmost 30,000 firefighters.
The initial NIOSH dermal PAH exposure study for firefighters begins (resulting in a NIOSH re


in many departments.
Although many fire safety organizations have developed and promoted exposure reduction guidelines to protect their members from an increased risk of cancer, there is currently no precise, quantitative evidence on how widely these


ng and cancer is widely accepted among those in the industry and studies to date have shown an association or correlation between cancer and firefighting, it is difficult to establish direct causation. Nonetheless, for the purposes of analysis, we as


14, and Fent, Eisenberg, Evans, et al., 2013 as a turning point in efforts to prevent cancer in firefighters. Because the NIOSH study (Daniels, Kubale, et
42 This information is based on internal conversations with NIOSH experts in the study of cance


ene wet decontamination by using dish soap (Fent, Alexander, et al., 2017). However, there have been no systematic attempts to estimate the overall effect. Fent, a NIOSH researcher who studies
43 Specifically, the IAFF’s website notes that the associ


based on the implementation of the control measures compared with standard practice in the early 2000s. Specifically, we assumed that, in the early 2000s, firefighters did not regularly wear SCBA during overhaul or outside ventilation activities, di


medical costs and productivity losses of cancer together with the total assumed number of cases reduced to provide monetized estimates of the benefit associated with the NIOSH research. As noted above, we provide estimates for six cancers that were i


ighters are men (see NFPA, undated). We also adjusted initial-diagnosis costs by a factor of 1.2 and last-year-of-life costs by a factor of 1.5 to account for patterns of care that have been reported to be more aggressive for younger cancer patients,


’s WTP to avoid an illness. Given insufficient data on age of onset and survival probabilities, this was not feasible. Thus, for nonfatal illnesses avoided,
48 We derived the productivity loss estimates from the average population estimates in Grosse


90-percent effectiveness of the exposure reduction recommendations and an 85-percent adoption rate for how widely adopted the control measures are.
We made assumptions about job tenure, cancer latency, and the value of future benefits. Our analysis e


ity of 1.44.55 Therefore, for both approaches, we estimated that the value of future benefits would increase by approximately 2 percent annually.
Finally, to estimate the total benefit attributable to NIOSH research activities, we calculated the net


fighters in the main estimation, although NIOSH research on preventing cancer among firefighters also likely benefits them. We present estimates for the entire population in a sensitivity test; however, some caveats are noted in the “Sensitivity Anal


Buccal and pharynx cancers Malignant mesothelioma Esophagus cancer
Cancer of the large intestine Kidney cancer
NOTE: As in the silica case, the results are sensitive to the choice of
estimate for valuing avoided morbidity cases. The total estimated b


23
5 3
1
5 6
3
Sensitivity 3:
1.2 Million Firefighters Affected
14,074 17,807 155 209 248
54 35
11
48 64
34
          NOTE: Annualized benefits using VSL measures are $1.3
$3.5 billion for sensitivity 3. We have rounded these numbers, so totals do no


pter Four. Assessing and Disseminating Impacts of Ohio Safety Intervention Grants
Benefit payments under workers’ compensation programs totaled $62.3 billion nationally in 2014, with medical benefits and wage loss compensation accounting for roughly


ant Case Study
Year Event or Activity
2012 BWCworkswithNIOSHtoevaluatetheeffectivenessofthegrantprogram.
NIOSH, along with others, organizes a workshop, “Use of Workers’ Compensation Data for Occupational Safety and Health.”
2013 BWCquadruplesthesize


0 percent of the employees in Ohio, not all of those who are employed in Ohio.
   46
NIOSH Activities in the Ohio Evaluation Study
From 2009 onward, NIOSH became more heavily involved with BWC, and, in 2010, the two formalized a partnership that incl


e success of funded safety initiatives.57 Grants are available for all policyholders, regardless of premium size.
57 Interview with an MEM official, 2017.
 47
Texas Mutual piloted its own grant program in 2016, awarding around $100,000 of safety gran


safety grants issued per state insurer, along with the total sum of the awards per year since 2012.
Table 4.3. Number of Safety Grants per State Since 2012
      FY
2012
2013
2014
2015
2016
2017
2018
Ohio Safety Intervention Grants
126 491 561 521 47


nesses, and fatalities. We used these steps to estimate three categories of benefits associated with the safety grant programs: avoided workers’ compensation costs, associated productivity gains, and avoided uncompensated wage losses. We did not acco


d with employees affected by the grants. These are also provided in Table 4.4.
Table 4.4. Number of Covered Employees, by State and Year
     State Year
Ohio 2012 Ohio 2013 Ohio 2014 Ohio 2015 Ohio 2016 Missouri 2016 Missouri 2017 Missouri 2018 Texas


ers by 76.2 percent.65 To do this, we used statewide data on the total medical and wage replacement costs, by year, from Baldwin and McLaren, 2016, and McLaren and Baldwin, 2017, in order to capture costs that insurers incur more than 30 months after


f total costs can be significantly higher than 30-month costs. See Wurzelbacher, Meyers, et al., 2013. 52
 FY 2000 to FY 2009 study period was $35,000.67 In other words, we assumed that average total spending per grant by BWC and the employer is half


ants might also be pursuing workplace safety in other ways.
67 Average BWC spending per grant is between $25,000 and $27,000. Assuming that average BWC spending was constant and average employer spending was slightly higher than the match requirement


cement rate. First, we used the statutory wage replacement rate for temporary total disability to estimate the amount of uncompensated wages. If the wage replacement rate is 70 percent, we assumed that total lost wages were indemnity payments divided


ey (that BWC provided) of 147 employers that benefited from safety grants between 2009 and 2011 suggest that keeping workers healthy provides significant productivity benefits to employers, improves production quality, and reduces absenteeism. Overal


efit of the NIOSH research was that, from 2013 to 2017, the Ohio program was larger than it otherwise would have been in our counterfactual, in which the Ohio program did not expand until 2018. Another benefit is that the Missouri and Texas programs


be inspired to create similar programs in response to the NIOSH work. More than in the other cases, the impacts of this work are still developing. Nevertheless, we find evidence that, between 2013 and 2017, NIOSH research has been associated with $4


nificantly larger than medical costs.
In interpreting findings from the case studies, we must be mindful of the fact that the two approaches to economic valuation answer different types of policy questions, with averted medical costs and productivity


ded uncompensated wage losses per year.
The reader should exercise care in interpreting differences among the estimated benefits. First, as noted earlier, the results include a mix of avoided medical costs and productive losses and VSL, which address


ting reductions in injuries, illnesses, and deaths to research activities. Where assumptions were necessary, we explained the logic behind them, and, where possible given the project timeline, we conducted sensitivity analyses to clearly show the rea


orce—firefighters, who engage in fire suppression, outside ventilation, and overhaul, and C/P.72 The sources and patterns of exposure are different for these two groups. There are at least four personnel in the C/P category at a typical fire—includin


who do not follow HI/HO guidelines) because they do not enter burning structures or get very close to the fire. However, they typically do not wear respiratory protection.
   68
Table B.2. Command and Pump Operation Personnel’s Exposure to Carcinogen


-systemic exposure than the deposition or transfer of contaminants to skin. This is the main reason we estimate less reduction in dermal exposure for firefighters than for C/P personnel when implementing the HI/HO recommendations. Although direct upt


g to remove an accumulation of deposits from over the years. Flame retardant contamination at stations has been documented, but this is a particularly persistent type of compound. The
70
importance of this exposure is hard to assess, but low-level, l


arcinogen reduction for firefighters, bringing the total to around 35 to
45 percent. It is difficult to know whether benzene exposure is more toxic than PAH exposure. Also, given all the assumptions made above, a larger CI is prudent. Thus, a 25- to


lman, Jay Colinet, and Gregory Chekan, In- Depth Field Evaluation: Dust-Control Technology for Asphalt Pavement Milling at South Dakota Highway 79 Resurfacing Project, Border States Paving and Industrial Builders, Contractors, Buffalo Gap, South Dako


7, 2013. As of December 1, 2017:
http://www.ohiomfg.com/grip-assets/pdf/ 2013-06-07_lb_swc_sb_op-ed_05-30-13_statewide.pdf
BWC—See Ohio Bureau of Workers’ Compensation.
CDC—See Centers for Disease Control and Prevention.
CDC WONDER—See Centers for Di


c. As of August 10, 2017:
https://www.cdc.gov/niosh/nas/default.html
———, “Construction Program,” last updated January 13, 2017a. As of August 10, 2017: https://www.cdc.gov/niosh/programs/const/
———, National Center for Health Statistics, Health, Uni


iels, R. D., T. L. Kubale, J. H. Yiin, M. M. Dahm, T. R. Hales, D. Baris, S. H. Zahm, J. J. Beaumont, K. M. Waters, and L. E. Pinkerton, “Mortality and Cancer Incidence in a Pooled Cohort of US Firefighters from San Francisco, Chicago and Philadelphi


/2010-0156-3196.pdf
Fent, Kenneth W., Judith Eisenberg, John Snawder, Deborah Sammons, Joachim D. Pleil, Matthew A. Stiegel, Charles Mueller, Gavin P. Horn, and James Dalton, “Systemic Exposure to PAHs and Benzene in Firefighters Suppressing Controll


Health Care Cost Projections for the Medicare Population: Summary of a Workshop, Appendix A: Predicting Medicare Cost Growth, Washington, D.C.: National Academies Press, 2010. As of November 30, 2017: https://www.ncbi.nlm.nih.gov/books/NBK52815/
FSRI


Cincinnati, Ohio: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Institute for Occupational Safety and Health, Division of Applied Research and Technology, Engineering and Physical Hazards Branch R


nes, Hylton J. G., and Gary P. Stein, U.S. Fire Department Profile: 2015, National Fire Protection Association, April 2017. As of September 28, 2017: http://www.nfpa.org/news-and-research/fire-statistics-and-reports/fire-statistics/ the-fire-service/


7: http://monographs.iarc.fr/ENG/Monographs/vol68/index.php
———, “Silica,” in International Agency for Research on Cancer, IARC Monographs on Carcinogenic Risk to Humans, Vol. 68: Silica, Some Silicates, Coal Dust and Para-Aramid Fibrils, Lyon, Franc


er, “A Reference Lottery Metric for Valuing Health,” Management Science, Vol. 42, No. 8, August 1996, pp. 1118–1130.
’t Mannetje, A., K. Steenland, M. Attfield, P. Boffetta, H. Checkoway, N. DeKlerk, and R.-S. Koskela, “Exposure-Response Analysis and


ce
———, Standard on Selection, Care, and Maintenance of Protective Ensembles for Structural Fire Fighting and Proximity Fire Fighting, Standard 1851, 2014.
———, Standard on the Rehabilitation Process for Members During Emergency Operations and Traini


vember 30, 2017:
https://www.bwc.ohio.gov/employer/forms/publications/nlbwc/ SafeHygPubs1.asp?txtCID=516630970
———, “Safety Grants,” undated (b). As of November 30, 2017: https://www.bwc.ohio.gov/employer/programs/safety/EmpGrants.asp
———, “Safety In


d Other Diseases, Washington, D.C.: U.S. Department of Labor, Directorate of Standards and Guidance, Occupational Safety and Health Administration, OSHA-2010-0034-0469, December 7, 2004. As of December 8, 2017: https://www.regulations.gov/document?D=


alth and Human Services (National Institute for Occupational Safety and Health) Publication 2013-147, May 2013. As of November 30, 2017:
http://www.cdc.gov/niosh/docs/2013-147/
Valiante, David J., Donald P. Schill, Kenneth D. Rosenman, and Edward Soc


Alysha R. Meyers, Stephen J. Bertke, Michael P. Lampl, David C. Robins, P. Timothy Bushnell, Ibraheem S. Al-Tarawneh, D. Childress, and J. Turnes, “Comparison of Cost Valuation Methods for Workers Compensation Data,” in David F. Utterback and Teresa


 JUSTICE, INFRASTRUCTURE, AND ENVIRONMENT
Estimating the Global Cost of Cyber Risk
Methodology and Examples
Paul Dreyer, Therese Jones, Kelly Klima, Jenny Oberholtzer, Aaron Strong, Jonathan William Welburn, Zev Winkelman
Sponsored by the William and


onmaking as it relates to science and technology, and the concurrent effects that science and technology have on policy analysis and policy choices. The program covers such topics as space exploration, information and telecommunication technologies,


eedance Probability of Costs for Each Event Type............................................. 45
 vi
Tables
Table S.1. Summary of Case Study Results for Different Models ................................................ ix Table 3.1. Advisen Data Set


e F.2. Sector Variation Elicitation Worksheet....................................................................... 48
 vii
Summary
Cyber incidents have been increasing in frequency and cost in recent years, with some resulting in hundreds of millio


stemic) of $799 billion to $22.5 trillion (1.1 to 32.4 percent of GDP). The purpose of the tool we have developed is to make transparent the underlying assumptions that go into these calculations of the cost of cyber risk.
2 Dreyer, 2018.
  viii
Thes


dam Isles, for reviewing our document and Excel tool.
 x
Abbreviations
CGE computable general equilibrium DDoS distributed denial of service GDP gross domestic product
IP intellectual property
NIST National Institute of Standards and Technology
OECD


explored cyber cost data from more than 550 sources, including journal articles, reports, and websites.3 Some of these sources focused on catastrophic loss scenarios in different economic sectors, and some focused on the global costs of cyber crime.


t costs in the United States and Canada ($225 and $190 per capita) and the lowest in Brazil and India ($79 and $64 per capita). Costs were also far higher for health care ($380) and financial services ($249) than for media ($119), research ($101), an


ledges the considerable uncertainty in the frequencies and costs of cyber incidents. This methodology (1) identifies the value at risk by country and industry sector; (2) computes direct costs by considering multiple financial exposures for each indu


 this model, we first defined the following four sets:
Countries: 𝑐 ∈ 𝐶 Industry sectors: 𝑖 ∈ 𝐼 Economic exposures: 𝑒 ∈ 𝐸 Perils: 𝑝 ∈ 𝑃
That is, we constructed a model with a set of countries 𝐶, industry sectors 𝐼, economic exposures 𝐸, and perils 𝑃.


untry Levels
We calculated direct costs to output and GDP for each sector 𝑖 in country 𝑐 by relating the sets (c, i, e, and p) to 𝐺), the GDP of country c.
First, we define 𝑤), as sector 𝑖’s share of GDP in country 𝑐. Thus, 𝑤), ∗ 𝐺) is the value adde


t model. When a disruption takes place within a firm, the costs are not necessarily contained within the firm, and there are broader supply-chain impacts that could be realized. Our approach allows for the inclusion of upstream impacts within the sup


e input-output model is its computational tractability arising from the linearity assumption regarding production functions.19 Because the supply chain is embedded within the SAM, it is simply a matter of matrix manipulation to obtain the economic im


ion, $B in systemic costs (upstream) in banking, and so on, regardless of whether the damages were caused by a hurricane, a war, a cyber attack, or something else. Thus, when we use direct costs to inform systemic costs, we only need the direct costs


the direct output costs of cyber attacks on sector i, and 𝑧),? be the entry in the ith row and jth column of (𝐼" − 𝐴))CD. Systemic output costs in sector i (𝑠),*) arising from direct costs to output are
𝑠),* =
20 OECD, 2017c.
21 Lloyd’s, 2015.
22 Llo


ted values and probability distributions of outputs.
Sets
Country (C)
Recall that we defined c as a particular country within a set of countries, C. Per the Ponemon Institute reports, the countries and regions currently suffering the most cyber attac


sibly be harmed, independent of hazard type. We consider these exposures, not in their direct contribution to GDP, but as an input to production that when damaged by a cyber attack could reduce a firm’s total revenue. Following the work of Jacobs, Bu


ystemic costs can be renormalized to payments to labor and payments to capital, but the OECD data do not disaggregate these two forms of payment. Because we are not calculating loss of life in the current version, we will not be pursuing the role of


Lockheed Martin.34 The phases of the cyber kill chain include reconnaissance, weaponization, delivery, exploitation, installation, command and control, and actions on objective. Rather than explore actions throughout the kill chain, we focus on unde


 Democratic National Committee attack are widely considered acts of cyber warfare, while the U.S. Office of Personnel Management hack is widely considered an example of espionage.
We emphasize that this approach removes the need to define the point o


 income of utilities and systemic cost in net income of users
Loss of $81 million in assets
Direct costs to Dyn’s net income and systemic costs to net income of customers
Net income losses borne through business interruptions
Utilities
Public
Technol


Specifically, research by Jacobs, Bulters, and van Wieren37 investigates financial sheets from approximately 50 Dutch companies and defines a relationship between financial exposures (e) and company revenue as given in Table 3.3.
Table 3.3. Module Y1


to add transparent rigor to the analysis of potential cyber costs to the economy. While we draw inspiration from the Jacobs, Bulters, and van Wieren38 analysis using income, assets, and IP as inputs, each term is slightly ambiguous. For example, inc


squares regression for each sector. Table 3.4 displays the coefficients, number of observations, and fit for each sector-specific regression, including a regression with all sectors. The regression makes two notable limiting assumptions. First, the r


at R&D has generally weak impacts on revenue, while total assets have the strongest impact. In fact, most intervals on 𝑌),,t&u include the value 0, implying that R&D (and, therefore, IP) may, in some cases, have no impact. In comparison, 𝑌),,v*wxy z{


lthough the model allows entering peril-exposure estimates by country and industry sector, due to limitations in available data, we were unable to distinguish the exposure-peril relationship by industry sector. For the examples shown in this document


is to measure potential economic damage rather than predict the level of damage in any given year. That is, we aim to provide a high-level estimate which, while only met in the extremes, provides insight into the total risk to the economy at large pr


ail
Oil, gas, and chemicals*
Utilities*
Other
       NOTE: Some sectors are not included due to insufficient data.
* Due to an insufficient amount of data on the oil, gas, and chemicals and utilities sectors, bootstrapped values for the "other" secto


cost of cyber crime in the Netherlands, build on the strengths of our model to allow for an improved understanding of annual impacts. Others, such as Lloyd’s business blackout, demonstrate how our model may be applied to specific extreme events. Tabl


, we ran the model for the cost of cyber crime in the Netherlands, which has been estimated to be 1.5 percent of the Netherlands GDP.41 We assumed base inputs. We found the following:
• Model 1: Dutch Peril and Exposure Estimates. Direct GDP losses a


ution of risk. This can be achieved by adjusting the severity and scale inputs to the desired period where 1 = 100% = loss of all output for one year. For example, if the attack resulted in loss of all output for one month and then a complete recover


ervices. A potential stand-in or shortcut for this data- intensive task is to leverage previous studies on sensitivities by industry to particular inputs, such as information and communication technologies use or energy.
Sensitivity to the Choice of


sts?
Users will likely want to update model assumptions to investigate their research question of choice. The examples provided in the model were built with publicly available data, and there are a number of potential improvements to sets and inter-s


iven that utilities are damaged, there is likely damage both upstream (companies that used to sell parts to the sector of utilities but cannot sell as many because the demand is reduced) and downstream (lights are out and other sectors cannot functio


and Generate Sector-Category Map buttons. These buttons will generate new tables for the user to fill on the Peril-Exposure Template, Sector-Exposure Template, and Sector-Category Map worksheets, removing all data from the sheets while doing so. The


Data Element Lists sheet is clicked, it updates both worksheets listed above. The intent is that the user would then make copies of these
38
worksheets as needed and populate them with point estimates and/or probability distributions for analysis. Th


termediate calculations in the GDP and systemic costs estimation. If the GDP by sector table is selected, a pie chart displaying GDP, direct costs, and systemic costs (in diagonal and horizontal hatching, respectively) for each sector is shown. The c


from specifically including cybersecurity controls, insurance, or other mitigation methods in this model. Accordingly, the model can be updated to include them but does not focus on them. To include controls or insurance, a user could alter the secto


 1.00 –0.02
0.05 3.62 0.00
0.09 1.61 0.13
0.03 0.54 0.59
0.03 –1.80 0.07
0.16 –0.16 0.88
  NOTE: Some sectors are not included due to insufficient data.
9 0.99
0.11 0.45
0.24 0.82
43
R&D
  All sectors
Banking
Business and professional services Consum


ed are omitted.
48 See Advisen (2017) for more information about the data set. 45
   Appendix E: Characterizing Attackers and Perils
This appendix provides a taxonomy for characterizing attackers and perils. Potential attackers are categorized by the


versary communications in conjunction with electronic warfare, disruption of adversary civilian Incident Command System– managed systems leading to physical effects
   46
Appendix F: Potential Expert Elicitation Format
We developed an expert elicitat


panies for Data Breaches, 2010. Advisen, “Cyber Loss Dataset,” 2017. As of January 2, 2018:
https://www.advisenltd.com/data/cyber-loss-data/
Association for Financial Professionals, 2015 AFP Payments Fraud and Control Survey: Report
of Survey Results


ss-blackout20150708.pdf
Lloyd’s, Counting the Cost: Cyber Exposure Decoded, 2017. As of January 4, 2018: https://www.lloyds.com/news-and-insight/risk-insight/library/technology/countingthecost
Lockheed Martin, “The Cyber Kill Chain,” 2014. As of Janu


com/content/en/us/enterprise/media/security_response/whitepapers/
ISTR2016_Ransomware_and_Businesses.pdf
Thompson, I., “NotPetya Ransomware Attack Cost Us $300m—Shipping Giant Maersk IT
Crippled So Badly Firm Relied on WhatsApp,” The Register, August


 JUSTICE, INFRASTRUCTURE, AND ENVIRONMENT
Law Enforcement Cyber Center
Final Technical Report
Sasha Romanosky, Karlyn D. Stanley, Jirka Taylor, Zev Winkelman
Prepared for the Bureau of Justice Assistance
For more information on this publication, visi


was conducted in the RAND Justice Policy Program, which spans both criminal and civil justice system issues with such topics as public safety, effective policing, police–community relations, drug policy and enforcement, corrections policy, use of tec


. Link crime units
4. Prevention education
5. Technical assistance
6. Justice Executive Cyber Roundtable
7. Administrative requirements
Design and Build Cyber Center
1a. Development of the LECC website
1b. Continuous efforts to improve user experienc


nce its utility for users. The database grew to contain 1,007 entries, representing, to our knowledge, the most comprehensive resource available for this type of information.
Prevention Education
To enhance cybercrime prevention, the LECC website hos


thank our peer reviewers, Meagan Cahill at the RAND Corporation and Patrick Mays at the Office of the State’s Attorney for Montgomery County, Maryland.
 ix
Abbreviations
BJA Bureau of Justice Assistance
CCDE Computer Crime and Digital Evidence
CJTFG


en the management of the LECC transitioned to the NW3C under a different funding arrangement. Therefore, the last six months of the original grant duration were focused on ensuring a smooth transition of LECC ownership and end-of-grant reporting and


nt a new website framework. Outsourcing the technical aspects of the website development accelerated the creation of the website, as well as its subsequent updates. Second, the project team encountered issues with the process for modifying the websit


this article helpful?” The response options were initially limited to “yes” and “no,” with accompanying thumbs up or down. Subsequently, those who indicated that information was not helpful were also invited to provide additional information, althou


o made use of information available from Google Analytics regarding what content tends to be viewed by which types of users and how they access it.1 The objective of this activity was to ensure that, given the constantly growing content on the LECC w


ups for a comprehensive redesign of the LECC website landing page. The objective of this effort was to make the landing page more user-friendly and help users better navigate the LECC website and the resources available there. Building on internal di


en a continuous growth in traffic. Figure 2.1 shows the growth in monthly sessions and page views by domestic users (i.e., traffic coming from the United States2) at the beginning of the last three semiannual reporting periods.3 It shows that between


where visitors to the LECC website come from. Perhaps unsurprisingly, the majority of visitors came from a relatively small number of large metropolitan areas. These are summarized in Table 3.1, including a breakdown by user type based on an analysi


frequently visited section of the website, followed by, depending on the month in question, content on cybercrime investigations (under the “investigators” tab) or information on digital evidence (under the “officers” tab). The development of traini


relevant information. To identify content for this section, in 2016, the project team began working with two RSS (Rich Site Summary) aggregators—Google and Primal—to select items that may be of interest to the cybercrime law enforcement community. A


and functions. The objective of the cyber alerts was to inform law enforcement personnel on issues pertaining to emerging technology and to provide guidance for investigations, where applicable. The first two NW3C cyber alerts uploaded to the LECC w


organization’s capabilities, particularly those that were not covered by the original search. Users of the website are also offered the opportunity to provide feedback (e.g., if any information displayed is incorrect) and to contribute new content. T


(see Law Enforcement Cyber Center, undated[g]). The Report Card was intended to help police executives and their information technology departments to build and maintain robust information systems by offering a measure for an agency to evaluate its c


ance
The main effort under this task was the development of an instructional document providing information on the creation of a successful cybercrime unit. The model selected for the study was the Utah Department of Public Safety (DPS), also known a


king with the FBI, state information-sharing initiatives, and information sharing with the private sector. The information presented in the
 20
report was intended as a resource for other state and local agencies that are in the process of establishi


t’s working sessions. The topics identified during these preparatory calls related to challenges in the investigation and prosecution of cybercrimes. Specifically, the challenges identified were: (1) ability to recognize all levels of digital evidenc


oncerns and challenges for cyber investigation and prosecution that merit further discussion and consideration. These are summarized in Box 7.1.
Box 7.1. Key Concerns and Challenges Identified at the Justice Executive Cyber Roundtable
     1. The“goi


ment of the LECC website, web analytics, ownership of LECC documentation, and regular reporting requirements. The transition was completed successfully during the reference period.
 24
9. Conclusions and Recommendations
We provide these conclusions a


ment and prosecutors with cybercrime prevention, investigation, and prosecution should continue to broker the exchange of knowledge within and across law enforcement stakeholder groups. The LECC’s
 25
experience with its function as a clearinghouse f


Table A.1. List of LECC Training and Technical Assistances (TTAs)
      Courts, judges, law
St. Louis, enforcement, legislators,
IACP LEIM conference Investigations May 2017 Missouri Federal, state, local prosecutors
       Technology Technology Tec


al justice
 Technology implementation,
Meeting with Utah State Bureau of crime analysis;; technology January
Investigation implementation, cybercrime 2016 Sandy, Utah Local Law enforcement
 Equipment, technology, February Washington,
Task Team Meetin


ciation Section on Science and Technology, Alexandria, Virginia
Terry L. Sult
Chief of Police
Police Department, Hampton, Virginia
Robert Merner
Assistant Chief of Police
Seattle Police Department, Seattle, Washington
 30
Michael Yu
Sergeant
Montgome


 Transnational Cyber Investigations: Michael Stawasz, Brendan McHugh, and Anne Beck Tokoph
• Panelists provide an overview of current case law and processes
for obtaining overseas data, and the impact on state law
enforcement investigation and prosec


 27, 2017:
http://www.iacpcybercenter.org/news/lecc-cyber-report-card/
Law Enforcement Cyber Center, “Strategic Partnerships,” webpage, undated(g). As of December 27, 2017:
http://www.iacpcybercenter.org/about-the-cyber-center/partners/
Law Enforceme


  CORPORATION
Looking to the Future
of the Department of
Immigration and Border
Protection (DIBP)
Assessment of the Consolidation of the Australian Customs and Border Protection Service (ACBPS) and the DIBP (2016–2017)
 Daniel M. Gerstein, Karen Edwa


 analysis. As such, it does not endeavour to repli- cate the previous effort, but rather commences from that starting point.
Relevant documents and interviews with senior leaders from across the department, including the ABF, were undertaken. While d


agement areas (ECM), RAND identified shortfalls. In summary, many of the goals of integration and reform remain a work in progress. In these ECM areas, it will require continued progress and leadership attention to meet the goals set out in the origi


................................... 44
5.1. Organisation Design Theory as Applied to U.S. Department of Defense...........61
5.2. Assessment of DHS Progress in Addressing the Strengthening
DHS Management Functions High-Risk Area, as of March 2016 ...


nce the publication of the previous study.
This initial analysis required RAND to take a deeper and longer view of the two organisations as they existed, understand how the combined organisation could incor- porate into a single entity, and assess th


sisted in developing lasting institutions and building the foundations for the twenty-first-century border management capacity envisioned in the integration plan. Examples are in areas such as resources for internal capability growth through research


c Findings
Follow-On Analysis of Key Operational Metrics
The integrated DIBP has improved the effectiveness and efficiency of its support for the government of Australia in the customs, immigration, and border protection mission area during the perio


held dual citizenship and were subsequently released. On a positive note, the better use of intelligence was important to rapidly resolving these cases. However, the changing demographics of the detention population and officer training deficiencies


on-ABF and part of the existing workforce reportedly are disenfranchised by the per- ceived lack of opportunity.
Specific Findings: Lessons Learned and Looking Toward the Future
An institutional predisposition toward POI versus ECM reform enabled the


the department has been laudable in some areas and lacking in others. Continued efforts to mature and, in some cases, build the organisation must continue. While the POI areas have seen important growth, the ECM sides of DIBP have not seen the same o


ble future direction for concentration DIBP efforts.
The second phase entailed a follow-up to the original RAND study. Specifically, the follow-up effort built on the first study, which examined performance during the period 14 May 2014 to 14 May 201


from which determinations could be made about the state of working toward the goals and objec- tives of integration and reform. As an example, the DIBP Corporate Plan 2016–2017 provided a basis for understanding the department’s broader outcomes, go


on the same data but are assessing two different issues.
The RAND team’s assessment was designed to examine how the integrated organ- isation has been performing in the areas of operational effectiveness and efficiency, building capabilities, and cul


indicators of the effects on culture of DIBP and personnel programs. Capability examines whether programs and processes designed to build organisational capacity have been devel- oped. Action plan for reorganisation looks at the planning done before,


back under a first assistant secre- tary and efforts have once again slowed, particularly in ECM areas where progress has lagged. With the establishment of HA, deputy secretary leadership has again been established through the creation of the Home Af


ng to the Future of the Department of Immigration and Border Protection
Table 2.1—Continued
 Category
Action Plan for Integration and Reform
Monitoring Progress
Description Assessment
 Specific Findings
Examination of the reorganisation planning for


 are expected later in the reorganisation.
A total of 168 operational program performance measures were provided by the DIBP for consideration; however, either no data or insufficient data was available for 90 metrics, as the data collection methods


ictoria, 18 August 2017. As of 7 December 2017: http://newsroom.border.gov.au/releases/abf-continues-to -smash-tobacco-syndicates-in-victoria
5 Interview with senior DIBP regional official on August 24, 2017, in telephone interview, Australia. (19)


ficulties with clarity on delegations and authorities were determined to have an operational impact, introducing ambiguities into roles and obli- gations and unhelpful friction into the operations. An example is in detention and case resolutions, whe


andards and professionalise the workforce.
6 Telephone conversation with DIBP regional official on August 24, 2017, Australia. (19)
 12 Looking to the Future of the Department of Immigration and Border Protection
The area of L&D, particularly in rela


7.
8 Interview with senior DIBP official on August 22, 2017, in Belconnen, Australian Capital Territory. (16)
9 Interview with senior DIBP official on August 22, 2017, in Belconnen, Australian Capital Territory. (16)
 Follow-On Operational Assessment


h a senior DIBP official on August 25, 2017, in Belconnen, Australian Capital Territory. (22)
11 Interview with senior DIBP official on August 22, 2017, in Belconnen, Australian Capital Territory. (16)
12 Interview with senior DIBP official on August


prototypes or, at best, as the early stages of incorporation within DIBP. The Transformation Delivery Framework and Blueprint provides the documentation and platforms for tracking and building capacity. These tools had been under development, were e


lligence and investigations.22
16 A definition of contestability from the Defence website is: “Contestability is a decision support function not a decision-making function. Contestability Division will contest programs and projects across the Capabil


press support for further consolidations for ECM areas. Doing so would provide opportunity for greater focus on core operational elements of the mission.
Implementation of the Integration and Reform Elements
Action Plan
In the initial study, findings


ropriately covered. Examples include international engagement and corporate records management.26 Another example is the development of a contestability function for the DIBP, which, to date, remains a recognised shortfall needing to be corrected.27


on and reform to the deputy secretary level was part of this effort to regain momentum.
During the 2016–2017 period, the Transformation Delivery Framework and Blue- print was developed, which should provide a tool and forum for monitoring progress. H


d issues with span of control.
Data management and lack of metrics continues to be a source of concern. With the Transformation Delivery Framework and Blueprint now underway, these issues could be mitigated, but progress will need to be monitored clo


in those areas where progress has been the greatest and as lessons to improve upon where progress has been lagging.
CHAPTER THREE
Follow-On Operational Assessment: Examination of the Five Key Areas
 Introduction
Five key areas were identified for fur


laid out as part of the integration.
Several important initiatives were underway at the time of the initial RAND study. Based on a 2016 Integrated Intelligence Capability Review (Classified), 107 intelligence recommendations were identified, with man


e efforts, a Border Intelligence Fusion Centre (BIFC) was devel- oped in June 2016 to assist with threat identification across the border continuum. The centre includes intelligence gathering and targeting functions, including those previ- ously deli


raining, educating, and developing the experience of the individual officer is accomplished; preparing focused intelligence products with the operational consumer in mind becomes commonplace; and forward- looking intelligence products are prepared. O


ation of the Five Key Areas 25
attempting to manually analyse and correlate the inputs. Investments would be benefi- cial to support these data-heavy processes and would undoubtedly require specialised hardware, software, and analysts to operate the


ctivities Need for standardised training curricula and career paths
Lack of centralised access to intelligence information
No strategic intelligence function in either organisation.
• Customs had already started to build the capacity to support a nat


nto transnational crime syndicates.
The Joint Organised Crime Group investigations into high-level organised crime syndicates, such as the Jomaa syndicate, represent a major investigation success that was significantly undermined by the involvement o


s and prioritisation are made on a case-by-case basis. This prioritisation then sup- ports the selection of the cases to be investigated. This process is in its early stages and is continuing to be refined. The overarching intent is to have a departm


y focused on correcting this issue was available in October 2017.16
The increased capability to conduct external investigations has been further strengthened by the development of human intelligence capabilities feeding investiga- tions and by combin


 30 Looking to the Future of the Department of Immigration and Border Protection
Table 3.2
Investigations Assessment
Pre-Integration (Pre-2015)
• Recognised shortfalls across several areas, including internal and external investigations
• Lacked emp


amination of the Five Key Areas 31
higher-risk detention population. This reflects improvements in risk management and staff professionalism that had been lacking in the former organisation.
Also noteworthy is the development of documentation for man


contains information on the Child Protection Panel terms of reference and report. As of 12 December 2017: https://www.border.gov.au/about/reports-publications/reviews-inquiries /child-protection-panel-terms-of-reference
24 Interview with senior DIBP


 (2)
26 Australian Department of Immigration and Border Protection, Immigration Detention and Community Statistics Summary, 31 May 2017. As of 7 December 2017: http://www.border.gov.au/ReportsandPublications/Documents /statistics/immigration-detentio


rations (including for contract staff). Despite improvements, several officials commented on the slow pace of development and adoption of policies, the need to refine roles and responsibilities, and the importance of better managing interactions and


n not paying sufficient attention to detention operations.35
Regional authorities expressed similar sentiments regarding accountability in the detention space. Handoffs between the DIBP and the ABF were identified as unneces- sarily hindering case re


rough the detention capability report.
• While in early stages, these efforts should improve the department’s operational capacity across the DIBP mission space to include within the detention area.
• Progress noted in developing guidance/documentati


consequences of lapses in integrity and anti-corruption were foundational elements of the integration and reform efforts.
Updates from 2016–2017
Corruption and integrity issues have been a significant challenge and remain one of the highest prioritie


or leader provided an example of the progress, highlighting a recent trip to Adelaide during which the workforce expressed support for the recent arrest of one of their own, stating they were “pleased that the workforce is clean,” and that there is “


 and 2011–2012) consistently highlighted the importance of integrity.
• The ACBPS Review in June 2013 highlighted the broad need for leadership and workforce reform. Specific language included such areas as “leadership behaviours, responsibilities an


an overview of the findings for the pre-integration, post- integration, and reform periods. The pre-integration and post-integration sections have been taken from the earlier report.
Learning and Development
Initial Findings
In the initial planning,


ividual highlighted that corporate L&D is ade- quate, but strongly assessed that the ABF College roles must be clarified.
The ABF College is responsible and funded for all non-leadership and non-APS core skills–related training for ABF and certain de


ard operations due to a lack of proper training and experience.53 Such skills are part of an individual’s core responsibilities and, therefore, must be mastered, beginning with a firm educational foundation.
Several issues serve to highlight the lack


College.
In some cases, the ABF College curriculum was not seen as being focused on the highest-priority areas. One senior ABF interviewee lamented that “use of force” train- ing needed to be incorporated into the curriculum.58 Another commented that


mination of the Five Key Areas 43
Regarding the self-service online training products developed by DIBP corporate, these have been described as “adequate, but not inspiring.” The courses have been described as repetitive and not updated from year to


al official on August 24, 2017, Australia. (20)
 Table 3.5
Learning and Development Assessment
Pre-Integration (Pre-2015)
• Shortfalls had been observed throughout the documents, beginning with the earliest findings dealing with the Alvarez and Rau a


• Individuals characterised as non-ABF and part of the existing workforce reportedly are disenfranchised by the perceived lack of L&D opportunity.
• Teaching methods and the use of technology have received significant criticism.
• The Administrative


utiny as the rest of the department.
While a broader study effort would be required to fully examine these issues, the perception is there and would benefit from leadership attention.
The Pace of Change
Senior leaders stated that the pace of change h


ogress but also high- light that it has been uneven across the areas.
Figure 3.1 provides a graphic depiction of the overall assessment of the five areas based on a synthesis of information provided, collected, analysed, and discussed in this report.


te, leadership, and APS core skills training materials
 Limited improvement Making progress Goals achieved
 L&D I&C Inv Det Intel
   48 Looking to the Future of the Department of Immigration and Border Protection
through the DIBP Learning and Develop


ased on analysis of data and key documents, interviews with senior staff over the course of the two study efforts, and assessments and observations made by the RAND team.
In this chapter, only the most substantial lessons are provided with an eye tow


uous assessment and improvement is essential for high- performing organisations.
– Even in functions than have been assessed to have largely met the goals of inte-
gration and reform, a process of continuous improvement must be in place to
Lessons Le


d cor- rectly or were omitted in the delegations.
– Over the last year, attention has turned to identifying these issues and correct-
ing the shortfalls. A related issue concerns horizontal alignment across other government departments and agencies w


des horizontal
integration between DIBP elements.
• Where successes in integration and reform have been noted, understanding what
contributed to those successes and replicating those efforts would be prudent.
– For example, understanding what contrib


ween offices could be problematic, as it requires synchronisation and coordination above the FAS level.
Conclusions
The lessons learned in this chapter demonstrate the progress made and highlight areas where improvements could be introduced. The degr


uding lessons learned resulting from these transformations: http://www.anao.gov.au/work/performance-audit /management-machinery-government-changes
 55
56 Looking to the Future of the Department of Immigration and Border Protection
organisations have


nd-to-end capability development function within the Department to maximise the efficient, effective and professional delivery of military capability
3. Fully implement an enterprise approach to the delivery of corporate and military enabling service


tained in the First Principles Review of Defence highlights the impor- tant cultural and workforce changes that were planned. Clearly the effort was designed to build capability, with emphasis on enterprise management issues. Regarding the action pla


gned to those commands
• to increase attention to strategy formulation and contingency planning
• to provide for the more efficient use of defense resources
• to improve joint officer management policies
• otherwise to enhance the effectiveness of mi


s, Army, Navy, Air Force, Marines) also has a transfor- mation office. These transformation efforts seek to gain effectiveness and efficiency in the operational forces and support activities.
The point of recounting these U.S. DoD efforts is to highl


ts?
What are the activities DOD engages in that enable it to make its unique contribution to national security? What are the functions or areas in which the department must build and maintain excellence? Are there functions or tasks that are better s


eak central staff, and failing to deconflict the roles and authorities of the organisations.14
Even today, tensions exist where roles and authorities are unclear and, in some cases, overlapping. As shortfalls have been identified, efforts to rational


 and rally around.
3. Focus on a key set of principles and priorities at the outset of the transformation. A clear set of principles and priorities serve as a frame- work to help the organisation create a new culture and drive employee behaviors.
4.


ge. This entails involving all levels of the organisation, from the most senior leaders to the lowest levels of the organisation. It also speaks to the importance of communicating key messages throughout the organisation and gaining buy-in for change


 Act of 2004 was the most significant legislation affecting the U.S. intelligence community since the National Security Act of 1947 and created the Office of the Director of National Intelligence (ODNI) in 2005.19
The overarching goal of the reform e


ate elements are not “adequately organised and did not have resources appropriately allocated to fulfill its missions”
• Others have indicated that the Intelligence Reform Act has not gone far enough in reforming the IC and that “recalcitrant ele- me


ons and surveillance at the external borders.”23 To achieve this goal, FRONTEX supported security at borders and facilitated legitimate flows of people, goods, and services. Its primary mechanism was to support coordination on border management effor


arding border management, they decided against a more centralised structure with provisions such as the right to intervene. The provision was seen to be an “unrealistic idea that drew objections from frontline states because it was seen as a violatio


he GAO framework have been introduced, including effectiveness and efficiency and culture and personnel. Indications are that action plans and monitoring progress are being considered, but are seen more as national requirements and far less central t


cript
28 GOV.UK, Single Departmental Plan 2015 to 2020.
 Lessons Learned from Other Transformation Efforts 71
In turning toward the reform of the Home Office, May identified five principles that should guide the reform:
The right institutions and str


 to reducing its operating costs over the Parliament, while continuing to improve the efficiency and effectiveness of its services, including through:
• increased use of automated data analytics to better identify risks, allowing resources to be targ


ing with SMEs by 2020
• working in partnership with the Cabinet Office to deliver Arms Length Bodies’ transformation plans, and the Infrastructure and Projects Authority on major projects and programs and prioritisation.
SOURCE: UK Home Office websit


The timelines for change in the integration of the DIBP were overly aggressive and the speed of transition hindered taking a more measured view of the roles, missions, and functions of the elements of the newly formed department.
• Excessive personne


n period and resulted in a loss of momentum. Today, it hinders the DIBP from assessing progress on its path forward for developing the twenty-first-century border management capacity for Australia.
Personnel issues continue to serve as unnecessary so


assets identified to become part of HA, other government and interna- tional stakeholders that currently interface with future HA elements, and interested external parties.
Another major concern expressed by senior officials related to the aggressive


applicable national strategies; and the assessment of capability devel- opment requirements and associated resourcing strategies.3
The degree to which ECM functions will be brought up to HA-level as shared services has not been fully determined. As s


nlike in the DIBP, where one of the initial goals was to build a single culture, most expressed that a single culture should not be the stated outcome, but rather it would be more important to find ways to embrace the different cultures of the organi


ty.
• The development of HA cannot be a “pick-up” game. It will require first-class
leadership and execution at all levels and important follow-through.
• Key documents such as business plans must be developed and disseminated early in the lifecycle


e totals have gone from approximately 1.2 million to 1.5 million per month, a 20 per cent increase. These increases translate to transactions (that is, additional work to be performed) in support of arrivals and departures. Regarding imports, the val


blishment will require significant deconfliction between the agencies being integrated. Determining design models—centralised versus decentralised, shared services for key functions, or a more limited hub and spoke for key support functions such as i


land Michael Milford
POSITION
Secretary
Acting Commissioner (ABF)
Acting Deputy Secretary Policy Group (DIBP)
Acting Deputy Secretary Visa and Citizenship Services Group (DIBP)
Deputy Commissioner Support (ABF)
Deputy Secretary Intelligence and Capab


ome Affairs.
Questions:
1. Biographical Information a. Name?
b. What is/was your position(s) in the government? c. How long have/did you serve?
d. What are/were your dates of service?
2. PreIntegration (2014–2015)
a. How was your organisation functi


ersonnel and goods has increased in recent years. Each of these movements creates an increased workload and requires a decision, verification, and potentially an inspection by DIBP organisations and personnel.
The overseas arrivals and departures (Fi


constant (Number of Visa Cancellations), and the other is decreasing (Illegal Worker Warning Notices Issued) (see Table C.1).
For the border infractions related to illegal weapons and drugs, the trends are also mixed, with the number of undeclared fi


ous and inclusive society, and advance Australia’s economic interests through the effective management of the visa and citizenship programs and provision of refugee and humanitarian assistance.
Data associated with this outcome reflects little change


7
100 Looking to the Future of the Department of Immigration and Border Protection
Table C.2
Categories of Border Revenue (AUS Dollars)
       Category
DIBP
Taxation Revenue
2015–2016 2016–2017 2017–2018
17,322,357 17,635,995 18,579,332
17,259,053 17


tizenship Management Deputy Secretary Policy Group
Finance Division
Major Capability
Digital Transformation and Channels Strategic Policy
Corporate Services
Traveller Customs and Industry Policy Integrity Security and Assurance
People Division
SOURCE


le C.3 provides detailed overviews of the gain and loss for large organisational elements.1
1 Large was defined as those organisations larger than 100 staff over the period of analysis.
 Abbreviations
 Australian Border Force ABF Australian Customs a


ust 2017: https://www.border.gov.au/ReportsandPublications/Documents/reviews-and-inquiries/Capability -Review-Action-Plan-Web.pdf
———, DIBP Corporate Plan 2015–2019, Canberra: Commonwealth of Australia, 2015. As of 14 August 2017:
https://w w w.borde


. As of 7 December 2017: http://www.defence.gov.au/publications/reviews/firstprinciples/Docs/FirstPrinciplesReviewB.pdf ———, L&D Operating Structure, Canberra, undated.
———, Leadership and APS Core Skills Learning Curriculum 2016–2017, Canberra, unda


 Report 2013–2014, Canberra, 2014. As of 14 August 2017: http://www.border.gov.au/ReportsandPublications/Documents/annual-reports/DIBP_AR_2013-14 .pdf
———, DIBP Annual Report 2014–2015, Canberra: Commonwealth of Australia, 2015.
———, DIBP Annual Repo


shboard 20160901, Canberra, 1 September 2016.
———, Minister’s Portfolio Dashboard 20161006, Canberra, 6 October 2016.
———, Minister’s Portfolio Dashboard 20161103, Canberra, 3 November 2016.
———, Minister’s Portfolio Dashboard 20161201, Canberra, 1 D


ww.aph.gov.au/About_Parliament/Parliamentary_Departments/Parliamentary_Library/pubs /rp/rp1718/Quick_Guides/HomeAffairs
Comrie, Neil, Inquiry into the Circumstances of the Vivian Alvarez Matter, Canberra: Commonwealth Ombudsman, September 2005. As of


 Monica, Calif.: RAND Corporation, RR-1713-AUS, 2016. As of December 22, 2017: https://www.rand.org/pubs/research_reports/RR1713.html
GOV.UK, Single Departmental Plan 2015 to 2020, 19 February 2016. As of 7 December 2017: http://www.gov.uk/government


erra, 9 June 2017.
U.S. Government Accountability Office, Department of Homeland Security: Important Progress Made, but More Work Remains to Strengthen Acquisition Management Functions, 16 February 2017. As of
17 January 2018:
https://w w w.gao.gov/p


 Addressing Emerging Trends
to Support the Future of
Criminal Justice
Findings of the Criminal Justice Technology Forecasting Group
John S. Hollywood, Dulani Woods, Andrew Lauland, Brian A. Jackson, Richard Silberglitt
 CORPORATION
For more informati


s, foundations, and the private sector.
This program is part of RAND Justice, Infrastructure, and Environment, a division of the RAND Corporation dedicated to improving policy- and decisionmaking in a wide range of policy domains, including civil and


omplete Notes and Mind Map. . . . . . . . . . . . . . . . . . online only
E. Charters for Global/Criminal Justice Technology Forecasting Group Task
Teams on Key Information Exchanges to External Service Providers. . . . . . . . . . . . . . . online o


 41 42 46
D.1. Emerging Themes and Constituent Trends from the Criminal Justice TechnologyForecastingGroup....................................................... onlineonly
Tables
S.1. Themes, Trends, and Ways Ahead from the Criminal Justice Technolo


sed bail and sentencing support; and tools to identify services needed by those on community supervision. The emergence of situational-awareness displays in improving form factors (e.g., tablets, smartphones) offers opportunities to get needed infor-


ppendix F, available online).
• Need to define and dis- seminate the value of fusion centers
• Lack of established business processes for operationalizing new technologies
• Emergence of analytics and enabling big data
• Emergence of • situational-aw


rity, privacy, and civil-rights planning and protections into the business cases and processes recommended in Table S.1.
There is a need to advance from small information-sharing pilots to field-wide inte- gration of information. Group members discus


lling business cases, a lack of established processes for implementing technologies, a lack of nationwide shar- ing capabilities to get needed information, and problems with ensuring security, privacy, and civil-rights protections. These issues repre


reats, which are most likely to affect criminal-justice practice significantly in the next three to five years. The CJTFG was tasked to (1) identify major, emerg- ing social and technology trends; (2) assess the impacts that these trends could have c


ling the future of information-sharing. Panelists also discussed and rank-ordered technologies and technology issues by how much impact they would likely have on criminal justice in the next three to five years (i.e., 2015 to 2020).
Meeting 2’s break


a and Civil Rights, 2015); and the graduated-reentry concept for reducing incarceration rates.
Our approach to capturing the deliberations was based on the Futures Wheel concept (Glenn, 1972), which captures an initial trend, potential direct (first-


eing addressed elsewhere
 Response A Response B
 At the end of meeting 3, we had captured 22 trends and associated impacts, potential responses, and relationships among them. Appendix D, available online, provides detailed notes from the meeting; the


 problems and oppor- tunities recognized in the individual trends
• nonredundant, in terms of not being addressed through other major initiatives.
Meeting 4 (2016), a conference call, was a structured review of the analyses described above and the dr


erging from CJTFG
Emergence of situational-awareness
data streams
data and analytics and the challenges Emergence of big of using them
Need to advance from small-scale to nationwide information-sharing
process, security, abound but are privacy, and c


cy, and civil-rights lines, especially for surveillance-related technologies.
Business Cases and Business Processes for Technologies Are Lacking
This theme brings together the lack of core business cases for new technologies with the lack of core bus


sonal data. Civil-rights protections refers to safeguards against technologies being used for discriminatory purposes.
Discussion and Findings 7
 8 Addressing Emerging Trends to Support the Future of Criminal Justice
Senate, 2012]). Panelists noted t


ices they need (e.g., Casey et al., 2014).
Panelists noted that a smart integration of technology, data, analytics, and good, community-based practices at a pilot site could provide an example of what good policing could look like. The panel also dis


physical burden of carrying too many devices and the cognitive burden of being overloaded with too much information, and a lack of in-house information and analytical capabilities needed to share, process, and integrate all of the data needed to pop


evoted substantial time to cybersecurity in its meetings. On the positive side, the CJTFG heard presentations on increased federal support for cyber, including increased Federal Bureau of Investigation (FBI) support for states’ cyber investigation an


 the public expects product advertising and television show levels of performance (the so-called CSI effect) resulting from new tech- nologies; expectations are often highly unrealistic.
This is one of the most central trends, complicating every surv


nt), combined with a major barrier (have versus have-not agencies).
There are needs for information integration to enable new models of criminal jus- tice across the enterprise. To maximize the potential from emerging technologies that pro- vide more


rs discussed. Panel- ists noted a strong need to develop systems and business models that permit “have” agencies to share information and capabilities with have-not agencies in the same region.
There are needs to enable digital-evidence management on


d terrorism—and these pressures can vary enormously from day to day. In the post-Ferguson and -Baltimore environments, demands to demilita- rize and “be less violent” appear persistent. Similarly, demands to be more community-centric appear persisten


ders) and desires to make law-enforcement operations transparent. The lack of settled law and case law on surveillance-related technologies is a major barrier here.
Agencies need to further develop and disseminate less-lethal weapons. As back- ground


ld lack training, often being the best
real-world alternative to using deadly force. As a result, panelists agreed that new development, testing, and fielding of less-lethal devices was needed.
Outside of device development, the CJTFG discussed that


 common policy challenges around emerging surveillance technologies.
Technologies to detect firearms and other weapons remotely might be emerging.
Panelists noted pilot efforts to use millimeter-wave technology to detect weapons from stand- off dista


value of fusion centers
Lack of established business processes for operationalizing new technologies
Legend
Most central trend Central trend Other trend
  Develop core business cases for key technologies, starting with exchanges to external service p


cycle
• integrationandinteroperabilityrequirements
• requirements for community and external expert participation
• cyber, civil, and privacy rights protections, along with other policy provisions, building
on the IACP Technology Policy Framework (IA


unknown and often not well suited operationally to criminal-justice practice practitioners. As a few exceptions, panelists noted that they knew of agencies learning about technologies from the IACP’s Police Chief magazine and the Police Executive Re