anticipate and prepare for change.
 1 Transit refers to all modes of intracity public transportation, including heavy rail, light rail, and bus service.
 ix
 Methodology
To develop alternative scenarios of the future of mobility, we applied a process


, even as more and more cities adopted constraints on driving to try to address growing problems of parking and congestion. Long-distance travel increased as well, even with oil prices of USD 150 per barrel. China maintained its position as the world


In scenario planning, wild cards provoke thinking about events that break with trends and constitute paths that differ from the projections that underpin the formally developed scenarios. In this report, the wild card is based on the possibility that


of Michigan State University; Abhas Jha of the World Bank; Ziqi Song of Utah State University; and Loraine A. West of the U.S. Census Bureau. Our economic experts were Patrick Chovanec of Silvercrest Asset Management Group; Damien Ma of the Paulson I


in travel has also contributed to the country’s serious air-pollution problems, yet the government’s stated plans to increase the use of electric vehicles (EVs) have fallen far short of expectations.
How these varying trends will play out over the ne


l prices have a substantial effect on the amount of driving because drivers are sensitive to the price of gasoline. However, other factors that are taking place simultaneously, such as investments in public transit systems or an economic downturn, mi


work. Ideally, this report will become one of a series of reports that can be likewise updated or that use similar methodologies for other countries.
Creating the Scenarios
Our methodological approach represents a state-of-the-art scenario process wh


ors in each influencing area; Chapter Two summarizes these papers.
Elicit Projections on Descriptors
The research team held one workshop for each influencing area to gather expert opinion on projections for each descriptor. Two workshops (those on de


d group descriptors and projections into distinct scenario frameworks (for a more-detailed discussion of these analytical tools and how they compare with other analytic methodologies, see Amer, Daim, and Jetter, 2013). We first used cross-impact anal


rd. We included the fourth cluster specifically because it differed so greatly from the other three. For example, it had a lower average gross domestic product (GDP) growth rate. We thought it was important to include different growth rates because,


sible ideas for wild cards were put forward, we selected one in which China experiences a fairly severe financial crisis with long-reaching implications. The economic experts were in agreement that some type of crisis is likely but that the course of


at could lead to one scenario versus the other: economic growth, constraints on cars and driving, and environmental concerns. Economic growth and environmental concerns depend on various determinants outside transportation. In addition, how our scena


been driven by rural-to-urban migration as the country industrialized, we also included commute distance and household type. These capture the demographic dimensions of factory work, in which many workers live in dormitories colocated with their plac


ulation between the four regions did not shift significantly during the 20-year period. The eastern region contained 34 percent of total population in 1990 and 38 percent in 2010. Each of the others saw its percentage of the overall population fall b


 9.7 km (6.0 miles) in 2009; in 2009, the average car trip was just over 15 km (9.3 miles) (Zhu, 2012; Lu and Gu, 2011). Beijing’s rail transit trip length more than doubled from
8 km (just less than 5 miles) in 2000 to 16.4 km (10.2 miles) in 2011 (


in the remaining years, with a sample size of less than 1 percent (National Bureau of Statistics of China, 1997–2000, 2004–2005, 2007–2010, 2012).
  17
Economic Growth and Regional Economic Shares
We define economic growth as change in GDP. China’s G


9 in 2009 (Sicular, 2013).
 18 THE FUTURE OF MOBILITY _ PAST TREndS In InFLUEncIng AREAS
Labor-force participation is defined as the ratio of working adults to all adults, with adult defined as people ages 15–64. In China, this ratio has declined fro


a because the cost of gasoline and the availability and cost of alternatives affect both the number of miles and the types of vehicles people drive. This section includes background on oil price and oil consumption, the possibility of legislation or


ported is consumed in country. One observer notes that imported oil might be entering a stockpile or exported as refined petroleum products (Russell, 2013).
  20 THE FUTURE OF MOBILITY _ PAST TREndS In InFLUEncIng AREAS
Introduction of Effective Gree


be set to CNY 10 (USD 1.61) per ton of carbon dioxide and then gradually increased over time (Liu, 2013).
 21
Adoption of Alternatively Fueled Vehicles
China has adopted aggressive targets for producing electric and hybrid vehicles, but it seems unli


initively. E-bikes are less common than e-scooters; according to one survey, in 2010, only 16 percent of E-2Ws in the market were e-bikes; 26 percent were e-scooters; and 58 percent were “something in between” (Ruan et al., 2012).
 14 From the origin


ntrol vehicle ownership through limiting the number of license plates available, either through lottery or auction. Those cities with auctions (including Shanghai and Guangzhou) charge an additional fee as well (Yu, 2013; “Shanghai Car Plate Auction


 residential areas can cost as much as CNY 100,000 (USD 16,150), which is about the cost of a car (Taiyuan Daily, 2014).
Taxis are already in widespread use, although many taxis operate illegally (in Beijing, for example, illegal taxis outnumber the


rrent)
SLOwIng BUT gROwIng
   Total population
Geographic distribution of population
1.44 billion (annual growth of 0.34%)
41% of the population lives in the eastern region.
1.39 billion (annual growth of 0.18%)
41% of the population lives in the eas


ffordable fares, and the quality of the vehicles. We used convenience instead of demand because we were interested in the factors that might encourage people to ride transit, not demand itself.
b Black taxis are taxis that operate illegally.


ould occur as a result of a successful shift to consumption-driven growth, a reduction in corruption, reform of the hukou system, or policies to allow more competition among companies (as opposed to propping up state-owned enterprises). On the other


did not include in our initial set of descriptors but rather identified in the process of writing the scenario narratives. Participants at several of the workshops discussed the impact of environmental conditions intensively as a reason for certain


r the economy toward more consumption-based growth, which tends to require more urban consumers. China is now about where most developed countries stand with regard to urbanization; geographically large countries tend to top out at about 70 to 80 per


ame from public demands for environmental cleanup, a realization that degraded environmental conditions were affecting quality of life and thus the party’s legitimacy, a change in the attitudes of the business community, and a desire to be viewed as


risis in 2008 marked a defining moment. To avoid an economic downturn caused by slowing demand for exports, China adopted a roughly USD 585 billion stimulus package (Tsuruoka, 2014), which was largely fueled by borrowing.
Although the overall level o


hold. In Shenzhen, for example, the ratio of the average house price to average income was almost 18 to 1 (Yi and Tan, 2013), with any ratio over 5 to 1 considered unaffordable. Although the falling prices in 2014 (Shao and Yao, 2014) might have help


d for more-aggressive protection of intellectual property rights— the laws had been on books, but now they were enforced far more vigorously—and this allowed both start-up businesses and foreign firms to function more normally, as opposed to fearing


omic growth. This realization, along with some early successes on the anti- corruption front, compelled the central government to take stringent steps to curb the worst offenders.
One place to make some immediate gains was in better enforcement of ex


predictions did not come to pass. Some had suggested, back in the 2010s, that the aquifer in the North China Plain might dry out if rainfall remained at historic lows. Climate change might have well aggravated the drought, but rainfall resumed norma


ears. From roughly half the population living in cities in the early 2010s, the proportion is now about 70 percent. Demand for urbanization had been kept artificially low through the 2000s via the hukou system, but China is now at an urbanization lev


t a slower rate than the number of total kilometers traveled was.
Car Manufacturing Expands and Ownership Rises
The Chinese car industry has been expanding. In 2030, production is roughly 50 million vehicles per year, both passenger vehicles and truc


n Beijing and Shanghai. The Shanghai model of auctioning license plates has been viewed as more effective at dampening vehicle ownership—indeed, as early as 2011, auto ownership was considerably lower in Shanghai than Beijing despite that city having


 as incomes rose. Additionally, climate regulations passed in 2020 made gasoline-powered vehicles less attractive because of the increase in gas prices that resulted.
With regard to technological change, several important developments shaped the use


rs are equipped with mobile Wi-Fi hotspots so that passengers need not miss a minute of online chatting.
Although there was a flurry of activity in the mid- to late 2010s around the promise of autonomous vehicles, so far, very little commercializatio


HSR. Much of the travel is for business purposes; even though the economy continues to concentrate in the eastern coastal cities, front-office types are still visiting factories in the central and western areas. Domestic tourism has been growing stea


benefit. Growth in travel demand will eventually slow now that the intercity network is built out and constraints remain on owning and driving personal vehicles. The environmental movement will likely play a larger role in the future as more Chinese


t growing Scenario: Key drivers and Reactions
 52 THE FUTURE OF MOBILITY _ THE ScEnARIOS
A Financial Crisis Sparks Economic Slowdown
Although the central government signaled its intentions to make some key reforms that would have steered the economy


ming loans because they had large amounts of capital available. But many of the shadow-banking entities were forced to close, and some investors lost substantial sums.
 Reform Remains Difficult
The weak banking reforms were just one sign of the broad


th rebounded to about 3 to 4 percent annually through the mid-2020s and rose to about 5 percent by 2030. This rebound was attributed to efforts undertaken after the crisis to focus more on the sectors of the economy that support growth in consumer de


testing. These drew more government response than previous demonstrations had as the government sought to hang on to its legitimacy. One response was more transparency in information related to soil pollution levels, and, by the late 2020s, many majo


ins how what many obser- vers think is a respectable rate of GDP growth coexists with stubborn poverty.
55
 56 THE FUTURE OF MOBILITY _ THE ScEnARIOS
Energy Prices Remain Level
Oil has remained at its 2013 price, at about USD 100 per barrel. Demand i


rging infrastructure stymied ambitious plans to increase numbers of NEVs produced. It was, of course, easier for NEVs to amass a larger share of the fleet in China than in developed countries because so many car purchases in China represented additio


tempted to ban them previously, those days are over, and cities have instead tried to accommodate them with separate lanes and, in some cases, even off-street paths. Some models travel too fast to mix safely with pedestrian traffic, so a few places h


ride high-quality transit where it is available. These cities have also been active in promoting transit-oriented development, building apartment buildings and office towers within walking distance of new rail stations.
The same pattern holds true fo


n, similar to the wild card we explored in the U.S. scenario project (Zmud, Ecola, et al., 2013).
  62 THE FUTURE OF MOBILITY _ wILd-cARd ScEnARIO
Debt Comes Due
The central government tried to intervene to stave off the mounting debt ratio but inste


ercent per year. Several problems contributed to this unexpectedly sluggish growth.
First, productivity remained stagnant. Chinese productivity had been slowing even as GDP grew during the 2000s (Hoffman and Polk, 2014), in large part because lending


red as officials retired and fewer replacements were hired, so polluters had no fear of being made to pay fines or shut down. Plans to build additional treatment plants were similarly put on hold because the original plan had called for these to be b


tion system might be made. The scenarios account for both the current state of affairs (because past trends informed the projections) and the various forces that might shape the state of affairs in 2030. These forces, which might be more or less like


ew infrastructure for other modes.
Analyzing the differences between the two scenarios, we can identify the key drivers that caused one path to emerge over another. Our analysis revealed three factors as being significant in this regard: (1) economic


ween the two scenarios.
Implications for Transportation Decisionmaking
Our two scenarios present transportation policymakers, planners, transportation suppliers, and private-sector users of the system with the different challenges and opportunities t


ent status. At every stage, the planner asks, “What must be done at the previous stage to reach this stage?” Making sense of past events and monitoring potential future developments when working in a high-pressure environment (as transportation often


ly one such wild card, Debt Comes Due, other wild cards should be considered. These could be either positive or negative. Positive wild cards might be based on health technology breakthroughs that produce extreme longevity and declines in disability,


ity is one key element—depends heavily on overall growth. Although policy levers certainly influence economic growth, factors outside of decisionmakers’ control, as well as constraints based on previous economic trends (such as the use of debt), coul


plausible combination of possible future developments. Scenarios support what-if thinking and lay the foundation for alternative strategies to reduce uncertainties in mid- and long-term planning (Gausemeier, Fink, and Schlake, 1998; Lempert, Popper,


t trends.)
 80 THE FUTURE OF MOBILITY _ METHOdOLOgY
Step 2: Elicit Projections on Descriptors
We held four workshops, one for each influencing area. We held workshops on demographics and on economics in the RAND Washington office in December 2013. We


of 1)
 (b) Remained equal (Palma ratio of 2)
 (c) Has increased (Palma ratio of 2.5)
2.4
Labor-force participation
(a) 76% of the adult population
 (b) 80% of the adult population
2.5
Domestic vehicle production
(a) 120 total vehicles per CNY 100 mil


opulation has a strong influence on urbanization, so that relationship would be rated 3, while total population has no effect on the convenience of public transit, so that was rated 0. This exercise establishes the degree of interconnectedness among


tal vehicles per 100 million CNY (same as current)
240 total vehicles per 100 million CNY
2%
4% (same as current)
USD 100 per barrel (real price in 2014)
USD 150 per barrel (real price in 2014)
70% of oil is imported
60% of oil is imported
Economic g


alysis that preceded this step. The exploratory scenario-construction toolbox in RAHS isolated clusters made up of homogeneous groups of descriptors and projections based on the consistency-analysis results. (More details on the application of consis


0
Oil consumption
 70% of oil consumed is imported
0
0
0
 100
60% of oil consumed is imported
100
100
100
0
                                              88 THE FUTURE OF MOBILITY _ METHOdOLOgY
Table A.2. Share of Projections within Each cluster — co


is type of exercise difficult because it cannot be based on past trends. As such, we instead discussed changes in travel demand in a more qualitative manner (e.g., strong increase versus moderate increase). We also wove in more broadly some thoughts


       93
   94
Bibliography
  “All Dried Up: Northern China Is Running Out of Water, but the Government’s Remedies Are Potentially Disastrous,” Economist, October 12, 2013. As of April 5, 2015: http://www.economist.com/news/china/21587813-northern-c


ards a User’s Guide,” Futures, Vol. 38, No. 7, September 2006, pp. 723–739.
Bradsher, Keith, “Speedy Trains Transform China,” New York Times, September 23, 2013. As of November 11, 2014: http://www.nytimes.com/2013/09/24/business/global/ high-speed-t


tarting next year],” October 19, 2010. As of November 19, 2014, in Chinese: http://inf.315che.com/n/2010_10/131082/
cherry, christopher, email exchange with Liisa Ecola and Kun Gu, December 23, 2013.
“China: A Question of Trust,” Economist, May 10, 2


k, September 25, 2013, pp. 12–18.
durst, carolin, Thomas Kolonko, and Michael durst, “Kooperations- dilemma in der Zukunftsforschung Ein IT-basierter Lösungsansatz der Bundeswehr [Cooperation dilemma in future analysis: A web-based approach by the G


47-chinas-new-rail-network- already-worlds-longest-will-soon-stretch-considerably-farther-faster
Hoffman, david, and Andrew Polk, The Long Soft Fall in Chinese Growth: Business Realities, Risks and Opportunities, New York: Conference Board, R-1563-14


ce.mit.edu/handle/1721.1/73607
Krieger, Axel, Larry wang, Philipp Radtke, and christian Malorny, “Recharging China’s Electric-Vehicle Aspirations,” McKinsey and Company, July 2012. As of April 5, 2015: http://www.mckinsey.com/insights/energy_resource


———, China Statistical Yearbook, 2003. ————, China Statistical Yearbook, 2004. ————, China Statistical Yearbook, 2005. ————, China Statistical Yearbook, 2006. ————, China Statistical Yearbook, 2007. ————, China Statistical Yearbook, 2008. ————, China


w.chinadaily.com.cn/china/2010-12/29/content_11768648.htm
98 THE FUTURE OF MOBILITY _ BIBLIOgRAPHY
  Randall, Tom, “Pain at the Pump: Gasoline Prices by Country,” Bloomberg Visual Data, September 3, 2013. As of November 11, 2014: http://www.bloomberg


ldbank.org/content/dam/Worldbank/document/ Poverty%20documents/Inequality-In-Focus-0813.pdf
Song, Sophie, “Prices of Beijing’s Black Market License Plates Now Double the Cost of Most Popular New Car,” International Business Times, February 4, 2014. A


拥有量排名 北京第三 [Ranking of new energy vehicle ownership: Beijing ranks third],” Legal Evening News, September 6, 2013. As of November 19, 2014, in Chinese: http://www.fawan.com/Item/210481.aspx
wang, Yichuan, “High-Speed Rail Is at the Foundation of Chin


er 24, 2013, in Chinese: http://www.tranbbs.com/Advisory/TPlan/Advisory_63801.shtml
Zmud, johanna P., vincent P. Barabba, Mark Bradley, j. Richard Kuzmyak, Mia Zmud, and david Orrell, Strategic Issues Facing Transpor- tation, Vol. 6: The Effects of S


  CORPORATION
Measuring the Resilience of Energy Distribution Systems
Henry H. Willis and Kathleen Loa
For more information on this publication, visit www.rand.org/t/RR883
  Published by the RAND Corporation, Santa Monica, Calif. © Copyright 2015 RAN


aking in a wide range of policy domains, including civil and criminal justice, infrastructure protection and homeland security, transportation and energy policy, and environmental and natural resource policy.
Questions or comments about this report s


Systems Will Have Different Resilience to the Same Disruption . . . . . . . . . . . . . . . . . 5
2.3. Different Responses Will Lead to Different Resilience at Different Costs. . . . . . . . . . . . . . . . . . . 5
2.4. Resilience of a System Also De


cold spells in the Northeast during the winter of 2014 sharply increased demand for natural gas, leading the prices of gas and elec- tricity to rise (U.S. Energy Information Association, 2014). Furthermore, in 2013, Pacific Gas and Electric Company’


the state of service being provided by a system in response to a disruption. When assessing resilience, key questions would be whether the service has been degraded, how much of the service has been degraded, how quickly the service has been restore


reater frequency of intense flooding (Figure 2.4, Response 1). If the system is continually maintained and upgraded, the service provided could improve, but at a cost (Figure 2.4, Response 2). However, if maintenance and upgrades are not made, operat


ue set of metrics. But across levels of decisionmaking, the metrics ought to be organized within a consistent measurement framework. Logic models provide a consistent framework for organizing metrics in the fields of program evaluation and qual- ity


e the amount of energy delivered or operating characteristics of the system, such as efficiency, reliability, fault tolerance, sustainability, or robustness.
In the end, the performance of energy systems depends on how the systems generate the outcom


etric, natural gas, nuclear, petrochemical, petroleum, pipelines, power, production, redundancy, reliability, resilience, transmission, vulnerability
   1 The citations listed in Tables 4.2, 4.3, and 4.4 are instances where the metric can be found in


efined Oil, and Natural Gas Systems 11
Maturity refers to how well defined the metrics are, how systematically they are collected, and how well they are organized. Our assessment of maturity reflects the practical challenges that extend beyond defini


ls and potentially create a systematic means of collecting and aggregating data across facilities and systems if efficient means of doing so can be implemented.
3 Although only 154 metrics were identified, some metrics are relevant to more than one s


ng, 2007)
devices—mean time to repair (M) (Yeddanapudi, 2012) Protective and switching
2012; Bagchi, Sprintson, and damage caused by fire to the electrical system (M) (Lucia, Load loss damage index— Singh, 2013)
frequency index of sags (M) (Shun et v


ctrical protection and metering Hazard rate relating function— Equipment positioning (L) (Keogh
(Bompard, Napoli, and Xue, 2010)
(Bompard, Napoli, and Xue, 2010)
Reserve/spare capacity (M) (Willis
Gen/load bus distribution (L)
and Garrod, 1997; Molyn


mission, and Ogden, and Sperling, 2007) distribution (H) (McCarthy, Functional zones—
Transmission lines available (#) (M) (Roe and Schulman, 2012)
only generating facilities, Hierarchical levels (HLI, facilities, and HLIII includes all three functio


ns (CELID-X; metric; measures the percentage of (H) (Rouse and Kelly, 2011) 2011)
18 Measuring the Resilience of Energy Distribution Systems
   Outcomes
  Performance
  Capabilities
  Capacities
  Inputs
  mode to minimize line losses, making full us


e fraction (Nadeau, 2007; Ellison, Corbet, and Brooks, of gas burned as compressor fuel (H) and Brooks, 2013) 2013)
emergency shutdown system (M) (Ellison, Corbet, and Brooks, 2013) Maximum/minimum flow (H) Emergency procedures/ (Hsu, Shu, and Tsao,


(McCarthy, Ogden, and primary energy supply, energy processing, and conversion and transport Functional zones— Sperling, 2007)
system lacks constant levels of productivity (M) Network resiliency—measured by its ability to keep supplying and distribut


 Sperling, 2007)
NOTE: Metric appears in bold. Maturity level appears in parentheses: H = high; M = medium; and L = low. The citations included in this table are instances where the
metric can be found in the literature, but they are not necessarily


become more resilient, better use of metrics will be crucial to guiding planning and evaluating progress. The literature review presented in this report suggests three recommendations that could improve the metrics available to support energy policy


’s proficiency requires public- and private-sector cooperation to identify which capabilities to measure, practical and valid exer- cises or tests of proficiency in those capabilities, and efficient means of collecting and manag- ing data on capabili


 program. Albuquerque, NM: Sandia National Laboratories.
Bhattacharyya, S. C. (2009). Fossil-fuel dependence and vulnerability of electricity generation: Case of selected European countries. Energy Policy, 37(6), 2411–2420. doi: 10.1016/j.enpol.2009.


ystem. Paper presented at the 14th International Middle East Power Systems Conference, Cairo University, Egypt.
Eto, J. H., & LaCommare, K. H. (2008). Tracking the reliability of the U.S. electric power system: An assessment of publicly available inf


tilities.pdf
Linkov, I., Bridges, T., Creutzig, F., Decker, J., Fox-Lent, C., Kröger, W., Lambert, J., Levermann, A., Montreuil, B., Nathwani, J., Nyer, R., Renn, O., Scharte, B., Scheffler, A., Schreurs, M., & Thiel-Clemen, T. (2014). Changing the


uctural Dynamics, 41(1), 61–79. doi: 10.1002/eqe.1118
Reymond, M. (2007). European key issues concerning natural gas: Dependence and vulnerability. Energy Policy, 35(8), 4169–4176. doi: 10.1016/j.enpol.2007.02.030
Roe, E., & Schulman, P. R. (2012). T


litative resilience analysis of petrochemical supply chains to a hurricane. Process Safety Progress, 30(3), 280–290.
Wang, H., Lin, Z., Wen, F., & Huang, J. (2012, September 8–9). A comprehensive evaluation index system for power system operation. Pa


  CORPORATION
The Potential of Blind Collaborative Justice
Testing the Impact of Expert Blinding and Consensus Building on the Validity of Forensic Testimony
Carolyn Wong, Eyal Aharoni, Gursel Rafig oglu Aliyev, Jacqueline Du Bois
 For more informati


f expert blinding and consensus feedback in a single representative sample of scientific experts. Evidence that expert blinding and/or consensus feedback can improve the validity of expert testimony would be of great value to the programmatic efforts


y view back to the individuals, and repeating this cycle to move toward a group consensus. This technique has been successfully used in a variety of disciplines, but has not been investigated in a courtroom scenario. Empirical support for this effect


dback responses (without advanced consensus interpretation) were incorrect, and higher levels of consulting experience and education were not associated with greater response accuracy. Our finding that delivery of expert consensus feedback resulted i


Commission as part of the DNA Sexual Assault Justice Act of 2004 (Public Law 108-405). The Department of Justice (DOJ) fulfilled this mandate by awarding a contract to the National Academy of Sciences (NAS) to conduct a study that addressed congress


g or “woodshedding” witnesses toward a desired conclusion, as well as expert-driven factors, such as outright adversarial allegiance, random drift from the consensus view, and other unconscious cognitive biases—such as the tendency to favor outcomes


more recent study by Murrie et al. (2013), the investigators hired forensic psychologists and psychiatrists to score risk assessments for the same offender case files but
2
manipulated their ostensible party representation. Those who believed they we


 positive diagnosis relative to blinded experts.
Finally, researchers in the United Kingdom investigated the likelihood that fingerprint examiners would produce false-positive matches depending on whether they were blinded to extraneous information a


nciple that judgments arrived at by a panel of experts using a structured process are more likely to reflect the truth than those by unstructured groups or individuals.2 The Delphi expert feedback consensus method was developed by the RAND Corporatio


dence might benefit from an expert consensus building–based approach.
Expert panels in consensus building exercises need not be large. Researchers have noted that ten experts could be enough in homogeneous cases (Delbecq, Van de Ven, and Gustafson, 1


en a particular base rate of true and false cancer diagnoses in a broader population), research has shown that responses to such inquiries are not random, but are clustered and indicative of a small number of specific, often erroneous, cognitive stra


on or defense) and these differences would be greater without exposure to the consensus feedback.
7
2. Methods and Data Collection Methods
The purpose of this study was to examine two potential contributors to biased testimony within adversarial liti


e with a solution based on theories that would be familiar to a target audience of professionals with advanced degrees in scientific fields.
The hypothetical case developed for the study involves criminal trespass and grand theft larceny of property


re of the 14 professional scientific societies we describe next, then culled our resulting list to include only those with doctorates. Eighteen employees were invited to participate, but none were informed about the goals or purpose of the study befo


onic format by email or on a dedicated protected website. In the first round of the exercise, each panel member submits assessments (responses to the questions on the questionnaire) and justifications for the responses. An exercise coordinator comput


ck) was given the opportunity to provide a different answer after their initial responses had been recorded. The random assignment produced a relatively even split among the various subsamples. In terms of party representation, there were three possi


k. In this case, the mean, median, and mode were all equal to 50-percent probability that the defendant was in the stairwell. In Round 2, the ten panelists who submitted correct responses in the first round made no changes to their responses. The com


ions to participate in the justice experiment were delivered by the nine professional scientific societies who agreed to distribute the invitation to their membership. Six hundred and eighty five individuals accessed the questionnaire and provided co


e in question (median = 25 percent, mode = 10 percent). However, such measures of central tendency are limited due to the multimodal nature of the distribution, which reveals possible evidence of discrete cognitive strategies. As expected, evidence o


cution condition (MD = 2.83, SE = 1.73, p = 0.10). However, this test violated the assumption of homogeneity of variance, using Levene’s test of equality of error variances, F (5, 545) = 10.52, p < 0.001, so we re-examined the direct effect of party


e degree), the expected main effect of consensus feedback was observed with accuracy increasing following the feedback, F (1, 218) = 50.56, p < .001, MD = 7.40 (SE = 1.04). However, no interactive effect was found, F (1, 218) = 1.42, p = 0.23. Moreov


itive rate provided in the hypothetical scenario, suggesting broad neglect of the base rate information, as predicted. This latter finding is highly consistent with classic research on Bayesian reasoning. Our own pattern of results could indicate tha


ritical thinking provide a plausible explanation for this effect, additional research would be needed to determine whether such motivations consistently evoke more accurate results or simply more effort or creativity of approach. Finally, we did not


Unfortunately, it is not possible to know the reasons for participant dropout. We speculate that the level of difficulty of the reasoning problem might have deterred some individuals from participating. If so, the true error rate of this population


posure approaches in a criminal justice environment.
 5 We experimented with an online means for providing such refresher education with a web application called TeachApp: Are You An Expert? (See the bibliography for the link.)
23
Appendix: Supplemen


26.98, 36.71] [24.85, 31.48] [26.20, 32.71] [29.61, 36.82]
               In addition to testing the between-subjects effect of feedback, we also sought to examine the within-subjects effect of feedback, namely the error differential before and after


iences, 2nd ed., Hillsdale, N.J.: Erlbaum, 1988.
Delbecq, Andre L., Andrew H. Van de Ven, and David H. Gustafson, Group Techniques for Program Planning. Glenview, Ill.: Scott, Foresman, and Co., 1975.
Dror, Itiel E., and David Charleton, “Why Experts


Hua Kei, Assumpta Madu, “Setting Priorities for the IT Industry in Taiwan—A Delphi Study,” Long Range Planning, Vol. 24, No. 5, 1991, pp. 105–118.
McCabe, David, and Alan Castel, “Seeing is Believing: The Effect of Brain Images on Judgments of Scient


- Crofts, 1966.
Sackman, Harold, Delphi Critique: Expert Opinion, Forecasting, and Group Process, Lexington, Mass.: Lexington Books, 1975.
Schneps, Leila, and Coralie Colmez, Math on Trial: How Numbers Get Abused in the Courtroom, New York: Basic Boo


   CORPORATION
Using Future Internet Technologies to Strengthen Criminal Justice
John S. Hollywood, Dulani Woods, Richard Silberglitt, Brian A. Jackson
SUMMARY Web technologies just over the horizon, including semantic tagging, intelligent agents, an


titioners
Another major theme was improving practitioners’ knowl- edge of web technologies. Priorities included general education on key web technologies, and model policies and procedures for using them. Panelists also called for procure- ment check


ose proximity
Top 5 Needs of Courts
• Video links to correctional facilities
• Procurement checklist for courts’
information technology
• High-speed Internet connections for courts
• Virtual courtrooms
• Educational materials on key web
technologies


d been safely taken into custody—where he would remain for a very long time.
———
How we use information technology (IT) is on the verge of
dramatic change. In the past, the World Wide Web and the Internet largely have involved passive activities: rea


d on key- words and phrases, online mapping and navigation, and social media. Table 1 adapts Kevin Kelly’s perspective on the history of the web and associated information technologies (2007). While there are no canonical definitions of web generatio


d illegally or in the immediate vicinity of an emergency), an officer could direct the vehicle to move to a new location (with the vehicle’s intelligent agents recognizing “officer” and “directions to move”) and automatically notify its owner and occ


cations like those written by these specified authors”
• “Find pictures of Georgian chests of drawers for sale near my house”
• “Find other employees whose backgrounds make them a good fit for
my project”
• “Display a network chart of all technical d


ost them on secure file transfer protocol sites, transferring files from one host to another for law enforcement use. These are transmitted in “standard” formats, such as extensible markup language (XML) or Excel worksheets. Web 3.0 technologies, suc


es of the circumstances under which such monitoring is justified. Policies, procedures, and technologies to help ensure that new technologies are not abused were a major discussion throughout the workshop.
Methodology
To consider the implications of


this report. In brief, panelists rated how valu-
able a solution to each need might be to each community of practice (law enforcement, courts, and corrections). They then assessed how technically feasible and operationally feasible it would be to dev


ith any panel exercise involving a limited number of participants, a different group may well have produced a different set of results. That said, our findings are consistent with our prior studies on criminal justice technology needs (Hollywood et a


of web technologies across the criminal justice community and (2) being comparatively easy to develop and implement.
DISCUSSION
In prior research on priority criminal justice technology needs (International Association of Chiefs of Police, 2005, Kope


 the predictive analytics tool but had noticeably higher scores than other Tier 2 needs
 10
 The call for “technological infrastructure” closely matches the general desire to improve criminal justice information sharing, in a single need.
for law enf


seful to explore whether seman- tic tagging might help facilitate data sharing in general, along with a range of data safeguards (on data provenance, privacy, and data access) and whether intelligent agents might be used to monitor and enforce access


ble”)
and about the past crimes they committed (from the incidents table). The difficulty is compounded if one wants to pull a person’s criminal history across multiple jurisdictions where he or she committed crimes—one has to put in very specific qu


atabases also are being deployed, with a key type being documented-oriented databases.
11
 12
about the person—name, address, scars/marks/tattoos, or
any other specific information known about the person.
• It will be much easier to put together the


NSTIC Trustmark Pilot, undated). As with searching, intelligent agents can simplify and automate the process of interpreting trustmarks and other security tags to get access to needed information.
Opportunities for and Obstacles to Improving Informat


 System (part of the FBI’s larger Next Generation Identi- fication system).
The call for “situational awareness displays” to use the shared information effectively closely parallels a similar need for law enforcement IT (Hollywood et al., 2015). Also


lp practitioners learn more about technologies, use them properly and effectively, and acquire them. Improving practitioners’ knowledge had the second-highest number of top-tier needs (five)
• educational materials on key web technologies (ranked sec


ess by the criminal justice communities of practice (see Gordon et al., 2012, for
a larger discussion of this issue, focusing on the DoJ side).
Even within portals, material is commonly presented as lists
or databases of disjointed documents. At the


Friday, you need to come to
your initial counseling appointment at the Center after work. I know you know where that is.”
“Yes, Dr. Nassar, looking forward to it.”
“Okay, sounds like we’re done here. Good luck and congratu- lations on getting early p


at led down a hall to the kitchen.
15
 16
 Three of the four needs concerned furthering officer health and safety.
The door felt warm, but not enough to really ring any alarm bells in her head. Suddenly, an alarm sounded in her ear, and a voice shout


otos of the city’s undercover officers and informants would be his. The initial plan was to dump all that information online. But then he wondered how much he could get by selling it to the local crime syndicates. Very tempting.
At last he was in. He


 response and operational deconfliction but also as a threat if criminals could recognize an undercover agent’s sensor during an operation.
Standards were not seen as posing inherent risks to civil rights, privacy rights, or security. Rather, the con


form day-to-day workloads of criminal justice staff ranging from law enforcement officers to judges
Interviews and report writing
Need intelligent agents to assist with field interviews and writing incident reports, along with the controlled vocabula


ence)
Video links to correctional facilities
Need to develop and/or assess video teleconferencing tools to allow incarcerated offenders to establish relationships and build rapport with community corrections officers before release
Real-time translat


  Table 3—Continued
21
     Community of Practice in Need
Description
Tier
Privacy Rights/ Civil Issue?
Security Issue?
      Part 4: Policies, Procedures, and Guidance
    1
 1
 1
 1
 2
 2
   Technology training
Need educational materials for key we


 of serious events
Semantic search assessment
Need to assess existing digital evidence toolkits to determine whether semantic search tools could substantially speed up searching and analyzing digital evidence
Analytics for social media and community


ionships of criminal justice concepts)
Emergency data sharing
Need common data models and data access mechanisms that will support integrating data feeds from multiple sources (including IoT sensors from the public and private sectors) and providing


risks of false positive identifications. The new Trustmarks (GTRI NSTIC Trustmark Pilot, undated) and other security tagging technology might be leveraged here to expedite setting up memoranda of understanding for sharing mug shots and other facial


systems (as discussed above).
The more general next step is to begin converting key technology portals from lists of disjointed materials to learning portals, which provide both technical developers and agencies seeking to acquire technologies with a


upport the location and tracking of offenders under community cor- rections supervision. Technologically, this means experiment- ing in two directions. The first is on tracking systems, both wearable and contained in infrastructure, that can leverage


e technology experts and experts on civil rights, privacy rights, and information security should be participants.
Notes
1 A precision immobilization technique (PIT) is a driving maneuver designed to cause a suspect’s vehicle to spin out and stop. 2


oordinating Council, undated. As of June 12, 2015: http://www.ise.gov/standards-coordinating-council
International Association of Chiefs of Police, Law Enforcement Priorities for Public Safety: Identifying Critical Technology Needs, Alexandria, Va.,


/324602
Silberglitt, Richard, Brian G. Chow, John S. Hollywood, Dulani Woods, Mikhail Zaydman, and Brian A. Jackson, Visions of Law Enforcement Technology in the Period 2024–2034: Report of the Law Enforcement Futuring Workshop, Santa Monica, Calif.:


ivil and criminal justice, infrastructure protection and homeland security, transportation and energy policy, and environmental and natural resource policy. Questions or comments about this report should be sent to the project leader, Brian A. Jackso


erprise.
This report is one product of that effort. It presents the results of the Web 3.0+ Criminal Justice Needs Workshop, a group convened in September 2014 as part of the NIJ/NLECTC Priority Criminal Justice Needs Initiative to identify and prior


 Energy-Sector
Workforce Development
in Southwestern
Pennsylvania
Aligning Education and Training with Innovation and Needed Skills
Gabriella C. Gonzalez, Reema Singh, Rita Karam, David S. Ortiz
 CORPORATION
For more information on this publication,


The results should be of interest to persons developing training programs to meet the needs of the energy sector and to firms in the energy sector, which need to plan for their future workforce needs.
iii
iv Energy-Sector Workforce Development in Sou


to the extraction, production, and use of energy resources. (This number includes corporate employees and
ix
x Energy-Sector Workforce Development in Southwestern Pennsylvania
the self-employed.) Since 2007, the energy sector has experienced a 23.4-p


.
The findings from this study can be used as a springboard for
deeper discussion among regional education and training institutions, business leaders in the energy industry, and nonprofit organizations devoted to supporting the employability of loca


energy-sector stakeholders could employ to support the education and training of regional talent. We intend for these suggested measures to spark discussion among representatives from the region’s energy-sector businesses and postsecondary education


skills, as well as broad (generic) skills. Curricula should also address the devel- opment of soft skills, such as the ability to work in teams, com- munication skills, entrepreneurship, customer relations, and work discipline (OECD, 2010). Studies h


for the degree or cer- tification. And most training programs and community colleges in our study indicated that their curricula incorporate generic and soft skills in addition to the occupation-specific content knowl- edge. Business partners, contra


mmunity colleges, unions, and private certification and training institutions—to forecast demand, provide regular input on curricula, support the acquisition of equip- ment and supplies used in hands-on training, and institute formalized workplace le


 to monitor whether their programs are meeting industry needs.
To complement accreditation, we suggest that education and training providers in the region develop and implement a mechanism that provides continuous and rigorous feedback. Two possible


ank the careful reviews of Keith Crane, Tom Witt, and Robert Bozick. Any errors and omissions are the fault of the authors.
xxi
Abbreviations
 AD associate’s degree; degree completed after two years of full-time schooling beyond high school
BD bachel


).
 1
2 Energy-Sector Workforce Development in Southwestern Pennsylvania
Figure 1.1
The 32 Counties That Make Up the Southwestern Pennsylvania Region
  Mercer
Lawrence
Butler
Beaver
Allegheny
Brooke
Washington
   Mahoning Columbiana
Ohio
Hancock Jeff


 number of direct jobs (those that are cre- ated specifically by businesses that produce, extract, transport, or
3 All information in this paragraph comes from the U.S. Energy Information Administra- tion, 2014.
 4 Energy-Sector Workforce Development


 of the Pennsylvania targeted-industry clusters in three steps:
• Identify occupations that are important to the success of industries within the clusters.
• Review occupational statistics, and apply filters to hone the list.
• Incorporate input from


of on-the-job training or an apprenticeship. MT-OJT = medium-term on-the-job training; basic tasks and skills are learned through a period of on-the job training, and a high-school diploma may be required. BD = bachelor’s degree; degree completed af


 energy industries, such as oil and gas extraction and coal mining, and renewables—due to increased production and extraction, coupled with impending retire- ments from the baby-boomer generation. More jobs will be opening up
Aligning Workforce Educa


 the workforce in new competencies.
In the SWPA region, these issues are even more pronounced than in the United States as a whole, given the burgeoning field of natural- gas extraction that exists alongside the well-established, traditional coal-min


fying these technologies enabled us to con-
12 Energy-Sector Workforce Development in Southwestern Pennsylvania
ceptualize trends and motivators of technological innovations, which helped inform discussion protocols.
We then conducted interviews in p


onal and training programs located in the 32-county SWPA region that train workers for the regional energy sector, examining how their programs compare with the characteristics determined as model practices from the literature review on exemplary TVE


anization of This Report
We have structured the remainder of this report according to the study’s objectives. We first document trends in innovation and tech- nology in the energy sector and the demand for labor and skills likely to be needed in the


 Workforce Development in Southwestern Pennsylvania
In this section, we organize the innovations in the energy sector that are likely to affect the demand for labor and workforce skills into four areas:
1. increasing productivity in energy extraction


n the productivity and efficiency of energy extraction or production from these sources.
Reducing Environmental Damage and Emissions of Carbon Dioxide
Increased concerns about environmental impacts brought about by energy production and consumption (


ality, energy-storage systems, and other key technologies. Such technologies include grid-sensing and market-pricing systems to ensure that intermittent resources are priced accordingly (Anadon et al., 2011). Some technological innovations in these e


agement systems Thermal storage systems
Fuel utilization (e.g., advanced fuel chemistry and testing, engine–fuel interactions)
Advanced power electronics
Vehicle systems (e.g., advanced heating and cooling, vehicle thermal management)
Home charging s


read and interpret more sensors and instru- mentation. A solar-energy expert explained that working with new grid architecture would likely require an electrician to learn new codes and regulations but would not necessarily affect the skill set neede


g steel
• climbing a steel column
Skills Needed to Meet Changing Industry Demand 25
• using surveying equipment with precision • rigging.
Manufacturing and automation, on the other hand, place special emphasis on skills in digital technologies, writi


iew of U.S. and international education training systems. We identified exemplary systems through a review of Organisation for Economic Co-operation and Development (OECD) publications that have documented the quality of training programs. There is e


Other countries, such as Germany, where responsibility for vocational training is divided between the federal government and localities have developed close partnerships among government, employers, and trade unions at all levels. For example, region


areers. For example, many current jobs across various sectors, including the energy sector, use computers or computerized instrumentation that require those operating these instruments to be able to read the data generated and make the appro- priate


ical skills and upgrade their skills to keep up with changes in occupations and the technologies used.
Some exemplary education and training programs included in our review select instructors who also work in the industry (OECD, 2010). These programs


nd one postsecondary career- and technical-education center. Descriptions of these programs are provided in Table A.1 in the appendix. Further details on the case-study site methodology we employed and site selec- tion are available in Chapter One.
T


 the process for industry–educator partnerships tends to be employer-initiated: Employers would ini-
Four Education and Training Providers in Southwestern Pennsylvania 35
tiate contact with the program directors to inform them about spe- cific workpl


itical thinking. With respect to technical skills, program directors emphasized the importance of students acquiring a “core knowledge and foundation” of the industry, as well as a skill set that was portable. They identified math and science, facili


lized skill sets and expertise in their trades.
Programs use professional and academic networks, former retir- ees, and general hiring strategies to identify potential candidates for teaching. Although some retirees have been hired as full-time fac-


dology was necessary in industries in which technologies constantly evolve.
Four Education and Training Providers in Southwestern Pennsylvania 39
In terms of ensuring the quality of educational curriculum, one program’s director plans to update the u


ucation and training providers through involvement with cur- riculum development, identifying job needs and vacancies, advertising programs, providing feedback, and providing equipment and materials for training. The key to success is sustained and c


est or refer recent retirees to education and training pro-
viders as possible instructors.
Incorporate Continuous Quality Assurance into Programs
Accreditation is the first step to ensure quality; all education and train- ing programs need to be acc


t Evolving Energy-Sector Needs 45
When conducting an evaluation, one can examine two areas:
• implementation, which determines whether programs are being implemented as designed and whether inputs are sufficient to ensure that the program can be prov


ns among regional stakeholders to ensure that the workforce is trained and educated in ways that best meet the changing labor-market needs of the energy sector.
APPENDIX
Descriptions of Education and Training Facilities Used for Case Studies
 Table A


ational and Training Programme,” International Journal of Lifelong Education, Vol. 24, No. 2, March– April 2005, pp. 137–147.
Anadon, Laura Diaz, Matthew Bunn, Gabe Chan, Melissa Chan, Charles Jones, Ruud Kempener, Audrey Lee, Nathaniel Logar, and Ve


János, Workplace Literacy Requirements and Unskilled Employment in East- Central and Western Europe, Budapest Working Papers on the Labour Market 2006/7, Budapest: Institute of Economics, Hungarian Academy of Sciences, December 2006. As of October


onsors-Partners-1
Smits, W., “Industry-Specific or Generic Skills? Conflicting Interests of Firms and Workers,” Labour Economics, Vol. 14, No. 3, June 2007, pp. 653–663.
U.S. Energy Information Administration, “Profile Data,” last updated September 1


 Energy-Sector
Workforce Development
in West Virginia
Aligning Community College Education and Training with Needed Skills
Gabriella C. Gonzalez, Sean Robson, Andrea Phillips, Gerald Paul Hunter, David S. Ortiz
 CORPORATION
For more information on th


ral gas and natural gas liquids from shale resources and their industrial application have increased demand for workers in these industries of the energy sector. In 2013, the National Energy Technology Laboratory (NETL) asked the RAND Corporation to


e Energy-Sector Workforce-
DevelopmentPipelineinWestVirginia............................... NationallyRecognizedPromisingPractices..................................
43 44
. 49 . 55
. 57 . 57 66 84
87
87 . 93
Example Energy-Sector Public-Private Partn


 2008–2009 Through 2012–2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5. Percentage of Students Enrolled in at Least One Developmental Course in Mathematics in 2008–2009, 2010–2011, and 2012–2013, by Age Category. . . . . .


Energy-Sector Workforce Development in West Virginia
sponsored education consortia that provide technical and career training or support students’ transitions from secondary to postsecondary institutions—to collaborate to ensure that the local talent


rkforce for the energy sector, and suggested paths forward.
Key Findings
To meet these objectives, the study undertook four tasks.
Task 1: Identify key knowledge, skills, and abilities needed for energy-sector jobs in West Virginia. For this task, we


revealed that while employers noted core skills needed for semi- skilled positions within their firms, they reported difficulty in finding local West Virginia talent with the necessary skills to fill these jobs. Education and training providers (wit


been systematically evaluated to determine their effectiveness. Therefore, we first described the characteristics of regional and high- profile energy partnerships and initiatives identified through Internet searches using several combinations of te


pared to support local talent’s training in the energy sector.
We order these suggestions based on ease of implementation— first those that we recommend occurring earlier than others, to address crucial bottlenecks, and those feasibly implemented in


nd the following improvements to programming:
• Integrate technical, occupation-specific training along with work- place readiness and other soft skills through cross-college collabo- ration with continual input from industry.
• Design the curriculum


r counseling positions were typically not a priority, and, when counselors were available, students sometimes found them inadequate. Improving career counseling, both during students’ course
Summary xxi
xxii Energy-Sector Workforce Development in Wes


tate institutions is to ask the recently reestablished West Virginia Workforce Planning Council to oversee the implementation of the strategy. The council has been tasked with
coordinating initiatives, leveraging resources, and planning for the deliv


States in the bottom third of all Organisation for Economic Co-operation and Development (OECD) countries (President’s Council on Jobs and Competitiveness, 2011). The gap between employer needs and workforce skills is stark- est as more jobs open up


ucation” (White House, 2014). Third, on July 9, 2014, Con- gress passed (and the President then signed on July 11) the Workforce Innovation and Opportunity Act of 2014 (WIOA) (Pub. L. 113-128). This act reauthorizes the Workforce Investment Act of 19


ral West Virginia to separate these liquids from dry natural gas (West Vir- ginia Department of Commerce, 2014). West Virginia also has large underground natural gas storage capacity, accounting for about 6 per- cent of the U.S. total.
Figure 1.1 sho


l Survey, New York Department of Environmental Conservation, Virginia Department of Mines, Minerals and Energy, U.S. Geological Survey (USGS), Wrightstone, 2009.
NOTE: Only wells completed after January 1, 2003, are shown. Updated June 1, 2011.
RAND


ve typically been in occu- pations that require a semiskilled workforce, i.e., those that require only some training or certification after high school (Kauffman and Fisher, 2012). Table 1.2 lists key occupations in Marcellus Shale–related industries


er
   Table 1.2—Continued
44,844 Derrick operator, Industrial 42,267 machinery
oil and gas
mechanic
Welder 39,350
Wellhead pumper 36,674
   10 Energy-Sector Workforce Development in West Virginia
is unnecessary. Occupations in Table 1.2 are listed in


                      External context that may influence the flow of talent through pipeline:
• State, local, and federal education, labor, and commerce policies • Regional economy
   12 Energy-Sector Workforce Development in West Virginia
to functi


ear colleges and trade schools are the primary arena for imparting the skills and training needed to suc- ceed in the subbaccalaureate local labor market. This is particularly the case for non–college-bound youths, who often remain in their home- tow


this study indirect jobs that typically build up around newly established energy-sector locales, such as hotels, res- taurants, and other hospitality or service-sector businesses, or govern- ment or environmental jobs that have grown to monitor or re


Introduction 15
16 Energy-Sector Workforce Development in West Virginia
Analytic Approach
Four tasks informed the development of the strategy. In this section, we describe each task, associated research questions, and the data sources we used to ans


loyers in aggregate, including type of industrial activity, firm size (number of employees), and geographic location. We contacted 20 employers and interviewed five of them. Interviews inquired about types of jobs and skill sets (content knowl- edge,


ipeline
To develop effective strategies that West Virginia may choose to adopt to fill gaps in the workforce-development pipeline, we reviewed sev- eral national partnerships that aim to develop key workforce skills, knowledge, and behavioral compete


ed in subse- quent chapters. We start with a description of the academic achieve- ments of the potential talent pool in the state. Although the potential talent pool includes recent high school graduates as well as adult work- ers, this section focus


ignificant progress on the state assessments. In the 2011–2012 school year, fewer than half of the students in West Virginia were deemed proficient or above in mathematics and in read- ing on the WESTEST2 (48 percent of students in mathematics and 49


es of tests in mathematics, reading, and science literacy administered by the Organisation for Economic Co- operation and Development (OECD) to representative samples of 15-year-olds in 65 countries, corroborate the WESTEST2 results. PISA results dem


aduate high school as “completers,” indicating that they completed four courses in a concentration and acquired an industry certification or accreditation. Depending on the course or career cluster in which students are interested, they may par- tici


 income level was no longer the sole criterion by which a person could access services. Funds for train-
Characteristics of the Energy-Sector Workforce-Development Pipeline 27
ing services were therefore distributed through individual training accoun


eship agencies), (3) programs provide on-the-job learn- ing alongside job-related technical instruction in a classroom, (4) on- the-job learning is conducted in the work setting under the direction of one or more of the employer’s personnel, and (5)


udy vary by college. For example, two of the nine colleges offer an associ- ate’s degree in electrical engineering technology, while, in the coming years, five colleges will offer mechatronics. Students rarely attend a col- lege far from where they l


e (West Virginia Code, Chapter 18, Article 13, 2013a). Goals of the West Virginia EDGE initiative are to provide an opportu- nity for all students to establish a college transcript while in high school, to increase the number of students attending co


ridgeValley Community and Technical College catchment area, rather than New River Community and Technical College.
Improving geographic coherence is one way in which West Virgin- ia’s workforce-development pipeline could be strengthened: The lack of


rce needs of the energy sector. For the energy sector, the most important degrees from these institutions tend to be within science and engineering departments. Public and private four-year institutions are selective. Offers for enrollment depend, in


age weekly wages by county in those states in the Marcellus Shale Play area (West Virginia, Ohio, Pennsylvania, and New York) between 2008 and 2012 for industries categorized in the “Oil and Gas Extraction” 2012 NAICS code subsector. These industries


from 48 to 360 jobs in the industry. In Pennsylvania, by 2012, total employment had shifted: Some counties experienced a decline to three to 25 jobs, while employment in other counties rose to as high as 719 to 1,202 jobs. The industry expanded geog


12. The most noteworthy growth occurred in Pittsburgh, Pennsylvania, and Charleston, West Virginia, where an estimated 36 to 38 estab-
Characteristics of the Energy-Sector Workforce-Development Pipeline 39
lishments were based. Counties surrounding t


Marcellus Shale drilling opportuni- ties did not cause a large influx of new businesses into the state, but job opportunities and wages increased. The number of establishments increased slightly, jobs spread across more counties than previously, the


 under three scenarios.
The O*NET data set provides a national- and state-level portrait of the KSAs needed in all sectors. O*NET contains extensive infor- mation about a wide range of occupations (n = 947), which can be linked to NAICS codes. Specif


r Occupations 45
• 2121Coalextraction
• 2131 Support activities for mining
• 2211 Electric power generation, transmission, and distribution
• 2212 Natural gas distribution
• 2371 Utility systems construction
• 3241 Petroleum and coal products manufac


the eight shale areas that account for over 90 percent of the shale gas produced nationally. Marcellus Shale play was compared to several shale areas across the nation: Haynesville (Louisiana and Texas), Eagle Ford (Texas), Bakken (North Dakota and


Western Pennsylva- nia, and Eastern Texas/Western Louisiana (Haynesville Shale), the growth in employment in this occupation has been coinci- dent with substantial increases in shale gas production. However, prior to 2010, in each case employment le


Education
The most frequent survey responses indicated that a bachelor’s degree was often required (28 percent) followed by a high school diploma (27 percent). Other responses indicated that an associate’s degree (7 percent), some college (9 percent)


postsecond- ary certificate, and about 11 percent needed some college courses or an associate’s degree. About 11 percent required a bachelor’s degree or higher.
Knowledge
The top five most important knowledge areas determined by ratings of importance


gher included oral expression, infor- mation ordering, reasoning (inductive and deductive), manual dexter- ity, and arm-hand steadiness.
Two abilities included in the top five for energy occupations were not rated as important abilities for general h


important skill areas determined by ratings of importance across occupations by survey responses were active listen- ing, critical thinking, speaking, monitoring, and reading comprehen- sion. Several other skills also received an average rating of “I


. Mathematics 3. English
language 4. Administra-
tive and
management 5. Public safety and security
1. Critical thinking 2 Active listening 3. Monitoring
4. Operations
monitoring 5. Speaking
1. Problem sensitivity
2. Near vision 3. Oral
comprehension


ta for all courses taken in the spring and fall semesters between 2008 and 2012 for the ten CTCS colleges. These data were snapshots of each semester in which each individual student is at a different point in his or her respective enrollment period.


red to enroll in more classes (about six courses) than their older counterparts, who typically enrolled in five or fewer classes per year. Though this pattern of courses by age groups remained consistent between the 2008–2009 and 2012–2013 academic y


gineering-related fields
Mathematics and statistics Physical sciences
Science technologies/technicians Construction trades
Mechanic and repair technologies/ technicians
Transportation and materials moving
2008– 2009– 2010– 2011– 2012–
2009 2010
555 6


 both developmental subjects, while Blue Ridge and Bridgemont tended to have lower percentages of stu-
Table 4.5
Percentage of Students Enrolled in at Least One Developmental Course in Mathematics in 2008–2009, 2010–2011, and 2012–2013, by Age Catego


, 2008 Through 2012, by CTCS College
 Institution
Blue Ridge Bridgemont
Eastern West Virginia
Kanawha Valley
2012–
Type of Degree Granted
Energy related Non–energy related Energy related Non–energy related Energy related Non–energy related Energy rel


and courses. The following section of this chapter explores these questions.
Insights on Education and Training in Energy
This section describes findings from interviews with individuals in sup- port positions for the education and training component


pants reported that they believe that West Virginians are familiar with four-year colleges, yet unfamil- iar with CTCS colleges because the colleges were still in the process of separating from their four-year partner institutions. Students enrolled


in the many high schools in their service area. Some CTCS service areas include five counties with many high schools. One promising practice is a weeklong grant-funded program in high schools, run by staff from a community college. The program focus


erved “declining enrollment of EDGE students in community college.” In an attempt to increase the number of CTE students enrolled in CTCS colleges, administrators from CTCS implemented various strategies. For example, program leaders or direc- tors v


exempt them from technical courses that align with their interests instead of the core general edu- cation courses in which they may be less interested. For example, a student may be exempt from a level 1 or 2 circuit course required in a concentrati


ere long term, as opposed to a large-scale short-term needs. Respon- dents shared figures reported in local newspapers or meetings attended with representatives of the energy sector but were unable to be more specific. A few expressed concerns about


pants are currently employed, a two-year degree-seeking program may provide an opportunity for advancement to middle management
74 Energy-Sector Workforce Development in West Virginia
or supervisor positions at a faster rate. At the time of the inter


If students are required to enroll in developmental coursework, their academic timeline is delayed: The more developmental courses in which a student is required to enroll, the longer the time to gradu- ation. Participants agreed that, depending on


The traditional semester-long schedule and successive courses no longer exist. Students may com- plete several skill sets, or the full set of required skills, in a significantly shorter time period. All interviewees who discussed the restructured dev


es might benefit from advising from program leaders. They perceived that program leaders are better able to encour- age students to persist in the developmental courses because they can more specifically speak to the benefits of persistence and what


nts reported “some” students, more than a few in each program, dropped out of each program of study. Some staff reported concerns that those students did not have the skills required to obtain employment in the energy sector at the time they disconti


als involved in new
80 Energy-Sector Workforce Development in West Virginia
programs reported that the course requirements often changed from semester to semester. The changing course requirements extended the length of the program; if a required cou


ems errors quickly. Employers are actively engaged in the creation of new programs, but many interviewees reported it was diffi- cult to recruit employers to participate in reviews of current programs. Participants suggested employers are critical pa


after students receive their degrees, employers of students, and how long new employees stayed in positions within the energy sector. Interviewees rarely had access to data related to these questions. They relied on their profes- sional experience an


ntified various efforts to understand students’ employment after graduation. Some reported fielding a paper or phone survey once a year asking if the student is employed, who the employer is, and whether the student felt prepared for the position. Ot


ia (CTCS, CTE/K–12, EDGE) exist, but the reported effectiveness varies: Col- leges do not have an incentive to accept EDGE credits. Because job
CTCS Education and Training Programs 85
postings typically request job experience, rather than a specific


e the chapter with a summary of promising practices most useful for West Virginia’s workforce- development system in the energy sector.
Nationally Recognized Promising Practices
A variety of strategies have been developed to promote skill develop- me


ing participants, or to commit to interviewing graduates of training programs (National Network of Sector Partners, 2010; Giloth, 2004). The organizations most likely to act as intermediaries are WIBs and community-based organizations. Other types of


ll and com- petency needs of local employers.
Sector-based partnerships served as a platform for the development of career pathway models. Within a career pathways model, industry- recognized credentials that can be “stacked” and are portable across


y another independent evaluation (Grote and Roder, 2005).
Teaching workplace competencies. In addition to industry- specific technical skills needed to enter the workforce, additional skills are necessary to remain competitive and function effectivel


coursework (Jenkins et al., 2010; Cho et al., 2012; Fong and Visher, 2013), providing support services such as tutoring or learning communities alongside the developmental education courses (Edgecombe et al., 2013), augmenting curriculum to include s


. Consequently, we expanded our search to include other workforce-development programs and part- nerships outside the West Virginia region. Most of these evaluations were supported by grants provided by ETA. For each evaluation we
94 Energy-Sector Wo


unity Development and the Pennsylvania Independent Oil and Gas Association (PIOGA).
ShaleNET leverages its partnerships by providing a talent match service through which interested jobseekers can find job opportunities and potentially even be paired


of alternative fuel technologies, was evaluated in 2012 as part of DOE’s Annual Merit and Peer Evalu- ation Meeting. This included evaluations by five reviewers who found that the program is consistent with DOE’s objectives and has done well to estab


States to provide industry-focused job training.
 98 Energy-Sector Workforce Development in West Virginia
Despite the success of HPTC, several challenges were noted. The large geographic region served by HPTC was a challenge for part- nering organiza


g advising at least once; 4) the percentage of stu- dents receiving transportation assistance at least once; and 5) the frequency of student meetings with an assigned transition coor- dinator. (Bragg et al., 2010, p. 17)
The percentages of students p


of student goals. Some programs provide individual courses; others provide education leading to certifi- cates and bachelor’s and master’s degrees.
A key component of the EPCE model is that industry represen- tatives (e.g., utility companies) meet tw


in sup- port of a strong, viable U.S. oil and natural gas industry to meet the energy needs of consumers in an efficient, environmentally responsible manner” (American Petroleum Institute, 2014). In support of this mis- sion, API engages in a wide r


 KSAs such as an indi- vidual’s professionalism, interpersonal skills, teamwork, math- ematics and science knowledge, and ability to follow directions (Figure 5.1).
3. Recognize potential structural barriers to program usage and quality of programmin


 to Improve the Workforce-Development Pipeline to Meet West Virginia’s Energy-Sector Needs
The Workforce-Development Pipeline for West Virginia’s Energy Sector
The analyses undertaken in this study revealed that a number of energy-sector training opp


s
  Students completing
regional consortia
providing adequate
RAND RR812-6.1
External context that may influence the flow of talent through pipeline:
• State, local, federal education and labor policies • Regional economy
Summary of Findings and Reco


d a central component of any workforce-development system (Alssid, Goldberg, and Schneider, 2011). Therefore, CTCS could be the central purveyor, or intermediary, for energy-sector career pathways, structur- ing and linking education and training off


 that meet their needs.
• Employers should establish formalized internship programs that offer hands-on, practical experience for students across several energy-related CTCS programs. Having such a joint program between employers and training institu


s to engage with and help students in need. We therefore recommend that CTCS provide stronger career counseling and reach out to local social service pro- viders and community-based organizations to help its student popula- tion with child care, tran


evelopment in West Virginia
development program fills immediate skills needs as opposed to long- term workforce needs. This relationship provides an opportunity to tap the adult workforce already in the labor pool but in need of credentials or a degr


mend the following measures:
• Include formalized internships, cooperatives, or apprenticeships with partner employers to provide on-the-job training as a way to enhance the practicality and usefulness of education programs.
Students also noted that


ess of talent entering postsecondary education and training programs. West Virginia high school students’ gradua- tion rates are on par with national averages, but only about half of high school students are “meeting the standards” on state achieveme


tor Workforce Development in West Virginia
coordinating initiatives, leveraging resources, and planning for the delivery of a comprehensive workforce strategy that ensures an integrated and strategic approach in meeting the educational and training n


entation
RAND RR812-A.3
Oral Expression
Time Sharing
Trunk Strength
Speed of Closure
Memorization
Glare Sensitivity
Wrist-Finger Speed
Peripheral Vision
Explosive Strength
Near Vision
Rate Control
Fluency of Ideas
Far Vision
Visualization
Reaction Ti


 Operations General and Oil, Gas, and Assemblers Operators, Team Mining Service Unit
Roustabouts, Oil and Gas
First-Line Supervisors of Construction Trades and Extraction Workers
Electrical Repairers Laborers Installers and Construction Power-Line
Sk


◦ Data, CTCS leadership team, local need expressed by sector,
interviews with employers or students, etc.
◦ Centralized decision or local decision?
– Who participated in the program or course of study decision- making team?
• How many programs has t


e programs?
– Increase in popularity over the last five years, decreased interest in other programs?
– Have there been targeted efforts to increase the popularity of these programs? If so, please describe those efforts.
• In your professional capacit


ese technical skills? How did you arrive at this understanding?
• To the best of your knowledge in your professional capacity, to what extent does location of your institution affect the number of applicants? How did you arrive at this understanding?


If they exit the program, why do they leave? Do they enroll in another program within the institution? Do they enter the
workforce and, if so, what sector and type of job?
• What proportion of graduates is hired within three months of
graduation?
– W


ectors might stu- dents in these programs be prepared to enter? Are these sectors ones students are familiar with?
• In your professional capacity, what are the perceptions of enrolled students of the energy sector? How do these perceptions compare t


led job categories? (Note to interviewer: Be sure to retrieve information for each job cat-
egory.)
◦ Required minimum degrees or certifications (if any)
◦ Required minimum years of experience (if any)
◦ Required recertification timeframes for key sk


tionship with CTCS or any
CTEs in the state or across the region? If so, please describe this relationship: when it started, the motivation for starting the relationship, and what are the characteristics for this rela- tionship?
Hiring Process and Ch


es?
– Any other differences within workforce in retention rates that your firm has documented?
Focus Groups with Students in Energy-Related CTCS Program Discussion Protocol
Background of Participants
• I’d like to go around the room. Can you please i


st widely followed; the ten community colleges attracted 951 students to this major in 2012–2013, almost double the number of students who had studied it four years earlier. Computer Science was offered at eight different colleges, while Biology and


X x X
Mechatronics X
Technical Programs Offered by CTCS and Enrollment by Major
149
      X
      X
X
        Table C.1—Continued
X
Highway Technician
Major Appliance
Manufacturing X
Technology
Maritime
Repair
Mechanical X
Engineering
Technology
Mine


 paralegal
Manufacturing engineering technology
Mechanical engineering technology
Nursing
Institution
Kanawha Valley Mountwest New River Pierpont
West Virginia Northern Blue Ridge
Kanawha Valley Mountwest
New River
Pierpont
West Virginia Northern Bri


l
WESTEST2 West Virginia Educational Standards Test 2
WIA Workforce Investment Act of 1998
WIB Workforce Investment Board
WIOA Workforce Innovation and Opportunity Act of 2014 WVONGA West Virginia Oil and Gas Association
WVU West Virginia University


Implementation Experiences and Evaluation Findings, Kalamazoo, Mich.: W. E. Upjohn Institute for Employment Research, 2011.
BLS—See U.S. Department of Labor, Bureau of Labor Statistics.
Bragg, Debra D., Timothy Harmon, Catherine L. Kirby, and Sujung


mmunity College Research Center, February 2013. As of June 4, 2015:
http://files.eric.ed.gov/fulltext/ED539910.pdf
Energy Providers Coalition for Education, homepage, undated. As of June 2, 2015: http://epceonline.org/
Finley, Ashley, and Tia McNair,


unity College Remedial English Students: Is the Accelerated Learning Program (ALP) Effective and Affordable? New York: Community College Research Center, Working Paper 21, September 2010. As of June 4, 2015:
http://files.eric.ed.gov/fulltext/ED512398


he Potential of the Lower-Wage Workforce,” Oakland, Calif.: Insight Center for Community Economic Development, 2010.
National Research Council, Emerging Workforce Trends in the U.S. Energy and Mining Industries: A Call to Action, Washington, D.C.: Na


 National Apprenticeship Act of 1937, August 16, 1937.
Public Law 105-220, Workforce Investment Act of 1998, August 7, 1998. As of July 6, 2015:
http://w w w.gpo.gov/fdsys/pkg/PL AW-105publ220/content-detail.html
Public Law 107-110, No Child Left Beh


ucation Statistics, NCES 2014-391, 2014. As of June 5, 2015:
http://eric.ed.gov/?id=ED544798
Tai, R., An Examination of the Research Literature on Project Lead the Way, Indianapolis, Ind.: Project Lead the Way, 2012.
Tsapogas, John, The Role of Commu


of Labor, U.S. Department of Commerce, U.S. Department of Education, and U.S. Department of Health and Human Services, What Works in Job Training: A Synthesis of the Evidence, Washington, D.C., July 2014.
U.S. Energy Information Administration, West


, D.C., July 2014.
WorkForce West Virginia, home page, undated(a). As of July 6, 2015: http://www.wvcommerce.org/business/workforcewv/default.aspx
———, “Economic and Labor Market Information,” website, undated(b). As of June 29, 2015:
http://workforc


th representatives of academic and training providers within CTCS, apprenticeship programs, a regional Workforce Investment Board (WIB), and CTCS students enrolled in energy-related programs to determine whether education and training is aligned with


   Improving Information-Sharing Across Law Enforcement: Why Can’t We Know?
John S. Hollywood, Zev Winkelman
SUMMARY Law enforcement agencies increas-
ingly demand sophisticated information technology (IT) capabilities to support their operations. Th


id, others see infor- mation sharing as a competitive advantage.
• A common concern focuses on how much RMSs and CAD systems cost, especially for smaller agencies.
• Commercial providers have reported challenges in gathering requirements.
2
 For the


tion).
For the longer term, we recommend developing a common business process that brings together practitioners and developers in identifying requirements for law enforcement IT systems. We also recommend creating a multilayer frame- work for sharin


g the largest agency in a metropolitan region maintain crates of radios to distribute during major events, but solutions and workarounds like this do exist.
However, practitioners have reported far spottier progress when it comes to sharing data. Dif


gy Standards Council (LEITSC) (2010).
4
• Why Can’t We Share? from the National Criminal Justice Association (NCJA) Initiative (NCJA, 2004).
• The Global Justice Information Sharing Initiative’s information-sharing standards packages (Global Standard


wer cost. This phenomenon extends beyond processing speed, memory size, hard disk size, or network capacity when one considers the number of sophisticated sensors that now come standard on an average commodity smartphone. As a result, however, expect


commonly called for throughout the materi- als and meetings described above. The figures show which law enforcement roles need the various types of information and the sources of that information. Figure 1 presents tactical policing information that


e IJIS Institute provides an introductory guide to connecting to N-DEx (Chawdry et al., 2013).
• National Law Enforcement Telecommunications System (NLETS). NLETS is a state-owned nonprofit organiza-
tion that facilitates more than 100 interstate dat


anges (LInX, RISS, NLETS)
National LE info exchanges (NCIC, N-DEx, SAR, LEEP, etc.)
Local RMS
Local CAD
Local RMS
State/regional info exchanges (LInX, RISS, NLETS)
National LE info exchanges (NCIC, N-DEx, SAR, LEEP, etc.)
NOTE: ALPR=automated license


te, local, and federal criminal registries
State, regional, and national protective order registries
External crime analysis systems
NOTE: MS=management system; NGI=next-generation identification; and NIBIN=National Integrated Ballistic Information N


, NCIC, SAR)
External crime analysis systems
Courts MS
Intelligent transportation system
Agency billing / accounting systems
GIS/AVL License plate reader (LPR)
Automated victim info and notification
Media outlets
HAZMAT DB
Next Generation 911 (Voice,


hanisms for checking compliance
• federal, state, and regional exchanges and repositories for sharing various types of information across agency boundaries
• information-assurance technology intended to ensure con- tinued access to, and integrity and


ard to say now, other than some groups are working on it . . .
—Adapted from a conference session on cybersecurity
Major strides have been made in the technology support- ing information-sharing. That said, there is still a great deal to accomplish.


ks missing any IEPDs shown in red (Pointers to IEPDs: Parker and Wisely, 2009; IACP, undated-a; Global Standards Council, undated; and OJP, undated-c). The blue labeling merely indicates the existence of a relevant IEPD—it does not describe the matur


 web (crime maps, info)
HR & staffing systems Fingerprinting device
Federal & state fingerprint ID systems
(e.g, IAFIS, NGI)
Mug-shot system
Case MS (Prosecutor) Jail MS
RMS/CAD
LE mobile
Fusion center systems
State & regional databases (e.g., LInX,


ta tables and systems.7
Training, Verification, and Validation. Requests for additional training material starting with introductory “read this first” tutorials and checklists have been made. Note that NIEM now offers detailed in-person and online tr


do not, as well as auditing usage. It also supports single-sign on, which means law enforcement personnel only have to sign with a set of credentials once to access information across a variety of sys- tems (OJP, undated-a). As mentioned, semantic t


ems include information-sharing; the Project Manager’s Guide includes a general discussion of information- sharing standards, including NIEM and IEPDs. The APCO International and IJIS Institute developed an updated func- tional standard for CAD syste


’ of Geospatial Markup
Language . . .”
“There are over 200 attributes to describe a person . . . pick a
new one . . .”
—Adapted from comments during Center-conducted focus groups and interviews on information-sharing
Governance issues commonly have b


standardized policy documents describing exactly how data will be used and protected after sharing. To a certain extent, lack of standard- ized policy is similar to a technology issue—right now, too many policy documents, including point-to-point in


es, and regionally adjacent RMS or CAD systems.
Similarly, Center researchers have had discussions with commercial providers on two major disincentives for develop- ers to support standardized information. The first is the cost and difficulty of supp


ware agencies are of those offerings.
Agencies are facing other costs well beyond RMS/CAD fees. These include the cost of IT staff (employees or contrac- tors) and the cost of building out the communications network. Network costs typically include a


nable to meet CJIS requirements (Gould, 2011). In discus- sions regarding cloud and SaaS models, the security of law enforcement–sensitive data is frequently discussed, including whether civilian contractors could physically manage sensi-
tive inform


es where deployment has large costs (as it does for RMS/CAD).
CONCLUSIONS—TOWARD INFORMATION-SHARING
We begin with short-term recommendations for agencies fac- ing near-term systems acquisition decisions. We make recom- mendations on near-term langua


es and regions
−interfaces to share data with the local prosecutor’s office and jail
• confirmation of compliance with the above, such as demonstrations, test results, and references. Over time, as standards and standards testing mature, agencies wil


r information-sharing are sufficient.
It is also useful to identify what must be strictly imple- mented and what is flexible. Too much inflexibility will come at a high price and result in inabilities to share information with other agencies; at the


atly lower costs.
Technology Development and Policy Recommendations
We describe technology and policy steps that would help address the barriers to information-sharing, building on the progress that has been made to date.
Technology Recommendations
I


on center systems
• GIS/AVL location (There is also a critical need for a CAD-
GIS/AVL interface.)
• license plate readers. (There is also a critical need for a
CAD-LPR interface. Note that there is some work going on to create an LPR data-sharing st


and what are key problems in both
situations
−what sorts of information they need during routine and
stressed conditions, and with what attributes
−what works well and what needs improvement about
what they have now.
• periods of observation in which


g technical references and case studies, but is just the start of producing a coherent cur- riculum for information-sharing and safeguarding, as well as mechanisms for delivering it (e.g., portals, e-learning sites).
• determining and employing busin


necessary for that state or region
• language on privacy and civil rights. This should specify
defaults on who will have access to specified data, for what purposes, how the usage will be audited, and how long the data should be retained. Special pro


ndard” framework for future • Super standard developed and disseminated Develop remaining critical interface standards • Critical interface standards developed and Develop a master data model for key elements of disseminated
information as part of th


ing out much of the analysis leading to the report’s specific recommendations. However, the expert practitioners on the task force determined which system-to-system links should be considered critical.
4 The police department in St. Louis, Mo., was t


hop%20-%20Technical%20Track%20-%20State%20of%20 LEA%20INFOSEC.pdf
Alaska Law Enforcement Information Sharing System, Frequently Asked Questions, undated. As of August 6, 2015: http://www.aleiss.org/FAQ.htm
ALEISS—See Alaska Law Enforcement Informatio


us/cjis/cjis-security-policy-resource-center/ view
Foy, Paul, “John Anthony Borell III, ‘Anonymous’ Member, Charged in Utah Police Hacking,” Huffington Post, April 16, 2012. As of January 24, 2014: http://www.huffingtonpost.com/2012/04/16/john-anthon


al Guide to Getting Started, 2013. As of October 2013: http://www.intrado.com/sites/default/files/documents/NextGen%20 9-1-1%20The%20Essential%20Guide%20to%20Getting%20Started. pdf
Kimball, L. R., Plan for Next Generation 9-1-1, NG9-1-1, January 2011


nforcement_agencies_
Morgan, Kenneth E., Computer Aided Dispatch Technology: A Study of the Evolution and Expectations of CAD and a Comparative Survey of CAD in the U.S. Fire Service and the Clark County Fire Department, dissertation, University of N


8, 2014:
https://it.ojp.gov/default.aspx
OJP—See Office of Justice Programs.
Ott, Michael, and Aaron Gorrell, “CAD/RMS Implementation in Palm Beach County Sheriff,” presentation to the 2013 IACP Law Enforcement Information Management Conference, Scot


ergency Communications Task Force, Ashburn, Va.: IJIS Institute and Association of Public Safety Communications Officials– International, 2013.
Wyllie, Doug, “SaaS Technology Helps Mid-Sized Ky. PD Fight Crime,” PoliceOne, Feb. 23, 2013. As of Octobe


 criminal justice system, including violence, policing, corrections, courts and criminal law, substance abuse, occupational safety, and public integrity. Program research is supported by government agencies, foundations, and the private sector.
This


 Managing Water
Quality in the Face
of Uncertainty
A Robust Decision Making Demonstration for EPA’s National Water Program
Jordan R. Fischbach, Robert J. Lempert, Edmundo Molina-Perez, Abdul Ahad Tariq, Melissa L. Finucane, Frauke Hoss
 CORPORATION
F


tice, Infrastructure, and Environment, a divi- sion of the RAND Corporation dedicated to improving policy and decisionmaking in a wide range of policy domains, including civil and criminal justice, infrastructure protection and homeland security, tra


T Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4. Modeling Schematic for North Farm Creek Case Study . . . . . . . . . . . . . . . . . . . . . . . . . .
4.5. Range of Climate Projections Used in This Stu


. . . . . . . . . . . . . . . xviii
S.4. Futures in Which TMDL Targets Are Met, by Plan and Pollutant for
NorthFarmCreek...................................................................... 2.1. Attributes of Adaptive Decision Strategies. . . . . .


d Use Scenario Characterizations Used in Scenario Discovery. . . . . . . . . . . . . . 110
C.3. Characterization of Climate Scenarios (2055–2065 Hydrology) . . . . . . . . . . . . . . . . 114
ix
x Managing Water Quality in the Face of Uncertainty
D.1


tes the pursuit of this goal. USEPA and its partners have often grappled with imperfect data regarding the hydrology, pollution sources, and pollutant flows in the water bodies they seek to protect. This uncertainty has been significantly exacerbated


nalytic approaches do not necessarily facilitate the development of robust and adaptive plans due to a lack of appropriate methods for using uncertain simulation model results as part of forward-looking analysis. Climate change and the opportunities


rs that dis- tinguish futures in which TMDL plans meet and miss their goals. In the fourth step, analysts and decisionmakers identify and evaluate ways to improve plans to increase the range of futures over which they succeed.
This overall process ai


age delivered loads of nitrogen, phosphorus, and sediment from the Patuxent River to the Chesapeake Bay—and focuses on pollutant loads from the urban sources. We also considered the cost of implementing best management practices (BMPs) as a performan


ents implemented as specified in Maryland’s WIP) were simulated in this study in 259 different futures. Each future represents a single realization of the model reflect- ing one set of assumptions about future climate-influenced hydrology, population


 average precipitation would need to stay constant or decline and impervious area would need to remain at the mid- to-low end of the plausible range to consistently meet the nitrogen TMDL with the Phase II WIP implemented as currently constructed.
A


lysis suggests that future climate change could significantly increase pollution loads in North Farm Creek (by 30–60 percent for nitrogen, and 85–200 percent for phosphorus, respectively) under Current Management.
Table S.4 shows that the modeled BMP


to meet its goals even if the green infrastructure worked as well as projected.
The information from this vulnerability analysis can help inform the design and implementation of the North Farm Creek adaptive management plan. Monitoring the two factor


over a wider range of assumptions about their effectiveness. The treatment of inputs in both models could also be streamlined to better integrate with simulations of a more complete range of biophysical and socioeconomic processes and to facilitate


ffective risk management for the nation’s overall efforts to improve water quality. RDM and similar analytic decision support methods can expand the range of conditions in which water quality implementation plans will be successful—that is, they can


Gopal Bhatt, Matt Johnston, and Peter Claggett for helping to scope the study, providing the water qual- ity modeling and supporting its use in this setting, and for feeding in a variety of key model inputs for the Robust Decision Making analysis. Fo


ural Research Service U.S. Environmental Protection Agency
U.S. Geological Survey
watershed implementation plan
Weather Research and Forecasting model
uncertainties, policy levers, relationships, and performance measures
CHAPTER ONE
Water Quality Dec


ended to make plans more robust. Overall, the report suggests that RDM-based analyses can help USEPA and its partners successfully manage uncertainty by developing flexible and robust plans within a process designed to facilitate stakeholder input an


source of uncertainty potentially affecting future water quality and the effectiveness of actions intended to improve it (USEPA, 2010c). Traditionally, water quality planning has assumed that future streamflow, precipita- tion, and other relevant hyd


many types of significant and irreducible uncertainty.
Robust Decision Making
To address this uncertainty challenge, the USEPA Office of Research and Develop- ment (ORD) asked RAND to consider how RDM methods might help USEPA/OW better manage climate


inty 5
when setting drinking water standards, water quality for the nation’s water bodies, and standards for TMDLs, the agency follows a recursive process of monitoring, analysis, public input, decision, and review (see Appendix A). RDM and related m


A and its state, tribal, and local partners to develop robust and flexible TMDL implementation plans in the face of climate change and other hard-to-predict uncertainties. The approach described is designed to support a transparent, objective, and sc


agement practices on similar plots of land in order to gather scientific data on the most effective practices. The latter, some- times called passive adaptive management, might involve a water management agency that pursues a portfolio of investments


responses to those vulnerabilities and decide whether and when to adopt these responses.
RDM thus differs from many of the decision processes currently used by USEPA in that it employs a “backward” analysis. Rather than beginning with an agreed upon


nhibit efforts to gen- erate consensus when the choice of any single distribution is insufficiently supported by the science and when different distributions are correlated with the differing economic interests and ethical values of the parties to th


process of deci- sion support, in which parties to the decision deliberate on their objectives, options, and problem framing; analysts generate decision-relevant information using the system models; and the parties to the decision revisit their objec


 plans.
 12 Managing Water Quality in the Face of Uncertainty
Figure 2.2
Notional Scenario Discovery Analysis
    Uncertain Parameter 2
RAND RR720-2.2
Coverage: 88% Density: 82%
 Uncertain Parameter 1
and throughout the remainder of this report, we u


he plan is vulnerable.
Facilitating Robust Adaptive TMDL Implementation Plans
To understand how RDM and the scenarios it helps to identify might facilitate robust and adaptive TMDL planning, it is useful to differentiate explicitly between three attr


 of the plans themselves and the processes by which the plans are created and implemented.
Recent years have seen a growing interest in formalizing the concepts, methods, and tools for developing and evaluating adaptive strategies (Haasnoot et al., 2


unities) of near-term plans and potential responses to those vulnerabilities.
Specify signposts that indicate need for plan adjustment and contingent actions to take in response to those signposts.
Combine management of multiple elements of a system


zed decisionmaking, tend to relate to decision support processes and the institutions in which they are embedded.
RDM has demonstrated success in supporting the development of plans with the first three attributes listed in Table 2.1. RDM is forward


 Management 17
RDM process shown in Figure 2.1 and provide only initial explorations of potential new planning options and the trade-offs among them.
Overall, RDM and related analytic approaches may offer the potential for an improved approach to TMD


res, which can diminish the capacity of bay grasses to grow (Najjar et al., 2010).
After a series of discussions with Chesapeake Bay Program representatives, RAND and USEPA identified one key tributary of the Chesapeake Bay, the Patuxent River, as a