fewer refinery-worker deaths
 –changes in employment
 –uninterrupted fuel supply
 –uninterrupted employment in the oil sector.
We could not estimate some costs because of a lack of data. These include crimi -
nal liability charges, reputation damage,


y in the refinery industry. Despite this, we believe that it is possible to estimate the
amount of revenue lost in at least some types of refinery incidents, as we describe later
in this report. Refiners were similarly unable to identify benefits o


s. They further attempt to elicit enough context for the answers to allow us to identify which costs are marginal costs that the regulations add (these are
the focus of this study) and which are total or historical safety-related costs. They fur -
t


demand in one sector will stimulate
demands for inputs across other regional sectors, generating additional demand.
Industry and academia use IMPLAN widely. In recent studies, the oil and gas
industry has used IMPLAN for macroeconomic studies in Ca


 economies of the United States are usually calibrated using the IMPLAN data because the model assumes that the
IMPLAN data are in equilibrium. One of the main advantages of using a CGE frame-
work as opposed to an input–output framework is the abil


or which we have data of reasonable qual -
ity). We developed an estimate of these costs by examining the 1999 Tosco incident, the 2012 Chevron Richmond incident, and the 2015 ExxonMobil Torrance incident.
For the Richmond case, we obtained data on


 to support the assertion that Pp is lower than Pc.
However, we do not find evidence that the consequences of MRIs will be reduced
under the proposed regulations, i.e., that Cc – Cp > 0. The costly major incidents that
have occurred have varied sig


the pro -
posed regulations’ costs, we excluded any safety-related initiatives already under way that the regulations do not directly mandate.30    Cost–Benefit Analysis of Proposed California Oil and Gas Refinery Regulations
Methods Used to Obtain A


at the refin -
ers actually reported as a “best” or average cost estimate. We take the 10th-percentile
(second-lowest) and 90th-percentile (second-highest) estimates as the likely lower and
upper bounds for this cost. Note that most estimates clust


t cost to comply with them. We have incorporated all answers regarding ongoing cost of com -
pliance into the estimates presented here. Differences in opinion along these lines have Figure 4.1
Estimated Annual Recurring Program Costs, in Dollars
RAND


sts of the
proposed regulations, with most of the cost being the ongoing operational costs of run -
ning facilities as the new regulations would require. Under most refiners’ estimates, the first one to five years could cost more than the following


nt reporting by all California refineries,
would be insufficient to provide statistically significant results.36    Cost–Benefit Analysis of Proposed California Oil and Gas Refinery Regulations
Classification of Refinery Incidents
Local government o


mbustibles that is more than
5,000 pounds1
Characteristics of Major Refi  nery Incidents
Th  e defi  nition of a level 3 MCAR above does not provide quantitative criteria that can
be used to defi  ne MRI , nor does it provide information to estimat


t. Califor -
nia regulators denied this request (Penn, 2015). ExxonMobil began to import spe-
cially produced batches of gasoline from overseas refineries capable of producing the
California blend of reformulated gasoline or nearly finished Californ


 shut down, and repairs were made within a matter of days. Full production
at the refinery was possible, and production capacity was restored at the refinery in a
few days. But county leaders and regulators asked the refinery operator to shut down


 chemicals released and the area to which they could spread beyond the facility itself. Some of the
hazardous chemicals stored and used at refineries include ammonia, sulfuric acid, and
hydrogen fluoride, as well as a host of hydrocarbon molecules


haracterized by relatively few worker deaths but long refinery-outage periods. In a few cases, a large number of nearby residents have suffered significant adverse health effects, while, in
others, the health of nearby residents has not been affecte


, the ISO and Richmond ISO contained similar regulatory guid -
ance. The ISO was amended in 2006 to extend ISO requirements for facility HF
programs to include maintenance and safety culture assessment. The revised ISO also
expanded MOOC requiremen


ineries because, prior to 1999, the ISO did not exist, and, between 1999
and 2000, the ISO had not yet come fully into effect.
Refinery-Worker Deaths
Table 5.6 shows refinery incidents in California that resulted in the deaths of one or
more refine


e than 200), we can approximate key statisti -
cal measures of the binomial distribution by the normal distribution (this holds in the NISO case but not for the ISO case) (Rosner, 2015). We can employ the normal Table 5.6
California Refinery-Worker D


he MCARs that have taken place in Contra Costa County since the
ISO was enacted. CCHMP tracks MCARs each year and reports these to the public
in its annual performance report. CCHMP also maintains a website that has data on Contra Cost County–regul


 possible refinery incidents that could have released large amounts of
hazardous chemicals into the air and caused adverse health effects off-site (thereby
constituting an MRI). In addition, we used data available from CCHMP and that Randall Sawyer


    49
refinery-years or samples available for the NISO case (237 refinery-years). The normal
approximation reveals the CI to be 0.032 to 0.094.
We cannot use the normal approximation to determine the CI for PMRI (ISO)
because of the small size of


in the rest of the United States. A premium is paid for California gasoline because of the unique California formulation requirements for gasoline and because
large-scale pipelines do not exist from the eastern United States into California. Nev -
e


ance in the California gasoline prices ( R2 = 0.98). From this estimation, we can Figure 6.1
California Versus U.S. Gasoline Prices
SOURCE: U.S. Energy Information Administration.
RAND RR1421-6.1January 1,
2010January 1,
2005January 1,
2000January 1,


the fire.
1 And at the same time, the delivery
of crude oil to the refinery was probably stopped.
Figure 6.3 shows the impact that the Chevron Richmond Refinery incident had
on California gas prices. We see that, immediately after the incident, the


the
elderly, and children to shelter in place and for residents to shut down air-conditioning
units. Children at local schools were told to shelter in place for several hours. The fire
department recommended that the same precautions be taken at c


is small for the size of the industry. The best estimate of $58 million is only 0.04 percent of industry revenue not devoted to inputs
and about 0.005 percent of industry revenue overall.
IMPLAN estimates total compensation in the California refiner


. For example, the impact of a shutdown of a 150,000-bpd plant for 12 months is twice the impact
of shutting it down for six months in all categories of impacts. Our base impact is to
consider a 150,000-bpd refinery being shut down for six months.


the regulations to reduce this probability.
We can use this analysis to estimate the total macroeconomic impact of the three
costly major incidents that have occurred in the state of California in the past two
decades. Here, we are interested in th


Potential Benefits to Industry
The safety improvements that might result from the proposed regulation could reduce
the number of future refinery incidents. If this is the case, refiners that implement the
measures called for in the proposed regula


be subject to fines that regulators impose after such incidents. ExxonMobil was recently fined $566,000 in conjunction with the 2015 incident. The
CSB investigation into the cause of the incident continues. So regulators might levy
additional fines


exposure to the hazardous smoke plume from the fire. Six employees received first aid as a result of the incident, but there were no injuries (Chevron USA, 2013).
Repair Costs
Chevron was also unwilling to provide us with the final costs to repair


rmission to resume full production), we can develop a rough
estimate of the lost production at the refinery. We estimate the total value of this lost
production to be approximately $900 million. From this, we estimate the lost profit to the refiner


 on the California gas market. At today’s prices—say, $4 per gallon—its output for six months is worth about $2.8 billion. As
described earlier in this report, a production loss of this magnitude qualifies this inci -
dent as a CMRI.
We use the same


nia refin -
ers $220 million. This is a cost that could be avoided if the proposed regulations are
implemented and do, as intended, improve refinery and worker safety.
It should also be noted that the loss to the refiner is not the same as the impac


ffset the costs, while lower estimates
of expected annual loss (because, for example, of lower estimates of refinery incident frequency) must produce greater improvements in risk to be worthwhile. Figure 9.2 70    Cost–Benefit Analysis of Proposed C


te of break-even point.72    Cost–Benefit Analysis of Proposed California Oil and Gas Refinery Regulations
an expected annual cost of $1.2 billion, the critical risk-reduction values are 4.8 per -
cent in the refiner-estimated case, 1.7 percent in th


as of this writing
are that the refinery might be off-line for a full 12 months. This might lead to a sig -
nificant increase in the estimate for expected annual losses. If so, this would drive the
critical risk-reduction levels downward, making th


timates to
construct the range of estimates by calculating a size-adjusted cost for each refiner and
then taking the 10th percentile (second-lowest) and 90th percentile (second-highest)
estimates as the likely lower and upper bounds for the unit c


ber of CMRIs. Therefore, a benefit to indus -
try of the proposed regulation is that the costs of major incidents could be avoided in the future. Below, we estimate the costs of a major incident for a California refinery that suffers the incident.
Re


ajor incident rate for refineries operating
under the proposed regulation will also likely lead to
• improved reliability of systems
• reduction in certain workers’ compensation premiums
• improved community relations
• improved labor–management rel


 and find that these data provide
statistically significant evidence to support this hypothesis.
Table 10.3 summarizes our analysis of refinery-worker death rates.80    Cost–Benefit Analysis of Proposed California Oil and Gas Refinery Regulations
St


esources
  required .

  We
  are
  meeting
  virtually
  all
  of
  the

requirements
  already.

2. Marginal
  additional
  resources
  required .

  We
  are
  doing
  the
  many
  of
  the
  things

required,


s)
  in
  the
  refinery?

If
  you
  have
  calculated
  it,
  what
  is
  the
  Nelson
  Complexity
  Index
  of
  your
  refinery?

Safety
  Training

The
  new
  regulations
  require
  a
  well-‐docume


  new

requirements?

  Please
  describe
  the
  costs
  associated
  with
  these
  changes.

  Please
  estimate
  any
  cost
  savings
  associated
  with
  the
  DMR
  process
  and
  the
  actions

that
  fol


se
  describe
  the
  costs
  associated
  with
  these
  changes.

Please
  estimate
  any
  cost
  savings
  associated
  with
  the
  RCA
  process
  and
  the
  actions

that
  follow
  the
  recommendations.


regulations
  or
  just
  for
  a
  subset?

Are
  HCAs
  incorporated
  into
  the
  PHA,
  Root
  Cause
  Analysis,
  or
  Mechanical
  Integrity

program?

About
  how
  many
  HCAs
  do


eport
  and
  action
  plan
  that
  is

communicate d
  to
  employees.
  The
  regulations
  require
  that
  employees
  and

their
  representatives
  participate
  in
  all
  aspects
  of
  the
  safety
  culture


fety,
  operational,
  and

maintenance
  information.

  They
  would
  also
  state
  how
  findings
  of
  all

prevention
  elements
  are
  communicated
  to
  employees
  and
  how
  employee

participation
  is


I
  RP
  570
  Tier
  1
  and
  2
  incidents.

Is
  this
  currently
  being
  done
  at
  your
  facility?
   (For
  example,
  does
  your
  company

currently
  use
  API
  754?)

  Do
  you
  currently
  t


s.

How
  familiar
  are
  you
  with
  the
  proposed
  Human
  Factors
  regulations?

Do
  you
  currently
  have
  a
  written
  human
  factors
  program?

  If
  you
  have
  such
  a
  program,
  does


ule
  for
  actions
  to
  be
  taken.
  Structured Interview Questions    91

Do
  you
  currently
  do
  L OPAs?

  If
  not,
  do
  you
  currently
  do
  some
  other
  form
  of
  SPA?

  Do
  you
  currently


  outage
  caused
  by
  an
  external
  utility
  provid er
  please

include
  but
  specify.
  If
  there
  are
  other
  unplanned
  downtimes
  that
  occur
  due
  to
  other
  external
  factors
  please
  specify.


u
  have
  given?

  Please
  indicate
  the
  answer
  that
  describes
  your
  confidence
  level.

1. Not
  very
  confident.
  Structured Interview Questions    93
2. Somewhat
  confident.
  Significant
  uncertain


ry Regulations
Chevron USA Richmond Investigation Team, “Richmond Refinery 4 Crude Unit Incident,
August 6, 2012,” April 12, 2013. As of January 19, 2016: http://richmond.chevron.com/Files/richmond/Investigation_Report.pdf
City of Richmond, Californ


Texas Tribune , March 22, 2015. As of
January 19, 2016: http://apps.texastribune.org/blood-lessons/disaster/Bibliography    97
Penn, Ivan, “Exxon Mobil Scraps Plans for Temporary Fix to Damaged Torrance Refinery,” Los
Angeles Times , September 23,


rt: Chevron Richmond Refinery Fire , April 2013. As of January 19,
2016:
http://www.csb.gov/assets/1/19/Chevron_Interim_Report_Final_2013-04-17.pdf———, Regulatory Report: Chevron Richmond Refinery Pipe Rupture and Fire , Report 2012-03-I-CA,
May


Developing a
Risk Assessment
Methodology
for the National
Aeronautics
and Space
AdministrationCORPORATION
Daniel M. Gerstein
James G. Kallimani
Lauren A. Mayer
Leila Meshkat
Jan Osburg
Paul Davis
Blake Cignarella
Clifford A. GrammichThe Nationa


fer and more secure, healthier and more prosperous. RAND is nonprofit, nonpartisan, and committed to the public interest.
RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.
Support RAND
Make a tax-deduc


dministration
nology have on policy analysis and policy choices. The program covers such topics as
space exploration, information and telecommunication technologies, and nano- and biotechnologies. Program research is supported by government agencies


.5.
 S
huttle Cancellation/ISS Transport Cost and Schedule Evaluation
(Alternative #1) (Notional)  ............................................................ 68
 5
.6.
 S
huttle Cancellation/ISS Transport Decision Risk Assessment (Notional)  ...


was to inform NASA’s ultimate interest in evaluating specific risk postures of the agency; such evaluations are beyond the scope of this study but could be follow-on projects for RAND.x    Risk Assessment Methodology for the National Aeronautics and


cal risk assessment format, when risks are represented by the likelihood of some outcome occurring
(e.g., 10 percent of five fatalities), absolute values are very important. Once these risk values have been normalized
to a common ordinal scale (e.g


d be incorporated into analysis at various levels of fidel -
2 For example, if the value of the cost growth component is less than 5 percent, this may be mapped to the risk
scale value of 0 (low), while 5–10 percent may be mapped to a 1 (some), all


risk-reward strategies (e.g., high-risk, high-payoff versus low-risk, low-payoff), and uncertainties from forecasting technology development.
As such, this methodology provides a comprehensive approach for guiding staff
and decisionmakers through a


e ability to understand comparisons within or among them or the overall risk burden associated with a decision.
The normalization process allows for determining relative component risks, and
even risk indicators and mitigation strategies. However,


interdependencies, it is certainly possible to define, understand, and assess how various relationships among the risk factors, components, indicators, and mitigations interact. Such reflection will provide insights into how best to manage and mitig


int, only from a method -
ological perspective. For this reason, all the tables for our case study are marked as having notional data.
Moving Forward
The steps developed in the methodology provide a structured way to consider a risk-
informed decisi


of how various disparate factors contribute to overall risk to project- or mission-level success that is associated with NASA-level decisions. The specific mandate was to provide NASA’s Office of Strategy and Plans with a risk-informed decision suppo


pproaches analyzed in the study, will be summarized in this report, along with extensive citations on each of these topics.
Any organization faces a range of risks to meeting its objectives. Risks cannot be
eliminated, but they can be consciously m


roadmaps, and risk-reward strategies (e.g., high-risk, high-payoff versus low-risk, low-payoff), and uncertainties from forecasting technology development.
Study Approach
Development of the study approach ensured that the team would begin from a posi


risk factors, each with multiple components—and within those, a set of indicators that suggest the level of risk (i.e., the likelihood that a certain consequence of concern will occur) associated with that area. We chose this method of risk assessme


for every risk factor, the risk factors are then graphically displayed on a radar chart, allowing graphical comparisons to be made.
In the case study analysis section, two decisions—one past and one future—were
considered. This served to test and p


 and
application. Thus, there is no one definition of risk. The Society for Risk Analysis (SRA)
provides both qualitative and quantitative definitions of risk in their glossary (SRA, 2015). Qualitatively, risk is the “possibility of an unfortunate


s, the component would be expected to fail eight times (the expected risk).
Risk Assessment Methods
In this section, we focus on examples of different methods for assessing a risk that may
be used to obtain the lower-level inputs needed for the meth


A further disadvantage with the risk matrix approach is that these matrices are typically structured with only one consequence and one probability. In reality, most risks have multiple consequences, each with an assigned probability (e.g., a car acc


interpretation of the risk characterization; and, in the case of continuous distributions, probability distributions remove the need for assigning (what may be arbitrary) prob -
ability and consequence categories. Overall, compared with a risk matri


milarly, the Integration Readiness
Level (IRL) scale provides a measure on the maturity between systems (Sauser et al., 2009). Figure 2.3 shows the TRL scale and its acquisition phases. The concept behind this readiness level is that the current sta


a percentage of the earned value
 –c
ost performance index (CPI): CPI = EV/AC; efficiency ratio for cost variance
•
 s
chedule:
 –s
chedule variance (SV): SV  = EV – PV ; measures the conformance of actual
progress to the schedule
 –s
chedule varian


triangular distribution may be developed by asking experts how long it will take for a technology to be ready for deployment. An elicitation facilitator could ask experts for the least and most amount of time it could take, followed by the most likel


order of concern. Here, we review these three concepts.
Acceptable Risk
Once risks have been characterized, the logical next step of any individual or organi -
zation being exposed to those risks is to determine whether they are at an acceptable leve


er -
tainty may choose the sure payment, while a risk-seeking individual (someone with a high tolerance for risk) may choose the coin toss. A risk-neutral individual would be indifferent to the choice. Individuals with different risk attitudes will v


ability Distributions
Similar to the risk matrix, probability distributions cannot accommodate comparisons of disparate risks unless given a common consequence metric. But, unlike the risk matrix approach, probability distributions provide more infor


tem safety risk (e.g., helicopter crash due to material failure) and an occupational health risk to the organization’s employees (e.g., injury of workers on the manufacturing line). If the organization does not prefer to monetize all risks, or if its


f Two Disparate Risks
Attribute System Safety Risk Occupational Health Risk
Maximum possible cost ($ millions) 10 2
Most likely cost ($ millions) 5 0.01
Expected frequency per year 0.25 10
Expected number of fatalities per occurrence 1 0.05
Uncertain


bute weights (Keeney, 1974). In addition to linear models, utility functions may take on nonlinear forms. An applicable nonlinear form includes that of threshold values for specific attributes to indicate their acceptability levels. For example, if t


-
ple, consider three weights w
1, w2, and w3. Using an absolute pairwise judgment method,
we may assert that attribute 2 is twice as important as attribute 1, and attribute 3 is three times as important as attribute 1. The following set of equation


the case studies presented in Chapter Five. Additional components or indicators may exist for each risk factor but may not be relevant for our case stud -
ies. Our methodology does allow the choice of other components and indicators where appropriate


while others may only be able to be characterized qualitatively. To ensure missions are able to operate as planned, a number of components should be analyzed as part of supply-chain risk. These include the following items:
•
 a
vailability of materia


ital projects, though different components may exist for each. 26    Risk Assessment Methodology for the National Aeronautics and Space Administration
NASA’s Office of the Inspector General released a report in 2012 detailing some
cost and schedule


e delays faced by the program prior to the first flight of Columbia  in
1981. Original plans called for the shuttle to begin flying in 1977, four years earlier
than what was achieved (Armagh Planetarium, 2013). This would have allowed the shuttle t


y on work. Federal funds must be used within a relatively short period of time; thus, many NASA programs and projects are well-detailed for two years’ worth of budget. Within those two years, managers can assess the stability of the funding and there


her modifications to a new aircraft. The following staff skills and experience are unique to flight research:
•
 r
esearchers
•
 a
irworthiness and flight safety staff (particularly for NASA-unique safety
standards and procedures)
•
 e
xperimental t


 c
ost of talent
•
 a
daptable skill mix/adaptability to changing missions
•
 t
raining programs in place.
Technical Expertise
NASA needs skilled individuals who understand and have responsibility for all tech -
nology, with support pyramids within


asses the strength of leadership, congressional interest, and funding dispersion. Leadership and culture is different at each center.
The current organization of NASA is the result of decades of development. NASA
HQ in Washington, D.C., is supported


g of individual locations
•
 c
ultural differences between locations.
Strength and Interest of Leadership
Leadership strength affects an organization. Leaders give organizations direction, set expectations, and provide an interface to the public. The


, Navy ships with a well deck (a large opening in the rear of certain ships that can be flooded so equipment and ships can float in and out of the host vessel) are required for safe process -
ing and transport of Orion capsules after splashdown (Figu


 and Cruise Emissions II
(ACCESS-II) testing? Partnerships provide NASA projects and programs with the potential for higher aggregated funding and scope, but also rely on foreign govern -
ments to provide what NASA needs. Alternatively, NASA may pa


budget cuts. Foreign government agencies face similar risks. Private institutions, which rely on the support of shareholders or directors, have their own rules for operations. Every organization has its own strategic plan. Working with NASA may or m


uld internally analyze the possible
priorities.
Accounting for political risks should also account for changes in national strategy
and goals. The national strategy toward space exploration, aeronautics research, and science can change, meaning ex


er programs.
Documentation for Key Technology Cost and Schedule Assumptions
Documentation and peer reviewing of assumptions would help NASA develop more-accurate assessments.
2 This will also allow persons responsible for the key technology to
asses


her program. Hence, consideration of technology risk should consider whether a program or project assumes that a key piece of technology will be available from another program. For example, if Program A is developing a widget that Program B plans to


on top of lower-level assessments that allow for analyzing individual risk elements. The subordinate analyses will allow for normalizing and assessing the NASA-level risk associated with a particular decision or set of decisions. As such, the method


tion measures must be done to ensure that all boundary conditions are eventually met. Should a component boundary condition not be met, the overall risk is determined to be unacceptable for the entire risk factor being analyzed.
1 In the classical r


ll vary by
risk factor and component. Some will lend themselves to quantitative measures, while
others will be analyzed qualitatively. A mix of objective and subjective measures will likely be employed to evaluate each component. A number of these


 with each. Development of the Risk-Informed Decision Methodology    47
For each risk, the components are defined, the indicators identified, evaluation
methods presented, and mitigation possibilities identified. Figure  4.4 presents an
example of


ce on analysis
using the full gambit of tools ranging from cost, schedule, and performance methods such as EVM to modeling and simulation (where available) to assess a component in detail. Input for such tools will likely rely on subject-matter expe


reshold values to be /f_illed out
using subject-matter expertsCalculation of risk
and mitigation50    Risk Assessment Methodology for the National Aeronautics and Space Administration
for a component. These normalized risk values can be summed to pro


total mitigated value may also be calculated. In this case, a mitigation will be assigned to an indicator (e.g., M1 is assigned to R1) and will reduce its overall score. Thus, if R1 had a score of 4, M1 may result in a mitigated score of 3. In some


0-5) Value
(0-5) Weighted
Value
(Score X
Weight) Normalized
Value
(Weighted
   Value/
Total
Weight)
Stability of
sources ! !
Alternative sources   ! !
Materials
! !
Services
! !
 ! !
Quality
management
Total 52


if several different strategies for a particular NASA issue were consid -
ered, one could run the analysis multiple times, employing different threshold values for each strategy, to arrive at a different perception of risk. In this way, one could gra


s for one strategy and 27 for another should likely be considered as depicting equivalent risk.
Fourth, the radar chart depictions are highly dependent on the assumptions and
threshold values for the strategies under consideration. In conducting a


 The team
developing the risk indicators and mitigation strategies serves as one set of experts, while senior-level NASA personnel would be used for setting threshold values. The final sets of experts to be consulted are the decisionmakers who inter


acceptable risk category to provide a way to characterize the risk associated with a particular assess -
ment. The term “unacceptable” serves as a boundary condition indicating that the risk burden must be mitigated or a different strategy devised.


tations. These objective methods must be augmented using subject-matter expertise and elicitation to arrive at the measure for each component. As additional information becomes available, the component evalua -
tions should be updated and the overall


ology
Table 4.1 summarizes the steps for the risk-informed decision methodology. While the
steps have been presented in detail already, this step-by-step presentation provides a consolidated description of the methodology. The table is designed to e


zation and subsequent comparison of risk factors and their components. Expert elicitation is essential for establishing these values. These values should also be developed in isolation from those charged with “scoring” the risk indicators.
4d Establ


se and even various mitigation strategies. One can make ordinal decisions between options using this methodology, but should not attempt to quantify the actual intervals between them. 60    Risk Assessment Methodology for the National Aeronautics and


n the lessons learned from ISS to improve life support, crew health, extravehicular activity capabilities, radiation shielding, and other areas. Much of the mission will validate pro -
cesses, procedures, technology, and operations to be used on a hu


ctor assessments for the supply-chain and political components, and the abbreviated Case Studies    63
assessments for the technical, human capital, cost and schedule, organizational and
managerial, and external dependency components (Figure 5.1).


ed cost, and “some” if there was 10-percent cost growth. We repeated this process for all elements in the table. Because the risk of R1 was assessed as 25-percent growth being too high, it was given a score of 4. The corresponding mitigation M1 was d


.  Expected
schedule 6-month
delay 1-year
delay 18-month
delay 2-year
delay >2-year
delay 3 2 6 0.50 /uni00A0
R3: Commercial Transportation
unreliable due to less heritage . Same
reliability 0.1% less
reliable 0.2% less
reliable 0.3% les


r factor, we calculated a score of 3.95, approaching the threshold of “extremely high” risk.
After evaluating all risk factors for the ISS transport decision, we plotted them on
a radar chart. Figure 5.6 shows the overall scores for each risk facto


aintaining supply
chains Expected
schedule 1-week
delay 2-week
delay 3-week
delay 4-week
delay >5-week
delay 4 5 20 1
Budget stability Expected
stability  5%
unstable 10%
unstable 20%
unstable 30%
unstable > 35%
unstable 4 4 16 0.8


 in Table 5.1 and
Figure 5.8. As with the ISS transport analysis, the values refl  ect the unmitigated risks only. (Th  e worksheets for the three options are provided online in the supplemental notional data for Appendix C.) Th  e radar chart allow


blic-private partnershipsNASA-driven   N O T I O N A L  D A T A  O N L Y73CHAPTER SIX
Findings and Conclusions
In this chapter, we summarize our accomplishments, our thoughts on the methodol -
ogy we presented, and recommendations for future work.
A


 thinking process that will allow for clearer and more thoughtful decisions. While employing the methodology may not change decisions, gaining better understanding of key sensitivities will undoubtedly be an important outcome.
The use of a normaliza


eliminate these interdependencies, it is possible to define, understand, and assess how various relationships among the risk factors, components, indicators, and mitigations interact. Such reflection will provide insights regarding how best to manag


iated case studies validating it, conducting a more robust analysis for a future NASA decision is important. Given the work we have begun on it, applying this model to a more expanded analysis of the pending Cislunar Habitat decision may be most appr


er to meet the cost constraints:
 –U
se off-the-shelf hardware components and inherited designs to the maximum
extent possible.
 – U
se analysis and modeling as an acceptable lower-cost approach to system test
and validation.
 –L
imit changes to t


panies. Agreements between NASA and these organizations are controlled by the executive and legislative branches of the U.S. government. In the last few years, the relationship between NASA and Roskomos, the Russian Space Agency, has been the subject


ical leaders of the country
•
 R
eliance on foreign entities
•
 C
ongressional restrictions
•
 F
ederal budget process
Technical
•
 D
ocumentation for key technology cost and schedule assumptions
•
 P
ayoff of technology
•
 R
eadiness levels
•
 S
a


n make use of existing concepts.
Individual competency or skills can be tracked and assessed annually. Performing this task will cover many of the metrics that were listed earlier.
An external example of managing human capital can be seen in the U.


 U.S. Nuclear Regulatory Commission, 2005.
Ayyub, Bilal M., Elicitation of Expert Opinions for Uncertainty and Risks , Boca Raton, Fla.: CRC
Press, 2001a. ———, A Practical Guide on Conducting Expert-Opinion Elicitation of Probabilities and Conseque


.org/pubs/technical_reports/TR756.html
Davis, Paul K., Russell D. Shaver, and Justin Beck, Portfolio-Analysis Methods for Assessing Capability
Options , Santa Monica, Calif.: RAND Corporation, MG-662-OSD, 2008. As of May 31, 2016:
http://www.rand.o


st U.S. Space Station’s Re-Entry,” History.com, July 11, 2012. As of April 20, 2016: http://www.history.com/news/the-day-skylab-crashed-to-earth-facts-about-the-first-u-s-space-stations-re-entry
Hastie, Reid, and Robyn M. Dawes, Rational Choice in an


1, 2015. As of April 20,
2016:
https://www.nasa.gov/directorates/heo/scan/engineering/technology/txt_accordion1.html
Mankins, John C., “Technology Readiness Assessments: A Retrospective,” Acta Astronautica,
Vol. 65, No. 9, 2009, pp. 1216–1223.


 General, National Aeronautics and Space Administration, IG-12-021 (Assignment
No. A-11-009-00), September 27, 2012. As of April 20, 2016: https://oig.nasa.gov/audits/reports/FY12/IG-12-021.pdf
Office of the Secretary of Defense Manufacturing Techn


n Heavy Rocket Debuts in 2016,” International Business Times , July 21, 2015. As of April 20, 2016:
http://www.ibtimes.com/elon-musk-spacex-complacency-contributed-falcon-9-crash-falcon-heavy-rocket-debuts-201780992    Risk Assessment Methodology fo


. For this effort, a methodology was developed for normalizing and comparing supply chain, external dependency, cost and schedule, human capital, organizational and managerial, political, and technical risks. The goal ultimately was to develop a sing


CORPORATION
Preservation of
Affordable Rental Housing
Evaluation of the MacArthur Foundation’s
Window of Opportunity Initiative
Heather L. Schwartz, Raphael W. Bostic, Richard K. Green, Vincent J. Reina,
Lois M. Davis, Catherine H. AugustineThis


 public investment in this country’s increasingly critical affordable housing stock.
Second, we did not foresee the Great Recession. Its significant shock to the national
economy made it harder for nonprofit owners to acquire properties, and dimini


 -
thropic initiative started in 2000 and will end in 2020, allocating $187 million in grants and
loans to preserve privately owned affordable rental housing. The purpose of the evaluation is
to assess whether the initiative achieved its goals, as


ity, transportation and energy policy, and environmental and natural resource policy.
Questions or comments about this report should be sent to the project leader, Heather
Schwartz (Heather_Schwartz@rand.org). For more information about RAND Infrast


ally Reinforcing  ....... 33
 5.1.  Number of States with Preferences for Preservation  .......................................... 90
 5.2.  Dollar and Share of Tax Credit Investments in Preservation  ................................. 91
 5.3.  Numbe


e Rental Housing: The Window of Opportunity Initiative
 4.4.  Overview of Three Grantees’ Accomplishments  ............................................... 70
 5.1.  Outcomes from Window of Opportunity  ................................................


 and grants to a wide range of recipients, including nonprofit developers, loan funds, networks of non -
profit owners of affordable rental housing, advocacy organizations, state and local government agencies, and researchers. The foundation’s ultima


cy framework to achieve preservation of 1 million affordable rental homes by 2020 Modest yes
a. Increased resources for preservation Yes
b. New regulations and legislation for preservation Modest yes
c. New federal incentives for preservation No
4. G


aterialize, nor have there been new federal laws or regulations
to promote preservation.
• More groups have become involved in preservation than before WOO began in 2000.
Interviewees credited WOO with accelerating changes ongoing within the affor


ing their assets to pro -
duce cash flow, building databases, and buying out co-general partners to become sole owners of properties they intended to preserve.
A third feature of WOO is that the MacArthur Foundation’s two competitive solicita -
tion


h as housing
are vulnerable to significant macro-economic forces  that affect the broader market. The Great
Recession that started in 2007 fueled significant declines in the development and preservation
of affordable rental housing, and it slowed


bills related to preservation, each of which died in committee. Such federal action would have likely have caused much more preservation
activity. xvi    Preservation of Affordable Rental Housing: The Window of Opportunity Initiative
A fourth reason


iew by Jeffrey Lubell (Abt Associates). The two reviews helped us improve the report. Finally, we would like
to acknowledge the publishing contributions of James Torr for editing the report and Sandy Petitjean for creating and helping design the fig


s to put in place a policy framework—i.e., laws and regu -
lations, incentives, and funding—that would enable the preservation of 1 million affordable
1 Preservation can also include public-sector-owned housing, such as public housing, but the MacAr


ement
of its other philanthropic initiatives. The report’s overview of preservation and the evolution
of practices to preserve affordable rental housing is also intended for developers/owners, advo -
cates, and policymakers who work on affordable r


ivity post-WOO and to identify lessons learned that might inform other jurisdictions interested in replicating similar initiatives.
Limitations
There are two primary limitations to this evaluation. The first and most important one is
that we do not


on defined preservation as “Affordable rental housing is preserved when an owner acts to keep rents affordable for low- and moderate-income house-
holds while ensuring that the property stays in good physical and financial condition for an extended p


rks, and types of preservation activity from the 1980s to 2015. Even though Window of Opportunity focused largely (but not exclusively) on the pres -
ervation of multifamily rental housing (defined as five or more rental units per property), we inclu


f federal housing subsidies go to them.
The set of U.S. federal housing policies to promote rental housing has been and remains
far less than the large federal apparatus to support homeowners. Policies and programs that
promote homeownership inclu


 we select a relatively low income of 50 percent of the area median income as a
benchmark. By this metric, Census data indicate that 68 percent of the 34 million rental units
(which is a little over 23 million homes) occupied in 1990 were affordabl


tion within their MSA. We repeat that same matching of income percentile to rent percentile for the 40th, 60th, and 80th
percentiles.
5
3 The 30 percent threshold is a common threshold for the appropriate share of income a household should devote t


, and more than 50 percent in none. Affordability conditions were acceptable in all but 3 out of 238
MSAs at the 60th percentile. Those at the 80th percentile of the income distribution could
afford an 80th percentile rental unit in all the MSAs as


white decreased. As a result, the five percentage point decrease in the share of the population that was white over the 1990s, combined with the differences in homeowner and renter pro -
file for white and nonwhite households, increased the nationa


. The net impact of the demographic changes was a slight uptick in the share of households demanding
ownership compared with rental housing. At the same time, the U.S. population grew, and
the growth in the total number of renter households from 19


 2000 that many
of the lowest-income renters would have had to pay more than 30 percent of their income to rent a home at a corresponding percentile of rental prices. Those at the 20th percentile of the
rental income distribution for their MSA woul


ffordable; it can become unaffordable due to increased rent
levels; it can become owner-occupied housing; it can become vacant; or it can fall out of the
stock. For the current analysis, we apply this methodology to the AHSs of 1991 and 2001 and tr


or preservation
because of the expiration of rent restrictions on units produced with some sort of federal rental
subsidy. To understand the supply of affordable housing, we look at changes in rents, renters’
incomes, and rent-to-income ratios ove


d LIHTC programs, administered by HUD and by the Internal Revenue Service (IRS), respectively, are relevant from a preservation
perspective, as each provides financing for private developers and owners, and in return the
owners agree to maintain th


portive Housing for the Elderly loan program (HUD)Low-interest 40–50-year loans to nonprofit owners in exchange for affordability restrictions on rents. Income eligibility rules have changed over time, but generally for low-income elderly. As of 1990


nded,
codified at 42 U.S.C., Section 1485. Section 521  refers to that section of the Housing Act of 1949, as amended,
codified at 42 U.S.C., Section 1490A.18    Preservation of Affordable Rental Housing: The Window of Opportunity Initiative
ized i


bsolescence and potential demolition.
This simple characterization captures a large majority, but not all, types of preservation
activities, which are difficult to include in a single universal definition because of the wide vari -
ety in the types


tention.
How Does Preservation Work?
As noted, there are three distinct channels by which preservation can become a need: strong
market conditions, subsidy opt-out or expiration, and building obsolescence. In this section,
we describe these channe


idy and the affordability period both typically
have a set term. For example, the subsidy and affordability requirements for a Project-Based Section 8 property continue for 20 years. For an LIHTC property, the subsidy is provided over
10 years, and


able rentals to households
at income levels previously served by that property, 14 percent were mixed-income rentals, and only 16 percent became unaffordable rentals (Blanco et al., 2015).
The Dynamics of Preservation: Building Obsolescence
Buildin


d means (e.g., market-rate loans for rehabilitation), although the added
cost of doing so must presumably be recaptured in the form of at least modestly increased rents that could nonetheless remain affordable.
Regardless of whether preservation oc


 down-side risk, 5 percent have an up-side risk, and so every year you should find
a way to deal with the “X” percent that are rolling into their risk profile. These come in
cohorts or waves that are reasonably predictable.
Preservation from the 19


converting to unaffordable market-rate rents—as one HUD field staff person put it, “convincing owners to stay in, or [else convincing]
owners [to sell] projects to other entities that would keep them affordable.” By the mid-1990s,
funding through t


edits, and volume cap
bonds. Compared with the 1990s, preservation “grew up” and got ahead of the curve by antici -
pating upcoming expirations or opt-out periods and re-underwriting properties to infuse them
with capital for building system renewa


nd this only increased to seven states as of 2000. (We revisit these data to examine 2000–2010 in Chapter
Six.)Rental Housing and the Role of Preservation    25
Conclusion
The landscape for affordable rental housing in the years leading up to the st


n illustrates four of these strategies via case studies.
Overview
The Launch of Window of Opportunity
The MacArthur Foundation first learned of the importance of preservation in 1999 when a
program officer attended a Chicago symposium at which a pa


is tour that it could issue PRIs to finance products
such as loan funds and for “walking around [predevelopment] money” for nonprofit developers/
owners to compete for with for-profit developers who had deeper pockets that enabled them to more quick


 the largest
single source of subsidy for the construction and preservation of affordable rental housing in the United States. State hous -Window of Opportunity    29
from a wide variety of housing markets that could influence preservation policy re


ts.
• 2002: MacArthur issues regional preservation leaders RFQ to select ten regional non -
profit owners.
ing finance agencies must allocate the tax credits to developers through a competitive process specified in their state-created
qualified allo


s of affordable rental housing from 2003 through 2013, according to the Joint Center for Housing Studies (1999).
In recognition of the stronger focus on policy, the second iteration of the initiative dou -
bled the budget (from $5 million to $10 mil


n strategies is provided in another section below.
To achieve the outcomes shown in the right-hand column of Figure 3.1, the WOO initia -
tive has ultimately adopted both a bottom-up and a top-down approach by funding actors who worked at state, mult


isits,
identi/f_ied:
(1) lack of working capital as a barrier for nonpro/f_it owners; (2) need
for regulatory
and legislative change to ease preservation;  and (3) variety of housing types in need of preservation.
 Vetted strategy
in 2001 with


/f_ication
(1999–2001)Designing the
initiative
(1999–2007)The strategies of window of opportunity
(2000–2020)Desired
outcomesWindow of Opportunity    33
Investments Made Through Window of Opportunity
As shown in Figure 3.1, there were ultimately se


State and
local
governmentsNetworks
of
ownersRegional
preservation
ownersFederal
government Seed loan funds and /f_inancial
vehicles for preservation
 Increase private and public
capital for preservation34    Preservation of Affordable Rental H


o build
a cadre of large nonprofit owners started with the MacArthur Foundation issuing an RFQ in 2002 for regional nonprofit preservation leaders. The intention of the open-call request (as
opposed to inviting only particular developers to apply)


 (SAHF), the Housing Partnership Network, and
NeighborWorks (to form Strength Matters). Ultimately, the MacArthur Foundation awarded
22 grants to these three networks amounting to $4.6 million during the period from 2002 to 2014. In this manner, WO


the New York City Acquisition Fund or to preservation owners
for the preservation of affordable rental housing in the Chicago area.
Reflecting the interagency structure of the The Preservation Compact (described in more
detail in a case study in C


erences on preservation
• create or expand state or city databases of subsidized rental properties
• develop early warning systems
• provide access for grantees to a subscription-based web mapping tool
• conduct studies of local housing markets
• cre


 words, was putting preservation at the center of a state and local agenda. As a
MacArthur staff person put it, “Are you making people say, ‘What about preservation? How
do we need to work together in a different way on it?’”
The foundation’s goals


at risk of converting from subsidized to unsubsidized rental housing.
Strategy to Improve Funding, Regulatory, and Legislative Context for Preservation
This strategy is the collection of the previous six strategies, plus the 20 general operating gra


er what those
strategies involved. We selected activities that we or the MacArthur Foundation believed were
successful and had the potential to be sustained post-WOO. But the chief aim of the case studies is to identify the structural features that


tive
tion Compact (TPC) and the state and local initiatives studied. These themes are discussed in
detail in each section, and summarized at the end of this chapter.
Seeding Innovative Preservation Finance Vehicles: Housing Partnership
Equity Trust


l $75 million. Prudential agreed to provide $10 million in
preferred equity, and Citibank agreed to provided $65 million in debt financing.
Having raised the $100 million, HPET management sought to deploy it by purchasing
properties that meet the


ommitment had been secured. HPET approached Prudential’s social investment division about the possibility of obtaining $10 million in mezzanine debt.
1 Pruden -
tial was not interested in providing a loan: According to our interview with this investo


 costs
than the HPET deals, the development fees associated with such deals are higher.
The Housing Partnership Equity Trust Has Been Able to Close Deals Quickly
Interviewees frequently noted the speed at which HPET was able to close deals. In the


reholder, and HPET must provide an equity return to its common shareholders. The key distinction between HPET and the typical REIT is that enti -
ties investing in HPET are looking to profit—just not by as much as if they invested elsewhere.
The key


s $127,140 per unit for this property. In other words, HPET would be willing to purchase the property at a price up to $127,140 per unit.
In contrast, the benefits from a LIHTC deal come largely from the credits themselves, as
well as from financin


ties. Thus,
the investor in the LIHTC requires an 8 percent rather than a 6 percent return. Now we can
compute the costs that a developer is willing to pay in exchange for $7,628 in rent per year:
• $7,628/0.08 capitalization rate = (hard construct


n
those of alternative real estate investments.
Lessons Learned
This case study has highlighted the structural features of HPET that have helped it recalibrate the balance of competitive power between nonprofit and for-profit developers interested i


n transactions, while also increasing standardiza -
tion of practices among preservation owners, which could ultimately help owners access larger
amounts of funding and more diverse types of private capital.
SAHF is a structure that enables informat


worked with more than one mayor, more than one HUD office, [and] generally in more than
one state.” The organizational scale of the owner members gives them the resources to frame
and address policy questions and to execute longer-range planning, w


f Affordable Rental Housing: The Window of Opportunity Initiative
ing HUD, write regulations that are conducive to good preservation work has been one of
the strengths of SAHF.
A nonmember explained that SAHF has a “unique ability as a union of the


ta collection about resident services and resident outcomes with an emphasis on identifying potential innovative funding sources for resident services.2012
Acquisition of National Affordable Housing Trust:  This financial intermediary and advisory fi


 of transactional problems required technical knowledge
by SAHF staff, as discussed in more detail below. Staff’s ability to facilitate SAHF member conversations required understanding the history of the “old [housing subsidy] programs,”
as well as


p Network (described in the first case study in
this chapter) and the Data Warehouse through Strength Matters. SAHF’s participation in Strength Matters gave other nonprofits a link to the property management and portfolio
data of some of the larges


with large
memberships, SAHF has not had to bridge disparate members’ concerns and interests.
SAHF members share an orientation toward the use of metrics, business analytics, and
data to run more effective businesses. This orientation propels an i


would have had anything like the impact we had. In the absence of adequate support, we
6 As of February 2015, SAHF’s mission statement is “to lead policy innovation and advance excellence in the delivery of
affordable rental homes that expand oppo


own organizational practices, and sets the stage
for the participation of other partners, industries, and stakeholders, through such means as pay-for-success programs. Another example, according to an SAHF member, is SAHF’s opera -
tions and mainten


pri -
vate, and nonprofit sectors established to coordinate and prioritize preservation efforts and to increase awareness of the importance of affordable rental housing preservation in Cook
County. Its central features include interagency coordinati


 TPC’s goal was to devise solutions and implement strategies that could assist developers, owners, tenants, government agencies, and nonprofit organizations concerned with preserv -
ing affordable rental housing and to develop a policy framework for


he 2008 recession, the housing market collapsed, and neighbor -
hoods struggled with foreclosed and abandoned housing. Although real estate values declined across Chicago about 35 to 38 percent on average during the recession, in communities with
a


and
their primary accomplishments. Below we discuss in detail two of the strategies often cited as
key accomplishments of TPC.
Key Accomplishments of the Preservation Compact
Of the strategies listed in Table 4.2, there are two particular initiativ


vers Loan fund.Case Studies of Window of Opportunity    59Table 4.2
The Preservation Compact’s Strategies, Actions, and Accomplishments
Challenge and Resulting Strategy Actions Accomplishments
Challenge: High operating costs.
Strategy: Manage operati


using Studies, TPC working groups monitor subsidized properties and intervenes when contracts are expiring or when building condi -
tions deteriorate.
• For properties with building condition problems, or if contracts are at risk of being lost in str


mily rentals are an underserved but significant share of the affordable housing market. Provide assistance to 1–4-unit property owners:
• Analysis by DePaul University showed that 1–4-unit properties comprised 48 percent of Chicago’s rental stock.
•


s required to invest in energy efficiency upgrades.
The Energy Savers Program was able to address some of these challenges through its pro -
gram delivery method. TPC created a working group to further expand resources for multi -
family retrofits a


analysis surprisingly showed that approximately 33 percent of the rental
housing stock in Cook County was composed of 2–4-unit buildings (DePaul University,
2012). In light of the 2008 recession, these buildings also accounted for a large portion o


ail -
able to rehabilitate and preserve groups of 1–4-unit buildings as affordable rental housing. The program is intended to help finance clusters of distressed 1–4-unit buildings in neighbor -
hoods by providing a takeout term loan for groups (9-un


 long-
term funding commitment of the MacArthur Foundation. This long-term funding allowed
TPC to convene the diverse group of government agencies, nonprofits, building owners, tenant
advocates, and other housing stakeholders many times over severa


ressing energy efficiency; the interviewee noted that the Energy Savers Program is now the largest energy conservation program of its kind in the coun -
try, and other jurisdictions are replicating it. The PRI also enabled the Energy Savers Program t


vation of Affordable Rental Housing: The Window of Opportunity Initiative
1. Seek a founding funder, such as a foundation or other entity, that is seen as a neutral
and respected broker. It is important to bringing a diverse set of stakeholders to t


, tax credits, real estate transaction-related surcharges and user fees, trust funds,
guaranty products and bond insurance programs.
The MacArthur Foundation also gained a deeper understanding over the course of the
Window of Opportunity initiative


the
coordinated preservation efforts of several state and city actors would both deepen their under -
standing of the importance of preservation and elevate its priority in the state—changing the general climate and culture around preservation. The


dized properties throughout their
states. Prior to their work, there was no single source in Ohio that stored the location of all the
subsidized properties in the state. Developers can now search for properties by city, and both Ohio and Oregon/Por


llion
Vermont Accelerate preservation of and reinvestment in affordable rental housing through targeted outreach and technical assistance for private owners and nonprofits, energy efficiency, predevelopment, bridge loans, and a demonstration using Me


ation in the Minnesota metropolitan area. Groups meet
monthly (in Minnesota) to quarterly (in Ohio) to biannually (in Oregon/Portland). Members of these working groups tend to be similar in each of the three states. State finance agencies,
HUD, USD


s. No program has come out of this work yet, but the Family
Fund and the Greater Minnesota Housing Fund have developed a joint work plan to further
investigate capacity building, data, financing, policy, and other tools to support unsubsidized affo


 it has also led to funders now consistently applying capital needs assessments and coordinating on appraisals.
Outcomes
We now turn to the outcomes grantees realized during their tenure in the State and Local Housing Preservation Leaders initiative


ars to continue the preservation work at the Net -
work for Oregon Affordable Housing. In 2012, this Network received a $3 million HUD
Energy Innovation Fund grant to support the MPower program, a wholly owned subsidiary
that provides financing to


g even before they were awarded the MacArthur Foundation funding. Minnesota started to focus on preservation in
the late 1990s as some of the earliest LIHTC developments, which had been allocated relatively
small tax credit awards, began to physica


ter-organizational group would not have been as successful without the funding for Network for Oregon Affordable Housing to ensure that
the group met on a regular basis. An Ohio interviewee noted that the operational funding
there also led them to


” Oregon/Portland inter -Case Studies of Window of Opportunity    75
viewees highlighted the trust that had developed among both staff and management group
members. Some wondered if excluding owners from these working groups had helped to build
thi


at although this may be the case, there are not enough non -
profit developers to do the “more difficult-to-develop projects,” and that the projects they need
to now address, which include “old HUD programs and old tax credit properties,” are more


preservation through either a set-aside in their QAP or a point system similar to the one used in Minnesota. The balance Minnesota Housing has struck between the burden
placed on developers to demonstrate property risk and rewarding points toward


t the benefits of investing in a research and development function of the network, with staff explicitly tasked with time to pool data across member organizations, conduct analyses, and structure pilot programs for members to test. State housing fina


ils about the interviews and the methods for analyzing their themes are included in Appendix A.
We augmented our interviews with secondary data to the extent that they were available.
To understand whether WOO recipients preserved the desired number


Increased Organizational Size
Nonprofit real estate developers’ growth in size has been fueled partly by mergers and the acquisition of property for preservation. Those with whom we spoke stressed the importance
of these changes with phrases such a


tead of one, or internal bridge
funds can be drawn on for quick acquisition. Interviewees stressed that timing is everything; as one expert noted, “Flexible and nimble and speedy tends to win the day.”
Better Ability to Partner with Resident Servic


rviewees wrestled with the question of who is the ideal organization to preserve affordable rental housing. In general, experts we spoke with described a changing nonprofit developer/
owner industry that is shedding small CDCs and moving away from a


lly painful to have to move up the income stream because you’re moving away from mission for the financial benefits. And it’s true;
it does move you away from mission, and yet you just can’t grow without that rebalancing.
Interviewees expressed con


helped them shift into more financially sustainable
practices with greater clout and policy influence. Notably, only one of the 30 owners provided 84    Preservation of Affordable Rental Housing: The Window of Opportunity Initiative
a PRI defaulted;


ting out there is for sophisticated groups, the kind of money that we really need.
The intermediate objective of the PRIs was to build better-capitalized, more-durable,
more-scaled nonprofit organizations that can, in turn, preserve more units. Inte


 an avenue by which MacArthur hoped owners could influence policy through large nonprofit owners’ practices. Interviewees felt that Mac -
Arthur’s investment in both individual nonprofit developer/owner organizations and in owner networks were mutual


e. As one interviewee noted, “there is an increasing amount of
intent at least to standardize processes and other tools of business” as a means to become more
efficient and to access more capital. Importantly, networks can serve this information-sh


of acquisition
and predevelopment funds for affordable housing developers. Several large nonprofit develop -
ers with whom we spoke indicated that predevelopment funding is no longer in as short supply, although permanent sources of capital are stil


tion Loan Fund; Southland Community Development Loan Fund (Chicago); Mas -
sachusetts Preservation Loan Fund (WOO); Vermont Loan Fund (WOO); Family Housing Fund (Minneapolis; WOO); Preservation Compact Fund (WOO); and Ohio Preservation Loan Fund (WOO


be below.
3 Section 8 overhang refers to the potential for revenue loss for a property if Section 8 rents are not recognized in locali -
ties where rent levels paid through Section 8 are substantially higher than the maximum rent levels allowed thro


ty of funding to increase the preservation of affordable rental housing
throughout the WOO years (Schwartz, 2015; National Housing Law Project, 2003) likely influenced the perspective of respondents. There was broad consensus that permanent sources


d from 2008 to 2010 and then peaked in 2011. There are many factors that could explain this dip between 2008 and 2010, not the least of which is that the odds of an owner
trying to leave a subsidy program during this time were likely less due to the


ousing Trust, many interviewees who were familiar with
WOO credited the MacArthur Foundation for funding high-impact national advocacy orga -Figure 5.2
Dollar and Share of Tax Credit Investments in Preservation
SOURCE: Data from the National Housing


 2007 2008 2009 2010 2011 2012 Outcomes of Window of Opportunity    93
preservation-related changes at the agency. Interviewees credited HUD with substantial, if
“incremental,” progress under the Obama administration to make preservation easier.
Wh


 persistently advocated for preservation policies that were in a bill called the Housing Preservation and Tenant Protec -
tion Act, sponsored by Barney Frank, that had been repeatedly introduced over several years but died in the House Financial Serv


nt for advancing preservation. Given the central role
played by state and local government—especially housing finance agencies—in preservation, one interviewee told us that this particular strategy was the foundation’s “best bet” to change
the fiel


two questions. First, was the affordable housing preservation sector larger after WOO’s 14
years of existence than it was before WOO began in 2000? If yes, the second question was whether WOO contributed to this growth. As was the case for the othe


 years for a profit “before the roof caves in,” without recapitalizing the property. Interviewees worry that such buyers, with their “short-sighted dumb money,” are
doing a “disservice” to the subsidized inventory, leaving a mess for future preserva


uncil of State Hous -
ing Agencies) were the second most frequently named, each with 18 mentions.
Several interviewees concluded that the majority of champions are at the local level—
many of them MacArthur WOO grantees or borrowers—which is where t


rvation, if not general appreciation of renting, to “balance” housing policy. We found a diversity of views about preservation’s definition and importance, suggesting that awareness of
preservation did not result in a unified understanding of the co


ndents saying they aspired to homeown -
ership (Hart Research Associates, 2015). While “the infatuation with ownership as the means of wealth accumulation” might be over, opined one interviewee, “the love affair with home ownership is ready to burst


among state and federal government
employees to WOO advocates’ education of state housing finance agencies, a greater awareness of the scope of the problem due to data advances, the proliferation of examples of preservation
done well, a greater und


policymakers that preservation of that stock was important, because there was not stock like that being built, and so you have to save it like an endangered species or else it will be gone.
Grantees described a two-step process to first try to ident


xisting subsidized rental housing stock.
Several interviewees also praised HUD for putting out more information (and more fre-
quently updated information) about subsidized rental housing through the national LIHC database (HUD PD&R, no date-a), the


ouse-
holds. This discussion covered the same terrain as the trade-offs of targeting housing aid,
which we discuss in Outcome 1 regarding nonprofit owners’ struggle to “maintain mission” while running a financially sound, sustainable business.
Wher


gree with me
on that. A lot of the people, even on the rural preservation working group call, they
want project-based rental assistance for forever. It doesn’t always make sense.”
3. Prioritize preservation of properties in “opportunity-rich areas.


nging
from the economic, to the moral, to the politically expedient. These reasons, along with exam -
ples, are listed in Table 5.2. Among the 15 justifications, cost-effectiveness was the most com -Outcomes of Window of Opportunity    103
Table 5.2


cause of physical deterioration.”
• “The risk in preservation environment is, I got housing authorities who are incompetent, I got 515 projects that have obsolete owners. I’ve got 202s and other very old properties that need renovation and reinvigora


d-ism] quite as frontally.”
• “Preservation of existing [housing] is an easier path than getting a re-zoning and going through a public process around affordable [housing]….”
9. Tenants, particularly seniors, can remain in place.• “It just makes so


o, you know, it’s just more efficient to get control of it rather than let it get lost and gone forever.”
14. Preservation is less risky than new construction.• “We do not have entitlement risk. We do not have construction risk. We do not have demand


ation means ten years into it.” The
closest interviewees came to a distilled schematic for preservation is the “bifurcated challenge” of, on the one hand, the upside risk of displacing low-income residents due to owners opting
out of maintaining th


t
an issue? Are they campaigning for Grandma? Is it poor people? I mean it’s very retail, very
basic, political.
The relative invisibility of preservation and its inherent variation, which depends on local
market conditions, tenant mix, ownership


y homeless.
Note that we do not know how many of these units would have been lost from the afford -
able stock if not for WOO. And three interviewees questioned whether WOO substantially increased the number of units preserved. Despite the wide agre


himberg Center and the Furman Center, have better approximations of which owners
renewed their subsidies, with and without a repositioning, as far back as 2004 and 2002,
respectively, but not before then and not on a national level.
In light of th


Units out that year
Units out that year as a share of existing units
Cumulative units out as a share of units ever built
1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Outcomes of Window of Opportunity    109
was sem


ded to change housing pol -
icies and practices to facilitate more preservation of affordable rental housing. The MacArthur
Foundation identified five principal barriers to achieving this goal:
• too few significant nonprofit owners who had made pr


oritized changing the nonprofit owner industry, not the much larger for-profit owner industry,
and it prioritized the policy environment over the unit count.
Interviewees continue to grapple with fundamental questions about preservation. These
dile


ver, these accomplishments did not resolve structural problems for preservation that
we believe inhibited WOO’s ability to achieve impacts at the desired scale. The largest inhibi -
tor of preservation was the economic downturn in the WOO years. Due


 choice by the
foundation to focus on large nonprofit preservation-oriented owners who, by and large, were preserving subsidized rental housing, which is most often in multifamily dwellings. The ratio -
nale for focusing on large nonprofit preservat


hereby expanding the resources devoted to preservation.
• Few funding vehicles and policy efforts to engage the largest owner type of affordable
rental housing: for-profit owners of small numbers of units.
• A lack of significant new funding sources


nsensus policy motivation for, preservation. That the initiative is not likely to reach its goal to catalyze major preservation-related policy changes speaks to the limitation of a single philanthropic initiative targeting a massively large asset suc


w subsidized rental housing for lower-income families.
Lessons for MacArthur and Other Philanthropies
Through our review of WOO and its outcomes, we conclude with six lessons for philanthropies
as they consider the design of future initiatives.
Les


ditional funds on it. The designated beneficiaries of WOO were accorded an advantage
that helped to attract outside dollars. 116    Preservation of Affordable Rental Housing: The Window of Opportunity Initiative
Also, in selecting a relatively narro


ard goals. A fourth challenge is the continued lack of data to mea -
sure preservation activity and resources to quantify changes over time. The four challenges, in combination, prevented the foundation from knowing empirically along the way whether


. population. The IPUMS is a series of data compiled by the Univer -
sity of Minnesota that allows us to follow the evolution of metropolitan area characteristics,
keeping boundaries constant.
We looked at homeownership by marital status, race/ethn


 income is matched with the unit having the second-lowest rent, and so forth,
until we reach the 20th, 40th, 60th, and 80th percentiles. Consider a household with income at the 20th percentile of the income distribution. The perfect distributional m


ille−Cocoa−Palm Bay, FL
Columbus, OH
Albuquerque, NM
Santa Fe, NM
San Jose, CA
Norfolk−VA Beach−Newport News, VA
Tacoma, WA
Fort Collins−Loveland, CO
Salt Lake City−Ogden, UT
Dayton−Spring/f_ield, OH
Columbia, SC
Macon−Warner Robins, GA
Spring/f_ield


oga, TN/GA
Killeen−Temple, TX
Colorado Springs, CO
Indianapolis, IN
Houma−Thibodaux, LA
Hickory−Morgantown, NC
Lancaster, PA
Houston−Brazoria, TX
Anchorage, AKMethods for the Evaluation of Window of Opportunity    123
Figure A.5
Rent-to-Income Ratio


015).
NOTE: Figure shows the ratio of 60th percentile gross rent to 60th percentile income in 1990.
RAND RR1444–A.780 60 40 20 100 0
PercentageMetropolitan Statistical Area
Bryan−College Station, TX
Miami−Hialeah, FL
Bridgeport, CT
Flint, MI
Daytona


L
Danbury, CT
Akron, OH
Knoxville, TN
South Bend−Mishawaka, IN
Charleston−N. Charleston, SC
Montgomery, AL
Galveston−Texas City, TX
Anniston, AL
Reno, NV
Richmond−Petersburg, VA
Greensboro−Winston Salem−High Point, NC
Youngstown−Warren, OH−PA
Oklahom


−PA
Spring/f_ield, IL
Joplin, MO
Longview−Marshall, TX
Wausau, WI
Benton Harbor, MI
Appleton−Oshkosh−Neenah, WI
Johnstown, PA130    Preservation of Affordable Rental Housing: The Window of Opportunity Initiative
Figure A.12
Rent-to-Income Ratio for t


fit into
these changes
• key remaining challenges facing the field of affordable rental housing preservation.
All interviews were audio-recorded, and virtually all were subsequently transcribed. At
the end of each interview, interviewees were asked


ere confidential. In this report, interviewee comments and opinions are not attributed to specific individuals unless permission was explicitly sought
(as was done for the case studies).
Selection of Case Studies
We worked with the MacArthur Foundat


in 2007 and continues to be an important organization for addressing preservation issues in Cook County. In addition to grants supporting the establishment and coordination of TPC
and participation in it, the MacArthur Foundation has funded between


ity Initiative
in Chapter Five), and they were also asked for permission to audio-record the interviews. All
of the interviewees consented to having their interview audio-recorded. The audio-recordings
were transcribed for use only by the research


 asked questions about SAHF and peer networks of each interviewee. The spe-
cific interview protocol for this case study covered the following topics: (1) perceptions of the most significant accomplishments of SAHF, (2) whether SAHF enabled members t


State and Local Housing Preservation Leaders Initiative Interviews
For this case study, we interviewed Robyn Bipes, director of programs and lending, Greater
Minnesota Housing Fund; Anne Heitlinger, preservation manager, Minnesota Housing;
Robert


nergy
• Francie Ferguson, director of the Multifamily Initiative, NeighborWorks 138    Preservation of Affordable Rental Housing: The Window of Opportunity Initiative
• James Grow, deputy director and senior staff attorney, National Housing Law Proje


isors
• James Logue, chief operating officer, Great Lakes Capital
• John Markowski, president, CIC (interviewed with Stacie Young)
• Patrick Nash, managing director, JP Morgan Capital Corporation
• Daniel Nissenbaum, managing director, Goldman Sachs


mpact’s List of Events (2009–2014)
Table C.1
The Preservation Compact’s List of Events (2009–2014)
Event Title Purpose
Green Retrofit Program
Workshop
May 2009
(50 attendees)A workshop to market the Green Retrofit Program, a program created with fe


vation of Affordable Rental Housing: The Window of Opportunity Initiative
Event Title Purpose
Preserving Chicagoland’s
Small Multifamily Housing
Stock
May 22, 2014(75 attendees)The forum examined the stock of 5–49-unit rental properties in the Chic


Housing in Cook County:
Current Conditions and Forecast , Chicago.: DePaul University Real Estate Center, 2007. As of April 20, 2016:
https://www.macfound.org/media/article_pdfs/REPORT_STATE_OF_RENTAL_HOUSING_1.PDF
Chetty, Raj, Nathaniel Hendren, a


icy Development and
Research.IPUMS— See Ruggles, Steven, Katie Genadek, Ronald Goeken, Josiah Grover, and Matthew Sobek,
Integrated Public Use Microdata Series: Version 6.0 [Machine-readable database], Minneapolis: University of
Minnesota, 2015.
J


w.law.umaryland.edu/marshall/crsreports/crsdocuments/RL34728_01282013.pdf
Preservation Compact, “Strategies That Work: Preserving Affordable Rental Housing,” Chicago, 2014. As of
April 20, 2016: http://www.preservationcompact.org/wp-content/uploads/


er 8, Low-Income Housing, Subchapter I, General
Program of Assisted Housing, Section 1437f, Low-Income Housing Assistance, January 3, 2012. As of April 22, 2016: https://www.gpo.gov/fdsys/granule/USCODE-2011-title42/USCODE-2011-title42-chap8-subchap


CORPORATION
Driving to Safety
How Many Miles of Driving Would It Take to Demonstrate
Autonomous Vehicle Reliability?
Nidhi Kalra, Susan M. Paddock
• Autonomous vehicles would have to be driven hundreds
of millions of miles and sometimes hundreds of


for autonomous vehicles, as well as for human drivers, and autonomous vehicles might perform worse than human drivers in some cases (Gomes, 2014). There is also the potential for autonomous vehicles to pose new and
1 This does not mean that 53.5 per


enables laboratory testing and verifi-cation. Curtain-style air bags, for example, are tested with a combina-tion of component tests to assess inflation time, fill capacity, and other responses in a range of temperature conditions and impact configur


rt assess the miles needed to demonstrate the reliability of this last class of fully autonomous vehicles. Therefore, we use the total fatality, injury, and crash rates of human drivers in the United States as benchmarks against which to compare auto


 driven
autonomous vehicles for a certain number of failure-free miles
and wishes to know the reliability (or, equivalently, the failure rate) that can be claimed at a particular level of confidence. Alternatively, for a given confidence C and reli


It is also known that injuries and crashes may be
significantly underreported; one study suggests by 25 percent and 60 percent, respectively (Blincoe et al., 2015). Therefore, we have also shown reference lines that could reflect a truer estimate of


ere      is 100(1 −α/2)th quantile of standard normal
distribution.10 The half-width of the CI, provides
an estimate of the precision of the failure rate estimate,
We can calculate the precision relative to the failure
rate estimate:
which simp


. The 8.8-billion mile data point corresponding to this example is annotated in Figure 2.
These results show that it may be impossible to demon -
strate the reliability of high-performing autonomous vehicles (i.e., ones with failure rates comparable


ct, λ
alt. We set
12 The subscript on z is 1− α here because this is a one-sided hypoth-
esis test.Figure 2. Miles Needed to Demonstrate Failure Rates to a Particular Degree of Precision
Failure rate (failures per 100 million miles)Miles needed to b


te of autonomous vehicles. This is shown on the horizontal axis and de/f_ined as a percent improvement over the human driver failure rate. The comparison can be made to the human driver fatality rate (blue line), reported injury rate (purple line), e


take 518 years—about a half a millennium.
Figure 4 shows how many miles fully autonomous vehicles
would have to be driven to demonstrate with 95% confidence and 80% power that their failure rate is A % better than the
human driver failure rate. The


f human driver failures. Second, injuries and crashes are significantly underreported and there is conflicting evidence about the rate of underreporting. Experiments in which injuries and crashes are accurately recorded could yield different rates. T


esting (Nelson, 2009), virtual testing and simulations (Chen and Chen, 2010; Khastgir et al., 2015; Olivares et al., 2015); mathematical modeling and analysis (Hojjati-Emami,  Dhillon, and Jenab, 2012; Kianfar, Falcone, and Fredriksson, 2013); scenar


es?
(1) without failure to demonstrate with 95% confidence that their failure rate is at most…275 million miles
(12.5 years)3.9 million miles
(2 months)1.6 million miles
(1 month)
(2) to demonstrate with 95% confidence their failure rate to wit


ashes 2010 (Revised) , Washington, D.C.: National Highway Traffic Safety
Administration, DOT HS 812 013, 2014, revised May 2015. As of March 3, 2016: http://www-nrd.nhtsa.dot.gov/pubs/812013.pdf
Bureau of Transportation Statistics, Motorcycle Rider


logy & Therapeutics, Vol. 97, No. 3, March 2015, pp. 234–246. As of April 1, 2016: http://dx.doi.org/10.1002/cpt.59
Fagnant, Daniel J., and Kara Kockelman, “Preparing a Nation
for Autonomous Vehicles: Opportunities, Barriers and Policy Recommendatio


ons, 1982. Mathews, Paul, Sample Size Calculations: Practical Methods for
Engineers and Scientists , Fairport Harbor, Ohio: Mathews, Malnar,
and Bailey, Inc., 2010.
Misra, Krishna B., ed., Handbook of Performability Engineering , New
York: Spring


is a senior information scientist at the RAND Corporation, a codirector of RAND’s Center for Decision Making
under Uncertainty, and a professor at the Pardee RAND Graduate School. Her research addresses energy, environment,
and science and technolo


science and technology, and the concurrent effects that science and technology have on policy analysis and policy choices. The program covers such topics as space exploration, information and telecommunication technologies, and nano- and biotechnolog


Programs for Incarcerated
Parents
Preliminary Findings from a Pilot Survey—
Appendixes
DIONNE BARNES-PROBY, CELIA J. GOMEZ, MONICA WILLIAMS,
MATT STRAWN, ISABEL LEAMON
CORPORATIONFor more information on this publication, visit www.rand.org/t/RRA1


and their children; however, little is known about exactly what programs exist, how they are implemented, and the extent to which incarcerated parents participate. In this pilot study, RAND researchers sought to explore and describe the current lands


understandable, organized, relevant, and straightforward; to evaluate respondents’ ability to recall information and answer survey questions; to assess the ease of following skip patterns within the survey; and to confirm projected time estimates to


cation (e.g., courses designed to improve parenting skills); literacy development and/ or book reading programs (e.g., activities designed to connect incarcerated parents and their children through shared reading and literacy activities); written com


f that is applicable)? (enter a number)
1. Box that accepts integers
• No
 [If yes, move on to “Program Name and Description” section]  [If no, show text below]
Some facilities do not have parenting programs that meet our definition, but they m


 (select all that apply)
• Child development
• Parenting techniques
• Communication with child
• Anger management
• Co-parenting
• Parenting self-esteem or self-efficacy
• Plans to re-enter child’s life upon release
• Other (specify):
 10. D


 children:
(e.g., children participate in monthly support groups)
1. Free response
 16. Does [PROGRAM NAME] include any of the following general education courses or job training specifically for parents in the program? (select all that apply)
On


eligibility requirements (e.g., only biological or adoptive parents are eligible for the program, or only parents with children of a certain age are eligible for the program)? (select one response)
• Yes  9 • [If yes, appears on next screen] Please


 length varies
(e.g., Some parenting classes last for one month; some reading and literacy
activities last for three months)
§ Free response
• Don’t know

30. Does [PROGRAM NAME] include any supports or services for parents after they have
lef


ls, Wi-Fi) used for [PROGRAM
NAME]:
§ Free response
• Classroom space
• Visitation space
• Other resources (specify):

[Perceptions of Program]
37. To what extent do you agree that [PROGRAM NAME] is meeting its goals and
objectives? (select


tion about other parenting programs or services, supports, or resources for
incarcerated parents that were not captured in the survey. If you do not have
additional information to share, select next.
• Free response box


A Green
Costa Rican COVID-19 Recovery
David G. Groves, Edmundo Molina-Perez, James Syme, and Gabriela Alvarado
(RAND Corporation)
Felipe De León Denegri
(Ministry of Environment and Energy, Climate Change Directorate)
Juan Daniel Acuña Román and Agri


s is authorized, without any special
permission of the owner of rights, provided that the source is indicated. UNDP-Costa Rica will appreciate a copy of any text prepared
based on t his publication.
Telephone: (506) 2 2961544
http://www.cr. undp.or


COVID-19 has had a significant impact on prevailing economic conditions, particularly for vulnerable indi -
viduals and households. From 2019 to 2020, the pandemic led to a sharp increase in unemployment, from about 12 percent to more than 24 percent


 and governmental responses
in 2020. We then use a general equilibrium model of the Costa Rican economy—Plataforma de Modelación Económico-Ambiental Integrada (Integrated Economic-Environmental Modeling [IEEM]; Banerjee and Cicowiez, 2020a, 2020b; B


 The numbers in each chart indicate the percentage change relative to the no COVID-19 baseline case. The results for employed
workers are based on a calibration optimized for employment; results for value added are based on a calibration optimized f


five-year recovery trajectory; the numbers in each chart indicate the percentage change relative to the baseline. The results for employed workers are based on a calibration optimized for employment; results for value added are based on a calibratio


We mapped low-productivity jobs to low-wage and low- and mid-skilled jobs, which we refer to as “lower-skilled” jobs in this figure. Green bars represent women; orange bars represent men. Results are based on a five-year recovery trajectory and calib


show that upfront decarbonization investments of the scale
required to implement the NDP could accelerate employment and economic recovery from COVID-19, and these investments would lead to higher employment and value added by 2025 than would have o


or a green transition are very often different from current employment opportunities, and workers will require training to transition to these new jobs.
•
 Co
sta Rica should continue to develop an analytical framework for identifying, measuring, an


nization Investments Will Vary Across Sectors and Socioeconomic Groups  .............. 37
CHAPTER FIVE
Aligning Decarbonization with Broader Sustainable Development Goals  ....................................... 45
CHAPTER SIX
Conclusion  ...........


r the National Decarbonization Plan, by Lines of Action,
for the First Five, Ten, and 15 Years from the Benefits and Costs of Decarbonization Study  ....... 13
 2.
1. Va
lue Added by Sector as a Percentage of GDP, 2019  ............................


e Development Goals for Each Nationally Determined Contribution
Decarbonization Action  ...................................................................................... 661CHAPTER ONE
Introduction
COVID-19 in Costa Rica
On January 30, 2020, th


al Response to the Pandemic
At the onset of the pandemic, the Costa Rican government imposed numerous measures to reduce mobility and slow the spread of COVID-19, such as
•
 Wo
rkplace closings:  On March 9, 2020, public sector workers were asked to


es, and it was considerably more stringent than Nicara-gua’s. In each country, stringency was highest during the early period of the pandemic—April to June 2020—and then it gradually declined to a steady state through the deployment of vaccinations (


wever, declined by 4.1 percent, a loss of 1.76 trillion Costa Rican colones (about $3 billion U.S. dollars [USD]), according to the International Monetary Fund (IMF, undated; see Figure 1.4). In the early months of the pandemic, a report by the Minis


lined only modestly. The exact source of this discrepancy is not yet known but could be due in part to FIGURE 1.4
Annual Change in GDP, 2010–2020
YearAnnual GDP change (%)
Annual GDP (trillion colones)
SOURCE: Author analysis of preliminary figures f


owners, the government also temporarily reduced the origi -
nal rate of social security taxes by 5 percent.
Costa Rica’s Performance on the United Nations Sustainable Development Goals
The United Nations sustainable development goals (SDGs), adopted


ic activity, are likely to be long-lasting without mitigation. Green government investment and policies, when properly directed, can increase employment and economic development (Ianchovichina et al., 2012). These green industrial policies are sector


, or a strong state, these policies can also lead to inefficiencies and corruption (Acemoglu and Robison, 2012, 2019). There are numerous benefits to public infrastructure investment, including increased private sector productivity, a higher rate of


 S.1.A Green Costa Rican COVID-19 Recovery: Aligning Costa Rica’s Decarbonization Investments with Economic Recovery
12economic and infrastructure sectors of Costa Rica’s economy. Table 1.3 summarizes these investments, and
Table C.1 (in Appendix C)


 as well as in transport and circular economy (Figure 1.8).
More recent estimates of the total investment required to both support ongoing economic development
and decarbonization in the transport sector (South Pole Carbon Asset Management [hereaft


e Effect of COVID-19 and
Decarbonization Investment on Costa Rica’s Economy
Costa Rica’s Economy: More Than Just Tourism
Costa Rica’s economy is ranked 13th out of 40 in Latin America and the Caribbean, with a GDP of $63.9 bil -
lion in 2019 (World


a). Petroleum oil and diesel accounted for about $1.4 billion of imports, followed by textiles and clothing, medications, and automobiles at around $500 million each. Most employment comes from commercial (retail trade) and ser -
vice provision secto


l Costa Rican market. This distinction is relevant because it deter -
mines how much an economic sector can expand in response to higher capital investments; export-oriented
1 IEEM was provided to the study team by BCCR, through a partnership with O


is report.A Green Costa Rican COVID-19 Recovery: Aligning Costa Rica’s Decarbonization Investments with Economic Recovery
18would incorporate additional information about how the relative share of jobs for women and men within
a sector might change


, we assume that these parameters are exogenously determined even though these parameters are deter-
mined by economic conditions and policies. Such consideration falls outside the scope of this study. FIGURE 2.2
Framework for Modeling COVID-19 Effec


metric it was possible to match the behavior of the
model to the statistical reports produced by INEC. Future work can take this approach further to consider both value added
and the demand for intermediary inputs as metrics for calibration. This a


use existing prepandemic economic models have been configured and calibrated to account for more-common economic conditions. Further research and model development will allow IEEM and similar models to reflect COVID-19 effects more closely. Despite t


tion and the economic census data. Estimated changes in these economic indicators using the economic census data are calculated by comparing census data between 2019 and 2020. Estimated changes in these economic indicators using the numerical results


collaboration of multilateral organizations, such as multilateral development banks or the Green Climate Fund. However, further research is needed to understand in more detail how these investments can be incentivized through the deployment of publi


).
The South Pole studies estimate total investment costs for the NDP, including those that would have been
otherwise incurred in BAU conditions; they do not specify the incremental costs associated with implement -
ing the NDP. For the transport l


Circular economy 1.20a1.20a1.20a
Livestock 0.06a0.78c1.56d
Agriculture 0.08a0.42c0.84d
Forestry 0.12a0.92c1.84d
Total 4.85 6.17 8.29
a Cost derived directly from Groves et al., 2020.
b Transport NDP net cost based on South Pole, 2019, estimates of


 -
anced calibration case, which simulates lower employment losses. In both cases, the differences in employ -
ment between no COVID-19 and with COVID-19 conditions lessen over time inversely proportional to the
assumed recovery time. For both calib


y to leave the workforce to care for sick family members or to support children who are no longer going to school due to school closures.
To better visualize the changes in employment by skill classification and gender, we show total employed
worker


Between COVID-19 Conditions and the No COVID-19 Baseline, by Gender and Skill Classification
Gender Skill ClassificationDifference in 2020
Jobs Compared
with 2019 Jobs
(Absolute and
Percentage)Difference in Jobs in 2025 Compared with the No COVID-


 as measured by value added (i.e., a decline of 4.3 percent in 2020 versus the Central Bank’s estimated FIGURE 3.4
Lower-Skilled Jobs, by Gender, and Change in Lower-Skilled Jobs, by Gender, 2019–2020 and
2025, With and Without COVID-19
Scenario
No


s:
• The significant declines in employment due to the COVID-19 pandemic that have been observed in
2020 and replicated by our models will largely recover; however, there will be slightly fewer total jobs in 2025 than there would have been without


m COVID-19.
• De
clines in economic activity, measured by value added, are modeled to mostly recover by 2025, regard -
less of the recovery time frame.
• In 2
025, value added would be 0.5 percent lower under a five-year recovery trajectory than it


it would have been without COVID-19 and decarboniza -
1 Higher productivity is positively correlated with higher wages in the job market, and higher wages increase the probability
of workers saving the financial resources needed to deal with unexpec


 a calibration that
balances model performance across employment and value added. The value labels indicate the percentage change relative to the no COVID-19, no investment baseline.Employment calibration
0
–100
–200–300Difference in number of emplo


n COVID-19 Recovery: Aligning Costa Rica’s Decarbonization Investments with Economic Recovery38economy, because these investments will change its underlying structure. Figure 4.3 shows how employment
could differ from the conditions in 2025 that cou


r employment than otherwise would have occurred without decarbonization spending is seen in
all but one sector and funding scenario.
3 The most significant effect is seen in the construction sector, which
could experience between 7.3 percent and 13


s of female workers, such as commerce, home employees, and hotels and restaurants. The higher spending sce -
narios show continued employment gains by lower-skilled female workers and even greater gains by lower-skilled male workers.
This modest incr


. We mapped low productivity to low-wage and low- and mid-skilled jobs, which we refer to as “lower-skilled” jobs in this figure. Results are based on a five-year recovery trajectory and calibration optimized for employment.Decarbonization
investmen


 of these decarbonization investment scenarios as follows:
• Decarbonization investments induce a faster COVID-19 recovery in terms of employment and value
added by about two years.
• Decarbonization investments lead to improved recovery outcomes


e investments.
• As c
urrently modeled, decarbonization investment only slightly reduces existing gender inequalities in
Costa Rica’s labor market because many of the jobs needed for decarbonization are from sectors with a larger share of male work


ong link between NDP benefits and SDGs, we describe how the modeled NDP benefits would contribute to or work against progress toward the SDGs. We summarize this analysis in Table 5.1 and in our subsequent discussion.
1 See the performance-by-indicato


021–2050 due to reduced transportation pollution is valued at nearly $1.6 billion.
Increased freight transport efficiencySDG 9
(industry,
innovation, and
infrastructure)Logistic Performance Index: Quality of trade and transport-related infrastruct


 connected to treated water supplies and wastewater systemsSDG 6
(clean water
and sanitation)Population using at least basic drinking water and sanitation services (%)The NDP would ensure that all households have access to clean water and that 80% o


rban solid waste generated by cities (11.6.1)
a The NDP would increase trash collection and treatment by 20% and increase the share of recyclable materials recycled 11-fold.
Agriculture Improved agricultural practices to reduce emissionsSDG 2
(zero


rted by the most decarbonization actions are SDG 12 (responsible consumption and production), SDG 8 (decent work and economic growth), and SDG 3 (good health and well-being). This reflects the significant investment in the circular economy and the ec


, Afrodescendants, and other local communi -
ties. In addition, with billions of dollars of spending required to implement the NDP, Costa Rica will need to ensure that graft and other forms of corruption are limited. The potential for special interes


chnical decarbon -
ization trajectories laid out in the NDP and provide a framework for just decarbonization centered on the most-vulnerable people, communities, and ecosystems.
Our analysis shows that the upfront decarbonization investments based o


i -
tional research is needed to improve understanding for informed policymaking:
• Co
sts of decarbonization. Detailed estimates of the upfront costs of decarbonization have yet to be
developed for all parts of the NDP. New action-by-action estimat


ining of vulnerable groups (lower-skilled workers, racial/ethnic minority groups, and women) when meeting the employment requirement of decarbonization.
Next, we suggest that employment outcomes across the sectors be carefully monitored as NDP invest


rofesionales y administrativas de apoyoCommerce
a
Professional servicesa
Communication and services, public administration and othersOtrosAdministración públicaComunicación y otros serviciosNo especificadoOther servicesPublic administration
a
Constru


ees ls_a-domest (domestic services)0.85 0.88 1.04 0.96 1.05
Hotels and restaurantsls_a-hotelrest (hotels and restaurants)0.85 0.86 0.87 1.00 1.05A Green Costa Rican COVID-19 Recovery: Aligning Costa Rica’s Decarbonization Investments with Economic Re


-IDPM and indicates the cost drivers that are multiplied by cost factors to estimate costs.
In general, costs are not estimated based on specific investments or programs, but instead they are based
on cost factors applied to sector-specific cost dri


 Investments to Sector Capital Stocks
Table B.4 shows how we linked the capital investments for the different NDP lines of action (columns)
with private investment levels, as represented in IEEM across different economic activities (rows) in Costa


ts and urban areas.
•
 By 20
50, have highly efficient
freight transport with 20% reduced emissions compared with 2018 emissions.
NOTE: LPG = liquified petroleum gas; PT = public transport.A Green Costa Rican COVID-19 Recovery: Aligning Costa Rica’s


ally Determined Contribution (INDC) previously commissioned by the Climate Change Directorate with
support from the German Agency for International Cooperation.1 That effort used document review, aug -
mented by interviews with subject-matter exper


s of the well-being of people and ecosystems.SDG1, SDG2, SDG3, SDG4, SDG5, SDG6, SDG7, SDG8, SDG9, SDG10, SDG11, SDG12, SDG13, SDG14, SDG15, SDG16, SDG17
0.2By 2022, Costa Rica will have carried out an analysis of Costa Rican green and blue jobs, whi