ble to investigate these complaints (OIG, undated). Additionally, it is not clear from its website how LASD handles anonymous complaints. Anonymous com- plaints can be valuable for documenting behavior, but that is likely the main benefit (Walker and


g polic- ing in specific ways and in accordance with a clear normative vision; 4) fact-finding; 5) the identification and promulgation of key experiences and narratives on both sides; 6) the specification of concrete changes in policies and practices


rust, legitimacy, effectiveness, and safety. For instance, Loyola Marymount University’s StudyLA project has surveyed county residents about their general perceptions of LASD in the recent past,10 but its more-detailed recent survey on per- ceptions


nt steps that LASD and the county should take. As discussed in detail in this report, the long-term solution involves mitigating the pressures and assumptions that lead deputies to create exclusive subgroups in the first place and addressing the norm


Prison Project and ACLU Foundation of Southern California, Cruel and Usual Punishment: How a Savage Gang of Deputies Controls LA County Jails, September 2011. As of May 5, 2021: https://www.aclusocal.org/sites/default/files/field_documents/ 97165508-


eflects-on-sheriff-deputies-fight-raises/docview/865443591/ se-2?accountid=25333
Barling, Julian, Kathryne E. Dupré, and E. Kevin Kelloway, “Predicting Workplace Aggression and Violence,” Annual Review of Psychology, Vol. 60, 2009, pp. 671–692.
Barr


Los Angeles County Sheriff’s Department, Knock LA, March 30, 2021. As of August 25, 2021:
https://knock-la.com/grim-reapers-jump-out-boys-lasd-gangs/
Chan, Janet, “Changing Police Culture,” British Journal of Criminology, Vol. 36, No. 1, Winter 1996,


Policing: An International Journal of Police Strategies & Management, Vol. 26, No. 2, 2003, pp. 186–207.
Crank, John P., Understanding Police Culture, 2nd ed., New York: Routledge, 2004.
Cronin, Matthew A., Laurie R. Weingart, and Gergana Todorov, “D


isconduct,” Police Quarterly, Vol. 17, No. 2, 2014, pp. 103–126.
Engel, Robin Shepard, “The Effects of Supervisory Styles on Patrol Officer Behavior,” Police Quarterly, Vol. 3, No. 3, 2000, pp. 262–293.
Engel, Robin Shepard, and Robert E. Worden, “Po


nt Review Annual Report, Los Angeles: Office of Independent Review, County of Los Angeles, 2012. As of May 5, 2021:
https://oig.lacounty.gov/LinkClick.aspx?fileticket=yH9RxxIBbr4%3d&portalid=18
Getty, Ryan M., John L. Worrall, and Robert G. Morris, “


p. 216–225.
Hassan, Shahidul, Jongsoo Park, and Jos C. N. Raadschelders, “Taking a Closer Look at the Empowerment‐ Performance Relationship: Evidence from Law Enforcement Organizations,” Public Administration Review, Vol. 79, No. 3, May–June 2019, pp


ss-Lethal Force Policies: Examining the ‘Transmission Belts’ of Police Departments,” Journal of Crime and Justice, Vol. 34, No. 3, 2011, pp. 221–233.
Ingram, Jason Robert, Robert R. Weidner, Eugene A. Paoline III, and William Terrill, “Supervisory In


–306.
Klinger, David A., “Organizational Accidents and Deadly Police-Involved Violence: Some Thoughts on Extending Theory, Expanding Research, and Improving Police Practice,” Annals of the American Academy of Political and Social Science, Vol. 687, N


heriff’s Department
Los Angeles County Sheriff Civilian Oversight Commission, “Secret Deputy Subgroups,” webpage, last updated October 15, 2020. As of May 4, 2021: https://coc.lacounty.gov/Our-Work/Secret-Deputy-Subgroups
Los Angeles County Sheriff’s


nd the Use of Coercion in Police–Suspect Encounters,” Police Practice and Research, Vol. 6, No. 1, 2005, pp. 19–37.
McDowall, Almuth, Paul Quinton, David Brown, Indira Carr, Emily Glorney, Sophie Russell, Natasha Bharj, Robert A. Nash, and Adrian Coy


 How Psychological Contract Violation Develops,” Academy of Management Review, Vol. 22, No. 1, 1997, pp. 226–256.
Mujal, Gabriela N., Meghan E. Taylor, Jessica L. Fry, Tatiana H. Gochez-Kerr, and Nancy L. Weaver, “A Systematic Review of Bystander Int


Office of the Attorney General, State of California Department of Justice, “Attorney General Becerra Launches Civil Rights Investigation of the Los Angeles County Sheriff’s Department,” press release, January 22, 2021. As of May 5, 2021:
https://oag.


f the Job, Durham, N.C.: Carolina Academic Press, 2013.
Parker, Martin, “Secret Societies: Intimations of Organization,” Organization Studies, Vol. 37, No. 1, 2016, pp. 99–113.
Patton, Michael Quinn, “Evaluation, Knowledge Management, Best Practices,


y/2020-08-06/lapd-whistleblower-swat-mafia-homeless
Reisig, Michael D., “Community and Problem-Oriented Policing,” Crime and Justice, Vol. 39, No. 1, 2010, pp. 1–53.
Reuss-Ianni, Elisabeth, Two Cultures of Policing: Street Cops and Management Cops, N


. 2, Hoboken, N.J.: John Wiley & Sons, 2010. Shareef, Reginald, “Subsystem Congruence: A Strategic Change Model for Public Organizations,”
Administration & Society, Vol. 25, No. 4, 1994, pp. 489–517.
Sherman, Lawrence W., Scandal and Reform: Controll


 Vol. 41, No. 2, 2007, pp. 457–492.
Tyler, Tom R., and Jeffrey Fagan, “Legitimacy and Cooperation: Why Do People Help the Police Fight Crime in Their Communities?” Ohio State Journal of Criminal Law, Vol. 6, 2008, p. 231.
U.S. Department of Justice C


County Sheriff’s Department and Condemning His Failure to Cooperate with Civilian Oversight,” Los Angeles County Sheriff Civilian Oversight Commission, October 15, 2020. As of May 4, 2021: http://file.lacounty.gov/SDSInter/bos/commissionpublications/


Evaluation of Energy Systems Network’s Moving Forward Initiative in Indiana
Initial Program Outcomes and Potential Impacts
NICHOLAS E. BURGER, AIMEE E. CURTRIGHT, JACOB DEWEESE, HARDIKA DAYALANI, RAHIM ALI, BRITTANY L. JOSEPH
 Sponsored by the Centra


med since the launch of its first phase in 2015, to allow ESN to continue to track performance as
MF matures, and to provide insight into the potential replicability of the MF model.
This report is intended for several audiences. The first is ESN and


oving Forward Development Site Locations .......................................................... 27 Figure 2.4. Key Building Attributes and Innovative Technology Deployment for Moving
Forward 1.0–3.0 Developments ..................................


5. Efficiency Design Elements and Specifications for Moving
Forward Developments......................................................................................................... 73
Table B.6. Development Mobility and Transportation Metrics...


ble housing in Indiana outside of the standard Qualified Allocation Plan (QAP) used to approve funding for affordable housing projects. In 2015, ESN worked with IHCDA to develop a process for awarding these set-aside LIHTC funds and in so doing affor


evaluation of the impact the program had. We laid out four broad research questions that focus on whether the program was being implemented effectively, whether the program produced the desired outcomes, what factors were associated with success, an


ncorporated many of the concepts and technologies introduced through the workshops, including multiple solar PV systems and energy-efficient building designs. Some developers felt MF pushed them to adopt new concepts and technologies, while other dev


ssing the program’s core goals (i.e., outcomes) and ensuring that MF is structured, staffed, and resourced to achieve those goals:
• Clarify which program goals are the priorities and ensure the program is structured and resourced to achieve those go


y aligning energy companies and project developers.
• Build on partnerships, knowledge, and technology development as a starting place for successive phases.
• Help developers engage with subject-matter experts following the workshops in a way that s


e, Jenna Rosswurm, Alisha Jones
• MF 2.0, H38 East: Elva James, Jeff Ryan, Tyler McKee, Ronda Shrewsbury Weybright, Anna Rankin, Melanie Reusze, Donald Peters, Steven Jackson, Michelle Carter
• MF 2.0, The Union: Melissa Benton, Robin Davis, James Ta


ffordable housing produced in Indiana by integrating enhanced energy efficiency, transportation, and access to services into the housing design and construction processes. An overarching goal of MF is to reduce household spending on housing—including


where 27 percent of renter households are ELI and only 38 affordable housing units are available for every 100 households, the shortage of rental homes available for ELI renters was more than 130,000 units in 2018. In terms of cost burden, 19 percent


ent of monthly income at the VLI threshold and 42 percent of monthly income at the ELI threshold (Dolin, 2019).
8 In the following paragraphs, we report data on rental costs (i.e., rent payments). For some households, that payment will include some b


(Lafayette)
Median Income for a 4-Person Family
$75,547 $85,096 $84,412 $97,159 $94,307
30 percent of Median Income for a 4-Person Family
$22,664.10 $25,528.80 $25,323.60 $29,147.70 $28,292.10
Max Monthly Rent and Utilities, 30 percent of Eligibility


 While “other renters” spent less than 10 percent of their income on utilities, both nationally
Figure 1.2. Average Utility Cost as a Percentage of Median Household Income
SOURCE: U. S. Census Bureau (2009–2017), U.S. Bureau of Labor Statistics (2009


d, whereas other renters’ proportion spent on transport was relatively flat. Table 1.1 provides the calculation of how spending on utilities and transportation translate, on average, into a dollar amount in the counties where developments in the firs


 local level (e.g., county, city, municipality). State and local governments typically adopt these established codes and standards and make changes to reflect local building practices or EE goals
15 American Society of Heating, Refrigerating and Air-


 Code based on the 2000 International Residential Code—neither of these versions was adopted.
18 The bill would have required adoption of the latest IECC and ASHRAE 90.1 codes at that time as well as requiring that subsequent updates be adopted withi


 goals. The share of this
20 These numbers were current as of September 9, 2020. See the HUD USER (undated) database summary and related FAQ (HUD USER, 2020).
investors receive a dollar-for-dollar reduction in their federal tax liability in exchange


 MF program showed a decline in the total points assigned to “Sustainable Development Characteristics” as well as the percentage of total points for this category. Specifically, this category awarded a maximum
25 points (of 195 total points, or less


ce) was a novel concept, and the initiative served to increase visibility for and investments in energy efficiency for affordable housing (Schwartz et al., 2018). Consistent with these findings, the American Council for an Energy-Efficient Economy (A


g developers to improve energy performance and transportation connectivity. The initiative seeks to increase quality of life while also decreasing the cost of living for low- to moderate-income individuals and families in new Moving Forward developme


ble affordable housing developments where feasible.
This evaluation sought to achieve certain goals and explicitly did not seek to achieve others. We tried to gather perspectives from a range of stakeholders, primarily those involved in designing and


ion and analysis of MF, explaining how the program is designed to work. We focus in detail on the workshop element that is central to the program’s design. We discuss MF funding briefly and then develop a theory of change for the program, which repre


e MF idea, IHCDA requested and received authorization to use the general set-aside to fund and promote the Moving Forward initiative, including selection of two MF development teams in each phase2 through a competitive request for qualifications (RFQ


phase 1.0 was energy efficiency to reduce housing and transportation costs for residents, which is the core focus of MF from its inception. In contrast, in MF 2.0, IHCDA asked applicants to add a supplemental focus on poverty alleviation and economi


ection processes. In 2016, 2017, and 2018, two of the total projects each year were MF projects, constituting
$1.5 million, $1.8 million, and $1.86 million, respectively.
5 For MF phases 1.0, 2.0, and 3.0, a request for qualifications (RFQ) had to be


ful creation of a development concept, were eligible to apply for up to $750,000 in annual Section 42 credits outside of the standard points-based QAP process.8
All three RFPs for MF 1.0 through 3.0 share common objectives for the creation of “innova


the formal scope of this evaluation, the changes in emphasis for MF 2020 provide an illustrative example of shifting priorities and adding layers of complexity. Specifically, MF 2020 added a focus on populations with intellectual and developmental di


elop solutions that target stretch goals. The goal at the end of the workshop is to have a “high-level plan” that development teams could further refine and ultimately implement.
Each workshop begins with an open-minded visioning session where partic


ources of funding needed to support staffing, operations, and overhead associated with running the program. Nevertheless, we want to provide some general
12 Because these materials were created expressly to support MF, they are all ESN’s intellectual


and
13 For example, in its first year, ESN applied for $120,000 from IHCDA to support the initial round of projects (ESN, 2015a), and each phase of Moving Forward received $100,000 to support the associated workshop. We emphasize that these are not c


 the MF program or workshop process, so pro bono support was a key supporting activity.
Inputs and activities are designed to generate program outputs. Some outputs are intermediate and support other activities and outputs. For example, the minimum a


or some projects, while service coordination for other projects focused on off-site, nearby services.
 25
Aside from impacts focused on residents, MF is also working to produce general social and environmental impacts, such as reduced greenhouse gas


takeholders could use a TOC like this to identify the resources required to make it possible to achieve specific outcomes and impacts. Alternatively, they could use a TOC to assess the feasibility or desirability of outcomes/impacts based on constrai


d solar electricity (PV). Gray shading of text/icons indicates a planned feature that had not been completed at the time of this evaluation. Broadway Lofts (MF 3.1) is eligible for LEED certification but is not planning on going through the process o


 we are limited in what we can conclude
 29
about factors associated with success. In Chapter 4 we draw conclusions where possible, focusing on implementation-based outcomes.
4. To what extent is the overall success of Moving Forward—or successful el


ject was still actively conducting its initial leasing during our evaluation period, and MF 3.0 developments were still under development. Appendix B provides electricity bill data for one of the MF 1.0 developers that also manages properties and pay


 cases, we interviewed multiple individuals from a single organization at the same time, and we have counted this as one “respondent” here.
 31
Our interviews were not without limitations, partly because of Covid-19. We conducted all interviews virtu


andemic restrictions. We discuss the limited survey results in more detail in Chapter 4, and in
4 A $5 Amazon electronic gift code was included in the survey invitation letter, which the residents were able to redeem regardless of participation in th


ent possible, and include some highlights of our analysis, the details of which can be found in the technical appendixes at the end of this report. Then we turn to additional themes that emerged over the course of the research and that are also relev


le, when IHCDA added the supplemental focus areas starting in phase 2.0, having a design document, like a theory of change, would have allowed IHCDA and ESN staff to consider how the emphasis would interact with existing MF goals, what resources it w


ment costs without necessarily conferring a commensurate benefit to residents and thus, in fact, could undermine the core program goals.
The supplemental focus areas layered onto the program in MF 2.0 and subsequent phases is another example of the p


es of MF.
 36
principally the workshops.5 However, ESN does not separately budget or track costs for MF, so we were unable to look at program revenue and spending in detail.
One risk of the current funding model for MF is that the program has relied


h they successfully conducted for each phase of MF. And all developers interviewed provided one or more positive comments or observations about the workshop. Along with the positive feedback, many of the developers also offered caveats by noting limi


ogies that exceeded standard affordable housing design, but both had to back down substantially from their workshop plans, with implications for cost and project timeline.11
8 This characterization is consistent with RAND’s own observation and analys


program did not adapt to their different capabilities. For example, some developer teams had little experience with EE technologies or transportation-oriented design. In contrast, others had done extensive previous work on EE designs and had experien


path to final outcomes. Developers generally described a positive experience with the MF process, and there is some evidence of increases in developer knowledge and examples of engagement with nontraditional partners, such as community groups.
Severa


ject] to another project.” Additionally, interviewees noted that relative to the traditional LIHTC application process, MF allowed “a little bit more time to do a more intentional project and really flesh out . . . [the more extensive] partnerships.”


lar was able to include geothermal heating and cooling. The Leadership in Energy and Environmental Design (LEED) certification for the Posterity Heights Scholar House is a clear example of an outcome that followed directly from the aggressive project


ncluded this development in this tally. Water conservation: QAP points are awarded based solely on the inclusion of low/dual-flush toilets, whereas four of the five MF developments included in the tally above have multiple additional water-saving fea


is investment (to the MF residents) relative to the overall fewer units that may be possible to produce in the state of Indiana with limited LIHTC funding, which would impact non-MF households in terms of the affordable housing supply in the state ov


ons in some cases. For example, one of the MF 1.0 teams engaged with BlueIndy, IndyGo, and Uber to explore options for providing “integrated housing development with transportation” (ESN, 2016a). According to the developer, MF was their first signifi


ee mobility scores, and one scoring worse in all three. The two MF 3.0 projects both lack transit scores; on the other two scores, one is slightly better than the comparable properties while one is slightly worse. Therefore, as a group, the six MF pr


electricity generated will not directly offset utility costs because the project is not net metered. Increased awareness of these issues up-front, including more explicit discussion of barriers and challenges to solar integration during the workshop


ects remain susceptible to changes in the political environment (e.g., a changed tax incentive value mid-project).
Moving Forward developments were found to cost more than comparable developments by as little as 12 percent and as much as 100 percent


esentations on new or emerging technologies from external experts (ESN, 2018).
For one development team, the workshop provided a lot of big ideas, which they worked to incorporate and which took substantial effort to scope and design. However, in the


uding from IHCDA, for post-workshop support. ESN used the VISTA program to supplement support (e.g., through regular meetings with developers), but VISTA member capabilities were limited. On the other hand, ESN was able to conduct extensive follow-up


ly helpful. But at the end of the day, when you’re really getting ready to implement it and you’re constrained by budget and space, I don’t know how helpful it is to think outside the box, because I don’t think there was anything in that workshop tha


lopers expressed a relative lack of previous substantial experience with incorporating transportation and mobility options into their planning and design processes.
 51
future LIHTC projects. On balance, however, the developers as a group have likely


nd. Electricity bills range from $20–$39 up to $140 or more. Of the seven respondents who answered the question about how their electricity bill compares to their previous residence (and who paid their bill separately at both residences), one reporte


through our evaluation. For instance, Is MF achieving reductions in total housing costs on a life-cycle basis for residents, while still producing as many or more units as non-MF projects? Are the total benefits of MF larger than total program costs


the traditional approach to tax credit–based affordable housing planning under IHCDA by encouraging developer teams to form first, then identify a potential project and carry out the design. We do not have enough evidence to definitively say this app


ansportation space.
Revisiting the Workshop Structure and Goals
The workshop has been the central element of MF in its first five years. We found positive and negative aspects of the workshop, and we recommend taking the time to reflect on the worksh


rom earlier MF phases to inform subsequent phases. This could also be a unique opportunity for developer competitors to share lessons learned with peers and to avoid roadblocks experienced—and leverage opportunities identified—in ongoing or even earl


nefit from establishing standard metrics and information that developers would report, along with a reporting schedule. That process would allow ESN to have up-to-date information on process status and more effectively track how designs change over t


e that the added cost of engaging expertise to support innovative design may not be financially feasible or may require cost trade-offs.
Recognize that knowledge of and access to cutting-edge technology is necessary but not sufficient for real-world


redit–incentivized housing process was a goal, but the program has not been used this way formally. MF could be explicitly structured as a way to test out technologies, EE designs, financing options, or specific transit-oriented demand approaches, wh


to draw more precise conclusions about how MF improves outcomes for residents. It would also allow ESN to be able to explain to others the specific mechanics of the program: what resources are necessary and at what levels, what the most effective wo


building commissioning
Permanent affordable and workforce housing
SOURCE: ESN, 2015b.
a IAQ is indoor air quality.
b VOC is volatile organic compound. c CO2 is carbon dioxide.
d ppm is parts per million.
Stretch Goals
Meet Living Building Challenge 3


 Workshop
  Minimum Goals
Personal vehicles minimized (no more than one car for each person)
Viable transportation alternatives accepted by residents (reduce personal vehicle demand)
Paradigm change of residents
Voice of customer in design phase
SOUR


ding solar in 2021
• Unit-level metering
• Planned on-site electric vehicle charging infrastructure for residents and car- share vehicles
a •
vehiclesc • Site selection made for access to
multiple transit lines
           MF Phase 2.0: Energy Efficie


visitors
• On-site car share
• Car-sharing will be provided by Sway Mobility
• Bikeshare partnership currently in development
• Transit access
• Car-sharing will be
provided by Sway
Mobility
• Transit access
• ILFIf Petal certification
• On-site rain


; local redevelopment plan; federally assisted revitalization award; off-site amenity and facility improvement within 0.25 mile of site; tax credit per unit; tax credit per bedroom; and internet access.
b Subcategories include smart use training on-s


s across each of the relevant five geographic locations.2
 Table B.1. Moving Forward and Comparable Project Information
      ID #
MF1.1
1 2 3
MF2.1
6 7
MF2.2
Property Name
B-Line Heights
Crawford Apartments II Kinser Flats
Southern Knoll
H38 East
Ch


ercent (MF 1.1), 14 percent (MF 1.2), 12 percent (MF 2.1), and 100 percent (MF 3.2) more expensive to build than their average comp costs. As a ballpark figure, BWI indicated during our interview that development of the Posterity Scholar House was “1


ved finalized cost approximations from Pedcor and BWI. Other cost approximations are still pending for other Moving Forward projects.
a Cost estimates early in the project that incorporated Net Zero designs were in the region of $11.6 million. b Squa


category. So, among the 81 percent of developers that sought points for some type of building certification, the average score obtained was 83 percent of the maximum score possible in that category.
The categories and subcategories listed correlate w


0.050 0.055
0.060
2005 Indiana Residential Code
2020 2018 Indiana International
Residential Residential Code Code
            SOURCE: Indiana Administrative Code, Article 14, One- and Two-Family Dwelling Code; Indiana Residential Code 2005 Edition Ti


Forward Developments
      ID #
MF1.1
MF1.2 MF2.1 MF2.2
Roof Insulation
Property Name R-Value
B-Line Heights R-24
Posterity Heights N/A H38 East
The Union R-30
Broadway Lofts
Harbor Square R-54
Exterior Wall R-Value
N/A
N/A
R-21 for fiber cement sidi


ospheric conditions, and allows for an apples-to-apples comparison between buildings. These differences are emphasized when comparing buildings in different climate zones or in different years within the same climate zone. There is no obvious differe


of gas and electricity, whereby a simple unit conversion can be made to adjust common gas units of cubic feet or British thermal units (BTU) to kWh. These calculations could be used, for example, to determine cost savings as well as cost-effectivene


o MF-specific goals and outcomes from the program theory of change, including more detailed questions on energy, transportation, and comfort outcomes. The first survey (administered on lease signing) should focus extensively on the resident’s previou


s that you consider yourself to be. a. White
b. Black or African American
c. American Indian or Alaska Native d. Asian
e. Native Hawaiian or Pacific Islander f. Other
6. Thinking about all of your sources of income, which category best describes the


evious home for the same month(s)?
a. I pay more in my current home
b. I pay the same in my current home
c. I pay less in my current home
d. I did not get a separate electricity bill at my previous residence
19. How does the comfort of your current h


e, or gotten worse since moving to your current home?
a. Health care b. Child care c. Groceries d. Restaurants e. Recreation f. Libraries
g. Entertainment
h. Other city/county services
30. Which are the top three services that are most important to y


fordability Index FAQ,” webpage, undated b. As of December 23, 2020:
https://htaindex.cnt.org/faq/
CNT—See Center for Neighborhood Technology.
DOE—See U.S. Department of Energy.
Dolin, Marissa, HUD PD&R Housing Market Profiles, Indianapolis-Carmel-An


 Innovation Workshop: Final Report Prepared for IHCDA,” February 2016a.
———, “Moving Forward 2.0 Final Report: Executive Summary,” Indianapolis, December 2016b. As of December 30, 2020: https://www.in.gov/ihcda/files/Moving%20Forward%202_0%20-%20Exec


y,” PD&R Edge, Office of Policy Development and Research, Washington, D.C., August 14, 2017. As of May 6, 2021: https://www.huduser.gov/portal/pdredge/pdr-edge-featd-article-081417.html#:~:text=Home %20%3E%20Featured%20Article-,Defining%20Housing%20A


 Hernández, and Arline T. Geronimus, “Energy Efficiency as Energy Justice: Addressing Racial Inequities through Investments in People and Places,” Energy Efficiency, Vol. 13, 2019, pp. 1–14.
Lyons, Craig, “HUD Approves Demolition of Contaminated Wes


n Wilson, Who Can Afford to Live in a Home? A Look at Data from the 2006 American Community Survey, U.S. Census Bureau, 2008. As of May 6, 2021: https://www.census.gov/housing/census/publications/who-can-afford.pdf
State of Indiana, “2016–2017 Qualif


mer Expenditure Survey: Interview Survey and Detailed Expenditure Files,” data file, 2009–2017. As of January 19, 2021: https://www.bls.gov/cex/pumd_data.htm
U.S. Department of Transportation, “Housing and Transportation Affordability,” webpage, Augu


Examining Implementation of the Los Angeles County Office of Diversion and Reentry Supportive Housing Program
STEPHANIE BROOKS HOLLIDAY, ALEX R. DOPP, MALLIKA BHANDARKAR, SARAH B. HUNTER
Sponsored by The Rose Hills Foundation
  CORPORATION
For more i


cipant interviews to describe how the program is implemented, identify its key facilitators and challenges, and illustrate participant experiences. This report will be relevant to entities interested in alternatives to incarceration and permanent sup


........... 15 Figure 6.1. Demographic Characteristics of People Experiencing Homelessness and
Incarceration in Los Angeles County ................................................................................... 56
Tables
Table 1.1. Summary of Fin


(2) describe program facilitators and challenges as expressed by program administrators and key service providers; and (3) obtain participant perspectives. We attended service-provider meetings and interviewed program administrators (n = 8), key serv


s an issue. Although the program was designed to link clients to the DMH Full Service Partnership (FSP) program for ongoing mental health services support, there have not always been enough slots available for the number of ODR clients in need. ODR d


aff training opportunities. This will ensure that staff from all organizations, including interim housing sites, are prepared to serve clients who have significant clinical needs.
 Expand ODR’s focus on equity in program implementation and outcomes.


 County)
PSH permanent supportive housing
RHO Returning Home-Ohio
SAPC Substance Abuse Prevention and Control (Los Angeles County)
 xi
1. Introduction
In 2015, the Office of Diversion and Reentry (ODR) was established in Los Angeles County. Housed wi


h shows that these programs can increase housing stability and reduce the use of crisis care (Aubry et al., 2020; Gilmer, Manning, and Ettner, 2009; National Academies of Sciences,
1 Number of individuals diverted as of October 2020 (Health Services,


(Aidala et al., 2013; Thomas et al., 2020).
Other program models feature elements similar to the FUSE model, such as combining PSH with mental health treatment. The Corporation for Supportive Housing Returning Home
2 Note that we also include a preli


or example, in the first eight months of the D.C. FUSE program, 110 individuals were identified as potential participants; of those, 51 met with the program’s transition coordinator, and only 14 were enrolled in the program (Fontaine, Gilchrist-Scott


duction) than a matched
in permanent housing at one year and 86% at
respectively.
were 21 times more likely to be arrested in the
follow-up period.
A preliminary-outcomes study found that 10 of
Single-group
PSH; assertive community
treatment
incarcer


es and
who were conditionally
2.76 times more likely to commit a new offense
against another person than those released to
significantly different. RHO clients were 40%
behavioral health services
current risk of homelessness
(Fontaine et al., 2012; F


experimental study of the New York/New York program (CUSC Institute, 2019) found that the subset of participants who had a history of justice-system involvement experienced significant decreases in days spent in prison or jail (Culhane, Metraux, and


lly significant reductions (Somers et al., 2013). These findings are similar to those of a study of a Housing First program in Australia, which also found that clients in scattered-site housing experienced a decrease in the mean number of justice-sys


ame unstable during the intervention (Kerman et al., 2018). By contrast, those who obtained housing later in the program then experienced a decrease in prison time. A study of New York/New York III, which focused on individuals with substance use dis


the Los Angeles County ODR Housing Program by doing the following: (1) describe the program’s implementation, including the ways that participants are identified and enrolled, the services that are offered, and the resources that are needed to operat


ce after March 2020. Given that most data collection took place pre-COVID-19, our description of the program and its operation focuses on that time frame. However, the program has continued to evolve in response to COVID-19, and to the extent possibl


artner-organization settings), and (5) the process of implementation. The codebook was developed and refined by three members of the research team, using a combination of deductive and inductive coding. We deductively sought codes that fit within the


ews conducted by the master’s level researcher. Our protocol was reviewed and approved by the RAND Institutional Review Board. All client responses were kept confidential. It was made clear during the consent process that there was no expectation tha


services agencies under contract with ODR. They also receive mental health services, typically provided in partnership with the Los Angeles County Department of Mental Health (DMH) Full-Service Partnership (FSP) program. Individuals directly enter i


an individual’s legal history to determine whether there are factors that might bar them from candidacy (i.e., a judge is unlikely to view certain serious charges, such as murder or rape, as acceptable for diversion). If initial clinical and legal re


h the client to prepare their medications for release (e.g., shifting to a long-acting injectable medication). ODR also assigns an ICMS provider, who typically arranges to meet the client in jail, completes an intake assessment, and then requests pla


l Health, undated). However, there are a limited number of FSP slots, which means that, at times, no slots are available, especially in certain geographic areas. Given the clinical vulnerability of clients during their initial weeks in the program, O


m housing. ICMS case managers support clients’ ability to attend their ongoing probation meetings. ODR clinical staff complete and submit FSP referrals to DMH on the client’s behalf. In addition, when clients are released from jail, they typically ha


recommendation, input from interim housing providers, FSP providers, and legal stakeholders is a key consideration. ODR clinical staff review each recommended client. If a client is not approved, a plan is put into place to address the reasons the pe


 work independently with their case managers to identify units; in this case, Brilliant Corners begins their involvement when the lease is signed. The lease is a standard renter’s lease.
On behalf of ODR, Brilliant Corners pays rent for ODR participa


, it might be their permanent housing setting.
Residential Care Facility
Some ODR Housing clients have significant clinical needs that require a higher level of care. For these clients, PSH may be in the form of a licensed residential care facility,


to clear the warrant and are sometimes successful but that other times the client returns to custody. Clients may also be rearrested on new charges and return to custody.
In the case of a bench warrant or rearrest for a less serious charge, ODR will


s the program, staff members described the development of policies and practices as collaborative, with ODR willing to consider input from those directly serving clients. Second, there are regular meetings that provide an opportunity for interagency


al services, which can be difficult to achieve with such a high-need and chronically underserved client population. Indeed, the suggestion that supportive housing programs should be expected to produce cost savings has been rejected on ethical ground


he Los Angeles County Board of Supervisors in November 2021 provided the treatment and housing cost- estimates for interim housing (Los Angeles County Department of Health Services, 2020a). These estimates exceed the supportive housing costs typicall


ability hearing
Suitability hearing
County court
Per hearing
Court holds hearings to make assignments
Public Defender/Legal Aid
Conditional release, interim housing, jail release,
PSH placement
interim housing
monthly rate paid by ODR
clients to achi


onse to provider input, client needs, and the expansion to other court hubs in Los Angeles County. The program also changed in certain ways as a result of the COVID-19 pandemic. Although the majority of our data collection with staff took place befor


tive housing. In the next chapter, we examine provider perspectives on the implementation of ODR Housing.
31
4. Provider Perspectives
To explore the factors that facilitated the implementation of ODR Housing, as well as challenges to implementation,


do more to promote clients’ integration and functioning in the community, such as assisting with vocational opportunities and prosocial network-building.
 32
Other challenges related to the ODR Housing Program model were identified as well. One area


out ‘Are we keeping you housed?’ because that’s what [we do], but ‘Are you really able to thrive?’” (SH provider)
“We have been operating for several years with a kinda one-size-fits-all housing. . . . We have clients who are in interim housing for q


using is having staff that’s trained and equipped to deal with the client base . . . so working on de-escalation tactics, and understanding mental health on a deeper level.”
(ICMS provider)
  NOTE: SH = supportive housing (interim or permanent). Inne


k that is a lasting
effect. . . . You need to teach how to fish, and [to do that], you don’t need to give people the fish, you got to let them make mistakes.” (ICMS provider)
  NOTE: SH = supportive housing (interim or permanent). Outer Setting
For p


s mental health services were described as not having enough capacity to take on clients. On the other hand, staff reported that Los Angeles County offers access to a robust social safety-net system for clients (e.g., Social Security, Medi-Cal, Count


 in LA county?” (SH provider)
“The ones that can’t make it through the program are probably our most severe cases. They need a higher level of care—that’s either drug rehab or something like that. It’s mostly drugs. Sometimes a mental health issue. .


lates to where they are with their mental health.” (ICMS provider)
“It’s hard to provide just case management, be asked to do clinical stuff, then have to tell the FSP you have to do x and y—they don’t like to be told what to do. There are a lot of c


relationships with local communities in advance of opening new housing options.
More generally, interviewees described how ODR Housing has rapidly grown as a program in the past few years. Overall, this was described as a facilitator and indeed a wel


roject-based housing [is needed] . . . housing that has all services onsite so it’s not a fractured treatment team.” (ODR staff)
We also have coming online our own Board and Care. . . . We’ve worked for a year on getting this up and running—that’s th


he program based on feedback from providers and the needs of the clients.
However, there have also been certain challenges to implementing ODR Housing. Though partnerships are a key element of the program, we learned that coordinating across so many


set of the pandemic, however, do not have a frame of reference for the “typical” operation of the program, which means some of their impressions of the program and its services were likely shaped by the pandemic in ways they may not have even known o


 facilities and assistance applying for jobs and completing benefits paperwork.
 Challenges included problematic behavior by other residents and the need for additional staff support.
 Clients in interim housing were largely aware of the steps need


nd those clients were sent to isolate in a separate location before being placed in their designated interim housing sites.
Interim Housing
When asked about the most helpful services received in interim housing, participant responses included the ava


nced a need to complete a checklist provided by ODR before looking for permanent housing:
They have a list, a checklist, to make sure we have our ID, make sure we have our social security, they make sure we have our income-verification form, and they


or and therapist appointments, and on other errands. They also assisted clients in
46
finding jobs. Overall, the clients in permanent housing were extremely happy with the program, and many made some version of the comment, “ODR saved my life.”
Behav


 helpful to stay inside rather than be outside and getting triggered to use.
Other clients reported appreciating that they had a reason not to leave their apartments or not to have to travel to their ICMS provider’s office, now that services were pro


you get jobs, help you get assistance. It’s been a good program for people that want to straighten their lives up and get back on track.
48
Summary
These interviews provided important insights into client experiences in ODR Housing, including those e


ers can prove challenging.
The ODR Housing Program provides wraparound services, including long-term housing subsidies, intensive case management, mental health services, and time-limited community supervision. These services are provided by differen


is challenge and of the need for continuity of mental health services immediately upon release from jail, ODR
has filled this gap by funding bridge mental health providers through the ICMS agencies, even though this comes at greater cost to the progr


 Watkins et al., 2017). However, this has yet to become a widespread practice, and more resources are needed to support further integration of care for multiple co-occurring conditions that many participants experience.
ODR Housing serves clients wit


ir case managers or mental health providers, the clients we spoke with were largely satisfied with the services they received. Clients who had been in the program longer seemed to be more aware of the range of services available to them, which sugges


pandemic, when organizations were focused on adapting to the emergency situation.
Regarding client interviews, we relied on program staff to advertise the opportunity for clients to participate in our data collection, so it is possible that the recru


o staff members who do not have formal clinical training, such as staff of interim housing sites. As the program establishes additional interim housing sites, standardized staff training opportunities may help to ensure high quality services across t


plore process measures, including early attrition rates.
Early reports on the program’s outcomes evaluated only housing stability and new felony convictions among clients who had reached the PSH phase of the program (Hunter and Scherling, 2019). It a


terms were identified to capture these concepts, depending on the database. Wildcard characters were used to expand search terms (e.g., “incarcerat*” to capture both “incarcerated” and “incarceration”). Results were limited to English-language artic


,
and Collins, 2013
Pre/post with
matched control
group
community-based or site-based service support, or congregate-
diagnosis and a recent history of homelessness, either in shelters or
Single-group
Permanent housing subsidies,
Veterans experiencin


mary care services,
N = 1,937
either onsite or by referral.
long-term housing placement with case management for and assistance with income.
with a disability that limits their ability
 Target Population
Veterans enrolled in a supported
Los Angeles,


02
homelessness with behavioral
62
 Various, Canada,
USA USA
   supportive housing in California Permanent
Henwood, Katz, and Hickert and Taylor, Gilmer, 2015
2011
Mentally Ill Offender Crime-Reduction
Emergency Service
Seattle Downtown
Center housin


Design
Key Services
and Sample Size
Geographic Area
Program
controlled trial
vouchers, and placement
treatment providers.
program. N = 1,160
and peer supporter) or
illness diagnosis. N = 2,255
Randomized
Scattered-site units with Assertive
treatment-


s receive while in interim housing?
5) Tell me about the process of an individual getting permanent housing.
a. What is the wait-list process like?
b. Are there milestones that clients have to achieve to enter housing?
 65
c. Are there different requ


th clients? How does this relate to service eligibility? (Potential probes: Housing readiness, substance use/mental health, physical health)
Case Management Providers
14) What type of screening or assessment(s) do you do with ODR Housing clients? (Po


g; other resources)
28) What are the major resources that you and your agency need to deliver the program?
a. We would like to set up a separate meeting to discuss how we can track program costs and potential savings. Who from your agency should be i


[If additional clarification is needed: The case management providers may have a number of roles in your care, including linking you to services, helping to prepare you for the move to permanent housing, and overseeing your psychiatric medication.]


mination or unfair treatment since being in the program? This could include unfair treatment from any source, including providers, landlords, or neighbors, or based on any characteristic (age, race, gender, mental health status, criminal justice stat


/12/Report_FUFBooklet.pdf
———, Returning Home: Ending the Cycle of Homelessness and Criminal Justice Involvement Through Supportive Housing, New York, N.Y.: Corporation for Supportive Housing, 2011. As of February 9, 2021: https://d155kunxf1aozz.clou


of February 8, 2021: https://www.urban.org/sites/default/files/publication/25716/412632-Supportive-Housing-for -Returning-Prisoners-Outcomes-and-Impacts-of-the-Returning-Home-Ohio-Pilot-Project.PDF
73
Fontaine, Jocelyn, Carey Anne Nadeau, Caterina Ro


15, pp. 80–87.
Hickert, Audrey O., and Mary Jane Taylor, “Supportive Housing for Addicted, Incarcerated Homeless Adults,” Journal of Social Service Research, Vol. 37, No. 2, 2011, pp. 136–151.
Hilton Foundation, “Just in Reach and Pay for Success Fac


odd P. Gilmer, “Implementation and Outcomes of Forensic Housing First Programs,” Community Mental Health Journal, Vol. 52, No. 1, 2016, pp. 46–55.
Larimer, Mary E., Daniel K. Malone, Michelle D. Garner, David C. Atkins, Bonnie Burlingham, Heather S.


ahsa.org/documents?id=4558-2020-greater-los-angeles-homeless-count -presentation
Malone, Daniel K., “Assessing Criminal History as a Predictor of Future Housing Success for Homeless Adults with Behavioral Health Disorders,” Psychiatric Services, Vol.


L. Streiner, Helen-Maria Vasiliadis, Paula N. Goering, and At Home/ Chez Soi Investigators, “Effect of Scattered-Site Housing Using Rent Supplements and Intensive Case Management on Housing Stability Among Homeless Adults with Mental Illness: A Rando


h Services Research, Vol. 48, 2021, pp. 707–717.
Watkins, Katherine E., Allison J. Ober, Karen Lamp, Mimi Lind, Claude Setodji, Karen Chan Osilla, Sarah B. Hunter, Colleen M. McCullough, Kirsten Becker, and Praise O. Iyiewuare, “Collaborative Care fo


   Research Report
SHAMENA ANWAR, PATRICK BAYER, RANDI HJALMARSSON, MATTHEW L. MIZEL
Racial Disparities
in Misdemeanor Speeding Convictions
CORPORATION
 Racial Disparities in Misdemeanor Speeding Convictions
For more information on this publication,


re are racial disparities in who benefits from this discretion and why these racial disparities might exist.
Justice Policy Program
RAND Social and Economic Well-Being is a division of the RAND Corporation that seeks to actively improve the health an


 court stages. Because of the relatively small numbers of individuals in other racial groups, our analysis focused on disparities between Black and White motorists.
Findings on Racial Disparities in Misdemeanor Speeding Convictions Overall Racial Dis


ney present significantly lowered the likelihood that a motorist was convicted of a misde- meanor, but in Virginia, attorneys are not provided by the court for these violations and must be retained at the motorist’s expense.
About one-fifth of the ra


. 29 Regression Analysis at the Law Enforcement Stage .................................................................. 32 Regression Analysis and Oaxaca-Blinder Decompositions at the Court Stage................................... 36
CHAPTER SIX
Con


ts Cited by Officers Who Did Not Discount Speeds ............................................ 48
Tables
2.1. Summary Statistics for the Main Analysis Sample......................................................... 9 3.1. How Court Pathways Differ Acr


. For instance, although Black individuals made up 13 percent of the U.S. population in 2015, they made up 27 percent of misdemeanor arrests (Natapoff, 2018). Mayson and Stevenson (2020) analyzed misdemeanor case filings in multiple jurisdictions and


g record, and can result in increased penalties for any future criminal conviction.
For several reasons, studying misdemeanor speeding violations presents a unique opportunity to under- stand both the overall disparities present in a misdemeanor offe


eive a charge reduction by the officer are generally required to go to court to adjudicate the charge, whereby the judge or prosecutor can reduce the charge to a simple traffic infraction. This allowance for discretion brings up a natural question: A


legally relevant reason that some of these controls are being taken into account. The models used for analysis here are directly informed by our process-mapping results. Because of the relatively small numbers of individuals in other racial groups, o


st, in our study, the benefits that motorists received from getting their speeds or charges dis- counted were quite different from the benefits noted in the earlier studies. In previously studied settings, only a higher fine (rather than a more serio


 amend charges.
 4
CHAPTER TWO
Data and Analysis Sample
Data Sources, Cleaning, and Variable Construction
Collecting data on speeding violations in Virginia requires obtaining data from the district courts. Virginia is composed of 95 counties and 38


nts per county used in our analysis (Virginia Demographics, undated). These counties, however, are geographically dispersed through much of Virginia.
1 Most counties had either no data or incomplete data for 2006. Only two of the counties (Shenandoah


ar, and thus driver age is unknown in our sample); city, state, and zip code of residence; and the county in which the alleged violation took place. We used the motorist’s address to code whether they were an out-of-state driver.
Although the data do


a felony are eventually convicted of a misdemeanor in those courts; the
7
Racial Disparities in Misdemeanor Speeding Convictions
 courts also handle probation violations. Our criminal history measure is thus a combination of whether the individual ha


arrested drivers were Virginia residents, and Virginia law enforcement officers are likely to mainly receive alerts from in-state incidents. Because we wanted to focus only on incidents in which the speeding violation was likely the only offense bein


ack individuals. Almost 60 percent of these drivers were from out of state, and almost two-thirds of the violations occurred in areas where speed limits were either 65 or 70 mph. Note that we report summary statistics as decimal num- bers (e.g., 0.58


d Brunswick Chesterfield Essex Fluvanna Fredericksburg Halifax
Louisa Madison Mecklenburg Prince George Rockbridge Shenandoah Southampton Warren
York
Observations
Overall Black White
0.650 0.624 0.667 0.287 0.301 0.277 0.063 0.075 0.055
0.592 0.592 0


0 mph at any speed limit.1 This charge is classified as a first-degree misdemeanor, punishable by up to 12 months in jail. Motorists traveling over the speed limit but under the reckless speeding limits are to be charged simply with speeding, which i


 charged with a reckless speeding violation that was
1 Virginia Criminal Sentencing Commission, 2015, p. 133. As noted earlier, in Virginia, this violation is technically listed as reckless driving—speeding, but for simplicity, we refer to it in this


officers in Virginia record only one speed on a citation and do not note whether this speed has been discounted, the easiest way to examine the extent to which discretion is used by law enforcement is to examine the data for the frequency distributio


rivers to be relatively continuous throughout the whole distribution.
Distribution of Cited Speeds
In Figures 3.1–3.3, we present three histograms, each presenting the distribution of miles per hour above the misdemeanor cutoff that the motorist was


cutoff is that motorists intentionally drive at a speed just below the cutoff to avoid receiving the harsher penalty. Although there is potential that some of this bunching reflects motorist behavior, we believe, for several reasons, that the majori


14), we can employ the following measure as a proxy for the amount of discounting used:
Number of motorists cited at the misdemeanor cutoff
Number of motorists cited between the misdemeanor cutoff and 15 mph above the cutoff (inclusive)
 17
Racial Di


it is still a useful indication of how the use of discretion by law enforcement varies across groups. The value of this measure (i.e., the number of motorists cited at the misdemeanor cutoff divided by the number of motorists cited between the misde


on, which indicates that offi- cers are discounting charges at a substantial rate. In this section, we consider what factors might influence whether law enforcement officers choose to discount a misdemeanor charge down to an infraction.
Figure 3.4 ma


in the previous section, law enforce- ment officers in our sample classified 58 percent of these motorists’ cases as infractions. Although these motorists could still go to court to try and further reduce their charges, we considered that discounted


orist prepaid the fine (which means that the officer made the offer and the motorist accepted it). Whether motorists exer- cised the prepayment option depended heavily on the county of the stop: Almost all instances of prepaid mis- demeanor charges o


 with upgraded charges had prepaid their tickets, so the data indicate that an upgrade typically happened only if the individual chose to challenge the original infraction citation in court.
 21
Racial Disparities in Misdemeanor Speeding Convictions


graded or dismissed. Table 3.1 shows the proportion of motorists with a given characteristic who prepaid the ticket, did not attend court, attended court without an attorney, or attended court with an attorney. We estimated the numbers in this table


ve the court outcome (n = 109,183). The table shows the proportion of each group in each court pathway.
FIGURE 3.6
Percentage of Motorists Who Attended Court and Had the Misdemeanor Charge Downgraded or Dismissed
        100
         90
         80


tion can go to jail and can have their driver’s license suspended by the judge, these two outcomes are relatively rare for motorists convicted of reckless speeding and are likely inflicted only on the very riskiest of drivers (who likely would not ha


xchange (generally, offenses for which fingerprints are collected).
  25
Racial Disparities in Misdemeanor Speeding Convictions
 Black applicants than for similar White applicants: A previous audit study by Pager (2008) indicated that the impact of a


ve the charge classified by law enforcement as an infraction, (3) charged with a misdemeanor by law enforcement but have the charge downgraded to an infraction or dismissed by the court, (4) or charged with a misdemeanor by law enforcement and convic


ls whose charge was amended to an infraction by the court, although the amount was still higher than if the motorist had been charged by law enforcement with an infraction. For motorists whose charge was amended to an infraction by the court, the fin


mit, motorists would receive 3 demerit points, and if the charge was amended to speeding between 10 and 19 mph above the speed limit, they would receive 4 demerit points (Virginia Department of Motor Vehicles, 2021). The table indicates that there is


isparities in outcomes, which we define as the racial disparities that exist when no other factors are controlled for. As noted in Chapter Three, there are four key outcomes that can occur when an individual is stopped for speeding in the reckless ra


ocess indicates that a much larger percentage of Black motorists than White motorists were forced to go to court to have the charge adjudicated. Furthermore, our discussion in Chapter Four on fines and license points indicates that one of the most pr


ppened at the law enforcement stage ((0.166−0.075) / 0.166 = 0.55), and the remaining 45 percent of the disparity occurred at the court stage. Next, we delve into what factors might explain the racial disparities at each of these stages.
31
Black mot


aracteristics. How- ever, this coefficient can also reflect racial differences in relevant characteristics that we are not accounting for in our model. As we discussed in Chapter Three, these relevant variables include the actual speed at which the m


s among that subsample who were either charged with a misdemeanor by law enforcement or cited with a speed at the cutoff (depending on the dependent variable used).
The results presented in Panel B of Table 5.1 indicate that driving record and crimin


for the speed limits of 65 or 70 mph are relatively small; this is likely because law enforcement officers rarely used discretion to classify speeds above the cutoff as an infraction at these high speeds. The low use of discretion implies a low oppo


be stopped in areas where all motorists were less likely to be cited for a speed at the cutoff. Unfortunately, we are unable to determine whether there is a race-neutral reason that, in counties in which a larger share of stops were of White motoris


ited motorists are actually driving at the cutoff. To the extent that speed- discounting is responsible for the disparity, researchers would need to determine whether there is a legitimate reason why officers in counties with more White motorists eng


ns (guilty, not guilty, or dismissed). To examine the
36
Racial Disparities in Misdemeanor Convictions
 raw racial disparity in misdemeanor convictions at the court stage, and to determine the extent to which this raw disparity can be explained by va


a in which the motorist was cited nor the addition of controls for previous driving violations or non-traffic criminal history seem to affect the central results.
The first key result is that, among this sample of individuals charged with a misdemean


p in an outcome variable Y between Black and White individuals (denoted as B and W) can be split apart into two terms (Jann, 2008):
YB -YW =(XB -XW )βW +XB (βB -βW ). (5.4)
XB and XW represent the average values of all covariates included in Equation


dicate that court attendance was the largest driver of the racial disparity observed at the court stage. The racial gap in misdemeanor conviction rates would close by 7 percentage points if White
5 A negative decomposition value attached to a covaria


generally are not provided indigent defense representation because the potential for jail time on these cases is typically waived in advance. Anecdotal evidence suggests that retaining a private attorney for these types of cases can cost about $500,


the most effec- tive intervention. That is, is it mainly important to implement interventions that encourage people to go to court or that get individuals to complete the driver’s clinic? A necessary first step is for the court to record when drivers


 played a role here. Overall, we were able to explain a higher percentage of the disparity at the court stage than at the law enforcement stage.
Data Limitations
All of our conclusions necessarily focus on racial disparities in misdemeanor speeding c


and send citations to speeding vehicles. These automated enforcement methods, which are common in Europe, are widely considered to reduce speeding- related crashes, fatalities, and injuries (NTSB, 2017; Wilson et al., 2010). Automated methods of enf


these two interventions had on court appearance rates for misdemeanor summons in New York City and found that redesigning the summons form reduced failures to appear by 13 percent, and text message reminders reduced failures to appear by 21 percent.


is restriction dropped 33 percent of the observations in areas where the speed limit was 25–60 mph.1 The hallmark of officers speed-discounting when issuing cita- tions is that there is bunching right at the misdemeanor cutoff. Thus, to identify the


y fit either group, so we did not use these observations in analyses that compare outcomes for motorists stopped by officers who did and did not discount speeds.3 Figure A.1 shows the distribution of cited speeds in speed limit zones of 25–60 mph and


comes of stops by officers who did discount speeds. The fact that such a small subsample of the stops were conducted by officers who did not discount speeds indicates that this comparison is not likely to be too useful, because this small subsample m


were differences in the locations where Black and White motorists were cited that need to be controlled for, Table B.1 does not present straight speed tabulations for each of the four officer and motorist combinations, although the numbers can be int


from the rate that White motorists’ speed was discounted.
ing columns 1 and 3 indicates that speed-discounting increases the proportion of Black motorists who were cited at the cutoff and reduces the proportion who were cited at speeds 1–5 mph above


s exploratory because we were able to include only a small (and somewhat nonrepresentative) portion of our full sample. Furthermore, the research design relies on comparing cited speed distributions among officers who did and did not discount speeds;


w of Law and Economics, Vol. 38, June 2014, pp. 11–24.
Anwar, Shamena, and Hanming Fang, “An Alternative Test of Racial Prejudice in Motor Vehicle Searches: Theory and Evidence,” American Economic Review, Vol. 96, No. 1, March 2006, pp. 127–151.
Beck


5, 2020. Luh, Elizabeth, “Effects of Disparate Policing on Long-Term Outcomes: Evidence Using Chicago Parking
Tickets,” working paper, March 20, 2020.
Lynn, Cheryl W., Wayne S. Ferguson, Nicholas J. Garber, and Jonathan C. Black, Executive Summary: A


, No. 2, 2021, pp. 1199–1253.
Saez, Emmanuel, “Do Taxpayers Bunch at Kink Points?” American Economic Journal: Economic Policy, Vol. 2, No. 3, August 2010, pp. 180–212.
Spohn, Cassia C., “Thirty Years of Sentencing Reform: The Quest for a Racially Neu


ocuses on understanding the sources of racial disparities in the criminal justice system, as well as evaluating criminal justice system reforms. She holds a Ph.D. in economics.
Patrick Bayer is a professor of economics at Duke University. His researc


 Research Funding for Women’s Health
MODELING SOCIETAL IMPACT
Matthew D. Baird Melanie A. Zaber Annie Chen Andrew W. Dick Chloe E. Bird Molly Waymouth Grace Gahlon Denise D. Quigley Hamad Al-Ibrahim Lori Frank
     CORPORATION
 A MESSAGE FROM WHAM
 T


ese data and citing this report.
 WHAM’s LEAD PARTNERS
WHAM’s sponsorship of this research project was enabled through the generous financial support from the following partners.
American Heart Association
The Association is a relentless force for a


es, Founder and Former CEO of CAROLEE
Meryl Comer, Vice Chair and Global Chair Co-Founder, UsAgainstAlzheimers/ WomenAgainstAlzheimers
Chair, Global Alliance on Women’s Brain Health
Anula Jayasuriya, Vice Chair and Chief Scientific Officer
Founder an


chiatry in the Field of Women’s Health, Brigham and Women’s Hospital
Dr. Wendy Klein, MD, MACP, Former Medical Director, Health Brigade
Dr. JoAnn Manson, DrPH, MD, Michael and
Lee Bell Professor of Women’s Health, Medicine, Harvard Medical School Co-


ol of Pharmacy, University of Washington
Hadine Joffe, MD
Vice Chair for Research, Department of Psychiatry, Executive Director, Mary Horrigan Connors Center for Women’s Health and Gender Biology, Paula A. Johnson Professor of Psychiatry in the Field


EXECUTIVE SUMMARY 1
  2 RESEARCH FUNDING FOR WOMEN’S HEALTH: MODELING SOCIETAL IMPACT
NIH portfolio for AD/ADRD, 4.5 percent for CAD, and 7 percent for RA. The impact of a current investment was assumed to occur in ten years, with benefits accruing a


 to help them plan investments that can yield the greatest public health benefits.
As part of an initiative of the WHAM nonprofit foundation, RAND Corporation researchers examined the impact of increasing funding for women’s health, beginning with a


For late onset AD/ADRD, gender differences in informal caregiv- ers are a prominent part of the differential gender impact, with more women than men serving as informal caregivers. The caregiving burden associated with CAD is not quite as high, and


ng the Research Investment
To construct models of the impact of research investment, we used current levels of funding from the NIH. Certainly, the universe of fund- ing extends beyond the NIH and includes advocacy organizations,
the biopharmaceutica


f approxi- mately one million adults and simulated the pro- gression of each person’s health in the sample over a 30-year time horizon. For CAD and RA, the focus was on working- age adults age 25 and older. For AD/ADRD, the cohort represents the U.S.


thetical cohort, we chose to base earnings calculations for everyone on the earnings of non-Hispanic white males. This avoids the gender- and race-based labor market discrimination that is inherent in the different (and lower) earnings for women vers


 and reduced mortality due to disease.
We simulated the effects of increasing funding for health research on women in terms of economic outcomes, including the monetary value of patients being able to stay in the labor force longer as a result of dec


e years with RA, and men have about 9,000 fewer life years with RA.
Increased Life Expectancy
For AD/ADRD, a 0.01 percent health improvement results in nearly 4,000 additional life years lived, more than 6,000 fewer years with AD/ADRD, and nearly 4,0


regivers pick up the increased care need instead. Countering this effect is the smaller number of individuals with the disease, which ultimately reduces overall productivity loss for the informal caregivers.
For CAD, caregiver productivity drops by a


e not changing the onset or severity of the disease, decrease the burden of the disease for a given severity. For these reasons, the QALYs capture a much larger effect, which is represented by approximately 223,000 more life-year equivalents of
a ful


50,000 100,000
50,000 0 –50,000
Women Men
      –4,514
34,446
 223,074
   –24,535
  –3,869
–732
 Lost productivity (self)
Lost productivity (caregivers)
QALYs
NOTE: Figure shows a 0.1 percent health improvement, which is three times larger for women


e next-largest driver. The overall reduc- tion in costs was around $10.5 billion over 30 years (in 2017 dollars). About 87 percent of the costs are from female patients, and 13 per- cent are from male patients. Approximately 90 percent of the cost re


 devoted to women’s health relative to general research, according to the specifications used here.
The overall societal cost savings from modest investment in wom- en’s health research could be $932 million for AD/ADRD, $1.9 billion for CAD, and $10


nal database, but these estimates are larger than some reports in the literature. Smaller estimates still result in large ROIs but change the nature of the health improvements.
  Policy Implications
The results of these analyses suggest several polic


 factors and, given the uncertainty of the future, requires assumptions. Future investment
in women’s health may result in large gains in condition status with resulting gains in health-related quality of life. The limitations that result from the im


y and Systems, Vol. 16, No. 1, February 8, 2018, p. 8.
Committee on Family Caregiving for Older Adults, Board on Health Care Services, Health and Medicine Division, “Economic Impact of Family Caregiving,” in Richard Schulz and Jill Eden, eds., Famili


ex and Gender Differences in Alzheimer’s Disease and Alzheimer’s Disease–Related Dementias (AD/ADRD),” August 5, 2020b. As of April 5, 2021: https://grants.nih.gov/grants/guide/notice-files/ NOT-AG-20-038.html
——— “Research Portfolio Online Reporting


Molinuevo, Matthew Norton, and Alireza Atri, “Tip
of the Iceberg: Assessing the Global Socioeconomic Costs
of Alzheimer’s Disease and Related Dementias and Strategic Implications for Stakeholders,” Journal of Alzheimer’s Disease, Vol. 70, No. 2, 2019


s (RePORTER),” updated November 7, 2020c. As of November 17, 2020: https://projectreporter.nih.gov/reporter.cfm
Nichols, Gregory A., Timothy J. Bell, Kathryn L. Pedula, and Maureen O’Keeffe-Rosetti, “Medical Care Costs Among Patients with Established


uce, Bryan R., Josephine Mauskopf, Frank A. Sloan, Jan Ostermann, and L. Clark Paramore, “The Return on Investment in Health Care: From 1980 to 2000,” Value in Health, Vol. 9,
No. 3, 2006, pp. 146–156.
National Institute of Arthritis and Musculoskele


’s health research, with a focus on the following three disease areas: brain health, immune and autoimmune disease, and cardiovascular disease. Using microsimulation analyses, the research team studied the societal cost impact of increasing research


 Societal Impact of
Research Funding for
Women’s Health
IN CORONARY ARTERY DISEASE
Matthew D. Baird Melanie A. Zaber Andrew W. Dick Chloe E. Bird Annie Chen
Molly Waymouth Grace Gahlon Denise D. Quigley Hamad Al-Ibrahim Lori Frank
     CORPORATION
 A


n more about using these data and citing this report.
 WHAM’s LEAD PARTNERS
WHAM’s sponsorship of this research project was enabled through the generous financial support from the following partners.
American Heart Association
The Association is a re


Founder, AccessCircles, Founder and Former CEO of CAROLEE
Meryl Comer, Vice Chair and Global Chair Co-Founder, UsAgainstAlzheimers/ WomenAgainstAlzheimers
Chair, Global Alliance on Women’s Brain Health
Anula Jayasuriya, Vice Chair and Chief Scientif


iate Professor of Psychiatry in the Field of Women’s Health, Brigham and Women’s Hospital
Dr. Wendy Klein, MD, MACP, Former Medical Director, Health Brigade
Dr. JoAnn Manson, DrPH, MD, Michael and
Lee Bell Professor of Women’s Health, Medicine, Harva


OICE) Institute, School of Pharmacy, University of Washington
Hadine Joffe, MD
Vice Chair for Research, Department of Psychiatry, Executive Director, Mary Horrigan Connors Center for Women’s Health and Gender Biology, Paula A. Johnson Professor of Ps


 outcomes.
Investing in women’s health research for CAD yields benefits simi- lar to investing in general research, with improved health-related qual- ity of life for women from women-focused research.
The results establish the potential for investme


, researchers, and policymakers to help them plan investments that can yield the greatest public health benefits.
As part of an initiative of the Women’s Health Access Matters (WHAM) nonprofit foundation, RAND Corporation researchers exam- ined the i


orbidity and mortality, investment in women’s health could be expected to yield a favorable return for society.
The lack of societal-level evidence on the economic costs, ben- efits, and social impacts of attention to sex and gender in health researc


CAD and examine the impacts of additional investments
 INTRODUCTION 7
  the future impact of funding on health outcomes and changes in soci- etal burden from CAD.
Determining the Research Investment
We used current levels of funding from the NIH as t


ed to disease impact and has been used as a metric for disease impact and impact of health innovation, incorporating length of life with the quality of life (Grant and Buxton, 2018).
The model assumes improved health as a result of increased fund- in


ls on health outcomes (Grant and Buxton, 2018). The return on research investment calculation was a function of the fol- lowing specific health outcomes: age incidence of disease, improved detection rates and earlier detection in the disease course,


ting in Women’s Health
Research
To further understand investment impact, we also examined the prob- ability of success of research investment levels. We calculated the minimum probability of success of the investment to generate a target of 15 percen


0,000
0 –20,000 –40,000 –60,000
 540
12,628
 8,097
35,712
 19,538
     –39,524
  –13,496
–8,297
–3,410
2,014
  Life years
CAD years
Lost productivity (self)
Lost productivity (caregivers)
QALYs
NOTE: Figure represents the U.S. population age 25 and o


atients)
NOTE: Figure shows a 0.01 percent impact, which is three times larger for women than men.
13
7
–159
32
 –78
  9
 –465
 –1,311
 Women Men
RESULTS 19
  20 SOCIETAL IMPACT OF RESEARCH FUNDING FOR WOMEN’S HEALTH IN CORONARY ARTERY DISEASE
   Dis


y through many path- ways. The models examined here focused on a small but important subset of potential impacts on population health using investment in women’s health research. Although a cure and/or preventive interven- tion may be possible for CA


men in the workforce.
DISCUSSION 23
  24 SOCIETAL IMPACT OF RESEARCH FUNDING FOR WOMEN’S HEALTH IN CORONARY ARTERY DISEASE
Conclusion
Understanding the full range of societal impacts from health research investment requires consideration of multiple


o. 1, February 8, 2018, p. 8.
American Heart Association, “Making Progress: Making a Difference,” fact sheet, undated. As of December 10, 2020: https://www.heart.org/-/media/files/about-us/policy-research/ fact-sheets/ucm_497880.pdf?la=en
Baird, Matt


Measuring and Improving the Societal Impact of Health Care Research,” Eurohealth, Vol. 19, No. 3, 2013, pp. 32–35.
Luce, Bryan R., Josephine Mauskopf, Frank A. Sloan,
Jan Ostermann, and L. Clark Paramore, “The Return on Investment in Health Care: Fro


randa R. Jones, “Air Pollution and Cardiovascular Disease: A Focus on Vulnerable Populations Worldwide,” Current Epidemiology Reports, Vol. 5, No. 4, December 2018,
pp. 370–378.
Zhao, Min, Mark Woodward, Ilonca Vaartjes, Elizabeth R. C. Millett, Kers


Going to the Doctor
Rideshare as Nonemergency Medical Transportation
LAURA FRAADE-BLANAR, TINA KOO, CHRISTOPHER M. WHALEY
 Sponsored by Lyft Healthcare
 CORPORATION
For more information on this publication, visit www.rand.org/t/RRA1019-1.
About RAND


del, the role of rideshare in NEMT, and the policies that support such a role. We conclude by considering how best to support NEMT broadly and optimize rideshare within NEMT specifically. This report is intended for a broad general audience. We condu


et need for on-demand cost-effective solutions within the NEMT ecosystem. Current RB-NEMT capabilities are most appropriate for individuals with medical conditions that result in the need to request rides and those who use Door2Door, Curb2Curb, or Ar


number of patients must show up—and show up punctually.
Nonemergency medical transportation (NEMT) services aim to improve health and health care delivery by facilitating access to medical care for transportation-vulnerable people. It is transport sp


T is a mandatory Medicaid benefit: States are required to provide free or low-cost transportation to and from medical appointments if beneficiaries have no other means of accessing services. But what exactly is covered, how that coverage works, and s


ecipients, to reduce costs (Price and Verma, undated), although protocol for applying for a waiver has changed with time and under different administrations. The potential for cost reduction is slight; NEMT comprises what the U.S. Government Accounta


ies for the NEMT benefit and, if the ride is appropriate, will assign a transit mode. A broker works with multiple transportation providers available to fulfill a request. With rideshare- based NEMT (RB-NEMT), the TNC becomes such a provider.
3 Addit


 mile, and will the total ride be
under two hours?)
• environmental conditions (e.g., is it over 100oF, below-freezing temperatures?)
• what the ride involves (discussed more in Chapter Four); for example, because most modes, such as taxis
or paratra


urer found a 66 percent decrease in rider complaints when using RB-NEMT, and a public health system in Oakland, California, found that using RB-NEMT led to a 73 percent decrease in no-show rates at a primary care clinic and a $400,000 annual savings


e requirements for their drivers, but these may not match state require- ments for NEMT drivers, such as that drivers learn first aid and be able to provide CPR. Currently, TNCs are
5
Going to the Doctor: Rideshare as Nonemergency Medical Transportat


ions remain about what types of riders and what types of rides work best for NEMT provided by rideshare.
6
CHAPTER TWO
Who Are the Transportation Vulnerable? Characteristics of Potential NEMT and RB-NEMT Users
Who needs NEMT? Broadly, NEMT users are


.1
Individuals with Transportation Challenges Among Those Who Have a Medical Condition That Makes It Hard to Travel (by Age)
   Number of people with a medical condition (millions)
14 12 10
8 6 4 2 0
      Ages
16 to 19
20 to 34 35 to 50 51 to 64 65


 steps Difficulty lifting ten pounds
Limitation in physical functioning*
                                                     0 10 20 30 40 50 60 70 80 90 100 Percentage of respondents reporting stated limitation
SOURCE: MEPS data (see Agency for Hea


ese barriers can be based on the physical, cognitive, or sensory health of a patient, or logistical barriers relating to geography and car ownership, among other issues.
• Thesebarriersresultinrealchangestotransportationpatterns—forexample,avoidingpu


ambulatory, able to walk distances (e.g., a half-mile to a bus stop) independently and without strain, and able to enter a vehicle without assistance. People in this group could walk to a nearby medical service or to a public transit stop at some di


y one characteristic of the group. By nature, people are intersectional in their vulner- abilities; very few of us have only one morbidity. Thus, some morbidities affect the required level of service, and some do not. For example, an individual with


that can be provided to them.
C2C or A2A transportation can accommodate various physical and cognitive challenges, depending on the severity and level of outside support needed.5 According to the 2017 NHTS, of respondents with a medi- cal “condition


ons dressing or bathing
FIGURE 3.2
RB-NEMT Groups Associated with Cognitive and Physical Ability
Which Riders Are Best for RB-NEMT?
   Level of assistance required Patient independence
 H2H D2D C2C A2A
Possible RB-NEMT users Likely RB-NEMT users
  th


rting technology. Recall that, in the pilot program with Lyft and CareMore described in Chapter One (footnote 1 in the Summary), smartphone apps were among the methods used to increase user comfort. Although owning (and being able to use) a smartphon


can broadly come down to three ques- tions:
– How ambulatory is a patient on their own?
– How much assistance is needed to carry items they have with them? – Cantheytravelalone?
• Table 3.1 provides details on a selection of NEMT providers, along wit


them there, and another may drive them home.1 The rideshare driver driving to the appointment may have difficulty finding a ride going back to the small town, and the rideshare driver driving from the appointment may have difficulty finding a rider w


astructure made it easy to book regular rides for an individual. Additionally, using RB-NEMT for regular travel by low-needs individuals (e.g., those traveling to a methadone clinic) freed up other NEMT resources for those who need more in-transit ca


t, and Sobhani, 2019).
Reciprocally, rideshare may increase interest in and use of NEMT. A focus group from the MACPAC (2021) noted that using rideshare helped normalize accessing NEMT and removed the stigma generated by using traditional NEMT vehicl


fficult. Part of this difficulty stems from the diversity of ways that NEMT is implemented in research and in practice and the need to match service level with personal needs and resources. The net result is an unusual situation where positive indica


ing to the Doctor: Rideshare as Nonemergency Medical Transportation
 utes for traditional NEMT, and a difference-in-difference analytical approach in Philadelphia found fewer missed appointments in patients who used RB-NEMT services (Chaiyachati et a


ew York, 2018). GAO noted vulnerabilities in terms of how NEMT providers were selected, how NEMT was administered, and how eligibility was verified for patients (Yocom and Goldstein, 2016b), a concern supported by interviewee anecdotes of systemic mi


re specifically, variation can result from the following scenarios:
1. TNC performance and user experience differs between TNC providers (e.g., Lyft, Uber, Veyo), so results from a specific TNC may not be generalizable to other TNCs. Similarly, servi


s, along with other social determinants, such as health behavior, employment, and education (Health Research and Educational Trust, 2017; Healthy People 2030, undated). Recognizing the important role that transportation plays in accessing health care


al factors that determine an individual’s required level of service, transportation needs can vary. As we articulated in Chapter Two, MA and Medicaid beneficiaries are more transportation-insecure because of logistical and medical factors. These indi


t an expensive benefit: One interviewee put the average ride at $25 at the unit level. This cost compares with the costs of missed medical appointments, pegged by an industry publication as $200, or emergency room visits, at an average of $530 (Moore


on may grow because of the aging of the U.S. population. TNCs may explore a tiered rideshare system, possibly starting with the four levels of service described in Chapter Three, where riders could request a driver with specific expertise, willingnes


re research and to think about best practices. Other important research areas include understanding how the rideshare model can be used to deliver needed health-related goods and medical services to individuals who cannot travel. These could and have


are system runs. Such enablers as NEMT broadly and RB-NEMT specifically deserve support.
5 For example, Uber has worked with Meals on Wheels (Uber, undated).
 29
Going to the Doctor: Rideshare as Nonemergency Medical Transportation
  BOX 5.1
Chapter


014 was assigned where missing.
• Living alone. We defined living alone as not married or living with a partner, relative, or other unrelated adult. Nursing-home residents were considered not living alone.
• Functional limitations. Respondents who an


rip files to describe the relationship between select household, per- sonal, and trip characteristics. Our analysis comprised 264,234 respondents and 923,572 trips. Estimates of personal characteristics, such as demographics and medical conditions, w


ce an Important Benefit?” Health Affairs Blog, September 20, 2017. As of October 20, 2020: https://www.healthaffairs.org/do/10.1377/hblog20170920.062063/full/
Agency for Healthcare Research and Quality, Center for Financing, Access, and Cost Trends,


ww.cms.gov/Research-Statistics-Data-and-Systems/Monitoring -Programs/Medicare-FFS-Compliance-Programs/Prior-Authorization-Initiatives /Prior-Authorization-of-Repetitive-Scheduled-Non-Emergent-Ambulance-Transport-
Chaiyachati, Krisda H., Rebecca A. Hu


& Medicaid Services, Department of Health and Human Services, April 24, 2019. As of December 23, 2020: https://www.cms.gov/Medicare/Health-Plans/HealthPlansGenInfo/Downloads/Supplemental_Benefits_ Chronically_Ill_HPMS_042419.pdf
eHealth, Inc., “Medic


-of-transportation-insecurity-an -exploratory-factor-analysis
Health Research and Educational Trust, Social Determinants of Health Series: Transportation and the Role of Hospitals, Chicago, Ill., November 2017. As of July 19, 2021: https://www.aha.or


” Washington, D.C.: United States Securities and Exchange Commission, March 1, 2019. As of December 23, 2020: https://www.sec.gov/Archives/edgar/data/1759509/000119312519059849/d633517ds1.htm
MACPAC—See Medicaid and CHIP Payment and Access Commission


, 2018. As of July 19, 2021: http://www.nadtc.org/wp-content/uploads/NADTC-Transportation-Options-Booklet-2018.pdf
Okoro, Catherine A., Tara W. Strine, Stacy L. Young, Lina S. Balluz, and Ali H. Mokdad, “Access to Health Care Among Older Adults and R


tes Between 2016 and 2018,” July 2018. As of October 20, 2020:
https://www.statista.com/statistics/916253/ride-sharing-total-ridership-us/
Syed, Samina T., Ben S. Gerber, and Lisa K. Sharp, “Traveling Towards Disease: Transportation Barriers to Healt


ss/house-bill/133/text
Weber, Chris, “Introducing Uber Health, Removing Transportation as a Barrier to Care,” Uber Newsroom, March 1, 2018. As of October 20, 2020:
https://www.uber.com/newsroom/uber-health/
Westat, Main Study Retrieval Questionnaire,


Implementation and 12-Month Health Service Utilization and Cost Outcomes from a Managed Care Health Plan’s Permanent Supportive Housing Program
SARAH B. HUNTER, ADAM SCHERLING, RYAN K. MCBAIN, MATTHEW CEFALU, BRIAN BRISCOMBE, WILLIAM MCCONNELL, PRIYA


report, we provide an update from our previous reports in terms of program implementation through September 2020, describing participant characteristics, program referrals, enrollment timelines, housing placements, and program exits. We also assess t


the key social determinants of health, stable housing, but little is known about the associated program costs and potential impacts on health care utilization among insured populations. We sought to understand the program’s implementation, the progr


 the PSH program prior to 12 months had higher health care utilization and associated costs in the pre-enrollment and post- enrollment periods relative to enrollees who remained in stable housing through the PSH program.
• The PSH program was associa


ity of life.
• We observed that enrollees who exited the program before reaching 12 months of enrollment had higher utilization patterns and associated costs both before and after enrollment, relative to enrollees who were stably housed at 12 months


 Health Care Costs in the Treatment and Comparison Groups
Prior to and After Program Enrollment ................................................................................ 16 Figure 3.3. Change in Per-Member Health Costs Pre-Enrollment Versus Po


ed with poorer physical and mental health outcomes. For example, compared with the general U.S. population, people experiencing homelessness are more likely to suffer from unmanaged chronic physical health problems (Baggett, Liauw, and Hwang, 2018),


ealth care providers and public payers. Through the Affordable Care Act and state Medicaid expansion, many individuals experiencing homelessness are now eligible for Medicaid. Also, a growing movement away from fee-for-service models to bundled payme


adequately planning PSH program implementation.
Previous studies have also tended to report the average effects over a specified intervention period (e.g., six or 12 months) rather than exploring changes over time in service use and associated costs.


cant component of services delivered included recuperative care for which members were provided shelter, meals, and transportation. Recuperative care has become an important service since the passing of California Senate Bill 1152 (Discharge Planning


lled in the PSH program. Data included member demographic information and records about individual use of health care services. For program enrollees, we received information about program referral and enrollment dates, receipt of supportive housing


CD-10)
diagnosis code Z59.0, indicating homelessness, between January 2018 and September
2019 (coinciding with the recruitment period for the treatment group)
• who were not enrolled in the PSH program during the analytic period.
This process identif


i.e., Current Procedural Terminology or Uniform Billing codes), place of service, ICD-10 diagnosis code, and net pay amount. A service event could include one or more health care claims or encounters. Costs, as ascribed to health care services and th


ogram enrollment on health care service utilization and costs (Austin, 2011). The use of a propensity score permits the analysis of an observational, nonrandomized study to mimic the characteristics of a randomized controlled trial by balancing treat


o that they sum to 162 to match the number of members in the treatment group.
Health Care Service and Cost Effects
The effects of the PSH program on health care service utilization and costs were examined using propensity score–weighted regression mo


ervice) in each person-month
2. a mixed effect linear regression predicting the number of services in person-months with at least one service
3. a mixed effect linear regression predicting costs in months with at least one service.
All models control


anced the groups were before and after weighting. Group differences were deemed notable when the maximum ASMD was greater than 0.20 (Cohen, 1992).
Propensity score weighting was effective in attenuating the observed differences between the treatment


osts during the post-enrollment period were segmented according to housing status (pre-housing transitional period, post-PSH placement period, and post-exit period among those who left the program), costs were notably lower during the period in which


36 percent and 66 percent; see Figure 3.3), while the number of patients served stayed relatively stable and the number of patient events increased.
17
                      Table 3.2. Most-Common Health-Related Costs: Permanent Supportive Housing Pr


hich constitute a majority of the observed health care costs (see Figure 3.9). Looking at costs in other settings, we were able to identify statistically significant treatment effects for home health and temporary lodging of $217 PMPM and $166 PMPM,


d health and social care so that high-cost health care events become less frequent (National Academies of Sciences, Engineering, and Medicine, 2018).
 24
Figure 3.7. Cumulative Share of Post-Enrollment Inpatient Costs, by Member, Treatment Group (n =


t 12 Months
  27
Figure 3.11. Monthly Health Costs for Members Who Left the Program, by Exit Reason
 28
The Effect of Housing Versus Program Enrollment
The previously described modeling approach aggregates data into annual summaries—i.e., the average


hs post-enrollment, we find that their probability of having any health care services is no different than had they not enrolled in the program.
Next, we explore the effect of the program on the total number of health care services during months with


ere is a nonstatistically significant but meaningful decrease of $1,337 PMPM during months with at least one health care service after the receipt of housing relative to no enrollment.
The total effect of the program on health care costs is the combi


nt). Custodial care and recuperative care were infrequent after PSH housing placement.
Looking at this 12-month window, total programmatic costs were $3,884,374, or $23,978 per participant (n = 162). The average cost per enrollee over the first 12 mo


Housing, ongoing Housing rental Housing utilities Damage mitigation
Enrollee services Transportation
Meals and food
Total $960,553
Grand total     $1,488,103
a The cohort perspective follows each enrollee over a 12-month period from
enrollment.


ivariable regression analyses that adjusted for demographic differences between the treatment and comparison groups, we did not find a statistically significant difference between groups in terms of change in health care costs from the pre-enrollment


18—a reduction of $21,418 over the period.
• Those in the comparison group observed a similar reduction of $25,273 between the pre- and post-enrollment periods, despite not participating in the PSH program.
• Health care costs declined further among


did not demonstrate cost offsets over the initial 12-month period of analysis, this may be partially due to high costs observed in the transitional period (the period after program entry but before receipt of PSH). Health care costs during the transi


etes care among enrollees following enrollment that occurred despite overall diabetes care costs remaining flat (see Appendix B, Table B.1). To examine whether these shifts in health care translated to improved health outcomes among enrollees compare


n those panel studies, an increase in pharmacy spending was observed in the post-PSH placement periods. Also, DeLia et al., 2021, did not observe net total health care cost savings after PSH placement: The reduction in emergent care costs was largely


t and comparison groups.
Similarly, we did not have information about the comparison group’s status in terms of whether members of this group had died, moved out of the area, or lost insurance coverage. All
39
of these events might have resulted in a


he drop in utilization because of the spike prior to enrollment.
We also compare pre-enrollment and post-enrollment costs using nominal cost data unadjusted for inflation. This method slightly overestimates (in real terms) the costs associated with t


 was primarily provided by another payer, and we did not consider other social service use, such as criminal justice utilization and associated costs.
Furthermore, we do not advocate that the financial outcomes specific to the health plan should driv


s compared with the transitional period. Our findings should be of interest to PSH program planners who are interested in understanding PSH implementation, including programmatic costs and their relation to health care utilization and associated cost


 the comparison group is larger than the treatment group, it was necessary to iterate through the treatment group members several times, each time in random order.
 43
Propensity Score Weights
Having identified enrollment dates for the comparison gro


ollment, using manual classifications based on CCSR.
d This variable indicates that the member was diagnosed with at least one chronic condition in the 12 months prior to program enrollment, based on the chronic condition indicator from HCUP, 2020.
4


atric facility Emergent
ER
Ambulance Outpatient
General hospital Primary care facility
Urgent care facility Nonmedical location
Custodial care facility
Temporary lodging
Home health Other classifications
Pharmacy
All other locations
Patient N Events


 518, ESS = 249) Standard
     Variable
Intervention
Baseline (intercept)
PSH enrollment (treatment)
Demographic characteristic Age
Gender: male
Ethnicity: black
Ethnicity: Hispanic Ethnicity: not reported/other County: Riverside
Estimate Error t p –


4 1.13 0.26 2.09 0.04
–1.19 0.24 0.35 0.72
Table C.14. Monthly Emergent Health Costs (n = 518, ESS = 249) Standard
NOTE: p = probability value; t = t-test statistic.
56
                    Variable
Intervention
Baseline (intercept)
PSH enrollment (tr


 = probability value; t = t-test statistic.
Variable
Intervention
Baseline (intercept)
PSH enrollment (treatment)
Demographic characteristic Age
Gender: male
Ethnicity: black
Ethnicity: Hispanic Ethnicity: not reported/other County: Riverside
Estimat


Enrolled (not housed)
Months since enrollment
–12
–11
–10
–9
–8
–7
–6
–5
–4
–3
–2
–1
0 (enrollment month) 1
    2
    3
    4
    5
    6
    7
    8
    9
    10
    11
County
Riverside
San Bernardino
Race/ethnicity
3.36 4.03
2.77 3.47
2.44 4.29
3.2


h E.10 show the results for the number of health care services by type of claim, and Tables E.11 through E.20 show the results for the health care costs by type of cost.
Models: Number of Monthly Services
Table E.1. Total Number of Monthly Services (


p = probability value; t = t-test statistic.
68
          Table E.6. Monthly Number of Inpatient Health Services (n = 445, ESS = 206) Estimate SD Error t p
     Variable
Intervention
Baseline (intercept)
PSH enrollment (treatment)
Demographic charact


phic characteristics Age
Gender: male
Ethnicity: black
Ethnicity: Hispanic Ethnicity: not reported/other County: Riverside
Estimate
–783.10 1,230.40
–24.26 –1,222.67 501.82 –1,454.67 –2,512.07 652.83
SD Error
1,707.54 735.97
27.21 786.56 794.73 1,139


Intervention
Baseline (intercept)
PSH enrollment (treatment) Demographic characteristics
Age
Gender: male
Ethnicity: black
Ethnicity: Hispanic Ethnicity: not reported/other County: Riverside
Estimate
SD Error t p
      –389.47 1,573.05
 –0.25 0.80
 7


oom
ESS effective sample size
HCUP Healthcare Cost and Utilization Project
ICD-10 International Classification of Diseases, Tenth Revision ICMS intensive case management services
PMPM per member per month
PSH permanent supportive housing SD standard


gelo Asa, Wendy Hartogensis, Stephanie E. Cohen, Monica Gandhi, Elvin Geng, and Katerina Christopoulos, “Evaluating the Impact of Housing Status on Gonorrhea and Chlamydia Screening in an HIV Primary Care Setting,” Sexually Transmitted Diseases, Vol.


alth Status of Homeless Adults Entering Permanent Supportive Housing,” Journal of Public Health, Vol. 40, No. 2, 2018, pp. 415–418.
Hollander, Mara A. G., Evan S. Cole, Julie M. Donohue, and Eric T. Roberts, “Changes in Medicaid Utilization and Spend


ousing to Help Homeless Patients,” JAMA, Vol. 321, No. 9, 2019, pp. 822–824.
Kushel, Margot B., Sharon Perry, David Bangsberg, Richard Clark, and Andrew R. Moss, “Emergency Department Use Among the Homeless and Marginally Housed: Results from a Commu


 pp. 287–294.
Salem, Benissa E., Angela L. Hudson, Kartik Yadav, Jaemilyn Lucas, Joy Toyama, Stephanie Chen, Mark Faucette, Maria L. Ekstrand, and Adeline M. Nyamathi, “Correlates of Posttraumatic Stress Symptoms Among Formerly Incarcerated, Homeless


Translational Impacts of World Trade Center Health Program Research
A Mixed Methods Study
THOMAS W. CONCANNON, LAURA J. FAHERTY, JAIME MADRIGANO,
SEAN MANN, RAMYA CHARI, SAMEER M. SIDDIQI, JUSTIN LEE, LIISA HIATT
 Sponsored by the National Institute


(2) stakeholder perspectives gathered through focus groups and interviews, and (3) a review of program documentation. This report is intended to guide program planning by the World Trade Center Health Program leadership as it aims to maximize the im


WTC and related sites or located within the New York City disaster area during and after the terrorist attacks of September 11, 2001. Scientific research supported by the Program provides an evidence base for the health monitoring and clinical care o


existing research in future research, planning, policy, and care. Finally, we applied a machine-learning algorithm with these same words and uses of research to identify related text passages within thousands of pages of Program documentation and gr