ctured Protocol for Post-Implementation Evaluation with Participants (RE -AIM,
2021)

Prior to the interview:
• Review the consent forms with all participants and provide hard copies of the
consent form for individuals’ personal records.
• Co


 health knowledge/attitudes/stigma within your
agency?
14. How has MHFA changed staff awareness of mental health resources available to clients, to City employees, and to New Yorkers in general?
 Implementation
15. The 8-hour training was offe


Sponsored by the Mayor’s Office for Economic OpportunityEvaluation of Mental Health
First Aid in New York City
EUNICE C. WONG, MICHAEL STEPHEN DUNBAR, DANIEL SICONOLFI,
ANTHONY RODRIGUEZ, CHESTER JEAN, VANESSA N. TORRES,
ROSEMARY LI, MICHELE ABBO


rofits, and community-based organizations in 2015 to support the mental health of all New Yorkers through 54 priority initiatives. One of these initiatives involved the provision of in-person Mental Health First Aid (MHFA) trainings, first to staff a


ations of the evaluation plan. We would like to acknowledge Annie Chen (who assisted with Chinese translation) and Ninna Gudgell (who assisted with Spanish translation) for their contributions to study recruitment materials for the focus groups. Addi


 suspended.
During the March 2020 COVID-19 shutdown, planning for an independent evaluation of the
MHFA program was underway but had not yet been launched. In November 2019, the Mayor’s Office for Economic Opportunity (NYC Opportunity), in collabo


re (16 percent), law enforcement/public safety (2 percent), other (26 percent), or no occupation (15 percent). Because city agency employees were prioritized for MHFA training and were a primary group of interest (because of OCMH’s focus on changing


ty])
• city agency employee frontline staff (participation was only possible from the Human Resources Administration/Department of Social Services, a city agency with high survey response rates).
A total of five focus groups (four CBO groups and on


A skills.
• Most survey respondents (90 percent) were correctly able to identify depression
symptoms from a vignette, but performance on a general knowledge test of MHFA
content was low (50 percent on average), indicating room for improvement in s


of CBOs viewed MHFA as well-aligned with their organizational and
programming priorities, and some alluded to cultural shifts within their organizations.
• Mental health stigma was identified by focus group participants as a potential barrier to


ed with  those who had completed MHFA training more recently (within the past
two years), respondents who had completed training three or more years ago were less likely to engage in certain helping behaviors (e.g., active listening, assistance with


f mandatory training could be explored further with a range of stakeholders with distinct vantage
points (e.g., city leadership, MHFA implementors, city agency leadership, city agency
staff, and agency clients). The three city agency employee focus


 their
communities.
Overall Acceptability of MHFA Training Was High, But Areas for Further Cultural
Adaptation Were Identified
• Survey respondents indicated overwhelmingly favorable attitudes toward MHFA training
with respect to its utility, co


ss. xiii Although those who had completed training three or years ago were less likely to engage
in certain helping behaviors compared with those who had completed training within the
past two years of the survey, the continued application of many


 of suicide or harm, listen nonjudgmentally, give reassurance and information, encourage appropriate professional help, and encourage self -help and other support strategies (Forthal et al., 2022).
MHFA has a standard adult course that focuses on ho


s a limitation in the research literature (Hadlaczky et al., 2014; Narayanasamy et al., 2018). A systematic review of mental illness stigma interventions in the workplace (which included MHFA) noted the lack of high-quality evidence (e.g., lack of RC


ortunity), in collaboration with
DOHMH and OCMH, commissioned the RAND Corporation to conduct an independent evaluation of the MHFA program. At the time of the COVID-19 shutdowns, planning for a full-scale evaluation of NYC’s MHFA trainings was unde


ings from focus groups conducted with CBOs and a city agency involved in facilitating MHFA training.
In Chapter 3, the individual-level impact of MHFA and training needs are examined by
addressing the following questions:
• How are trainees curre


ctivities: a web-based survey of those who had received training since the
program’s launch in 2015 and a series of focus groups composed of community leaders and select staff from a city agency. This chapter describes the methodological decisions a


nt) and those who had no work affiliation (23 percent). Community-based trainees also represented a variety of occupational fields, including mental health and social services (19 percent), education (17 percent), health care (16 percent), and law en


 survey assessed MHFA training-related outcomes that have been widely documented in
the literature (Hadlaczky et al., 2014; Morgan, Ross, and Reavley, 2018). Specifically, the
primary MHFA training-related outcomes include trainees’ mental health k


asured with a single item that has been found to be longitudinally
predictive of subsequent application of MHFA skills (Morgan, Ross, and Reavley, 2018).
Self-perceived impact of MHFA on helping behaviors measures the frequency with which
trainees


 within trainees’ social networks, respondents were asked to
indicate in the past six  months the number of people they applied MHFA skills to for six  types of
members within their social network (e.g., friend/family member, neighbor) (Ashoorian e


resentative survey of U.S. adults that was sponsored by the American Psychiatric Association (American Psychiatric Association, 2020). Items measured levels of comfort with discussing mental health with coworkers and supervisors and seeking mental he


encies, agency affiliation was
treated as a single categorical variable (i.e., affiliations were treated as mutually exclusive). Individuals who indicated being affiliated with multiple agencies were re-coded as “multiple agencies.” Data for agencie


blic safety /criminal justice
Department of Information
Technology &
Telecommunications (DoITT)  Medium Low Housing /transportation/infrastructure
Department of Probation (DOP)  Medium Medium  Public safety/criminal justice
Department of Tran


 for analysis as Bronx, Brooklyn, Manhattan [reference category], Queens, Staten Island, or other).
Analyses
We first examined item-level univariate descriptive statistics (means; frequencies) to
characterize participant responses in the pooled s


ttan, 20 versus 21 percent Queens, 4 versus 4 percent Staten Island, 14 versus 12 percent other), gender (78 versus 73 percent female), and ethnicity (24 versus 27 percent Hispanic).
However, differences were observed with respect to age distributi


ty?
• How, if at all, have community members who have been trained in MHFA shared their MHFA knowledge or skills with other community members who did not
receive MHFA training?
• How has MHFA changed mental health
knowledge/attitudes/stigma wit


ion. Recruitment emails to Latinx community leaders included both English and Spanish translations, and emails to Chinese community leaders included both English and Simplified Chinese translations. Following these initial emails, RAND staff also att


based leaders (July 2021)  11 4 4 4
Latinx community leaders (August 2021)  10 4 3 3
LGBTQ+ community leaders (September 2021)  7 5 3 3
HRA/DSS agency staff (November 2021)  4,995  14 5 3
a Invitations were sent by email from NYC governmental/age


taff debriefs held after each focus group, and review of the final transcripts (Guest, MacQueen, and Namey, 2012; McMahon and Winch, 2018; Nowell et al., 2017; Braun and Clarke, 2006). Together, this process generated theory-based codes (i.e., codes


use among recipients of MHFA?
• What was the impact of MHFA on trainees’ well -being?
• How are training experiences (i.e., time since completed training; MHFA trainer status, number of MHFA courses completed; receipt of additional non-MHFA mental


HFA required for job
Yes 814 30.85
No 1,722 65.25
I don ’t know/not sure  103 3.90
Occupation
Mental health and social services  586 22.21
Faith based  86 3.26
Education  558 21.14
Health care 371 14.06
Law enforcement/public safety


le and administrative data across most sociodemographic variables of interest. For example, survey respondents were well matched to trainees represented in administrative data in terms of borough of home residents, gender (e.g., survey: 78 percent fe


gh (home residence)
Bronx  16.29 17.68
Brooklyn  28.57 29.58
Manhattan  17.32 15.11
Other  13.83 12.29
Queens  20.12 20.91
Staten Island 3.87 4.42
Missing  0.00 --
SOURCE S: RAND MHFA web survey, summer 2021, and DOHMH administrative data .


ercent) agreed or strongly agreed that most people in their community think less of someone with a history of mental health problems.
Helping Behaviors
Respondents reported on their confidence in helping others with a mental health problem,
which


and assess for risk of suicide or harm; listen nonjudgmentally; give reassurance and information;
encourage appropriate professional help; encourage self -help and other support strategies . For more information on
the MH FA Action Plan, see Mental


th whom trainees use MHFA to help others in the community,
respondents reported on their recent interactions with individuals who had a possible mental
health problem, as well as their application of various MHFA skills with different types of indi


ighbor or someone from my communityCo-worker/someone who I  see as part of my jobClient/someone I provide services as part of my jobFriend/family member
% respondents32 Table 3. 3. Estimated Num ber of Individuals with Whom Respondents Used  MHFA Ski


ting rarely (12 percent) or never (6 percent) using skills to help themselves. Many respondents (39 percent) also endorsed having frequently (20 percent) or occasionally (19 percent) used MHFA skills or knowledge to get counseling or therapy for thei


th respect to the types of mental health training
received, which could be associated with variations in MHFA training-related outcomes. For example, individuals in occupations that are often characterized by the receipt of professional education in


gnificantly more likely to report being very confident in helping someone with a mental health problem (OR = 3.67, 95% CI 2.56–5.25, p < 0.001), to endorse knowledge of referral resources (OR = 1.70, 95% CI 1.06–2.72, p < 0.05), and to endorse occasi


ector) was significantly associated with confidence,
knowledge of resources, and helping behaviors. Although the magnitude of differences varied slightly across specific outcomes, individuals in nearly all job types/fields show ed consistently
lowe


Group contrasts (OR, 95% CI)
Occupation type (employment
sector)  p = 0.001 p < 0.0001  p < 0.0001  p < 0.0001  p = 0.003
Mental health and social
services  (ref) (ref) (ref) (ref) (ref)
Faith based  0.64 (0.32, 1.28)  0.85 (0.5 0, 1.45)  0.39


 related to mental health literacy could be augmented to best meet the needs of New Yorkers. This section provides information regarding respondents’ perceptions of MHFA training and future training needs.
Perceptions of MHFA Training
Respondents


meone in distress, and more than two-thirds (68 percent) occasionally or frequently offered first aid information and reassurance. Despite the fact that virtually all trainees had completed city-sponsored MHFA training more than one year prior to com


lem in the past six months. This is slightly higher than rates of such contacts reported in other studies of MHFA trainees. For example, among government employees in Australia, Kitchener and Jorm, 2004, reported that approximately 72 percent of MHFA


Being
Over 80 percent of survey respondents endorsed occasionally or frequently using information
from MHFA training to support their own well-being. Of significant note, approximately 40 percent indicated obtaining counseling or therapy for their


cency of MHFA Training
Were Associated with a Greater Likelihood of Applying MHFA Skills, But Employer-
Required MHFA Training Was Unrelated to Skill Application
Attaining certification as a MHFA trainer, completing more than one MHFA course, and


dorsed Community Mental Health Stigma as a Problem
Although most respondents did not endorse beliefs that mental health problems reflect
personal weakness (i.e., suggesting low personal stigma), perceptions of community attitudes
toward mental he


s correctly identified symptoms of depression after completing online MHFA training.
However, performance in this study on other training-related targets indicated potential room
for improvement. For example, on a test of MHFA knowledge correspond


n the ongoing application of MHFA and related skills to help others in their communities.  47 Chapter 4 . Survey Results  Among Subset of City Agency
Employee Respondents
Because of OCMH’s focus on changing the culture of government agencies, pro


sion  1.29
Department of Veterans’ Services  <1.0a
Department of Corrections <1.0a
Department of Housing Preservation and Development  <1.0a
Department of Information Technology & Telecommunications  <1.0a
Department of Probation  <1.0a
Departm


 transportation/infrastructure
Human Resources Administration
(HRA) Large Medium Health, community, social, or human
services
Office of Labor Relations  Large Medium Other
Police Department (NYPD)  Large Medium Public safety/criminal justice


ks.
Finally, we assess differences in use of MHFA skills among city agency employee
respondents by agency affiliations and agency characteristics. To address questions related to differences in MHFA training-related outcomes across city agencies a


1 5.36
Offered a distressed person basic “first aid ”
level information and reassurance about mental
health problems  22.37 44.55 18.58 14.51
Assisted a person who was dealing with a
mental health problem or crisis to seek professional help  24.


lls with clients in social services or health care settings), we also explored whether groups differed by self-reported occupation type to help contextualize findings related to use of MHFA skills with different types of individuals. City agency empl


02). Compared with  individuals affiliated with agencies that provide
health, community, and social/human services (reference category), respondents were significantly less likely to endorse being very confident in helping others with a mental healt


those who worked in agencies focused on education (OR = 0.65, 95% CI 0.49–0.88, p = 0.005), housing, transportation, and infrastructure (OR = 0.33, 95% CI 0.18–0.61, p = 0.0003), public safety/criminal justice (OR = 0.54, 95% CI 0.30–0.97, p = 0.04)


gh their employer. In contrast, fewer city agency employee respondents —but notably more than one in ten (11 percent)—indicated concerns about employer retaliation or fear of being fired if they were to seek mental health care. 57 Figure 4. 3. City A


kely to
agree or strongly agree that they could discuss mental health openly and honestly with coworkers (OR = 0.59, 95% CI 0.42–0.82, p = 0.002).
Perceived Need for Additional Training
To assess the extent to which city agency employee responden


Department of Housing Preservation and
Development  -- --
Department of Information Technology &
Telecommunication  -- --
Department of Parks & Recreation (DPR)  -- --
Department of Probation (DOP)  -- --
Department of Transportation (DOT)  --


Among those affiliated with a city agency, more than half of respondents indicated that they
felt comfortable discussing mental health with coworkers (65 percent) and supervisors (58
percent). Similarly, most indicated that they would feel comfor


dose agencies may be less inclined to want more training because training principles are infused throughout the organization. It is also possible that individuals in higher-dose agencies have greater access or exposure to refresher trainings or relat


a survey of educators in both rural and urban settings, Moon, Williford, and Mendenhall, 2017, found that 96 percent of respondents reported that they were likely or very likely to encounter students with mental health issues in their work, and 97 pe


or multiple tests. Additionally, we conducted sensitivity analyses to examine whether group differences
persisted after accounting for training and occupation-related factors (see Chapter 3). As detailed in Chapter 3 of this report, the effective su


kely to report being very confident compared with  those who identified as non-Hispanic White. With respect to 65 knowledge of resources (agree or strongly agree with the statement “I know where I can refer
individuals for help with their emotional


Group Differences Adjusting for Training and
Occupation-Related Covariates
To assess whether sociodemographic group differences from bivariate models persisted after
accounting for training and occupation-related factors found to correlate with MH


hic group variables and respondents ’ self-perceived frequency of engaging in helping
behaviors “as a result of Mental Health First Aid training, ” confidence in helping others with a mental health problem, and knowledge of mental health treatment o


onal help). Hispanic and Black respondents were also more likely than their non-Hispanic White counterparts to agree that MHFA training address es topics important to their
community. Moreover, compared with  respondents who identified as non-Hispan


address the specific needs of their communities in a culturally appropriate manner. These findings should be considered in the context of some limitations. For example, respondents differed from the broader population of trainees on some characterist


mi -structured focus groups, one for each population, guided
by the RE-AIM (Reach, Effectiveness, Adoption, Implementation, and Maintenance) domains. These domains are described in more detail in Table 2.3 (Chapter 2) and were used to organize the f


 subsequently contacted MHFA coordinators to implement the training in their community.
Stakeholders in the Chinese, African American, and SGM groups explicitly described how
offering MHFA training was aligned with their organizations’ existing mis


their hope was that the MHFA training would allow trained community members and volunteers to serve as a “first step” and “a bridge” to formal services:
they can be able to give better suggestions, and they encourage them [persons
with mental heal


, I think it ’s still —for a short time we ’ll have to change the thousand years
of these cultural barriers is not that easy. So we try to open up the dialog to
encourage people, as many a s we can, to participate in the event. Of course, there
ar


ty leaders described how trainees were using MHFA-related knowledge and skills.
They also identified specific aspects of MHFA training that they or their community members
perceived to be helpful or unhelpful.
Trainees ’ Use of MHFA Knowledge  and


e on.” They continued, “We’re
able to use a label and say this is what it is. It’s mental health.” Similarly, a Latinx community leader explained, “Identificamos muchas cosas que quizás antes, nosotros diríamos, «¿qué es lo que está pasando?».” [“We


hat the difference between depression,
schizophrenic, you know, bipolar and all of those things made it a little —it was
helpful for them to understand when their —especially when their family
members got a diagnosis, they could kind of put “Oh, t


al for diffusion was cited by one leader in the African American community as a motivator for offering MHFA training to their constituents.
Other aspects of diffusion were also identified. For example, an African American leader
described conductin


ations in families, and the workshops created “el canal de apertura” [“an opening channel”] to talk about mental health.
Leaders from the SGM community perceived that MHFA had empowered staff to better
serve their organizations’ clients. As one lea


. So they knew the topic and had empathy with the people, and they had— They were very flexible. . . . So they selected the staff very well. ]
In one case, a leader from the Chinese community noted that some trainers would benefit
from a less “tech


that their constituents were unlikely to be accessing web sites that advertise the trainings, and they recommended “creando alianzas con todas las organizaciones de las ciudades donde van a presentar estos talleres y las organizaciones hacer la invi


ing like that. It could last two or three hours. It would make
it easier for the people to participate. ]
A leader from the Chinese community said that they had initially planned to offer the standard
eight-hour training but ultimately offered it


ese lines, a leader from the Chinese community group expressed a desire for 84 more “case stud[ies]” during the training, which would “help people to understand the theory of
the situations,” although it was not clear whether this was referring to m


roups overwhelmingly asked for more MHFA trainings and were hopeful that MHFA 85 would eventually resume. As described earlier, SGM leaders expressed a desire for more
culturally tailored MHFA, if it were to be offered in the future.
Leaders in the


omes, and participation barriers and facilitators. We remind 86 readers that findings from this group represent the perspectives of only three staff members;
therefore, they should be generalized to other agency employees with extreme caution.
Moti


nd then the agency as an organization could follow up by identifying and codifying best practices for working with clients. Finally, to promote participation in future MHFA-like initiatives, one of the participants pointed out the advantage to 87 “a


that the dual-trainer approach led to clashes in style. One of them elaborated:
the way their style comes across, like, or the way that the information comes
across or is delivered to  you, it’ s like, it ’s totally different. . . . one is believab


ligned with Community Organizations’ Needs
Leaders of community organizations in all four populations were aware of long-standing
mental health needs in their communities, and this was a significant motivator to offer MHFA
training to their commun


 al., 2017) and as articulated by
community leaders, stigma and shame are barriers to accessing mental health treatment. Prior
research suggests that aspects of stigma, specifically negative attitudes toward help-seeking and greater self-stigma, ar


demic-
related disruptions to community activities and resource constraints. If MHFA or similar trainings are offered again in the future, city implementors might consider engaging communities in developing formal maintenance plans. For example, impl


y integrated other ministry activities in the course of the MHFA training day and offered breakfast and lunch as part of the training. There may be other considerations or supports (e.g., providing child care resources) that would promote participati


d for
videoconference trainings (e.g., for persons who could not travel, or persons with compromised immune systems). For context, we consider this finding in light of what is known from prior research examining differences in MHFA delivery and the


Participants Identified Priority Areas for Future Trainings and Other
Resources
Leaders in the Chinese, Latinx, and African American communities all desired a continuation
of MHFA trainings, and SGM leaders also expressed a desire for MHFA trainin


 participants did not appear to know each other prior to
participation, and therefore they may have been uncomfortable or unwilling to spontaneously disclose how training impacted their own mental health. We did not explicitly ask workers about the


orted using the
knowledge or skills learned in MHFA to help support their own well-being. This paralleled feedback from some focus group participants, who felt that MHFA training had a positive impact on their personal well-being. These findings are


n-focused agencies.
Community
MHFA trainings were offered in multiple CBOs and other publicly accessible venues
throughout the city in order to maximize training accessibility and extend the reach of MHFA to
all New Yorkers, including those from


of MHFA training in their native language was a major facilitator of MHFA’s success in their communities. In this respect, focus group feedback revealed potential opportunities to improve reach and perceived benefits of MHFA training in diverse comm


• Survey respondents presented with documented mental health needs, with over 50 percent reporting needing help for emotional or mental health problems in the past year and 8 percent meeting criteria for serious psychological distress in the past 30


respondents indicated the presence of community stigma—
agreeing that their community thinks less of someone with a history of mental health problems and that seeking treatment is seen as a sign of personal weakness.
• CBO leaders in the focus group


arency and reproducibility. A range of adaptation frameworks exist, and these
typically share common steps (Escoffery et al., 2019). The most relevant frameworks for
adaption of MHFA training in this context may be those frameworks that explicitly


hifting community norms around mental health. Social
marketing campaigns have gained traction as a tool to counter public stigma, but
reductions in stigma tend to be strongest among persons reporting awareness of the
campaign rather than among the


d to design and apply survey weights, the use of an English-language–only survey, and the closing of the survey when the target number of community-based trainee respondents was reached. Although the survey sample was broadly representative of the po


rying
exposure to MHFA training, as well as population-level outcomes (e.g., workplace absenteeism
and climate; community rates of unmet mental health need and discrimination).
Several limitations are specific to the focus groups. First, participa


orks, organizations, and communities that are primed to recognize and come to the aid of those who are in need of assistance. The positive findings must be tempered by the fact that this evaluation is limited in its ability to causally link the self-


-MFANYC, 2018. As of June 7, 2022:  https://www.rand.org/pubs/research_reports/RR2497.html
Banh, M. K., J. Chaikind, H. A. Robertson, M. Troxel, J. Achille, C. Egan, and B. J. Anthony,
“Evaluation of Mental Health First Aid USA Using the Mental Hea


pp. 1–10.
Crisanti, A. S., L. Luo, M. McFaul, H. Silverblatt, and C. Pyeatt, “Impact of Mental Health First
Aid on Confidence Related to Mental Health Literacy: A National Study with a Focus on Race-Ethnicity,” Psychiatric Services, Vol. 67, No. 3


e with a 20-Year Review,” Frontiers in Public Health, Vol. 7, 2019, p. 64.
Grant, J. Busby, C. P. Bruce, and P. J. Batterham, “Predictors of Personal, Perceived and Self-
Stigma Towards Anxiety and Depression,” Epidemiology and Psychiatric Sciences


g Messages in New
Zealand’s Like Minds Campaign and Its Effect on Stigma,” Social Marketing Quarterly, Vol. 27, No. 2, June 2021, pp. 82–98.
Kessler , R. C., P. R. Barker, L. J. Colpe, J. F. Epstein, J. C. Gfroerer, E. Hiripi, M. J. Howes, S.
L.


tal Health Systems, Vol. 3, No. 1, September 7, 2009, p. 19. 110 Mohatt, Nathaniel Vincent, Robert Boeckmann, Nicola Winkel, Dennis F. Mohatt, and Jay
Shore, “Military Mental Health First Aid: Development and Preliminary Efficacy of a
Community Tra


hief Technology Officer,
2018.
NYC Mayor’s Office of ThriveNYC, “ThriveNYC: Promoting Mental Health for All New
Yorkers,” 2021.
Olfson, Mark, Shuai Wang, Melanie Wall, Steven C. Marcus, and Carlos Blanco, “Trends in
Serious Psychological Distr


cial Determinants of Mental
Health Among New York City Adults, New York: New York City Department of Health and Mental Hygiene, 2019.
Wang, Philip S., Patricia Berglund, Mark Olfson, Harold A. Pincus, Kenneth B. Wells, and
Ronald C. Kessler, “Fai


CORPORATION
SAMUEL PETERSON, BRIAN A. JACKSON, DULANI WOODS, ELINA TREYGER
Finding a Broadly
Practical Approach for
Regulating the Use of
Facial Recognition by
Law EnforcementResearch ReportFinding a Broadly
Practical Approach for
Regulating th


gislators and policymakers, criminal justice offi -
cials, and society in general that should be understood, and further developments need to include efforts to
maximize the benefits of such technology while limiting the potential risks. This report


Approach for Regulating the Use of Facial Recognition by Law Enforcementivofficer, Clearview AI). We would also like to thank our peer reviewers, Kristin Warren from RAND and
Katie Kinsey from the Policing Project, who provided valuable feedback tha


uracy complicated by the true positive not
being included in the gallery (testing always includes correct answer)?
• What is the additive or interactive effect of human and algorithm error on overall accuracy?Finding a Broadly Practical Approach for


s That Might Come from
Facial Recognition Use
• Could FR use change police investigations by overemphasizing video evidence or diminishing other
investigative skills to solve crimes?
• Could the community view FR results as definitive or reduce rep


 willingness to exercise constitutional rights (e.g., because of
fears of state surveillance and the abuse of power), the potential for FR systems’ errors to lead to miscarriages
of justice, and analyses that have suggested that biases embedded in


 resources if technology acquisition efforts are disrupted or canceled; may
create societal harms from lack of restriction on inappropriate use of these technologies, given uncertainty in
management structures and oversight; and may sacrifice the o


 to address
those concerns effectively.
This report is intended as a first step in accomplishing those goals. We examine, objectively, the various
dimensions along which use of FR for law enforcement purposes can be considered. We also summarize


o note that each of the above methods of identification—both the primary and
secondary—has accuracy limitations that we will not discuss further in this report.
FR can be viewed as a key advancement in methods of identification, with characteristic


e database, FR algo-
rithms calculate the similarity in faces represented by pairs of images. Early geometry-based approaches
plotted points on faces (see Figure 1.2) to measure the distances between facial features. Recent advances
have used deep-


ion)
• Many
(identification/
surveillance)Probe image
• One (verification/
Identification)
• Many
(surveillance)Review
• Trained examiner
• Peer review
• Decision
• Submission of
another image FR comparison and
similarity scoresIntroduction: Th


inding a Broadly Practical Approach for Regulating the Use of Facial Recognition by Law Enforcement6fidence score at which they would like the software to return results. This is set typically to limit the number
of false positives. The software can


survey show that those who trust the police more also support police using FR technologies (see
Figure 1.4). When there is lower trust among the community members, there is also lower acceptance of the
use of these technologies (dark blue in Figure


g and how
FR could exacerbate both. In the most extreme application, a law enforcement organization, if given access to
a large-enough network of cameras equipped with FR, could track everyone’s public movements extensively
and with much less effo


gh-trust respondents
Low-trust respondents
High-trust respondents
Low-trust respondentsIntroduction: The Use of Facial Recognition by Law Enforcement9historical reasons, there are racial disparities in arrests and therefore an overrepresentation of r


bout
the practical implications of small differences in accuracy (e.g., 0.06 percent or lower depending on the
application).12 Questions of how well these tests generalize to real-world conditions also need to be answered.
12 Testing of demographi


moratoria are often temporary. Indeed, of the two states that had enacted full bans, Vir -
ginia has recently enabled use through regulation (the other state being Vermont).
Where FR is allowed, a more standardized approach would be of interest to a


tant features of different types of FR use; the risks, benefits, and trade-
offs; and policy guidelines that might address those risks. This description is intended to reflect the state of
knowledge and practice at present, in an effort to summarize


 Law Enforcement12Broader Concerns About Pervasive Surveillance and Artificial Intelligence
The technological capability to surveil large areas or track people across several areas using FR or other sur -
veillance technologies already exists today.


e questions as part of RAND’s ongoing American Life Panel survey, which periodically surveys a nation -
ally representative panel of Americans. In particular, we were interested in how trust in law enforcement
might be affected by people learning th


eir applications
by individuals and organizations. A use case is defined generally as an example application of a technology or
system to perform a specific task or achieve a specific goal (see, e.g., Brush, 2022). Often applied in the devel -
opme


volve a system of inputs and outputs involving both human decisions (e.g., as influ-
enced by policy, training, and experience) and FR software (e.g., data inputs, algorithm quality, software
settings). The entire system involved in any given use of


ddresses) of romantic interests, ex-partners or their
new partners, and celebrities. Officers could use the technology to identify both personal information and
personal locations or travel patterns, clear examples of misuse. Finally, another clear


igh-resolution images,
meaning that there are multiple potential ways to enhance an image, and it would not be clear which is cor -
rect (Menon et al., 2020). Thus, the choices made in image modification could vary with unknown impact
on the final


count for.
2 In this case, the FR algorithm would return the top-ranked results in terms of similarity or confidence score. This might be
used in an exploratory way but could return low-quality results.
3 We borrowed the label for this category fr


s (repeat); and providing private security, safety at
public events, and safety in public spaces—including automated CCTV and movement tracking, police body
cameras, and student-attendance tracking.
Use and Misuse Cases in the Current Study
Many o


 2.1
Hypothetical Use Cases for Interviews with Subject-Matter Experts
Type Description
Investigative use for theft CCTV images of a jewelry theft suspect are captured from multiple private businesses in the
same area. Police use multiple still imag


olars ( n = 5), researchers or oversight officials with expertise on FR use by
law enforcement ( n = 3), and FR software company representatives ( n = 4).
Recruitment
We used a combination of independent outreach, referrals from existing contacts,


ne to many, or many to many?
Data sharing What type of data are being shared? With whom and for what purposes?
Practicality Would other means of identification be overly burdensome, or is the situation time
sensitive? What other biometric identifi


ntifying individuals and concerns from civil
society groups stems partly from different emphases on the risks and benefits of using FR. Legal and policy
actions in reaction to FR use or proposed FR use can respond to those risks and benefits, seeki


centage of crimes for which a suspect is identified and arrested is quite low (e.g., homicide clearance rates
are near 50 percent nationally and decreasing over time; Braga and Dusseault, 2018; Ousey and Lee, 2010;
Rodricks, 2021). However, the us


e of FR, risks mostly outweigh any benefits. These
individuals fall into two groups: individuals whose photo is taken in a particular location and submitted to a
FR system for identification and the larger number of people whose photos are included


ot necessarily require malfeasance by investigators. Mental biases
can lead individuals to seek out information that confirms or reinforces their existing beliefs and theories, and such biases
could shape how investigators collect and use data in c


, and
increases resources to the criminal justice system. Perceptions of or actual misuse or wrongful arrests could
negatively shift societal approval. Some less serious types of crime may disproportionately be committed by
poor or marginalized gr


citated or missing carries similar risks
and benefits. One key question for this group is whether FR results can be used against the subject being
scanned (e.g., the subject has an open warrant). This also applies to victims, witnesses, and associa


s are chosen and that features of the photos themselves are similar (e.g., lighting, pose, size), so as
to reduce the chances of a false identification, although there are arguments that selecting lineups by similar
facial features using FR could i


oint often means that less intrusive means have been explored or are determined to be likely to fail. Concerns about whether FR use by law enforcement is justified across a variety of
situations:
• Will police use FR for minor crimes?
• Will FR use


any searches that are worth distinguishing. First, with aggrega -
tion, law enforcement is using archival or stored data to search many unknown faces and compare them to
gallery images. In this case, law enforcement might be going back through video


e of the search moves from one-to-one, to one-to-many, to many-to-many applications, the
level of individualized suspicion decreases, and FR is then being used more as a dragnet than a focused tool
that maximizes the benefit of its use. Additionall


lice by efficiently establishing subject identity, possibly improv -
ing officer safety, or ensuring that the right person is released from custody.
Despite substantial improvements in accuracy since even the late 2010s, there are still areas that n


k, and
White, 2019). Some of research in this area notes the importance of confidence judgments in assessing the
quality of the positive identification and finds that facial examiners are more consistent as a group, suggest -
ing the importance of


of our interviewees, however, had reserva-
tions about making image alterations beyond cropping and basic image edits, such as brightness, contrast,
and sharpness. Making more-drastic changes, such as closing mouths, opening eyes, or mirroring image


negative
identification rate) but produce weaker results because a top result can be returned that would not have met
the threshold had it been applied (Grother, Ngan, and Quinn, 2017). This is because although the top-ranked
candidate may be corr


t answer)?
• What is the additive or interactive effect of human and algorithm error on overall accuracy?Finding a Broadly Practical Approach for Regulating the Use of Facial Recognition by Law Enforcement32Accuracy: Sources of Bias
Concerns about t


n
• Contains preexisting biases (e.g., disparities in arrests)
Probe images
• Potentially more likely to reflect crimes committed in public
• Potential disparities in CCTV placement
Human
decisionmakingImage submission
• Requires deciding which ima


erns about accu -
racy (Grother, Ngan, and Hanaoka, 2019a). The larger the gallery, the likelier that more nonsuspects will
show up in FR searches and the lower the accuracy. Moreover, larger databases mean that more people will be
matched on a mul


eir images from the internet, when companies collect these data, they remain at a level of permanence that
prevents individuals from being in control of their image.
A key question for data sources, particularly when it comes to size, is the exact


nding the Landscape of Concerns35subject.13 In this application, the first step will be to look for any other supportive or disqualifying informa -
tion, including, but not limited to, whether the subject has a twin, whether the subject was out of to


everal steps, such as legislation, policy, training, and checks and balances (e.g., internal auditing, external auditing).
14 Ongoing tests in London do not appear to have shown improvement (Rollet, 2022). Concerns about what happens after police us


some police departments have come under
scrutiny for using FR on protestors, leading to concerns about retaliation, due-process violations, and First
Amendment violations.16 This also recalls the previous discussion on the scope of the search or th


stitutes a search,
and is therefore subject to the reasonableness standard, depends in part on whether it violates “reasonable
expectations of privacy” ( Katz v. United States , 1967). Fourth Amendment jurisprudence has long held that
government’s


in court without
that expert’s appearance ( Melendez-Diaz v. Massachusetts , 2009; Bullcoming v. New Mexico , 2011). Similarly,
introducing FR evidence, which could contain statements describing the results of FR analysis by the indi -
vidual that


rors that also have higher confidence. Jackson (2019) explores this idea, but more research is needed,
as is more research on human-machine interactions and how they affect decisionmaking (Peeters, 2020).
A second due-process concern FR raises is r


he Due Process Clause calls for a preliminary judicial inquiry into
the reliability of that eyewitness identification. Understanding the Landscape of Concerns39defendant. Some cases (e.g., People v. Beckley , 2010) have rejected photographs from the


specific individuals and their locations, which makes tracking and surveillance applications more efficient
and increases risks of pervasiveness. If these data are kept long term, there would be historical records for law
enforcement to access at


of potential matches is more invasive than just having your image
included in a search. Next, most people are not aware that their images are included in gallery data, and there
are questions about whether there should be automatic notice. There ar


 FR searches, meaning that
FR use is more easily tracked and authorized users can develop more-specialized expertise.
Data Quality
Next, data-quality variation is a potential concern. There is little known about how image quality, the most
importa


us on, leading them to overlook other potential suspects or types of informa -
tion that could aid the investigation. Unintentionally, continued reliance on FR could lead to diminishing
investigative skills over time. If a large part of the job is u


R, developing policies that balance the
risks and benefits of FR use by law enforcement will likely be a location- and jurisdiction-specific endeavor.
In this chapter, we present a road map for developing that path—looking at the different componen


nterviews. The options fall
broadly into the categories shown in Figure 4.1, with community approval being an overarching factor for the
other categories. Different options in each category are responsive to different concerns about risks affect -


 consequences if
implementation deviates
from intent
Community buy-in pre-FR implementationCan support
accountabilitySupports
appropriate
useDeveloping a Policy Road Map to Help Navigate the Landscape of Concerns About Facial Recognition45the basi


 the more intangible effects (both on direct victims and others around them, such as their families)
of violent crimes is more difficult. Nonetheless, estimates done by a variety of methods can provide some
insight. Figure 4.2 shows approximate val


e means that there is a reasonable basis based on the totality of circumstances to believe that a crime
has been committed or evidence of a crime exists. Probable cause is required for searches or arrests. FIGURE 4 .2
Relative Estimates for the Cost


ement
has sought to identify large numbers of individuals present in a given area—from mass detentions during
protests where there were some violent participants to the use of geofence warrants to identify all of the
cell phones present in an area


on data and, therefore, the ability to track specific vehicles visiting locations that reveal
potentially sensitive information about their owners.8
Prohibitions or restrictions on the scope of FR use could therefore be elements of a policy approac


9 Reportedly, some states have prohibited the combination of FR with police body-camera footage, which would be a broad-
scope application with officers moving among the population in many areas during the course of their daily activities (Slaight


fit of a specific case being closed being better positioned to assess the overall benefits
and risks of use.10
Centralized decisionmaking about use can also help support external accountability, since patterns of
use and misuse could be more easily


ecurity and Bureau of Justice Assistance, U.S. Department of Justice (2017, p. 17) for a reflec-
tion of some of these issues in a model policy document.
13This is the level of external control proposed in Lynch, 2019, and similar to the enacted Mass


oth from practical constraints, including the
conditions in which a photo of a criminal incident was recorded or the quality of the camera doing so, 16
but also from manipulation of photos by users before submission (e.g., mirroring a partial photo,


in the outcome of FR systems, since there may be racial, socioeconomic, and other factors that
affect the likelihood of an individual being arrested (and therefore photographed at booking) during an
encounter with police (Slaight and LeCloux, 2020)


the results) and how results should be returned (number and ranked or unranked) are important factors, but
different agencies appear to be using different approaches.
With the development of independent testing, quality levels for different algori


ner knowing that some of the individuals returned by the system committed crimes
previously may create a confirmation bias and skew the examiner’s assessment of whether those individuals
match the photo associated with the new criminal incident. An


or retrospective tracking
of one’s movements and activities could chill important activities and degrade individual rights. Retention of
just FR matches is potentially problematic from a risk perspective, since not retaining the associated images


stan -
dards for the types of action that can be taken at different levels of match quality or certainty
• limits on how long individuals’ photos can be stored in FR databases used in the jurisdiction and how
long data on matches from past searches


stored (e.g., security), managed, shared, retained, and purged.
This would provide critical feedback as to how well this biometric information is being handled and uncover
potential risks in the data life cycle. Accuracy audits might select a rando


 to whatever path a jurisdiction has set for use of FR, there should be conse -
quences for straying from that path. Without some consequence associated with policy violation, there is real
risk that the perceived benefits to individual officers or


dministered may effectively regulate officer use of FR, it may not have significant
external trust value if skeptical audiences cannot see its results.23
Any discussion of the regulation of law enforcement use of technological or other capabilities


nd when police should use FR is public concern that the
risks of such use may outweigh its benefits, what the public knows about law enforcement utilization of FR
and whether the public authorizes or approves of that use matter. Members of the publ


p. 7,
emphasis added).Policy and law should define
• what the penalties are at both the individual and agency levels when use deviates from approved use
cases
• what types of exigent circumstances justify relaxation of jurisdiction-level restrictio


d not utilize FR: Then, inappropriate use would become a substantial liability
for such cases, balancing out any perceived value that using FR could let investigators close the case
more quickly. Disclosing the quality of matches could also manage


s mistrust and opposition to the technology. However, more-substantial
community involvement  in authorizing FR use, as well as development of the ground rules for its use, could
31 Although not framed in the context of transparency or making their


y being measured in FR use?), trans -
ferring inequity (e.g., do the data being used have embedded inequity?), exacerbating inequitable harms (e.g.,
is FR used in a way that enhances current inequities?), or compromising inequity oversight (e.g., ar


ays that maximize benefits and
minimize risks, along with careful review to ensure that these regulations were upheld. This narrowing could
consist of limiting FR to investigating serious crimes, increasing the level of authorization required, redu


imit susceptibility to improper interpretation or application of FR results. The next section (green section)
addresses some of the features of FR systems as they are applied in law enforcement settings, which always
include a combination of comput


gencies use FR. Moreover, were there accepted standards, software companies could create
software, interfaces, and settings that adhere to those standards and could not be manipulated by individual
1 Stanford University’s Institute for Human-Center


forcement, but it would theoretically be a valuable alternative method
of remote biometric identification in cases in which there is no usable image of a suspect’s face or could be
combined with FR to strengthen identification. Object recognition,


sted above (A, B, C). In this order, the question about FR systems were presented
before questions about trust in police. The second group received the questions about police using FR sys -
tems first, followed by the question on trust in police (sp


 priority of this section):
• What are the necessary regulatory or oversight elements that should govern the use of this technology?
 –Would certification be a potentially useful route to ensure ongoing regulation?
 –What about limits to access?


g/en/latest/news/2021/06/scale-new-york-police-facial-recognition-revealed/
angus mcdiarmid, “Joseph C Ilacqua,” October 28, 2012. As of January 20, 2023:
https://www.flickr.com/photos/angusmcdiarmid/8130830700
Anwarul, Shahina, and Susheela Dahiya,


the Impact of Citizen Complaints and Crime Rates on Drug Arrests,” Criminology and Public Policy , Vol. 11,
No. 4, November 2012.
Facial Identification Scientific Working Group, “Published Documents,” webpage, undated. As of September 4,
2022:
ht


meras-
bc9036918ae0
Goldberg, Rebecca Darin, “You Can See My Face, Why Can’t I? Facial Recognition and Brady ,” HRLR Online ,
Vol. 5, 2020.
GRC World Forums, “Can European Regulators Stop Clearview AI?” May 27, 2022. As of January 10, 2023:
https:


ala, and Li Fei-Fei, “Evaluating Facial Recognition Technology: A
Protocol for Performance Assessment in New Domains,” Denver Law Review , Vol. 98, 2020.
Howard, John J., Laura R. Rabbitt, and Yevgeniy B. Sirotin, “Human-Algorithm Teaming in Face Re


Vol. 39, No. 4, 2022.
Kahn, Jeremy, “After Roe, Fears Mount About A.I.’s Ability to Identify Those Seeking Abortions,” Fortune ,
June 28, 2022.
Katz v. United States , 389 U.S. 347 (1967).
Kindt, Els J., “Transparency and Accountability Mechanisms


e Effectiveness of an Acoustic Gunshot Location System in
St. Louis, MO,” Policing: A Journal of Policy and Practice , Vol. 6, No. 1, 2012.
Mares, Dennis, and Emily Blackburn, “Acoustic Gunshot Detection Systems: A Quasi-Experimental Evaluation
in


itimacy and Social Networks on Active Gun Offenders,” Journal of Criminal Law and
Criminology , Vol. 102, No. 2, Spring 2012.
Payal, Parekh, and Mahesh M. Goyani, “A Comprehensive Study on Face Recognition: Methods and
Challenges,” Imaging Science


 Baltimore
Sun, March 9, 2021.
Rodriguez, Roger, “Facial Recognition: Art or Science?” Vigilant Solutions, undated. As of September 2, 2022:
https://www.sheriffs.org/sites/default/files/Whitepaper%20Facial%20Recognition.pdf
Rollet, Charles, “Londo


Ester Gonzalez-Sosa, and Julian Fierrez,
“DeepFakes Detection Across Generations: Analysis of Facial Regions, Fusion, and Performance Evaluation,”
Engineering Applications of Artificial Intelligence , Vol. 110, April 2022.
Turner v. United States


leted his Ph.D. in criminal justice at the University of Cincinnati.
Brian A. Jackson is a senior physical scientist at the RAND Corporation. His research focuses on homeland
security, criminal justice, and emergency preparedness. Jackson received


CORPORATIONResearch Report
JASON M. WARD, RICK GARVEY, SARAH B. HUNTER
Recent Trends Among the
Unsheltered in Three Los Angeles Neighborhoods
An Annual Report from the
Los Angeles Longitudinal  Enumeration and Demographic Survey (LA LEADS) Project


bservation period, these activities did not result in longer-term decreases in the overall levels of unsheltered people, vehicles, and encampments.
 QOur counts and Los Angeles Homeless Services Authority (LAHSA) annual point-in-time count data from


c-tive allocation of resources for addressing the city’s homelessness crisis.
Study Geography
For this study, we selected three neighborhoods—Hollywood, Skid Row, and Venice—partly because of their historically high rates of unsheltered home-lessness


ull 4and Venice. We alternated the timing of these counts
between early-morning hours (6 to 9 a.m.) and night-time hours (9 p.m. to midnight) to allow us to assess whether there was significant variation in population counts depending on the time of


ility Criteria
Each potential respondent was screened for eligibil-ity with two questions. The first question was taken from the LAHSA demographic survey and asked where the individual had spent most of their nights over the past 30 days. The second


ce by using signs suggesting the vehicle was not being regularly moved (such as low tire pressure) and other signals, such as having covered windows, fogged up windows, or large amounts of possessions in or around the vehicle. Additionally, several o


spent most of the
past 30 nights. If a screened individual answered that they had been staying in one of our specified infor-mal shelter types (a car, van, RV, tent, or makeshift structure), then we asked them how many people they shared this shelte


ltipliers are not directly compara-ble, but we reproduce the relevant LAHSA multipliers alongside the more aggregated LA LEADS multipli-ers to give a sense of how they compare with one another. In general, our multipliers are slightly larger than the


n the relative shares of individuals, cars, vans, RVs, tents, and make-shift shelters. For example, if an area were counted and found to have 90 individuals and 10 tents and we applied a multiplier for the tents of 1.5 (suggesting that every other te


ounting t he weeks
of our study period beginning with the last week of September 2021, the date of our first enumeration of Skid Row. The annual p ercent-
age change value given in the table is scaled from the change-per-week point estimate to refle


ge share of each category of informal shelter (cars and vans, RVs, or tents and m akeshift
shelters) in our enumerations by neighborhood across the 12 months of data collection. The “Unadjusted” column presents the act ual sum
of individuals and


January 2022February 2022March 2022April 2022 May 2022 June 2022 July 2022
August 2022
September 2022October 2022
Unadjusted count Range of adjusted estimates11FIGURE 4
Skid Row Unsheltered Population
COVID-19 testing operation in area on day of enum


nt); in Hollywood and Venice, one-quarter of respondents reported that their current spell was drawing individuals to the area from elsewhere and
potentially drawing individuals onto the streets from the considerable stock of permanent support-ive h


(81 percent) among respondents in Skid Row. The length of these TABLE 3
Demographic Characteristics of Survey Participants (percentages)
Participant CharacteristicAll
(N = 418)Hollywood
(n = 104)Skid Row
(n = 211)Venice
(n = 103)
Age
18–24 5 5 2 9*25


ods for each category by creating indicator variables for membership in each group (e.g., age 18–24) and regressing each indi cator variable on indica-
tor variables for the Hollywood and Venice neighborhoods (with Skid Row as the omitted neighborhoo


 homeless in Los Angeles 40 38 38 43
Factors that prevented housing move-in
Never contacted for move-in 44 36* 47 45
Lack of privacy 40 49* 38 35
+
Housing safety 35 45* 31 33+
Paperwork issues 29 31 27 29Hours or curfew 25 23 23 33*
Housing location


tistically significantly different from Hollywood at the 90-percent confidence level are indicated with a plus sign  (+).
a Of respondents not answering “yes” to this question, one-half (5 percent of the total) answered “maybe” in a similar pattern


provide follow-up for housing placements because of the forced relocation of people experiencing home-lessness (Cohen, Yetvin, and Khadduri, 2019). About 40 percent of respondents were concerned with a lack of privacy offered in the housing options a


ey is conducted in each neighborhood only over a few days. Thus, leaving aside any differences arising from the use of weighting, the LAHSA data may more strongly reflect short-term population characteristics while the LA LEADS data may reflect longe


mates of age shares differ in several
cases. For example, our share of survey respondents who were 18 to 24 years old in Venice was more than double the LAHSA estimate. On Skid Row, we surveyed considerably fewer 25- to 54-year-olds and more individ


nsus tracts as our blocks), matches that are exact over multiple census tracts (where, for example, three to six census tracts might equal two of our blocks), and very close area matches across multiple LA LEADS sub-geographies or census tracts where


o better understand the extent of such mecha-
nisms affecting the accuracy of the annual count, we generated small-area comparisons of our own count data occurring prior to and following the LAHSA PIT count data collected in late Febru-ary 2022. Our


191203, and 191204 in Hollywood
020406080100120140160Individuals
January February March April May0510152025303540Cars and vans
January February March April May
05101520RVs
January February March April May020406080100120140160180Tents and makeshift sh


s, and RVs. The LAHSA count for tents in these tracts is similar to our count from around one month before but sig-nificantly higher than our result from approximately one week later. However, it is possible that such a discrepancy could be explained


eks.
We observe a similar pattern for the area
immediately to the south, shown in Figure 9. Again, the discrepancy for cars and vans may relate to a similar modest geographic difference in the com-parison areas (LA LEADS geography is similarly slig


 match except that we count one  block farther north to Marine Street west of Sixth Avenue and Ozone Street between Lincoln Boulevard and Sixth Avenue.LA LEADS LAHSA
FIGURE 12
Comparison of LA LEADS and LAHSA Unadjusted Count Data for Census
Tract 2


informal overnight shelter. However, this distinction is unlikely to explain the higher number of RVs (26 versus our surrounding counts of 7 and 15), which are much more likely to actually be overnight shelter, conditional on being on the street in t


being used as shelter.
These comparisons also suggest potentially
important differences in the way tents are counted (for example, whether long, contiguous makeshift shel-ters are counted as a single unit or multiple segments and how tents obviousl


peated
enumerations suggest that outlier events—such as
cleanup activities, inclement weather, or other one-
off occurrences—can meaningfully influence esti-mates of this hard-to-track population.
Our exploration of multiple approaches to the
app


housing receipt may yield large dividends in
housing those most in need.Notes
1  We also included the Veterans Row encampment along San
Vicente Boulevard in front of the West Los Angeles Veterans Affairs campus in our early survey and enumeration


IT on
these nights also likely leads to a large, systematic undercount
because of the increased likelihood that people will attempt to
shelter out of sight for increased protection from the elements
(Boone, 2019). 30LeGras, Christopher, “I Partic


e, Seung Choi, et al., “Initial Item Banks and First Wave Testing of the Patient-Reported Outcomes Measurement Information System (PROMIS) Network: 2005–2008,” Journal of
Clinical Epidemiology , Vol. 63, No. 11, 2010, pp. 1179–1194.
Centers for Dise


D Corporation, RR-A1890-3, 2023. As of
January 20, 2023: https://www.rand.org/pubs/research_reports/RRA1890-2.html
Weare, Christopher, Counting the Homeless: Improving
Knowledge of the Unsheltered Homeless Population , Goldman
School of Public Pol


w.rand.org/about/research-integrity.
RAND’s publications do not necessarily
reflect the opinions of its research clients and sponsors.
 is a registered
trademark.
Limited Print and Electronic Distribution Rights
This publication and trademark(s) c


Sponsored by the Lowy Family GroupRecent Trends Among the
Unsheltered in Three Los Angeles Neighborhoods: An Annual Report from the Los Angeles Longitudinal Enumeration and Demographic Survey (LA LEADS) Project
Appendixes
JASON M. WARD, RICK GARVEY,


systematic effort to monitor changes in the number of people living unsheltered over a full year, helping to bolster what is captured by the annual Los Angeles County point-in-time count, which provides a snapshot in time of this population. The rese


 the project. We also thank Heather Schwartz at RAND and Nichole Fiore at Abt Associates for providing peer review for this report.

   v Contents, Figures, and Tables
Contents
About This Report ................................................


Poinsettia
Recreation
CenterBarnsdall
Art Park
Wilshire
Country ClubN. Fuller Ave.N. Fuller Ave.N. Martel Ave.N. Vista St.
Romaine St.190401
18920 1 189201
.e.....Fountain Ave.190402 1190301 b
191204191203191301a9 a
191301 b
191000c
1910000aa00191520


191 85 60 33 45 654 995 804
08/23/22 (PM)  269 173 56 57 31 80 666 1,007 792
09/15/22 (AM)  226 166 64 52 28 65 601 917 737
Table B.4. Enumeration Totals by Category: Veterans Row
Date and Time  People  Cars and Vans  RVs Small Tents  Medium Ten


 Annual Report from the Los Angeles Longitudinal Enumeration and Demographic Survey (LA
LEADS) Project , RAND Corporation, RR -A1890 -2, 202 3. As of January 15, 2023:
https://www.rand.org/pubs/research_reports/RRA1890 -2.html   8 Table C .1. Full


on tests differences between Venice
and Hollywood (omitting Hollywood). Results for Hollywood and Venice that are statistically signif icantly different from
Skid Row at the 90 -percent  confidence level are indicated with a star (*). Results for V


percent  confidence level are indicate d with a plus sign (+).   11 Table C.3. Housing Needs and Preferences of Survey Participants , with Recoded “ Other”
Responses Included  (percentages )
Need or Preference  All
(n = 430) Hollywood
(n = 104)


liness  50 49 51 49 50
Negative interactions with staff 42 35 41 50+ 42
Hours or curfew  35 40* 29 44* 33
Other rules  26 23 26 30* 33
Othera 4 5 2 8* 0
NOTE:  Mutually exclusive percentages may not sum to exactly 100 due to rounding. We perform


s stopped you from moving into housing in
LA?” (see the row labeled “Factors that prevented housing move -in” in Tables  5 and C.3) on answering “yes” to
the question , “Since you have been homel ess in LA, have you been offered housing?”  Mutually


Sponsored by Adagio HealthA Needs Assessment
of Women Veterans in Western Pennsylvania
Final Report to Adagio Health
DANA SCHULTZ, SUSAN L. LOVEJOY, KAYLA M. WILLIAMS, KERRY
LINDQUIST, TEAGUE RUDER
CORPORATIONFor more information on this publicatio


er understanding of the needs of women veterans and the extent to which these needs are being met will help veteran-serving organizations provide the best support possible to women who served.
Adagio Health, a health care provider based in Western


hose of men, and women veterans
also differ in significant ways from their nonveteran counterparts. Few studies have explored these variations, and this has translated to potentially missed opportunities to improve support for women during and after


r complicate women veterans’ treatment needs.
Our assessment of the needs of women veterans in Adagio Health’s service area drew on
national and regional survey data and patient data from Adagio Health. We supplemented this information with a revie


nuary 2018
and April 2022 differed somewhat from the nonveteran women in its service area. Specifically, they tended to be older and higher earning than their nonveteran counterparts and were more likely to be White and married. However, the two gro


Q -9), which assesses depression
severity, and the Cut down, Annoyed, Guilty, and Eye -opener Adapted to Include Drugs (CAGE -AID) tool, which
assesses substance use.  p = 0.001 for any positive SBIRT prescreen; n ot significant  (n.s.) for any pos


o Health service area face in navigating services and benefits and accessing the support they need. Interviewees noted that these challenges are often interconnected. For example, women veterans who are unable to access adequate care for mental healt


d them or to address the needs of women veterans specifically.
Opportunities to Address Challenges and Improve Supp ort for Women
Veterans
Since launching its Women Veterans Initiative in 2018, Adagio Health has made strides in
addressing the c


s for reproductive health care and increase its own capabilities for providing and referring for infertility care.
Provide Multiple Channels to Ensure That Women Veterans Are Aware of Available
Services and Supports
Women veterans noted that thei


es veteran engagement training to providers and staff, but there are still opportunities to improve the identification of women veterans with trauma histories through routine screenings and to ensure seamless care transitions to strengthen trust when


gure 5.1. Marital Status of Women Served by Adagio Health ............................................. 36	Figure 5.2. Family Size of Women Served by Adagio Health ................................................ 36	Figure 5.3. Family Income of Women


.............................. 42	Table 5.10. Pregnancy Testing for Women Served by Adagio Health ................................... 45	Table 6.1. Resources in the Adagio Health Service Area for Women Veterans ..................... 48	Table 6.2. Men


 2022). Furthermore, as of 2017, women who served in the post-9/11 era accounted for more than one-third of all women veterans (see Figure 1.1).   2 Figure 1.1. Service Era Percent age of Women Veterans

SOURCE: Features data from National Center


owever, these rates are lower than for nonveteran women, 13.7 percent of whom live in poverty
(National Center for Veterans Analysis and Statistics, 2019).
In terms of health status, veteran women nationally reported having more days of poor
physi


of the health status of veterans, VA conducted a
2012–2014 follow-up study of Gulf War–era veterans (Dursa et al., 2019). Among women veterans who had deployed during that conflict, the most-prevalent physical health conditions 61%
30%
25%22% 21% 20


s), and health care utilization (Creech et al., 2021).
For the reproductive health of veteran women, there is mixed evidence on whether women
veterans have higher rates of stillbirth, miscarriage, birth defects, and other birth outcomes than nonve


ith annual estimates of PTSD ranging from 11 to 20 percent (National Center for
PTSD, undated). Both military women and men have experienced combat trauma, an important predictor of both PTSD and depression. Almost 75 percent of Iraq and Afghanistan


al support exacerbated mental health problems, particularly for those with complex trauma histories (Scoglio et al., 2022). Another study noted the challenges women veterans face in engaging with social support networks because of mental health issue


 progress (Pub. L. 117-135, 2022).
Despite these efforts, women veterans continue to have unmet health care needs, including
for mental health and family planning services.
Veteran women are less likely than veteran men to receive evidence-based


nt rate for veteran women
was 4.2 percent, which is lower than the rate for both veteran men and nonveteran women (Diaz ,
2022). Confounding factors, such as MST and IPV, are related to worse occupational functioning for veteran women compared with


sent a greater proportion of
the military population than the general population (10.7 percent versus 4.2 percent), there is
little information about this population of women veterans (Lehavot and Simpson, 2013). Some studies have examined whether


Adagio Health asked the RAND Corporation to conduct a community health needs
assessment to characterize the needs of women veterans in the Adagio Health service area, the extent to which existing services and supports meet their needs, gaps in the


lth to launch its Women Veterans Initiative in 2018. The initiative has four primary objectives:
• to make Adagio Health a significant provider of non-VA health care for women veterans
in the areas it serves
• to position Adagio Health  as part of


h care with physical health care
services for all patients, but it was motivated in large part by the needs of women
veterans. Staff were trained in the Screening, Brief Intervention, and Referral to
Treatment (SBIRT) model, as well as motivationa


 vet centers, which provide counseling and intervention for veterans and service
members who have deployed to a combat theater or experienced MST, including many
who are otherwise ineligible for VA care.
Outreach, Education, and Awareness
Regiona


ssess the local availability and capacity of different programs, services, and supports for women veterans. Finally, we describe our approach to the qualitative assessment of women veterans in the Adagio Health service area.
Literature Review
In o


tinuous variables and chi-squared tests for categorical variables.
Patient  Data  Analysis
Adagio Health provided us with deidentified individual-level data from its Enterprise
Practice Management and electronic health record systems.
This combi


dagio Health
service area.
• United Way of Southwestern PA 211 is an online search engine for resources available in Allegheny, Armstrong, Beaver, Butler, Fayette, Greene, Indiana, Lawrence, Mercer, Washington, and Westmoreland counties. It includ


nt identification and recruitment, capacity, needs of women veterans, barriers to providing services, coordination of services, client engagement with services, and advocacy.
We then conducted phone interviews with 12 women veterans. Adagio Health


.0001 .

Veteran women in the Adagio Health service area are less often White than nonveteran
women (81.5 percent versus 89.7 percent), whereas the percentages of Hispanic or Latino are similar between the two groups (see Table 4.2). This racial a


f rounding . Missing, refused, or unknown not shown.
p < 0.0001.
There are no differences in the employment status of veteran and nonveteran women in the
Adagio Health service area. Fifty-seven percent of veteran women and 55 percent of nonvete


n
Women  Veteran
Women  Nonveteran
Women
Hearing difficulty  2.5%  0.8%  3.7%  2.6%  13.7%  12.3%
Self-care difficulty  3.2%a 0.8%  3.6%  3.2%  11.3%  10.0%
Vision difficulty  4.1%b 1.3%  4.0%  2.7%  4.8%  7.0%
Independent living difficu


and April 2022, Adagio Health served 26,736 women, including
890 women veterans. The data in the table and figures presented here are the deidentified
individual-level data that Adagio Health provided and include the entire January 2018 to April 2


. Race  and Ethnicity of Women Served by Adagio Health
Race  and Ethnicity  Veteran  Women  Nonveteran
Women
Racea
White  alone  86.0%  81.3%
Black or African American  alone  11.5%  13.2%
Asian alone  0.3%  0.9%
American Indian or Ala


e 50 and over age group had a single visit. We also looked at counts of patients and visits by month and quarter and did not see any strong trends across the two groups. There was a small uptick in veteran utilization around COVID-19, but that appear


em visit.
a p = 0.014.
b p = 0.040.
c p = 0.004.
 Initial birth control methods were similar for veteran women patients and nonveteran women
patients (see Table 5.6). About 22 percent of both groups used oral contraceptives, a patch, or a
v


ve health
educationb 26.6%  36.9%
Smoking cessation counseling  10.2%  12.0%
Pregnancy options counseling  3.5%  4.4%
Other counselingc 22.1%  28.8%
a p = 0.026.
b p < 0.001.
c p < 0.001.
 Around 38 percent of veteran women patients ha


 screened positive on the PHQ-9  43 with a score of 10 or higher. For the CAGE-AID questionnaire, similar percentages of veteran
and nonveteran women patients received a clinically significant score of two or higher.
Figure 5.5. Positive Behavioral


counseling during an Adagio Health visit. Thirty-one percent of women patients with a positive pregnancy test had at least one prenatal visit at Adagio Health.
We also looked at contraceptive care for women patients who had a negative pregnancy tes


, the regional resources in the inventory are either exclusively for
veterans and their family members or are programs or organizations that serve broader populations but offer veteran-specific programs (see Appendix B, Table B.1). Of the 63 resourc


le 6.1. Resources  in the Adagio Health Service Area  for Women Veterans
Category  Types of Services  Number of
Regional
Programs or
Providers  Number of
National
Online
Resources
Mental health and
substance use Treatment facilities; suici


facilities. Furthermore, completion of the surveys on which the data are based is entirely voluntary, so some facilities may not be represented, or their responses may not be complete. The survey results indicated that, of the 138 mental health trea


e an hour of their time each week to provide free counseling sessions to
service members and their families as part of the Military and Veterans Program.
Potential clients may submit a request for an in-person, telephone, or video meeting;
review


eriences, Trips, and Gifts  for Veterans
This group of 25 programs and services includes those that we deemed lower priority for
referrals by Adagio Health care navigators for various reasons, such as a low probability of
selection for the program


auma in the women veteran population emerged as a salient theme. This includes not only high rates of MST but also combat trauma and other types of traumas encountered before, during, or after military service. This history of trauma and possibly rel


 being implicitly given that message when a veterans’ service organizations offered her the application for auxiliary rather than the regular membership when she inquired about joining. Another stakeholder said , “We’re kind of
invisible. Women don’


facilities would not serve them, “I thought, I’m not going to file a claim. Claims were for guys who lost their arms or legs.”
The group of women veterans we interviewed learned through word of mouth from other
veterans that VA care may be an optio


w to help us care for you better?” to open space for women veterans to request any accommodations they may need or find helpful without specifically asking them to retell—and perhaps reexperience—traumatic events from their past. Another recounted wi


identified by stakeholders is growth in nontraditional families seeking housing support, including family units with adult children and/or grandchildren. Single-room occupancy emergency housing programs are often unable to support women veterans and


fy women veterans who may be in need; ensure they are made aware of services that can support them and their families; and engage them in these programs, services, and support. This is often a highly personalized, labor-intensive process that many or


g on-site child care or supporting telehealth appointments with an
iPad, hot spot, or other resources to promote widespread internet access. One woman veteran said that she would have taken advantage of telehealth but she did not own a working compu


ts for those age 50 and older (24.6 percent versus 20.4 percent). When it came to behavioral health screenings conducted during office visits, veteran women patients were more likely to respond positively on the SBIRT prescreening than nonveteran wom


ny of the national online resources align with the issues and challenges faced by women
veterans. The inventory includes 32 national online resources that focus on employment and career development, 31 that offer individual and family support, 20 th


atients with resources and services in the community through outreach activities and strategic partnerships with veteran-serving organizations. With these efforts, Adagio Health made strides toward addressing the issues and challenges that women vete


f available resources combined with trust issues. Through its Women
Veterans Initiative, Adagio Health has made strides in establishing relationships with veteran-serving organizations, such as PAServes, VA vet centers, and VLP. Despite these effort


the qualitative assessment, we learned that it is important to recognize that partner agencies may face challenges with staff turnover and continuity of care, which might interfere with their ability to effectively serve their clients. Adagio Health


Many women veterans lack knowledge about services and supports available to veterans.
With transition support that was overwhelming, not individually tailored to their needs, or many
years in the past, there can be a vacuum in women veterans’ awar


 could
partner with another agency to provide activities, such as cooking or yoga classes, at Adagio Health  office locations. For example, Adagio Health could partner with the Veterans Yoga
Project to offer yoga classes at an Adagio Health locatio


erans with mentors (e.g., eMentor, VETS—Beyond the Uniform, American Corporate Partners, and Stand Beside Them), assist ing with resume preparation (e.g., Military
and Veterans Appreciation Trust Foundation, O*NET Online—Military Crosswalk, and Skil


 time to build
trusted relationships with underserved populations.
• Systematic, ongoing data collection will help identify unmet needs and gauge the progress of improvement efforts.
• Explore additional opportunities to solicit input and feedback


6%  11.6%  8.1%
Divorced  13.1%  12.9%  23.1%  11.9%  11.7%  22.5%   72    Women in
the United States  Women in the Adagio Health Service Area
Characteristic  All Nonveterans  Veterans  All Nonveterans  Veterans
Household Type
Married


usehold  49.7%  49.7%  44.9%
Cohabiting -couple household  2.5%  2.5%  3.0%   76  Women in Allegheny County
Characteristic  All Nonveterans  Veterans
No spouse or partner  30.8%  30.7%  34.3%
Household Size
1 or more under 18  26.2%  26.


 private coverage  19.2%  19.1%  30.1%
No health insurance coverage  3.4%  3.4%  2.7%
SOURCE: Features data from the five -year ACS data file (2016 though 2020) from
U.S. Census Bureau, undated.
NOTE: Some percentages do not sum to 100 because


%  78.2%  74.9%
Mean percentage of income spent on
housing  21.8% 21.8% 22.8%
Disability Status
Age 20 –39
Hearing difficulty  0.4%  0.4%  0.0%
Self-care difficulty  0.7%  0.7%  0.0%
Vision difficulty  1.5%  1.5%  0.0%
Independent


.0%
Household Size
1 or more under 18  30.8%  30.9%  17.8%
Average household size  2.68 2.68 2.84
Average family size  3.11 3.11 2.99
Average number of children  0.51 0.52 0.39
Educational Attainment
Less than 9th grade  0.0%  0.0%


dated.
NOTE: Some percentages do not sum to 100 because of rounding.

   87 Table A.5. Characteristics of Women Veterans in Fayette County
 Women in Fayette County
Characteristic  All Nonveterans  Veterans
Age
20 to 24 years  6.2%  6.


ion difficulty  2.2%  2.2%  0.0%
Independent living difficulty  5.4%  5.4%  0.0%
Ambulatory difficulty  3.2%  3.2%  0.0%
Cognitive difficulty  7.4%  7.4%  0.0%
Age 40 –64
Hearing difficulty  2.8%  2.8%  0.0%
Self-care difficulty  5.2%


 Attainment
Less than 9th grade  0.0%  0.0%  0.0%
9th–12th grade, no diploma  6.5%  6.6%  0.0%
High school graduate  40.2%  40.4%  28.9%
Some college, no degree  16.4%  16.4%  20.2%
Associate ’s degree  11.3%  11.2%  22.0%
Bachelor ’s


Age 40 –64
Hearing difficulty  2.1%  2.0%  8.4%
Self-care difficulty  2.7%  2.7%  3.0%
Vision difficulty  2.8%  2.8%  1.9%
Independent living difficulty  6.3%  6.2%  9.6%
Ambulatory difficulty  9.1%  9.1%  11.2%
Cognitive difficulty


raduate  37.8%  38.0%  21.8%
Some college, no degree  19.8%  19.6%  33.7%
Associate ’s degree  11.3%  11.4%  7.0%
Bachelor ’s degree  15.9%  15.8%  23.1%
Graduate or professional degree  8.3%  8.3%  9.8%
Missing or unknown  1.8%  1.8%  1.8%


care difficulty  2.9%  3.0%  0.6%
Vision difficulty  3.0%  3.0%  0.6%
Independent living difficulty  6.1%  6.1%  0.9%
Ambulatory difficulty  9.5%  9.6%  6.3%
Cognitive difficulty  6.1%  6.2%  2.8%
Age 65+
Hearing difficulty  12.5%  12.


 degree  17.2%  17.1%  23.7%
Associate ’s degree  9.2%  9.2%  9.4%
Bachelor ’s degree  13.3%  13.3%  9.7%
Graduate or professional degree  8.0%  8.0%  11.3%
Missing or unknown  3.3%  3.4%  0.0%
High school graduate or higher  91.5%  91.4%


40 –64
Hearing difficulty  3.4%  3.4%  5.5%
Self-care difficulty  3.5%  3.5%  9.6%
Vision difficulty  3.3%  3.3%  1.5%
Independent living difficulty  7.1%  7.1%  10.5%
Ambulatory difficulty  12.1%  11.9%  30.0%
Cognitive difficulty  7.9


Less than 9th grade  0.0%  0.0%  0.0%
9th–12th grade, no diploma  5.3%  5.4%  1.5%
High school graduate  46.7%  46.9%  34.6%
Some college, no degree  14.8%  14.8%  20.5%
Associate ’s degree  10.8%  10.8%  17.8%
Bachelor ’s degree  12.5%  12


ndiana Counties
 Women in Armstrong and Indiana
Counties
Characteristic  All Nonveterans  Veterans
Age
20 to 24 years  10.2%  10.3%  0.0%
25 to 34 years  12.7%  12.7%  10.9%
35 to 44 years  13.0%  13.0%  13.3%
45 to 54 years  15.9%


2.4%  6.6%
Ambulatory difficulty  1.5%  1.5%  0.0%
Cognitive difficulty  3.3%  3.3%  6.6%
Age 40 –64
Hearing difficulty  2.7%  2.7%  2.3%
Self-care difficulty  2.2%  2.2%  0.0%
Vision difficulty  3.6%  3.5%  18.5%
Independent living d


ng
Armstrong County
Veterans Affairs  Helps veterans and their survivors file claims for state and
federal veteran  benefits  Armstrong
Balanced Heart Healing
Center  • Offers group and individual health education and
healing services by ment


ois Vet Center  • Offers confidential free help for veterans, service
members, and their families in a nonmedical setting
• Includes counseling services for such needs as depression, PTSD, and the psychological effects of
MST
•
 Offers a Women ’


utic needs  Allegheny
Jefferson County Veterans
Affairs  Provides referrals for benefits, employment, financial
assistance, mental wellness and substance use,
posttraumatic stress, and more  Jefferson
Lawrence County
Community Action
Partners


Provides homes in Pittsburgh to help homeless
veterans with transitional housing  All
Pennsylvan ia CareerLink  Connects veterans with employment representatives at any
of the statewide Pennsylvania CareerLink offices to receive free job counselin


he greater Pittsburgh and Western Pennsylvania area
throughout their transition and after  Allegheny
Venango County Veterans
Affairs  Provides referrals for benefits, employment, financial
assistance, mental wellness and substance use,
posttraum


egheny Health Network Department of Psychiatry  Pittsburgh  Allegheny
East End Behavioral Health
Hospital  Pittsburgh  Allegheny
Glade Run Lutheran Services  Pittsburgh  Allegheny
Jefferson Hospital  Behavioral Health Inpatient  Clairton Alleghe


llsville Fayette
Family Behavioral Resources  Outpatient Mental Health Clinic  Uniontown  Fayette
Southwest Behavioral Care, Inc.
Uniontown  Fayette
Cornerstone Care  Waynesburg  Waynesburg  Greene
Cornerstone Care  Mount Morris  Mount Mor


Partial hospitalization or day treatment  18
Federally Qualified Health Center  10
Payer or payment information
Cash or self -payment  73
Medicaid  76
Medicare  64
State -financed health insurance plan other than Medicaid  68
Private healt


burgh  Allegheny
UPMC McKeesport Hospital
McKeesport  Allegheny
UPMC Mercy Hospital  Acute Medical Detox  Pittsburgh  Allegheny
UPMC Western Psychiatric Hospital  Narcotic Addiction Treatment Program  Pittsburgh  Allegheny
VA Pittsburgh He


id
Society
Franklin  Venango
Family Services and Children ’s Aid
Society   Oil City  Venango
Oil Region Recovery
Franklin  Venango
Deerfield Centers for Addictions
Treatment   Warren  Warren
Family Services of Warren County   Warren


0
Federally certified opioid treatment program  15
Cognitive behavioral therapy  68
Individual counseling  69
Group counseling  68
Family counseling  66
12-step facilitation  56
Brief intervention  54
Contingency management and motivational i


mprehensive
process of medical care, rehabilitation, and professional development
Army Resilience
Directorate  Provides policy, resources, and capabilities for soldiers and U.S. Army
leaders to increase their r esilience and readiness
Army Wom


eer transition advice
and resources (online mentoring program)
Esposas Militares
Hispanas USA Armed
Forces  Supports Hispanic military spouses through awareness and information in
Spanish about everything that surrounds them in the military tha


ry -friendly
employers by industry, function, and location
Hire Heroes USA  Provides free employment assistance to help veterans and military spouses
get hired, and helps companies hire and retain them
Hire Veterans  Connects veterans with civi


 • Offers a blog for service members and veterans to help them take
advantage of their military benefits and achieve financial independence
• Offers articles on military life, including military pay and bene fits,
insurance, money management, and


obtaining VA benefits and participating in local and national issues pertaining
to all women veterans
Network of Care for
Veterans, Service Members, and Their
Families  Provides an interactive web portal for service members, veterans, their
f


, and support
Project Welcome Home
Troops  Provides techniques for stress relief, greater health, and well -being and
empowerment to improve the quality of life for returning veterans and their
families
Purple Heart Foundation  • Supports prog


agement, referral services, and temporary
financial assistance for housing to veterans facing homelessness
• Provides an opport unity for a veteran to meet with staff at a location
convenient for the veteran
Splunk  Offers access to e -learning


to the U.S. Court of
Appeals for veterans ’ claims
Veterans Employment and
Training Service  Offers employment service s, programs, and resources to help veterans,
service members, and military spouses get jobs
Veterans in Pain • Provides a na


njured in Combat  Offers peer support groups, retreats, military sexual trauma intensive therapy
retreats, workshops, summits, and more in partnership with Challenge
America
Women Veterans Call
Center  Provides a call center dedicated to answeri


are gifted homes
Canines for Service  Provides veterans from all conflicts with service -connected mobility
challenges, PTSD, TBI, and MST with highly trained service dogs at no
cost or promotional consideration
Freedom Hunters  Offers therapeu


o teach fly -fishing and fly -tying techniques to
disabled service members and veterans
Sierra Club —Military Outdoors  Provides veterans with the opportunity to get outdoors when they return
home after service
Sisterhood 4 Vets Provides vacati


r health care professionals
VA Blind Rehabilitation
Services  Supports blind and low -vision veterans and active -duty service members
in regaining their independence and quality of life to enable their
successful integration into family and com


ervice- connected disabilities to build, remodel , or purchase an adapted
home
VA Suicide Prevention Helps veterans find local mental health and suicide prevention resources,
family and friends connect to care and resources for coping with a suic


ance Abuse and Mental Health Services Administration
SBIRT Screening, Brief Intervention, and Referral to Treatment
STI sexually transmitted infection
TBI traumatic brain injury
UMPC University of Pittsburgh Medical Center
VA U.S. Department of


Sciences, Vol. 3, No. 2, 2017.
Creech, Suzannah K., Carey S. Pulverman, Jennifer N. Crawford, Ryan Holliday, Lindsey L.
Monteith, Keren Lehavot, Jennifer Olson-Madden, and Ursula A. Kelly, “Clinical Complexity in Women Veterans: A Systematic Review


ison B., Ines Poza, Vivian Hines, and Donna L. Washington, “Barriers to
Psychosocial Services Among Homeless Women Veterans,” Journal of Social Work Practice in the Addictions, Vol. 12, No. 1, 2012.
Henry, Meghan, Tanya de Sousa, Caroline Roddey, S


w American Security, November 17, 2015.
Lehavot, Keren, Jodie G. Katon, Jessica A. Chen, John C. Fortney, and Tracy L. Simpson, “Post-
Traumatic Stress Disorder by Gender and Veteran Status,” American Journal of Preventive Medicine, Vol. 54, No. 1,


st updated September 7, 2022.
Office of People Analytics, 2021 Workplace and Gender Relations Survey of Military Members:
Overview Report, U.S. Department of Defense, September 2022.
Orshak, Jennifer, Lacey Alexander, Andrea Gilmore-Bykovskyi, and


visible Wounds of War: Psychological and Cognitive
Injuries, Their Consequences, and Services to Assist Recovery, RAND Corporation, MG-720-CCF, 2008. As of February 26, 2023: https://www.rand.org/pubs/monographs/MG720.html
U.S. Census Bureau, “Amer


Sponsored by Westlands Water DistrictGrowing Toward a
Low-Carbon Future
Estimating Greenhouse Gas Emissions in
California’s Westlands Water District
MICHELLE E. MIRO, NIDHI KALRA, JONATHAN LAMB, NIHAR CHHATIAWALA
CORPORATIONFor more information on


work has been informed through structured collaboration with Westlands staff and key interested parties. This report will be of interest to Westlands and other irrigation and agricultural districts in California, as well as various natural resources


ount for the most and least GHG emissions?
• How do GHG emissions from crop production and related land uses change under climate change?
• How would changes in crop production and related land use practices impact GHG emissions and key resource tr


 and sequestering carbon dioxide (CO 2) when land is converted to and from
cropland or grassland, and from carbon stocks stored in those lands, resulti ng in net emissions of approximately 12.1
MTCO2e q. The net emissions are the sum of fluxes in f


ent of Conservation, undated).
California crop emissions have already declined over time, in part because of (1) a reduction
in acres cultivated, resulting in a decrease in synthetic fertilizer use, and (2) an uptake of drip and sprinkler irrigatio


gy, and climate—and addressing them requires a cross-sectoral approach to policy and planning. Furthermore, prioritizing and implementing changes in agricultural practices must account for future uncertainty that would affect the trade-offs presented


tural GHG emissions and strategies to reduce emissions. Chapter 3 presents findings of our integrated model, and Chapter 4 offers conclusions and recommendations for California agriculture and water management based on our findings.
 5 Chapter 2.


round the world, in such works as Johnson et al., 2016; Dyer et al., 2017; Halverson et al., 2012; De Gryze, 2011; Necpalova, 2016; Weiler et al., 2017; Lemma et al., 2021; Sanz-Cobena et al., 2017; and Afroz et al., 2021.
Considering Future Uncerta


ied a number of alternative practices that might reduce GHG emissions from Westlands, as shown under “Policy Levers” in Table 2.2. We also identified a number of metrics relevant to Westlands to evaluate these options (see “Performance Metrics” in Ta


d land use practices that are most similar to current conditions in
Westlands. The baseline scenario assumes no change in the mix of crop types and no expansion or reduction in land used for solar energy generation. To be able to isolate the relativ


e and Precipitation Values for  GCMs  and Clima te Scenarios

Land Use Model
Overview
Emissions from soil and field management practices were estimated with a dynamic Land
Use Model adapted from the Intergovernmental Panel on Climate Change (I


ng of native grassland, or kept fallowed), and an equal area assumed to be permanently fallowed. All crop characteristics required to simulate the carbon and nitrogen cycle per IPCC guidelines, including assumptions related to crop yield, were either


or each crop, we gathered estimates of the carbon, nitrogen, and lignin content of above- and below-ground dry mass, as well as any trimmings or deadfall, to calculate nutrient inputs each period (shown in Table 2.3). Soil carbon was divided between


re uniform across the district each year (silt and clay composition were not utilized in the model calculations). The net change across all three pools results in a positive or negative carbon
flux in units of carbon, which, following IPCC guideline


ndergo some form of rotation to maintain soil health and
improve crop yield. Because these vary by grower, year, and other conditions outside of our
modeling framework, we applied a crop rotation scalar to emissions to account for reductions in emi


ed data carry their own assumptions around data
relevance (for example, crop yields vary with time, agricultural practices, etc.), we attempted to pull as many data from similar sources as possible. Unless explicitly described in the “Modeling Alter


 Acreage Category  Lifespanᵃ
(years)  Annual
Irrigation
Monthsᵃ  Annual
Irrigationᵇ
(AF) Equipment
Emissionsᵃ
(lbs./acre ) Annual Fertilizer ᵃ
(lbs.
nitrogen/ acre) Yieldᶜ
(lbs./acre )
Broccoli  1 4 1.8 5,148 170 16,000
(U.S. Departme


we used the following hierarchy for yield estimates:
1. yield reported by Westlands (Cartwright, 2022)
2. recent yield figures from Fresno County Department of Agriculture, 2020
3. California-specific yield from U.S. Department of Agriculture Nati


om historical data and management practices. It is intended to capture (1) the
crop water demand and resulting energy use from pumping for irrigation and (2) fluctuations in water available for irrigation and, thus, the irrigated acreage of Westland


In addition to reporting the annual water supply and demand, the Water Use Model passes
key information to the other two models. The amount of groundwater and surface water used in a given year is passed to the Energy Use Model to calculate emissio


er is used and 270,000 AF of groundwater is used under the current-state scenario, the average water EF is calculated in Equation 1.

(0
.38)∙(504)+(0.62)∙(9.7)=	197.5	lbs	CO#	AF%"  (1)
Farm Fuel Consumption
UC Davis Cost Studies, 2022, includes


rbon intensity of the California grid is then estimated  by Equation 3, where CI is
carbon intensity.
𝐶𝐼&)&*+!,-
)./=𝑁𝐺%&∗450)&*+!,-
)./	+(1−𝑁𝐺%&)∗7)&*+!,-
)./  (3)

To calculate the offset of solar for Westlands, we used Equation 4, where 𝑆𝐶𝐼 i


those authors validated their own results based on previously measured California data because of the similarities in climate between Greece and California. The study by Steenwerth et al., 2015, simulates wine grape cultivation in Central California


d is assumed to remain
fallowed. The restoration case converts the land to a composite native grassland. Each
solar case converts the area from fallowed to utility-scale solar in the indicated year
(2025, 2035, or 2045).
• Energy use options: Thi


cycle processes. During dry years, less precipitation is available, and CVP allocations may be reduced. This requires more land to be fallowed or more water to be pumped from groundwater if crop water demands exceed available water supply. Fallowing


re
positive when tree crops turn over. On average, this results in net positive emissions under the assumptions of this model, but CO
2 emissions are likely sensitive to the initial and final age
composition of orchards. We can also see that per-ac


ertain crops. However, if all farms in the district moved from current tillage practices to a no-till system, net emissions would be reduced to approximately 70,000 MtCO2eq annually. According to our model, this reduction would not require changes in


d or no tillage might have additional benefits in the reduction of crop water
demand, but modeling these was beyond the scope of this study.
 38 as opposed to 2033 in the current system. Solar 2035 only provides marginal emissions
reductions, an


 in this study was further expanding solar in the district by converting a portion

16 We account ed only for future climate change and future changes in the carbon intensity of California’s electricity
grid. Other future sources of change were no


es are not altered. It suggests that introducing no-till tillage practices can significantly reduce emissions by retaining more carbon in soils and reducing the amount of on-farm equipment needed for full tillage. This practice, according to our

1


to the crops shown in this study, such as processing tomatoes, rotations of row crops could result in emissions reductions of around 25 percent or more. Each of these strategies offers additional co-benefits for agriculture and the environment not c


ic
lightning, the decomposition of prior crop residues, animal droppings, and applied fertilizers. However, this is typically in the form of nitrites (NO
2-) and ammonia or ammonium (NH 3, NH 4+)
that cannot be used directly by plants. Nitrifying b


JV South;
Fresno County  2018 Yes Cited cost study is current and geographically comparable.
Pistachios  Baldwin et al., 2020  SJV South  2020 Yes Cited cost study is current and geographically comparable.
Pima cotton  Hutmacher et al., 2012a  S


92 No There is no current comparable cost study or alternative
source, so an older study was applied.
Pomegranates Day, Klonsky, and
Moura , 2010  SJV South  2010 No There is no current comparable cost study  or alternative
source, so an older s


ramento
Valley  2015 Yes Estimates combine data from both cost studies, utilizing a
representative value across both studies when possible.

 47 Abbreviations
AF acre-feet
CDFA California Department of Food and Agriculture
CH 4 methane
CO


t Hembree, Craig E. Kallsen , Giulia Marino, Blake Sanden, Themis J. Michailides, Florent Trouillas, Daniele
Zaccaria, Linda Harris, and Donald Stewart, Sample Costs to Establish and Produce Pistachios, University of California Agriculture and Natur


s Greenhouse Gas Inventory,”
webpage, September 6, 2021b. As of February 7, 2023: https://ww2.arb.ca.gov/sites/default/files/ghg-inventory-doc/doc/docs3/3c4_agsoilmanagement_direct_cropresiduenitrogen_n2o_2019.htm
 51 California Climate Investment


house Gas Inventories, 2019.
Chao Liang, Joshua P. Schimel, and Julie D. Jastrow, “The Importance of Anabolism in
Microbial Control over Soil Carbon Storage,” Nature Microbiology, Vol. 2, No. 8, 2017.
Christensen, Libby O., Ryan E. Galt, and Aliss


isleading Trends in Sustainability Attributes of No-Till,” Global Change Biology, Vol. 26, No. 6, 2020.
Day, Kevin R., Karen M. Klonsky, and Richard L. De Moura, Sample Costs to Establish and
Produce Pomegranates, U.C. Cooperative Extension, 2010.