mes. Research on the effectiveness of most of the health promotion and education efforts is
45
scarce, although there is some limited evidence of positive effects on certain risk behaviors known to lead to poor outcomes. Home visiting has been widely


on timing for programs, services, and supports clusters in the prenatal and postnatal periods, with relatively few appropriate for delivery in the preconception period (Figure 7).
47
Figure 7. Categories of Programs, Services, and Supports by Interve


ngle mother are less likely to be retained in programs (Roggman et al., 2008) and that ethnicity or minority status is also related to attrition, with some groups more likely to engage with services while others are less likely (Brand and Jungmann, 2


recruitment, retention, and participation. For instance, there is potential to incorporate principles like default opt-in for appointment scheduling (Chapman et al., 2010), gains-framed messaging about behavioral changes or treatments (Toll et al., 2


evention and intervention services
• ensuring access to high-quality, patient-centered care
• investing in prevention and public health at the community level
• fostering health equity by eliminating disparities and unequal treatment
• developing dat


, 2019).
Outside the federal government, other national efforts to prevent and reduce infant mortality include the March of Dimes’ Prematurity Campaign, which focuses on reducing prematurity, the leading cause of infant mortality (March of Dimes, 201


eceived a Medicaid waiver that covered long-acting reversible contraception for women on Medicaid.
In 2009, Ohio’s Infant Mortality Task Force issued a report with a series of recommendations that included the formation of a statewide organization to


 and Infant Mortality Rates in 2009, 2012, and 2015
    Initiative Launched
        Location
Primary Focus
2009
Infant Mortality Rate
         2012
2015
                  Black
White
Black
White
Black
White
         Baltimore (city) Milwaukee (city)


per 1,000. When Cradle Cincinnati launched, the infant mortality rate stood at 8.4 per 1,000. While the infant mortality rate returned to 9.4 per 1,000 in 2015, it is still too soon to examine the possible impact of the strategies just outlined.
Star


nsin
Wisconsin’s Lifecourse Initiative for Healthy Families was launched in 2009 to reduce disparities in birth outcomes between black and white infants (Lifecourse Initiative for Healthy Families, n.d.). In Milwaukee, the initiative’s Community Acti


 strategies, and implementing specific activities. Last, client-centeredness is at the core of many of the initiatives, such that the strategies and activities are designed and carried out to maximize engagement.
58
5. Inventory of Allegheny County P


x family medicine practices
 59
affiliated with UPMC participate in the IMPLICIT (Interventions to Minimize Preterm and Low Birth Weight Infants Using Continuous Quality Improvement Techniques) interconception care project, which involves maternal sc


 Start initiative that tested the effectiveness of the birth center model, and it continues to offer a suite of pregnancy and childbirth supports to its mothers.
The Birth Circle, part of the UPMC Department of Family Medicine, is a doula program tha


rams. Pregnant and postpartum women can also access these programs directly or through referrals from other programs or services.
ACHD supports three nurse home visiting programs. The Nurse-Family Partnership program follows the evidence-based nation


ormation on program components, capacity, and reach.
Table 10. Allegheny County Health Promotion and Education Interventions
Home Visiting Programs
        Program Components
     Capacity and Reach
     Allegheny Link
     Nurse-Family Partnership
H


effectiveness in improving birth and infant outcomes.
Care Coordination
Pregnant women in Allegheny County with Medicaid coverage can access maternity care coordination, which has emerging evidence of improving outcomes such as birth weight, preterm


ch operates out of Homewood Children’s Village, offers education, resources, and support to expectant parents and parents of young children.
Pregnant women in Allegheny County have access to Text4baby and its weekly information texts through local pa


ent services, and an in-home substance abuse assessment and mentoring program. Familylinks’ two Family Treatment Centers prioritize pregnant women for its residential treatment program.
Two outpatient clinics situated within hospitals offer services


e evidence-based or evidence-informed models
Serves about 6,000 children and families annually
       Genesis House
Assistance in finding permanent housing after delivery, counseling sessions, classes on prenatal care, parenting, smoking cessation, a


o six months postpartum)
Halfway house program: Residential treatment with psychiatric and health care services
Outpatient program: Partial hospitalization, intensive outpatient programming, counseling, therapy, and resource coordination
In-home serv


 Supports (as of 2018)
     Preconception Prenatal
Neonatal Postneonatal
       Medical Setting
Preconception/ interconception care
 One Key Question PATH
 IMPLICIT ICC
One Key Question PATH
Prenatal care/ enhanced prenatal care
 Healthy Beginnings P


ram
Healthy Start Life Skills Classes
Healthy Start Fatherhood Program
AIU Responsible Fatherhood Program
Healthy Start Life Skills Classes
         72
6. Allegheny County Birth Outcomes, Risk and Contextual Factors, and Intervention Participation
To


ry (14 percent); SIDS or other ill-defined causes
74
(12 percent); and perinatal respiratory or cardiovascular disorders (9 percent). The remaining 21 percent of infant deaths were caused by other factors, including perinatal infections, fetal blood


8.3 per 1,000 births, whereas the rate among women who smoked during pregnancy was 9.3 per 1,000 (Figure 14). Among all
          78
births, the rates of low birth weight and preterm births were close to 215 per 1,000 among women with any substance a


These were the only interventions for which we had participation information from any of the data sources that fed into the IMPreSIv database. For all of the other programs, services, and supports described in Chapter 5, we were not able to obtain da


es Mental health support services
Preconceptiona Pregnancy Postpartum
      ✅
          82
between WIC and income benefits was 0.55. However, what this figure shows most starkly is that there is almost no correlation between the medical-setting inter


UPMC system. Among these deliveries, infant mortality rates ranged from 10.2 per 1,000 live births for contraceptive counseling to 13.7 per 1,000 for genetic counseling, with all of the rates well above the average rate for Allegheny County (see Fig


 63.1 per 1,000 for women who had a birth doula were both below the rates across all deliveries (see Figure 19 note).
86
Figure 19. Birth Doulas in the Infant Mortality Prediction System with Intervention Management Database
  1 00
75
Rate per 1,000


or all deliveries, the infant mortality rate was 6.49 per 1,000 live births, the low birth weight birth rate was 86.52 per 1,000, and the preterm birth rate was 109.61 per 1,000.
Behavioral health services represent another category of support and re


s of infant mortality generated during this step fit the three types of data described in Chapter 2 (reportable information, potentially reportable information, and health care information) and the different time periods of our analysis (preconceptio


service providers and health care providers have access to different information, the different data types reflect the information each type of provider may be able to obtain.
With these predictors, missing data could be handled by excluding observat


ry. On the other hand, when predicting behavioral health service participation after delivery, we add indicators of other intervention participation postdelivery, as well as preconception and during pregnancy.
93
Predictors of Intervention Participat


f receiving any prenatal care were the race or ethnicity of the father, insurance status, and mother’s smoking behavior. Finally, missing birth interval information was another important predictor. Birth interval is recorded in the birth certificate


s. It is important to note that Family Support Centers offer a wide array of services for parents, some of which pertain to pregnant or postpartum women but many of which do not. Further, each Family Support Center offers its own unique mix of servic


ikely to utilize. Methodologically, greater accuracy in the predictive models enables us to control for selective intervention participation. The most common and accepted measure of model accuracy is the AUC scores, which combine the true positive ra


ders can also utilize the accurate predictions of intervention participation to better understand which women are most likely to use specific interventions and direct them to programs, services, and supports to meet their needs. Overall, the models p


these methods enable us to interpret our estimates as causal with only one reasonable assumption, conditional unconfoundedness. This assumption means that our predictions of infant mortality and intervention participation account for enough of what


effects indicate more effective reduction of infant mortality risk, even small effects are important. This is because of the terminal and dichotomous nature of the infant mortality outcome. For example, an estimate of −1 percent would mean that the


sisted housing). The limitations section of Chapter 10 has a fuller discussion of the limitations of our methods (including the conditional unconfoundedness assumption, missing data, selective observation, and multiple hypothesis testing).
105
Medica


of preconception/interconception care were observed only for those who received services at a UPMC-affiliated facility. Vaccinations, genetic counseling, and contraceptive counseling were also offered by other health care providers, such as FQHCs, Pl


ll women. The estimates for contraceptive counseling, genetic testing, and vaccinations are clustered, large, and increasing as the risk of infant mortality grows. In contrast, the estimated effects of preconception exams are near zero and generally


lity during different time periods, or it could be a by-product of having fewer observations with which to detect statistically significant differences (see Appendix C for results).
Doula Support
The effectiveness of doula support (i.e., having a dou


in the risk of infant mortality for each level of risk above 50 percent, with the effects ranging from 2.1 to 10.0 percent across the risk levels (Figure 38). However, the estimated effects of home visiting after delivery were not statistically sign


d
 114
by the wide array of services and supports available at Family Support Centers, many of which are unrelated to pregnancy, and the differences across centers in the mix of services offered, this does not explain how Family Support Centers could


educed risk of infant mortality (Figure 41). Participating in Family Support Center services for two years doubled the effect to a 3-percent reduction in the risk of infant mortality (multiplying two years by 1.5 percent gives this estimate). Longer


fant deaths occur shortly after birth, it is likely that the nutrition counseling drives this result. The estimated impact increased as the risk level increased, as did the size of the confidence interval (Figure 42). For those predicted at a greater


articipation in behavioral health services during specific time periods (Figure 44). However, for participation in behavioral health services before conception at the greater than 50 percent predicted risk level, the estimated effect is a statistical


or Medicaid. Given the correlation between socioeconomic status and infant mortality risk, it may be that the women who participate in ACDHS-funded behavioral health services have higher overall risk profiles than the women who participate in behavio


lower bounds of the true effects.
Intervention category. The results demonstrate that medical interventions, which are the most targeted to the health of the mother and baby, were estimated to have the largest impact. While our estimates of the effec


Further, the effects of home visiting during pregnancy were magnified as the risk level increased. This indicates that earlier involvement in home visiting programs may pay off with better outcomes, particularly for those most at risk for poor birth


ns are designed to achieve other objectives, they play an important role in community-based efforts to reduce infant mortality, particularly for those most at risk of poor birth outcomes. Since estimated intervention effects varied by risk level, the


ically, first pregnancy).
Next, for each intervention, we explored how the effects of the interventions are altered when combined either with other interventions or with income, medical, or other benefit programs. Any given intervention is unlikely t


s for us to assess their combined effects (for reference, see Figure 16 in Chapter 6).
Second, the results in this chapter focus on those with a predicted risk of infant mortality greater than 50 percent. Focusing on those with a greater than 50 perc


ffect of preconception care/interconception care was a significant increase in risk of infant mortality by about 10 percent. This finding may be related to the multiple hypotheses being tested, which can produce results that incorrectly appear to be


 Education
Within the health promotion and education intervention category, the IMPreSIv database includes information on home visiting, including the Healthy Start, Early Head Start, and Nurse- Family Partnership programs.
Home Visiting
Of all the d


of ever receiving WIC benefits for women with the specific risk factors and a greater than 50 percent predicted risk of infant mortality (Figure 53). Among the demographic risk factors examined, WIC was estimated to be most effective for teens, youn


21). We also present results describing the effects of the interventions for women with additional risk factors in Appendix C.
Table 21. Summary of Intervention Effectiveness by Risk Factor and by Other Interventions
Medical Setting
Doula support Tee


For these women, the estimates of increased effectiveness for most of the interventions suggest that the programs, services, and supports are reaching high-risk populations and having an impact. The findings related to these demographic risk factors


women with an opioid use disorder. Similar to its effect for smoking, the effect of WIC in reducing risk for women with opioid use disorder may represent the program’s general focus on nutrition and other factors that contribute to healthy developmen


ticipation, parental characteristics, maternal health risk factors and behaviors, pregnancy and delivery observations and diagnostics, postdelivery observations and diagnostics, and socioenvironmental risk factors. The quantity and quality of informa


ons.
     • Provider awareness of the availability and effectiveness of different types of interventions for a range of issues may be limited.
• Effectiveness of many interventions is magnified when combined with other interventions addressing differ


n, interconception, and prenatal care are particularly pronounced among black women.
• Early and regular care addressing maternal health has potentially cascading effects for infant health.
 8. Leverage opportunities to connect with women early durin


re able to accurately identify women at high risk of infant mortality. This is critical because the estimates show that interventions are most effective for women with the highest predicted risk of infant mortality. Identifying women most at risk for


ervention categories, we found virtually no correlation between the medical-setting interventions (preconception/interconception care, prenatal care, and doula support) and the non-medical-setting interventions in the health promotion and education c


ed risk. The risk assessment and referral tool (see recommendation 1) will help health care and community-based service providers to understand the individual risk profiles and make referrals to interventions based on the predicted risk of infant mor


ctions of intervention participation that will be implemented in the risk assessment and referral tool to better understand which women are most likely to use specific interventions, and provide optimized referrals to additional programs, services, a


viders is important but not sufficient to coordinate medical and nonmedical care and to ensure that the wide-ranging needs of high-risk women are met. Previous studies have demonstrated that care coordination in the health care setting can generate s


rdinator, and the client engagement grant program.
Because our models demonstrate that infant mortality can be accurately predicted using a limited set of information available either during pregnancy or preconception, the risk assessment should be c


st pregnancies likely generates longer- term benefits that we could not observe in this study. Whether the interventions contribute to women feeling better informed about pregnancy and childbirth, address risky behaviors, provide breastfeeding suppor


who make up 2.8 percent of the population in the city of Pittsburgh. With Allegheny County’s growing immigrant population, programs that provide doula support, such
150
as the Allegheny Health Network’s Immigrant Health Program, should be expanded so


 by the risk assessment and referral tool.
Limitations
Our approach has several limitations, including two types of missing data, selective observation, conditional unconfoundedness, and multiple hypothesis testing.
151
Missing data. While the IMPreS


examined (vaccinations, genetic counseling, and contraceptive counseling) were observed only for those who received services at a UPMC-affiliated facility. However, these services are also offered by other health care providers not included in the d


ense, this study is a combination of many studies. Consequently, we cannot recommend focusing too heavily on any one of the many hypotheses that we tested. We can be most confident in the lack of a statistical aberration for the results that contribu


implemented, their progress in improving birth and infant outcomes, and their sustainability—and the opportunities to build on successes. The integration of a risk assessment and referral tool into systems and practices in different health care and


icians and Gynecologists.
Alhusen, J. L., E. Ray, P. Sharps, and L. Bullock, “Intimate Partner Violence During Pregnancy: Maternal and Neonatal Outcomes,” Journal of Women’s Health, Vol. 24, No. 1, 2015, pp. 100–106.
Allegheny County Health Departmen


pe 1 Diabetes Mellitus: A Systematic Review and Metaanalysis,” Journal of Clinical Endocrinology and Metabolism, Vol. 94, No. 11, 2009, pp. 4284–4291.
Bayrampour, H., and M. Heaman, “Advanced Maternal Age and the Risk of Cesarean Birth: A Systematic


rics and Gynecology, Vol. 216, No. 2, 2017, pp. 185.e1–185.e10.
Brooten, D., J. M. Youngblut, L. Brown, S. A. Finkler, D. F. Neff, and E. Madigan, “A Randomized Trial of Nurse Specialist Home Care for Women with High-Risk Pregnancies: Outcomes and Co


s and Newborns Initiative: General Information,” last updated February 11, 2019. As of June 12, 2019:
https://innovation.cms.gov/initiatives/strong-start/
159
Chamberlain, C., A. O’Mara-Eves, J. Porter, T. Coleman, S. M. Perlen, J. Thomas, and J. E.


isk of Adverse Perinatal Outcomes: A Meta-Analysis,” JAMA, Vol. 295, No. 15, 2006, pp. 1809– 1823.
Connor, K. A., D. Cheng, D. Strobino, and C. S. Minkovitz, “Preconception Health Promotion Among Maryland Women,” Maternal and Child Health Journal, Vo


iedmann, I., E. M. Dahdouh, P. Kugler, G. Mimran, and J. Balayla, “Maternal and Obstetrical Predictors of Sudden Infant Death Syndrome (SIDS),” Journal of Maternal-Fetal and Neonatal Medicine, Vol. 30, No. 19, 2017, pp. 2315–2323.
Frisch, R., and F.


4–26.
Grigoriadis, S., E. H. VonderPorten, L. Mamisashvili, G. Tomlinson, C. L. Dennis, G. Koren, ... J. Martinovic, “The Impact of Maternal Depression During Pregnancy on Perinatal Outcomes: A Systematic Review and Meta-Analysis,” Journal of Clinica


 of Stillbirth: A Systematic Review,” Canadian Medical Association Journal, Vol. 178, No. 2, 2008, pp. 165–172.
Huang, Q. T., Q. Huang, M. Zhong, S. S. Wei, W. Luo, F. Li, and Y. H. Yu, “Chronic Hepatitis C Virus Infection Is Associated with Increase


. Stewart-Brown, and L. Polnay, “Parenting Interventions and the Prevention of Unintentional Injuries in Childhood: Systematic Review and Meta-Analysis,” Child: Care, Health and Development, Vol. 34, No. 5, 2008, pp. 682– 695.
Kochanek, K. D., S. L.


dical Informatics, Vol. 5, No. 1, 2017, p. e3.
166
Lifecourse Initiative for Healthy Families, homepage, n.d. As of June 12, 2019: https://lihf.wisc.edu/
Logan, C., E. Holcombe, J. Manlove, and S. Ryan, “The Consequences of Unintended Childbearing: A


merican Journal of Preventive Medicine, Vol. 50, No. 5, 2016, pp. S13–S19.
May, Linnea Warren, Serafina Lanna, Jordan R. Fischbach, Michelle Bongard, Shelly Culbertson, Rebecca Kiernan, Ricardo Williams, Elizabeth DeWolf, Victoria Lawson, and Julia B


, 2012, pp. 219–228.
Ohio Collaborative to Prevent Infant Mortality, Ohio Infant Mortality Reduction Plan 2015– 2020, 2015. As of June 10, 2019: https://odh.ohio.gov/wps/portal/gov/odh/know-our-programs/infant-and-fetal- mortality/reports/mortality-p


hman, “No Health Without Mental Health,” The Lancet, Vol. 370, No. 9590, 2007, pp. 859–877.
Quigley, M. A., G. Henderson, M. Y. Anthony, and W. McGuire, “Formula Milk Versus Donor Breast Milk for Feeding Preterm or Low Birth Weight Infants,” Cochrane


ng in the Extended Postpartum Period,” International Journal of Gynecology and Obstetrics, Vol. 124, No. 3, 2014, pp. 193–197.
Srinivasan, S., L. Schlar, S. E. Rosener, D. J. Frayne, S. G. Hartman, M. A. Horst, J. L. Brubach, and S. Ratcliffe, “Deliv


urnal of Medicine, Vol. 360, No. 2, 2009, pp. 111–120.
Toll, B. A., S. S. O’Malley, N. A. Katulak, R. Wu, J. A. Dubin, A. Latimer, B. Meandzija, T. P. George, P. Jatlow, J. L. Cooney, and P. Salovey, “Comparing Gain-and Loss-Framed Messages for Smoki


whamglobal.org/whamblog/316-advisory-charts-steps-to-make-pa-a-leader-in- maternal,-child-health
Witt, W. P., K. Litzelman, E. R. Cheng, F. Wakeel, and E. S. Barker, “Measuring Stress Before and During Pregnancy: A Review of Population-Based Studies


COVID-19 and the Experiences of Populations at Greater Risk
Description and Top-Line Summary Data—Wave 2, Fall 2020
KATHERINE GRACE CARMAN, ANITA CHANDRA, DELIA BUGLIARI, CHRISTOPHER NELSON, CAROLYN MILLER
 Sponsored by the Robert Wood Johnson Founda


bsequent top-line reports will update these findings.
Researchers from RAND and RWJF jointly conducted the research reported here; the report is intended for individuals and organizations interested in learning more about public attitudes about a Cul


harder for them to get the care they need than it is for those who live in urban areas, or is there not much of a difference? ............................................................................................27
Q21. [w2_pocimpact1] People o


...30 Views, Mindset, and Expectations: Understanding What Drives Health................................................31
Q29. [w1_drivers1-w1_driver4] Here is a list of some things that may affect people’s health and well-being. Please rate each on


context of COVID-19, RAND and RWJF have partnered again to build from the NSHA to implement a longitudinal survey to understand how these health views and values have been affected by the experience of the pandemic, with particular focus on populati


me under $125,000. The ALP sample limited White individuals to those with household incomes under $125,000 but imposed no income limits on Black, Hispanic, or Asian respondents. We were most interested in understanding the experiences of historically


color and low- to middle-income households. This report describes the second of four surveys conducted as part of the COVID-19 and the Experiences of Populations at Greater Risk Survey, a survey that RWJF and RAND Corporation researchers developed a


; Carman et al., 2019) and the report describing the first wave of this survey series (Carman et al., 2020).
Survey Design Process
Our survey was designed with the goals described above in mind. Where possible, survey questions were drawn from existi


Lab
Lydia Saad, “In U.S., More Fear COVID-19 Illness Than Financial Harm,” Gallup, April 16, 2020. As of October 9, 2020: https://news.gallup.com/poll/308504/fear-covid-illness- financial-harm.aspx
NORC at the University of Chicago, “American Health


g America Study
University of Southern California, “Understanding America Study: Understanding Coronavirus in America,” webpage, undated. As of October 9, 2020: https://covid19pulse.usc.edu
    A large section of our survey was focused on measuring r


 2, we broadened the question asking about protesting for whatever issues respondents believe in.
We added two questions. First, in wave 1, more than 70 percent of respondents reported that the pandemic could be an opportunity for positive change (w1


rstanding the experiences of the historically underserved individuals in U.S. society. The income limits allow us to focus on individuals in households that were likely eligible to receive the federal COVID-19 stimulus checks. In both panels, we over


e randomly selected to participate, whereas all panelists of
7
other races or ethnicities were invited to participate.3 This resulted in a sample of 1,331 panel members being invited to participate in the first wave survey. Of those, 1,098 (82.5 perc


rman et al., 2016; Carman et al., 2019). In our previous work, we took several steps to assess the appropriateness of combining these data. First, we took care to ensure that the implementation of the survey in the two panels was the same. Both panel


the characteristics of the U.S. population as estimated in the CPS. The survey sample distributions represent characteristics of the ALP and the KnowledgePanel at the time of sampling.6 Some households had changes in income or other characteristics


the two panels when controlling for demographic differences, we might not be able to discern from the social and demographic profiles of the sample every underlying difference regarding attitudes and perspectives. However, this combining of panels h


urvey data, who might apply other weighting approaches in their analyses. Weighted demographic characteristics are shown in Table 1.4.
We report the percentage of respondents who chose not to answer each question and label this as “missing.” None of


st Some of none of the time time
All of the time
Missing
            Wave 1
Wave 2
SOURCE: Santa Monica Well-Being Survey.
25.9 54.1 25.0 53.4
4.3 0.1 4.3 0.2
       D. Have a lot of energy Sample
None or almost Some of none of the time time
All of t


d not have work due to coronavirus (COVID-19) pandemic related reduction in business
(including furlough)
I am/was laid off due to coronavirus (COVID-19) pandemic
My employment closed temporarily due to the coronavirus (COVID-19) pandemic
My employme


 the coronavirus (COVID-19) has affected you.
Q11. [w2_coviddx] Has a doctor or other health care provider ever told you or a loved one that you or they have coronavirus (COVID-19)?
    Yes No
Missing
    29.0 70.7 0.3 X X X
    Select only one answe


4 0.4
     w2_exp6. Disruption in childcare or schooling for children in your household
Sample Yes No
Wave 2 13.2 85.9 0.9
             w2_exp8. Difficulty caring for a chronic condition (yours or others)
Missing
Missing
    Sample Yes
Wave 2 10.8 SO


g
      Wave 1 Wave 2
2.1 13.4 2.0 13.9
79.8 0.8 81.0 0.6
      24
 C. Food stamps (SNAP) or Special Supplemental Nutrition Program for Women, Infants, and Children (WIC)
      Yes, my household was already receiving assistance from this program befo


ed
1.5 1.6
No, my household has not received nor applied for assistance from this program since coronavirus (COVID- 19) started
Missing
      Wave 1 Wave 2
94.4 0.9 94.7 0.7
       H. Programs related to caring for my children (including school-based


       27
Q22. [w2_pocimpact2] People of color (e.g., African Americans, Latinos) are facing more of the financial impact of coronavirus (COVID-19) than whites.
        Sample
Somewhat Strongly agree agree
Neither agree Somewhat Strongly nor disagre


 people in need
      Sample Most of the time
Some of the time
Rarely Never
Missing
      Wave 1 37.6
Wave 2 36.2 SOURCE: Developed by RWJF and RAND.
47.7 48.8
10.7 3.8 0.2 10.5 3.9 0.6
      29
Views, Mindset, and Expectations: Views of Government R


they were born (their genetics/DNA)
No effect
Sample 1 2 3 4 5 Missing
               Very strong effect
       Wave 2: Not asked X
SOURCE: Developed by RWJF and RAND.
X X X X X
       31
Views, Mindset, and Expectations: Views on Future U.S. Approac


eloped by RWJF and RAND.
8.2 12.6 10.8 13.8
13.9 18.5 46.3 0.4 15.5 19.8 39.6 0.5
       33
Q35. [w2_protest] I think it is worth protesting for things I believe in, even if it may pose a health risk to myself or my family.
Select only one answer.
No


ity
Wave 2 16.1
Medium priority
25.2
Medium priority
50.5
Low priority
Missing
     4.2 1.5
           Low priority
Missing
     31.9 1.4
     35
 H. Children ages 0–18 with serious illness who are at high risk of dying from the coronavirus
     Samp


arried
22.5 21.6
            Respondents by Number of Household Members
          9 or Sample 1 2 3 4 5 6 7 8 more
          Wave 1 29.1 33.0 15.2 11.5 6.1 Wave 2 29.9 33.9 14.7 11.1 5.7
Respondents by Level of Family Income, in Dollars
2.9 2.8
1.1 0


ixes to the Report on the Economic Well-Being of U.S. Households in 2019—May 2020,” webpage, last revised May 21, 2020. As of October 9, 2020: https://www.federalreserve.gov/publications/2020-supplemental-appendixes-report- economic-well-being-us-hou


20. As of October 9, 2020: http://www.igmchicago.org/surveys/policy-for-the-covid-19-crisis/
University of Southern California, “Understanding America Study: Understanding Coronavirus in America,” webpage, undated. As of October 9, 2020:
https://covi


  AUSTRALIA
MEAGAN CAHILL, RYAN ANDREW BROWN, GARRETT BAKER, DIONNE BARNES-PROBY, HANNAH SANDRINI
Australia’s Third Action Plan of the National Plan to Reduce Violence Against Women and Their Children, Priority Area 2
Aboriginal and Torres Strait Isl


hes Used in the Prior Two Weeks. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
3.6. Therapeutic Approaches Used in the Prior Two Weeks by Program Type . . . . . . . . . . . . 63
3.7. CommonChallengesEncountered.................


uation
PM&C provided funding to 14 different service providers across Australia that work specifi- cally on family issues in Aboriginal communities. These sites are mostly located in remote and sparsely populated areas, creating challenges for servic


valuation approaches such as quasi-experimental designs.
The second major challenge to our work was the time taken to receive approval from the Australian Institute of Aboriginal and Torres Strait Islander Studies (AIATSIS), which provides review of


tions focus on ways to improve the implemen- tation process and on strategies for building evaluations into implementation of similar programs in the future.
Evaluation of Third Action Plan
Data Collection
Inside Policy, the organisation providing te


rams that were able to engage informal and formal institutions in communi- ties, from law enforcement to local clubs, created extended systems of support for
program clients that had positive impacts on program goals.
3. Clinicians noted that storyte


e things smoother by clearly identifying the roles and responsibilities for the regional and national offices, and for the sites. Throughout implementation, the national office staff may have ben- efited from more engagement with the regional office


ers called “Getting to Outcomes” (Chinman, Imm and Wandersman, 2004). The recommendations drawn from our evaluation of the 3AP effort start at the very beginning of the process—when an agency decides to put funding towards a specific social need, in


pted therapeutic frameworks. We need instruments that capture such subtleties.
Lessons Learned for Program Success
1. Understand that the timeline for change is long and requires sustained invest- ment. Sites emphasised that family violence is an iss


e of locally developed, culturally grounded measurement approaches. These include custom scales developed by and validated in the com- munity context, as well as methods to quantify narrative or descriptive information.
9. Focus on evaluating impleme


NIAA National Indigenous Australians Agency
NPYWC Ngaanyatjarra Pitjantjatjara Yankunytjatjara Women’s Council
NT Northern Territory
PM&C Department of the Prime Minister and Cabinet
PUTP Picking Up the Pieces
RCT randomised controlled trial
RSES Ros


the National Plan framework for more than a decade (COAG, 2012). The Second Action Plan, implemented from 2013 to 2016 and called “Moving Ahead”, focused on using evidence gathered during the First Action Plan to identify and address gaps in the appr


policy was officially abolished in 1972, but one of the last schools that housed stolen chil- dren did not close until the early 1980s. An official public document acknowledging the Stolen Generations was released in 1997, with an official apology c


r, 2020a). Although the Intervention officially ended in 2012, many of its policies remain in effect today in some form under the current Building Stronger Futures policies (Aus- tralians Together, 2020a).
Impacts on Child Well-Being
For at least a c


cted to a disproportionately high rate of family violence, especially when compared with the non-Indigenous population in Aus- tralia (Braybrook, 2015). According to the AIHW (2018), just over one out of every five Indigenous Australians indicated th


ity as opposed to more individu- alised approaches, and Walker and Shepherd (2008) present two model interventions— addressing child literacy and child health and family well-being—that demonstrate the importance of cultural considerations in these e


an Aboriginal Child Care Agency (VACCA)
Men’s Outreach Service (MOS) Rekindling the Spirit (RTS) Yoowinna Wurnalung
Birrang Maranguka
Kornar Winmil Yunti Aboriginal Corporation (KWY)
Mookai Rosie Wurli-Wurlinjang
Aboriginal Family Legal Services Sout


es
 SOURCE: Esri, HERE, Garmin, USGS, Intermap, INCREMENT P, NRCan, Esri Japan, METI, Esri China (Hong Kong), Esri Korea, Esri (Thailand), NGCC, 2018; overlay by RAND.
Table 1.1
Third Action Plan Intensive Family-Focused Case Management Program Chara


nal and Torres Strait Islander people who are traumatised by family violence or sexual assault.
Participants will be women who have experienced family violence, specifically those with complex needs and a child at risk.
Aboriginal women who are victi


Inside policy
Roles:
• Conduct 3 Design Shops
• Assist sites with implementation
RAND
Role: Conduct process and outcomes evaluation
     Intensive family— focused case management
programs
 1. Birrang/Maranguka 2. KWY
3. Mookai Rosie
4. Wurli-Wurlinj


ties for the period and make funding decisions. Thus, pro- grams funded as part of 3AP had at maximum three years for implementation, but in real- ity implementation had happened for just over two years.
Thus, we expected that 3AP programs were likel


 Strait Islander peoples, all project-related data collection efforts were paused until AIATSIS approval was received. Several rounds of submissions, revisions and resubmissions took place over a one-year period.
AIATSIS is part of the PM&C funding p


e design for 3AP programs, including details such as defining the client population, outlining the intended services provided, the process by which such services would be provided, and expected outcomes. This information was used to develop logic mod


 3AP-funded programs. We used the ideas generated during these sessions to further refine the sites’ logic models and to frame analysis related to possible outcome evaluations in a culturally sensitive manner.
Project Background 11
12
Australia’s Thi


d versions from Inside Policy for analysis (see Appendix A). In preparation for the site visits and phone calls with sites starting in October 2017, we analysed the logic models for specific sites and refined the stated activities, outputs and outcom


 interview protocols from these early interviews).
In October 2017, team members visited six sites over a two-week period:
• KWY
• Mookai
• NAAFLS • NPYWC • Yoowinna • VACCA
We selected these sites based on availability of program staff, travel logis


erimental design – program enrolment
– data collection.
At the time that the evaluability assessments were conducted, all sites were early in their implementation periods. The assessment helped us to determine which sites were advanced enough in orga


ll available data col- lected throughout the evaluation period, including baseline information gathered through early 2018 and from interviews with all sites in 2019–2020. The summaries put this infor- mation into the context of the logic model, whic


sational capacity to provide services to families in need.
Inputs
PUTP included four service components: case management, individual trauma-informed clinical interventions, family functioning therapy, and referrals for additional services (e.g., subs


prevent them from being exposed to the perpetrator or his family.
Observed Outcomes
In 2019, Birrang staff reported that the services they were able to give clients—and mothers in particular—as part of 3AP funding led to real change for clients and f


communities of Adelaide and the Riverland and as a result reduce the number of children entering the out of home care and juvenile justice systems by breaking the cycles of family violence”. Short-term goals for the program included engaging clients


CEO as one of their primary implementation challenges. KWY staff did not feel that keeping client data in Excel was efficient for the staff. KWY aimed to col- lect extensive data on each of their clients through dynamic risk assessments and behav- i


been using for some time now and are experienced with. One individual oversees all data in the system. In 2018, Mookai reported interest in tracking how their cli- ents interact with Australia’s legal, justice and education systems. They discussed w


d it employed local Aboriginal staff. The organisation had a community liaison office to ensure client feedback was incorporated into service design and keep its finger on the pulse of changes in the community. Wurli also employed two community engag


to their families, and to the land itself. VACCA staff reported their approach has allowed some families to connect to their extended families—their clans or “mob”. This also allows family members to connect or reconnect to country, including visiti


 historical and structural context in mind.
VACCA has made several changes and developments in its therapeutic process over the years of 3AP funding. For example, they have started to build an internal culture of evaluation, assisted in part by evalu


complemented by on-country activities, such as cultural camps, with plans to conduct a check-in with the client at 12 months.
Initially, MOS planned a five-person team (with four full-time employees) for Change ’Em Ways comprised of one senior clinic


her assistance. For Burubi, RTS was funded to run an early intervention program providing a wide range of supports for the perpetrator; the program was designed to be complementary to RTS’s existing behaviour change program.
The six-week program cons


am has been refunded for the next three-year funding cycle, and the team hoped to keep demonstrating the successes of their program to sustain funding into the future. RTS reinforced the necessity of consistency in funding to measure outcomes and mai


vism, increased employment and trained workers, and increased feelings of safety among women and children. At the community level, the organisation aims to inspire greater awareness of family violence and its impacts, establish “community champions”


hey sought to provide support that was broader than merely addressing clients’ legal needs.
The services that AFLSSQ had previously worked with for counselling services were unable to provide counselling for its 3AP program. It was difficult to hire


we’ve assisted very well is a lady who went from being very intimidated in relation to dealing with her children to being more confident, a turnaround in personality, it took 12 months. And I was shocked, but the information she received that enable


people who were recently or previously victimised by family violence and/or sexual assault. As of December 2017, the program was monitoring and refining specific procedures for recruiting and referring clients, as well as the caseload for the two soc


ding higher-quality and more comprehensive care. The staff used a well-being scale from 1 to 10 to rate how clients were doing in different areas, although it is unknown whether and how this measure may be repeated and tracked over time across sessio


ice clients, they wanted to cap the number of clients that the social workers can have and monitor their caseloads to make sure they did not feel overwhelmed or experience burnout.
Individuals were generally referred to the social work services after


e new programming.
Marninwarntikurra Women’s Resource Centre
MWRC works with children and families affected by domestic and family violence and provides legal services to these groups through its Family Violence Prevention Legal Unit (FVPLU). MWRC ex


s was approximately 650 km from the main office in Darwin; the counselling team, always travelling in pairs for safety, would go out on a Monday and return at the end of the week “totally exhausted from the long travel”. Also, because the communities


ve experience working in such communities.
Site Profiles 31
32 Australia’s Third Action Plan of the National Plan to Reduce Violence Against Women and Their Children
Focus on developing the workforce in this area would be beneficial to programs acros


mat by every staff member. While the data collection provided excellent narrative histories of clients’ progress, it would not be conducive to more formalised evaluation.
Because of the mode of data collection and because the 3AP program is closely i


program staff were working to manualise the program they had developed, establish training for their legal staff on family violence, and embed the social service part of the organisation into the existing legal service activities.
Site Profiles 33
CH


ultiple interviews with different program staff that were conducted individually. We uploaded these 14 transcripts (one for each of the fourteen 3AP sites) to the cloud-based collaborative qualitative analysis software platform Dedoose (SocioCul- tur


g child codes and writing up content were often responsible for differ- ent parent and grandparent codes than they were assigned during Coding Step 2. This allowed team coders to check through each transcript as they created child codes and read thro


y
– future plans for sustainability
– advice for PM&C on sustainability.
Interactions with Department of the Prime Minister and Cabinet, Inside Policy or Other Third Action Plan programs
Type and Level of Interaction
Interviewees from all 14 3AP site


heir work—typically involving program funding or cost. Also, interviewees from four sites discussed some type of miscommunication or disagreement with PM&C that had to be settled. For example,
My view is that Canberra were number crunching and saw fr


a good bridge between us and PM&C on a fed- eral level. I was quite impressed.
Another site’s interviewee echoed those thoughts regarding Inside Policy’s helpfulness:
They were good—really good. Especially initially when we were putting framework pap


dy”. An interviewee from this same site also mentioned that while PM&C didn’t support their site in their challenges, they conceded that it “might have been our fault because we didn’t contact them much about it until we did the report . . . from the


as not enough opportunity to have effective sharing of models and best practices among sites:
What I really think is that out of that meeting, the initial get together with all the partners who were funded by 3AP, I think that was a wasted opportunit


er, they receive comprehensive support to address all needs as they arise.
The holistic approach to care was also described as evolving over time and starting with meeting families where they are. One site’s interviewee discussed how they sought to m


ut in one of the cases the client had no other options for the care of an older child, so we informed [the client] that we had the facilities and vacancy at our women centre to do this.
To address these gaps as they were identified, programs brought


 support and doubled the number of group sessions being offered:
The reason that we did that was because we found in the 4–5 weeks [that clients were with the program] there were a couple of issues. Firstly, it was too short of time to be able to del


er that date. If it was a 3-year contract at that time, [the social workers] would have stayed.
Barriers to service delivery. One of the biggest challenges to program implementa- tion was the numerous barriers to delivering services to the target pop


s, two days a week, and after that [we] follow up with 10–11 months of coun- selling. We only have a couple of guys that have maintained that all the way through, they have been consistent with their change, but the men coming through the justice sys


of [as a sign of our success] is that we have a waiting list, and communities and agencies are wanting to send Aboriginal men to us, and some of these men have gone through other men’s behaviour change programs and have still main- tained being viol


les, including that “they [men] might drink and all that but they don’t drink as much, or their kids have said that. When they come back from school, Daddy isn’t lying on the lounge and growling at me, but that they have come home, and Dad is sitting


tes recognised the need to develop programming for chil- dren. For example,
We have also just finished piloting a children’s program because we really want to develop a trauma informed program for the women and children. So, we have just finished run


rviewees reported that their sites used some type of data system, typi- cally for case notes. The systems used by sites included CLASS (two sites), NFILES, Penel- ope, and IRIS (each used by one site), while one site did not specify the type of data


staffing, outreach meth- ods, cultural reference groups of an advisory nature, participatory activities, and education tools as means of community involvement.
Process Evaluation 49
50 Australia’s Third Action Plan of the National Plan to Reduce Viol


ams, this conversation recognised the role of men in perpetuating violent behaviour, encouraging accountability, sharing stories and recognising change. One program interviewee mentioned that the community responded by asking questions, seeking clari


t the organisational focus from the individual to the family. 3AP funding also provided for the upskilling of workers at one site.
One site’s interviewee noted that 3AP funding provided “enthusiasm and drive” to consider different initiatives around


victims of violence felt more empowered and were able to stand up for their own rights and life goals as a result of participating in the 3AP program:
An example of this is a woman that has been a client of the service for about 20 years. . . . The a


the cycle of violence because they [clients] have a connection to something. All of us human beings need to belong to something, we need to believe in something, it’s part of our human nature, and when they have a connection to their country, to thei


f the National Plan to Reduce Violence Against Women and Their Children
Also, interviewees from two sites mentioned that they felt they just could not provide enough services; one of these interviewees indicated they thought couples counselling was i


. Interviewees from three sites indicated that an increased understanding of family violence in the community encouraged responsibility among community members for creating change; the program became sustained by the community. Meanwhile, inter- view


bed by one site as being “generational”, requiring significant time. One site’s interviewee mentioned that the size and number of communities that required assis- tance was a significant barrier for their program. They noted that for “the community a


 with police, which is why they [the police] are coming to us to set up the vulnerable people’s unit. . . . Police are first responders to domestic violence so if they can refer to us, we know it’s a win.
An interviewee from one site mentioned that a


ncy plan involves employing and training local residents to the extent possible and encouraging empowerment and education among community residents. Eventually the community will no longer need the program if they have sustained it themselves. Inter-


Evaluation 57
58 Australia’s Third Action Plan of the National Plan to Reduce Violence Against Women and Their Children
Interviewees from four sites called for greater cultural understanding from PM&C in the future. This included more flexibility and


Surveys
Methods
Early in the implementation of 3AP programs, we collected survey data from program staff and program managers regarding the progress of implementation and program details. The idea for these surveys came from one of the 3AP providers


beginning each survey, participants were provided information about the confidentiality and voluntary nature of the survey as well as contact details to direct questions about their rights as participants. The respon-
Figure 3.2
Monthly Manager Surve


uld also be related to your organisation or management.
5. Please describe one lesson learnt over the past fortnight.
The lesson learnt can be small, such as a minor adjustment in how you work with clients, or it could be more significant, such as le


received 124 surveys from 50 client-facing staff. Table 3.3 provides the response rates for staff and managers for each wave of the survey; managers took the survey monthly so response rates from only three surveys are reported for those respondents.


management surveys administered three times during that period. Staff respon- dents replied to between zero and six surveys. Among staff who completed at least one survey, the average was three surveys completed per staff member. All management respo


therapy
CBT
Strength-based approaches Active or deep listening Client-led or centred approaches Risk, needs assessments
Focus on Aboriginal traditions and cultures Motivational interviews
Trauma-focused approaches Communication
Intensive Family- Focu


bstance abuse treatment or mental healthcare not offered at their organisation. Respondents also reported discord between staff within agencies, sometimes stemming from general dislike of other staff, but sometimes reporting more serious concerns. In


ts in sessions, receiving training on specific techniques, or even realising their need for specific types of training. This also included more informal lessons, such as learning to “trust but verify” client reports of their situation, and personal s


nd broader social networks within communities. Such an approach yielded positive results in terms of program interest, engagement, retention and reductions in family violence.
2. Programs that were able to engage informal and formal institutions in c


re hands-off and waited for sites or the national PM&C to indicate a need for assistance.
3. At the outset, the national office could have made things smoother by clearly iden- tifying the roles and responsibilities for the regional and national offi


out the intervention period. In this chapter, we provide insights that we were able to extract from each data set and suggest a way forward for future out- come evaluations.
Client-Level Quantitative Data
One of the 3AP intervention sites, RTS, colle


cale (URICA; Habits Lab at UBMC, n.d.)4 and the RTS-Q. The research team calculated the overall scale and subscale means and standard deviations (SD; see Table 4.1).
2. Differences in Men Who Completed the Intervention
We reasoned that men who comple


100 16 100 16 81 14 71 16 75 16 75
16 6 16 88 16 94 16 6 12 67 17 76 17 100 17 88
Outcome Evaluation 71
      When you are violent or aggressive, do you see that as a problem?
When you are violent or aggressive, do you feel powerful?
When you are vio


ales.5
Client Case Vignettes
Two additional sites provided de-identified client case vignettes with narrative data on cli- ents and their families, including basic demographic data; which types of therapies were provided; and challenges, successes an


explained how VACCA is developing more knowledge and confidence with respect to the uniqueness of its own approach, and that it is currently engaging in a process to develop a customised scale to measure progress along the healing pathway among Abor


llect the array of data that was initially planned and thus unable to conduct rigorous analyses of potential program outcomes, this report has significant limitations. We are not able to draw any conclusions as to the actual effective- ness of any of


are such activities as assessing community needs for programming; developing a logic model (including the goals, activities and outcomes of the program); determining community “fit” so that any new program complements without duplicat- ing existing


e the program development process as part of the implementation evaluation, gather rel- evant information about each site that can be incorporated into the evaluation plan, and work with the technical assistance provider to help sites develop systems


e that all hands (including technical assistance) are on deck to assist programs consistently. Large implementation programs like 3AP necessarily involve many different agencies and entities, some of which could have been formally brought into the pr


ally thought their program would be implemented in one way before attending the Design Shops and then completely redesigning their approach. Our observation, then, is that this process was not sufficient to prepare sites for implementation.
Continuin


sed programming but on their capacity and willingness to work through a robust needs assessment and design and implementation process. While more resource-intense at the front end, this approach may result in increased program longevity and success.


me degree, better coordi- nation with regional PM&C offices (and between regional and national staff) could help to address this. However, local variability in drivers of family violence, cultural dynamics and other ecological challenges (including l


sions (and preparation for data collection) should have started much earlier in the process. Further exacerbating this challenge was the time taken to secure ethics approval and the subsequent pause in interacting with 3AP sites, as discussed earlier


upport for an inclusive, coordinated effort would ben- efit more community service providers.
Any tools implemented should also accommodate the various Aboriginal- centred approaches to treatment that may be used, such as efforts to create connec- ti


hildren is commendable for its bold vision over a long time frame, implementation of the plan is broken into three-year periods, and fund- ing for the 3AP sites was initially provided for only two years. Several 3AP sites did receive follow-on fundin


 for program clients, increased the pro- grams’ ability to refer clients to other needed assistance, and made positive progress towards their goals.5
11. Storytelling and narrative therapy are critical for success.
Storytelling and narrative therapy


d, and no sites indicated it was needed, we believe identify- ing local cultural advisory structures in each of the sites would have improved our findings. This effort requires planning time and resources to establish but would increase the likelihoo


uantita- tive tools (Muir and Dean, 2017; Palinkas, 2014).
15. Emphasise the use of locally developed, culturally grounded measurement approaches.
The need for culturally appropriate measurement tools was discussed above under Recommendation 7. That


ainst Women and Their Children
the program design and implementation phases (i.e., building it into the structure of the program) would support long-term sustainability by allowing sites to better plan for the changing needs of the communities they s


s client goes back to partner
   Community: • Decreased
FV incidents and crime in community
• Increased connectedness
• Economic inclusion of Aboriginal families
Organisational: • Increase
in quality partnerships and networks
• Become a lead provider


llors
Staffing:
   Kornar Winmil Yunti Aboriginal Corporation Logic Model
Figure A.2
NOTE: This logic model was modified by RAND (as a part of the outcome evaluation) to include additional information gleaned from qualitative interviews with program


lop strategies to be implemented in
• One-to-one counseling—men, women,
• Lateral violence—together and all
– staff training CNS and city based
• For community to use/promote
• Workshops—staff training and
• Health assessment and service
• Family the


ation
• Children and family gain awareness of DV and coping strategies and
• Minimisation of involvement by TFs services trauma trauma skills
   trauma transfer assessment process
• Community staff
• Increase inter-agency
• Improve stakeholder
• Comm


s and communities are free from family violence
Blue = TF Red = 10YP Orange = OL Green = FVTI Black = LTU
 Outcomes (6–12 months)
Recreational well-being, social and cultural connectedness
 Increased ability to sustain positive relationships
 Stronge


ily and community relationships, membership of CRG is reconstituted greater connection to extended culture, and country, and are more • Clients (partner and children) have engaged in societal activities. • The Terms of Reference and based upon lesson


with partner and children.
Participation:
Cultural Mapping (stage 2), for
has been formed, and members
• Cultural Reference Group (CRG)
• Clients have undergone CEW
participation in the workshop
understand role (ToR).
(stage 3).
Culture:
engaged, wh


 0–2 years) Changes in knowledge, awareness, convenience, motivation, etc.
   OUTPUTS Units of service delivered to stakeholders/to meet project objectives
abusive behaviour on self,
• Raise awareness of positive
• Raise awareness of the impacts of v


retation services
• Advisory group/board
• Community based
• Mediator
champions
country
Staffing:
• Data input systems and IT
• Policies and procedures
(e.g., for recruitment)
• Operational support
• Accreditation
Processes:
• Winnebago for travel
•


oject objectives
   ACTIVITIES Events or processes to meet needs/ objectives
• Client support officer accompanies client to
• Counsellors do narrative
• Advertise to community about services on offer
• Engage with existing
network services
• Refer to


rs
procedures
Staffing:
worker
• Manager
Processes:
• Finance
• New marketing materials
• New office space
Resources:
   Figure A.10
Far West Logic Model
NOTE: This logic model was modified by RAND (as a part of the outcome evaluation) to include add


ng mediation
• Training/upskilling
including board
healing space and
• Redesigning of
to discuss needs
workshops
stakeholder
resources
service
– Staff
   INPUTS
Staff, money,
previous research
• Internal knowledge/
research including
cultural (such a


ional:
stakeholders.
and policies
   INPUTS
  • Professional development
member (advising)
• Capable staff
• Elders and TO and
• Interpreters
Staffing:
• Promotion
legal and social services to
(e.g., for recruitment)
• Create barriers between
protect


fident life home care be drastically
• Ongoing commitment from and parents to be involved
• Changes in attitude toward
• Improvement in well-being functioning, well-being and
• Exit plan—as long-term
• Want clients to be happy with pride and free fro


commitment
– 1x part-time support
– Training for all staff
social workers (SW)
• History of region:
(cultural/security/
• Understanding the
different language
• Outreach induction
– 2x social workers
in Geraldton and
and all staff
• AFLS induction
fr


ence that this includes in the community you serve?
 101
102 Australia’s Third Action Plan of the National Plan to Reduce Violence Against Women and Their Children
b. [low priority] How did the family violence issue originate? [less important—skip if


 specific data they will track, etc.]
1. [high priority] What types of data will you be keeping from the clients
who are in your program?
a. [high priority] Will you keep case notes for each client? What is
included in those case notes? v. Are there


er context of your community and then we will speak more specifically about your program.
1. Tell me about your community. [We are looking for broad contextual information about the community; we will talk more specifically about family violence and


ird Action Plan effort
i. Specific activities undertaken as a result of the funding
ii. If/How it is changing from the original proposal to PM&C iii. Do you have sufficient resources to do the work?
f. [low priority—important to know so we can determ


ges to [organisation]’s program. This will help us to ensure we understand the context of the issues with violence your community faces, and the ethical considerations we should follow while we work together. Over time, we’ll be talking with people a


funding . . . other?]
g. [high priority] What are best long-term solutions to reducing family violence?
i. What are persistent barriers to solving the issue? [probe on systemic, dis- crimination, resources, etc.]
ii. What solutions have been tried al


 more than an hour.
I will be leading today’s discussion and [NAME] may ask additional questions for clarification. [NAME] will be taking notes.
Do you have any questions before we begin?
The first half of our interview will focus on the broader cont


e new program aligns with other organisational services
ii. Is there any program you’re currently running that is most similar to the new program?
1. Do you believe this or any of the other programs you offer through your organisation could be suitab


hat aspects of your interaction with regional PM&C staff did you feel worked well? What aspects of your interactions could have been improved? Were regional PM&C staff accessible? Responsive to requests? Hands-off or too involved? Open to discussing


lence? SOURCE: Data from 3AP site, analysed by RAND.
What were the goals of the 3AP program? How did this change over time?
• What populations did you plan to serve? How did this change over time?
• What was your staffing plan? How did this change ov


.au/getmedia/b0037b2d-a651-4abf-9f 7b-00a85e3de528/aihw-fdv3-FDSV-in-Australia -2019.pdf.aspx?inline=true
Australians Together, “The Intervention,” webpage, last updated January 14, 2020a. As of August 22, 2020:
http://www.australianstogether.org.au/


97. As February 19, 2020: https://humanrights.gov.au/sites/default/files/content/pdf/social_justice/bringing_them_home_report.pdf
———, Closing the Gap: Prime Minister’s Report 2017, Canberra, ACT: Department of the Prime Minister and Cabinet, 2017. A


ogy and Law, Vol. 19, No. 1, pp. 104–117, 2012.
Eckhardt, Christopher I., and Angela C. Utschig, “Assessing Readiness to Change Among Perpetrators of Intimate Partner Violence: Analysis of Two Self-Report Measures,” Journal of Family Violence, Vol. 2


01_2016/3-1-respectful-relationships-evaluation -summary-of-findings.pdf
Levesque, Deborah A., Richard J. Gelles, and Wayne F. Velicer, “Development and Validation of a Stages of Change Measure for Men in Batterer Treatment,” Journal of Cognitive The


 Adolescent Psychology, Vol. 43, No. 6, pp. 851–861, 2014.
Piper, Alana, and Ana Stevenson, “The Long History of Gender Violence in Australia, and Why It Matters Today,” The Conversation, July 14, 2019. As of September 14, 2020: https://theconversati


er
and Cabinet’s (PM&C’s) Third Action Plan (3AP) of the National Plan to Reduce Violence Against Women and Their Children, and specifically 3AP’s Priority Area 2: Aboriginal and Torres Strait Islander Women and Their Children. PM&C provided funding


  Research Report
ANDREW R. MORRAL, TERRY L. SCHELL, BRANDON CROSBY, ROSANNA SMART, ROSE KERBER, JUSTIN LEE
Preliminary Findings from the Aerial Investigation Research Pilot Program
In this report, we provide a preliminary description of the clearanc


reet jus- tice because they do not believe going to the police will produce results. In addition, low clearance rates could undermine community support and coopera- tion with police if citizens believe that the police have little interest in solving


e June, a second plane began to be used, and it was declared fully operational by PSS staff on July 6, 2020. The AIR pilot program col- lected an average of 5.7 hours of imagery per day
in May, 7.7 hours of imagery per day in early June, and 9.1 hour


t least one of the crimes associated with a case has been cleared by arrest or by exception, accord- ing to either the database used by the investigating detective or the BPD records management system. Case status updates in the record management sys


icts, and clearance rates vary across districts. Similarly, AIR cases consist of a higher percentage
of homicides than non-AIR cases, and homicides
are cleared at a higher rate than other MOU crimes. Therefore, we would expect cases with AIR evidence


when the suspect is arrested at the crime scene (Cook et al., 2019). These cases close before AIR evidence would ever need to be used. The primary benefit of the AIR pilot might be expected to be found among the more-difficult cases that are not reso


 Were Not Cleared on the Day of the Crime or the Next Day
100
75 Percentage closed/solved 50 25
0
AIR Non- Matched AIR non-AIR
Provisionally cleared
AIR
Non- Matched AIR non-AIR Provisionally solved
      NOTES: Sample sizes for AIR and non-AIR cases


o AIR cases also has limitations. Although the matched cases do, on average, occur at times
and places comparable to AIR cases, they might still differ from AIR cases in important and systematic ways that make them a misleading comparison group. For


vestigates the crime, they enter a more accurate crime description in Lotus Notes. Information in the Lotus Notes–based data system and RMS is later reconciled through a formal UCR review, although this review sometimes occurs months after the crime


hursday day:Friday day:Saturday day:Sunday month:5 month:6
month:7
month:8 month:9 month:10 time:12am_2am time:2am_4am time:4am_6am time:6am_8am time:8am_10am time:10am_12pm time:12pm_2pm time:2pm_4pm time:4pm_6pm time:6pm_8pm time:8pm_10pm time:10pm


e sample size of the matched non-AIR group. More-difficult cases = all MOU cases that did not close on the day of the crime or the next day.
Aerodrome Report data from Baltimore/Washington International Thurgood Marshall Airport describ- ing cloud co


ases (difference = 6 per- centage points, t = 1.27, p = 0.2). (Recall that this was also true for the analysis with the full sample of MOU cases.)
FIGURE A.1
Rates at Which Crimes Were Provisionally
Cleared and Solved, July 1 to October 31 Subset
100


bberies, and carjackings. The pilot began on May 1, 2020, and continued until October 31, 2020. In this preliminary report, we provide an initial description of the clearance rates achieved by the BPD during the six-month pilot test, which might prov


  CORPORATION
BENJAMIN M. MILLER, DAVID METZ, JON SCHMID, PAIGE M. RUDIN, MARJORY S. BLUMENTHAL
Measuring the
Value of Invention The Impact of Lemelson-MIT Prize Winners’ Inventions
                For more information on this publication, visit www.


cuses on such topics as infrastructure, science and technology, community design, community health promotion, migration and population dynamics, transportation, energy, and climate and the environment, as well as other policy concerns that are influe


tions by Year, 1990–2019 ...................................78 Figure 5.6. Bertozzi’s International Collaboration on Scientific Publications..............................79 Figure 5.7. Distribution of Bertozzi’s Patents by Technology Sector..........


cases, the inventions of the Lemelson-MIT Prize winners not only provided the scientific and technological foundation for new companies but were revolutionary ideas that
1 In addition, 22 more of these companies were involved in acquisitions for whic


Evidence suggests that a lack of exposure to role models and support networks is limiting diversity in the population of inventors. Programs and policies that support the development of both inventions and inventors remain important for ensuring eve


, and more-productive lives. They provide new ways to build, move, communicate, heal, learn, and play.1
The potentially life-changing benefits of invention make it important to ensure that policies, programs, and initiatives support the development o


assachusetts Institute of Technology (MIT) Prize. The Lemelson- MIT Prize was one of the most prominent efforts to recognize and reward invention.
This report is not an assessment of the effectiveness or impact of the Lemelson-MIT Prize or similar pr


pt of the award.
Chapters 3 through 5 of this report focus on the impacts of the inventions created by winners of the Lemelson-MIT Prize from three particular years: 1996 (Herbert Boyer and Stanley
3 For studies showing that ex post technology prizes


a better world. Jerome Lemelson was one of the most prolific independent inventors in the United States, amassing more than 600 patents.4 The Lemelson-MIT Prize was created to recognize highly accomplished and highly promising mid-career inventors wh


llecting, analyzing, and publishing statistical data related to the U.S. business economy” (U.S. Census Bureau, undated).
8
4
3
3
3
2
2
1
1
1
1
1
     4
Nearly half of the Lemelson-MIT Prize winners produced inventions in the field of research and de


rch and development in the physical, engineering, and life sciences (except nanotechnology and biotechnology); U.S. Office of Management and Budget, Economic Classification Policy Committee, 2017.
5
In addition to—or often, in the process of—providin


old patents than their peers in the top 1 percent of parent income (Bell et al., 2019). Furthermore, as of 2016, the “women inventor rate” had reached only 12 percent (increasing from less than 5 percent in 1976), and growth in that metric has been s


a key role in the promotion of their inventions, design, production, and distribution functions are typically addressed by new or existing organizations, which may or may not be led by said inventor.
Often, many organizations are involved in the deve


cture Fogarty’s device. Today, Edwards Lifesciences still
 manufactures Fogarty’s balloon catheter, which it sells to medical professionals worldwide.
 Ultimately, the balloon catheter is estimated to have saved the lives or limbs of as many as 15 mi


 objects on the road to improve artificial intelligence for Google’s Waymo autonomous driving systems (Healy and Flores, undated; Kidd, 2019).
1 The quotes and information in this paragraph, as well as the statistic on the number of patients affected


category, inventions must have provided a foundation for a public or private company to grow over time and remain an independent entity that exceeds the U.S. Small Business Administration’s threshold for a small business. These thresholds vary by in


   SOURCE: RAND analysis of data sourced from Yahoo! Finance, Dun & Bradstreet, and other financial news sources.
The most-frequent pathways of invention proliferation among the 26 Lemelson-MIT Prize winners include commercializing an invention as a


 a lot of pressure. It was well beyond the technology at this point, right? It’s branding, it was operations, it was [a] tough, tough business. (DeSimone, 2013)
A decision by Congress to not pass an anticipated tax credit that would have favored the


ed with existing businesses, and the majority remained successful smaller companies or were start-ups attempting to commercialize new ideas or technologies. Many companies did not succeed.
The combined technologies, resources, and capabilities of the


unt for each inventor, we searched the author field within the WOS database using the author’s last name, followed by their first initial. We then identified a publication that we knew to have been written by the target inventor to serve as a confirm


e issue of variability in a given article’s impact by also examining prize winners’ citations (i.e., the number of instances in which a scientific peer has cited a winner’s publication portfolio).
Findings
Table 2.1 summarizes the aggregate scientifi


  49
         48
         43
         41
         40
         39
         37
         35
         34
         34
         34
    International co-authorship can be used to measure the extent of international scientific collaboration; it also reflects


Third, patents can be filed strategically to prevent other firms from using an innovation (Blind, Cremers, and Mueller, 2009; Noel and Schankerman, 2013; Thumm, 2004), and even the choices of which citations to cite as prior art may be a strategic de


       Langer
Mead
Fogarty
Kurzweil
Kamen
Hood
Holonyak
Norris
Fergason
Swager
DeSimone
Mirkin
Bertozzi
Rogers
Quake
Belcher
Bhatia
Whitacre
Raskar
Zhang
vonAhn 12 128 49 Friesen 87 37 22
831 7,487 1,369 90 2,824 414 215 8,259 795 117 883 232 479 4,5


e winners. During the patent application process, patents are assigned International Patent Classification (IPC) codes by patent examiners to facilitate subsequent applications’ search for prior art. IPC codes classify a given invention in terms of i


nces in several
21 In addition, 22 more of these companies were involved in acquisitions for which our research team was not able to find publicly available financial details.
      3% 11%
  34%
Instruments
Chemistry
Electrical engineering Mechanical


 inventions in these contexts—although the inability to do so does not diminish the impacts of the inventions because the benefits may be diffuse and not solely
22 Financial data are not available for all companies, including many privately held comp


cientists, generated billions of dollars in revenues, and would have some of the largest public offerings in U.S. history at the time. In 2009, Genentech would become one of the first biotechnology companies to achieve a market valuation exceeding $1


Linn and Arber, 1968; Meselson and Yuan, 1968; Smith and Wilcox, 1970). Boyer and others envisioned that restriction enzymes would be useful for cutting, recombining, and characterizing DNA. In 1972, Robert Yoshimori, a graduate student in Boyer’s la


oyer presented on the properties of EcoRI, Cohen recognized that this restriction enzyme could be a potential solution to the challenges of inserting plasmid DNA into bacteria for cloning. One evening after the conference, Boyer and Cohen met at a lo


nvention
rDNA technology
Beneficiaries
rDNA technology gave rise to the field of genetic engineering and laid the foundation for the biotechnology industry, opening the door to gene therapy and the manufacturing of new drugs and existing synthetic bi


Cohen created a novel combination of antibiotic resistance genes. Conveying unease within the scientific community, a majority of conference attendees voted to send letters to the National Academy of Sciences and the National Institute of Medicine r


tologous, Homologous, and Heterologous Normal Skin Grafts in the Hamster Cheek Pouch” in the Proceedings of the Society for Experimental Biology and Medicine (Cohen, 1961). Since 1961, Boyer and Cohen have published an additional 247 WOS-indexed arti


cription of Eukaryotic DNA in Escherichia coli,” which were published by the Proceedings of the National Academy of Sciences of the United States of America in 1973 and 1974, have been cited 1,172 times and 364 times in WOS publications, respectively


al limitation of Kelly et al., 2018, is that text-based measures focus on scientific value rather than market value; however, the literature finds these tend to be related (Hall, Jaffe, and Trajtenberg, 2005; Nicolas, 2008; Odasso, Scellato, and Ughe


these patents around the time it financed the acquisition of Gen-Probe Incorporated by Hologic, Inc. in 2012.
36
Table 3.4. Notable Inventions with Forward Citations from Boyer and Cohen’s Patent Portfolio
    Citing Organization
Genentech
Goldman Sa


company and Swanson departing the firm. Swanson would later approach Boyer with a business proposal.
By 1978, Cetus Corporation had finally completed construction of genetic engineering research facilities to conduct their first rDNA experiments. Cet


lergic reactions in some humans. rDNA technology could replicate human insulin, which would pose lower risk of an adverse reaction and mitigate a global insulin shortage. The company’s goal would be to license the genetically engineered bacteria to a


 of directors.
 39
hormone and rapidly approve Genentech’s human growth hormone. Genentech would gain exclusive rights to sell recombinant human growth hormone under the brand name Protropin in the United States, while KabiVitrum would sell it intern


ch generated sales of about $7.5 billion (about $9.7 billion measured in 2018 dollars). In 2018, the same four products accounted for approximately $21.1 billion in sales for Hoffman-La Roche (Trefis Team and Great Speculations, 2019). Figure 3.7 sum


ed-b). Estimated national expenditures for cancer care in the United States were $150.8 billion in 2018 (National Institutes of Health, National Cancer Institute, 2020). Modern rDNA drugs and protein therapeutics are used in cancer treatments for mil


$10,000 plus downstream royalties based on a percentage of final product sales (Hughes, 2011). Over the lifetime of the patents, which jointly expired in December 1997, the technology was licensed to 468 companies, resulting in more than $35 billion


wide impacts cannot be solely attributed to individual inventors, they reflect the enormous and prolonged impact of invention. They might not have happened nearly as quickly—or at all—without the determination of the inventors and the risks they took


s for diabetes, growth hormone deficiency, leukemia, AIDS, hepatitis B, and multiple sclerosis—all of which have reduced the risk of premature death and provided tangible quality of life improvements for many.
46
4. Hood: The DNA Sequencer and Modern


impacts.
 47
Overview of Inventions
In 1960, Hood completed a B.S. in biology at California Institute of Technology (Caltech) and enrolled at the Johns Hopkins School of Medicine, where he was granted an M.D. in 1964. Hood then enrolled in the bioche


nk-jet oligonucleotide arrays synthesizer.
2 Discretizing the innovative contributions of a researcher as prolific as Hood is inexact, here our assessment is consistent with that of Hood himself. In a 2002 article providing a personal account of his


science, zoology, neuroscience, agronomy, parasitology, toxicology, and polymer science.4 Hood and his colleagues used the instrument to sequence platelet-derived growth factor (a human blood hormone), which showed that the N-terminal
3 Based on a se


t important scientific contributions of the DNA synthesizer was facilitating the synthesis of oligonucleotides. The DNA synthesizer was used to assemble oligonucleotides in an important study on genetic disease detection methods (Barany, 1991). The D


pidly determine the order of the four bases. The DNA sequencer also allowed rapid genotyping, a related process that focuses on determining the precise genetic variants that an individual possesses. Of the instruments invented at Caltech, Hood descri


e student at Caltech, published his first WOS-indexed article, “Evidence for Amino Acid Sequence Differences Among Proteins Resembling the L-Chain Subunits of Immunoglobulins,” in the Journal of Molecular Biology (Bennett et al., 1965). Over the cour


ns in which at least one co-author from each country was also listed on a publication with Hood. The network graph depends on data from Hood’s WOS-generated author profile. As described earlier, we found Hood’s author profile to undercount his scient


ation has been cited more than 14,000 times in WOS-indexed articles and more than 25,000 times according to Google Scholar.
11 Publications are reported as of October 15, 2020; therefore, the 2020 data may be incomplete. 55
 Technological Impact
As o


s,13 and Applied Biosystems (which merged with Invitrogen Corporation and later was acquired by Thermo Fisher Scientific). The separate appendix provides a summary of the top citing organizations to Hood’s patent portfolio.
Table 4.3 provides example


cientific evidence.
58
Economic and Social Impacts
Hood laid the technological foundation for genomics and proteomics through the invention of cutting-edge scientific instruments incorporating automation with integrated laser and computer technology.


immerman, 2016). Hood’s lab at Caltech, with funding from the U.S. Department of
59
Energy and the National Institutes of Health, became one of dozens of research centers in the United States, Europe, and Asia that were sequencing the human genome. T


orporated, initially Genetic Systems Company (GeneCo), was founded in 1981 in Foster City, California, to commercialize the technology developed by Hood and his colleagues. Its first commercial instrument, the Model 470A Protein Sequencer, was releas


ed cost
• Contributed to development of expressed sequence tags, a revolutionary new method for gene discovery and sequencing
• One of the key technologies that enabled the completion of the Human Genome Project
• Produced high yields of high-quality


ientific in 2014 for approximately $13.6 billion (Berkrot and Kelly, 2013).
Amgen
In 1980, William Bowes recruited George Rathmann from Abbott Laboratories and Winston Salser from the University of California, Los Angeles, to found Amgen (originally


9
Kyprolis
Xgeva
Prolia
Nplate
Vectibix Sensipar/Mimpara Neulasta
Enbrel
Aranesp Neupogen
Epogen
     In 2019, Amgen had about 22,000 employees and revenues of $23.4 billion, and invested more than $4 billion in new research and development (Amgen, 2


eatments pose the risk of serious side effects that must be weighed against the potential benefits of treatment.
In addition to becoming one of the most successful biotechnology companies, Amgen endeavored to make a positive social impact. In 1994, A


esearch/Project
Regulatory network for glioblastoma using systems genetics network analysis (published in Cell Systems)
High-resolution mapping of Huntington’s disease in mice (published in Human Molecular Genetics)
The Human SRMAtlas
Project Feed 10


) for $76 million in 1997. SyStemix’s pioneering work in stem cell research was shut down by Novartis in 2000. Weismann was able to license key technologies from Novartis, and more than a decade later, resumed research to grow and deliver blood stem


 and Phenogenomics Corporation (a biotechnology company seeking to develop rapid methods for detecting proteomic interactions
68
designed to discover new antibody-based drugs for cancer treatment) (Agilent Technologies, 2019).
In 2003, Hood co-founde


et, and other financial news sources. Note that in Chapter 2 where we reported the collective impacts of all of the Lemelson-MIT Prize winners, we only included economic and financial data for companies founded by the inventors, but not other compani


still in their infancy, and the full scale of her work’s impact is yet to be realized. Nevertheless, Bertozzi has already published hundreds of scientific articles, developed more than 60 patents, and co-founded seven companies.
Overview of Invention


om the 1990s suggested that such chemical reactions might exist or [at] least could be invented with some clever mechanistic thinking.” (Sletten and Bertozzi, 2011). In 2003, Bertozzi’s team published the first use of the term bioorthogonal (Hang et


ls at the site of their action” (Devaraj, 2018). In other words, a bioorthogonal reaction could be used to assemble a medically effective compound at the site of disease, rather than trying to direct a prefabricated medical compound to the right loca


son, the ultimate impacts of Bertozzi’s work remain to be seen. She has co- founded seven different companies based on different aspects of her research, but medical applications of her discoveries are still in trial phases, so the final outcome is u


, and with high accuracy.
Enables research advances in such areas as oncology, autoimmunity, metabolism, and neurology.
The goal is to discover and develop “first-in-class therapeutics that degrade extracellular and membrane-bound proteins that drive


ublish frequently, having more than 300 WOS publications. Figure 5.5 depicts Bertozzi’s annual publication output from 1990 to 2019 (the most recent year for which complete data are available), and Table 5.2 provides statistics on her publications an


ornia, Berkeley; 14 co-authored papers), Jeremy Baskin (another alumnus of Bertozzi’s University of California, Berkeley lab; 13 co-authored papers), Steven Rosen (a biology professor whom Bertozzi worked for as a postdoctoral researcher at UCSF, and


Center at Oakland. The separate appendix provides a summary of the top citing organizations to Bertozzi’s patent portfolio.
Table 5.4 provides examples of new inventions among the top 25 citing organizations (i.e., entities with the highest number of


cusses each organization’s activities and describes their potential impacts.
Thios Pharmaceuticals
The first company Bertozzi co-founded was Thios Pharmaceuticals, which she co-founded with Steve Rosen, Stefan Hemmerich (another former postdoctoral r


ite-specific protein modification tool. The technologies developed by Redwood “enable the generation of homogenous bioconjugates engineered to improve performance and ease of manufacturing.” This “enables the identification of superior drugs from lib


lop additional treatments for additional diseases—seems promising.
Enable Biosciences
In 2015, Bertozzi co-founded a third company, Enable Bioscience, along with Jason Tsai, Peter Robinson, and David Seftel; Tsai and Robinson are both graduate studen


1 diabetes, and the presence of any two or more of these autoantibodies generally causes type 1 diabetes to occur (Ziegler et al., 2013). These autoantibodies can be present well in advance of
84
disease symptoms; they can be present at as early as s


ies of receptors known as Siglecs turn off that response whenever they bind to any one of many sialic acid–containing glycans that cover the surface of the cancer cell. Targeting this “off switch” has been difficult because there are many different t


dditional parts of the body, but only 20 percent of ovarian cancers are detected in earlier stages because the disease is typically asymptomatic in early stages and an effective screening test is not available.3 Ovarian cancer is the fifth-most commo


a variety of drugs are available to cure cases of the disease, but the drugs must be taken for six to nine months. Some strains of tuberculosis are becoming resistant to certain drugs because patients stop taking the drugs too soon (Centers for Dise


ich Bertozzi has recently published several papers (Ahn et al., 2020; Banik et al., 2019). It has been suggested that Lycia has now advanced beyond the information described in Bertozzi’s publication (Tong, 2020). At this point, further details on th


as an excellent mentor to students of all ages, including postdoctoral fellows, graduate students, undergraduate students, and K–12 students. Many of Bertozzi’s companies and scientific publications are collaborations with students whom she has ment


 the inventions was integral to the selection of the winners of the Lemelson-MIT Prize. However, there have been and will continue to be many highly successful and impactful inventors beyond this small group.
Chapters 3 through 5 presented case studi


nia National Primate Research Center to work on optimizing convalescent plasma therapy for COVID-19 patients (Lee, 2020). InterVenn Biosciences (another company co- founded by Bertozzi), announced in October 2020 that it had “identified marked differ


n is an ongoing social investment—not a largely depleted checklist of opportunities. Future work should be focused on shaping such efforts to address inequality and assessing their effectiveness.
Several notable efforts are already underway. For exam


20.
Albert, M. B., D. Avery, F. Narin, and P. McAllister, “Direct Validation of Citation Counts as Indicators of Industrially Important Patents,” Research Policy, Vol. 20, No. 3, 1991, pp. 251– 259.
———, “Survival Rates for Ovarian Cancer,” webpage,


Genetic Disease Detection and DNA Amplification Using Cloned Thermostable Ligase,” Proceedings of the National Academy of Sciences of the United States of America, Vol. 88, No. 1, 1991, pp. 189–193.
Proceedings of the National Academy of Sciences of


11, pp. 651–653.
 Blind, Knut, Katrin Cremers, and Elisabeth Mueller, “The Influence of Strategic Patenting on
 Companies’ Patent Portfolios,” Research Policy, Vol. 38, No. 2, 2009, pp. 428–436.
 96
Bonifacio, Ezio, “Predicting Type 1 Diabetes Using


74, pp. 1030–1034.
———, “Construction and Characterization of Amplifiable Multicopy DNA Cloning Vehicles Derived from the P15A Cryptic Miniplasmid,” Journal of Bacteriology, Vol. 134, No. 3, 1978, pp. 1141–1156.
Chetty, Raj, John N. Friedman, Nathani


 4, No. 8, 2018, pp. 952–959. As of January 26, 2021: https://pubs.acs.org/doi/10.1021/acscentsci.8b00251#
Diamond v. Chakrabarty, 447 U.S. 303, 1980.
Drake, Penelope, “Catalent’s SMARTag® ADC Stands Out in the Crowd of HER2-Targeted Conjugates,” ADC


 Reticulum,” Journal of Biological Chemistry, Vol. 264, No. 36, 1989, pp. 21522–21528.
Flynn, Gregory C., Jan Pohl, Mark T. Flocco, and James E. Rothman, “Peptide-Binding Specificity of the Molecular Chaperone BiP.” Nature, Vol. 353, 1991, pp. 726–73


 Jaffe, and Manuel Trajtenberg, “Market Value and Patent Citations,” RAND Journal of Economics, Vol. 36, No. 1, 2005, pp. 16–38.
Hall, Bronwyn, Jacques Mairesse, and Pierre Mohnen, “Measuring the Returns to R&D, National Bureau of Economic Research,”


icago: University of Chicago Press, 2011.
“Human Insulin Market Size, Share & Industry Analysis, By Type (Analogue Insulin, Traditional Human Insulin), By Diabetes Type (Type 1, Type 2), By Distribution Channel (Retail Pharmacy, Hospital Pharmacy, On


t, William V. Treuren, Neil Martinson, Rainer Kalscheuer, Bavesh D. Kana, and Carolyn R. Bertozzi, “Rapid Detection of Mycobacterium Tuberculosis in Sputum with a Solvatochromic Trehalose Probe,” Science Translational Medicine, Vol. 10, No. 430, 2018


in, Scott T., Jeremy M. Baskin, Sharon L. Amacher, and Carolyn R. Bertozzi, “In Vivo
 Imaging of Membrane-Associated Glycans in Developing Zebrafish,” Science, Vol. 320, No.
 5876, 2008, pp. 664–667.
 Lee, Tim, “Cerus Forms Group to Research Optimal


en to Medicine and Farm,” New York Times, May 20, 1974.
Meselson, Matthew, and Robert Yuan, “DNA Restriction Enzyme from E. Coli,” Nature, Vol. 217, 1968, pp. 1110–1114.
Montague, P. Read, Carolyn D. Gancayco, Mark J. Winn, Richard B. Marchase, and M


ynthesis of Porcine Cardiodilatin 88,”
 Journal of the American Chemical Society, Vol. 110, No. 23, 1988, pp. 7847–7854.
 107
Olson, Maynard, Leroy Hood, Charles Cantor, and David Botstein, “A Common Language for Physical Mapping of the Human Genome,


ts, 2009–2019. As of January 21, 2021: https://www.roche.com/investors/downloads.htm
Rogers, Mary A. M., Catherine Kim, Tanima Banerjee, and Joyce M. Lee, “Fluctuations in the Incidence of Type 1 Diabetes in the United States from 2001 to 2015: A Lon


 Molecular Biology, Vol. 51, No. 2, 1970, pp. 379–391.
Smith, Lloyd M., Jane Z. Sanders, Robert J. Kaiser, Peter Hughes, Chris Dodd, Charles R. Connell, Cheryl Heiner, Stephen B. Kent, and Leroy E. Hood, “Fluorescence Detection in Automated DNA Seque


ffice, “Glossary,” webpage, 2021. As of January 16, 2021: https://www.uspto.gov/learning-and-resources/glossary
USPTO—See U.S. Patent Trademark Office.
U.S. Small Business Administration, “Table of Small Business Size Standards Matched to North Ameri


ses 1 and 2 in Mouse Collagen-Induced Arthritis,”
 BMC Immunology, Vol. 7, No. 1, 2006.
 112
Yoshimori, Robert, Daisy Roulland-Dussoix, and Herbert W. Boyer, “R Factor-Controlled Restriction and Modification of Deoxyribonucleic Acid: Restriction Muta


Health Service Utilization and Cost Outcomes from a Permanent Supportive Housing Program
Evidence from a Managed Care Health Plan
SARAH B. HUNTER, ADAM SCHERLING, RYAN K. MCBAIN, MATTHEW CEFALU, BRIAN BRISCOMBE, WILLIAM MCCONNELL, PRIYA BATRA
 Sponso


ronmental Policy Program
RAND Social and Economic Well-Being is a division of the RAND Corporation that seeks to actively improve the health and social and economic well-being of populations and communities throughout the world. This research was con


ocument the short-term programmatic costs of the PSH program.
Our previous work (McBain et al., 2020) demonstrated that members enrolled in the program were high health care utilizers with one or more chronic health conditions and high service costs


alo School of Social Work and Jeanne Ringel at RAND.
 ix
Abbreviations
ACG The Johns Hopkins ACG® System ADG Aggregated Diagnosis Groups
ASMD absolute standardized mean difference COPD chronic obstructive pulmonary disease DME durable medical equipme


duals maintain independence and autonomy (Rog et al., 2014). Research has demonstrated that PSH can increase housing stability among individuals who were formerly homeless (National Academies of Sciences, Engineering, and Medicine, 2018). Although it


nter et al., 2017). Because of shortages in affordable housing in Southern California, housing providers report that it often takes several months to obtain an appropriate unit (Hunter et al., 2020). In the meantime, PSH programs can have significant


(i.e., first six months) health care service utilization and associated costs of members enrolled in a PSH program, compared with a similar cohort of individuals who were not enrolled in the PSH program
• to better understand the interim housing and


lacement) periods. Together, these factors allowed us to take a preliminary look at the costs and the benefits of the program.
Data and Sources
Health Plan Member Demographics
The managed health plan provided the following health plan member characte


35.6% 64.4%
19.8% 27.7% 39.8% 12.7%
44.6% 55.4%
36.7% 46.3% 16.9%
      NOTE: ACG risk category refers to groupings based on ADGs experienced by patients using the ACG System.
Health Care Services and Associated Costs
The program provided health care


Additionally, we did not capture a preponderance of behavioral health services provided to patients, because, in most cases, these services were carved out within the counties’ behavioral health departments. In characterizing costs, the aim of this r


h sample size equal to the ESS. Similarly, when reporting the weighted mean of a particular characteristic of the comparison group (e.g., mean health care cost), we report the weighted sample size as the sum of the propensity score weights (i.e., the


teristics
Descriptive Characteristics Before and After Weighting
Table 3.1 provides descriptive information about the treatment and comparison group members, before and after weighting. We calculated absolute standardized mean differences (ASMDs) com


pendix B, Table B.3). The total cost during the transitional period was $3,463,229, compared with $1,022,796 during the PSH period.
Comparison group. As shown in Table 3.3, the propensity score–weighted total cost associated with health service utili


ommon procedure type.
b Pharmacy as a health condition refers to the dispensing of medications to patients, which may take place at a physical pharmacy or else in other settings, such as ERs or inpatient facilities.
17
88
6,422 $2,170,428 $24,592 $33


estimate of the treatment effect on the change in total costs, as shown by the large 95-percent confidence interval for the outcome labeled “Total.” This is largely attributable to inpatient costs, which contribute a large proportion of the total val


rspective. From April 2018 to March 2020, total programmatic costs amounted to $4,292,201, or $26,495 per enrollee (n = 162). The typical individual was enrolled for 11.0 months over this period, translating to $2,405 PMPM. Segmented by enrollment pe


 hospital discharge who do not have a place to recuperate, while custodial care reflects longer-term supportive health services for patients in need of more-consistent medical monitoring.
Among the 162 enrollees, 116 clients received transitional hou


er reduction in health care costs between pre- enrollment and post-enrollment periods (see Tables 3.2 and 3.3). Specifically, program enrollees observed an $18,035 reduction in health care costs between baseline and intervention periods, while nonenr


d in the PSH program. Finally, we examined the PSH costs since program start-up and examined PSH costs in relation to individual health service utilization to explore whether any cost offsets were observed. In summary, we found the following:
• PSH e


ing into an independent living arrangement (i.e., PSH). Interim housing provides an added level of care (e.g., recuperative care) that can add to the transitional costs observed during this period.
Overall, our findings are similar to those of other


lly significant effects between the treatment and comparison groups.
Because of the program design, the health plan had more information about the status of PSH program enrollees than members in the comparison group. For example, we prorated the data


enrollment periods. This uncertainty is due, in part, to the high volatility and the highly skewed nature of health
30
claim costs (and, in particular, inpatient costs). It is also due, in part, to the short-term observational window of six months po


tation was the time between enrollment and housing placement. The average time was about three months, with a range of ten to 258 days. The time was highly variable, primarily based on the health status of participants. The length of time is also par


n group, ten candidate enrollment dates were generated by adding a number of days drawn from the aforementioned distribution to the Z59.0 diagnosis date. After candidate enrollment dates were selected, a propensity score model was fit to estimate the