al
Alignment and the Use of Incentives Can Promote a Culture of Health: Stakeholder Perspectives , Santa Monica,
Calif.: RAND Corporation, RR-1889-RWJ, 2017. As of February 13, 2019: https://www.rand.org/pubs/research_reports/RR1889.html
National A


Johnson Foundation.
U.S. Bureau of Labor Statistics, American Time Use Survey: 2013 Results , Washington, D.C., June 18, 2014. As
of January 6, 2016:
http://www.bls.gov/news.release/archives/atus_06182014.htm
U.S. Census Bureau, “Current Population


Examining Civic
Engagement Links to Health
Findings from the Literature and
Implications for a Culture of Health
Christopher Nelson, Jennifer Sloan, Anita Chandra
CORPORATIONLimited Print and Electronic Distribution Rights
This document and tradem


cross various (mostly nonhealth) disciplines, is still mostly formative in terms of research rigor, and involves complex rela -
tionships among actors at the local, regional, national, and interna -
tional levels. Furthermore, much of the empirical r


s in knowledge that might point to possible investment opportunities in new research?
This narrow focus is intended as a building block for additional
reviews and analyses describing and characterizing the full range
of interventions, social moveme


ate this may be related to differences in social stigma (e.g., the extent to which individuals are deemed responsible for their condition), sophistication of pressure groups formed around a condition, and more general differ -
ences in national cultu


nd just a few studies on civic engagement related to religious organizations. Perhaps most importantly, although there is a reasonable body of emerging evidence linking individual civic engagement to individual health, there is little attention to da


onsoring conferences, forums, and networks to provide venues in which researchers can collab -
orate (specifically around shared data sets, as mentioned above) and more actively contribute articles to build the literature base.
• Support collaborati


eclaiming health as a shared priority. RWJF iden -
tified three drivers of change needed to make progress in this Action Area: (1) mindset and expectations (how the U.S. public views and pri -
oritizes health, well-being, and related investments); (2


collective
action through movement in perspective, shared narrative, and cultural meaning” (Chandra et al., 2016)—in other words, this action requires a change in mindset among other cultural shifts. A mindset shift in health is currently most evide


ng and characterizing the full range of interventions, social movements, and other efforts that use civic engagement as a tool for addressing the social determinants of health was beyond the scope of this initial study. These elements also are key to


political science, and other social sciences. Each term was paired with “health” and “well-being.” This yielded a total of 1,329 results, from which two members of the team independently identified 157 articles for data abstraction, most of which wer


information about the links between civic engagement and health outcomes in the most rigorous studies available to date.
During our title vetting, articles were included in the “possibly
relevant” bucket if there was any indication that they might


 exploring whether the nature of the conclusions differed across quality ratings.  11CHAPTER THREE
Health and Voting Participation
We begin with key findings from studies related to voting participa -
tion and health. Most of these studies focus not


ld Development Study, a longi -
tudinal study of all people living in Ireland and Britain who were born in one week in March 1958 examined correlates of voting in the 1979, 1987, and 1997 elections (sample sizes were 8,463, 6,653, and 6,782, respecti


r than men, because they are, on average, widowed and live alone at an earlier age than men, and because women live longer and are typically younger than their husbands. Also, older
1  This is also one a few studies we located that explored the rela


 explicit comparison of the effects of
poor health on voting with other forms of civic participation. Söderlund and Rapeli (2015) analyzed a 2012–2013 cross section of more than Health and Voting Participation    15
8,000 respondents to the European


008 U.S. presidential election. After adjusting for sociodemographic characteristics and some health-related confounding factors, they found that individuals with cancer diagnoses are more likely to vote, while those with heart disease diagnoses are


ealth
Some studies focus specifically on the health-voting connection in the
context of behavioral and mental health. The next sections summarize those findings, starting with behavioral choices.
Voting and Health Behaviors
There are three studies


duals taking precau -
tions to reduce health risks, including HIV risk. This is important in a population that could be considered less healthy and marginalized and perhaps less likely to engage given prior study findings on poor self-reported health


s, wave 2 had 14,738 respondents, and wave 3 had
15,197 respondents.20    Examining Civic Engagement Links to Health
community engagement might have mental health benefits for youths,
and that depression might reduce later civic engagement in adult


t the low voting rate among psychiatry inpatients and individuals with mental illness might be the result of issues of access, administrative issues (e.g., lack of an identity card), lack of knowledge, and neglect from political candidates because of


e relationship between regional turnout rates in Finland (district level) and the United States (state level) from 1995 to 2015 with measures of local influenza prevalence. In both countries, regression models sug -
gested that influenza outbreaks we


ed inequality amplifies psychosocial comparisons up and down the hierarchy. This suggests that socioeconomic inequality in voter turnout is associated with poor self-rated health, independent of both income inequality and state median household incom


es, civic engagement is conceptualized as part of
the broader construct of social capital. However, we include only those studies that contained a clear and specific measure of civic engage -
ment as a part of the broader social capital construct. F


organization or any group concerned with the environment, pollution, and related issues. Using data from the 1974 and 1994 versions of the Alameda County Study ( n = 6928), the authors prospectively analyzed
the association between 1974 volunteeris


that “rather than having a contextual influence on health, the benefi -
cial properties of social capital can be found at the individual level.” Similarly, a global analysis of adults older than age 50 by Van Groezen, Jadoenandansing, and Pasini (201


ture. For instance, Veenstra’s (2000) analysis of 534 observations from a 1997 cross-sectional study of adults in Saskatchewan, Canada, found that an engagement index (including voting, writing letters to editors, and paying attention to community is


lder were less likely to be hypertensive but more likely to have lipid dysregulation than nonvolunteers. Gold et al. (2002),  in turn, used cumulative data from
the U.S.-based General Social Survey for 1986–1990 (put out by the Centers for Disease C


volunteering was a stronger predictor for physical activity than “other volunteering,” perhaps because environmental vol -
unteering could require more-active outdoor engagement. Two addi -32    Examining Civic Engagement Links to Health
tional artic


nxi -
ety and depression) after controlling for sociodemographic variables. CMD was measured using the General Health Quesionnaire-12, which includes three constructs of anxiety, social dysfunction, and loss of confidence. Similarly, Buck-McFayden et


rs also noticed a theme of political participation “as a component of empowerment for minority groups in general.”
Finally, a study by Acevedo, Ellison, and Xu (2014) suggests that
the effect of civic engagement on health can be indirect. In a 2004


 the mediating role of sense of community and empowerment. Social well-being was measured using the Mental Health Continuum Short Form. The authors found that group membership, other than in recreational groups, is associ -
ated with higher social we


trust decreased significantly and then rose (i.e., were most discrep -
ant from each other across the study) and had the strongest association with high scores on a “contribution” scale that included community leadership, service, helping behaviors,


 family and community during adoles -
cence predicted greater likelihood of voting, community volunteer ser -
vice, involvement in social action groups, and a better sense of civic trust. At a broader contextual level, exposure to community violence


mbership in civic organizations were associated with lower neighborhood death rates (for ages 45–64) for total mortality and heart disease mortality for whites and blacks; no association was found with cancer mortality.
In another study, Cutlip, Bank


omote civic engage -
ment and/or to help realize the health benefits of engaging civically. We include only studies that explicitly seek to estimate an interven -
tion’s effects. Therefore, we do not seek to include the full range of studies that des


Brown et al., 2017).
In another study, Dabelko-Schoeny, Anderson, and Spinks (2010)
piloted an intervention in 2008 in a Midwestern city in the United States to promote civic engagement in adults with functional limita -
tions. Subjects came from a


effects of these participation-increasing measures.
1  More information on this scale is available online at the Positive Psychology Center’s
website (undated).44    Examining Civic Engagement Links to Health
Civic Engagement Interventions to Impro


 indirect effects (such as affordable housing). The authors concluded that community organiz -
ing–based interventions designed and led by community stakeholders could identify environmental and policy solutions to address structural inequities relat


e social support among youths. The authors argue that youths with more social support and meaningful participa -
tion are more healthy in general. Conducted in Kansas, the campaign started in 1998, targeted adults, and provided information about and


get populations vary in demographic background; therefore, more work is needed to determine how well these interventions might work across a more-diverse population mix. 48    Examining Civic Engagement Links to Health
InterventionTarget
Population


nity into tan -
gible actions that could lead to new partnerships, improvements in the health conditions found in communities, and the degree of integration among health services and systems. However, the evidence base linking civic engagement and he


f Civic Engagement
Health Outcome Voting Civic Engagement
Physical health 10 20
Mental health 9 13
Patient populations 2
Community health 4 4
Well-being 0 9
Adolescents 0 8Discussion    53
findings (e.g., Chola and Alaba, 2013) (see Table 6.2). Moreo


cess, (2) the lack of participation leads to underrepre -
sentation and a lack of policies that benefit those with depression, and (3) the lack of beneficial policy outcomes perpetuates the experience of depression. And so it repeats.
If true, and d


t
with this viewpoint—for example, physical disability, which may 56    Examining Civic Engagement Links to Health
increase the private costs and burdens of participation, has been shown
in some studies to be related to lower voting turnout (Schur


e teen and early adult years. There is reason to believe that stigma can apply to health behaviors, such as smoking (Albright et al., 2015; Denny and Doyle, 2007). However, the nature of this relationship might be mediated by culture, as is sug -
ges


disease
but not from cancer (Lochner et al., 2003). There is also some evi -
dence that the nature of the health–civic engagement connection might vary by social group, as blacks with low education and cancer are more likely to turn out than well-e


esearch is needed that leverages longitudinal designs to unpack causal relationships and further elucidate how health experience and voting behavior evolve over time. In addition, further analyses on how the health status of a community, and not simp


, environments, and community power structures (and, in turn, can be causally  linked
to health outcomes) is needed.
• Support the development and sustainment of multi-
disciplinary communities of practice. Similarly, work in this area involves a


: The Role of Participation”
Abu-Ras (2013) “American Muslim Physicians’ Public Role Post-9/11 and
Minority Community”“Empowerment: Serving the Underserved”
Acevedo, Ellison,  and Xu (2014)“Is It Really Religion? Comparing the Main and Stress-Buffer


Illness Experience in the United States”
Buck-McFadyen (2018) “Social Capital and Self-Rated Health: A Cross-Sectional Study of the General Social Survey Data Comparing Rural and Urban Adults in Ontario”
Bulanda and Jendrek (2016)“Grandparenting Role


 Civic Engagement in Young Adulthood”
Emerson et al. (2014) “Perceptions of Neighbourhood Quality, Social and Civic Participation and the Self-Rated Health of British Adults with Intellectual Disability: Cross-Sectional Study”
Fang et al. (2018) “Hap


c Participation”
Leedahl, Sellon, and Gallopyn (2017)“Factors Predicting Civic Engagement among Older Adult Nursing Home Residents”
Lenzi et al. (2012) “Family Affluence, School and Neighborhood Contexts and Adolescents’ Civic Engagement: A Cross-Nat


on Turnout in 30 Countries”
Mattila et al. (2018) “Sick Leave from Work and the Voting Booth? A Register-Based Study on Health and Turnout”
McAllister (2008) “Public Support for Democracy: Results from the Comparative Study of Electoral Systems Proje


on in Political Organizations: The Role of Generativity and Its Impact on Well-Being”
Seyfang (2003) “Growing Cohesive Communities One Favour at a time: Social Exclusion, Active Citizenship and Time Banks”
Shin and McCarthy (2013)“The Association Bet


ngagement, Community Size, and Recall of Health Massages”
Wagenaar  et al. (2000)“Communities Mobilizing for Change on Alcohol: Outcomes from a Randomized Community Trial”
Wakefield  et al. (2001)“Environmental Risk and (Re)action: Air Quality, Healt


nal Self-rated
healthCivic
engagementCivic engagement
not related to
better health
Bartkowski
and Xu (2007)“Religiosity and
Teen Drug Use
Reconsidered:
A Social Capital
Perspective”U.S. Cross-sectional Alcohol use Civic
engagementHigher c


e (2014)“Connection
Between
Adolescent’s
Exposure to
Community
Violence and
Future Civic
Engagement
Behaviors During
Their Young
Adulthood”U.S. Longitudinal Exposure to
community
violenceCivic
engagementExposure to
violence
associate


ts
with Intellectual
Disability: Cross-
Sectional Study”UK Cross-sectional Self-rated
healthCivic
engagementHigher civic
engagement
related to better
health among
those with and
without intellectual
disabilities
Fang et al. (2018) “Happine


ngagement
related to better
health
Kelleher et al.  (2002)“Indicators of
Deprivation, Voting
Patterns, and
Health Status at
Area Level in the
Republic of Ireland”Ireland Cross-sectional Self-rated health Voting Higher voting rates
related to


agement
Indicators Association
Mattila et al. (2013) “Healthy Voting:
The Effect of Self-
Reported Health
on Turnout in 30
Countries”30 European
countriesLongitudinal Self-reported
healthVoting Better health
associated with
higher voter
turn


nsome et al. (2016)“Social Capital Is
Associated with
Late HIV Diagnosis:
An Ecological
Analysis”U.S.
(New York)Cross-sectional HIV diagnosis Civic
engagementCivic engagement
related to lower rates of late HIV
diagnosis94    Examining Civic


th health
Veenstra (2002) “Social Capital
and Health (Plus
Wealth, Income
Inequality and
Regional Health
Governance)”Canada
(Saskatchewan)Cross-sectional Mortality Civic
engagementHigher civic
engagement
related
to lower
mortalityDetai


olitical Science Review , Vol. 84, No. 2, 1990,
pp. 377–393. References    101
Bolton, M., I. Moore, A. Ferreira, C. Day, and D. Bolton, “Community
Organizing and Community Health: Piloting an Innovative Approach to Community Engagement Applied to


 Young People: Understanding Pathways from Civic Participation to Social Well-Being,” Voluntas , Vol. 26, No. 1, 2015, pp. 24–44.
Collins, C. R., and S. Guidry, “What Effect Does Inequality Have on Residents’ Sense of Safety? Exploring the Mediating


opment , Vol. 42, No. 4, 2018,
pp. 425–433.
Finlay, A. K., and C. Flanagan, “Adolescents’ Civic Engagement and Alcohol Use:
Longitudinal Evidence for Patterns of Engagement and Use in the Adult Lives of a British Cohort,” Journal of Adolescence , V


e Around the World , Cambridge, UK: Cambridge University Press, 2003.
Jennings, M. Kent, and Richard G. Niemi, Generations and Politics: A Panel Study of Young Adults and Their Parents , Vol. 68, Princeton, N. J.: Princeton University
Press, 2014.


Classroom Climate: The Mediating Role of Self-Efficacy,” Journal of Applied Developmental Psychology , Vol. 41, 2015, pp. 8–18.
Marquez, B., P. Gonzalez, L. Gallo, and M. Ji, “Latino Civic Group Participation, Social Networks, and Physical Activity,


hers/psychological-well-being-scales
Ramlagan, S., K. Peltzer, and N. Phaswana-Mafuya, “Social Capital and Health
Among Older Adults in South Africa,” BMC Geriatrics , Vol. 13, No. 1, 2013.
Ransome, Y., S. Galea, R. Pabayo, I. Kawachi, S. Braunstein


e First South African National Income Dynamics Study,” Journal of Affective Disorders ,
Vol. 144, No. 1–2, 2013, pp. 101–105.
Urbatsch, R., “Influenza and Voter Turnout,” Scandinavian Political Studies ,
Vol. 40, No. 1, 2017, pp. 107–119. Van Groe


of August 1, 2019: http://www.ncbi.nlm.nih.gov/pubmed/23629592
Zhu, L., “Healing Alone? Social Capital, Racial Diversity and Health Care
Inequality in the American States,” American Politics Research , Vol. 45, No. 6,
2017, pp. 1059–1087.
Ziegenfu


CORPORATION
Future-Proofing Justice
Building a  Research Agenda to Address the Effects
of Technological Change on the Protection of
Constitutional Rights
Brian A. Jackson, Duren Banks, Dulani Woods, Justin C. Dawson
As changes occur in society, fit


s of the panel included require -
ments for best practice and training develop -
ment, addressing such issues as criminal justicedata quality and its implications for individuals’rights; evaluation work to better understand howanalytic tools (such as


t data from those devices? The correct answer to such questions is not obvious, but what is clear is that simple analogies between new and old technologies are likely not enough to understand their implications. For example, it might be easy to make


enforcement through the issuance of warrants and determining the length and nature of punishment for an individual in the correctional system—our focus broadened to explore questions of how technological change might affect the protection of individu


 to shape trial outcomes (Davis, 2013).
As discussed earlier in the context of virtual presence,
when it comes to protecting individuals’ rights, a single tech-nology may produce positive effects, negative effects, or both. For example, significant


rovide government might begin to affect individual behavior and infringe on freedoms protecting expression, association, and political action (see Kaminski and Witnov, 2015) .
To move beyond these contemporary examples, we devel-
oped a framework for


ntire business models rely on wresting data about individuals from their control in order to sell the information
 —for e
xample, for
delivering targeted advertising or building customer profiles for commercial purposes.4Next, we identified personal


y, we defined a class of the wider societal technology
ecosystem  to capture the fact that sensors are being installed in a wide variety of public places—not just by government but by private property owners—for security and other purposes. The more


rns about the following:o
 fair
ness (e.g., in analytic tools)
o influ
ence (e.g., whether virtual simulations might
influence juries beyond the facts of the case)
o effec
tiveness (e.g., whether virtual presence
meets justice objectives as well as


ls’ (e.g., purchasers, other occupants, and visitors) knowledge or understanding.
•	Technologies (e.g., voice-command systems) may monitor constantly and send data to external businesses’ systems for analysis.
Vehicle-integrated technologies•
 Conn


ore the meeting date, we sent panel members a back-
ground document and pre-workshop questionnaire that drew on the research team’s review of published literature on these issues. The materials were structured using different categories of rights—the


dividuals’ rights. To
provide structure to the set of identified needs, we asked the panel members to rank each need based on its expected benefit (how important they thought it would be if the need was met) and the probability of success of actuall


at were identified, the research team also identified five key overarching themes.
2 Although the
workshop was framed to explore both challenges and opportuni -
ties that new technologies present to the protection of indivi -
dual
s’ rights, the gre


y problems and responding more effectively to deter and solve crimes when they occur, and these initiatives are now broadly accepted in many departments (Police Executive Research Forum, 2013; Ratcliffe, 2016). As alluded to in the introduction, the


With data-sharing, inaccuracy can travel—because of either objective inaccuracy from data-entry errors or more-
 subj
ective
inaccuracy from how uncertain information is recorded.4 And
when data are copied and ingested into many separate data sets,


, 1987). Academic stud-ies have shown that although some risk assessment tools have reasonable levels of predictive power, results from others are much less predictive, and some assessment tools show different rates of false positives and negatives f


, Joh, 2016, p. 40) but also for forensics tech -
niques and data sets (e.g., National Research Council, 2009, pp. 273–274; Murphy, 2007), and even physical devices used by law enforcement for collecting data (e.g., Liebow, 2010).
During the panel’s


how people see themselves; for example, people spend money on virtual goods that exist only in online systems and see such goods as part of their effort to define their own identities (see, for example, Koles and Nagy, 2012; Nagy and Koles, 2014). M


nd will continue to provide protections against self-incrimination. We are already at a point where several programs are unbreakable.”
– Panel Memberet al., 2015, p. 58), although they also indicated that what is
shared on social media is “less auth


text of lifelogging—research has sug-gested that using these technologies has had an effect on how their users think, what information they remember, and where
they remember it. Studies have suggested that given connec-tivity to extensive resources


Internet adver-tisements to help shape sales messages. Biometric data from personal fitness devices can also be analyzed to make inferences about individuals’ emotions or activities. Data collected from a personal fitness tracker has already been us


nd no longer part of their “papers and effects?” The case of a technology that is implanted within a person would seem to cross that line and be more appropriately viewed as part of the person rather than something they own or possess. But how far be


 should have walked the 15 miles to the grocery store? In practical terms, his other choice was basically not to eat or go to the doctor .
In the criminal justice system, the third-party doctrine
governs government access to data that individuals h


, at least for some people, may take the place of person-to-person phone calls, capturing communication in written form that might have once been conveyed verbally in a way that was not routinely stored.
11 The revealing nature
of these streams of c


use become ubiquitous in products—for example, all automobiles are equipped with black boxes that record driving behavior and performance, or future intelligent transportation systems capture such data themselves (see Douma, Garry, and Simon, 2012)—t


t before involved actions that were much more clearly invasive—and generally required warrants. For instance, access to smartphone location logs provides data similar to physical surveillance of a person or installation of a tracking device on the pe


e
to the top tier—fundamentally examining how the validity of
the third-party doctrine may shift given the increasing require-ment for disclosure of data to third parties and addressing the pressures that social media use by varied criminal justice


without
massive disclosure of data to third parties. This is especially important given that modern life necessitates individuals turning over large amounts of data about themselves to others.
Tier 2 •
 Crea
te improved mechanisms (e.g., educational


argue about a defendant’s emotional state during an alleged crime, what do judges, prosecutors, and defense counsel need to know to sufficiently understand how electronic devices compile, store, and process data? When varied types of data are brough


ec -
tions related to ESI has focused on how to effectively apply the existing rules of evidence. Frieden and Murray (2011) argue that counsel should draw analogies between the source of ESI and the most similar, non-ESI or traditional source of evi


he need for stronger knowledge in the legal system
regarding such digital evidence is known, and there is no short-age of courses and opportunities for continuing legal education on such topics. For example, the National District Attorneys Associati


otected hinges on whether judges and counsel are suf-ficiently informed about ESI to appropriately and effectively apply that guidance. Such knowledge is critical to challenge the admissibility of electronic evidence resulting from poor chain of cust


rants at all parts of the process. This is especially important given the role of prosecutors and judges in issuing warrants and of defense attorneys in challenging search and seizure of new technological data.
•
 Deve
lop best practices and qualific


at night. We would like to bring all of this together for you. Please put on the headsets next to each of your seats.
The jurors put on the virtual reality headsets, which
cover their eyes and ears. The reenactment begins.
Prosecutor: What you are s


he country for hearings that are presumed to not affect the outcome of a case, such as hearings to determine bail, waive right to a jury trial, receive a jury verdict, enter a plea, sentence a defendant, and conduct post-conviction and parole reviews


during the study period, which was significantly greater than the 13-percent increase in bond amount experienced by the control group of felony defendants who appeared in person for bail hearings (Diamond et al., 2010). Research on the use of remote


urtrooms through-out the country, innovators are developing technologies that push further into virtual space, and these might be used in courtrooms in the future. For instance, advances in technology are under way to support more-realistic virtual r


 Feigenson, 2006, and refer-ences therein; Nemeth, 2011, and references therein). With advances in video game technologies, simulation capability
“These new technologies
are [computer graphics] on steroids. The more real and personal interpretation


to do so by identifying research needs to understand and mitigate potentially negative effects of technology on the protection of individuals’ rights and by identifying and exploring how new technologies could aid in protecting those rights. During t


ariety of such resources are available. As a result, some of the best practices identified in the workshop’s research agenda are likely already available, while others still need to be developed. For example, because telepresence and video
-
confe
re


 quality and reliability with retention and use as articulated by the panel.
Beyond best practices, the panelists also identified needs
related to educating various actors in the criminal justice sys-tem to better take on the complexities of new tec


ers).
Fundamental Research
Because many of the issues discussed dur-ing the panel covered technologies that are still developing, related shortfalls in knowledge on several issues will require
fundamental research to inform future decisions. The top


 where data volume has been a core issue in considering e-discovery in commercial and other types of cases (Pace and Zakaras, 2012). Just as a large volume of information can complicate the inves -
tigation and prosecution of a case (see, for example


e top-tier needs, but panel members felt that these issues were somewhat more difficult to solve or that the solutions were more difficult to implement.
Within that second tier, needs fell into each of the three
research agenda categories, including


ation is ensuring that justice processes protect the rights of individuals guaranteed in the Constitution. While not the only vector of change in society, technology can be a powerful force, with the potential to transform what is possible for citize


ets of organizational behavior to find evidence of misbehavior that would otherwise be hidden. But maintaining balance in the effects of technology on individuals’ rights and protection of due process within the justice system is challenged by resour


and emerging technologies and the different categories of rights they may affect.1
Social media platforms have created opportunities for inappropriate activities, such as attorneys, judges, and jurors “friending” each other. Some media platforms (e.g


ce system participants on modern electronic technologies and scientific evidence. 1
Criminal justice agencies are creating, collecting, storing, and reusing data that are of questionable quality. Once recorded, such data lose their full context, ofte


ot be fully explained by the human analysts who apply them.Develop best practices related to what level of certainty is necessary from automated algorithms for different justice system applications (e.g., probable cause, evidentiary purposes) and wha


, integrating motor vehicle records, welfare records, and follow-up programs).2 None
The nearly ubiquitous availability of online and social media has dramatically increased the potential for inappropriate extrajudicial research by jurors and other p


urt proceedings.Create standards to quickly and transparently assess the authenticity of converted and admitted video evidence. 333Problem or Opportunity Associated Need TierRelated
Theme(s)
Body-worn technologies (e.g., fitness trackers)
may broad


equential decisions that affect individuals. In the credit context, however, provisions of the Fair Credit Reporting Act provide consumers access and procedures to dispute contents of their credit file that they believe are inaccurate.
5 Both analyst


t the telephone stations.
16 For discussion of this issue in the legal literature, see, for example,
Henderson, 2006–2007; Spencer, 2013; and Kerr, 2009.
17 Authenticating data must also consider the possibility that evidence
on a person’s device w


ors or corrections” (Code of Federal Regulations, 2015).
25 For a discussion of this issue, see, for example, Bushway and
Smith, 2007.
26 For examples of recent debates about the tools’ fairness, see discus-
sion from a variety of perspectives in Ja


I, No.  3, 20
11. As of December 12, 2016:
http://jolt.richmond.edu/v17i3/article9.pdf
Barry, Nicholas, “Man Versus Machine Review: The Showdown
Between Hordes of Discovery Lawyers and a Computer-Utilizing Predictive-Coding Technology,” Vanderbilt


cus.gov/sites/default/files/documents/Draft_Best%2520Practices%2520Video%2520Hearings_10-09-14_1.pdf
Chauriye, Nicole, “Wearable Devices as Admissible Evidence:
Technology Is Killing Our Opportunity to Lie,” Catholic University Journal of Law and Te


stiglione, Giuseppe Cattaneo,
Giancarlo De Maio, and Mario Ianulardo, “Automated Construction of a False Digital Alibi,” in A M. Tjoa, Gerald Quirchmayr, Ilsun You, and Lida Xu, eds., Availability, Reliability and Security for Business, Enterprise a


2016:
http://www.okbar.org/members/BarJournal/archive2008/Mararchive08/obj797esi.aspx
Electronic Privacy Information Center, “Algorithms in the Criminal
Justice System,” web page, undated. As of December 12, 2016: https://epic.org/algorithmic-tran


38Heintz, Michael E., “The Digital Divide and Courtroom Technology:
Can David Keep Up With Goliath?” Federal Communications Law Journal , Vol.
 54, N
o. 3, 20
02, pp.  567
–589.
Henderson, Stephen E., “Beyond the (Current) Fourth Amendment: Protecti


ngs: Legal and Empirical Issues and Directions for Research,” Law and Policy , Vol.
 28, N
o. 2,
20
06, pp.  211
–227.
“Juries ‘Could Enter Virtual Crime Scenes’ Following Research,” BBC News, May 24, 2016. As of December 12, 2016: http://www.bbc.co


After Underdahl,” Minnesota Journal of Law, Science, and Technology , Vol.
 11, N
o. 2,
20
10, pp.  853
–875. As of December 12, 2016:
http://scholarship.law.umn.edu/mjlst/vol11/iss2/15
Logan, Wayne A., and Andrew Guthrie Ferguson, “Policing Crimin


Practices.ashx
———, “Jury Managers’ Toolbox: Best Practices to Decrease
Undeliverable Rates,” 2009b. As of December 12, 2016: http://www.ncsc-jurystudies.org/~/media/Microsites/Files/CJS/Toolbox/Undeliverable%20Best%20Practices.ashx
National Legal A


ivacy-age-smartphone
R.A. Malatest and Associates Ltd., Evaluation of the Bail Reform Pilot
Project, Peace Region and Surrey , March 31, 2010 .
Ratcliffe, Jerry H., Intelligence-Led Policing , 2nd ed., Abingdon,
Oxon, UK: Routledge, 2016.
Raynes-Go


More from the #Jury Box: The Latest on Juries and Social Media,” Duke Law & Technology Review , Vol.
 12, N
o. 1, 20
14, pp.  64–
91.
St. Eve, Amy J., and Michael A. Zuckerman, “Ensuring an Impartial Jury in the Age of Social Media,” Duke Law & Techn


lem/Villasenor, John, “What You Need to Know About the Third-Party Doctrine,” The Atlantic , December 30, 2013. As of August 26, 2016:
http://www.theatlantic.com/technology/archive/2013/12/what-you-need-to-know-about-the-third-party-doctrine/282721/


urt system and criminal justice overall. We would also like to acknowledge the contributions of Martin Novak and Steve Schuetz of the National Institute of Justice. James Anderson of the RAND Corporation provided advice and assistance in legal elemen


officer on afloat and ashore assignments in Miami, Fla.; New London, Conn.; and Baltimore, Md.
Justin C. Dawson is a senior research scientist at RTI International. His research focuses on courts and corrections, criminal justice, and inter -
nationa


ations do not necessarily reflect the opinions of its research clients and sponsors.
R® is a
registered trademark.
RR-1748-NIJCORPORATION© Copyright 2017 RAND Corporation
www.rand.orgThis publication was made possible by Award Number 2013-MU-CX-K00


The Cost and Affordability
of Flood Insurance in  New York City
Economic Impacts of Rising Premiums and
Policy Options for One- to Four-Family Homes
Lloyd Dixon, Noreen Clancy, Benjamin M. Miller, Sue Hoegberg,
Michael M. Lewis, Bruce Bender, Sama


the national level. Congress instructed the Federal Emergency Management Agency (FEMA) to develop an affordability framework in light of legislation that directs FEMA to gradually eliminate certain program subsidies and to collect additional program


....................................................................... 119
To What Extent Is Purchasing Flood Insurance Burdensome for Households
Living in One- to Four-Family Homes in the Study Area?  ............................... 119
How Might


 U
nder PFIRM Versus 2007 FIRM for Properties in
High-Risk Zones of the 2007 FIRM  ................................................. 42
 3.4. Difference B
etween Structure Elevation and PFIRM BFE in the Study Area  ..43
 3.5. Premium Pro
jections U


 Burden–Based Subsidy: Program Outcomes
with Full Participation  ................................................................. 99
 6.8. Design 2: Enr
ollment and Program Cost Under Different
Program Participation Assumptions  ................


ehold Income and Household Net Worth  .......... 149
 D.1. Fraction of I
ncome Spent by One- to Four-Family Households in the Study
Area on the Components of PITI, Owner-Occupied Residences Only  ......... 153
 D.2. Fraction of I
ncome Spent on PIT


ions, with a focus on one- to four-family homes in areas of the city that are at high risk of flooding (the study area):
1.
 To what extent i
s purchasing flood insurance burdensome for households living
in one- to four-family homes in the study are


s than 40 percent of the households living in them are low income, and the percentage of low-income households is substantially higher in some of the five subareas examined. A considerable number of one- to four-family structures face substantial flo


at flood insurance is burdensome for 64 percent of extremely and very low–income households and for 41 percent of low-income households. The proportion of households for which flood insurance is burdensome varies across the five subareas, with the hi


 some policyholders are
offset by charging higher rates to other policyholders. Although this cross-subsidy does not come from outside the
program, it nevertheless is a subsidy for certain policyholders. Table S.1
Median Premium in Flood Insurance


into an 8-inch increase in BFE across the study area. We estimate that such an increase in BFE would cause the average full-risk rates projected using the PFIRM to increase by approximately 10 percent across the study area as a whole. However, more w


n the study area, resulting in defaults rising from roughly just more than 300 per year to roughly 450 per year, or 1.5 percent of homes with mort -
gages per year. Most of these defaults will be in the high-risk zones of the 2007 FIRM.
Table S.2
De


 larger benefits are delivered to the target popu -
lation by this design, even though the program cost is substantially less than that for the income-based subsidy (design 1). One downside of the housing burden–based sub -
sidy is the extra informat


 occurs, is reimbursed for deductible payments that exceed $2,000; benefit reduced for moderate- and middle-income households
4. Mitigation loans and grants Grants for the low-income households and low-interest  loans for moderate- and middle-income


nd the 2015 NFIP rate schedule
with pre-FIRM rates.
b Assumes that flood insurance premiums are based on the PFIRM and the 2015 NFIP rate schedule
without pre-FIRM rates or grandfathering.
c Does not include administrative cost.Summary    xxv
finan


City Mayor’s Office of Recovery and Resiliency for providing intellectual leadership throughout the course of the project. They also played key roles in efforts to encourage residents to participate in the study and helped us obtain data from other


 draft report. Elizabeth Asche, Claudia Murphy, and Andy Neal at FEMA provided helpful feedback on interim findings. The report is a better product because of their efforts.
At RAND, statisticians Stephanie Kovalchik and Michael Robbins designed the


andy struck New York City on October 29, 2012, with devastating conse-
quences. It highlighted the importance of programs and policies that promote greater resilience to flood events. Flood insurance is an important part of this resilience strat -
eg


levant not only to New York City, but also to the United States
as a whole. Congress recognized the challenges that the phase-out of subsidies could create for households and directed Federal Emergency Management Agency (FEMA) to study “methods of e


Beach, and Hamilton
Beach in Queens)
• Rockaway Pe
ninsula (in Queens)5
• East Shore, Staten Island (South Beach, Midland Beach, New Dorp Beach, and Oakwood).
Map
s showing the location of these subareas are provided in the discussion of our
findin


, so findings on the distribution of properties by residency are based on the 615 households completing the survey. In contrast, data from the EC are required to project flood insurance premiums, and thus premium projections are based on the 485 prop


result from sea level rise (SLR). It also examines the effects of these rate increases on housing burden.
While Chapters Two and Three focus on the study area as whole, Chapter Four
explores how the results vary across the five study subareas.
Cha


are in
the high-risk zones of the 2007 FIRM (A, AE, AO, V, and VE zones) with the remain -
ing properties newly mapped into the high-risk zones by the PFIRM.3
A high percentage (84 percent) of the one- to four-family structures in the study
area w


Flood Insurance in New York City    9
tures are eligible for “pre-FIRM construction rates” from the NFIP, or “pre-FIRM”
rates for simplicity (FEMA, 2015a, p. RATE-2).4 Pre-FIRM rates are on average less
than actuarially based rates for the structur


eas. Because all 48,100 properties are represented,
confidence intervals are not necessary.
c Based on the study sample with complete surveys ( N = 615).
d Based on the study sample with ECs ( N = 485).Table 2.1—Continued10    The Cost and Affordabi


ally affect the rental market.
9 The BFE is the elevation water reaches in a flood that occurs with a 1-percent annual chance.
10 For structures with basements, structure elevation is measured from the top of the basement floor. For struc -
tures wi


edian income in our sample is $74,500, which is higher than the median income for all households in New York City ($53,782 in 2016 dollars) (U.S. Census Bureau, undated[a]). It should not be surprising that median income for New York City homeowners


vides income cutoffs for these income categories (which vary by house-hold size) in the New York, New York Metro Fair Market Housing Area.
Table 2.3 shows that just less than 20 percent of households in owner-occupied,
one- to four-family homes in


) 7,200 17% 5,800–8,900 14–21%
Higher income (> 165% of AMI) 10,000 23% 8,300–11,900 20–28%
Low income and below
(≤ 80% of AMI)16,800 39% 14,800–18,900 35–44%
NOTE: These values are based on a sample of 569 owner-occupied residences. Categories may


P
policies were accurately assigned to property parcels. They found a match rate of 88 percent, suggesting that the
estimate in that study may overstate the actual take-up rate.
13 Kousky (2016) finds that the majority of the increase in flood ins


hat take-up rate increases as
household income increases, particularly for 2016. The relatively large confidence inter -
vals, however, prevent any firm conclusions. We will return to the relation between take-up rate and housing burden (the last se


ximately 45 percent carry the maximum $100,000
offered by the NFIP.
The relation between building coverage and structure replacement cost is one
indicator of the extent to which property owners are protected from flood losses. The
NFIP policy appl


s in the study sample with a flood insurance policy as of December 31,
2014 (N = 775).RAND RR1776-2.3Percentage of policies
1–50 51–100 101–
150151–
200201–250251–500501–750751–1,000>1,000Building coverage
Structure replacement cost The Affordabili


    increased c
ost of compliance (ICC) premium +
    deductible
 adjustment +
    reserve fun
d assessment +
    federal poli
cy fee +
    HFIAA su
rcharge.
Specifically, the premium in the PMF does not include the reserve fund assess -
men


s much higher than the $1,120 average for the elevation-rated structures.
24 A surprising 10 percent of policies are PRPs—
surprising because PRPs are not available in high-risk areas. A likely explanation is that the flood zone was incorrectly state


inition back to the National Housing Act of 1937.
The regulations promulgated pursuant to this act originally offered public housing to families with incomes less
than five to six times the rent. This rule was quickly changed to limit maximum rent


ity Department of Finance. Careful efforts were made not to double-count property taxes and insurance payments escrowed in the mortgage. Consistent with common terminology in the lending industry, we refer to these costs collectively as PITI (mortgag


. Six households that report
annual household income of less than $20,000 and have a PITI ratio greater than 2.5 but less than 4.1 are not shown on this chart.RAND RR1776-2.4
PITI ratio
200 100 0 300PITI ratio = 0.4Cutoff for
middle-incomehousehold


lds with PITI ratios above 0.4 do not have flood insurance, and purchasing flood insurance would only increase their PITI ratio further above 0.4. Consequently, flood insurance is also burdensome for these households using this measure of housing bur


miums that are currently paid and the
extent to which they are financially burdensome. That analysis was predicated on the FIRM currently in place (the 2007 FIRM), the NFIP rate schedule in effect between July 1, 2015, and June 30, 2016, and the ava


ructure t
ype (e.g., slab, basement, enclosure, crawlspace, subgrade crawlspace)
• number of floors
• location of M
&E
• flood zone
• structure el
evation relative to BFE
• residency (
whether primary residence of property owner).
When a structure i


 annual flood insurance premium increases to 18 percent per year, so large premium increases would be phased in over multiple years. For exam -
ple, if premiums triple as they do in some of the scenarios below, it would take approxi -
mately seven ye


d:
•
 building co
verage: the lesser of replacement cost and $250,000 (the maximum
offered by the NFIP)
• contents co
verage: $100,000 for structures with $250,000 in building coverage
and 40 percent of the amount of building coverage for structure


s to $2,500 and is shown in Table 3.4. Including the other premium components increases the total premium to $2,762.
The bottom part of Table 3.4 shows the non -elevation-based premium for this
structure. The building and contents rates are shown i


tructure
Location of M&E In basement
Building coverage $250,000 with $2,000 deductible
Contents coverage $100,000 with $2,000 deductibleFlood Insurance Premium Scenarios    37
Somewhat unexpectedly, changes in the NFIP rate schedule between 2012 and


62
Non-elevation-based rate
Base premium $2,850 $1,203 $4,053
Increased cost of compliance premium — — $55
Deductible adjustmentc— — 0
Reserve fund assessmentb— — $616
Federal policy fee — — $45
HFIAA surcharge — — $25
Total $4,794
a –7.5% of base pr


paid in the high-risk areas of the 2007 FIRM is $1,880 (see Table 2.8 in Chapter Two), just more than one-half of the $3,500 average projected in scenario B. Primary reasons for this for gap are:
•
 The coverage le
vels observed for policies in force


PFIRM affects flood insurance premiums by increasing the number of struc -
tures in high-risk areas and by increasing, for most properties, the property-specific BFE. As shown in Table 3.5, there are 25,900 “newly mapped” properties under the PFIRM—t


ifference for one- to four-family structures in the study is area is –5.0 feet.
Premiums for Properties Already in High-Risk Zones
How premiums would change under the increasing risk depicted by the PFIRM depends importantly on whether the property


perience a premium increase and the average premium across properties increases by $500.
Eliminating grandfathering does have a considerable impact for properties that
are already in high-risk zones according to the 2007 FIRM. As can be seen by com


igure 3.7), and at least
75 percent of properties see increases of $1,800 or more (first column of Figure 3.8). The reason is that Congress directed FEMA to gradually increase rates from the PRP rate to the standard X-zone rate for newly mapped pro


grandfatheringDrop
grandfathering,
keep pre-FIRM
ratesAdopt PFIRM
with risk-based
ratesAdd 8 inches of SLR
06,000
5,0004,0003,000
2,0001,0007,000
RAND RR1776-3.8
Premium change ($)
5th
percentile$400 $0 $0 $400 $0
25th percentile$1,800 $0 $300


he
PITI ratio when the premiums in scenarios B and G are assumed. In both of these sce-narios, all properties are assumed to have flood coverage at the same coverage amounts assumed in this chapter. Moving to premium scenario B does not increase the


E.
Outside the high-risk zones of the 2007 FIRM, the median premium remains at
$500, with or without pre-FIRM rates. This assumes that property owners continue to qualify for PRP rates outside the high-risk area. Continued qualification for the PRP


y as the Jamaica Bay subarea)
• South Beach/
Midland Beach/New Dorp Beach/Oakwood (referred to jointly at
the East Shore subarea).
We oversampled in these subareas to get a better representation of vulnerable
communities in our study.1 This chapter


Jamaica Bay 2,800 2,600
Rockaway Peninsula 11,300 9,700
East Shore 6,300 5,800
Rest of study area 21,100 18,300
Overall study area 48,100 42,700
SOURCE: Property address provided by the New York City Department and Finance. Residency status
determi


s full of summer bungalows by the early 20th century. Over time, these bungalows were winterized and became year-round residences. These two communities are home to approximately 11,000 people and more than 9,000 residen -
tial units.
Flood Risk Prof


head Bay comprise detached (see Figure 4.8) and semidetached one- and two-family homes, and there are numerous multifamily buildings. Elevating or retrofitting
Figure 4.5
First Floor Below Street Level in Gerritsen Beach
SOURCE: Image used with perm


 Boulevard. These communities were expanded by dredging and filling in marshland (NYC Plan -
ning, 2016).
Flood Risk Profile
These communities have long experienced flooding during storm events and even regu -
lar tidal flooding. These communities e


,000 residents. In mid- to late 19th century, the Rockaways was primarily a waterfront resort area. The construction of the Cross Bay Bridge, the Marine Parkway Bridge, and improvements to railway services between the 1920s and 1940s made the Rockawa


sion of the New York City Department
of City Planning. All rights reserved.
RAND RR1776-4.1266   The Cost and Affordability of Flood Insurance in New York City
East Shore Subarea
Background
The East Shore of the Staten Island study area consisted pr


ity in Five Vulnerable Subareas    67
Take-Up Rates and Housing Burden
Flood insurance take-up rates vary considerably across the five study subareas.
Table 4.2 reports estimates of the take-up rate in 2012 and 2016 for one- to four-family
propert


 (≤80% AMI)Percentage Housing
Burdened
(PITI Ratio >0.40)Percentage Older
(Financially Responsible
Person ≥65 Years Old)
Study SubareaEstimated
Percentage95%
Confidence
IntervalEstimated
Percentage95%
Confidence
IntervalEstimated
Percent


wner-occupied residences in Canarsie. Reflecting the large premium increases in the Rockaway Penin-sula, the increase in the percentage of households that are housing burdened is particu -
larly large—increasing from 16 percent under current conditio


d, flood insurance premiums are particularly burdensome
for households in Canarsie. A substantially higher proportion of households in Canar -
sie are currently housing burdened than in the other subareas examined and a substan-tially higher proport


d be affected by shifting from the effective FIRM to the PFIRM can provide insight into how households and neighborhoods will be affected by whatever change ultimately occurs.
Another source of uncertainty is that the magnitude of price changes will


 study area is in the high-risk zones of the PFIRM). Many factors beyond floodplain location and flood insurance prices go into determin -
ing property values; for example, the 2008 subprime mortgage crisis had an enormous effect on housing prices. O


eas
5th percentile $235,000
25th percentile $307,000
50th percentile $407,000
75th percentile $516,000
95th percentile $850,000
Median home price by study subarea
Canarsie $390,000
Southern Brooklyn Waterfront $361,000
Jamaica Bay $400,000
Rockaway P


ecline in value of some property
in the current high-risk zone is so large that the property is almost worthless. This
3 The average mortgage is prepaid in fewer than 30 years, but the value of the property is based on its resale value
and not how


00
NOTES: Sample limited to one- to four-family properties in the PFIRM with complete ECs. There are no
homes from Canarsie in our sample with an EC that are inside the high-risk zones of the effective FIRM. Maximum decline in value is bounded at th


ome drops, the household no longer ben -
efits from spending the same amount of money to purchase a now less-valuable home from the lender. Because the value of the home has dropped, selling the home also does not raise sufficient revenue to pay back


4) and Gyourko and Tracy (2014).
In January 2016, the mortgage default rate in New York City was 1.0 per -
cent (Barraza, 2016), or just over 300 mortgage defaults per year among owner- occupied residences in the study area. Table 5.4 shows over 150


uch of an empirical difference.7 These inelastic
estimates are also based on much smaller changes in price than those considered here. If the probability that a $5,000 increase in insurance costs causes a household to drop its insurance is more than


n flood insurance prices will cause sig -
nificant decreases in property values. Table 5.2 shows that households in the study area but outside the high-risk zones of the effective FIRM will see the value of their property decrease by roughly $10,000


ed by the National Academies of Sci -
ences, Engineering, and Medicine on behalf of the Federal Insurance and Mitigation Administration (FIMA).
1 The first three programs subsidize flood insurance premiums
in different ways. The fourth makes flood i


is complete.
We do not examine the potential for incorporating CRS participation into a flood insurance affordability
strategy in this report. That topic warrants further study.Table 6.1
Flood Insurance Affordability Program Designs
Design Name Desc


 designs are also modeled using premium scenario E (PFIRM, 2015 rate schedule without pre-FIRM rates, but with grandfathering).
5.
 Design para
meters.  Each design is modeled under a set of base-case design
parameters for eligibility and benefit l


2  The Cost and Affordability of Flood Insurance in New York City
decline in benefits avoids a sudden change in benefits from 80 percent of the premium
to zero and the associated incentive to reduce income to just below the threshold. It also opens


o 0.80, but remains high.6 As shown in the last column of
Table 6.3, this design provides benefits to many households that are not housing bur -
dened and thus could afford the flood insurance premium without the program. More than one-half of the


households that are housing burdened without the program increases when full-risk rates are used (from 9,700 to 14,200). For scenarios B and E the program causes little change in the percentage of households that are housing burdened in the study are


,000 to 15,400), while the number of beneficiaries who are housing burdened declines by 600 (from 9,700 to 9,100). Even though the cost of the more narrowly targeted program is 18-percent lower than in the base case, the outcomes for the housing-burd


d by HUD’s Section 8 housing choice vouchers for renters. That program expects renters to pay 30 percent of household income toward rent and then fills in the gap between the household contribution and a measure of market rent in the area.
8 In the d


his design tends to reward less financially conservative households. A household that, for example, took on the maximum amount of debt when times were good would be more likely to benefit from the program than households that did not borrow up to the


ects the number of beneficiaries
and program cost. Higher flood insurance premiums increase a household’s PITI ratio,
and, as expected, the number of beneficiaries is higher when the program is run using premium scenarios based on the PFIRM. As is


00)15,500
(13,000–18,000)
Annual premium subsidy $14 million
(7–21)$19 million
(11–26)$26 million
(18–34)
Outcomes for households that are housing burdened without the program
Number of beneficiaries 6,900 9,700 9,700
Annual premium subsidy $14


e pre-
mium charged for the $2,000/$2,000 and $10,000/$10,000 policies. NFIP actuaries consulted during this study indicated that the deductible factors are based on actu -
arial data and that differences in the factors provide a first estimate of th


ere could be years where
no flooding events take place or years where multiple flooding events take place.
12 Because of this relationship between expected outlays and the difference in premiums, the annual cost of a
program in which the government


vious designs have provided financial assistance for flood insurance premiums
but have not made any effort to modify the structure to reduce risk. This design makes flood insurance more affordable by modifying structures in ways that reduce the NFIP


 approach is attractive because, rather than funding subsidy
payments year after year, mitigation reduces risk and the need for subsidies in the first place. Mitigation also means that households experience fewer and less-severe flood losses and the


ned, but rather the cost of the loans after repayments have been made. This cost is due to the below-market interest rates and loan default. The subsidy rate for the low-interest loans provided by the SBA provides a basis for estimating the cost of


levation is determined using the
floor above the crawlspace or enclosure. Retrofitting a residential structure with flood vents typically costs between $6,000 and $9,000, and we assume a cost of $8,000 in the program simulations (see Table 6.15).
R


$70,000 in our program simulations. An obvious downside of this mitigation measure is the loss in utility and potential rental income that basements generate for homeowners.
Structure Elevation
The last and most expensive mitigation measure consider


ber of structures for which
mitigation measure is feasible and household is eligible2,600
(1,500–3.600)7,200
(5,300–9,100)21,100
(18,300–
23,900)15,700
(13,100 –18,100)
Number of structures for which mitigation measure is cost-effective and


households in the study area that are housing burdened (given the rate schedule that underlies pre-mium scenario B).
A final point to underscore regarding the projections in Table 6.16 is that they pre-
sume all eligible structures participate in th


ion $94 million $330 million
Average premium without program $9,100 $4,600 $5,000 $26,500
Average premium with program$5,300 $3,900 $1,200 $4,400
Average PITI ratio without program 1.32 0.82 0.83 1.11
Average PITI ratio with program1.07 0.80 0.72 0.5


 row six.
Finally, row seven reports the difference in costs between the two designs. The
results show that government cost for design 5 will be lower than for design 1 under particular circumstances. Government cost is lower if the subsidy is provi


out
Program
DesignFlood Insurance
UnaffordableFlood Insurance
AffordableBenefit Cost
with Full
ParticipationaWithout
ProgramWith
Program Pros Cons
1. Income-based subsidy 9,700 22,000 $33 million per
year$2,100 $650 Least data required on
ho


rs the relatively small number of middle-income households that are housing burdened from the program.
The premium reductions and program costs for the deductible subsidy design are
more modest than in other programs. We have modeled a program in w


art of the program.Table 6.20
Number of Beneficiaries and Program Costs Under Premium Scenario B and Premium Scenario G
With Premium Scenario B With Premium Scenario G
DesignNumber of
Beneficiaries Program CostNumber of
Beneficiaries Program Cost
1


be largely phased out by the time a revised FIRM is adopted for New York City, but even without pre-FIRM rates, the median premium with grandfathering is not much higher than under the 2007 FIRM. It should be remembered, however, that not all homeow


 450 per year, or 1.5 percent of homes with mortgages per year. Most of these defaults will be in the high-risk zones of the effective FIRM. Some study areas, such as the Rockaway Penin -
sula, could be particularly hard hit by increased default rate


tentially lower than with the premium subsidy alone. We illustrate that this can indeed be the case with savings up to hun -
dreds of millions of certain multiyear scenarios, but only if the low-income households that qualify for the income-based sub


. to provide a reliable and representative estimate of the affordability of flood
insuranc
e for one- to four-family homes in New York City
2. to provide acc
urate estimates of flood insurance affordability for one- to four-
family homes in each of


me. Study subarea was not a factor in sampling in this stage, so the additional sample would be expected to be proportional to the representation of each study area among the sample frame structures. The final sample size was 2,800 structures with an


2015 and May 2016. Waves were
organized in part by geographic location.
To enable survey results to be projected to the study area as a whole, properties
eligible to participate in the study were randomly selected; consequently, we could not accept


rt staff to ensure accurate information was communicated in response to inquiries. Staff fluent in Spanish, Chinese, Russian, and other languages were available on call as needed.
During the course of the study, 345 phone-based inquiries were receive


60 study participants.
Site Visits and Elevation Certificates
Following completion of the questionnaire by study participants, the project- scheduling team contacted each participant by phone and/or email to schedule the site visit for the land surve


ic command. The outcome variable is an indicator variable for participation in the study (1 indicating yes), and the explanatory variables are also all indicator variables. As shown in Table A.4, the following variables are included in the response-r


e Coefficient Prob > |z| Coefficient Prob > |z|
Study subarea
Canarsie 1.102 0.56 1.122 0.54
Southern Brooklyn
Waterfront0.679 0.04 0.908 0.63
Jamaica Bay 1.097 0.56 1.070 0.71
Rockaway Peninsula Reference — Reference —
East Shore, Staten Island 0.7


y duration, creation of maps for use by door-to-door canvassers, the process used to match the study addresses to the NFIP policy database, and the process used to match the study addresses to the Build It Back database.
Geodatabase Development: Dat


ch
 –Canarsie
 – Rockaway Pe
ninsula including Rockaway Park and Rockaway Beach
 – Sheepshea
d Bay, Gerritsen Beach
 – South Beach, M
idland Beach, New Dorp Beach, Oakwood.
Source Data Manipulation
The following data-manipulation steps were performed


l.
For riverine areas, the BFEs are stored in the FIRM database as attributes to line
features that cross the floodplain perpendicular to stream flow. Using the elevation of the BFE lines as input and the zone AE flood hazard areas as a mask, a Tri


d. PINs were assigned for the first 300 addresses in wave A. PINs were assigned for sub -
sequent mailing waves as the mailing waves were identified (number of addresses and neighborhoods).
Prepare Data for Mailings and Telephone Outreach
The study d


e addresses were geocoded and their points were moved to locations that better represented real-world locations.
To improve the display of tabular information in AGOL, only selected fields were
kept in the study geodatabase that was prepared for AG


aded into the tool. A starting and ending location for each route was chosen, and an ordered route and directions were output. Once the route was created, map sheets at a size and scale that could be printed and carried in the field were gener -
ated


au on the relationship between household income and household net worth. These data are based on statistics for households nationwide, but they provide an initial sense of how adding an asset test to an income-based eligibility criterion could reduce


3 15.6 33.8
NOTE: Data on household net worth by income quintile from U.S. Census Bureau, Survey of Income
and Program Participation (SIPP), 2008 Panel, Wave 10 (U.S. Census Bureau, undated[c]). Net worth is defined as total assets less total liabil


a change in jobs has reduced household income. It could also be the case that workers in the household have retired. The extent to which remaining costs are unsustainable to a household depends on their level of accumulated assets. The role of assets


auses a substantial movement of households to the higher-income share categories.
Relationship Between Building Coverage, Contents Coverage,
Replacement Cost, and Mortgage Balance
The top section of Table D.3 examines the relationship between build


h NFIP policies.
1 Mortgage balance is reported by survey respondents.
Properties with building coverage are divided into three groups:
1. those with a mortgage and mortgage balance less than or equal to replacement cost
2. those with a mortgage and


or building coverage.159APPENDIX E
Flood Insurance Premium Model
This appendix first describes the model that was developed to project flood insurance
premiums under a number of assumptions about the FIRM in effect and the availabil -
ity of pre-FIR


r
Elevation-based rates in VE zones
NFIP rate table 2012 and 2015
Flood zone VE
Structure type Elevated with enclosure, elevated without enclosure, all other
Elevation difference +4 to –13 in feet
PRP
NFIP rate table 2012 and 2015
Flood zone Xa
Stru


 2012 and $45
                    in 2015 for other policies
HFIAA charge = $0 in 2012; $25 in 2015 for primary residences; $250 in 2015
                           for nonprimary residences.
This premium information is used to develop the following


truc-
   ture and PR
3 is lower than the rate after Step 3 (allows for pre-
   FIRM rates fo
r grandfathered structures in high-risk zones of
   2007 FIRM and se
ts rates for newly mapped pre-FIRM prop
   erties to sta
ndard X-zone rate).3
Step


schedule used in the premium model went into effect. Rates generally increase over time, and one would expect the model to overpredict premiums for policies that Table E.3
Comparison of Model Predictions with Premiums in the 2012 Policy Master File (


se with coverage).Table E.4
Comparison of Flood Zone
Flood Zone According to
2016 Policy Master FileFlood Zone According to EC Completed for Study
A or AE V or VE B, C, and X Total
A or AE 154 0 3 157
V or VE 0 1 0 1
B, C, X 18 1 114 133
Total 172 2


e, but it is somewhat more common for structure to be declared pre-FIRM in the PMF when it is post-FIRM on New York City’s PLUTO property database than the reverse. Nineteen percent of post-FIRM properties are identified as pre-FIRM in the PMF (11 of


s and 18%  per property.If there is no lender requirement, the lapse rule does not apply.
SOURCE: FEMA, 2016a, Section 5 (Rating).170    The Cost and Affordability of Flood Insurance in New York City
Table F.2
Rating Options Available When the Flood


r V can be canceled and rewritten to a PRP  going back to the inception date of the current term (the term the map change took place in). If the building does not qualify for a  PRP (based on loss-
history requirements
b),
the policy can be endors


e pay-ment associated with taking a loan worth 80 percent of the 2016 assessed property value at a 5-percent interest rate. For the two remaining primary residences that have mortgages but are still missing mortgage payments, we impute mortgage payme


tional insurance expenditures.
2
Utilities
In addition to the costs associated with PITI, we collect information on utility expen -
diture as another component of housing cost. We ask homeowners to report their prior month’s electric and gas bills, a


ms unlikely based on current take-up rates, but would be socially desir -
able because it reduces the risk of households facing sudden unexpected loses. How -
ever, such a change in practice would also mean an increase in insurance prices causes a la


ults.Economic Effects Model   177
Table G.1
Decline in Property Value Because of Change in Flood Insurance Premium from Scenario B to Scenario G, Flood Insurance Purchased Indefinitely (2016 Dollars)
Properties in
the High-Risk Zones
of the Effecti


 RR1776-H.1-2182    The Cost and Affordability of Flood Insurance in New York City
Figure H.1—Continued
RAND RR1776-H.1-3Survey Instrument    183
Figure H.1—Continued
RAND RR1776-H.1-4184    The Cost and Affordability of Flood Insurance in New York C


a Monica, Calif.: RAND
Corporation, RR-328-NYC, 2013. As of March 15, 2017: http://www.rand.org/pubs/research_reports/RR328.html192    The Cost and Affordability of Flood Insurance in New York City
Dixon, Lloyd, Noreen Clancy, Seth Seabury, and Adr


S. Department of Housing and Urban Development.
“Income Limits,” Office of Policy Development and Research (PD&R), U.S. Department of Housing and Urban Development, undated. As of March 29, 2017: https://www.huduser.gov/portal/datasets/il.html#2015Re


orhoods/gerritsen-beach.page
———, “Resilient Neighborhoods,” nyc.gov, undated(d). As of March 20, 2017:
http://www1.nyc.gov/site/planning/plans/resilient-neighborhoods.page
———, “Rockaway Park and Rockaway Beach,” nyc.gov, undated(e). As of March 22


ualifying Ratios (4155.1),” hud.gov website, March 1, 2011b. As of
March 20, 2017: https://portal.hud.gov/hudportal/documents/huddoc?id=4155-1_4_secF.pdf
———, FHA Single Family Housing Policy Handbook 4000.1 , Washington, D.C., December 30,
2016. A


ns that reduce ﬂood
insurance premiums by making the home less susceptible to ﬂood risk. This report builds
on a previous work by the RAND Corporation on ﬂood insurance in New York City, Flood
Insurance in New York City Following Hurricane Sandy


SOCIAL AND ECONOMIC WELL-BEING
Evaluation of the Homeless
Multidisciplinary Street Team for the City of Santa Monica
J. Scott Ashwood, Karishma Patel, David Kravitz, David M. Adamson,
M. Audrey Burnam
Prepared for the City of Santa MonicaLimited P


cerns that are influenced by the
natural and built environment, technology, and co mmunity organizations and institutions that
affect well-being. For more information, email chep@rand.org.
  iv Contents
Preface ...................................


eral population to
seek services to address these conditions on their own. For thes e reasons, chronically homeless
individuals are often repeat us ers of emergency services—including medical, law-enforcement,
and paramedic-response services. Th i


, to date only one client has formally “graduated” and transitioned to step-down care and is no longer receiving services from the HMST. Transitioning clients to step-down care is currently the program’s greatest challenge.
How do community stakeho


 recommend that the HMST provide more in formation on the impact of their efforts to
the other stakeholders in the community. So me individuals we interviewed said they
would like to know the outcomes of encounters with the clients.
 We recommen


asons, chronically homel ess individuals are often repeat users of
emergency services—including medical, law-enfo rcement, and paramedic-response services.
This pattern of service use is expensive for ci ties both in dollars and manpower. A recent


nity and
organizations within it, and recommendations for improvement based on our interviews.
The goal of the HMST is to reduce the public co st of services by the highest utilizers of
emergency services among the homeless population in the City


sion in 2016 based on input from the city attorney’s office, the police
department, and the fire department, and using ad ded consideration for data from the emergency
rooms of local hospitals. Future additions to the HMST roster are made in a simi


gs he was willing to work on kind of snowballed.”1
In the early stages, HMST focuses on small, da ily tasks that a client wants done. Then, as
smaller goals are accomplished, the case manage r focuses on bigger goals, such as access to
general rel


w, June 25, 2018. 6 someone who can diagnose mental illness and develop meaningful treatment plans.2 However,
while there are advantages to the HMST’s abilit ies in the field, there is only so much that
specialists can do “with a backpack on.”3 Lik


2 Interview with city administrator, August 30, 2018.
3 HMST interview, June 25, 2018.
4 HMST interview, June 25, 2018.
5 Interview with Santa Monica Police Depa rtment representatives, June 27, 2018.
6 HMST interview, June 25, 2018. 7 Client g


 working with or  responding to calls involving
homeless individuals.
Clients of the HMST say that going beyond trad itional case management has contributed to
the team’s success but has also built high expe ctations. Many clients rely on the team


12 months following. We received Santa Monica
Police Department (SMPD) encounter data cove ring both the pre-period and post-period for the
clients and comparison groups. The HMST clients had a much higher average number of police
encounters (22.1


 Experienced Fewer Encounters with First Responders and Emergency
Departments Than Those in Comparable Ho meless Population 6 to 18 Months After
Engagement
Table 2 summarizes the change in outcomes between the year prior to the HMST and the
year


isit $1,3254
NOTE: The original estimates from Wu & Stevens are in  2014 dollars. We have converted their estimates to 2016
dollars.
1 This is the per-arrest cost.
2 This is the per-arrest cost plus the cost  of one day of jail in a male facilit


verag e annual spending of
$500,000. When we sum the total savings per client in the first 12 m onths and in months 6 to 18,
we estimate annual savings of $103,000 in the fi rst year and $259,000 between months 6 and 18.
This represents an offset


erve, such
as other factors that may be changing during th e same time period or random fluctuation in
encounter rates. We do have police department encounter data for a comparison population of chronically homeless individuals, but the comparison


ck outcomes beyond those within the na rrow focus of this evaluation. Our small
sample size limited our ability to conduct more  rigorous statistical analyses of outcomes
as well as our ability to tell a more comple te story of client experience. E


 contribute to the primary progr am goals that motivate the intervention.
For the HMST, the major inputs are the select ion of the cohort that  ensures the highest
utilizers are being served; the human, logistical, and financia l resources that mak


nesses
- Ways to Improve
Development of Individual Treatment Plans How do you know HMST to enroll? (if familiar)
- Strengths - Weaknesses
- Ways to Improve
Evaluation of Clients and
Provision of Services  How do you know HMST to evaluate servic


zing themes included how the HMST works, the visibility of the intervention, the
effectiveness of the intervention, and recommen dations. In addition to key themes, we noted
recommendations by the category of interviewee (e.g., staff member, client


to use name, middle name, a nd date of birth to match records. The SMFD
data were in an electronic fo rmat but required significant mani pulation before they could be
added to the database. Table A3 summarizes the data we received. Data from the HM


th the pre- and post-peri ods in our evaluation. 26 The patterns in the figures reflect the patterns described in the main text above. There is a
decline in encounters over time fo r most of the HMST clients and little change in encounters for
the


t that there are few clients of the
HMST. The small sample size prevented us from controlling for additional predictors in our
models, such as individual dem ographics or individual measures of health or behavior. We did
not have the statistical p


ng return
on investment over time, we are able to  capture this aspect of the HMST.  30 References
City of Santa Monica, City Council Report from Public He aring of the Annual Review of the
City’s Plan for Homeless Services , Santa Monica, Calif.:


CORPORATION
Digital Resources for
STEM Educators and Recommendations for Cyberlearning Initiatives
Results from the National Science Foundation
Digital Library/Distributed Learning Program Evaluation
Tora K. Bikson, Susan G. Straus, Grace Agnew, Ni


ernally. Findings were made available to the RAND Corporation researchers who conducted the second and third phases of the evaluation.
The Phase 2 evaluation provided a preliminary assessment of the program, based largely on
existing information r


nment, a division of the
RAND Corporation dedicated to improving policy- and decisionmaking in a wide range of policy domains, including civil and criminal justice, infrastructure protection and homeland security, transportation and energy policy, a


gineering, and mathematics (STEM) education at all levels” (NSF, 2010). By default, when we use the term NSDL in this report, we are referring to the program. The NSDL portal refers specifically to the website NSDL.org, which is a starting point for


uture or similar enterprises. Lessons learned about the sustainability of digital initiatives for STEM education are relevant to a range of stakeholders, including NSF, the broader NSDL community of researchers and developers,
xiii
 STEM educators,


development and application of a framework for evaluating the health and sustainability of digital initiatives, a review of tools and services funded through the NSDL program, usability studies, and case studies of three full instantiations of the NS


es the service, and the customer, who often funds or pays for it; small enterprises often focus their attention on the client when the customer is more critical to their success (Rothschild, 2012). Communication with the target audience can be facili


ent institution to document the project’s contribution to advancing the parent institution’s mission. Another sustainability strategy is to look for natural alignments of the initiative across departments in the institution, such as the library, depa


d to enable users to display,
copy, download, or modify resources, as well as whether this information has been conveyed clearly. Ensuring an effective rights platform involves not only acknowledging and documenting the copyright of site content, di


e, could be viewed as sustainability strengths for NSDL. However, improvements were needed in integrated security beyond basic user identification and password protection and in support for users in terms of training and personalized services.
The N


a few key tools and services for further analysis. It should be noted that our analysis of NSDL services was limited; therefore, many of the conclusions discussed here should be considered tentative.
We found a diverse range of themes in the techno


earch service in comparison with a general search engine (Google) when seeking representative STEM content. In the second, we designed a usability experiment with preservice teachers and library science students to assess the relative effectiveness a


ess, is likely to have a positive effect on the uptake of NSDL. Adding graphical displays to NSDL may also influence use of search results.
Lessons from Case Studies
We evaluated three full instantiations of the NSDL logic model using a replicated


vice or preservice
science teachers).
All three cases show ed evidence of success. In all cases, teachers were enthusiastic about
the benefits of access to vetted, digital resources that are aligned with standards. They report ed
that the resourc


and persistent user-developer communities and partners. The communities and partners promote sustainability through their demand for resources and services and through the social capital they generate as they contribute back to the digital initiativ


 resources by vetting collections; on the other hand, this strategy excluded a good deal of dynamic and interesting content on the w eb, such as YouTube channels, Twitter feeds from
prominent scientists, and open-source online textbooks. Likewise, w


and innovate. In later stages, projects that have produced promising results and have designed plans for sustainability can be funded under contract or cooperative agreement to implement such plans and transition their products to other sponsors. Li


rs)
and students to evaluate program goals, NSF opted not to seek approval for data collection as
required by the Paperwork Reduction Act of 1995 (Public Law 104-13), which mandates approval for systematic data collection from ten or more persons u


ey themes and topics in these sources of secondary data.
• Finally, another critical recommendation is that cyberlearning initiatives, particularly
multiyear, collaborative initiatives such as NSDL, need a much greater focus on
sustainability from


the Phase 1 evaluation of National Science Digital Library/Distributed Learning at GOH in 2008. He subsequently served as a consultant to GOH for the Phase 2 evaluation and as a consultant to GOH and RAND for the Phase 3 evaluation. His principal ef


dvisory Board
TD-ESC  Teacher Domain Educational Standards Correlation
TNS  Technical Network Services
UCAR University Center for Atmospheric Research
UNESCO United Nations Educational, Scientific and Cultural Organization
URL uniform resource


vices or tools, such as search
capabilities, website authoring, and blogging applications, to support users, collection providers, and the Core Integration project to enhance the usability, usefulness, and value of the digital library
Targeted Rese


ive evaluation of NSDL, based on the assumption that the program would continue for several years. However, because the program was discontinued, NSF requested a shift in focus of

1 As noted in Bikson


brant community of developers and users who can create innovative content and tools and improve them on an ongoing basis.
The results of the Phase 3 evaluation of NSDL are intended to benefit the Directorate of
Education and Human Resources within


y funded contracts; NSF opted not to seek such approval, which greatly limited the extent and nature of primary data that we could collect to address program outcomes and impacts. The evaluation team also was not provided with annual progress reports


ions sought to develop, organize, and manage sets of digital content related to STEM learning within a coherent theme (e.g., subject area or grade level) (this track was discontinued in 2004).
• Pathways, which replaced the Collections track in 2004


 collaboration and no common elements of look and feel across pathways. These findings suggest that NSDL.org did not appear to be substantially greater than the sum of its parts.
Evidence of User Activities to Find and Access Resources
The NSDL po


ecause the NSDL portal and pathways are tightly associated with the collections they represent, we assess ed them jointly. NSDL.org and pathways are expected to
help users find their way to appropriate collections and to locate resources of interest


regarding the state of health of the collections. Therefore, in Phase 3, we focused more narrowly on dimensions of collections noted in the Phase 2 evaluation related to authoritativeness (e.g., evidence of peer review and professional oversight), c


ter Six ), and case
studies (Chapter Seven). Chapter Eight summarizes our results and presents recommendations.
The chapters are not necessarily stand-alone—i.e., understanding the content of some of the
chapters requires a general understanding


gital content (Griffin, 1998; Zia, 2008). DLI ran from 1994 through 1998; based on its impressive achievements—which included the original development of Google’s search technology—DLI -2 began in the spring of 1998
(Griffin, 2005). DLI-2 enjoyed su


geted Research. (Note that we use the term stage to refer to periods of development of the NSDL program, whereas we use the term phase to refer to the evaluation of the NSDL program.)
Table 2.1. NSDL Program Tracks , Stage 1 (2000– 2003)
Name  Des


s a basic metadata repository, and collections were intended to aggregate a very large number of online resources (Howe et al., 2011).
Table 2.2. New NSDL Program Tracks , Stage 2 (2004– 2008)
Name  Description and Relationshi ps
Pathways  Pathwa


program (see Table 2.3 for details).

 16 Table 2.3. NSDL Pathways Projects
FYs Funded
Pathway Name  URL Description  Pathways I  Pathways II
2004– 2007 2008– 2011  NSDL Applied Math and Science: Community College
(AMSER)  amser.org  All dis


o -cost extensions or rece ived supplementary support.
18 As these changes were taking place within the Services and Pathways tracks, the CI project
was overhauling the technical infrastructure of NSDL.org in ways that would support the needs
of


Similarly, the Pathways II grants were much smaller than the original awards and required the projects to implement a sustainability plan. The final award to UCAR in 2011 (see Table A.1) was intended to support the reorganization and business plannin


ding levels rather than
project counts, as shown in Figure 2.2. In comparison to Collections, far fewer Pathway projects were funded, although the level of funding for Pathways projects by the end of the program (approximately $43.4M) approached tha


 NSDL program solicitation thus influenced the projects that
were supported⎯but the changes took many years to substantially affect the program’s portfolio
and transform the NSDL community.

0 10 20 30 40 50 60 70 80
00 01 02 03 04 05 06 07 08 0


op a framework to delineate the elements of sustainability for digital initiatives that develop collections or curate collections assembled from other sources. In this chapter, we review and synthesize the relevant literature to develop a comprehensi


s must be able to attract customers in a competitive marketplace where they have many options, even in the nonprofit space.
Finally, we have included findings from the learning object repository and open educational
resources (OER) literatures. OE


 and the organizations that manage them, we will discuss search services that
are specific to the collections, since these affect the usability, and hence the sustainability, of the collections.
However, the broader range of digital library tools an


lection contributors, and parent organizations, who have varied interests in the collection and its success. However, a more nuanced view of the organization’s audience is critical when considering the sustainability of a digital collection initiativ


we et al., 2011, p. 1), and “NSDL’s ability innovate and achieve educational impact is depend[e]nt upon a diverse ecosystem of large and small entities whose interleaved efforts produce, consume, process, aggregate, reorganize, embellish and tinker w


ng success.
An initiative that functions in this way has come to be known in the literature as a community
of practice. Etienne and Beverly Wenger-Trayner defined a community of practice as “people who engage in a process of collective learning in


ders, and users. The idea that the “user knows best” also may not work well with emerging technologies, such as digital libraries, where users may not conceive of ways that technology can dramatically improve and transform their work processes.
Acc


development policy and safeguards for protecting copyright of resources (particularly for repositories that encourage user contributions) are other important policies that are relevant for accountability to the audience and are related to the usabil


ntified user needs. Different expectations for technical infrastructure among developers and users can pose a significant risk to the sustainability of grant-funded projects. Grants are usually awarded for innovative, groundbreaking technologies, whe


ices. Webster (1992) described the focus of marketing as relationship management. Madhusudhan (2008) echoed this view and delineated a number of marketing activities, including customer relationships, branding and corporate identity, marketing commun


profit	organizations.	Ongoing	review	of	the	strategic	plan	is	required	to	ensure	that	it	is	flexible	to	changing	circumstances.	The	organization ’s	stakeholders	and
audience	should	be	encouraged	to	provide	ongoing	feedback	about	the	strategic	plan	a


-defined audience, strong leadership, and measurable goals, it is difficult, if
not impossible, to obtain funding for digital initiatives. Even if projects demonstrate these attributes of sustainability, funding to sustain digital collection initiat


zed. Rothschild (2012) offered a similar approach in which organizations develop innovative resources that can be sold for profit, supporting the nonprofit arm of the enterprise. Minnesota Public Radio, for example, has many successful product spin-o


es for cost-savings and factoring them into a business plan, however, can be complex (Bishoff and Allen, 2004).
Collections
We end our sustainability literature review with a discussion of the health of collections. A
growing body of literature (


es that do not support the mission, do not conform to collection scope, or are not intended for the primary and secondary audiences can often be eliminated. However, elimination can be interpreted broadly to mean finding a new home for the resources,


le for an extended time? Although all usability factors are important, impact is
the primary criterion for weeding resources. Resources that can demonstrate high impact will help the portal be competitive for ongoing funding and stakeholder support,


building trust in the repository by its user base. However, an
essential attribute of usability is responsiveness to a user need. In the e-learning space, usability requires resources to be appropriate for the intended learning task (Ardito et al.,


ver, assessing quality in a federated metadata repository, such as the NSDL portal, has an added layer of complexity. In their study of metadata in federated repositories, Shreeves et al. (2005) noted that metadata may be of high quality within a loc


el. Creators of resources own the rights to the
further exploitation and use of the works of their creation, unless they transfer those rights to another person or organization. Copyright confers the right to control use of the resource, such as cop


sources, and metadata but also  documenting the licensing
that enables permitted uses of the resource by the audience. A sustainable repository will invest considerable time and effort in a rights management strategy that enables the fullest,
43 b


ous chapter, we reviewed the sustainability literatures on digital collections,
learning object repositories, and nonprofit businesses to develop a general model for assessing
the health and sustainability of digital initiatives (see Figure 3.1). T


ther ways to assess attributes of the model.
Table 4. 1. Sustainability Model Concepts and Attributes for Digital Library Initiatives
Audience  Organization  Collections
• Identification
− Mission, purpose, and value
proposition
− Clients an


 services of the portal, since these
should be aligned to the needs of the audience. Prominently advertising the identified audience
also tells potential users that the portal has collections and services suited to their particular needs. Questions


an help you engage student interest, shorten lesson preparation time, provide concept updates, and develop curricula that are in line with national standards for content, use of animals and humans, and student safety.”
The NSDL portal and all the p


Resource organization  Does the site provide private workspaces, where users can organize and
save resources in personal folders and share resources with others?
Audience participation  Does the site solicit or encourage participation through s


igned and implemented separately from the primary portal, such as the Middle School Portal, SMART (Science and Mathematics Accessible Resource Tool), which provides resources created by students for other students that are separate from the Middle Sc


data consist of “dynamic information about digital learning objects that is generated as they are used, reused, adapted, contextualized, favorited, tweeted, retweeted, shared, and all the other social media style ways in which educational users inter


ent been conducted, as indicated by pop -up surveys
when the portal or pathway site was visited, or information about  staff dedicated
to evaluation of the portal?
External assessment  Has an external assessment been conducted by an independent g


 pathways have made progress in identifying target
audiences, although there are opportunities for improvement:
• While NSDL.org and the pathways identify their audiences, in many cases, the audience
is very broad, such as K–12 or K–gray, and we f


ithin digital initiatives. This research could demonstrate how paradata influence or reflect user workflows and could validate the usefulness of different types of paradata to support teachers’ decisions about which resources to select and use.
Wit


 used several measures to assess evidence of sustainable organizations on projects’
websites. In particular, we looked for the staffing roles (described in Chapter Three), as shown in Table 4.9.
Table 4. 9. Questions for Leadership and Strategic St


 was limited information we could obtain about other aspects of organization, such as the
role of the parent organization and opportunities for alignment of the initiative across departments in the parent institution. Likewise, without access to the


e educational outreach missions of their parent
institutions. One example is SMILE, which is a portal for informal education, reflecting the key mission of its lead organization, the University of California at Berkeley’s Lawrence Hall of Science an


keholder confidence in the sustainability of the initiative. If not, and if the projects seek to become sustainable (either as independent start-ups or in coordination with their host institutions), our literature review provides guidance about how t


s in the collection been selected by a peer -review process
or other professional oversight , with selection (accessioning) according to
a documented process and removal (deaccessioning) when they are no
longer viable?
Availability (or attrition


g that resources were available for ten metadata records in each pathway. For this analysis, we conducted ten portal-appropriate searches and selected the first resource appearing in the results list. As Table 4.14 demonstrates, the low attrition rat


ilability was similar across both assessment periods .
Table 4.1 6. Search Strategies Supported
 Number of
Simple Search es Number of
Advanced Search es Number of
Brows ing Options
NSDL .org Phase 2  1 1 1
NSDL .org Phase 3  1 1 1
Pathways


educator workflows. NSDL has made these advances largely through the development of the Learning Activity Readiness (LAR) metadata schema (see box). The NSDL rubric for assessing metadata for LAR compliance includes the following elements: title, UR


a tional
standards?

We examined ten metadata records from each pathway and these same 180 records in NSDL
(using the same records assessed for availability described earlier). We rated each of these criteria (accuracy, completeness, and consiste


e value of NSDL as a one-stop shop. In particular, NSDL often provides composite records consisting of reviews and suggestions for use from annotation metadata, along with aggregated use from paradata, such as number of views and comments. This is on


 ChemEdDL (Chemical Education Digital Library) resources that are not available at the ChemEdDL portal.

67 • The NSDL portal leaders, working in conjunction with several pathways partners, have
compiled a growing collection of math Common Core r


ce for users about the rights associated with a resource (copyright metadata) and the uses they are permitted to make of a resource (permitted-use metadata).
Table 4.2 2. Rights and Permitted -Use Policies and Metadata
 Copyright and
Use Polic i


provide accurate and complete metadata that are consistent across records. Perhaps the most-important metadata improvements, however, consist of greatly enriched descriptions of resources, such as the alignment of NSDL materials to educational stand


ortal, presumably merely a gateway to the collections of pathways and the third-party resources selected by the NSDL RC, has a significant investment in its science literacy maps, professional development resources, and the policy and guidelines reso


le NSDL and most portals have an identified
audience, the audience identification is generally very broad. Although there is considerable improvement in metadata and metadata displays, there is no evidence that these changes are being made in collab


el as our organizing perspective. Tools and services are important attributes of sustainability because they support use of digital collections, and NSDL projects were expected to generate an array of diverse tools and services to discover, organize,


ysis of the development of and user communities associated with them (selection criteria for the key tools and services are described below). We asked two sets of questions related to the logic model, the first focusing on the products and outputs of


lection Services  - - - - 0 3 0 2 1 2 4 12
Integrated Services  - - - - 0 0 1 3 0 0 0 4
Usage Development  - - - - 4 4 1 1 0 7 5 22
Other  9 13 12 11 11 4 2 4 3 6 5 80
TOTAL 9 13 12 11 15 11 4 10 4 15 14 118
NOTE: “ Other ” refers to all Service


 simulations.
StandardConnection —
Mapping NSDL Educational
Objects to Content Standards (0121543)  The investigators are developing a natural language processing tool
(“StandardConnection ”) for the automatic assignment of content standards and


itutional partners.
Pedagogic Services for
Digital Libraries (0532768)  Building on a successful example, the Starting Point Digital Library connects pedagogic
materials to teaching materials in digital library collections. It is helping teachers


 courses.
Our review of abstracts also showed that some projects proposed redundant services.
For example, ten distinct award abstracts from 2000 to 2010 mentioned commenting tools, while more than 20 abstracts mentioned annotating resources.
Alt


evidence of whether the projects were completed,
partially completed, not completed, or of unknown completion status. Measurement of the relationship of services to NSDL captured two features of NSDL collaborative development: We classified projects


research. Finally, we found no evidence of tool or service development, or research products, for the remaining four projects.
Relationship of Project s to NSDL  Program
The columns in Table 5.3 show the sample of 22 services in terms of their rel


originally proposed to develop.
• Other projects apparently never committed to developing NSDL-ready services, although their products may have led to new ideas, new projects, or new services, some of which stayed within the NSDL community and some


ecific instantiation of the latter.

Although the services differed substantially in their purposes, user communities, and
longevity, we established a standard template to capture the common features of the services that
worked relatively well.


xpert Voices, for example, had many thousands of page views per month and hundreds of persistent users. IA has more than 7,500 registered users and has produced more than 17,000 projects. Nevertheless, of the services discussed here, only the Strand