in an educational program at some point while they were enrolled in CalWORKs. About half of the clients that were new to the CalWORKs program reported either being enrolled in an educational program or being interested in continuing their education.


for clients, noting that, “Students can dedicate more time to their schooling and not have to be burdened with the 20 hours of work a week on top of that.” At the same time, some community college staff voiced concern that only clients who specifica


es that can limit access to child care services. Staff from three of the six counties indicated that communication with CalWORKs clients in regard to child care could be improved, particularly when they are at risk of losing their child care slots be


h limited client ability to access supportive services and secure and maintain employment. Staff from counties that include rural areas were more likely to share this concern, although one of the urban focal counties emphasized that approving and fac


unties did staff report a strong domestic violence service system, with on-site clinicians that are equipped to work quickly with clients. Staff described their services as a “robust domestic violence program that assists our WTW population,” includi


the OCAT and the FS program has resulted in increased communication with external service providers, with more focus on tracking clients’ progress.
144
Learning Disability Services
If clients screen positive for a learning disability, they are referr


high school diploma to start with. So we are creating these very long-term plans at this point which is pushing for them to get work experience far into the future beyond the time limits of the CalWORKs program, so that has also been a concern of th


hanges so that counties can build on the programs in place. Staff in two counties reported that they struggled with the way in which SB 1041 was rolled out to the counties (a theme that also emerged in our 2015 site visits). The administrator in one


g and doing all that stuff, could better be used in focusing on the families and providing one-on-one services with them.” A caseworker from a different county said, “We spend so much time on these things that we forget why they are here. We don’t ha


he WPR.” An employment service provider from the same county also emphasized the importance of supporting the clients, stating, “[T]hey [the workers] need to let them [the clients] know that they believe in them and that they can make a difference .


ve easily said, “I would rather do the CalWORKs side of the clock, not the feds, I don’t want to do any core hours.” So our WPR would be zero, conceivably. And you can’t fault the counties for that, because we’re just doing what you kind of gave us a


so much. . . . When people have a place to live that’s stable and affordable, the rest can fall into line.”
Improve referral processes and coordination with service providers. Some service providers reported little to no increases in the number of re


tus, and
earnings) might be associated with the passage
of SB 1041, rather than other factors, such as the changing economy. We continue to consider
  Chapter Six Methods
• We analyze state administrative data from WDTIP, MEDS, and EDD wage data.
• A


TW clients were exempt from participating in WTW requirements? The percentage of clients who received at least one exemption during their first two years in the CalWORKs WTW program increased slightly from the February 2007 entry cohort to the Februa


employed in February 2016 than in February 2013 (26 percent in 2013 and 34 percent in 2016).
• How much did WTW clients who were employed earn? For the February 2016 cross- section, employed clients earned about $2,533 per quarter on average, a 17-pe


6.1 identifies which of the two studies (the status study, the tracking study, or both) addressed each research question. In general, the status study addressed questions about WTW clients at a point in time, and the tracking study addressed pattern


 2015 2016 0 Year
Unemployment rate (percent)
  CalWORKs WTW clients (left axis)
Unemployment rate (right axis)
SB 1041
                   SOURCES: WDTIP, 2006 to 2016; EDD unemployment data.
NOTES: CalWORKs WTW client status is measured in February


lients declined by 20 percent. (See the last four periods in Figure 6.1.) The 2013 status cross-section had 338,565 clients, compared with 337,388 for the 2014 status cross-section, 308,926 for the 2015 status cross-section, and 272,013 for the 2016


n exhausted, clients are subject to the stricter work requirements of the CalWORKs federal standards, which are similar, but not identical, to TANF requirements. If, however, the client meets the more stringent CalWORKs federal requirements, months o


most months counted on the 48-month time-on-aid clock (nearly 13 out of 36 months counted on average).
In addition, we examined how many months were counted on the WTW 24-month time clock for the 2013 and 2014 entry cohorts (the post–SB 1041 cohorts)


of ticks on the 24-month time clock for the 2013 and 2014 entry cohorts are presented in Figures E.1 and E.2 in Appendix E.
 160
Figure 6.6. Average Number of Months as a CalWORKs WTW Client, Counted on the 48-Month Time-on-Aid Clock, and Counted on


transitional group had at least one of the WTW 24-month time clock exemptions in 2013, which increased to 12 percent in 2014, and 14 percent in 2015 and 2016; less than 1 percent of the post–SB 1041 group had this type of exemption in 2013, which in


ry cohorts yielded similar findings as with status cross- sectional snapshots and adds a pre-policy trend. For each tracked entry cohort, the percentage of
163
clients who received at least one exemption within two years was substantially greater tha


nts Were No Longer CalWORKs WTW Clients One, Two, and Three Years After Entry?
This section describes clients’ exits from the CalWORKs WTW program. Leaving the CalWORKs WTW program is a fluid concept because clients may leave the program for a period


ployment and earnings. This section offers an initial examination of employment patterns because only about three years had elapsed since the policy change at the time of the analysis in this report.34
By several measures, CalWORKs WTW clients increa


Status Cross-Sections and Transitional and Post–SB 1041 Groups in February 2013, February 2014, February 2015, February 2016
         Status cohort
2013
2014
2015
2016
26 26 28
30 30 29
33 33 33
33
34
33
0 20 40 60 80 100
Percentage of clients
All cl


ly employed during their third year after entering CalWORKs WTW compared with their second year, with the exception of the February 2007 entry cohort. Among the February 2007 entry cohort, a little more than 22 percent were continuously employed duri


ffect.37 We isolated the group of clients
37 Clients with the short-term WTW exemption for young children refers to those who were exempt pursuant to AB X4 4 and were subject to reengagement (see Appendix A for further discussion). This short-term ex


rticipation in the Cal-Learn program. It also included a small number of exceptions such as penalty, good cause, extender, and repayment. The category “not a CalWORKs WTW client” was measured as a client not present in the WDTIP data system in Februa


families that are potentially eligible for the TANF program in California (i.e., potentially eligible for CalWORKs WTW) to similar families in other states. That analysis is not updated in this report but will be updated in our final report for this


ORKs WTW client (+1.9 percentage points). When disaggregated into mutually exclusive categories, the percentage who were employed-only was significantly lower (–2.1 percentage points), while the percentage who were both a CalWORKs WTW client and empl


d, WTW Client
   Employed, Not WTW Client
 Entry Quarter 0
 New Clients Enter CalWORKs WTW
 Two Years After Entry Quarter 8
 Not Employed, Not WTW Client
 Not Employed, WTW Client
  Employed, WTW Client
 WTW
Client Employed
  Employed, Not WTW Client


2. Percentage of CalWORKs WTW Entrants Who Are CalWORKs WTW Clients One Year After Entry, Percentage Employed One Year After Entry, and Average County Unemployment Rate One Year After Entry
75
50
Percentage
25
0
Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3


f CalWORKs WTW Entrants Who Were Employed One Year After Entry
50
45
Percentage
35
30
Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3Q4Q1Q2Q3Q4
2007 2008 2009 2010 2011 2012 2013 2014
Entry cohort
+ 1.9*
  SB 1041
 Post–SB 1041, with SB 1041
P


mployment rates and client characteristics, we see that the rates of those in CalWORKs WTW only one year after entry (and not employed) as well as the rates of those neither in CalWORKs WTW nor employed one year after entry were each about 1 percenta


er employed nor a CalWORKs WTW client one year after entry, which is 0.9 percentage points lower than we would have expected.
One Year After Enrollment, Was the Level of Earnings Different for Cohorts of CalWORKs WTW Clients Who Enrolled After SB 104


 and 2014 CalWORKs WTW Entry Cohorts)
    2013 Cohorts
2014 Cohorts Estimated
Estimated Impact for 2013 Different from 2014?
ü ü
ü
    Status One Year After Entry
Employed
WTW Client
Neither
WTW client only Employed and WTW client Employed only
Qu


ith SB 1041
 2,500
2,000
Real quarterly earnings ($)
 Change associated with SB 1041
  1,973
–18
1,954
 1,500
1,000
500
0
    Without SB 1041
With SB 1041
SOURCE: Authors’ analysis of 2010–2014 WDTIP files, EDD employment and earnings files, and EDD


e expect many of the clients who would benefit from the flexibility introduced with SB 1041 to have used it as intended—to increase their human capital and address barriers that would then put them on a more stable employment trajectory. We will also


 of June 2016
From the perspective of state-level stakeholders, all of the components of SB 1041 and the complementary Early Engagement activities were in place as of mid-2016, training had been provided, and data systems had been automated to the ex


g barriers to self-sufficiency and needed services.
• A majority of counties in the 2016 ACS indicated that it currently takes less than one month for a CalWORKs WTW client to begin participation in employment and/or job training and education. More


ate-level stakeholders viewed the ESE program as an underused mechanism, but use is
expected to grow with its consolidation with the AB 98 Subsidized Employment program.
• One challenge with the ESE program was getting employers to participate. Anoth


lients who received at least one exemption during their first two years in the CalWORKs WTW program increased slightly from the February 2007 entry cohort to the February 2014 entry cohort (from 44 percent to 46 percent). There was a similar increase


oss-section.
• Among the WTW clients who had the short-term WTW exemption for young children in December 2012, 16 percent still had this exemption in February 2014 and almost none had it in February 2015 and February 2016. Sixty-five percent of this


he WTW 24-month time clock. After three years, clients who started in the CalWORKs WTW program in February 2013 had used, on average, about 11 months of the 48-month time limit and about two months of the WTW 24-month time clock.
• The percentage of


t still had this exemption in February 2014 and almost none still had it in February 2015 and February 2016. Sixty-five percent of this group were not clients in the CalWORKs WTW program in February 2016.
Third, with respect to impact on clients, the


y obtaining a solid understanding of the WTW 24-month time clock and how it applies to them. And caseworkers continue to struggle with understanding how to calculate clients’ 24-month time clocks and how to explain and prepare CalWORKs clients for th


expected to increase the counties’ use of subsidized employment as a tool for increasing work participation. The extent to which this occurs will be of interest in our final round of data collection. Greater use of the program may require that count


e started later and other features in the law were scheduled to be phased in over time.
Overview of New WTW Services and WTW 24-Month Time Clock
Under SB 1041, the lifetime limit for CalWORKs receipt remained at 48 months (the time limit adopted in 2


anuary 1, 2013, have a 24- month period to engage in more-flexible WTW activities. However, those already enrolled in CalWORKs as of 2013 with more than 24 months of cumulative aid will likely reach their 48- month lifetime limit before exhausting th


ant difference between the clocks is the approved activities and the distinction between core and non-core activities. SB 1041 eliminated the WTW core and non- core hourly requirements, with the CalWORKs WTW 24-month time clock activities being more


• • • • •
•
• •
• •
Federal Work Activities
Core Activities
Unsubsidized employment • Self-employment
Subsidized private or public sector
• Grant-based on-the-job training
• Work study
Work experience
Community service
Vocational education (12-month


, WTW exemption, in the process of developing a WTW plan, good cause, domestic abuse waiver, etc.) (CDSS, 2012a; CDSS, 2012c). In general, the 48-month time-on-aid clock continues to tick each month while the nonexempt participant remains eligible fo


estimated for July 2015 through December 2015.
Transitioning to the Post–24-Month Time Clock Process
Caseworkers are to initiate the transition to the post–24-month time clock process when clients are nearing the end of the 24-month time clock. After


on of unticking a clock takes place in a manual system that communicates with the automatic system to send time-clock information to the state and federal welfare programs. As mentioned earlier, the 24-month time clock is unticked if a caseworker fin


ior to cuts in 2011 in response to the Great Recession.
• Supports for pregnant and parenting teens. The Cal-Learn program, after a suspension of case management services, was reinstated and allowed to resume its specialized case management and suppo


nsuccessful in finding employment (Figure A.1). This is consistent with a “work-first” approach, where job-search activities are required immediately upon receipt of assistance—typically for four weeks—and followed by a more thorough assessment of sk


e eligible for FS. Additionally, at least one adult in the family must not yet have exhausted his or her 24-month time clock and his or her 48-month cash-aid time clock to participate. If a client is granted good cause while in the FS program, partic


ment in a particular field (CDSS, 2013c). Through ESE, the county and the employer share responsibility for the participant’s wages during the job-placement period. The system used to divide responsibility for wages between the two parties varies amo


requirements by getting a caseload reduction credit, based on the extent to which the caseload has declined relative to 2005 for reasons other than eligibility changes (Schott and Pavetti, 2013; Brown and Derr, 2015).53 If a state program is unable


ews with CalWORKs WTW clients.
State-Level Key Informants
The state-level key informant interviews were conducted with individuals from the following agencies and organizations:
• CDSS
• CWDA
• Legislative Analyst’s Office
• legislative staff
• Offic


ask: How has your role – and/or your organization’s role changed with regard to the implementation of SB 1041 since we (or another member of the evaluation team) spoke with you in spring 2015?] What is your role – and your organization’s role – in re


el?
b. In the last year, what trainings have been offered to counties to prepare for OCAT implementation and to support caseworkers in the process?
i. How effective do you believe the OCAT trainings have been? [Probe for examples of how the OCAT trai


ams and the community colleges, in providing education services to CalWORKs participants?
a. What are some of the factors that facilitate collaboration at the state level between the state welfare program and California Community College Chancellor’s


ble for 48 months, and in 24 of those months there are different,
in some cases reduced, participation standards?
iii. That after the 24-month time clock is exhausted, that there is a change to the
CalWORKs federal participation standards?
11. Implem


, mental health services). The 2016 survey will ask about implementation activities that took place between the completion of the 2015 ACS (i.e., July 2015) and today. As you may know, the RAND Corporation and American Institutes for Research (AIR) a


 clicking the “save and exit” button. Please complete all sections before “submitting” the survey at the end of Section VII.
To review definitions related to the SB 1041 reforms, please see the last three pages of the survey. We encourage you to prin


ces for CalWORKs participants
   ¦
   ¦
   ¦
   ¦
   ¦
    k) Availability of line staff (e.g., caseworkers, employment services specialists) who work with and/or counsel WTW participants
     ¦
    ¦
    ¦
   ¦
     ¦
   m) Explaining the complexity


 or reconcile participants’ 24-month time clock status on a monthly basis.
r WTW caseworkers calculate participants’ 24-month time clock on a monthly basis for their caseload.
r Other
If you selected “Other,” please go to Question 5a. If not, please


 time clock
d) Effectively counsel CalWORKs participants on the flexibility of the 24-month time clock
Not At All Somewhat Very Confident Confident Confident
¦¦¦
¦¦¦
  c) Accurately reconcile discrepancies between your consortium’s database and your


_ If your WTW caseworkers have questions about the use of OCAT, what resources do they have
available to them? (Check all that apply.)
r Support from an administrator or supervisor
r Support from a content expert within your county social services de


ed
¦ ¦
¦ ¦
  c) Ability to save partially completed appraisals conducted using the OCAT
   ¦
   ¦
   ¦
   ¦
   ¦
   e) Ability of caseworkers to administer the OCAT in the participant’s preferred language
   ¦
   ¦
   ¦
   ¦
   ¦
 Overall, how satisf


 ¦
   ¦
 Transportation services (e.g., bus, Metro Rail)
Substance abuse Translation services Other services
¦¦¦
¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦ ¦
  e) Mental health
   ¦
   ¦
   ¦
   g) Domestic abuse
   ¦
   ¦
   ¦
   i) Housing assistance
   ¦
   ¦
   ¦
     Comp


 Stabilization Programs. Please continue to the next section.
   To return to a page in this section, please click “Back."
To continue to the next section, please click “Next.”
To exit the survey and return at a later time, please click "Save and Exi


     You have reached the end of Section VI: Questions for Supervisors About Line Staff and CalWORKs Participants. Please continue to the next section.
   To return to a page in this section, please click “Back."
To continue to the next section, plea


¦¦
  c) Provision of services (e.g., child care, domestic abuse assistance, housing assistance)
   ¦
   ¦
   ¦
   ¦
   ¦
   e) Coordination with community colleges
   ¦
   ¦
   ¦
   ¦
   ¦
   g) Coordination with other county agencies to provide serv


f SB 1041 Reforms
In this survey, we will be asking you about SB 1041 policy reforms to the CalWORKs Program that were implemented January 1, 2013. What we mean by SB 1041 reforms is the following:
• Changes to the Welfare-to-Work (WTW) time clock o


good cause for lack of services. SB 1041 replaced these exemptions with a once-in-a-lifetime exemption for young children.
§ Counties had until January 1, 2015, to reengage these exempt individuals.
• Earned income deduction
o Effective October 1, 20


ime clock with flexibility in welfare-to-work requirements; lowered the minimum hourly participation requirement for single parents; allowed recipients one lifetime exemption whereby one adult per family is excused from WTW activities if they are car


ew Questions
First we’d like to ask you a few background questions and ask about the overall status of SB 1041’s reforms implementation. (~5 minutes)
2. Role. Let’s start with your role within the county welfare department. What is your job title or


er caseloads?
6. We understand that the OCAT is currently available in English only. How
are the service needs and barriers to self-sufficiency assessed for non- English speaking clients?
a. For these clients, how does the appraisal process differ fr


ond to the amount of time it takes to
complete the OCAT?
c. What concerns, if any, do they have about doing the OCAT
interview?
d. How have clients responded to sensitive questions (e.g., about
substance use, domestic abuse, mental health issues)?
e.


ot had CalWORKs participants transition to the post 24- month time clock, when do you anticipate that will occur?
6. WPR: One topic that emerged from our interviews last year was the work participation rate (WPR) and counties’ efforts to meet WPR.
a.


ss supportive services (e.g., tuition assistance, money for books, transportation and child care assistance) to allow them to enroll and fully participate in education programs? What, if any, barriers exist to limit or prevent access to these service


ty social services department?
6. Do you interact with WTW caseworkers? If so, in what ways?
7. What types of supports does the CalWORKs Program provide students?
a. In your view, how knowledgeable are students about what services and supports they a


xpand the number of subsidized employment slots available to CalWORKs clients.
a. Expanded Subsidized Employment (ESE) Program
ii. Has your county developed an Expanded Subsidized Employment program
[Note: Interviewer should review the county-specifi


nt or organization?
a. By the county social services department?
b. Self-referral?
c. What information, if any, is needed by your department or organization from the county
social services department when a referral is made?
d. Do you interact with W


n education and training activities; and 5) the Family Stabilization and Expanded Subsidized Employment programs. The information you share with us today will help inform our understanding of the implementation and impact of SB 1041 reforms.
REVIEW O


e note that you may not be helped directly by participating in this study. However, others may be helped by what is learned from this research. The results of this focus group will help the RAND/AIR research team develop recommendations to the CDSS r


in your office?
b. As a result of using the OCAT, have there been any organizational changes in the
office, or a redefining of staff roles?
c. What type of training and guidance were provided to prepare you to implement the
OCAT?
i. Was it adequate –


tion, domestic abuse services]
d. To what extent do you feel the referral process is working effectively in your county?
e. In your experience, have there been any significant changes in the type and number
of referrals – meaning have there been any


ng adequate - – did it provide clear guidance on what activities necessitate “ticking” or “unticking” the 24-month time clock ?
e. How well do you think CalWORKs participants understand the 24-month time clock [Probe: increased flexibility; reduced p


ticipants with the education programs that they need or want to enroll in?
i. Since implementation of SB 1041, are more clients participating in educational programs now as part of the welfare to work program? Why or why not? [Probe for barriers or f


gram is working effectively in your county?
d. What, if any, are the barriers or challenges to implementing the ESE program? (probe:
insufficient number or type of employers, inadequate/insufficient participant work experience or interest, delays in


or are eligible to receive in the future.
D. I am going to ask you several questions about the types of services or assistance you have received from the CalWORKs Program.
E. We’re interested in your opinions and whatever you have to say is fine with


 When did you enroll in the CalWORKs Program?
2. What services are you currently receiving under the CalWORKs Program?
a. Which ones did you find most helpful and why?
b. Which ones did you find least helpful and why?
c. Are there any services that y


l and emotional health and about your exposure to domestic violence or any issues you have experienced related to drug or alcohol use.
a. What did you think about the questions that you were asked?
b. Did any of the questions make you uncomfortable?


would like to interview you again in about 3 months to see how things are going and to talk to you again about the types of benefits or services you have received or have tried to get from the CalWORKs program during those 3 months and any issues or


individuals in the state from 1998 to the present. CDSS receives quarterly updates from the California Department of Health Care Services and provides us with processed extract files semiannually. The extracts are in two forms: annual monthly extrac


d clients include those sanctioned because of WTW noncompliance; being a fleeing felon; violating a condition of probation or parole; being convicted of a felony for the possession, use, or distribution of a controlled substance; and being a custodia


cember 2012. “February 2013 status study cross-section, post-SB 1041 group” is defined as individuals who were CalWORKs WTW clients during February 2013 and first started in the CalWORKs WTW program in January 2013 or February 2013.
57 At the recomme


the first quarter of 2007 (January, February, or March 2007) and had not been clients any time between January 1998 and December 2006. “Impact study cohort 2014Q4” is defined as individuals who were clients in the fourth quarter of 2014 (October, No


iables in the tracking study analysis data set to a two-year span and the impact study analysis data set to the quarterly level.
Data Integrity
Staff at AIR and Stanfield Systems assessed the integrity of the data files and variables prepared for ana


l
Data Constructs
Caseload status Case discontinuance Applicant denials
WTW activities, exemptions, terminations
Demographics, sources of cash assistance, earnings, other income sources
        We expect that the final evaluation report will benefit


  All counties
Small counties Medium-sized counties Large counties
Number 12 Number 2 Number 5 Number 5
12 24 10 3 10 5 2 8 5 7 6 0
Number 24
Number 26 9 3 Number 105 5 0 Number 125 1 0
0 58 0 20 0 20 0 18
0 58 0 20 0 20 0 18
0 58 0 20 0 20 0 18
3 58


mber Number Number
Number Number Number Number
26 10 4 11 3 1 9 4 2 6 3 1
5 1 1 2 0 0 2 1 1 1 0 0
SOURCE: Authors’ analysis of 2016 ACS data.
(20 counties), from 1,000 to 4,999 for medium-sized counties
274


s to Which He or She Has Been Assigned or Scheduled (All Counties and by County Caseload Size)
     Employment/Job Training Activity
Subsidized employment All counties
Small counties Medium-sized counties Large counties
Job skills training All counti


18
1 51 0 17 1 18 0 15
                         SOURCE: Authors’ analysis of 2016 ACS data.
NOTE: Caseload size is up to 999 for small counties (20 counties), from 1,000 to 4,999 for medium-sized counties (20 counties), and 5,000 or more for large co


20 5 18
30 58 8 20 13 20 9 18
3 58 2 20
0 20
1 18
                  Benefited from enrollment in community college
      All counties
Small counties Medium-sized counties Large counties
Number Number Number Number
      Successfully transitioned to e


 Medium-sized counties Large counties
Number Number Number Number
Number Number Number Number
35 11 46 9 5 14 14 4 18 12 2 14
7 39 46 2 12 14
2 16 18
3 11 14
     SOURCE: Authors’ analysis of 2016 ACS data.
NOTE: Caseload size is up to 999 for small


ties
Small counties Medium-sized counties Large counties
Number Number Number Number
0 5 32 0 2 10 0 2 12 0 1 10
        How to use the flexibility of the 24-month time clock to the CalWORKs client’s advantage
        All counties
Small counties Medi


 Supervisor’s Rating of Caseworker Understanding of Other SB 1041 Changes and Related Supports
     Policy Change
Enhanced educational flexibility All counties
Small counties Medium-sized counties Large counties
Number Number Number Number
Not at All


3 Number 2 Number 1 Number 0
13 30 10 0 5 10 3 0 5 9 4 0 3 11 3 0
4 16 35 0 2 5 11 0 0 3 16 0 2 8 8 0
                        Increased flexibility offered by the 24-month time clock
All counties
Small counties Medium-sized counties Large counties
Nu


               Table D.14. Among Counties with an FS Program, Their Rating of How Well the FS Program Is Doing (All Counties and by County Caseload Size)
       FS Program Component
Helping stabilize families in crisis All counties
Small counties Med


 Number
        Ability to identify client exemptions
All counties
Small counties
Medium-sized counties
Large counties
SOURCE: Authors’ analysis of 2016 ACS data.
NOTE: Caseload size is up to 999 for small counties
(20 counties), and 5,000 or more fo


nties (18 counties). Counties were instructed to mark one rating for each type of need listed. County-level results for Table 4.28, Q27 on the ACS.
Needs
Total
Very Somewhat
Dissatis-
fied fied
Somewhat Very Satisfied Satisfied
Dissatis-
        CalW


Medium-sized counties Large counties
Number Number Number Number
0 7 0 3 0 3 0 1
291
13 7 13 5 11 6
                                                                                Table D.18—Continued
Much Somewhat About Somewhat Much Not


20 3 18
16 58 9 20 5 20 2 18
        All counties
Small counties Medium-sized counties Large counties
Number 2 Number 2 Number 0 Number 0
Very Well
24 5 10 9
19 3 7 9
23 4 8 11
24 3 10 11
20 1 9 10
13 1 5 7
6 1 2 3
23 1 13 9
Not at All


ured in February of each year. CalWORKs WTW clients include participating, exempt, or sanctioned CalWORKs WTW clients. These clients were members of two-parent families, all other families, and TANF timed-out cases. Unemployment rate is for Californi


 all other families, or TANF timed-out cases present in February 2013, 2014, 2015, and 2016. Categories may not add up to 100 percent because of rounding error.
296
     Table E.4. Number of New CalWORKs WTW Clients: Tracked Entry Cohorts 2007, 2009,


ions stop the WTW 24-month time clock, the 48-month time-on-aid clock, or both. For the tracking study, the presence of exemptions and sanctions were measured for 36 months including the entry month. For example, for the 2013 cohort, we counted month


 February 2013, February 2014, February 2015, February 2016
          Group and Indicator
Clients with short-term WTW exemption for young children in December 2012
New WTW exemption for young children Short-term young children exemption
Sanction 139


mployed only or also a CalWORKs WTW client
• CalWORKs WTW client: CalWORKs WTW client only or also employed
• employed only (and not a CalWORKs WTW client)
• both employed and a CalWORKs WTW client
 305
• CalWORKs WTW client only (and not employed)
•


. Karoly, Anamarie Auger, Beverly A. Weidmer, Ashley N. Muchow, and Italo A. Gutierrez, A Descriptive Profile of CalWORKs Recipients and Their Families: Initial Findings from the California Socioeconomic Survey, Santa Monica, Calif.: RAND Corporation


nfo/acl/2013/13-01.pdf
———, Pathways to Success: Overview of CalWORKs Change, Sacramento, Calif.: CalWORKs Employment and Eligibility Branch, 2013a.
———, “California Work Opportunity and Responsibility to Kids (CalWORKs) Program: Release of Third Set


lity to Kids (CalWORKs) Program: Additional Guidance and Forms Regarding Welfare-to-Work (WTW) 24-Month Time Clock Extensions, and Release of Related Welfare Data Tracking Implementation Project (WDTIP) Tracking Recipients Across California (TRAC) Im


.3, Sacramento, Calif., January 2016a.
———, “California Work Opportunity and Responsibility to Kids (CalWORKs) Assembly Bill (AB) 98 Subsidized Employment and Expanded Subsidized Employment (ESE) Program Changes,” All County Letter (ACL) No. 16-95, S


ports/RR919.html
315
Legislative Analyst’s Office, “The 2012–13 Budget: The Governor’s CalWORKs and Child Care Proposals,” webpage, February 22, 2012. As of June 16, 2016: http://www.lao.ca.gov/analysis/2012/ss/calworks-child-care-022212.aspx
O*NET R


tives and if there are any unintended consequences. This second evaluation report extends the analyses in the initial evaluation report through updated findings from the process study based on the second wave of the All-County Survey and qualitative


    EXECUTIVE SUMMARY
Local jurisdictions, faced with caseloads of increasing com- plexity and cost, have adopted alternative approaches to crimi- nal case processing—including new technologies—that have the potential to reduce backlog and improve ju


ogy in some capacity should be developed.
• Model configurations that can be used to help purchasers make intelligent buying decisions should be developed.
• Research is needed to better understand the effect of telepresence technology on defendants’


nefiel
9th Circuit Court, Florida
Robin Davis
ICF International
Jonathan P. Hein
Darke County, Ohio, Common Pleas Court
John W. Johnson
Miami-Dade Corrections and Rehabilitation
Casey Kennedy
Texas Office of Court Administration
Fredric Lederer
Cente


t appearances via video. According to the National Center for State Courts’ (NCSC’s) State Court Organization data set, which was last updated in
June 2017, state-level offices of courts oversee the use of video- conferencing in 35 states and the Dis


; Devoe and Frattaroli, 2009).1 In Crawford v. Washington, the Supreme Court further clarified that it is a violation of the defendant’s Sixth Amendment rights under the confrontation clause to be denied the opportunity to confront or challenge
an ad


 individuals to appear in court remotely, telepresence eliminates the need to trans- port defendants and offenders from correctional facilities to courthouses, reducing potential threats to the safety
of court personnel, corrections staff, victims, w


nd child abuse, to testify against the defendant without expe- riencing the revictimization and trauma of being physically present with their offender (Garvin et al., 2011; Kenniston, 2015).
Although telepresence technology might offer courts valu- a


s for the constitutional rights of court parties. More-serious technical issues could delay or disrupt court proceedings (Devoe and Frattaroli, 2009).
• data storage and security: Securely storing the large quantities of video and audio data generate


elays and challenges created by technical issues and the potential violation of the defendant’s legal and constitu- tional rights. Ultimately, the participants identified 24 specific research and development needs related to the use of telepres- ence


nsiveness when speaking (Landström and Granhag, 2010; Landström, 2008; Landström, Ask, and Sommar, 2015). Telepresence technology has the potential to negatively affect each of these elements. Although no known studies have confirmed this directly


s appearing remotely might be in a room where other people are talking or where other background noises are present. One panel member described an instance in which videoconferenc- ing equipment was installed in the loud boiler room of a jail, making


her the technical setup of the equipment has as much of an impact as the tech- nology itself on a judge’s determination during a bail hearing. Panel members also noted that research on the impact of social media platforms on communication suggests th


games that require remote communication might be more comfortable with the technology and view the court proceedings as something like a game. A lack of appreciation for the seriousness of the proceed- ing, coupled with the feeling of being removed


found satisfac- tion among patients, but not among providers, and suggested that although defendants might largely be satisfied with the quality of representation in proceedings involving telepresence technology, the defense attorneys could find the


fendant appears in person. Therefore, the use or availability of telepres- ence technology might encourage victims or other witnesses to participate in the proceedings. Telepresence technology might be especially beneficial in cases where physically


ce technology if the equipment setup is so complicated that “the bench looks like a cockpit.”
The panel members discussed the benefits of using telepres- ence technology to enable judges to sign warrants electronically after hours, including over the


eedings through a regular conference
call. More research is needed to understand the implications of these backup options. Although it might appear that allowing the proceedings to be conducted via conference call speeds up proceedings, saves money,


and case outcomes, several areas were identified as priority research and development needs. The panel identified and later ranked needs related to understand- ing the impact of telepresence technology on court proceedings. These needs include conce


the way in which telepresence technology is installed or operated can affect perceptions of credibility and case outcomes. Technical stan- dards and training on proper setup and use could help insure against these negative consequences.
According to


in mind that are easy to install in a variety of settings, including court and correctional facilities. Moreover, participants argued that telepresence vendors should automate system updates and provide ongoing technical support to promote the system


chnological devices and equipment to enable telepresence technology to be incorpo- rated.
Participants also touched on the importance of the back- ground for remote court appearances. Lighting and other environmental conditions of the remote court pa


ested that research should be conducted to identify best practices for videoconferencing audio setups and to determine which materials or other equipment could be used in the setup to minimize background noise and produce high-quality audio. The pane


 concern that multimedia court records would allow appellate courts
to overturn lower court rulings more easily and make assess- ments about credibility and other factors that could usurp the function of the trial court. As a result, trial and appell


lling telepresence technology equip- ment can be high, efficiencies could be realized by ensuring that telepresence systems are designed to enable multiple func- tions.
However, panel members pointed out that court purchasers often neither are aware


earch into options for improving network connectivity.
 Proprietary equipment and software that cannot be translated across devices might limit the ways in which telepresence technology can be used in court.
Potential Expansions of Telepresence Techn


as the best option for areas with less tech- nological bandwidth but that seek to use telepresence technol- ogy. Improving wireless coverage is less expensive and more feasible than adding more bandwidth or using satellites. A judge, attorney, or ot


courts might consider implementing some of these capabilities for use in certain legal proceedings. If everyone participated through telepresence technology, any bias or nega- tive consequences would be eliminated because the defendant or other key


a related concern about the potential impact a fully virtual courtroom could have on behaviors and interactions. As mentioned earlier, communicat- ing behind a screen might change how an individual engages with others. It remains an open question wh


thing to the objective and 9 indicated that meeting the need could result in a 20-percent or greater improvement in performance).
2. how practical it would be to meet the need (while meeting some needs might require only minor adaptation, meeting oth


rchasers; (7) conducting research into the effect of telepresence technology on defen- dants’ experiences with the court process and perceptions of procedural justice; (8) developing standards on video quality and image size; (9) conducting research


arch to identify best prac- tices and minimum standards for audio setup.
• Conduct research to assess the impact of telepresence technology on the expe- riences of witnesses and victims.
• Identify the technical issues that impact the effectiveness o


 practices for useful Yes performance indicators and the kinds
of data that should be collected
from these systems to inform the
indicators.
• Conduct research to assess the Yes impact of telepresence technology
on the experiences of witnesses and
vi


ccess to and sharing of telepres- ence feeds.
Need Received a High- Importance
Score Corresponding (Yes/No) Category
Yes Measuring outcomes
No Measuring outcomes
Yes Technical standards and training
Yes Technical standards and training
No Measuring o


 remote interactions on court
outcomes. One panel member referred to telepresence technol- ogy as being “guilty until proven innocent.” In other words, without rigorous efforts to assess the impact of telepresence technology on such outcomes as bail


 of the workshop’s focus areas.
In advance of the workshop, participants were provided an opportunity to identify the issues and topics that they felt would be important to discuss during the workshop. Using a comprehensive literature review and inpu


d be met and broadly implemented successfully. That is, they could assign the need’s chance of success between 10 percent (i.e., a rating of 1) and 90 percent (i.e., a rating of 9). This dimension was intended to include not only technical concerns (


version 3.5. We chose this algorithm to minimize within-cluster variance when deter- mining the breaks between tiers. The choice of three tiers is arbitrary but was done in part to remain consistent across the set of technology workshops we have cond


o indicate whether each problem and need pairing should be voted up or down on the list. An example of this form is provided in Table A.2.
We then tallied the participants’ third-round responses
and applied those votes to produce a final list of prio


suffer when some or all of the
participants are remote (judge, court reporter, prosecutor, defendant, etc.).
Need: Conduct research on the impacts on decisionmaking, communication, and 3 perceptions of satisfaction with the process when one or more p


 output from this process became the final ranking of the panel’s prioritized results.
Figure A.3. How a Need’s Increase in Expected Value Might Result in Its Movement Across Tier Boundaries
Higher score
Tier 1 score range
1
27
   1
1
        2
  Fig


 45, No. 1, 2014.
Concurrent Technologies Corporation and CONSAD Research Corporation, Evaluation of Video Conferencing Demonstration Projects: Final Report, Johnstown, Pa.: Pennsylvania Commission on Crime and Delinquency, 2000.
Cookingham, Lisa M.,


lywood, John S., Dulani Woods, Andrew Lauland, Sean E. Goodison, Thomas J. Wilson, and Brian A. Jackson, Using Future Broadband Communications Technologies to Strengthen Law Enforcement, Santa Monica, Calif.: RAND Corporation, RR-1462- NIJ, 2016. As


 Journal, Vol. 90, No. 4, 2015, pp. 1683–1705.
MacCabe, Jena, “Can You Hear Me Now?: Interpreters for California Civil Cases,” Loyola of Los Angeles Law Review, Vol. 49, 2016,
pp. 685–704.
Maryland Administrative Office of the Courts, Evaluation of t


cial Media its Impact with Positive and Negative Aspects,” International Journal of Computer Applications Technology and Research, Vol. 5, No. 2, 2016, pp. 71–75.
Strickland, S., R. Schauffler, R. LaFountain, and K. Holt, eds., “State Court Organizat


er by training and a research public health analyst in the Applied Justice Division at RTI International. She has provided training and technical assistance to research partners evaluating innovative prosecutorial strategies and conducted original sc


ention. Planty has led several projects for the U.S. Department of Justice and plays a key role in several ongoing studies related to school safety, incident-based crime statistics, identity theft, and sexual assault. Before joining RTI in 2017, Plan


ement by NIJ or the RAND Corporation.
 This publication was made possible by Award Number 2013-MU-CX-K003, awarded by the National Institute of Justice, Office of Justice Programs, U.S. Department of Justice. The opinions, findings, and conclusions o


 CORPORATION
STEPHANIE BROOKS HOLLIDAY, JIRKA TAYLOR, PRISCILLIA HUNT, IFEANYI EDOCHIE, SARAH B. HUNTER
Evaluation of a Trauma-Informed Program for Juvenile Justice- Involved Youth
The Pilot Program at Lookout Mountain Youth Services Center
 For more


d social and economic well-being of populations and communities throughout the world. The program focuses on such topics as access to justice, policing, corrections, drug policy, and court system reform, as well as other policy concerns pertaining to


am. To better understand the nature of the pilot program, we com- pleted a series of related procedures. We conducted key informant discussions, reviewed relevant documents, and conducted a site visit to observe the facility. To understand barriers a


 or proportion of youth who earned a diploma or GED during the evaluation period.
Staff Program Outcomes
Data indicated that staff absences were common. During the pilot period, staff members on both units missed approximately three days of work per


 across DYS facilities. Therefore, while the program holds promise, we offer the fol- lowing recommendations to Colorado DYS for the ongoing implementation of the program:
viii
Evaluation of a Trauma-Informed Program for Juvenile Justice-Involved You


t CI confidence interval
DYS Division of Youth Services FTE full-time equivalent
GEE generalized estimating equation
HB House Bill
LMYSC Lookout Mountain Youth Services Center MYSI Missouri Youth Service Institute
OR odds ratio
PREA Prison Rape Elimi


First, certain symptoms of mental health diagnoses can manifest as problem behav- iors such as anger, irritability, and aggression (Underwood and Washington, 2016). Similarly, youth with a history of complex trauma may be at greater risk of exhibitin


mbers. In the report, staff members noted that they lacked training to appropriately de-escalate behavioral incidents. Together, as indi- cated in the report, these practices represented a significant barrier to effective rehabilitation.
In recogniti


aff.
behavioral health needs (Ryan and Mitchell, 2011). Together with HB 17-1329, this Strategic Plan shaped the design and implementation of the trauma-informed pilot program at Lookout Mountain Youth Services Center (LMYSC).
Colorado DYS Pilot Prog


program is described in more detail in subsequent chapters.
Present Evaluation
In addition to establishing the pilot program, HB 17-1329 required that the program be evalu- ated to assess its effectiveness during its first year of implementation. The


SI submitted to DYS for the training contract, the Colorado DYS Policy on Reporting Critical Incidents (CDHS, 2017a), the Colorado DYS Strategic Plan (CDHS, 2017b), and the MYSI Assessment of the Colorado Division of Youth Corrections (MYSI, 2017).
F


selecting a comparison group from a different facil- ity may have introduced more variability with respect to these factors. Each unit at LMYSC is made up of multiple pods. The pilot unit was smaller than other units with regard to the number of pods


event the youth from being in a prone position; as well as the use of transport restraints. Levels of physi- cal responses were recorded using categories established by Colorado DYS. Level One refers to an “escort hold,” in which a staff member holds


eyes or skin). “Unauthorized/Incidental” incidents reflect fights that are not the result of an intentional act of aggression.
Method 7
8 Evaluation of a Trauma-Informed Program for Juvenile Justice-Involved Youth
Injuries. We received information as


nary conversations with represen- tatives of DYS and LMYSC. These discussions revolved around two principal topics: (1) data collected by the DYS that could be made available to the research team; and (2) reflections of DYS and LMYSC representatives


the youth and staff in the pilot and comparison groups. For youth, this included age, race/ ethnicity, and committing offense; for staff, this included role at the facility. We also examined whether there were differences in the mean length of time


mparison of costs over time, and (4) unit costs (to the extent possible and analytically meaningful). Note that in the main text, we focus on summarizing the results of this analysis and provide more detailed results in Appendix C.
CHAPTER THREE
Eval


perience and supervise the direct care staff.
At the beginning of the pilot program, the intention was that YSS staff would be specifi- cally assigned to the pilot program; however, based on our key informant discussions, this became more difficult t


nt, the training included ten modules and incorporated didactic elements and practical applications (e.g., case vignettes). The
Missouri Model training incorporates elements of existing models of leadership and group pro- cess (e.g., Tuckman Model of


ram model differed from implementation.
Features of Programming Across LMYSC
Youth involved in the pilot program participated in many of the same activities as youth in other areas of the facility. In this section, we describe the core features of pr


two individual and one family session, but if the family is unable to participate, it can be three individual sessions. In addition, youth receive at least one group therapy session per week. These treatment groups are determined based on the individ


o reported that he encourages staff to display youth achievements and artwork, and to incorporate aspects of the treatment philosophy (e.g., charts with different emotions, community standards). In addition, youth in the pilot program were given acce


s prior to parole, and continue providing this service for 90 days after parole in the family’s own community. The goal of this component is to help understand and address any issues that the family has and connect them with resources that they may n


articipate in regular behavioral health services, consistent with the broader LMYSC community
• All LMYSC staff are provided with training in the Sanctuary Model; pilot staff received additional intensive training in the months leading up to pilot im


columns summarize the outcomes expected from the program. In the short term, the pilot program is expected to decrease negative behavioral incidents, reduce use of restraints and seclusion as behavior management techniques, decrease injuries, decrea


mber acknowledged that eliminating seclusion is beneficial, in that seclusion can be traumatizing. However, this staff member described the value of “contain- ment” in more extreme situations (e.g., “causing physical harm every day”), noting that thi


ff members feeling unsafe.
20 Evaluation of a Trauma-Informed Program for Juvenile Justice-Involved Youth
In turn, being short-staffed has resulted in multiple challenges to implementation. First, inter- viewees described how one foundational tenet o


he jail system. Here, it’s a little different. If you attack a staff, we’ll do what we have to do, use community restoring techniques.” When asked if these techniques were effective, the staff member said, “With certain youth. Then certain youth care


ereas some found it helpful and engaging, those with more of a background in trauma-informed care found it less useful. The latter group suggested there could be ways to dive deeper into the content and spend more time discussing application of the i


on youth who resided on the pilot or comparison units during the evaluation period. This included youth who were on these units at the beginning of the evaluation period, as well as youth who entered these units during the year of implementation. Th


nts and Level Two physical responses were most common. Of note, the maximum number of physical responses with mechanical restraints or Level Two physical responses for a given youth in any month was seven incidents. On both units, most outcomes were


Juvenile Justice-Involved Youth
Table 3.4
Proportion of Youth Experiencing Each Outcome by Unit
 Outcomes
Without mechanical restraint With mechanical restraint Side hold
Transport restraint
Level One Level Two Critical
Seclusion < 4 hours Seclusion


)
40.20 (26.64) 22.0% (9)
p-value
p = 0.86 p = 0.49
   Within the subset of youth who had not yet earned their degree (n = 41 pilot youth, n = 31 comparison youth), we examined the mean number of credits earned during the pilot year. We also examined


nt difference in the proportion
Table 3.7
Staff Absences and Turnover by Unit
 Outcome
Total absences (days) Absences per month (days) Staff left facility
Pilot
M (SD) or % (n)
17.4 (27.14) 2.68 (2.81) 31.10% (14)
Comparison M (SD) or % (n)
15.38 (17


d (2) costs associated with the running of the facility of which the pilot is a part, but which are not specific to the pilot. We discuss these two types of costs in turn below to arrive at a composite measure of the pilot’s running costs. Thus, this


is share of facility-wide costs to the pilot.
1 Please note that the setup of the pilot also likely required staff time from personnel not explicitly dedicated to the pilot, such as participation in training or oversight and management by the facilit


90 $290 $417 Total $436,323
  CHAPTER FOUR
Conclusion
This report has documented the first year of implementation of the trauma-informed pilot program at LMYSC. In this section, we summarize the findings, discuss how the pilot pro- gram fits with the


need for behavioral management techniques that staff members can use to address these situations. In addition, though use of mechanical restraints was less common in the pilot program, it was still used with 45 percent of youth at least once during t


ning or the pilot program philosophy when interacting with youth on those other units. Therefore, nonpilot units also had exposure to certain aspects of trauma-informed care. This may have contributed to the lack of significant differences between th


s a substantial difference between the staff working in the pilot and nonpilot areas (i.e., where more senior or specialist-skilled individuals are recruited for the pilot).
Considerations Related to Fit with Colorado DYS
From a theoretical and empir


part because certain restrictions had to be put in place across the facility and in part because staff were pulled from the pilot to cover other units. To some extent, placing limits on the programming of the pilot program may be inevitable when ther


program. We did not interview staff of the comparison unit for this evaluation, but doing so would provide additional information about what practices and challenges were unique to the pilot versus experienced facility-wide.
Regarding the evaluation,


the cost analysis, data provided by DYS include only information on costs directly incurred by DYS and do not include other notable cost categories, including rent/ mortgage, utilities, insurance, and workers’ compensation. As such, the costs of the


all staff—including new program staff—would ensure that staff members are operating from a shared understanding of the program. It may not be feasible for MYSI staff to provide this type of training to all new staff members, especially given current


t recom- mendation) may also reduce absences and turnover.
4. Explore staff concerns regarding safety and the effectiveness of the program in more depth.
Staff reported that in recent months they had felt unsafe at the facility—and some staff noted t


ral health staff
Conclusion 37
38 Evaluation of a Trauma-Informed Program for Juvenile Justice-Involved Youth
run groups that only include pilot program youth. Staff members also had suggestions of addi- tional groups or activities that would be bene


as the Colorado DYS Strategic Plan specifically identifies the promotion of “prosocial, safe, and nonviolent interactions” as a goal, and the pilot program aims to “promote building trust and healthy relationships between youths and staff and to allo


. We understand that the team is a key part of this pilot program. How is your working
relationship with your colleagues? [probes] How is your working relationship with lead- ers? [probes]
Section E: Barriers and Challenges to Implementation (All Sta


clude annual leave payments and sick leave payments. Negative values represent reversals of earlier payments. The total costs from July 2018 to June 2019 were $83,655, or on average $6,971 per month. Of these, the prorated amount attributable to the


at would be added to an already existing pod.
References
 Abram, Karen M., Linda A. Teplin, Devon R. Charles, Sandra L. Longworth, Gary M. McClelland, and Mina K. Dulcan, “Posttraumatic Stress Disorder and Trauma in Youth in Juvenile Detention,” Arch


ry, “Fostering Implementation of Health Services Research Findings into Practice: A Consolidated Framework for Advancing Implementation Science,” Implementation Science, Vol. 4, Article 50, 2009, pp. 1–15.
DeLisi, Matt, Andy Hochstetler, Gloria Jones


hol Use Disorders in Primary Care. Santa Monica, Calif.: RAND Corporation, RR-2049- NIDA, 2017. As of September 30, 2019:
https://www.rand.org/pubs/research_reports/RR2049.html
IBM Corporation, IBM SPSS Statistics, version 21, 2012.
Ken Blanchard Com


velopmental Sequence in Small Groups,” Psychological Bulletin, Vol. 63, No. 6, 1965, pp. 384–399.
Underwood, Lee A., and Aryssa Washington, “Mental Illness and Juvenile Offenders,” International Journal of Environmental Research and Public Health, Vo


 SOCIAL AND ECONOMIC WELL-BEING
The Frequency and
Economic Impact of
Musculoskeletal Disorders
for California Firefighters
Trends and Outcomes over the Past Decade Michael Dworsky, Seth A. Seabury, Nicholas Broten
Prepared for the Commission on Healt


. ICJ is dedicated to improving the civil justice system by supplying policymakers and the public with rigorous and nonpartisan research. Its studies identify trends in litigation and inform policy choices concerning liability, compensation, regulati


...................... 52 Figure 7.1. Number of Chiropractic Visits in Three Years of Medical Care, All Workers’
Compensation Information System Injuries with One or More Chiropractic Visit.............. 80 Figure 7.2. Number of Occupational Therapy


 with Indemnity Benefits Receiving Any Capped
Services, by Injury Type, Service Type, and Occupation..................................................... 78
Table 7.2. Average Visits Injured Workers with Indemnity Benefits Receiving Any
Capped Servic


ters than for workers in similar occupations. MSDs in older firefighters (aged 55 or above at injury) had earnings and employment losses comparable to private-sector workers, however. The report also examined the effect of medical reforms enacted in


among firefighters with MSDs.
 Estimate the effects of disability rating reforms enacted in SB 863.
 Assess the effect of caps on chiropractic, occupational therapy, and physical
medicine visits.
We analyzed administrative data from the California


es are high compared to other workers. Between 2005 and 2017, the firefighter injury rate ranges from just under 200 per 1,000 workers to more than 250 per 1,000. Injury rates for police officers are similar though slightly lower, while the overall r


f Musculoskeletal Disorders for Firefighters Remain Less Severe than for Workers in Similar Occupations
We also examined postinjury earnings and employment for firefighters, using methods developed in previous RAND studies to compare injured worker o


milar impairment severity to police and other comparison occupations with the same type of rating. Firefighters had slightly higher final ratings than comparable occupations prior to SB 863, however. Firefighters have relatively high occupational adj


mented under SB 863.
Firefighters with Musculoskeletal Disorders Rarely Receive Treatment or Permanent Disability Benefits for Post-Traumatic Stress Disorder or Other Psychiatric Conditions
We examined diagnosis codes on medical services and prescrip


s on patterns of care or worker outcomes, so the Commission on Health and Safety and Workers’ Compensation requested that we study this issue. We ask what
xv
proportion of workers with and without MSDs have sufficiently high utilization of the capped


pensation system. Data from the workers’ compensation system also cannot speak to the incidence of PTSD or mental distress among workers who do not file claims for a workplace injury. The potential for stigma suggests a need to go beyond claims data


te sector (Bureau of Labor Statistics, undated). In California, the risks of firefighting have become even more salient in the past few years, with the record wildfires and resulting deaths that have occurred.
The health risks facing firefighters go


eform in 2012 with California Senate Bill (SB) 863. This bill was passed in response to the fact that employer costs and premiums were rising even though the earlier 2004 reforms had led to a dramatic cut in permanent partial disability (PPD) benefit


delays in medical treatment. However, while these reforms have the potential to improve worker outcomes and contain costs, their implications for California firefighters with MSDs should be relatively minor.
 2
workers appeared to be particularly har


e PTSD and other psychological injury reporting.
7. Empirically assess what percentage of MSD claims are ultimately determined compensable under workers’ compensation.
8. Of MSDs determined compensable, assess how many are subject to causal apportion


urces and methods to the relevant chapters, and additional details on sample construction are presented in Appendix A.
Chapter 3 uses only occupation descriptions, worker demographics, and injury characteristics from the First Report of Injury (FROI)


l factors relevant to public safety employees, particularly regarding disability compensation that could affect our findings. We then discuss previous work and offer a few descriptive statistics using national data to describe the overall MSD risk fo


lassification of Diseases (ICD), specifically ICD-9-CM diagnosis codes contained in the medical billing data. The ICD-9 codes have the advantage of providing more precise clinical detail about the nature of injury and an individual’s physical conditi


nclude full coverage of medical costs and partial coverage of lost wages. Indemnity benefits vary depending on whether the injury is permanent or temporary, as well as according to such other factors as the type of injury and/or the worker’s age or o


ear. As noted above, these workers are routinely asked to put their lives at risk to protect the public, and we might be concerned that workers would be less willing to take on these risks if they knew it would lead to significant future economic har


hus, comparing workers eligible for LC §4850 compensation to other workers could make it look as if injured firefighters and other safety employees were actively working after an injury even if they were out of work due to temporary disability. This


ing moral hazard“ while workers increasing their risk taking and experiencing more injuries is referred to as “risk bearing moral hazard“ (Butler and Worrall, 1991). Both are theoretically feasible, but generally speaking the former is considered to


is a growing concern in California. Using data from the Department of the Interior from 2003 to 2007, Britton et al. (2013) identified slips or falls as the most common type of nonfatal injury, followed by injuries involving equipment or machinery.


se is made up of information on individual workers’ compensation claims submitted to DIR from claims administrators using the Electronic Data Interchange (EDI) formats developed by the International Association of Industrial Accident Boards and Commi


, such as maximum medical improvement, end date of temporary total disability payments, date and receipt of permanent partial disability payments, return to work (if applicable), and claim closure.
In theory, combining data from FROI and SROI should


a semiannual survey of approximately 200,000 nonfarm establishments, for a total of about 1.2 million establishments over a three-year survey frame, which BLS uses to provide annual estimates. Because of the nature of the survey, the data exclude far


or on a scale of 1 to 5, with 5 indicating the factor is more relevant. As we discuss below, we use these data on job demands to select private- sector occupations similar to firefighting for inclusion in a comparison group.
Methods
We had to choose


d classification system, unlike the occupation description. Submission of class codes to WCIS is optional for self-insured employers, however, so class codes are not present in all FROI records, with differentially low rates of reporting among public


active firefighters identified by our current approach.
Comparing Job Demands for Firefighters and Workers in Other Occupations
As with the 2010 RAND firefighter study, our analysis compares injury frequency and outcomes for firefighters to a selecti


less relevant each factor is for a firefighter compared to someone in a randomly selected occupation, according to whether the difference is positive or negative, respectively. Even though the precise ordering should be taken as suggestive since the


 -0.51
3.11 -0.54
3.04 1.07
3.1 0.46
3.84 0.34
3.24 0.3
3.84 0.25
4.19 0.07
1.93 -0.12
             Spend time using your hands to handle, control, or feel objects or tools
Spend time making repetitive motions Indoors, environmentally controlled Spen


ghters based on
job characteristics.
Table 3.2. Firefighters Compared to Similar Occupations
0.0
5.3
7.9
9.5
10.6
   Occupation Title
Statistical Distance
  Municipal firefighters
Municipal firefighting and prevention supervisors Fire investigators F


s and attendants, except emergency medical technicians Refrigeration mechanics and installers Septic tank servicers and sewer pipe cleaners Millwrights Commercial pilots Occupational health and safety technicians
          25.3
   22
is unlikely to y


, the general patterns are the same even with additional years.
Millwrights; Occupational Health and Safety Technicians; and Police Identification and Records Officers. We did not identify the excluded occupations in the WCIS data, so we are unable t


or firefighters. For firefighters, MSDs account for 47.3 percent of all injuries, compared to 38 percent for police officers, 42.5 percent for other public-sector workers, 37.2 percent for workers in the private-sector comparison group, and 41.8 perc


kely (54.6 percent) than the statewide average to be female.
Table 3.5. Demographics of Injured Workers with Musculoskeletal Disorder Claims, by Occupation
   Active Active Rest of Public Firefighters Police Sector
Rest of Private Sector
40.4 13.9%
2


tiple body parts.
Figure 3.3. Injuries by Occupation and Body Part
   0.0% 20.0% Firefighters
Police Officers
Other Public-Sector Workers
Private Sector Comparison Group
Other Workers
Head, Face or Neck Upper Extremities
40.0%
60.0% 80.0%
100.0% 120.


 missing data on injury cause.
WCIS also includes some information on the cause of injury, which we compare for MSDs versus other injuries across occupations in Table 3.6. Specifically, we report the share of all injuries reported to WCIS that fall i


ot specifically referring to job security or injury compensation, as we expect that job security and injury compensation are similar between firefighters, police officers, and other public employees. While definitively studying the factors that drive


gain experience or find better jobs, in which case earnings losses would be greater than the observed drop in earnings from before an injury to after an
 32
injury. On the other hand, noninjured workers might experience declines in earnings over time


settings with extremely high job security, such as state and local government or unionized manufacturing, such an assumption could well be a reasonable approximation in the absence of better data.
However, this is not a justifiable assumption for the


Control Group
A key departure from previous RAND research on earnings losses is the use of medical-only injuries as an alternative control group. A limitation of our typical methodology is that the individual characteristics available for uninjured c


e UI program must report the wage and salary earnings of every employee to EDD on a quarterly basis. EDD stores these data in the Base Wage File. Both workers’ compensation and UI are nearly universal in California, so the wage and salary earnings of


3 Match rates were very similar across occupational categories: summing across all injuries and injury years available, between 57.5 and 61.5 percent of the sample used in Chapter 3 was successfully matched to EDD wage data with matched controls.
36


ed in Appendix A.
5 The BLS OES program reports an average hourly wage of $35.51 for firefighters and an annual mean wage of $48.12 for police. See Bureau of Labor Statistics (2017a) for police and Bureau of Labor Statistics (2017b) for firefighters.


laims is 92 percent, as against 82 percent for police, 87 percent for other public-sector workers, and 79 percent for similar private- sector workers.8 The differences in control worker earnings dynamics illustrated by Figure 4.1 are thus in line wit


y in earnings noted above. Similar recoveries in relative earnings are apparent for other occupations as well, but relative earnings in the second year after injury are sharply lower for all other occupations studied here: 88 percent for police, 85 p


ounced in nonmusculoskeletal cases. However, relative at-injury employment remains sharply higher for firefighters (90 percent) than for other comparable occupations (83 percent for police, 85 percent for other public-sector workers, and 76 percent f


phics and to provide a comparison to results obtained using uninjured control workers, we also present unadjusted relative earnings based on medical-only controls, as well as estimates based on uninjured control workers. Table 4.3 focuses on earnings


rred between 2005–2007 and 2010–2012 and that a similar deterioration in outcomes was evident in the other occupations as well. Comparing
43
workers injured just before SB 863 took effect (2010–2012 injury years) to those injured just after SB 863 to


ectories up to three years postinjury for injured and control workers in each occupation; figures plotting these trajectories are shown in Appendix B. We do not find evidence of a sharp decline in the third postinjury year for firefighters when compa


uggested by Boone et al. (2011). We would expect this mechanism to be less relevant for firefighters given the high levels of job security (including high postinjury job retention); we cannot rule out that the severe pressure and sharp cuts in state


viewed 37 studies and found that there were associations between work factors and common mental health conditions but concluded that the causal relationship between work and mental disorders remained unclear due to methodological limitations in the e


sychiatric Association’s Diagnostic and Statistical Manual of Mental Disorders (5th ed.) for more details (APA, 2013).
3 Hoge et al. (2004) surveyed veterans returning from Iraq and Afghanistan and found that fewer than half of those experiencing PTS


consequences of comorbid psychiatric conditions is referred to as sample 4 in Appendix A.
6 We conducted this analysis only on claims with dates of service.
7 The ICD-9 code for PTSD is 309.81. Our measure of psychiatric disorders includes all ICD-9


and police with MSD claims in comparison to other public-sector workers and similar private-sector workers. On nonmusculoskeletal disorder claims, psychiatric disorder rates for firefighters and police are closer to those observed among comparable p


s had relative earnings of 90 percent over the two years after injury, versus relative earnings of 71 percent for firefighters who also had psychiatric comorbidities. Consistent with the estimates in Chapter 4, relative earnings for workers in other


ychiatric comorbidities are associated with worse labor market outcomes compared to MSDs without such comorbidities. This finding was anticipated given previous evidence on earnings losses for workers with permanent disability due to psychiatric impa


ornia and the changes implemented under SB 863. This discussion draws heavily on Chapter 5 of Dworsky et al. (2016). Readers seeking further details may wish to consult that report.
The level of permanent disability (PD) benefits provided to an injur


er, the larger the magnitude of the adjustment. Again assuming a 10-percent rating after occupational adjustment, a 21-year-old worker would have the rating reduced to 8 percent and a 62-year-old worker would be increased to 13 percent. The age- adju


ed to calculate permanent partial disability. Second, SB 863 made several modifications to the Permanent Disability Rating Schedule. We briefly discuss these changes here, with an emphasis on the potential impacts on firefighters with MSDs.2
One majo


lic safety workers who are unable to return to work.
 57
below the statewide average weekly wage, the majority of all injured workers were receiving the maximum weekly PD benefit before SB 863. The workers examined in this report—particularly firefig


t. Like the add-on elimination, the bump-up / bump-down elimination was expected to have limited effect because the implementing language in SB 899 was interpreted by the courts to mean a substantial fraction of PPD benefits are paid out before the b


 a back injury. The injury in the Guzman case was carpal tunnel syndrome. 59
 Data
We rely on disability ratings performed by the state DEU. DEU performs three types of ratings, known as consult, summary, and formal ratings. The characteristics of ca


with PD, and it is therefore unclear whether DEU can be treated as representative of the full population of workers with PD. As discussed above, cases with different types of ratings (summary, consult, or formal) reach DEU through distinct pathways t


is of trends. Sample sizes by occupation and period (pre- vs. post- SB 863) are presented below.
Table 6.3 shows summary statistics on DEU ratings for workers in one of the occupations of interest in this study with MSDs. Panel A reports summary stat


Impairments and Disability Ratings for Firefighters and Other Workers
Before comparing ratings and benefits across occupations, we examine the distribution- specific impairments rated in MSD for firefighters and other occupations. We grouped impairme


 6.4. Body Part of Highest-Rated Permanent Impairment by Occupation, Constant-Maturity Musculoskeletal Disorder Cases
 Body Part
Knee
Lumbar spine
Shoulder
Cervical spine
Ankle
Hip
Elbow
Arm neuropathy
Hypertensive cardiovascular disease Other
Active


26) 13.0 13.0 (0.74) (0.40)
Other Private-Sector
 12.5 12.9
 Table 6.6 presents final ratings by occupation, both for the full 2005–2015 sample and by injury date, with injury dates grouped into cases rated under the pre–SB 863 disability rating sche


results presented in Table 6.6 reflect average outcomes over the entire period when the disability rating schedule established under SB 899 was in effect, from 2005 to 2012. Since outcomes may have changed during this period, a narrower-term comparis


n summary ratings, this appeared to reflect systemwide rating practices rather than anything particular to firefighters.
These findings suggest that increased apportionment among consult ratings may have offset some of the rating increases anticipate


D injuries Non-MSD injuries
Active Active Firefighters Police
0.6% 0.5% 4.0% 3.7%
Active Active Firefighters Police
0.6% 0.4% 0.0% 1.0%
Active Active Firefighters Police
0.0% 1.1% 7.7% 6.0%
Other Public Sector
1.6% 10.6%
Other Public Sector
1.3% 1.8%


the average statutory benefit across occupations and over time. Table 6.10 shows that there were substantial increases in statutory benefits after SB 863 took effect for both summary and consult ratings. As above, we compare 2010–2012 injury dates t


apportionment to nonoccupational cause. While we did not have a sufficient sample size of firefighters with consult ratings to examine differences across occupation, across all occupations the increased frequency of apportionment was concentrated amo


e targeted modes of treatment: chiropractic, OT, and PT. Although LC §4604.5 specified that the cap could be waived with written authorization by the employer, the treatment caps clearly established a new and substantial barrier to receipt of care be


899 differentially reduced spending on low back injuries and that this reduction in spending led to worse labor market outcomes for injured workers. Powell and Seabury (2018), however, did not have access to data on medical claims and were not able t


that are subsequently reversed are a reasonable indication of delays and hassles encountered by injured workers.
To study the treatment caps, we focused on visit counts within the three varieties of capped services: chiropractic, OT, and PT. We focus


sit counts up to three years after the first service date in the workers’ compensation bills.
Results
Table 7.1 reports the proportion of indemnity claims receiving each of the capped services at any time in the first three years of medical care. Uti


its.
Figures 7.1, 7.2, and 7.3 depict the distribution of chiropractic, OT, and PT visit counts for workers using each service type who had 36 or fewer visits in the first three years after the first service date. The fitted curve on each histogram i


93.85% 0.78% 5.36%
83.61% 1.56% 14.83%
Private-Sector Comparison
94.15% 0.68% 5.17%
96.20% 0.90% 2.90%
83.87% 1.69% 14.44%
 Under 24 visits Exactly 24 visits Over 24 visits
 Occupational Therapy
Under 24 visits Exactly 24 visits Over 24 visits
Physi


24 visits on pre–SB 228 injuries, it is difficult to know how many workers would have received more than 24 visits in the absence of the treatment caps. It is also plausible that the major reforms implemented in 2004–2005 (as well as the more recent


other workers. Similar to the 2010 report, we found that firefighters are significantly more likely to be injured than workers in other occupations, and a larger share of those injuries are MSDs. Specifically, we found that nearly half (47 percent) o


ss likely to separate from their at-injury employers than other types of workers. In most occupations, employment at the at- injury employer is well below the overall employment rate, indicating that injury can lead to increased job separations or ca


ssible to credibly evaluate their impact on patient outcomes.
There are a number of policy implications from the findings. Perhaps most importantly, California firefighters continue to face an elevated risk of injuries, particularly MSDs. While injur


rlying results in the text.
Sample Construction and Weighting
We construct four separate analytic samples for analysis. For each sample, we construct weights to correct for bias due to potentially nonrandom variation in data reporting across the popu


kers in California, not just those who pass this claim administrator screen, we construct weights to match the target distribution described above. To construct these weights, we first partition our data into mutually exclusive categories defined by


A limitation of our reliance on DEU data is that the population of workers who are rated at DEU may not be representative of the full population of workers with PD. Furthermore, because PD may take many years to emerge after injury, even the WCIS cl


osis
93
734; 735
Legg-Perthes disease; osteoporosis, spontaneous fracture, other diseases of the
781
Such symptoms as leg cramps, leg pain, muscle pain, joint pain
Carpal tunnel syndrome (CTS)
Table A.4 reports a cross-tabulation of this definition a


valence of the specific types of diagnoses used in our diagnosis- based definition for the sample of claims that do versus do not meet the FROI-based definition. The distribution of diagnoses is very similar across columns with one exception: workers


 advantage given our study’s focus on trends in case mix and outcomes.
Additional Details on Methods for Earnings Loss Estimation (Chapter 4)
Our primary estimation sample for analysis of earnings losses includes injuries between 2005 and 2015 matche


stimate outcomes for indemnity injuries relative to the medical-only control group. We use Poisson regression to accommodate the heavily right-skewed distribution of earnings. This regression model can be written as follows:
yit exp(Indemnityi 


ional Details on Disability Evaluation Unit Data Construction (Chapter 6)
DIR provided the RAND research team with data on DEU ratings. A member of the research team (Neuhauser) processed these data and produced a file with information on up to six r


w final rating using the formula set forth in the Permanent Disability Rating Schedule for combining multiple ratings.
We linked the WCIS data described in earlier chapters to DEU so that we could maintain consistency with the definitions of MSDs and


e do not believe there is evidence of changes over time in selection into DEU that would bias analysis of changes in ratings after SB 863.
In light of these issues, we do not attempt to reweight DEU records to obtain representative estimates for the


ess/eval; Physical therapist assess/eval; Physical therapist assess/eval; Physical therapist assess/eval; Physical therapist assess/eval; Physical therapist assess/eval; PT evaluation
PT reevaluation
OT evaluation
OT reevaluation Therapeutic exercise


eiving Any Capped Services, by Injury Type, Service Type, and Occupation, Alternative Service Definition
Other Private- Active Public Sector
 Type of Injury Firefighters
Active Police Sector Comparison
 Chiropractic
 Non-MSD 8.7 9.5
9.2 8.2 9.0 7.7
8


esponds to the time limit for LC §4850 benefits.
To evaluate the robustness of our analysis to this form of misreporting, we produced earnings loss estimates focusing solely on the second year after injury, after the expiration of LC §4850 time. Thes


)
–0.150*** (0.00509)
–0.145*** (0.00490)
–0.171*** (0.0235)
–0.164*** (0.0232)
 Figure B.1. Injured and Control Worker Earnings to Three Years Postinjury, by Occupation
         107
Additional Results for Disability Rating Analysis (Chapter 6)
Table


 occupations. Particularly sharp increases were apparent for firefighters and other public-sector workers with summary ratings.
Table B.7. Almaraz Ratings by Injury Date and Occupation, Consult Ratings
 Injury Date
2005–2012
2013–2015
2005–2015 Chang


Berninger, Amy, Mayris P. Webber, Hillel W. Cohen, Jackson Gustave, Roy Lee, Justin K. Niles, Sydney Chiu, Rachel Zeig-Owens, Jackie Soo, Kerry Kelly, and David J. Prezant, “Trends of Elevated PTSD Risk in Firefighters Exposed to the World Trade Cent


s.cdc.gov/view/cdc/39008
Department of Industrial Relations, California Safety Officer Workers’ Compensation Cancer Presumption—Impact of AB 1035, San Francisco, Calif.: Department of Industrial Relations, 2014.
DIR—See Department of Industrial Relat


org/10.1136/OEMED-2016-104015
Haynes, Hylton J. G., and Joseph L. Molis, United States Firefighter Injuries—2016, Quincy, Mass.: National Fire Protection Association, 2017.
Hoge, Charles W., Carl A. Castro, Stephen C. Messer, Dennis McGurk, Dave I. C


 University of Montana, Missoula, Mont: ScholarWorks, Graduate Student Theses, Dissertations, & Professional Papers, January 2017, 11059. As of July 29, 2019: https://scholarworks.umt.edu/etd/11059
116
Seabury, Seth A., and Christopher F. McLaren, Th


Industrial Cancers in California’s Workers’ Compensation System
Evidence on Earnings Losses and Disability Benefits
MICHAEL DWORSKY, CAROLYN M. RUTTER
  SOCIAL AND ECONOMIC WELL-BEING
Prepared for the California Department of Industrial Relations
For


orm the ongoing debate over compensation for industrial cancer in California. In this report, we provide new evidence on the following questions:
1. What level of earnings losses is experienced by workers who file workers’ compensation claims for ind


of Injury Under Alternative Case Definitions, by Occupation ....................................................................... 62
viii
Summary
California’s workers’ compensation system requires employers to provide medical care and disability (o


legislative attention. Much of this controversy stems from concerns about the fairness of permanent partial disability (PPD) benefits for workers with occupational cancer. Permanent disability (PD) compensation in California is based on impairment ra


ing of demographically identical workers in the same occupations and industries who filed minor, or medical-only, workers’ compensation claims expected to result in minimal long-term earnings losses. We used claims data to calculate the proportion of


and detection rise with age. They are also overwhelmingly male: just 13.6 percent of workers with cancer claims in our sample were female, while 41.2 percent of workers with a non-cancer claim were female. This gender breakdown largely reflects the


different characteristics from others with workers’ compensation claims, there are also important differences in demographics, benefit receipt, and claim denial rates across occupational groups (Table S.2). Lifeguards with cancer are substantially y


d workers to access compensation for occupational cancers. However, other explanations for this pattern cannot be ruled out without more detailed information on primary cancer site or stage at diagnosis; such information was not available in the clai


an half the level observed among peace officers and firefighters before the claim, their losses in dollar terms ($43,000) also exceed those for any occupation group examined in this study.
xiv
Figure S.1. Earnings Losses After Cancer Claims, by Occup


 firefighters.
Table S.3. Pretax Wage Replacement Rates over Five Years, by Occupation
          5-year earnings losses (extrapolated) (Standard error)
Average benefit payments
Pretax wage replacement rates
Peace Officers
$40,826 ($5,205) $19,860 49%


im filing for public safety workers. Even for peace officers and firefighters, however, rates of full or partial denial on cancer claims are several times higher than those observed for the average workers’ compensation claim. Detailed examination of


atively high (86 percent) for firefighters, but was far lower for peace officers (49 percent) and lower still for workers in other occupations (39 percent). Workers in these other occupation groups had high disability ratings and also had the highest


with cancer site and staging. Linking workers’ compensation data to group health claims with a consortium of large public employers or, potentially, a retiree health care system would dramatically improve the credibility of our claims-based case asce


artial disability
PTD permanent total disability
SB Senate Bill
SROI Subsequent Report of Injury
TD temporary disability
TTD temporary total disability
WCIS Workers’ Compensation Information System
 xxi
1. Introduction
As in other states, California’


e:
1. Do the standards for determining impairment due to occupational injury or illness accurately reflect the level of impairment caused by industrial cancer?
 1
2. Study and compare the differences between the fifth and sixth editions of the Americ


examination of the AMA Guides and the rating process in California in relation to industrial cancer claims” (DIR, 2019, pp. 4–5).
 2
three days or that requires an inpatient hospital admission. Workers with a temporary condition will collect weekly t


its are thus infrequently used. We estimate that 1.0 percent of cancer cases receive PTD benefits without also receiving PPD benefits. This group accounts for 3.4 percent of workers with PD (i.e., those with either PPD or PTD benefits). In this repor


stantial reduction in the size of PD ratings: the adoption of the AMA Guides led to a reduction because the average rating in the AMA Guides is lower than the rating assigned to a similar impairment evaluated using the pre-2005 PDRS. Neuhauser (2007)


ept those whose impairments already had an adjustment factor of 1.4.4 Second, the maximum weekly wage used to calculate PPD benefits was raised for the first time since 2006. Workers injured between January 1, 2006, and December 31, 2012, were eligib


eases arising out of and in the course of employment, including cancer. The most likely mechanism for work to cause cancer is exposure to carcinogens in the workplace or while carrying out job duties. The National Institute of Occupational Safety and


of workers with occupational disease filed a workers’ compensation claim (Biddle et al., 1998). We note that these estimates were derived from data on cases identified between 1992 and 1994 in Michigan, and so the external validity of these findings


care available to workers with cancer.
8
2. Data and Methods for Studying Earnings Loss and Benefits Due to Occupational Cancer
This chapter provides an overview of the data sources used in this study and our analytic methods. Our analysis of earning


the state Employment Development Department (EDD) to measure postinjury labor market outcomes.
The WCIS FROI/SROI and EDD data used in this study, which are quite complex, are described at length in Chapter 2 and Appendix A of Dworsky, Rennane, and B


e reported to claim administrator), worker demographics (i.e., age and sex), job and employer characteristics
10
(i.e., weekly wage, class code, and location), and information about the type of injury (nature, cause, and body part of injury). After a


d the resulting disability rating. To supplement the WCIS, we will use data on ratings from the California DEU. DEU is a state agency that performs between 60,000 and 80,000 ratings of permanent disabilities each year. The DEU data contain specific i


ultiple injuries when other occupational injuries have occurred or occupational diseases are present.
To address the shortcomings of our primary case definition, we conducted sensitivity analyses in which we used medical bills reported to the WCIS to


officer occupation group beyond police and sheriff’s deputies. However, this initial approach led to a small, but nontrivial number of public safety workers (primarily peace officers with less common job titles, such as special agents or investigato


efighter workforce remains overwhelmingly male, and there were too few cancer claims from female lifeguards to provide informative estimates of gender differences; the other occupations group, meanwhile, had a higher proportion of female workers with


trate reliable reporting of the SROI.5 We dropped 1,283 cases due to missing data or because they were submitted by claims administrators that systematically failed to report benefit payments and other claim outcomes, leaving 4,156 cases with complet


ation was motivated by concerns about cases in the other occupations category that appeared likely to be misclassified public safety workers or were otherwise ambiguous, it is not surprising that many of the cases dropped were in the other occupation


Agriculture, Forestry, Fishing and Hunting (11)
Mining, Quarrying, and Oil and Gas Extraction (21)
Utilities (22)
Construction (23)
Manufacturing (31–33)
Wholesale Trade (42)
Retail Trade (44–45)
Transportation and Warehousing (48–49)
Information (5


ensation claims other than cancer result in payment of death benefits, versus 2.8 percent of cancer claims.8
The above calculations include all cancer claims filed, including denied claims. Table 2.1 also makes it clear that cancer claims are subject


mographics prevented us from obtaining meaningful estimates of gender differences in postinjury outcomes for firefighters or lifeguards.
Table 2.2. Demographics, Benefit Receipt, and Employer Characteristics of Workers’ Compensation Claims, by Cancer


is unsurprising given the severity of cancer, we found major differences across occupations in the probability that claims resulted in paid death benefits. We did not observe paid death benefits for any lifeguards in our sample, while 2.4 percent of


the availability and scope of legal presumptions for cancer. Presumptions for lifeguards are limited to skin cancer, which can frequently be treated with limited side effects. It is therefore unsurprising that lifeguards with cancer claims experienc


main continuously employed, the inclusion of workers who leave employment after the quarter of injury outweighs wage growth (see Dworsky et al., 2016, for an example). Furthermore, wage growth and the risk of job loss are highly sensitive to economic


hich was also focused narrowly on injuries and labor market outcomes for firefighters and other groups of public safety workers.
The great advantage of using workers with medical-only injuries is that the filing of a workers’ compensation claim provi


ith the same type of employer). For other occupations, we control for the two-digit NAICS code (e.g., manufacturing, utilities, construction, or health care). We include similar controls for seven categories of employer size at the time of injury, wh


r terms could be misleading. We use the Poisson regression estimates when we describe earnings losses in percentage terms.
Methods for Calculating Pretax Wage Replacement Rates
Estimating wage replacement rates for this study posed some additional ch


ctions in earnings in the years following a cancer claim. Employment rates for these groups of workers are reported in the second panel of the table. Compared with workers in other occupations, public safety workers have very high employment rates tw


presents these event-study estimates by plotting two lines for each occupation. The solid colored lines with markers show the actual earnings of workers with cancer claims in each of the four occupation groups.3 The light gray dashed line plotted fo


e pattern observed for lifeguards reflects the seasonality of lifeguard employment; this distinctive pattern underscores the importance for the present study of using a control group of workers with medical-only injuries whose occupations can be obse


cers and firefighters before the claim, their losses in dollar terms ($43,000) exceed those for any occupation group examined in this study.
Table 3.2 summarizes the earnings losses depicted in Figure 3.1 by reporting the results from our regression


aluate the robustness of our estimates to reverse causation associated with claims that are filed with a substantial delay, we estimated our main regression models on a sample that excluded all claims (both cancer claims and medical-only claims) file


in substantive terms and is consistent with our expectation that SB 863 would have increased final ratings substantially. However, the sample of ratings performed at DEU was limited, and this difference was not statistically significant either.8
7 Mo


nefit receipt and average paid amounts for those receiving benefits differ sharply between public safety workers and those in other occupations. TTD payments for those receiving TTD benefits averaged $47,197 for peace officers and $58,326 for firefig


cross occupations for the vast majority
9 As noted above, we observed no lifeguards who received TTD benefits for cancer claims. 33
 of workers; while SB 863 raised the maximum weekly PD benefit substantially between 2012 and 2014, most workers with


6, we compare benefit payments with earnings losses in order to derive a measure of the pretax wage replacement rate provided to the average worker with a cancer claim in each
10 Unfortunately, we did not have access to any other reliable source of d


the second year after injury for workers injured between 2005 and 2008, for whom five-year losses were directly observable. This ratio was 4.74, meaning that the dollar value of losses over five years after injury is 4.74 times the value of losses in


05), they found that less-educated cancer patients experienced more severe earnings and employment losses. Occupational differences between blue-collar and white-collar workers were also substantial.
Other studies that did not provide longitudinal es


ts in Michigan. Bradley et al. (2007) found substantial reductions in employment at 6 months for both breast cancer patients (17-percentage-point reduction) and prostate cancer patients (10-percentage-point reduction), but that reductions in employme


table that we find persistently lower employment for all occupation groups other than lifeguards, when the U.S.-based studies on outcomes at two or more years after diagnosis generally do not find strong evidence of reduced employment. The peace offi


generous for patients who remain cancer-free, but inadequate for patients who experience recurrence or new cancers in the future. While the potential for long-term complications that lead to reduced work capacity in the future is relevant for a wide


              Annual earnings
Employment
Number of cases
2 years before injury 1 year before injury 1 year after injury
2 years after injury
2 years before injury 1 year before injury 1 year after injury
2 years after injury
Male
$116,880 $119,724 $1


for disability ratings and benefit payments conditional on benefit receipt by gender.
43
seem to suggest that female peace officers have, on average, slightly higher standard and final ratings compared with male peace officers. However, no gender dif


ficance of difference between male and female workers in each occupation. Significance of differences in percentage with benefits was assessed using a chi-squared test. Significance of differences in dollar amounts was assessed using a two-sample t-t


by the presumptions. Even for peace officers and firefighters, however, rates of full or partial denial on cancer claims are several times higher than those observed for the average workers’ compensation claim. Detailed examination of delays in benef


likely to contain more women. For the time being, however, the small number of women who filed workers’ compensation claims for cancer poses a major challenge to research on the role of gender in the workers’ compensation system’s treatment of cance


d with other injuries or diseases. Unfortunately, our inability to capture medical claims and diagnoses outside workers’ compensation (most notably from workers’ group health coverage) means that our case definitions based on medical claims are almos


to pension eligibility. However, multiple cancer screening guidelines are keyed to ages 40, 45, and 50, and these guidelines generate sharp increases in screening that are likely to translate into increased detection at exactly the same ages when pu


there were many cases with ambiguous occupation descriptions that we were unable to conclusively assign to one of our four occupational categories. When possible, we followed the approach taken in Dworsky, Seabury, and Broten (2019), which used occu


 of cases
Number of unique occupation titles
Percentage of Cases
Police officer
Deputy sheriff
Officer 11.9
Percentage of Cases
12.98
Percentage of Cases
40.79
Percentage of Cases
1.93
1.45
1.45
1.45
1.45
1.45
1.45 1.45 0.97
0.97
Peace Officer
Firefi