Input:
	OJJDP Established the Girls Study Group to Assess the Effectiveness of Girls’ Delinquency Programs

With an overall goal of developing research that communities need to  make sound decisions about how best to prevent and reduce girls’  delinquency, OJJDP established the Girls Study Group (Study Group) in  2004 under a $2.6 million multiyear cooperative agreement with a research  institute. OJJDP’s objectives for the group, among others, included  identifying effective or promising programs, program elements, and  implementation principles (i.e., guidelines for developing programs).  Objectives also included developing program models to help inform  communities of what works in preventing or reducing girls’ delinquency,  identifying gaps in girls’ delinquency research and developing  recommendations for future research, and disseminating findings to the  girls’ delinquency field about effective or promising programs. To meet  OJJDP’s objectives, among other activities, the Study Group identified  studies of delinquency programs that specifically targeted girls by  reviewing over 1,000 documents in relevant research areas. These included  criminological and feminist explanations for girls’ delinquency, patterns of  delinquency, and the justice system’s response to girls’ delinquency. As a  result, the group identified 61 programs that specifically targeted  preventing or responding to girls’ delinquency. Then, the group assessed  the methodological quality of the studies of the programs that had been  evaluated using a set of criteria developed by DOJ’s Office of Justice  Programs (OJP) called What Works to determine whether the studies  provided credible evidence that the programs were effective at preventing  or responding to girls’ delinquency. The results of the group’s assessment  are discussed in the following sections.

	OJJDP Efforts to Assess Program Effectiveness Were Consistent with Social Science Practices and Standards, and OJJDP Has Taken Action to Enhance Communication about the Study Group with External Stakeholders

OJJDP’s effort to assess girls’ delinquency programs through the use of a  study group and the group’s methods for assessing studies were consistent  with generally accepted social science research practices and standards.  In addition, OJJDP’s efforts to involve practitioners in Study Group  activities and disseminate findings were also consistent with the internal  control standard to communicate with external stakeholders, such as  practitioners operating programs.
According to OJJDP research and program officials, they formed the Study  Group rather than funding individual studies of programs because study  groups provide a cost-effective method of gaining an overview of the  available research in an issue area. As part of its work, the group  collected, reviewed, and analyzed the methodological quality of research  on girls’ delinquency programs. The use of such a group, including its  review, is an acceptable approach for systematically identifying and  reviewing research conducted in a field of study. This review helped  consolidate the research and provide information to OJJDP for  determining evaluation priorities. Further, we reviewed the criteria the  group used to assess the studies and found that they adhere to generally  accepted social science standards for evaluation research. We also  generally concurred with the group’s assessments of the programs based  on these criteria. According to the group’s former principal investigator,  the Study Group decided to use OJP’s What Works criteria to ensure that  its assessment of program effectiveness would be based on highly rigorous  evaluation standards, thus eliminating the potential that a program that  may do harm would be endorsed by the group. However, 8 of the 18  experts we interviewed said that the criteria created an unrealistically high  standard, which caused the group to overlook potentially promising  programs. OJJDP officials stated that despite such concerns, they  approved the group’s use of the criteria because of the methodological  rigor of the framework and their goal for the group to identify effective  programs.
In accordance with the internal control standard to communicate with  external stakeholders, OJJDP sought to ensure a range of stakeholder  perspectives related to girls’ delinquency by requiring that Study Group  members possess knowledge and experience with girls’ delinquency and  demonstrate expertise in relevant social science disciplines. The initial  Study Group, which was convened by the research institute and approved  by OJJDP, included 12 academic researchers and 1 practitioner; someone  with experience implementing girls’ delinquency programs. However, 11 of  the 18 experts we interviewed stated that this composition was  imbalanced in favor of academic researchers. In addition, 6 of the 11 said  that the composition led the group to focus its efforts on researching  theories of girls’ delinquency rather than gathering and disseminating  actionable information for practitioners. According to OJJDP research  and program officials, they acted to address this issue by adding a second  practitioner as a member and involving two other practitioners in study  group activities. OJJDP officials stated that they plan to more fully involve  practitioners from the beginning when they organize study groups in the  future and to include practitioners in the remaining activities of the Study  Group, such as presenting successful girls’ delinquency program practices  at a national conference. Also, in accordance with the internal control  standard, OJJDP and the Study Group have disseminated findings to the  research community, practitioners in the girls’ delinquency field, and the  public through conference presentations, Web site postings, and published  bulletins. The group plans to issue a final report on all of its activities by  spring 2010.

	The Study Group Found No Evidence of Effective Girls’ Delinquency Programs; in Response OJJDP Plans to Assist Programs in Preparing for Evaluations but Could Strengthen Its Plans for Supporting Such Evaluations

The Study Group found that few girls’ delinquency programs had been  studied and that the available studies lacked conclusive evidence of  effective programs; as a result, OJJDP plans to provide technical  assistance to help programs be better prepared for evaluations of their  effectiveness. However, OJJDP could better address its girls’ delinquency  goals by more fully developing plans for supporting such evaluations.
In its review, the Study Group found that the majority of the girls’  delinquency programs it identified—44 of the 61—had not been studied by  researchers. For the 17 programs that had been studied, the Study Group  reported that none of the studies provided conclusive evidence with which  to determine whether the programs were effective at preventing or  reducing girls’ delinquency. For example, according to the Study Group,  the studies provided insufficient evidence of the effectiveness of 11 of the  17 programs because, for instance, the studies involved research designs  that could not demonstrate whether any positive outcomes, such as  reduced delinquency, were due to program participation rather than other  factors. Based on the results of this review, the Study Group reported that  among other things, there is a need for additional, methodologically  rigorous evaluations of girls’ delinquency programs; training and technical  assistance to help programs prepare for evaluations; and funding to  support girls’ delinquency programs found to be promising.
According to OJJDP officials, in response to the Study Group’s finding  about the need to better prepare programs for evaluation, the office plans  to work with the group and use the remaining funding from the effort— approximately $300,000—to provide a technical assistance workshop by  the end of October 2009. The workshop is intended to help approximately  10 girls’ delinquency programs prepare for evaluation by providing  information about how evaluations are designed and conducted and how  to collect data that will be useful for program evaluators in assessing  outcomes, among other things. In addition, OJJDP officials stated that as a  result of the Study Group’s findings, along with feedback they received  from members of the girls’ delinquency field, OJJDP plans to issue a  solicitation in fiscal year 2010 for funding to support evaluations of girls’  delinquency programs.
OJJDP has also reported that the Study Group’s findings are to provide a  foundation for moving ahead on a comprehensive program related to girls’  delinquency. However, OJJDP has not developed a plan that is  documented, is shared with key stakeholders, and includes specific  funding requirements and commitments and time frames for meeting its  girls’ delinquency goals. Standard practices for program and project  management state that specific desired outcomes or results should be  conceptualized, defined, and documented in the planning process as part  of a road map, along with the appropriate projects needed to achieve those  results, supporting resources, and milestones. In addition, government  internal control standards call for policies and procedures that establish  adequate communication with stakeholders as essential for achieving  desired program goals. According to OJJDP officials, they have not  developed a plan for meeting their girls’ delinquency goals because the  office is in transition and is in the process of developing a plan for its  juvenile justice programs, but the office is taking steps to address its  girls’ delinquency goals, for example, through the technical assistance  workshop. Developing a plan for girls’ delinquency would help OJJDP to  demonstrate leadership to the girls’ delinquency field by clearly  articulating the actions it intends to take to meet its goals and would also  help the office to ensure that the goals are met.
In our July report, we recommended that to help ensure that OJJDP meets  its goals to identify effective or promising girls’ delinquency programs and  supports the development of program models, the Administrator of OJJDP  develop and document a plan that (1) articulates how the office intends to  respond to the findings of the Study Group, (2) includes time frames and  specific funding requirements and commitments, and (3) is shared with  key stakeholders. OJP agreed with our recommendation and outlined  efforts that OJJDP plans to undertake in response to these findings. For  example, OJJDP stated that it anticipates publishing its proposed juvenile  justice program plan, which is to include how it plans to address girls’  delinquency issues, in the Federal Register to solicit public feedback and  comments, which will enable the office to publish a final plan in the  Federal Register by the end of the year (December 31, 2009).
Mr. Chairman, this concludes my statement. I would be pleased to respond  to any questions that you or other Members of the Subcommittee may  have.

	Contacts and Acknowledgements

For questions about this statement, please contact Eileen R. Larence at  (202) 512-8777 or larencee@gao.gov. Contact points for our Offices of  Congressional Relations and Public Affairs may be found on the last page  of this statement. Individuals making key contributions to this statement  include Mary Catherine Hult, Assistant Director; Kevin Copping; and  Katherine Davis. Additionally, key contributors to our July 2009 report  include David Alexander, Elizabeth Blair, and Janet Temko.
This is a work of the U.S. government and is not subject to copyright protection in the  United States. The published product may be reproduced and distributed in its entirety  without further permission from GAO. However, because this work may contain  copyrighted images or other material, permission from the copyright holder may be  necessary if you wish to reproduce this material separately.

Output:
Why GAO Did This Study

This testimony discusses issues related to girls' delinquency--a topic that has attracted the attention of federal, state, and local policymakers for more than a decade as girls have increasingly become involved in the juvenile justice system. For example, from 1995 through 2005, delinquency caseloads for girls in juvenile justice courts nationwide increased 15 percent while boys' caseloads decreased by 12 percent. More recently, in 2007, 29 percent of juvenile arrests--about 641,000 arrests--involved girls, who accounted for 17 percent of juvenile violent crime arrests and 35 percent of juvenile property crime arrests. Further, research on girls has highlighted that delinquent girls have higher rates of mental health problems than delinquent boys, receive fewer special services, and are more likely to abandon treatment programs. The Office of Juvenile Justice and Delinquency Prevention (OJJDP) is the Department of Justice (DOJ) office charged with providing national leadership, coordination, and resources to prevent and respond to juvenile delinquency and victimization. OJJDP supports states and communities in their efforts to develop and implement effective programs to, among other things, prevent delinquency and intervene after a juvenile has offended. For example, from fiscal years 2007 through 2009, Congress provided OJJDP almost $1.1 billion to use for grants to states, localities, and organizations for a variety of juvenile justice programs, including programs for girls. Also, in support of this mission, the office funds research and program evaluations related to a variety of juvenile justice issues. As programs have been developed at the state and local levels in recent years that specifically target preventing girls' delinquency or intervening after girls have become involved in the juvenile justice system, it is important that agencies providing grants and practitioners operating the programs have information about which of these programs are effective. In this way, agencies can help to ensure that limited federal, state, and local funds are well spent. In general, effectiveness is determined through program evaluations, which are systematic studies conducted to assess how well a program is working--that is, whether a program produced its intended effects. To help ensure that grant funds are being used effectively, you asked us to review OJJDP's efforts related to studying and promoting effective girls' delinquency programs. We issued a report on the results of that review on July 24, 2009. This testimony highlights findings from that report and addresses (1) efforts OJJDP has made to assess the effectiveness of girls' delinquency programs, (2) the extent to which these efforts are consistent with generally accepted social science standards and federal standards to communicate with stakeholders, and (3) the findings from OJJDP's efforts and how the office plans to address the findings. This statement is based on our July report and selected updates made in October 2009.

What GAO Found

With an overall goal of developing research that communities need to make sound decisions about how best to prevent and reduce girls' delinquency, OJJDP established the Girls Study Group (Study Group) in 2004 under a $2.6 million multiyear cooperative agreement with a research institute. OJJDP's objectives for the group, among others, included identifying effective or promising programs, program elements, and implementation principles (i.e., guidelines for developing programs). Objectives also included developing program models to help inform communities of what works in preventing or reducing girls' delinquency, identifying gaps in girls' delinquency research and developing recommendations for future research, and disseminating findings to the girls' delinquency field about effective or promising programs. OJJDP's effort to assess girls' delinquency programs through the use of a study group and the group's methods for assessing studies were consistent with generally accepted social science research practices and standards. In addition, OJJDP's efforts to involve practitioners in Study Group activities and disseminate findings were also consistent with the internal control standard to communicate with external stakeholders, such as practitioners operating programs. The Study Group found that few girls' delinquency programs had been studied and that the available studies lacked conclusive evidence of effective programs; as a result, OJJDP plans to provide technical assistance to help programs be better prepared for evaluations of their effectiveness. However, OJJDP could better address its girls' delinquency goals by more fully developing plans for supporting such evaluations.


Input:
	Background

For more than 30 years, federal law has provided comprehensive health coverage for low-income children through Medicaid. The children eligible for such care have made up a significant and growing portion of the nation’s population, as eligibility for Medicaid benefits has expanded to cover increasing numbers of previously uninsured children. In 1998, Medicaid covered more than one-third of young children ages 0 through 5, and more than one-fourth of children under age 21 (see figure 1). The 21 million children covered by Medicaid that year composed slightly more than half of the 41 million people in the program while the $32 billion spent for their care was about 23 percent of the $142 billion spent on the program by the federal government and states.
An increasing number of children are also becoming eligible for EPSDT services, as federal policy designed to cover the growing number of uninsured children allows states to provide Medicaid services through the federally supported State Children’s Health Insurance Program (SCHIP). To implement SCHIP, states have the option of expanding their Medicaid programs, developing separate SCHIP programs, or doing some combination of both. If a state elects Medicaid expansion, it must offer the same comprehensive benefit package, including EPSDT services, to SCHIP beneficiaries as it does to Medicaid beneficiaries. In 2000, more than 1 million children were enrolled in SCHIP Medicaid expansion programs and were therefore also eligible for EPSDT services.

		EPSDT Services Under Medicaid Are Comprehensive

Although many coverage, eligibility, and administrative decisions are left to individual states, the federal government sets certain requirements for state Medicaid programs. Coverage of screening and necessary treatment for children is one of these requirements. EPSDT components are designed to target health conditions and problems for which growing children are at risk, including iron deficiency, obesity, lead poisoning, and dental disease. They are also intended to detect and correct conditions that can hinder a child’s learning and development, such as vision and hearing problems. For many children, especially those with special needs because of disabilities or chronic conditions, EPSDT is an important help in identifying the need for essential medical and supportive services, and in making these services available.
The federally required EPSDT components that constitute an EPSDT “screen” include a comprehensive health and developmental history, a comprehensive unclothed physical exam, appropriate immunizations, laboratory tests (including a blood lead-level assessment), and health education. Other required EPSDT services include   vision services, including diagnosis, treatment, and eyeglasses;   dental services, including relief of pain and infections, restoration, and   hearing services, including diagnosis, treatment, and hearing aids; and   services for other conditions discovered through screenings, regardless of whether these services are typically covered by the state’s Medicaid plan for other beneficiaries.
While state Medicaid programs must cover EPSDT, they have some flexibility in determining the frequency and timing of screens. States develop, in consultation with recognized medical and dental organizations, their own “periodicity schedules,” which contain age-specific timetables that identify when physical examinations and certain laboratory tests and immunizations should occur. These tables vary somewhat from state to state. For example, the number of recommended EPSDT screens ranged from 15 to 29 across the five states we visited (see table 1).

		The Growth of Medicaid Managed Care

States have increasingly turned to managed care as a way to deliver Medicaid services, including EPSDT. From 1991 to 1999, the proportion of all Medicaid beneficiaries enrolled in managed care—either capitated or in primary care case management models—rose from about 10 percent to about 56 percent. Only two states do not have at least some Medicaid beneficiaries in managed care plans.
Managed care, with its emphasis on preventive and primary care, is philosophically an ideal model for delivering EPSDT-type services. Under a capitated managed care model, states contract with managed care plans, such as health maintenance organizations, and pay a fixed monthly fee per Medicaid enrollee (a capitated fee) to provide most medical services. This model, with its fixed prospective payment for a package of services, creates an incentive for plans to provide preventive and primary care to reduce the chance that beneficiaries will require more expensive treatment services in the future. However, capitated managed care can also create a financial incentive to underserve or deny beneficiaries access to needed care. Moreover, Medicaid beneficiaries required to enroll in managed care may find it difficult to seek alternative care if their plan providers fail to meet their needs. Because of the potential to underserve, states must build in safeguards and accountability measures, such as grievance and appeals processes, to ensure that beneficiaries receive appropriate care.
The Congress has given states greater flexibility in moving Medicaid beneficiaries into mandatory managed care plans. Before the Balanced Budget Act (BBA) of 1997, a state could require Medicaid beneficiaries to enroll in managed care only if it first obtained approval from HCFA to waive certain statutory provisions, such as the freedom to choose providers. Under HCFA waivers, states have implemented a variety of mandatory managed care programs, ranging from programs serving limited populations in just a few counties to state-wide programs covering all Medicaid beneficiaries, including children with special needs. The BBA gave states new flexibility in implementing mandatory Medicaid managed care programs, allowing them to implement programs through an amendment to their state Medicaid plan without first obtaining a HCFA waiver.

		States Are Required to Report on the Delivery of EPSDT Services

The Omnibus Budget Reconciliation Act of 1989 (OBRA 89) made significant changes to improve the provision of EPSDT services to children in Medicaid. It required that the Secretary of HHS set state- specific annual goals for children’s participation in EPSDT; mandated state-established periodicity schedules for screening in dental, vision, and hearing services; required blood lead assessments appropriate for age and risk factors; and imposed new reporting requirements.
To fulfill the state-specific goal requirement, in 1990 HCFA set a participation goal of 80 percent by 1995 for every state. To measure progress towards participation goals and in accordance with the OBRA 89 requirement that states report certain EPSDT statistics, HCFA required, starting in 1990, that states submit annual EPSDT reports (known as the form 416). The EPSDT report captures, by age group, the number of children who (1) received EPSDT health screens; (2) were referred for corrective treatment; (3) received dental treatment or preventive services; and (4) were enrolled in managed care plans. Since fiscal year 1999, states also are required to report the number of blood tests provided to screen children for lead poisoning.

		Legal Settlements Highlight Challenges in Many States

Lawsuits have been filed in many states alleging shortcomings in the provision of EPSDT services. According to information from the National Health Law Program, at least 28 states have been sued by beneficiaries or advocates since 1995 for failing to provide required access to EPSDT services. These lawsuits range from single-issue suits—such as coverage of selected services including mental health services in Maine—to alleged programwide failures and deficiencies in Texas, Tennessee, and Washington, D.C. In several instances, the outcomes, including court orders and settlements agreed to by both parties to remedy known concerns, illustrate the difficulties states have encountered in providing services and also suggest strategies to remedy established EPSDT deficiencies.

	Limited Available Data Indicate Many Children Do Not Receive EPSDT Services

Despite statutory reporting requirements, reliable national data are not available on the extent to which children in Medicaid are receiving EPSDT services. However, a number of studies of limited scope indicate that many children in Medicaid are not receiving EPSDT services. These studies also show that several factors are at work in limiting the successful delivery of EPSDT services. Some factors are program-related, such as a lack of providers or systems to ensure access to covered services. Others are related to beneficiaries themselves, such as the beneficiaries’ lack of awareness about the importance of preventive health care and about services covered, or their difficulty in maintaining continuity of care with one provider.

		Reliable and Comprehensive National Data Do Not Exist

HCFA’s efforts to assemble reliable information about EPSDT participation in each state have so far been unsuccessful. State-reported data, upon which HCFA depends, are often not timely or accurate. For example, states were required to submit their fiscal year 1999 reports by April 1, 2000. As of January 2001, 15 states had not submitted their 1999 reports and another 15 states’ reports had been returned by HCFA because they were deficient. HCFA and state officials acknowledge long-standing difficulties that states face in their efforts to collect complete and reliable data, which are used as the basis for the EPSDT reports. These difficulties continue despite HCFA’s attempts to improve the reliability of state EPSDT reports by revising the report format and guidance.
One reason for the continued difficulty involves collecting data on EPSDT services provided under managed care. Under the more traditional fee-for- service approach, data on service delivery are often relatively easy to collect as part of the payment process because states pay providers for each service for which they bill the state. Under capitated managed care, however, states pay the managed care plan a prospective monthly per- enrollee fee that is not tied to the individual services provided. As a result, data on service utilization (often referred to as “encounter data”) are not necessarily captured. Instead, states have to rely on managed care plans to collect and report these data separately. Managed care plans, particularly those that also pay their participating providers on a capitated basis, often have difficulty collecting and reporting complete and accurate data.
States face continuing challenges in determining how to minimize the administrative burden on managed care plans and providers while still collecting information at the level needed to administer the program. For example, to facilitate the collection of EPSDT data, California uses a special EPSDT form for providers to use in documenting the components of EPSDT services provided. California’s managed care contracts also call for managed care plans to collect the EPSDT forms from their providers and submit detailed encounter data to the state. However, the state has had difficulty enforcing these requirements across the several layers of contractors involved in its managed care delivery system. For example, in the Los Angeles area, the state contracts with two large managed care organizations that subcontract with multiple commercial and nonprofit health plans, such as Blue Cross, that further subcontract with a network of providers. Most of these contracts are on a capitated basis. State officials said that some of the health plans had difficulty collecting the required encounter data and that one plan had never submitted the required data. Also, they said that capitated providers of health plans had little incentive to fill out and submit the EPSDT form because their payments are not linked to it. The state’s Medicaid agency has not imposed sanctions against noncompliant plans or providers, restrained in part by its reluctance to lose any providers given the shortage of providers willing to serve children in Medicaid.
Although problems are more extensive with managed care data than with fee-for-service data, most of the states we visited had some difficulty obtaining complete and accurate data from fee-for-service providers as well. Florida illustrates the kinds of difficulties that can be encountered. Providers in Florida are required to use a specific EPSDT code and a claim form to document the components of EPSDT services they provide. However, according to state officials, providers often choose to use other codes instead. For example, providers may submit a claim under a comprehensive office-visit code for a new patient that pays a higher rate than an EPSDT screen or they may submit claims under other comprehensive office-visit codes that require less documentation.
Compounding these difficulties are limitations in claims processing systems used by states for fee-for-service programs or by managed care plans. In Florida, for example, if a child receives laboratory work from one provider and the remaining components of a screening from another provider, some managed care plans’ data systems do not combine the services to correctly reflect that a full screening for the child has been provided. Similarly, some states have problems tracking referrals and follow-up treatment services. This tracking difficulty may explain why, in HCFA’s 1998 compilation of state reports, seven states reported that no children had been referred for corrective treatments.

		Some Studies Show Screening Rates Are Low

While HCFA’s data cannot present a reliable and comprehensive picture of the extent to which children in Medicaid receive EPSDT services, other studies indicate that many of these children are not receiving such services. These other studies have been narrow in scope, allowing analysts to overcome the kinds of problems that so far have thwarted attempts to gather comprehensive data. They have focused on specific EPSDT services or reviews of a sample of patients’ medical records. For example, in recent years we have conducted reviews of screening rates for lead poisoning and dental care, basing our analysis primarily on data from national health surveys. Both studies found low screening rates for these specific services among low-income populations served by Medicaid. For lead poisoning, about 19 percent of children in Medicaid aged 1 through 5 were screened—a serious concern, because these children are almost five times more likely than others to have a harmful blood lead level. The screening rate for potential dental problems was similar, with about 21 percent of low-income children aged 2 to 5 having had a dental visit in the previous year. Older children fared somewhat better, with 36 percent of low-income children aged 6 to 18 having had a dental visit within the previous year.
Studies by others have shown similar results. A 1997 study by HHS’ Office of Inspector General, which examined a sample of 338 children’s medical records from 12 health plans in 10 states, estimated that only 28 percent of children enrolled in Medicaid managed care received all prescribed EPSDT screens and that 60 percent received no screens at all. In several states, organizations responsible for external quality review of the Medicaid program have conducted sample medical record reviews of children enrolled in fee-for-service programs as well as those in managed care, and they have found similar results. For example, a study by Minnesota’s external quality review organization found that nearly half of the children in managed care plans whose files were reviewed had not visited a clinic in the previous year, and only 6 percent of those due for an EPSDT screen had received a comprehensive screen. A study in Washington State found that for the sampled files of children in managed care, 32 percent of infants (birth to 15 months) and 20 percent of children age 3 to 6 years received screenings for all aspects of EPSDT. The screening rates for children in fee-for-service care were also low—7 percent for infants and 24 percent for children age 3 to 6 years.

		Several Factors Contribute to Children Not Receiving Services

Studies such as those cited above have collectively identified a number of reasons why many children in Medicaid are not receiving EPSDT services. Some of these reasons involve program-related matters, such as limited provider participation in Medicaid. For example, low provider participation in Medicaid has been noted as a particular problem in dental and mental health. Our earlier study found that a shortage of dentists willing to treat Medicaid patients was the major factor contributing to the low use of dental services. Similarly, a study by the Economic and Social Research Institute for the Kaiser Commission on Medicaid and the Uninsured found shortages of mental health and substance abuse professionals willing to treat Medicaid patients.
Other program-related factors include inadequate methods for ensuring access to services. Our study of lead screening found problems with providers’ missing opportunities to perform follow-up tests when children returned for other care. Lawsuits brought in a number of states have also highlighted such problems as inadequate systems for informing beneficiaries about the availability of EPSDT services and poor coordination by managed care plans and state agencies. Several advocacy groups we interviewed echoed concerns that states and managed care plans do not adequately inform beneficiaries about the broad scope of EPSDT services or about beneficiary appeal rights. These groups also questioned the adequacy of the provider networks for serving children in Medicaid.
In addition to these program-related factors, some beneficiary-related factors have also been found to limit screening services. For example, many Medicaid beneficiaries change eligibility status over short periods of time, and they may move frequently, making it more difficult to maintain continuity in their medical care. Researchers have also found that parents whose children are eligible to receive services under Medicaid tend to be less aware of the importance of preventive care than the general population. Those who try to obtain preventive care face other barriers. In our reports on oral health and screening for lead poisoning, we noted several other contributing factors, such as difficulty in getting time off from work, finding child care, arranging transportation to the provider, and overcoming language differences. These factors may contribute to a higher rate of broken appointments—a major concern among providers, particularly dentists. An American Dental Association survey reported that about one-third of Medicaid patients failed to keep appointments. A 1999 study conducted for the Florida Medicaid agency found that the top three reasons given by survey respondents for missing pediatric appointments were not having a ride to the appointment, the child no longer being sick, and forgetting an appointment.

	States Have Taken Action to Improve Data Reporting and Delivery of EPSDT Services

The five states we visited have implemented a variety of initiatives intended to improve the provision of EPSDT services to children in Medicaid, including those in managed care. The state and health plan efforts we identified fall into three general categories: (1) improving data; (2) better ensuring that plans deliver services; and (3) improving beneficiary outreach and education. Although in most cases states and health plans could not provide information on their specific impact, these initiatives represented efforts that state and plan officials cited as helping to better ensure that children receive EPSDT services.

		Improved Data Reporting

The five states we visited have taken a number of steps to improve the quality of the data they collect—especially from managed care programs— to monitor the utilization of services and to compile EPSDT reports to HCFA. These steps have not yet solved the states’ data and reporting problems; however, by moving toward more timely and reliable encounter data, states can better assess progress toward participation goals, identify specific plans or providers experiencing problems, and target corrective measures. As table 2 shows, these steps involve four main types of actions: requiring plans to submit detailed encounter data, validating those data, linking data with other sources, and reporting summary data in print or on the Internet. For example, to encourage health plans to report complete and accurate data, and to publicize comparative data, New York publishes summary statistics on individual plans on its health department Web site.
These states’ experiences demonstrate that gathering complete and reliable encounter data is a long-term effort. Wisconsin, for example, worked collaboratively with capitated managed care plans for 4 years to formulate a uniform encounter data set and reporting system that all plans are required to use. Wisconsin’s system did not become functional until May 2000 and has not yet produced its first report to HCFA. New York has required managed care plans to submit encounter data for the past 6 years, but state Medicaid officials said the first few years of data were unreliable. The data became more reliable around the fourth year, after state officials worked with health plans to improve their data collection and verification efforts.

		Ensuring Service Delivery

States have also put into action a number of initiatives to help ensure that managed care plans and health providers deliver screening and treatment services to children enrolled in Medicaid. The broad package of benefits offered under EPSDT can result in confusion and potential under-service if health plans and providers are not clearly informed of their responsibilities to provide EPSDT services. In California, for example, officials said some health plans were not performing screens according to the state’s managed care periodicity schedule. Plan providers were confused, they said, because the state’s Medicaid fee-for-service periodicity schedule called for fewer screens than its managed care periodicity schedule (15 compared to 27) and physicians often served both fee-for-service and managed care patients. In addition, a recent HCFA- sponsored study of Medicaid managed care contracts in more than three dozen states found that states often fail to spell out the full range of EPSDT services that plans are responsible for providing. The study concluded, among other things, that while states routinely expect managed care plans to provide the full range of EPSDT service obligations, they do not always explain in contracts what this means and may not require contractors to educate beneficiaries about the benefit package offered under EPSDT.
To better ensure EPSDT service delivery, the states we visited have taken action in several areas (see table 3). Some of these actions have involved states’ laying out expectations for managed care plans or providers through extensive specification of responsibilities in contracts or provider education. Other actions have involved the monitoring of health plans, the use of incentives and sanctions for provision of services, and requirements for plans to coordinate care with public health departments. States have also increased reimbursement rates for EPSDT services. For example, in 1995, to encourage fee-for-service providers to screen more children, Florida more than doubled its reimbursement rate for a comprehensive EPSDT screen. The examples in table 3 represent a few of the promising actions these states and health plans have implemented.

		Improving Beneficiary Outreach and Education

The third area in which states have taken action is in educating and encouraging parents to better ensure that their children receive EPSDT services. Beneficiary outreach and education is typically a responsibility shared between the states and the health plans. At certain times in the process, the states may have primary responsibility for informing beneficiaries about covered services, such as when new beneficiaries are enrolled. Once a beneficiary is enrolled in a health plan, the state may require the plan to take measures to inform parents and families about covered services and how to access them. Officials from states and plans we visited reported a number of initiatives to better inform beneficiaries about EPSDT services (see table 4). These generally fell into four categories: designing clear and informative member handbooks, creating helpful and easy-to-understand materials to supplement member handbooks, developing programs to reach special populations such as children with disabilities, and conducting community outreach activities. For example, to encourage Medicaid beneficiaries, including those in managed care, to take advantage of preventive care, Florida mails reminder letters to families when their children are due for EPSDT screens.
In addition to these efforts in the five states we visited, children’s advocates also informed us that several states have implemented initiatives as part of settlement agreements arising from EPSDT-related lawsuits. Settlement documents and court orders from selected EPSDT lawsuits contain information on a number of state initiatives to improve delivery of EPSDT services. For example, Pennsylvania established a series of 18 performance standards and health outcome measures and incorporated them into managed care contracts. Standards and interim targets were established for the percentage of children to receive immunizations and EPSDT screens, and measures were established for treatment and prevention of asthma, anemia, and lead poisoning. Appendix II contains further information on the basis for selected lawsuits and actions taken by states in response.

	HCFA Efforts to Ensure Children’s Access to EPSDT Have Been Limited

HCFA, now called CMS, is currently reevaluating how best to carry out its role in helping to ensure that children receive access to EPSDT services. In recent years, HCFA’s efforts have focused largely on trying to improve the guidance to states about reporting the extent to which children are being screened. Attempts to improve reporting have been time-consuming, and progress has been slow. Because HCFA’s focus has been mainly on improving the format and specificity of the state EPSDT reports, it has placed little emphasis on the extent to which states are improving the underlying data or meeting HCFA’s EPSDT participation goals. At the regional office level, where much of the responsibility for working with states resides, a few offices have begun to help states identify problems and promote state progress in increasing children’s use of services. However, because most regional offices have focused their resources on priorities other than EPSDT, these efforts have not been widespread. In January 2001, HCFA’s central office proposed to regional offices and other stakeholders that the agency work more closely with states to improve both reporting and children’s use of services, but a specific plan for how to do so has not yet been developed.

		HCFA Is Acting to Improve State Reports, but Progress in Improving the Underlying Data Is Slow

Recognizing that progress in providing services is difficult to assess without good data as a starting point, HCFA has centered its monitoring efforts largely on revising the guidance and format in order to improve state EPSDT reports. These revisions were largely aimed at capturing more reliable and more consistent EPSDT information while minimizing the burden on states in completing the reports. For example, in 1999 HCFA changed the EPSDT report to, among other revisions, require new information on dental services and blood lead tests, and to add more precise definitions of certain required data elements. It also allowed states to use their own periodicity schedules to determine their participation and screening rates.
While these revisions have changed the reporting requirements, they have done little to address the continuing difficulties states face in their efforts to gather reliable and complete data. As our review of the five states showed, these problems require determined efforts at the state level, and because of the complexities associated with collecting managed care encounter data, such efforts take considerable time to accomplish. In the meantime, these EPSDT reports do not provide an accurate or complete picture of most state EPSDT programs, nor do they allow for reasonable national estimates of EPSDT screening and participation rates or for meaningful comparisons between states.
Although HCFA’s efforts to improve data collection are important, by themselves they do not represent a strategy for helping states meet EPSDT goals. In part because HCFA acknowledges the limitations of the state EPSDT reports, the agency has done little to address how well states are doing in meeting the goal of providing EPSDT services to 80 percent of children enrolled in Medicaid. The existing reports show that most states are considerably below this goal. However, even if issues regarding data and reporting are adequately addressed, improved EPSDT reports, taken alone, will not provide HCFA with sufficient program detail to perform other oversight duties, such as helping states identify and correct specific problems or share information on lessons learned from other states and model state practices.

		Studies by Some Regional Offices Have Identified Areas for Improvement and Innovative Practices

A few HCFA regional offices have conducted reviews of state EPSDT programs. HCFA regional officials reported to us that eight such studies have been completed since 1995. Four included EPSDT as one element of a broader review of a state’s Medicaid managed care program; four focused exclusively on EPSDT. While these EPSDT and managed care assessments varied widely in their methodology and coverage of EPSDT issues, they have helped illuminate policy and process concerns and innovative practices of states. They have also identified needed actions to improve children’s access to EPSDT care. For example: In Oklahoma, an EPSDT-focused study conducted jointly by HCFA’s Dallas Regional Office and state Medicaid officials found several ways to increase screening and improve the quality of data submitted. The team found that providers relied on a review of a child’s medical chart to determine whether an EPSDT screen was due—a step they generally took only when an office visit occurred. As a result, children not visiting for other reasons were often not screened. The study recommended that the state establish a system to notify providers when children were due for screens. The study team also found that Medicaid provider knowledge of EPSDT services varied widely, and that many providers did not know about a monetary bonus the state offered to those providers who increased, to 60 percent or more, the proportion of eligible children who had EPSDT screens. To increase provider awareness, the study team recommended that the state annually include a discussion of EPSDT at provider education meetings. In California, an EPSDT-focused study conducted by HCFA’s San Francisco Regional Office with the cooperation of state Medicaid officials found that families of children in Medicaid were not being effectively informed about the availability of services or how to gain access to them.
State officials who responded to the report’s findings acknowledged the need for a more cohesive effort to provide information about EPSDT services, and they indicated that the state would work to ensure that systems are in place to provide adequate information to families of children in Medicaid. The same HCFA study also singled out commendable practices including state efforts to coordinate care between Medicaid managed care plans and community health providers such as county mental health centers. In Michigan, a review of the state’s Medicaid managed care program conducted by HCFA’s Chicago Regional Office and others included an assessment of certain EPSDT policies and processes. These included EPSDT-covered services; processes and responsibilities for outreach, informing, and providing transportation services to beneficiaries; provider access and coordination; data reporting; and the achievement of screening goals. The review contained observations such as problems the state was having in collecting reliable data for the state EPSDT reports and differences in the usefulness of health plan member handbooks for describing how beneficiaries can obtain transportation services covered under EPSDT. Stated goals of the review were to gather information that would be useful in improving access and quality in the managed care program and to identify areas of innovation and best practices that could be shared with other states.
While these assessments have helped those state programs that were reviewed and have identified best practices that might be applicable to other states, HCFA has reviewed only eight states since 1995 and has not established a mechanism for sharing lessons learned or innovative practices already in place among states. Since there is no HCFA requirement to periodically focus on and promote EPSDT on the state level, the decision to do so resides with management of each HCFA region. Most regions have not devoted resources to actively monitor or promote EPSDT. Some regional office staff cited other priority efforts, such as SCHIP, as diverting their resources. We found that regions typically have one staff person designated as EPSDT Coordinator, but with multiple responsibilities other than EPSDT.

		Recent HCFA Proposal Aims to Improve EPSDT Performance, but Specific Plan Is Not Yet Developed

HCFA has recently begun to reevaluate the adequacy of its role in EPSDT. In a January 2001 letter to the agency’s regional offices, HCFA’s Director of the Center for Medicaid and State Operations introduced a proposal to broaden the agency’s role in promoting state EPSDT activities. In the letter, the Director sought input to a proposal designed to assure children’s access to services under the Medicaid program and to assist states in addressing problems in the collection and reporting of state EPSDT data.
HCFA officials told us that the goal of the letter was to obtain stakeholder comments on what HCFA’s focus and direction should be. As of April 2001, HCFA regional staff had reviewed and commented on the letter, as had representatives from the American Academy of Pediatrics, officials from HHS’s Health Resources and Services Administration, and the Maternal and Child Health Technical Advisory Group (an advisory group made up of 6 to 10 state Medicaid directors). HCFA officials informed us that stakeholder reaction to the proposed initiative had generally been positive. The current chair of the Maternal and Child Health Technical Advisory Group told us that the general tone of the letter represents a collaborative, partnership approach that would provide for needed technical assistance while affording the flexibility needed for states to address conditions and impediments unique to each state.
It is too early to determine whether this initiative will move forward, what form it will take, or what might result from it. The agency has not yet established a plan or devoted resources to develop and implement this proposal. HCFA officials said that they were continuing to solicit comments and input from stakeholders to develop a plan and that decisions about resources and implementation would depend on guidance and direction on agency priorities.

	Conclusions

More than a decade ago, the Congress passed legislative changes to help ensure that millions of low-income children under Medicaid have access to important health screening and treatment services. In the years since then, the Congress has placed even more emphasis on providing a health care safety net by expanding coverage to more and more children who do not have health insurance. This safety net, however, cannot be considered fully in place unless there are assurances that the covered health care services are actually provided. Unfortunately, reported data are unreliable and incomplete. They are inadequate for gauging Medicaid’s success in providing screening, diagnostic, and treatment services to enrolled children. Particularly for children served by managed care plans—a growing segment of the population—current information does not allow a thorough assessment of progress. However, the available information indicates that many children are still not receiving health screening services. Recognizing this concern, some states are taking a more active role in identifying ways to reach the at-risk population served by Medicaid.
HCFA, now called CMS, has recently indicated increased emphasis on EPSDT services and can build on these state efforts in several ways while still giving states the flexibility to administer the program. One way is to continue the important task of working with states to improve the reporting of information on service delivery. Many providers, plans, and states will need to improve their reporting in the long-term so that there will be a more accurate picture of how well they are doing in providing these services, especially in a capitated managed care environment. In the short-term, CMS can take action to obtain a better understanding of the many different state policies and practices so it can work collaboratively with states to improve data and reporting, monitor the provision of services, and better inform and reach beneficiaries. In its position of setting federal policy and assessing a broad array of state activities intended to help reach at-risk Medicaid children, CMS can help build on successful efforts by sharing successes among states and working with the many different agencies and parties to ensure a coordinated approach to this care. By signaling a broadening of its interest in state EPSDT efforts, the agency has taken a positive first step. An important next step is for CMS to develop a more specific plan and time frames for working with states to assess their efforts and results in providing services to children in Medicaid.

	Recommendations for Executive Action

To strengthen the federal role in ensuring the delivery of EPSDT services and to bring greater visibility to ways that states can better serve children in Medicaid, we recommend that the Administrator of CMS:   work with states to develop criteria and time frames for consistently assessing and improving EPSDT reporting and the provision of services, including requiring that states develop improvement plans as appropriate for achieving the EPSDT goal of providing health services to children in Medicaid; and   develop a mechanism for sharing information among states on successful state, plan, and provider practices for reaching children in Medicaid.

	Agency Comments

We obtained comments on a draft of this report from CMS and the five states we visited. CMS commented that, as noted in the draft report, the problem is complex and not subject to an easy resolution (CMS’s comments are included in app. III). CMS agreed that more could be done to work with states to help ensure children’s access to services and compliance with federal requirements and stated that the agency’s regional offices are already starting to work with some states where problems exist. CMS partially agreed with our recommendation that it work with states to develop criteria and time frames for assessing and improving EPSDT reporting and the provision of services, including developing state-specific improvement plans for achieving EPSDT goals. While acknowledging the importance of working with states to improve the provision of services, CMS indicated that it was not certain that improvement plans for all states were necessary as part of this effort. Because of the unreliability of EPSDT reports, we believe that a more consistent assessment across all states is necessary to provide greater insight into states’ progress in achieving EPSDT goals. Depending on the assessment outcomes, improvement plans may not be needed for every state. We have clarified our recommendation accordingly. CMS agreed with our recommendation that the agency do more to foster information sharing and cooperation among states to improve EPSDT. The agency indicated that, as a first step, it is planning several activities with states, foundations, and others to promote the value of EPSDT services. The agency also provided technical comments that we incorporated where appropriate.
California and Connecticut reviewed our findings concerning their state programs and said they had no comments. Florida, New York, and Wisconsin provided technical comments, which we incorporated where appropriate. New York also commented that the draft did not acknowledge that compliance rates with screening requirements are uniformly low, even for children not in Medicaid, and stated that EPSDT expectations may not be realistic. While some available reports, such as our past work on lead and dental screening, do show low screening rates in the aggregate, these reports also show wide variations among states. Because available data are insufficient to gauge states’ progress in providing EPSDT services, assessing whether the agency’s 80 percent screening goal is realistic is difficult. We anticipate that once state EPSDT data are more reliable, CMS will be in a better position to reevaluate whether the annual screening goals that it set more than a decade ago are realistic and achievable. New York also commented that the shortfalls in the provision of recommended levels of preventive health services identified in the report apply to all children, not just those served by Medicaid. Rather than perform a comparative analysis of the provision of services for children in Medicaid versus others, this report focused on the provision of EPSDT services to children in Medicaid, which our past work, as well as the work of others, has shown to be an at-risk population. New York’s comments are included in appendix IV.
As arranged with your offices, unless you release its contents earlier, we plan no further distribution of this report until 30 days after its issuance date. At that time, we will send copies to the Secretary of Health and Human Services; the Administrator of CMS; appropriate congressional committees; and other interested parties.
If you or your staff have any questions about this report, please contact me at (202) 512-7118. Other contacts and major contributors are included in appendix V.

Appendix I: Profiles of the Five States Visited

To obtain information about efforts states were taking to improve EPSDT services, particularly within managed care, we visited five states. These states—California, Connecticut, Florida, New York, and Wisconsin—were selected to represent different regions of the country and because they had relatively high numbers of children in managed care or a reputation for having an innovative EPSDT program or both. These states differed greatly in the size of their Medicaid populations and the number of participating health plans. Table 5 contains background information on the states we visited.

Appendix II: Examples of State Actions Resulting From EPSDT Legal Settlement Agreements

Lawsuits have been filed in at least 28 states alleging the states had failed to adequately provide EPSDT services. The seven cases summarized in table 6 were suggested by the National Health Law Program’s Director of Legal Affairs and other EPSDT advocates as examples of states that have adopted innovative or promising EPSDT practices as a result of lawsuits. The following information reflects our review of relevant court documents in each of these cases and, in some instances, follow-up contacts with state officials to obtain further information about the state’s efforts.

Appendix III: Comments From the Centers for Medicare and Medicaid Services

Appendix IV: Comments From the State of New York Department of Health

Appendix V: GAO Contacts and Staff Acknowledgments

	GAO Contacts

	Staff Acknowledgments

Other major contributors to this report were Matthew Byer, Bruce Greenstein, Sophia Ku, Behn Miller, and Stan Stenersen.

Output:
Why GAO Did This Study

The Early and Periodic Screening, Diagnostic, and Treatment (EPSDT) Program calls for states to provide children and adolescents under age 21 with access to comprehensive, periodic evaluations of health, development, and nutritional status, as well as vision, hearing, and dental services. There is concern that state Medicaid programs are not doing an adequate job of screening children for medical conditions or providing treatment for the children who need it. There is also concern about how these services are faring under managed care plans. This report examines (1) the extent to which children in Medicaid are receiving EPSDT services, (2) efforts that selected states are taking to improve delivery of EPSDT services, particularly within managed care, and (3) federal government efforts to ensure that state Medicaid programs provide covered EPSDT services.

What GAO Found

GAO found that the extent to which children in Medicaid are receiving EPSDT services are not fully known, but the available evidence indicates that many are not receiving these services. A Department of Health and Human Services Office of Inspector General study found that less than one-half of enrolled children in their sample received any EPSDT screens. GAO found that states are taking actions to improve delivery of EPSDT services, particularly within managed care. These actions include linking several state databases, publishing statistics that compare performance, contracting with local health departments to coordinate care for children, and mailing reminder letters to parents. Federal efforts to ensure that children are receiving services have largely focused on changing the state reports so that they can collect reliable information about the extent of the EPSDT screening.